2024-05-06T11:57:07.797 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-06T11:57:07.798 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --owner scheduled_teuthology@teuthology --name teuthology-2024-05-03_21:08:03-orch-squid-distro-default-smithi --archive /home/teuthworker/archive/teuthology-2024-05-03_21:08:03-orch-squid-distro-default-smithi/7688103 -- /home/teuthworker/archive/teuthology-2024-05-03_21:08:03-orch-squid-distro-default-smithi/7688103/orig.config.yaml 2024-05-06T11:57:07.846 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-06T11:57:07.929 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-05-03_21:08:03-orch-squid-distro-default-smithi/7688103 branch: squid description: orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7688103' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/quincy ' name: teuthology-2024-05-03_21:08:03-orch-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN sha1: 70ce9539b62a15b9de429c1f0e91a1e012b099aa ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 70ce9539b62a15b9de429c1f0e91a1e012b099aa kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 70ce9539b62a15b9de429c1f0e91a1e012b099aa owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 3735 sha1: 70ce9539b62a15b9de429c1f0e91a1e012b099aa sleep_before_teardown: 0 subset: 10/64 suite: orch suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_70ce9539b62a15b9de429c1f0e91a1e012b099aa/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 70ce9539b62a15b9de429c1f0e91a1e012b099aa targets: smithi092.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKqUT7+t55ukAQG9vthWS3R1xTo9XqzCpOxghMF3+w79mpeABKvmHmuRCD2yNsyxPS74zILZH4gZzvX9HXxMCMM= smithi190.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCQOkrFXfkgx4LrkVXglbo8zoGLSCcrkRtvo9k3P/rHzsGmOkOoyg15LqCqsTkDLFNwKNPNRNxNIC3KdErT1gpA= tasks: - install: branch: quincy exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: cephadm_branch: quincy cephadm_git_url: https://github.com/ceph/ceph conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:quincy roleless: true - print: '**** done end installing quincy cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay true - cephadm.shell: host.a: - ceph fs set cephfs inline_data true --yes-i-really-really-mean-it - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: - /home/teuthology/src/git.ceph.com_ceph_squid/qa/suites/orch/cephadm/mds_upgrade_sequence/tasks/3-upgrade-mgr-staggered.yaml meta: {} postmerge: [] variables: fail_fs: false teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-03_21:08:03 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs false || true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-05-06T11:57:07.929 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_70ce9539b62a15b9de429c1f0e91a1e012b099aa/qa; will attempt to use it 2024-05-06T11:57:07.930 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_70ce9539b62a15b9de429c1f0e91a1e012b099aa/qa/tasks 2024-05-06T11:57:07.930 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-06T11:57:07.931 INFO:teuthology.task.internal:Checking packages... 2024-05-06T11:57:07.953 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '70ce9539b62a15b9de429c1f0e91a1e012b099aa' 2024-05-06T11:57:07.953 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-06T11:57:07.953 INFO:teuthology.packaging:ref: None 2024-05-06T11:57:07.954 INFO:teuthology.packaging:tag: None 2024-05-06T11:57:07.954 INFO:teuthology.packaging:branch: squid 2024-05-06T11:57:07.954 INFO:teuthology.packaging:sha1: 70ce9539b62a15b9de429c1f0e91a1e012b099aa 2024-05-06T11:57:07.954 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-05-06T11:57:08.268 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2504.gb05eb275 2024-05-06T11:57:08.269 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-06T11:57:08.276 INFO:teuthology.task.internal:no buildpackages task found 2024-05-06T11:57:08.276 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-06T11:57:08.284 INFO:teuthology.task.internal:Saving configuration 2024-05-06T11:57:08.301 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-06T11:57:08.308 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-06T11:57:08.333 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-03_21:08:03-orch-squid-distro-default-smithi/7688103', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-06 11:51:58.915452', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKqUT7+t55ukAQG9vthWS3R1xTo9XqzCpOxghMF3+w79mpeABKvmHmuRCD2yNsyxPS74zILZH4gZzvX9HXxMCMM='} 2024-05-06T11:57:08.357 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-03_21:08:03-orch-squid-distro-default-smithi/7688103', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-06 11:51:58.917042', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCQOkrFXfkgx4LrkVXglbo8zoGLSCcrkRtvo9k3P/rHzsGmOkOoyg15LqCqsTkDLFNwKNPNRNxNIC3KdErT1gpA='} 2024-05-06T11:57:08.358 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-06T11:57:08.364 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-05-06T11:57:08.364 INFO:teuthology.task.internal:roles: ubuntu@smithi190.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-05-06T11:57:08.365 INFO:teuthology.run_tasks:Running task console_log... 2024-05-06T11:57:08.436 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f19dce340d0>, signals=[15]) 2024-05-06T11:57:08.437 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-06T11:57:08.443 INFO:teuthology.task.internal:Opening connections... 2024-05-06T11:57:08.444 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi092.front.sepia.ceph.com 2024-05-06T11:57:08.445 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-06T11:57:08.521 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi190.front.sepia.ceph.com 2024-05-06T11:57:08.522 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-06T11:57:08.593 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-06T11:57:08.600 DEBUG:teuthology.orchestra.run.smithi092:> uname -m 2024-05-06T11:57:08.617 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2024-05-06T11:57:08.618 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2024-05-06T11:57:08.674 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="CentOS Stream" 2024-05-06T11:57:08.674 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="9" 2024-05-06T11:57:08.674 INFO:teuthology.orchestra.run.smithi092.stdout:ID="centos" 2024-05-06T11:57:08.674 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="rhel fedora" 2024-05-06T11:57:08.674 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="9" 2024-05-06T11:57:08.675 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el9" 2024-05-06T11:57:08.675 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-06T11:57:08.675 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2024-05-06T11:57:08.675 INFO:teuthology.orchestra.run.smithi092.stdout:LOGO="fedora-logo-icon" 2024-05-06T11:57:08.675 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-06T11:57:08.675 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://centos.org/" 2024-05-06T11:57:08.675 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-06T11:57:08.675 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-06T11:57:08.675 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-06T11:57:08.676 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2024-05-06T11:57:08.700 DEBUG:teuthology.orchestra.run.smithi190:> uname -m 2024-05-06T11:57:08.717 INFO:teuthology.orchestra.run.smithi190.stdout:x86_64 2024-05-06T11:57:08.718 DEBUG:teuthology.orchestra.run.smithi190:> cat /etc/os-release 2024-05-06T11:57:08.775 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="CentOS Stream" 2024-05-06T11:57:08.775 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="9" 2024-05-06T11:57:08.775 INFO:teuthology.orchestra.run.smithi190.stdout:ID="centos" 2024-05-06T11:57:08.775 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE="rhel fedora" 2024-05-06T11:57:08.775 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="9" 2024-05-06T11:57:08.775 INFO:teuthology.orchestra.run.smithi190.stdout:PLATFORM_ID="platform:el9" 2024-05-06T11:57:08.775 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-06T11:57:08.775 INFO:teuthology.orchestra.run.smithi190.stdout:ANSI_COLOR="0;31" 2024-05-06T11:57:08.776 INFO:teuthology.orchestra.run.smithi190.stdout:LOGO="fedora-logo-icon" 2024-05-06T11:57:08.776 INFO:teuthology.orchestra.run.smithi190.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-06T11:57:08.776 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://centos.org/" 2024-05-06T11:57:08.776 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-06T11:57:08.776 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-06T11:57:08.776 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-06T11:57:08.776 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server 2024-05-06T11:57:08.791 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-06T11:57:08.801 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-06T11:57:08.807 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-06T11:57:08.807 DEBUG:teuthology.orchestra.run.smithi092:> test '!' -e /home/ubuntu/cephtest 2024-05-06T11:57:08.810 DEBUG:teuthology.orchestra.run.smithi190:> test '!' -e /home/ubuntu/cephtest 2024-05-06T11:57:08.832 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-06T11:57:08.838 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-06T11:57:08.839 DEBUG:teuthology.orchestra.run.smithi092:> test -z $(ls -A /var/lib/ceph) 2024-05-06T11:57:08.865 DEBUG:teuthology.orchestra.run.smithi190:> test -z $(ls -A /var/lib/ceph) 2024-05-06T11:57:08.908 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-06T11:57:08.963 INFO:teuthology.run_tasks:Running task kernel... 2024-05-06T11:57:08.977 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-05-06T11:57:08.977 INFO:teuthology.task.kernel:config {'host.a': {'kdb': 1, 'sha1': 'distro'}, 'host.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-05-06T11:57:08.978 DEBUG:teuthology.orchestra.run.smithi092:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-06T11:57:08.978 DEBUG:teuthology.orchestra.run.smithi190:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-06T11:57:08.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-06T11:57:08.994 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2024-05-06T11:57:08.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-06T11:57:08.995 DEBUG:teuthology.orchestra.run.smithi190:> uname -r 2024-05-06T11:57:09.050 INFO:teuthology.orchestra.run.smithi092.stdout:5.14.0-437.el9.x86_64 2024-05-06T11:57:09.050 INFO:teuthology.task.kernel:Running kernel on smithi092: 5.14.0-437.el9.x86_64 2024-05-06T11:57:09.050 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum install -y kernel 2024-05-06T11:57:09.052 INFO:teuthology.orchestra.run.smithi190.stdout:5.14.0-437.el9.x86_64 2024-05-06T11:57:09.052 INFO:teuthology.task.kernel:Running kernel on smithi190: 5.14.0-437.el9.x86_64 2024-05-06T11:57:09.052 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum install -y kernel 2024-05-06T11:57:12.586 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.0 MB 00:01 2024-05-06T11:57:12.835 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - BaseOS 3.8 MB/s | 8.0 MB 00:02 2024-05-06T11:57:15.686 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-05-06T11:57:15.744 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 9.1 MB/s | 19 MB 00:02 2024-05-06T11:57:19.878 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-05-06T11:57:21.478 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 16 kB 00:00 2024-05-06T11:57:22.419 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - CRB 1.9 MB/s | 6.2 MB 00:03 2024-05-06T11:57:23.283 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-05-06T11:57:23.803 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 16 kB 00:00 2024-05-06T11:57:24.691 INFO:teuthology.orchestra.run.smithi190.stdout:Extra Packages for Enterprise Linux 27 MB/s | 21 MB 00:00 2024-05-06T11:57:29.779 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 1.1 kB/s | 1.7 kB 00:01 2024-05-06T11:57:29.805 INFO:teuthology.orchestra.run.smithi190.stdout:lab-extras 44 kB/s | 1.7 kB 00:00 2024-05-06T11:57:31.619 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-06T11:57:31.619 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-06T11:57:31.644 INFO:teuthology.orchestra.run.smithi190.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-06T11:57:31.644 INFO:teuthology.orchestra.run.smithi190.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-06T11:57:31.714 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-06T11:57:31.715 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-05-06T11:57:31.715 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-06T11:57:31.736 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-05-06T11:57:31.737 INFO:teuthology.orchestra.run.smithi190.stdout:Nothing to do. 2024-05-06T11:57:31.737 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-05-06T11:57:31.815 DEBUG:teuthology.orchestra.run.smithi092:> echo no | sudo yum reinstall kernel || true 2024-05-06T11:57:31.856 DEBUG:teuthology.orchestra.run.smithi190:> echo no | sudo yum reinstall kernel || true 2024-05-06T11:57:32.355 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:03 ago on Mon 06 May 2024 11:57:29 AM UTC. 2024-05-06T11:57:32.397 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:00:03 ago on Mon 06 May 2024 11:57:29 AM UTC. 2024-05-06T11:57:32.485 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-06T11:57:32.495 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T11:57:32.496 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2024-05-06T11:57:32.496 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T11:57:32.496 INFO:teuthology.orchestra.run.smithi092.stdout:Reinstalling: 2024-05-06T11:57:32.496 INFO:teuthology.orchestra.run.smithi092.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-05-06T11:57:32.496 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T11:57:32.496 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-06T11:57:32.496 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T11:57:32.496 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T11:57:32.496 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 255 k 2024-05-06T11:57:32.496 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 0 2024-05-06T11:57:32.496 INFO:teuthology.orchestra.run.smithi092.stderr:Operation aborted. 2024-05-06T11:57:32.540 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-05-06T11:57:32.551 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T11:57:32.551 INFO:teuthology.orchestra.run.smithi190.stdout: Package Architecture Version Repository Size 2024-05-06T11:57:32.552 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T11:57:32.552 INFO:teuthology.orchestra.run.smithi190.stdout:Reinstalling: 2024-05-06T11:57:32.552 INFO:teuthology.orchestra.run.smithi190.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-05-06T11:57:32.552 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T11:57:32.552 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-05-06T11:57:32.552 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T11:57:32.552 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T11:57:32.552 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 255 k 2024-05-06T11:57:32.552 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 0 2024-05-06T11:57:32.552 INFO:teuthology.orchestra.run.smithi190.stderr:Operation aborted. 2024-05-06T11:57:32.557 INFO:teuthology.orchestra.run.smithi092.stdout:Is this ok [y/N]: 2024-05-06T11:57:32.558 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum reinstall -y kernel || true 2024-05-06T11:57:32.604 INFO:teuthology.orchestra.run.smithi190.stdout:Is this ok [y/N]: 2024-05-06T11:57:32.604 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum reinstall -y kernel || true 2024-05-06T11:57:33.055 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:04 ago on Mon 06 May 2024 11:57:29 AM UTC. 2024-05-06T11:57:33.098 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:00:04 ago on Mon 06 May 2024 11:57:29 AM UTC. 2024-05-06T11:57:33.172 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-06T11:57:33.173 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T11:57:33.173 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2024-05-06T11:57:33.173 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T11:57:33.173 INFO:teuthology.orchestra.run.smithi092.stdout:Reinstalling: 2024-05-06T11:57:33.174 INFO:teuthology.orchestra.run.smithi092.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-05-06T11:57:33.174 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T11:57:33.174 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-06T11:57:33.174 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T11:57:33.174 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T11:57:33.174 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 255 k 2024-05-06T11:57:33.174 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 0 2024-05-06T11:57:33.174 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-06T11:57:33.216 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-05-06T11:57:33.217 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T11:57:33.217 INFO:teuthology.orchestra.run.smithi190.stdout: Package Architecture Version Repository Size 2024-05-06T11:57:33.217 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T11:57:33.217 INFO:teuthology.orchestra.run.smithi190.stdout:Reinstalling: 2024-05-06T11:57:33.217 INFO:teuthology.orchestra.run.smithi190.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-05-06T11:57:33.217 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T11:57:33.217 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-05-06T11:57:33.217 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T11:57:33.217 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T11:57:33.217 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 255 k 2024-05-06T11:57:33.217 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 0 2024-05-06T11:57:33.217 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-05-06T11:57:33.523 INFO:teuthology.orchestra.run.smithi190.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.5 MB/s | 255 kB 00:00 2024-05-06T11:57:33.523 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-05-06T11:57:33.523 INFO:teuthology.orchestra.run.smithi190.stdout:Total 833 kB/s | 255 kB 00:00 2024-05-06T11:57:33.543 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-05-06T11:57:33.745 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-05-06T11:57:33.745 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-05-06T11:57:33.750 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-05-06T11:57:33.750 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-05-06T11:57:33.814 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-5.14.0-437.el9.x86_64.rpm 429 kB/s | 255 kB 00:00 2024-05-06T11:57:33.814 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-06T11:57:33.814 INFO:teuthology.orchestra.run.smithi092.stdout:Total 398 kB/s | 255 kB 00:00 2024-05-06T11:57:33.836 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-06T11:57:33.864 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-05-06T11:57:33.966 INFO:teuthology.orchestra.run.smithi190.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-05-06T11:57:33.966 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-05-06T11:57:34.021 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-06T11:57:34.021 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-06T11:57:34.026 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-06T11:57:34.027 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-06T11:57:34.165 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-06T11:57:34.257 INFO:teuthology.orchestra.run.smithi092.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-05-06T11:57:34.257 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-05-06T11:57:34.658 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-05-06T11:57:34.659 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-05-06T11:57:34.970 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-05-06T11:57:34.970 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-05-06T11:57:35.331 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-05-06T11:57:35.331 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T11:57:35.331 INFO:teuthology.orchestra.run.smithi190.stdout:Reinstalled: 2024-05-06T11:57:35.331 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-5.14.0-437.el9.x86_64 2024-05-06T11:57:35.331 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T11:57:35.332 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-05-06T11:57:35.528 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q kernel | sort -rV | head -n 1 2024-05-06T11:57:35.575 INFO:teuthology.orchestra.run.smithi190.stdout:kernel-5.14.0-437.el9.x86_64 2024-05-06T11:57:35.575 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-05-06T11:57:35.575 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-05-06T11:57:35.576 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-05-06T11:57:35.576 DEBUG:teuthology.orchestra.run.smithi190:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-06T11:57:35.607 INFO:teuthology.orchestra.run.smithi190.stdout:ttyS1 2024-05-06T11:57:35.620 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-05-06T11:57:35.620 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T11:57:35.620 INFO:teuthology.orchestra.run.smithi092.stdout:Reinstalled: 2024-05-06T11:57:35.620 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-5.14.0-437.el9.x86_64 2024-05-06T11:57:35.620 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T11:57:35.620 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-06T11:57:35.636 DEBUG:teuthology.parallel:result is None 2024-05-06T11:57:35.835 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q kernel | sort -rV | head -n 1 2024-05-06T11:57:35.881 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-5.14.0-437.el9.x86_64 2024-05-06T11:57:35.882 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-05-06T11:57:35.882 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-05-06T11:57:35.882 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-05-06T11:57:35.882 DEBUG:teuthology.orchestra.run.smithi092:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-06T11:57:35.911 INFO:teuthology.orchestra.run.smithi092.stdout:ttyS1 2024-05-06T11:57:35.932 DEBUG:teuthology.parallel:result is None 2024-05-06T11:57:35.932 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-06T11:57:35.941 INFO:teuthology.task.internal:Creating test directory... 2024-05-06T11:57:35.941 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-06T11:57:35.975 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-06T11:57:35.990 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-06T11:57:36.001 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-06T11:57:36.012 INFO:teuthology.task.internal:Creating archive directory... 2024-05-06T11:57:36.012 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-06T11:57:36.030 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-06T11:57:36.075 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-06T11:57:36.084 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-06T11:57:36.084 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-06T11:57:36.088 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-06T11:57:36.124 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-06T11:57:36.133 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-06T11:57:36.148 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-06T11:57:36.158 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-06T11:57:36.159 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-06T11:57:36.168 INFO:teuthology.task.internal:Configuring sudo... 2024-05-06T11:57:36.168 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-06T11:57:36.177 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-06T11:57:36.222 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-06T11:57:36.233 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-06T11:57:36.233 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-06T11:57:36.240 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-06T11:57:36.277 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-06T11:57:36.334 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-06T11:57:36.386 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-06T11:57:36.470 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-06T11:57:36.507 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-06T11:57:36.508 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-06T11:57:36.571 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-06T11:57:36.616 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-06T11:57:36.660 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-06T11:57:36.742 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-06T11:57:36.780 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-06T11:57:36.780 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-06T11:57:36.843 DEBUG:teuthology.orchestra.run.smithi092:> sudo service rsyslog restart 2024-05-06T11:57:36.845 DEBUG:teuthology.orchestra.run.smithi190:> sudo service rsyslog restart 2024-05-06T11:57:36.877 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-06T11:57:36.915 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-06T11:57:37.382 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-06T11:57:37.390 INFO:teuthology.task.internal:Starting timer... 2024-05-06T11:57:37.390 INFO:teuthology.run_tasks:Running task pcp... 2024-05-06T11:57:37.452 INFO:teuthology.run_tasks:Running task selinux... 2024-05-06T11:57:37.462 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-06T11:57:37.462 DEBUG:teuthology.orchestra.run.smithi092:> sudo service auditd rotate 2024-05-06T11:57:37.554 INFO:teuthology.orchestra.run.smithi092.stdout:Rotating logs: 2024-05-06T11:57:37.556 DEBUG:teuthology.orchestra.run.smithi190:> sudo service auditd rotate 2024-05-06T11:57:37.602 INFO:teuthology.orchestra.run.smithi190.stdout:Rotating logs: 2024-05-06T11:57:37.604 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-06T11:57:37.604 DEBUG:teuthology.orchestra.run.smithi092:> /usr/sbin/getenforce 2024-05-06T11:57:37.622 INFO:teuthology.orchestra.run.smithi092.stdout:Permissive 2024-05-06T11:57:37.623 DEBUG:teuthology.orchestra.run.smithi190:> /usr/sbin/getenforce 2024-05-06T11:57:37.651 INFO:teuthology.orchestra.run.smithi190.stdout:Permissive 2024-05-06T11:57:37.652 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi092.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi190.front.sepia.ceph.com': 'permissive'} 2024-05-06T11:57:37.652 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-06T11:57:37.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-06T11:57:37.685 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-06T11:57:37.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-06T11:57:37.716 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-06T11:57:37.716 DEBUG:teuthology.orchestra.run.smithi092:> sudo /usr/sbin/setenforce permissive 2024-05-06T11:57:37.755 DEBUG:teuthology.orchestra.run.smithi190:> sudo /usr/sbin/setenforce permissive 2024-05-06T11:57:37.780 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-06T11:57:37.793 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-06T11:57:37.809 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-06T11:57:37.810 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi092.front.sepia.ceph.com,smithi190.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-06T12:01:29.460 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi092.front.sepia.ceph.com'), Remote(name='ubuntu@smithi190.front.sepia.ceph.com')] 2024-05-06T12:01:29.461 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-05-06T12:01:29.462 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-06T12:01:29.543 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-05-06T12:01:29.622 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-05-06T12:01:29.622 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-05-06T12:01:29.623 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-06T12:01:29.702 DEBUG:teuthology.orchestra.run.smithi190:> true 2024-05-06T12:01:29.787 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-05-06T12:01:29.787 INFO:teuthology.run_tasks:Running task clock... 2024-05-06T12:01:29.797 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-06T12:01:29.798 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-06T12:01:29.801 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-06T12:01:29.804 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-06T12:01:29.804 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-06T12:01:29.833 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-06T12:01:29.847 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-06T12:01:29.858 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-06T12:01:29.872 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-06T12:01:29.872 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: ntpd: command not found 2024-05-06T12:01:29.883 INFO:teuthology.orchestra.run.smithi092.stdout:506 Cannot talk to daemon 2024-05-06T12:01:29.896 INFO:teuthology.orchestra.run.smithi190.stderr:sudo: ntpd: command not found 2024-05-06T12:01:29.898 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-06T12:01:29.906 INFO:teuthology.orchestra.run.smithi190.stdout:506 Cannot talk to daemon 2024-05-06T12:01:29.910 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-06T12:01:29.920 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-06T12:01:29.933 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-06T12:01:29.965 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-05-06T12:01:29.968 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-06T12:01:29.968 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-05-06T12:01:29.968 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-06T12:01:29.968 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-06T12:01:29.968 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-06T12:01:29.968 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-06T12:01:29.995 INFO:teuthology.orchestra.run.smithi190.stderr:bash: line 1: ntpq: command not found 2024-05-06T12:01:29.998 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-06T12:01:29.998 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2024-05-06T12:01:29.998 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-06T12:01:29.998 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-06T12:01:29.998 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-06T12:01:29.998 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-06T12:01:29.999 INFO:teuthology.run_tasks:Running task install... 2024-05-06T12:01:30.010 DEBUG:teuthology.task.install:project ceph 2024-05-06T12:01:30.010 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '70ce9539b62a15b9de429c1f0e91a1e012b099aa'}} 2024-05-06T12:01:30.010 DEBUG:teuthology.task.install:config {'branch': 'quincy', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': '70ce9539b62a15b9de429c1f0e91a1e012b099aa'} 2024-05-06T12:01:30.010 INFO:teuthology.task.install:Using flavor: default 2024-05-06T12:01:30.016 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-06T12:01:30.016 INFO:teuthology.task.install:extra packages: [] 2024-05-06T12:01:30.016 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '70ce9539b62a15b9de429c1f0e91a1e012b099aa', 'tag': None, 'wait_for_package': False} 2024-05-06T12:01:30.017 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-06T12:01:30.017 INFO:teuthology.packaging:ref: None 2024-05-06T12:01:30.017 INFO:teuthology.packaging:tag: None 2024-05-06T12:01:30.017 INFO:teuthology.packaging:branch: quincy 2024-05-06T12:01:30.017 INFO:teuthology.packaging:sha1: 70ce9539b62a15b9de429c1f0e91a1e012b099aa 2024-05-06T12:01:30.017 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-05-06T12:01:30.019 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '70ce9539b62a15b9de429c1f0e91a1e012b099aa', 'tag': None, 'wait_for_package': False} 2024-05-06T12:01:30.019 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-06T12:01:30.019 INFO:teuthology.packaging:ref: None 2024-05-06T12:01:30.019 INFO:teuthology.packaging:tag: None 2024-05-06T12:01:30.019 INFO:teuthology.packaging:branch: quincy 2024-05-06T12:01:30.020 INFO:teuthology.packaging:sha1: 70ce9539b62a15b9de429c1f0e91a1e012b099aa 2024-05-06T12:01:30.020 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-05-06T12:01:30.239 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/e81df4201214a93810b2eaca609e475a6319e220/centos/9/flavors/default/ 2024-05-06T12:01:30.240 INFO:teuthology.task.install.rpm:Package version is 17.2.7-925.ge81df420 2024-05-06T12:01:30.272 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/e81df4201214a93810b2eaca609e475a6319e220/centos/9/flavors/default/ 2024-05-06T12:01:30.272 INFO:teuthology.task.install.rpm:Package version is 17.2.7-925.ge81df420 2024-05-06T12:01:30.351 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/e81df4201214a93810b2eaca609e475a6319e220/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/e81df4201214a93810b2eaca609e475a6319e220/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/e81df4201214a93810b2eaca609e475a6319e220/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-06T12:01:30.351 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-06T12:01:30.351 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-06T12:01:30.375 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-05-06T12:01:30.376 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-06T12:01:30.376 INFO:teuthology.packaging:ref: None 2024-05-06T12:01:30.376 INFO:teuthology.packaging:tag: None 2024-05-06T12:01:30.376 INFO:teuthology.packaging:branch: quincy 2024-05-06T12:01:30.376 INFO:teuthology.packaging:sha1: 70ce9539b62a15b9de429c1f0e91a1e012b099aa 2024-05-06T12:01:30.376 DEBUG:teuthology.orchestra.run.smithi190:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-06T12:01:30.382 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/e81df4201214a93810b2eaca609e475a6319e220/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/e81df4201214a93810b2eaca609e475a6319e220/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/e81df4201214a93810b2eaca609e475a6319e220/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-06T12:01:30.383 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-06T12:01:30.383 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-06T12:01:30.409 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-05-06T12:01:30.409 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-06T12:01:30.409 INFO:teuthology.packaging:ref: None 2024-05-06T12:01:30.409 INFO:teuthology.packaging:tag: None 2024-05-06T12:01:30.409 INFO:teuthology.packaging:branch: quincy 2024-05-06T12:01:30.409 INFO:teuthology.packaging:sha1: 70ce9539b62a15b9de429c1f0e91a1e012b099aa 2024-05-06T12:01:30.410 DEBUG:teuthology.orchestra.run.smithi092:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-06T12:01:30.440 DEBUG:teuthology.orchestra.run.smithi190:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-06T12:01:30.475 DEBUG:teuthology.orchestra.run.smithi092:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-06T12:01:30.512 DEBUG:teuthology.orchestra.run.smithi190:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-06T12:01:30.549 DEBUG:teuthology.orchestra.run.smithi092:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-06T12:01:30.575 INFO:teuthology.orchestra.run.smithi190.stdout:check_obsoletes = 1 2024-05-06T12:01:30.577 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum clean all 2024-05-06T12:01:30.612 INFO:teuthology.orchestra.run.smithi092.stdout:check_obsoletes = 1 2024-05-06T12:01:30.614 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum clean all 2024-05-06T12:01:30.802 INFO:teuthology.orchestra.run.smithi190.stdout:50 files removed 2024-05-06T12:01:30.829 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-radosgw 2024-05-06T12:01:30.838 INFO:teuthology.orchestra.run.smithi092.stdout:50 files removed 2024-05-06T12:01:30.865 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-radosgw 2024-05-06T12:01:31.489 INFO:teuthology.orchestra.run.smithi190.stdout:ceph packages for x86_64 193 kB/s | 77 kB 00:00 2024-05-06T12:01:31.568 INFO:teuthology.orchestra.run.smithi092.stdout:ceph packages for x86_64 174 kB/s | 77 kB 00:00 2024-05-06T12:01:31.864 INFO:teuthology.orchestra.run.smithi190.stdout:ceph noarch packages 32 kB/s | 11 kB 00:00 2024-05-06T12:01:31.919 INFO:teuthology.orchestra.run.smithi092.stdout:ceph noarch packages 35 kB/s | 11 kB 00:00 2024-05-06T12:01:32.231 INFO:teuthology.orchestra.run.smithi190.stdout:ceph source packages 5.4 kB/s | 1.8 kB 00:00 2024-05-06T12:01:32.377 INFO:teuthology.orchestra.run.smithi092.stdout:ceph source packages 4.2 kB/s | 1.8 kB 00:00 2024-05-06T12:01:35.331 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - BaseOS 2.6 MB/s | 8.0 MB 00:03 2024-05-06T12:01:35.494 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 2.6 MB/s | 8.0 MB 00:03 2024-05-06T12:01:38.418 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 10 MB/s | 19 MB 00:01 2024-05-06T12:01:42.512 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-05-06T12:01:43.324 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - AppStream 2.7 MB/s | 19 MB 00:06 2024-05-06T12:01:44.079 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 16 kB 00:00 2024-05-06T12:01:47.299 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.2 MB 00:00 2024-05-06T12:01:47.472 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 6.4 MB/s | 21 MB 00:03 2024-05-06T12:01:48.750 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 16 kB 00:00 2024-05-06T12:01:50.146 INFO:teuthology.orchestra.run.smithi190.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-05-06T12:01:52.638 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-05-06T12:01:54.758 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-06T12:01:54.760 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:01:54.760 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-05-06T12:01:54.760 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:01:54.760 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-06T12:01:54.760 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw x86_64 2:17.2.7-925.ge81df420.el9 ceph 14 M 2024-05-06T12:01:54.760 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrading: 2024-05-06T12:01:54.760 INFO:teuthology.orchestra.run.smithi092.stdout: librados2 x86_64 2:17.2.7-925.ge81df420.el9 ceph 3.1 M 2024-05-06T12:01:54.760 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1 x86_64 2:17.2.7-925.ge81df420.el9 ceph 2.9 M 2024-05-06T12:01:54.760 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-05-06T12:01:54.760 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base x86_64 2:17.2.7-925.ge81df420.el9 ceph 5.0 M 2024-05-06T12:01:54.761 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common x86_64 2:17.2.7-925.ge81df420.el9 ceph 22 M 2024-05-06T12:01:54.761 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux x86_64 2:17.2.7-925.ge81df420.el9 ceph 25 k 2024-05-06T12:01:54.761 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-06T12:01:54.761 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2 x86_64 2:17.2.7-925.ge81df420.el9 ceph 638 k 2024-05-06T12:01:54.761 INFO:teuthology.orchestra.run.smithi092.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-06T12:01:54.761 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-06T12:01:54.761 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-06T12:01:54.761 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1 x86_64 2:17.2.7-925.ge81df420.el9 ceph 458 k 2024-05-06T12:01:54.762 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-06T12:01:54.762 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2 x86_64 2:17.2.7-925.ge81df420.el9 ceph 4.9 M 2024-05-06T12:01:54.762 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-06T12:01:54.762 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-06T12:01:54.762 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse x86_64 2:17.2.7-925.ge81df420.el9 ceph 45 k 2024-05-06T12:01:54.762 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common x86_64 2:17.2.7-925.ge81df420.el9 ceph 118 k 2024-05-06T12:01:54.762 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs x86_64 2:17.2.7-925.ge81df420.el9 ceph 156 k 2024-05-06T12:01:54.762 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-06T12:01:54.762 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados x86_64 2:17.2.7-925.ge81df420.el9 ceph 319 k 2024-05-06T12:01:54.762 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd x86_64 2:17.2.7-925.ge81df420.el9 ceph 293 k 2024-05-06T12:01:54.762 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw x86_64 2:17.2.7-925.ge81df420.el9 ceph 99 k 2024-05-06T12:01:54.762 INFO:teuthology.orchestra.run.smithi092.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-06T12:01:54.762 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:01:54.763 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-06T12:01:54.763 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:01:54.763 INFO:teuthology.orchestra.run.smithi092.stdout:Install 22 Packages 2024-05-06T12:01:54.763 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrade 2 Packages 2024-05-06T12:01:54.763 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:01:54.764 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 58 M 2024-05-06T12:01:54.764 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-06T12:01:55.285 INFO:teuthology.orchestra.run.smithi190.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-05-06T12:01:56.523 INFO:teuthology.orchestra.run.smithi092.stdout:(1/24): ceph-radosgw-17.2.7-925.ge81df420.el9.x 9.0 MB/s | 14 MB 00:01 2024-05-06T12:01:56.564 INFO:teuthology.orchestra.run.smithi092.stdout:(2/24): ceph-selinux-17.2.7-925.ge81df420.el9.x 609 kB/s | 25 kB 00:00 2024-05-06T12:01:56.848 INFO:teuthology.orchestra.run.smithi092.stdout:(3/24): ceph-common-17.2.7-925.ge81df420.el9.x8 12 MB/s | 22 MB 00:01 2024-05-06T12:01:56.873 INFO:teuthology.orchestra.run.smithi092.stdout:(4/24): libcephfs2-17.2.7-925.ge81df420.el9.x86 2.0 MB/s | 638 kB 00:00 2024-05-06T12:01:56.907 INFO:teuthology.orchestra.run.smithi092.stdout:(5/24): libradosstriper1-17.2.7-925.ge81df420.e 7.7 MB/s | 458 kB 00:00 2024-05-06T12:01:56.957 INFO:teuthology.orchestra.run.smithi092.stdout:(6/24): python3-ceph-argparse-17.2.7-925.ge81df 902 kB/s | 45 kB 00:00 2024-05-06T12:01:57.016 INFO:teuthology.orchestra.run.smithi092.stdout:(7/24): python3-ceph-common-17.2.7-925.ge81df42 2.0 MB/s | 118 kB 00:00 2024-05-06T12:01:57.083 INFO:teuthology.orchestra.run.smithi092.stdout:(8/24): python3-cephfs-17.2.7-925.ge81df420.el9 2.3 MB/s | 156 kB 00:00 2024-05-06T12:01:57.174 INFO:teuthology.orchestra.run.smithi092.stdout:(9/24): librgw2-17.2.7-925.ge81df420.el9.x86_64 16 MB/s | 4.9 MB 00:00 2024-05-06T12:01:57.200 INFO:teuthology.orchestra.run.smithi092.stdout:(10/24): python3-rados-17.2.7-925.ge81df420.el9 2.7 MB/s | 319 kB 00:00 2024-05-06T12:01:57.225 INFO:teuthology.orchestra.run.smithi092.stdout:(11/24): python3-rbd-17.2.7-925.ge81df420.el9.x 5.7 MB/s | 293 kB 00:00 2024-05-06T12:01:57.250 INFO:teuthology.orchestra.run.smithi092.stdout:(12/24): python3-rgw-17.2.7-925.ge81df420.el9.x 1.9 MB/s | 99 kB 00:00 2024-05-06T12:01:57.431 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-05-06T12:01:57.433 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:01:57.433 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2024-05-06T12:01:57.433 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:01:57.433 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-05-06T12:01:57.433 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw x86_64 2:17.2.7-925.ge81df420.el9 ceph 14 M 2024-05-06T12:01:57.434 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrading: 2024-05-06T12:01:57.434 INFO:teuthology.orchestra.run.smithi190.stdout: librados2 x86_64 2:17.2.7-925.ge81df420.el9 ceph 3.1 M 2024-05-06T12:01:57.434 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1 x86_64 2:17.2.7-925.ge81df420.el9 ceph 2.9 M 2024-05-06T12:01:57.434 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-05-06T12:01:57.434 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base x86_64 2:17.2.7-925.ge81df420.el9 ceph 5.0 M 2024-05-06T12:01:57.434 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common x86_64 2:17.2.7-925.ge81df420.el9 ceph 22 M 2024-05-06T12:01:57.434 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux x86_64 2:17.2.7-925.ge81df420.el9 ceph 25 k 2024-05-06T12:01:57.434 INFO:teuthology.orchestra.run.smithi190.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-06T12:01:57.434 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2 x86_64 2:17.2.7-925.ge81df420.el9 ceph 638 k 2024-05-06T12:01:57.434 INFO:teuthology.orchestra.run.smithi190.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-06T12:01:57.434 INFO:teuthology.orchestra.run.smithi190.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-06T12:01:57.434 INFO:teuthology.orchestra.run.smithi190.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-06T12:01:57.435 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1 x86_64 2:17.2.7-925.ge81df420.el9 ceph 458 k 2024-05-06T12:01:57.435 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-06T12:01:57.435 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2 x86_64 2:17.2.7-925.ge81df420.el9 ceph 4.9 M 2024-05-06T12:01:57.435 INFO:teuthology.orchestra.run.smithi190.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-06T12:01:57.435 INFO:teuthology.orchestra.run.smithi190.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-06T12:01:57.435 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse x86_64 2:17.2.7-925.ge81df420.el9 ceph 45 k 2024-05-06T12:01:57.435 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common x86_64 2:17.2.7-925.ge81df420.el9 ceph 118 k 2024-05-06T12:01:57.435 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs x86_64 2:17.2.7-925.ge81df420.el9 ceph 156 k 2024-05-06T12:01:57.435 INFO:teuthology.orchestra.run.smithi190.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-06T12:01:57.435 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados x86_64 2:17.2.7-925.ge81df420.el9 ceph 319 k 2024-05-06T12:01:57.435 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd x86_64 2:17.2.7-925.ge81df420.el9 ceph 293 k 2024-05-06T12:01:57.435 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw x86_64 2:17.2.7-925.ge81df420.el9 ceph 99 k 2024-05-06T12:01:57.435 INFO:teuthology.orchestra.run.smithi190.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-06T12:01:57.435 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:01:57.436 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-05-06T12:01:57.436 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:01:57.436 INFO:teuthology.orchestra.run.smithi190.stdout:Install 22 Packages 2024-05-06T12:01:57.436 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrade 2 Packages 2024-05-06T12:01:57.436 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:01:57.436 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 58 M 2024-05-06T12:01:57.436 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-05-06T12:01:57.484 INFO:teuthology.orchestra.run.smithi092.stdout:(13/24): ceph-base-17.2.7-925.ge81df420.el9.x86 2.0 MB/s | 5.0 MB 00:02 2024-05-06T12:01:57.668 INFO:teuthology.orchestra.run.smithi092.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 108 kB/s | 45 kB 00:00 2024-05-06T12:01:57.835 INFO:teuthology.orchestra.run.smithi092.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 263 kB/s | 160 kB 00:00 2024-05-06T12:01:57.993 INFO:teuthology.orchestra.run.smithi092.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 267 kB/s | 42 kB 00:00 2024-05-06T12:01:58.127 INFO:teuthology.orchestra.run.smithi092.stdout:(17/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 637 kB/s | 292 kB 00:00 2024-05-06T12:01:58.410 INFO:teuthology.orchestra.run.smithi092.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 171 kB/s | 49 kB 00:00 2024-05-06T12:01:58.502 INFO:teuthology.orchestra.run.smithi092.stdout:(19/24): libunwind-1.6.2-1.el9.x86_64.rpm 735 kB/s | 67 kB 00:00 2024-05-06T12:01:58.561 INFO:teuthology.orchestra.run.smithi092.stdout:(20/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 544 kB/s | 309 kB 00:00 2024-05-06T12:01:59.153 INFO:teuthology.orchestra.run.smithi092.stdout:(21/24): thrift-0.15.0-2.el9.x86_64.rpm 2.4 MB/s | 1.6 MB 00:00 2024-05-06T12:01:59.270 INFO:teuthology.orchestra.run.smithi092.stdout:(22/24): librdkafka-1.6.1-102.el9.x86_64.rpm 371 kB/s | 662 kB 00:01 2024-05-06T12:01:59.378 INFO:teuthology.orchestra.run.smithi190.stdout:(1/24): ceph-base-17.2.7-925.ge81df420.el9.x86_ 4.5 MB/s | 5.0 MB 00:01 2024-05-06T12:01:59.453 INFO:teuthology.orchestra.run.smithi092.stdout:(23/24): librbd1-17.2.7-925.ge81df420.el9.x86_6 9.8 MB/s | 2.9 MB 00:00 2024-05-06T12:01:59.520 INFO:teuthology.orchestra.run.smithi190.stdout:(2/24): ceph-radosgw-17.2.7-925.ge81df420.el9.x 11 MB/s | 14 MB 00:01 2024-05-06T12:01:59.545 INFO:teuthology.orchestra.run.smithi190.stdout:(3/24): ceph-selinux-17.2.7-925.ge81df420.el9.x 151 kB/s | 25 kB 00:00 2024-05-06T12:01:59.595 INFO:teuthology.orchestra.run.smithi190.stdout:(4/24): libcephfs2-17.2.7-925.ge81df420.el9.x86 8.3 MB/s | 638 kB 00:00 2024-05-06T12:01:59.620 INFO:teuthology.orchestra.run.smithi190.stdout:(5/24): libradosstriper1-17.2.7-925.ge81df420.e 6.0 MB/s | 458 kB 00:00 2024-05-06T12:01:59.679 INFO:teuthology.orchestra.run.smithi190.stdout:(6/24): python3-ceph-argparse-17.2.7-925.ge81df 776 kB/s | 45 kB 00:00 2024-05-06T12:01:59.712 INFO:teuthology.orchestra.run.smithi092.stdout:(24/24): librados2-17.2.7-925.ge81df420.el9.x86 2.7 MB/s | 3.1 MB 00:01 2024-05-06T12:01:59.716 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-06T12:01:59.716 INFO:teuthology.orchestra.run.smithi092.stdout:Total 12 MB/s | 58 MB 00:04 2024-05-06T12:01:59.746 INFO:teuthology.orchestra.run.smithi190.stdout:(7/24): python3-ceph-common-17.2.7-925.ge81df42 1.7 MB/s | 118 kB 00:00 2024-05-06T12:01:59.776 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-06T12:01:59.809 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-06T12:01:59.809 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-06T12:01:59.813 INFO:teuthology.orchestra.run.smithi190.stdout:(8/24): python3-cephfs-17.2.7-925.ge81df420.el9 2.3 MB/s | 156 kB 00:00 2024-05-06T12:01:59.880 INFO:teuthology.orchestra.run.smithi190.stdout:(9/24): librgw2-17.2.7-925.ge81df420.el9.x86_64 17 MB/s | 4.9 MB 00:00 2024-05-06T12:01:59.905 INFO:teuthology.orchestra.run.smithi190.stdout:(10/24): python3-rados-17.2.7-925.ge81df420.el9 3.4 MB/s | 319 kB 00:00 2024-05-06T12:01:59.930 INFO:teuthology.orchestra.run.smithi190.stdout:(11/24): python3-rbd-17.2.7-925.ge81df420.el9.x 5.7 MB/s | 293 kB 00:00 2024-05-06T12:01:59.956 INFO:teuthology.orchestra.run.smithi190.stdout:(12/24): python3-rgw-17.2.7-925.ge81df420.el9.x 1.9 MB/s | 99 kB 00:00 2024-05-06T12:02:00.014 INFO:teuthology.orchestra.run.smithi190.stdout:(13/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 776 kB/s | 45 kB 00:00 2024-05-06T12:02:00.056 INFO:teuthology.orchestra.run.smithi190.stdout:(14/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-05-06T12:02:00.140 INFO:teuthology.orchestra.run.smithi190.stdout:(15/24): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-05-06T12:02:00.165 INFO:teuthology.orchestra.run.smithi190.stdout:(16/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-05-06T12:02:00.215 INFO:teuthology.orchestra.run.smithi190.stdout:(17/24): python3-prettytable-0.7.2-27.el9.noarc 561 kB/s | 42 kB 00:00 2024-05-06T12:02:00.241 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-06T12:02:00.242 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-06T12:02:00.532 INFO:teuthology.orchestra.run.smithi190.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 168 kB/s | 49 kB 00:00 2024-05-06T12:02:00.633 INFO:teuthology.orchestra.run.smithi190.stdout:(19/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 661 kB/s | 309 kB 00:00 2024-05-06T12:02:00.741 INFO:teuthology.orchestra.run.smithi190.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 323 kB/s | 67 kB 00:00 2024-05-06T12:02:00.900 INFO:teuthology.orchestra.run.smithi190.stdout:(21/24): thrift-0.15.0-2.el9.x86_64.rpm 5.9 MB/s | 1.6 MB 00:00 2024-05-06T12:02:01.042 INFO:teuthology.orchestra.run.smithi190.stdout:(22/24): librados2-17.2.7-925.ge81df420.el9.x86 10 MB/s | 3.1 MB 00:00 2024-05-06T12:02:01.284 INFO:teuthology.orchestra.run.smithi190.stdout:(23/24): librbd1-17.2.7-925.ge81df420.el9.x86_6 7.7 MB/s | 2.9 MB 00:00 2024-05-06T12:02:01.506 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-06T12:02:01.904 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-05-06T12:02:01.934 INFO:teuthology.orchestra.run.smithi190.stdout:(24/24): ceph-common-17.2.7-925.ge81df420.el9.x 6.2 MB/s | 22 MB 00:03 2024-05-06T12:02:01.939 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-05-06T12:02:01.939 INFO:teuthology.orchestra.run.smithi190.stdout:Total 13 MB/s | 58 MB 00:04 2024-05-06T12:02:01.956 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librados2-2:17.2.7-925.ge81df420.el9.x86_64 2/26 2024-05-06T12:02:02.002 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-05-06T12:02:02.035 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-05-06T12:02:02.036 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-05-06T12:02:02.048 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:17.2.7-925.ge81df420.el9.x86_64 2/26 2024-05-06T12:02:02.126 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rados-2:17.2.7-925.ge81df420.el9.x86_64 3/26 2024-05-06T12:02:02.149 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs2-2:17.2.7-925.ge81df420.el9.x86_64 4/26 2024-05-06T12:02:02.228 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephfs2-2:17.2.7-925.ge81df420.el9.x86_64 4/26 2024-05-06T12:02:02.282 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-05-06T12:02:02.311 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-05-06T12:02:02.472 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-05-06T12:02:02.473 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-05-06T12:02:02.743 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-05-06T12:02:02.768 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librgw2-2:17.2.7-925.ge81df420.el9.x86_64 8/26 2024-05-06T12:02:02.831 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librgw2-2:17.2.7-925.ge81df420.el9.x86_64 8/26 2024-05-06T12:02:02.860 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-05-06T12:02:02.894 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-argparse-2:17.2.7-925.ge81df420.el9.x 10/26 2024-05-06T12:02:02.931 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cephfs-2:17.2.7-925.ge81df420.el9.x86_64 11/26 2024-05-06T12:02:02.993 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rgw-2:17.2.7-925.ge81df420.el9.x86_64 12/26 2024-05-06T12:02:03.021 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libradosstriper1-2:17.2.7-925.ge81df420.el9.x86_64 13/26 2024-05-06T12:02:03.086 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libradosstriper1-2:17.2.7-925.ge81df420.el9.x86_64 13/26 2024-05-06T12:02:03.125 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-05-06T12:02:03.154 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-05-06T12:02:03.187 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-05-06T12:02:03.479 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-05-06T12:02:03.517 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librbd1-2:17.2.7-925.ge81df420.el9.x86_64 18/26 2024-05-06T12:02:03.589 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:17.2.7-925.ge81df420.el9.x86_64 18/26 2024-05-06T12:02:03.663 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rbd-2:17.2.7-925.ge81df420.el9.x86_64 19/26 2024-05-06T12:02:03.694 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-common-2:17.2.7-925.ge81df420.el9.x86 20/26 2024-05-06T12:02:03.703 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-05-06T12:02:04.047 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-05-06T12:02:04.161 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librados2-2:17.2.7-925.ge81df420.el9.x86_64 2/26 2024-05-06T12:02:04.253 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-2:17.2.7-925.ge81df420.el9.x86_64 2/26 2024-05-06T12:02:04.332 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rados-2:17.2.7-925.ge81df420.el9.x86_64 3/26 2024-05-06T12:02:04.363 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephfs2-2:17.2.7-925.ge81df420.el9.x86_64 4/26 2024-05-06T12:02:04.450 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephfs2-2:17.2.7-925.ge81df420.el9.x86_64 4/26 2024-05-06T12:02:04.504 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-05-06T12:02:04.534 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-05-06T12:02:04.969 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-05-06T12:02:04.999 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librgw2-2:17.2.7-925.ge81df420.el9.x86_64 8/26 2024-05-06T12:02:05.045 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librgw2-2:17.2.7-925.ge81df420.el9.x86_64 8/26 2024-05-06T12:02:05.074 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-05-06T12:02:05.126 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ceph-argparse-2:17.2.7-925.ge81df420.el9.x 10/26 2024-05-06T12:02:05.162 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cephfs-2:17.2.7-925.ge81df420.el9.x86_64 11/26 2024-05-06T12:02:05.225 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rgw-2:17.2.7-925.ge81df420.el9.x86_64 12/26 2024-05-06T12:02:05.252 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libradosstriper1-2:17.2.7-925.ge81df420.el9.x86_64 13/26 2024-05-06T12:02:05.309 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libradosstriper1-2:17.2.7-925.ge81df420.el9.x86_64 13/26 2024-05-06T12:02:05.348 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-05-06T12:02:05.377 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-05-06T12:02:05.409 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-05-06T12:02:05.703 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-05-06T12:02:05.748 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librbd1-2:17.2.7-925.ge81df420.el9.x86_64 18/26 2024-05-06T12:02:05.980 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-2:17.2.7-925.ge81df420.el9.x86_64 18/26 2024-05-06T12:02:06.053 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rbd-2:17.2.7-925.ge81df420.el9.x86_64 19/26 2024-05-06T12:02:06.084 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ceph-common-2:17.2.7-925.ge81df420.el9.x86 20/26 2024-05-06T12:02:06.103 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:17.2.7-925.ge81df420.el9.x86_64 21/26 2024-05-06T12:02:06.141 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-common-2:17.2.7-925.ge81df420.el9.x86_64 21/26 2024-05-06T12:02:06.647 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:17.2.7-925.ge81df420.el9.x86_64 21/26 2024-05-06T12:02:06.711 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-base-2:17.2.7-925.ge81df420.el9.x86_64 22/26 2024-05-06T12:02:06.811 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-base-2:17.2.7-925.ge81df420.el9.x86_64 22/26 2024-05-06T12:02:06.811 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-06T12:02:06.811 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-06T12:02:06.811 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:02:06.846 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-selinux-2:17.2.7-925.ge81df420.el9.x86_64 23/26 2024-05-06T12:02:08.559 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:17.2.7-925.ge81df420.el9.x86_64 21/26 2024-05-06T12:02:08.597 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-common-2:17.2.7-925.ge81df420.el9.x86_64 21/26 2024-05-06T12:02:09.110 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:17.2.7-925.ge81df420.el9.x86_64 21/26 2024-05-06T12:02:09.151 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-base-2:17.2.7-925.ge81df420.el9.x86_64 22/26 2024-05-06T12:02:09.253 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-base-2:17.2.7-925.ge81df420.el9.x86_64 22/26 2024-05-06T12:02:09.253 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-06T12:02:09.253 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-06T12:02:09.253 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:02:09.285 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-selinux-2:17.2.7-925.ge81df420.el9.x86_64 23/26 2024-05-06T12:02:15.955 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-selinux-2:17.2.7-925.ge81df420.el9.x86_64 23/26 2024-05-06T12:02:15.956 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /sys 2024-05-06T12:02:15.956 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /proc 2024-05-06T12:02:15.956 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /mnt 2024-05-06T12:02:15.956 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /var/tmp 2024-05-06T12:02:15.956 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /home 2024-05-06T12:02:15.956 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /root 2024-05-06T12:02:15.956 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /tmp 2024-05-06T12:02:15.956 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:02:17.187 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-radosgw-2:17.2.7-925.ge81df420.el9.x86_64 24/26 2024-05-06T12:02:17.256 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-925.ge81df420.el9.x86_64 24/26 2024-05-06T12:02:17.256 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-06T12:02:17.256 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-06T12:02:17.256 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-06T12:02:17.256 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-06T12:02:17.256 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:02:17.258 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-05-06T12:02:17.340 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-05-06T12:02:17.340 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-05-06T12:02:18.545 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-selinux-2:17.2.7-925.ge81df420.el9.x86_64 23/26 2024-05-06T12:02:18.545 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /sys 2024-05-06T12:02:18.545 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /proc 2024-05-06T12:02:18.545 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /mnt 2024-05-06T12:02:18.545 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /var/tmp 2024-05-06T12:02:18.545 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /home 2024-05-06T12:02:18.545 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /root 2024-05-06T12:02:18.545 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /tmp 2024-05-06T12:02:18.545 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:02:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-05-06T12:02:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-base-2:17.2.7-925.ge81df420.el9.x86_64 1/26 2024-05-06T12:02:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-common-2:17.2.7-925.ge81df420.el9.x86_64 2/26 2024-05-06T12:02:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-radosgw-2:17.2.7-925.ge81df420.el9.x86_64 3/26 2024-05-06T12:02:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-selinux-2:17.2.7-925.ge81df420.el9.x86_64 4/26 2024-05-06T12:02:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs2-2:17.2.7-925.ge81df420.el9.x86_64 5/26 2024-05-06T12:02:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libradosstriper1-2:17.2.7-925.ge81df420.el9.x86_64 6/26 2024-05-06T12:02:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librgw2-2:17.2.7-925.ge81df420.el9.x86_64 7/26 2024-05-06T12:02:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-argparse-2:17.2.7-925.ge81df420.el9.x 8/26 2024-05-06T12:02:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-common-2:17.2.7-925.ge81df420.el9.x86 9/26 2024-05-06T12:02:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cephfs-2:17.2.7-925.ge81df420.el9.x86_64 10/26 2024-05-06T12:02:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rados-2:17.2.7-925.ge81df420.el9.x86_64 11/26 2024-05-06T12:02:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rbd-2:17.2.7-925.ge81df420.el9.x86_64 12/26 2024-05-06T12:02:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rgw-2:17.2.7-925.ge81df420.el9.x86_64 13/26 2024-05-06T12:02:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-05-06T12:02:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-05-06T12:02:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-05-06T12:02:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-05-06T12:02:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-05-06T12:02:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-05-06T12:02:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-05-06T12:02:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-05-06T12:02:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-05-06T12:02:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:17.2.7-925.ge81df420.el9.x86_64 23/26 2024-05-06T12:02:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-05-06T12:02:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:17.2.7-925.ge81df420.el9.x86_64 25/26 2024-05-06T12:02:19.235 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-05-06T12:02:19.235 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:02:19.236 INFO:teuthology.orchestra.run.smithi092.stdout:Upgraded: 2024-05-06T12:02:19.236 INFO:teuthology.orchestra.run.smithi092.stdout: librados2-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:19.236 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:19.236 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-06T12:02:19.236 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:19.236 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:19.236 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:19.236 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:19.236 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-06T12:02:19.236 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:19.236 INFO:teuthology.orchestra.run.smithi092.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-06T12:02:19.236 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-06T12:02:19.236 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-06T12:02:19.237 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:19.237 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-06T12:02:19.237 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:19.237 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-06T12:02:19.237 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-06T12:02:19.237 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:19.237 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:19.237 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:19.237 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-06T12:02:19.237 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:19.237 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:19.237 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:19.237 INFO:teuthology.orchestra.run.smithi092.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-06T12:02:19.237 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:02:19.238 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-06T12:02:19.520 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-test 2024-05-06T12:02:19.769 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-radosgw-2:17.2.7-925.ge81df420.el9.x86_64 24/26 2024-05-06T12:02:19.846 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-925.ge81df420.el9.x86_64 24/26 2024-05-06T12:02:19.846 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-06T12:02:19.846 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-06T12:02:19.846 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-06T12:02:19.847 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-06T12:02:19.847 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:02:19.848 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-05-06T12:02:19.913 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-05-06T12:02:19.913 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-05-06T12:02:20.139 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:28 ago on Mon 06 May 2024 12:01:52 PM UTC. 2024-05-06T12:02:20.265 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-06T12:02:20.266 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:02:20.266 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-05-06T12:02:20.266 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:02:20.266 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-06T12:02:20.266 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test x86_64 2:17.2.7-925.ge81df420.el9 ceph 48 M 2024-05-06T12:02:20.266 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-05-06T12:02:20.266 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite x86_64 2:17.2.7-925.ge81df420.el9 ceph 163 k 2024-05-06T12:02:20.266 INFO:teuthology.orchestra.run.smithi092.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-06T12:02:20.267 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-06T12:02:20.267 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:02:20.267 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-06T12:02:20.267 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:02:20.267 INFO:teuthology.orchestra.run.smithi092.stdout:Install 4 Packages 2024-05-06T12:02:20.267 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:02:20.267 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 48 M 2024-05-06T12:02:20.268 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 192 M 2024-05-06T12:02:20.268 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-06T12:02:20.915 INFO:teuthology.orchestra.run.smithi092.stdout:(1/4): libcephsqlite-17.2.7-925.ge81df420.el9.x 607 kB/s | 163 kB 00:00 2024-05-06T12:02:21.127 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-05-06T12:02:21.127 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-base-2:17.2.7-925.ge81df420.el9.x86_64 1/26 2024-05-06T12:02:21.127 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-common-2:17.2.7-925.ge81df420.el9.x86_64 2/26 2024-05-06T12:02:21.127 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-radosgw-2:17.2.7-925.ge81df420.el9.x86_64 3/26 2024-05-06T12:02:21.127 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-selinux-2:17.2.7-925.ge81df420.el9.x86_64 4/26 2024-05-06T12:02:21.127 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs2-2:17.2.7-925.ge81df420.el9.x86_64 5/26 2024-05-06T12:02:21.128 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libradosstriper1-2:17.2.7-925.ge81df420.el9.x86_64 6/26 2024-05-06T12:02:21.128 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librgw2-2:17.2.7-925.ge81df420.el9.x86_64 7/26 2024-05-06T12:02:21.128 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-argparse-2:17.2.7-925.ge81df420.el9.x 8/26 2024-05-06T12:02:21.128 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-common-2:17.2.7-925.ge81df420.el9.x86 9/26 2024-05-06T12:02:21.128 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cephfs-2:17.2.7-925.ge81df420.el9.x86_64 10/26 2024-05-06T12:02:21.128 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rados-2:17.2.7-925.ge81df420.el9.x86_64 11/26 2024-05-06T12:02:21.128 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rbd-2:17.2.7-925.ge81df420.el9.x86_64 12/26 2024-05-06T12:02:21.128 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rgw-2:17.2.7-925.ge81df420.el9.x86_64 13/26 2024-05-06T12:02:21.129 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-05-06T12:02:21.129 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-05-06T12:02:21.129 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-05-06T12:02:21.129 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-05-06T12:02:21.129 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-05-06T12:02:21.129 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-05-06T12:02:21.129 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-05-06T12:02:21.130 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-05-06T12:02:21.130 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-05-06T12:02:21.130 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-2:17.2.7-925.ge81df420.el9.x86_64 23/26 2024-05-06T12:02:21.130 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-05-06T12:02:21.130 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-2:17.2.7-925.ge81df420.el9.x86_64 25/26 2024-05-06T12:02:21.277 INFO:teuthology.orchestra.run.smithi092.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 176 kB/s | 64 kB 00:00 2024-05-06T12:02:21.327 INFO:teuthology.orchestra.run.smithi092.stdout:(3/4): socat-1.7.4.1-5.el9.x86_64.rpm 448 kB/s | 305 kB 00:00 2024-05-06T12:02:21.808 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-05-06T12:02:21.808 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:02:21.808 INFO:teuthology.orchestra.run.smithi190.stdout:Upgraded: 2024-05-06T12:02:21.809 INFO:teuthology.orchestra.run.smithi190.stdout: librados2-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:21.809 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:21.809 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-05-06T12:02:21.809 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:21.809 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:21.809 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:21.809 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:21.809 INFO:teuthology.orchestra.run.smithi190.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-06T12:02:21.809 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:21.809 INFO:teuthology.orchestra.run.smithi190.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-06T12:02:21.809 INFO:teuthology.orchestra.run.smithi190.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-06T12:02:21.809 INFO:teuthology.orchestra.run.smithi190.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-06T12:02:21.810 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:21.810 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-06T12:02:21.810 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:21.810 INFO:teuthology.orchestra.run.smithi190.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-06T12:02:21.810 INFO:teuthology.orchestra.run.smithi190.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-06T12:02:21.810 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:21.810 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:21.810 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:21.810 INFO:teuthology.orchestra.run.smithi190.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-06T12:02:21.810 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:21.810 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:21.810 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:21.810 INFO:teuthology.orchestra.run.smithi190.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-06T12:02:21.810 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:02:21.811 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-05-06T12:02:22.054 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-test 2024-05-06T12:02:22.589 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:00:27 ago on Mon 06 May 2024 12:01:55 PM UTC. 2024-05-06T12:02:22.710 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-05-06T12:02:22.711 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:02:22.711 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2024-05-06T12:02:22.711 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:02:22.711 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-05-06T12:02:22.711 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test x86_64 2:17.2.7-925.ge81df420.el9 ceph 48 M 2024-05-06T12:02:22.711 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-05-06T12:02:22.711 INFO:teuthology.orchestra.run.smithi190.stdout: libcephsqlite x86_64 2:17.2.7-925.ge81df420.el9 ceph 163 k 2024-05-06T12:02:22.711 INFO:teuthology.orchestra.run.smithi190.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-06T12:02:22.711 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-06T12:02:22.712 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:02:22.712 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-05-06T12:02:22.712 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:02:22.712 INFO:teuthology.orchestra.run.smithi190.stdout:Install 4 Packages 2024-05-06T12:02:22.712 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:02:22.712 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 48 M 2024-05-06T12:02:22.712 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 192 M 2024-05-06T12:02:22.713 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-05-06T12:02:22.965 INFO:teuthology.orchestra.run.smithi092.stdout:(4/4): ceph-test-17.2.7-925.ge81df420.el9.x86_6 21 MB/s | 48 MB 00:02 2024-05-06T12:02:22.967 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-06T12:02:22.967 INFO:teuthology.orchestra.run.smithi092.stdout:Total 18 MB/s | 48 MB 00:02 2024-05-06T12:02:22.993 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-06T12:02:23.012 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-06T12:02:23.012 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-06T12:02:23.373 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-06T12:02:23.373 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-06T12:02:23.882 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-06T12:02:23.949 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-05-06T12:02:23.997 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-05-06T12:02:24.025 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephsqlite-2:17.2.7-925.ge81df420.el9.x86_64 3/4 2024-05-06T12:02:25.404 INFO:teuthology.orchestra.run.smithi190.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 896 kB/s | 305 kB 00:00 2024-05-06T12:02:25.479 INFO:teuthology.orchestra.run.smithi190.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 852 kB/s | 64 kB 00:00 2024-05-06T12:02:25.596 INFO:teuthology.orchestra.run.smithi190.stdout:(3/4): libcephsqlite-17.2.7-925.ge81df420.el9.x 305 kB/s | 163 kB 00:00 2024-05-06T12:02:28.154 INFO:teuthology.orchestra.run.smithi190.stdout:(4/4): ceph-test-17.2.7-925.ge81df420.el9.x86_6 15 MB/s | 48 MB 00:03 2024-05-06T12:02:28.156 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-05-06T12:02:28.157 INFO:teuthology.orchestra.run.smithi190.stdout:Total 8.9 MB/s | 48 MB 00:05 2024-05-06T12:02:28.183 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-05-06T12:02:28.203 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-05-06T12:02:28.203 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-05-06T12:02:28.367 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephsqlite-2:17.2.7-925.ge81df420.el9.x86_64 3/4 2024-05-06T12:02:28.446 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-test-2:17.2.7-925.ge81df420.el9.x86_64 4/4 2024-05-06T12:02:28.568 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-05-06T12:02:28.568 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-05-06T12:02:29.103 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-05-06T12:02:29.172 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-05-06T12:02:29.220 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-05-06T12:02:29.256 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephsqlite-2:17.2.7-925.ge81df420.el9.x86_64 3/4 2024-05-06T12:02:30.753 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-test-2:17.2.7-925.ge81df420.el9.x86_64 4/4 2024-05-06T12:02:30.754 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-test-2:17.2.7-925.ge81df420.el9.x86_64 1/4 2024-05-06T12:02:30.754 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephsqlite-2:17.2.7-925.ge81df420.el9.x86_64 2/4 2024-05-06T12:02:30.754 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-05-06T12:02:31.039 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-05-06T12:02:31.039 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:02:31.039 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-06T12:02:31.040 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:31.040 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:31.040 INFO:teuthology.orchestra.run.smithi092.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-06T12:02:31.040 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-06T12:02:31.040 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:02:31.040 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-06T12:02:31.281 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph 2024-05-06T12:02:31.786 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:39 ago on Mon 06 May 2024 12:01:52 PM UTC. 2024-05-06T12:02:31.912 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-06T12:02:31.914 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:02:31.914 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-05-06T12:02:31.915 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:02:31.915 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-06T12:02:31.915 INFO:teuthology.orchestra.run.smithi092.stdout: ceph x86_64 2:17.2.7-925.ge81df420.el9 ceph 6.5 k 2024-05-06T12:02:31.915 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-05-06T12:02:31.915 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds x86_64 2:17.2.7-925.ge81df420.el9 ceph 2.1 M 2024-05-06T12:02:31.915 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr x86_64 2:17.2.7-925.ge81df420.el9 ceph 1.4 M 2024-05-06T12:02:31.915 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core noarch 2:17.2.7-925.ge81df420.el9 ceph-noarch 240 k 2024-05-06T12:02:31.915 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon x86_64 2:17.2.7-925.ge81df420.el9 ceph 3.9 M 2024-05-06T12:02:31.915 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd x86_64 2:17.2.7-925.ge81df420.el9 ceph 15 M 2024-05-06T12:02:31.915 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-06T12:02:31.915 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-06T12:02:31.915 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-06T12:02:31.916 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-06T12:02:31.916 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-06T12:02:31.916 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-06T12:02:31.916 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-06T12:02:31.916 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-06T12:02:31.916 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-06T12:02:31.916 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-06T12:02:31.916 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-06T12:02:31.916 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-06T12:02:31.916 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-06T12:02:31.916 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-06T12:02:31.916 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-06T12:02:31.917 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-06T12:02:31.917 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-06T12:02:31.917 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-06T12:02:31.917 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-06T12:02:31.917 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-06T12:02:31.917 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-06T12:02:31.917 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-06T12:02:31.917 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-06T12:02:31.917 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-06T12:02:31.917 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-06T12:02:31.917 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-06T12:02:31.917 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-06T12:02:31.917 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:02:31.917 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-06T12:02:31.917 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:02:31.918 INFO:teuthology.orchestra.run.smithi092.stdout:Install 33 Packages 2024-05-06T12:02:31.918 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:02:31.918 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 27 M 2024-05-06T12:02:31.918 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 96 M 2024-05-06T12:02:31.918 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-06T12:02:32.490 INFO:teuthology.orchestra.run.smithi092.stdout:(1/33): ceph-17.2.7-925.ge81df420.el9.x86_64.rp 40 kB/s | 6.5 kB 00:00 2024-05-06T12:02:32.741 INFO:teuthology.orchestra.run.smithi092.stdout:(2/33): ceph-mgr-17.2.7-925.ge81df420.el9.x86_6 3.4 MB/s | 1.4 MB 00:00 2024-05-06T12:02:32.966 INFO:teuthology.orchestra.run.smithi092.stdout:(3/33): ceph-mon-17.2.7-925.ge81df420.el9.x86_6 8.2 MB/s | 3.9 MB 00:00 2024-05-06T12:02:33.058 INFO:teuthology.orchestra.run.smithi092.stdout:(4/33): ceph-mgr-modules-core-17.2.7-925.ge81df 2.6 MB/s | 240 kB 00:00 2024-05-06T12:02:33.258 INFO:teuthology.orchestra.run.smithi092.stdout:(5/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-05-06T12:02:33.534 INFO:teuthology.orchestra.run.smithi092.stdout:(6/33): ceph-osd-17.2.7-925.ge81df420.el9.x86_6 19 MB/s | 15 MB 00:00 2024-05-06T12:02:33.578 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephsqlite-2:17.2.7-925.ge81df420.el9.x86_64 3/4 2024-05-06T12:02:33.650 INFO:teuthology.orchestra.run.smithi092.stdout:(7/33): ceph-mds-17.2.7-925.ge81df420.el9.x86_6 1.6 MB/s | 2.1 MB 00:01 2024-05-06T12:02:33.652 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-test-2:17.2.7-925.ge81df420.el9.x86_64 4/4 2024-05-06T12:02:33.701 INFO:teuthology.orchestra.run.smithi092.stdout:(8/33): python3-ply-3.11-14.el9.noarch.rpm 638 kB/s | 106 kB 00:00 2024-05-06T12:02:33.751 INFO:teuthology.orchestra.run.smithi092.stdout:(9/33): python3-cryptography-36.0.1-4.el9.x86_6 2.5 MB/s | 1.2 MB 00:00 2024-05-06T12:02:33.793 INFO:teuthology.orchestra.run.smithi092.stdout:(10/33): python3-requests-2.25.1-8.el9.noarch.r 1.3 MB/s | 125 kB 00:00 2024-05-06T12:02:33.818 INFO:teuthology.orchestra.run.smithi092.stdout:(11/33): python3-pycparser-2.20-6.el9.noarch.rp 808 kB/s | 135 kB 00:00 2024-05-06T12:02:33.844 INFO:teuthology.orchestra.run.smithi092.stdout:(12/33): python3-urllib3-1.26.5-5.el9.noarch.rp 2.3 MB/s | 215 kB 00:00 2024-05-06T12:02:33.877 INFO:teuthology.orchestra.run.smithi092.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 594 kB/s | 35 kB 00:00 2024-05-06T12:02:33.902 INFO:teuthology.orchestra.run.smithi092.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 872 kB/s | 51 kB 00:00 2024-05-06T12:02:33.927 INFO:teuthology.orchestra.run.smithi092.stdout:(15/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-05-06T12:02:33.953 INFO:teuthology.orchestra.run.smithi092.stdout:(16/33): python3-toml-0.10.2-6.el9.noarch.rpm 551 kB/s | 42 kB 00:00 2024-05-06T12:02:34.203 INFO:teuthology.orchestra.run.smithi092.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 145 kB/s | 43 kB 00:00 2024-05-06T12:02:34.262 INFO:teuthology.orchestra.run.smithi092.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 183 kB/s | 11 kB 00:00 2024-05-06T12:02:34.304 INFO:teuthology.orchestra.run.smithi092.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 457 kB/s | 172 kB 00:00 2024-05-06T12:02:34.321 INFO:teuthology.orchestra.run.smithi092.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 303 kB/s | 18 kB 00:00 2024-05-06T12:02:34.354 INFO:teuthology.orchestra.run.smithi092.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 463 kB/s | 23 kB 00:00 2024-05-06T12:02:34.379 INFO:teuthology.orchestra.run.smithi092.stdout:(22/33): python3-cherrypy-18.6.1-2.el9.noarch.r 840 kB/s | 358 kB 00:00 2024-05-06T12:02:34.396 INFO:teuthology.orchestra.run.smithi092.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 258 kB/s | 19 kB 00:00 2024-05-06T12:02:34.421 INFO:teuthology.orchestra.run.smithi092.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 295 kB/s | 20 kB 00:00 2024-05-06T12:02:34.447 INFO:teuthology.orchestra.run.smithi092.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 691 kB/s | 46 kB 00:00 2024-05-06T12:02:34.472 INFO:teuthology.orchestra.run.smithi092.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-05-06T12:02:34.531 INFO:teuthology.orchestra.run.smithi092.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-05-06T12:02:34.548 INFO:teuthology.orchestra.run.smithi092.stdout:(28/33): python3-portend-3.1.0-2.el9.noarch.rpm 164 kB/s | 16 kB 00:00 2024-05-06T12:02:34.573 INFO:teuthology.orchestra.run.smithi092.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 898 kB/s | 90 kB 00:00 2024-05-06T12:02:34.589 INFO:teuthology.orchestra.run.smithi092.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 612 kB/s | 36 kB 00:00 2024-05-06T12:02:34.648 INFO:teuthology.orchestra.run.smithi092.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-05-06T12:02:34.698 INFO:teuthology.orchestra.run.smithi092.stdout:(32/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.3 MB/s | 427 kB 00:00 2024-05-06T12:02:34.723 INFO:teuthology.orchestra.run.smithi092.stdout:(33/33): python3-zc-lockfile-2.0-10.el9.noarch. 149 kB/s | 20 kB 00:00 2024-05-06T12:02:34.729 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-06T12:02:34.729 INFO:teuthology.orchestra.run.smithi092.stdout:Total 9.7 MB/s | 27 MB 00:02 2024-05-06T12:02:34.856 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-06T12:02:34.899 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-06T12:02:34.899 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-06T12:02:35.170 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-06T12:02:35.171 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-06T12:02:35.886 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-test-2:17.2.7-925.ge81df420.el9.x86_64 4/4 2024-05-06T12:02:35.886 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-test-2:17.2.7-925.ge81df420.el9.x86_64 1/4 2024-05-06T12:02:35.887 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephsqlite-2:17.2.7-925.ge81df420.el9.x86_64 2/4 2024-05-06T12:02:35.887 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-05-06T12:02:36.161 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-05-06T12:02:36.161 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:02:36.161 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-05-06T12:02:36.161 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:36.161 INFO:teuthology.orchestra.run.smithi190.stdout: libcephsqlite-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:36.162 INFO:teuthology.orchestra.run.smithi190.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-06T12:02:36.162 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-06T12:02:36.162 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:02:36.162 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-05-06T12:02:36.305 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-06T12:02:36.376 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-05-06T12:02:36.409 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-05-06T12:02:36.432 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph 2024-05-06T12:02:36.441 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-05-06T12:02:36.534 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-05-06T12:02:36.620 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-05-06T12:02:36.661 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-05-06T12:02:36.704 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-05-06T12:02:36.734 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-05-06T12:02:36.769 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-05-06T12:02:36.802 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-05-06T12:02:36.842 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-05-06T12:02:36.877 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-05-06T12:02:36.911 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-05-06T12:02:36.945 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:00:41 ago on Mon 06 May 2024 12:01:55 PM UTC. 2024-05-06T12:02:36.950 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-05-06T12:02:37.024 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-05-06T12:02:37.075 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-05-06T12:02:37.077 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:02:37.077 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2024-05-06T12:02:37.078 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:02:37.078 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-05-06T12:02:37.078 INFO:teuthology.orchestra.run.smithi190.stdout: ceph x86_64 2:17.2.7-925.ge81df420.el9 ceph 6.5 k 2024-05-06T12:02:37.078 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-05-06T12:02:37.078 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds x86_64 2:17.2.7-925.ge81df420.el9 ceph 2.1 M 2024-05-06T12:02:37.078 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr x86_64 2:17.2.7-925.ge81df420.el9 ceph 1.4 M 2024-05-06T12:02:37.078 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core noarch 2:17.2.7-925.ge81df420.el9 ceph-noarch 240 k 2024-05-06T12:02:37.078 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon x86_64 2:17.2.7-925.ge81df420.el9 ceph 3.9 M 2024-05-06T12:02:37.078 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd x86_64 2:17.2.7-925.ge81df420.el9 ceph 15 M 2024-05-06T12:02:37.078 INFO:teuthology.orchestra.run.smithi190.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-06T12:02:37.078 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-06T12:02:37.078 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-06T12:02:37.079 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-06T12:02:37.079 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-06T12:02:37.079 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-06T12:02:37.079 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-06T12:02:37.079 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-06T12:02:37.079 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-06T12:02:37.079 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-06T12:02:37.079 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-06T12:02:37.079 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-06T12:02:37.079 INFO:teuthology.orchestra.run.smithi190.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-06T12:02:37.079 INFO:teuthology.orchestra.run.smithi190.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-06T12:02:37.079 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-06T12:02:37.079 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-06T12:02:37.079 INFO:teuthology.orchestra.run.smithi190.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-06T12:02:37.080 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-06T12:02:37.080 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-06T12:02:37.080 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-06T12:02:37.080 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-06T12:02:37.080 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-06T12:02:37.080 INFO:teuthology.orchestra.run.smithi190.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-06T12:02:37.080 INFO:teuthology.orchestra.run.smithi190.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-06T12:02:37.080 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-06T12:02:37.080 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-06T12:02:37.080 INFO:teuthology.orchestra.run.smithi190.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-06T12:02:37.080 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:02:37.080 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-05-06T12:02:37.080 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:02:37.080 INFO:teuthology.orchestra.run.smithi190.stdout:Install 33 Packages 2024-05-06T12:02:37.080 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:02:37.081 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 27 M 2024-05-06T12:02:37.081 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 96 M 2024-05-06T12:02:37.081 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-05-06T12:02:37.171 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-05-06T12:02:37.250 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-05-06T12:02:37.303 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-05-06T12:02:37.350 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-05-06T12:02:37.406 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-05-06T12:02:37.463 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-05-06T12:02:37.648 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-05-06T12:02:37.696 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-05-06T12:02:37.767 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-05-06T12:02:37.856 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-05-06T12:02:37.895 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-05-06T12:02:38.008 INFO:teuthology.orchestra.run.smithi190.stdout:(1/33): ceph-17.2.7-925.ge81df420.el9.x86_64.rp 29 kB/s | 6.5 kB 00:00 2024-05-06T12:02:38.010 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-05-06T12:02:38.173 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-925.ge81df420.el9.n 28/33 2024-05-06T12:02:38.205 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-2:17.2.7-925.ge81df420.el9.x86_64 29/33 2024-05-06T12:02:38.230 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-2:17.2.7-925.ge81df420.el9.x86_64 29/33 2024-05-06T12:02:38.230 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-06T12:02:38.230 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-06T12:02:38.230 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-06T12:02:38.230 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-06T12:02:38.230 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:02:38.317 INFO:teuthology.orchestra.run.smithi190.stdout:(2/33): ceph-mgr-17.2.7-925.ge81df420.el9.x86_6 2.6 MB/s | 1.4 MB 00:00 2024-05-06T12:02:38.367 INFO:teuthology.orchestra.run.smithi190.stdout:(3/33): ceph-mds-17.2.7-925.ge81df420.el9.x86_6 3.5 MB/s | 2.1 MB 00:00 2024-05-06T12:02:38.484 INFO:teuthology.orchestra.run.smithi190.stdout:(4/33): ceph-mgr-modules-core-17.2.7-925.ge81df 2.0 MB/s | 240 kB 00:00 2024-05-06T12:02:38.559 INFO:teuthology.orchestra.run.smithi190.stdout:(5/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-05-06T12:02:38.685 INFO:teuthology.orchestra.run.smithi190.stdout:(6/33): python3-cryptography-36.0.1-4.el9.x86_6 9.8 MB/s | 1.2 MB 00:00 2024-05-06T12:02:38.743 INFO:teuthology.orchestra.run.smithi190.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-05-06T12:02:38.810 INFO:teuthology.orchestra.run.smithi190.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 2.0 MB/s | 135 kB 00:00 2024-05-06T12:02:38.869 INFO:teuthology.orchestra.run.smithi190.stdout:(9/33): python3-requests-2.25.1-8.el9.noarch.rp 2.1 MB/s | 125 kB 00:00 2024-05-06T12:02:38.944 INFO:teuthology.orchestra.run.smithi190.stdout:(10/33): python3-urllib3-1.26.5-5.el9.noarch.rp 2.8 MB/s | 215 kB 00:00 2024-05-06T12:02:39.011 INFO:teuthology.orchestra.run.smithi190.stdout:(11/33): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-05-06T12:02:39.061 INFO:teuthology.orchestra.run.smithi190.stdout:(12/33): python3-markupsafe-1.1.1-12.el9.x86_64 695 kB/s | 35 kB 00:00 2024-05-06T12:02:39.128 INFO:teuthology.orchestra.run.smithi190.stdout:(13/33): ceph-mon-17.2.7-925.ge81df420.el9.x86_ 3.5 MB/s | 3.9 MB 00:01 2024-05-06T12:02:39.278 INFO:teuthology.orchestra.run.smithi190.stdout:(14/33): ceph-osd-17.2.7-925.ge81df420.el9.x86_ 16 MB/s | 15 MB 00:00 2024-05-06T12:02:39.303 INFO:teuthology.orchestra.run.smithi190.stdout:(15/33): python3-pytz-2021.1-5.el9.noarch.rpm 210 kB/s | 51 kB 00:00 2024-05-06T12:02:39.329 INFO:teuthology.orchestra.run.smithi190.stdout:(16/33): python3-toml-0.10.2-6.el9.noarch.rpm 208 kB/s | 42 kB 00:00 2024-05-06T12:02:39.454 INFO:teuthology.orchestra.run.smithi190.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 248 kB/s | 43 kB 00:00 2024-05-06T12:02:39.512 INFO:teuthology.orchestra.run.smithi190.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 183 kB/s | 11 kB 00:00 2024-05-06T12:02:39.538 INFO:teuthology.orchestra.run.smithi190.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 733 kB/s | 172 kB 00:00 2024-05-06T12:02:39.567 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-osd-2:17.2.7-925.ge81df420.el9.x86_64 30/33 2024-05-06T12:02:39.572 INFO:teuthology.orchestra.run.smithi190.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 301 kB/s | 18 kB 00:00 2024-05-06T12:02:39.599 INFO:teuthology.orchestra.run.smithi190.stdout:(21/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-05-06T12:02:39.599 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-osd-2:17.2.7-925.ge81df420.el9.x86_64 30/33 2024-05-06T12:02:39.599 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-06T12:02:39.599 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-06T12:02:39.600 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-06T12:02:39.600 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-06T12:02:39.600 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:02:39.622 INFO:teuthology.orchestra.run.smithi190.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 277 kB/s | 23 kB 00:00 2024-05-06T12:02:39.647 INFO:teuthology.orchestra.run.smithi190.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 258 kB/s | 19 kB 00:00 2024-05-06T12:02:39.681 INFO:teuthology.orchestra.run.smithi190.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 337 kB/s | 20 kB 00:00 2024-05-06T12:02:39.706 INFO:teuthology.orchestra.run.smithi190.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 554 kB/s | 46 kB 00:00 2024-05-06T12:02:39.732 INFO:teuthology.orchestra.run.smithi190.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 938 kB/s | 79 kB 00:00 2024-05-06T12:02:39.757 INFO:teuthology.orchestra.run.smithi190.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 324 kB/s | 16 kB 00:00 2024-05-06T12:02:39.790 INFO:teuthology.orchestra.run.smithi190.stdout:(28/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-05-06T12:02:39.807 INFO:teuthology.orchestra.run.smithi190.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 716 kB/s | 36 kB 00:00 2024-05-06T12:02:39.824 INFO:teuthology.orchestra.run.smithi190.stdout:(30/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 977 kB/s | 90 kB 00:00 2024-05-06T12:02:39.858 INFO:teuthology.orchestra.run.smithi190.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 3.3 MB/s | 230 kB 00:00 2024-05-06T12:02:39.875 INFO:teuthology.orchestra.run.smithi190.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 398 kB/s | 20 kB 00:00 2024-05-06T12:02:39.900 INFO:teuthology.orchestra.run.smithi190.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.5 MB/s | 427 kB 00:00 2024-05-06T12:02:39.903 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-05-06T12:02:39.903 INFO:teuthology.orchestra.run.smithi190.stdout:Total 9.6 MB/s | 27 MB 00:02 2024-05-06T12:02:39.968 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mon-2:17.2.7-925.ge81df420.el9.x86_64 31/33 2024-05-06T12:02:39.992 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mon-2:17.2.7-925.ge81df420.el9.x86_64 31/33 2024-05-06T12:02:39.992 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-06T12:02:39.992 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-06T12:02:39.992 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-06T12:02:39.993 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-06T12:02:39.993 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:02:40.019 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-05-06T12:02:40.055 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-05-06T12:02:40.055 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-05-06T12:02:40.194 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mds-2:17.2.7-925.ge81df420.el9.x86_64 32/33 2024-05-06T12:02:40.217 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mds-2:17.2.7-925.ge81df420.el9.x86_64 32/33 2024-05-06T12:02:40.217 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-06T12:02:40.217 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-06T12:02:40.217 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-06T12:02:40.217 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-06T12:02:40.218 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:02:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-2:17.2.7-925.ge81df420.el9.x86_64 33/33 2024-05-06T12:02:40.320 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-05-06T12:02:40.321 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-05-06T12:02:41.600 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-05-06T12:02:41.671 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-05-06T12:02:41.710 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-05-06T12:02:41.762 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-05-06T12:02:41.842 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-05-06T12:02:41.858 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-2:17.2.7-925.ge81df420.el9.x86_64 33/33 2024-05-06T12:02:41.858 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-2:17.2.7-925.ge81df420.el9.x86_64 1/33 2024-05-06T12:02:41.858 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mds-2:17.2.7-925.ge81df420.el9.x86_64 2/33 2024-05-06T12:02:41.858 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-2:17.2.7-925.ge81df420.el9.x86_64 3/33 2024-05-06T12:02:41.858 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mon-2:17.2.7-925.ge81df420.el9.x86_64 4/33 2024-05-06T12:02:41.858 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-osd-2:17.2.7-925.ge81df420.el9.x86_64 5/33 2024-05-06T12:02:41.858 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-925.ge81df420.el9.n 6/33 2024-05-06T12:02:41.859 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-05-06T12:02:41.859 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-05-06T12:02:41.859 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-05-06T12:02:41.859 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-05-06T12:02:41.859 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-05-06T12:02:41.859 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-05-06T12:02:41.861 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-05-06T12:02:41.861 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-05-06T12:02:41.861 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-05-06T12:02:41.861 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-05-06T12:02:41.861 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-05-06T12:02:41.861 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-05-06T12:02:41.862 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-05-06T12:02:41.862 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-05-06T12:02:41.862 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-05-06T12:02:41.862 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-05-06T12:02:41.862 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-05-06T12:02:41.862 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-05-06T12:02:41.862 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-05-06T12:02:41.862 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-05-06T12:02:41.862 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-05-06T12:02:41.862 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-05-06T12:02:41.862 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-05-06T12:02:41.862 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-05-06T12:02:41.862 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-05-06T12:02:41.862 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-05-06T12:02:41.930 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-05-06T12:02:41.970 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-05-06T12:02:42.006 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-05-06T12:02:42.036 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-05-06T12:02:42.070 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-05-06T12:02:42.104 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-05-06T12:02:42.144 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-05-06T12:02:42.170 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-05-06T12:02:42.205 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-05-06T12:02:42.243 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-05-06T12:02:42.315 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-05-06T12:02:42.453 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-05-06T12:02:42.533 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-05-06T12:02:42.587 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-05-06T12:02:42.626 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-05-06T12:02:42.675 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-05-06T12:02:42.730 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-05-06T12:02:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-05-06T12:02:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:02:42.804 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-06T12:02:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core-2:17.2.7-925.ge81df420.el9.noarch 2024-05-06T12:02:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-06T12:02:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-06T12:02:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-06T12:02:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-06T12:02:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-06T12:02:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-06T12:02:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-06T12:02:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-06T12:02:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-06T12:02:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-06T12:02:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-06T12:02:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-06T12:02:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-06T12:02:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-06T12:02:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-06T12:02:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply-3.11-14.el9.noarch 2024-05-06T12:02:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-06T12:02:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-06T12:02:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-06T12:02:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-06T12:02:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-06T12:02:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-06T12:02:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-06T12:02:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-06T12:02:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-06T12:02:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-06T12:02:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-06T12:02:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:02:42.806 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-06T12:02:42.914 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-05-06T12:02:42.955 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-05-06T12:02:43.025 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-05-06T12:02:43.112 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-05-06T12:02:43.154 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-05-06T12:02:43.162 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-base 2024-05-06T12:02:43.268 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-05-06T12:02:43.416 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-925.ge81df420.el9.n 28/33 2024-05-06T12:02:43.449 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-2:17.2.7-925.ge81df420.el9.x86_64 29/33 2024-05-06T12:02:43.474 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-2:17.2.7-925.ge81df420.el9.x86_64 29/33 2024-05-06T12:02:43.474 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-06T12:02:43.474 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-06T12:02:43.474 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-06T12:02:43.475 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-06T12:02:43.475 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:02:43.682 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:51 ago on Mon 06 May 2024 12:01:52 PM UTC. 2024-05-06T12:02:43.762 INFO:teuthology.orchestra.run.smithi092.stdout:Package ceph-base-2:17.2.7-925.ge81df420.el9.x86_64 is already installed. 2024-05-06T12:02:43.802 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-06T12:02:43.803 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-05-06T12:02:43.803 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-06T12:02:43.874 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install cephadm 2024-05-06T12:02:44.379 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:52 ago on Mon 06 May 2024 12:01:52 PM UTC. 2024-05-06T12:02:44.498 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-06T12:02:44.499 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:02:44.499 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-05-06T12:02:44.499 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:02:44.499 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-06T12:02:44.500 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm noarch 2:17.2.7-925.ge81df420.el9 ceph-noarch 82 k 2024-05-06T12:02:44.500 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:02:44.500 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-06T12:02:44.500 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:02:44.500 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-05-06T12:02:44.500 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:02:44.500 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 82 k 2024-05-06T12:02:44.500 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 372 k 2024-05-06T12:02:44.500 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-06T12:02:44.783 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-osd-2:17.2.7-925.ge81df420.el9.x86_64 30/33 2024-05-06T12:02:44.809 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-osd-2:17.2.7-925.ge81df420.el9.x86_64 30/33 2024-05-06T12:02:44.809 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-06T12:02:44.809 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-06T12:02:44.809 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-06T12:02:44.810 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-06T12:02:44.810 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:02:44.817 INFO:teuthology.orchestra.run.smithi092.stdout:cephadm-17.2.7-925.ge81df420.el9.noarch.rpm 259 kB/s | 82 kB 00:00 2024-05-06T12:02:44.818 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-06T12:02:44.818 INFO:teuthology.orchestra.run.smithi092.stdout:Total 258 kB/s | 82 kB 00:00 2024-05-06T12:02:44.818 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-06T12:02:44.823 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-06T12:02:44.823 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-06T12:02:44.825 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-06T12:02:44.825 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-06T12:02:44.893 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-06T12:02:45.184 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mon-2:17.2.7-925.ge81df420.el9.x86_64 31/33 2024-05-06T12:02:45.207 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mon-2:17.2.7-925.ge81df420.el9.x86_64 31/33 2024-05-06T12:02:45.207 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-06T12:02:45.207 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-06T12:02:45.207 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-06T12:02:45.207 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-06T12:02:45.207 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:02:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephadm-2:17.2.7-925.ge81df420.el9.noarch 1/1 2024-05-06T12:02:45.410 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mds-2:17.2.7-925.ge81df420.el9.x86_64 32/33 2024-05-06T12:02:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephadm-2:17.2.7-925.ge81df420.el9.noarch 1/1 2024-05-06T12:02:45.432 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mds-2:17.2.7-925.ge81df420.el9.x86_64 32/33 2024-05-06T12:02:45.432 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-06T12:02:45.432 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-06T12:02:45.432 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-06T12:02:45.432 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-06T12:02:45.432 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:02:45.487 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-2:17.2.7-925.ge81df420.el9.x86_64 33/33 2024-05-06T12:02:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephadm-2:17.2.7-925.ge81df420.el9.noarch 1/1 2024-05-06T12:02:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephadm-2:17.2.7-925.ge81df420.el9.noarch 1/1 2024-05-06T12:02:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:02:45.926 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-06T12:02:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm-2:17.2.7-925.ge81df420.el9.noarch 2024-05-06T12:02:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:02:45.926 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-06T12:02:46.162 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-immutable-object-cache 2024-05-06T12:02:46.681 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:54 ago on Mon 06 May 2024 12:01:52 PM UTC. 2024-05-06T12:02:46.802 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-06T12:02:46.803 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:02:46.803 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repo Size 2024-05-06T12:02:46.803 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:02:46.803 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-06T12:02:46.803 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-925.ge81df420.el9 ceph 137 k 2024-05-06T12:02:46.803 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:02:46.803 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-06T12:02:46.803 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:02:46.803 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-05-06T12:02:46.803 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:02:46.803 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 137 k 2024-05-06T12:02:46.804 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 379 k 2024-05-06T12:02:46.804 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-06T12:02:46.940 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-2:17.2.7-925.ge81df420.el9.x86_64 33/33 2024-05-06T12:02:46.940 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-2:17.2.7-925.ge81df420.el9.x86_64 1/33 2024-05-06T12:02:46.941 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mds-2:17.2.7-925.ge81df420.el9.x86_64 2/33 2024-05-06T12:02:46.941 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-2:17.2.7-925.ge81df420.el9.x86_64 3/33 2024-05-06T12:02:46.941 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mon-2:17.2.7-925.ge81df420.el9.x86_64 4/33 2024-05-06T12:02:46.941 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-osd-2:17.2.7-925.ge81df420.el9.x86_64 5/33 2024-05-06T12:02:46.941 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-925.ge81df420.el9.n 6/33 2024-05-06T12:02:46.941 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-05-06T12:02:46.941 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-05-06T12:02:46.941 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-05-06T12:02:46.941 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-05-06T12:02:46.941 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-05-06T12:02:46.941 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-05-06T12:02:46.941 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-05-06T12:02:46.941 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-05-06T12:02:46.941 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-05-06T12:02:46.942 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-05-06T12:02:46.942 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-05-06T12:02:46.942 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-05-06T12:02:46.942 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-05-06T12:02:46.942 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-05-06T12:02:46.943 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-05-06T12:02:46.943 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-05-06T12:02:46.943 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-05-06T12:02:46.943 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-05-06T12:02:46.943 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-05-06T12:02:46.944 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-05-06T12:02:46.944 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-05-06T12:02:46.944 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-05-06T12:02:46.945 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-05-06T12:02:46.945 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-05-06T12:02:46.945 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-05-06T12:02:46.945 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-05-06T12:02:47.006 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-immutable-object-cache-17.2.7-925.ge81df42 679 kB/s | 137 kB 00:00 2024-05-06T12:02:47.007 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-06T12:02:47.007 INFO:teuthology.orchestra.run.smithi092.stdout:Total 672 kB/s | 137 kB 00:00 2024-05-06T12:02:47.008 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-06T12:02:47.015 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-06T12:02:47.015 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-06T12:02:47.077 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-06T12:02:47.078 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-06T12:02:47.227 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-06T12:02:47.299 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-925.ge81df420.e 1/1 2024-05-06T12:02:47.324 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-925.ge81df420.e 1/1 2024-05-06T12:02:47.324 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-06T12:02:47.324 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-06T12:02:47.324 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:02:47.735 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-05-06T12:02:47.735 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:02:47.735 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-05-06T12:02:47.735 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:47.735 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:47.735 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:47.735 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core-2:17.2.7-925.ge81df420.el9.noarch 2024-05-06T12:02:47.735 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:47.735 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:47.735 INFO:teuthology.orchestra.run.smithi190.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-06T12:02:47.736 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-06T12:02:47.736 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-06T12:02:47.736 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-06T12:02:47.736 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-06T12:02:47.736 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-06T12:02:47.736 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-06T12:02:47.736 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-06T12:02:47.736 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-06T12:02:47.736 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-06T12:02:47.736 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-06T12:02:47.736 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-06T12:02:47.736 INFO:teuthology.orchestra.run.smithi190.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-06T12:02:47.736 INFO:teuthology.orchestra.run.smithi190.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-06T12:02:47.737 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-06T12:02:47.737 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ply-3.11-14.el9.noarch 2024-05-06T12:02:47.737 INFO:teuthology.orchestra.run.smithi190.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-06T12:02:47.737 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-06T12:02:47.737 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-06T12:02:47.737 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-06T12:02:47.737 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-06T12:02:47.737 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-06T12:02:47.737 INFO:teuthology.orchestra.run.smithi190.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-06T12:02:47.737 INFO:teuthology.orchestra.run.smithi190.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-06T12:02:47.737 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-06T12:02:47.737 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-06T12:02:47.737 INFO:teuthology.orchestra.run.smithi190.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-06T12:02:47.737 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:02:47.738 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-05-06T12:02:48.096 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-base 2024-05-06T12:02:48.156 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-925.ge81df420.e 1/1 2024-05-06T12:02:48.156 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:02:48.156 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-06T12:02:48.156 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:48.156 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:02:48.156 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-06T12:02:48.359 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr 2024-05-06T12:02:48.618 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:00:53 ago on Mon 06 May 2024 12:01:55 PM UTC. 2024-05-06T12:02:48.699 INFO:teuthology.orchestra.run.smithi190.stdout:Package ceph-base-2:17.2.7-925.ge81df420.el9.x86_64 is already installed. 2024-05-06T12:02:48.740 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-05-06T12:02:48.741 INFO:teuthology.orchestra.run.smithi190.stdout:Nothing to do. 2024-05-06T12:02:48.741 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-05-06T12:02:48.813 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install cephadm 2024-05-06T12:02:48.871 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:56 ago on Mon 06 May 2024 12:01:52 PM UTC. 2024-05-06T12:02:48.952 INFO:teuthology.orchestra.run.smithi092.stdout:Package ceph-mgr-2:17.2.7-925.ge81df420.el9.x86_64 is already installed. 2024-05-06T12:02:48.994 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-06T12:02:48.995 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-05-06T12:02:48.995 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-06T12:02:49.053 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr-dashboard 2024-05-06T12:02:49.318 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:00:54 ago on Mon 06 May 2024 12:01:55 PM UTC. 2024-05-06T12:02:49.438 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-05-06T12:02:49.439 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:02:49.439 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2024-05-06T12:02:49.439 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:02:49.439 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-05-06T12:02:49.439 INFO:teuthology.orchestra.run.smithi190.stdout: cephadm noarch 2:17.2.7-925.ge81df420.el9 ceph-noarch 82 k 2024-05-06T12:02:49.439 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:02:49.439 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-05-06T12:02:49.439 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:02:49.440 INFO:teuthology.orchestra.run.smithi190.stdout:Install 1 Package 2024-05-06T12:02:49.440 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:02:49.440 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 82 k 2024-05-06T12:02:49.440 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 372 k 2024-05-06T12:02:49.440 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-05-06T12:02:49.564 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:57 ago on Mon 06 May 2024 12:01:52 PM UTC. 2024-05-06T12:02:49.683 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-06T12:02:49.684 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:02:49.684 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-05-06T12:02:49.684 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:02:49.684 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-06T12:02:49.685 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard noarch 2:17.2.7-925.ge81df420.el9 ceph-noarch 3.4 M 2024-05-06T12:02:49.685 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-05-06T12:02:49.685 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards noarch 2:17.2.7-925.ge81df420.el9 ceph-noarch 24 k 2024-05-06T12:02:49.685 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts noarch 2:17.2.7-925.ge81df420.el9 ceph-noarch 15 k 2024-05-06T12:02:49.685 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-05-06T12:02:49.685 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-06T12:02:49.685 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-06T12:02:49.685 INFO:teuthology.orchestra.run.smithi092.stdout:Installing weak dependencies: 2024-05-06T12:02:49.685 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-05-06T12:02:49.685 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:02:49.685 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-06T12:02:49.685 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:02:49.685 INFO:teuthology.orchestra.run.smithi092.stdout:Install 7 Packages 2024-05-06T12:02:49.685 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:02:49.686 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 3.7 M 2024-05-06T12:02:49.686 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 80 M 2024-05-06T12:02:49.686 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-06T12:02:49.759 INFO:teuthology.orchestra.run.smithi190.stdout:cephadm-17.2.7-925.ge81df420.el9.noarch.rpm 257 kB/s | 82 kB 00:00 2024-05-06T12:02:49.760 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-05-06T12:02:49.760 INFO:teuthology.orchestra.run.smithi190.stdout:Total 256 kB/s | 82 kB 00:00 2024-05-06T12:02:49.760 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-05-06T12:02:49.765 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-05-06T12:02:49.765 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-05-06T12:02:49.767 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-05-06T12:02:49.768 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-05-06T12:02:49.845 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-05-06T12:02:50.027 INFO:teuthology.orchestra.run.smithi092.stdout:(1/7): ceph-grafana-dashboards-17.2.7-925.ge81d 128 kB/s | 24 kB 00:00 2024-05-06T12:02:50.119 INFO:teuthology.orchestra.run.smithi092.stdout:(2/7): ceph-prometheus-alerts-17.2.7-925.ge81df 52 kB/s | 15 kB 00:00 2024-05-06T12:02:50.211 INFO:teuthology.orchestra.run.smithi092.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 49 kB/s | 9.0 kB 00:00 2024-05-06T12:02:50.273 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: cephadm-2:17.2.7-925.ge81df420.el9.noarch 1/1 2024-05-06T12:02:50.361 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephadm-2:17.2.7-925.ge81df420.el9.noarch 1/1 2024-05-06T12:02:50.377 INFO:teuthology.orchestra.run.smithi092.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 185 kB/s | 31 kB 00:00 2024-05-06T12:02:50.444 INFO:teuthology.orchestra.run.smithi092.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 126 kB/s | 41 kB 00:00 2024-05-06T12:02:50.664 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: cephadm-2:17.2.7-925.ge81df420.el9.noarch 1/1 2024-05-06T12:02:50.694 INFO:teuthology.orchestra.run.smithi092.stdout:(6/7): ceph-mgr-dashboard-17.2.7-925.ge81df420. 3.9 MB/s | 3.4 MB 00:00 2024-05-06T12:02:50.720 INFO:teuthology.orchestra.run.smithi092.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 551 kB/s | 188 kB 00:00 2024-05-06T12:02:50.722 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-06T12:02:50.722 INFO:teuthology.orchestra.run.smithi092.stdout:Total 3.5 MB/s | 3.7 MB 00:01 2024-05-06T12:02:50.722 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-06T12:02:50.731 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-06T12:02:50.731 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-06T12:02:50.792 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-06T12:02:50.792 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-06T12:02:50.868 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephadm-2:17.2.7-925.ge81df420.el9.noarch 1/1 2024-05-06T12:02:50.868 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:02:50.868 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-05-06T12:02:50.868 INFO:teuthology.orchestra.run.smithi190.stdout: cephadm-2:17.2.7-925.ge81df420.el9.noarch 2024-05-06T12:02:50.868 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:02:50.868 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-05-06T12:02:51.057 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-immutable-object-cache 2024-05-06T12:02:51.083 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-06T12:02:51.146 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-05-06T12:02:51.180 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-05-06T12:02:51.226 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-05-06T12:02:51.258 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-05-06T12:02:51.287 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-925.ge81df420.el9.no 5/7 2024-05-06T12:02:51.566 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:00:56 ago on Mon 06 May 2024 12:01:55 PM UTC. 2024-05-06T12:02:51.686 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-05-06T12:02:51.686 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:02:51.687 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repo Size 2024-05-06T12:02:51.687 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:02:51.687 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-05-06T12:02:51.687 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-925.ge81df420.el9 ceph 137 k 2024-05-06T12:02:51.687 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:02:51.687 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-05-06T12:02:51.687 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:02:51.687 INFO:teuthology.orchestra.run.smithi190.stdout:Install 1 Package 2024-05-06T12:02:51.687 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:02:51.687 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 137 k 2024-05-06T12:02:51.687 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 379 k 2024-05-06T12:02:51.687 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-05-06T12:02:51.924 INFO:teuthology.orchestra.run.smithi190.stdout:ceph-immutable-object-cache-17.2.7-925.ge81df42 582 kB/s | 137 kB 00:00 2024-05-06T12:02:51.924 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-05-06T12:02:51.925 INFO:teuthology.orchestra.run.smithi190.stdout:Total 578 kB/s | 137 kB 00:00 2024-05-06T12:02:51.925 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-05-06T12:02:51.932 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-05-06T12:02:51.932 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-05-06T12:02:51.994 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-05-06T12:02:51.994 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-05-06T12:02:52.145 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-05-06T12:02:52.250 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-925.ge81df420.e 1/1 2024-05-06T12:02:52.274 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-925.ge81df420.e 1/1 2024-05-06T12:02:52.275 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-06T12:02:52.275 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-06T12:02:52.275 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:02:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-925.ge81df420.el9.n 6/7 2024-05-06T12:02:52.326 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-925.ge81df420.el9.noarch 7/7 2024-05-06T12:02:53.152 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-925.ge81df420.e 1/1 2024-05-06T12:02:53.152 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:02:53.153 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-05-06T12:02:53.153 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-immutable-object-cache-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:02:53.153 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:02:53.153 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-05-06T12:02:53.393 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-mgr 2024-05-06T12:02:53.473 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-925.ge81df420.el9.noarch 7/7 2024-05-06T12:02:53.473 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-925.ge81df420.el9.n 1/7 2024-05-06T12:02:53.473 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-925.ge81df420.el9.noarch 2/7 2024-05-06T12:02:53.473 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-925.ge81df420.el9.no 3/7 2024-05-06T12:02:53.473 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-05-06T12:02:53.473 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-05-06T12:02:53.473 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-05-06T12:02:53.739 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-05-06T12:02:53.739 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:02:53.740 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-06T12:02:53.740 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards-2:17.2.7-925.ge81df420.el9.noarch 2024-05-06T12:02:53.740 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard-2:17.2.7-925.ge81df420.el9.noarch 2024-05-06T12:02:53.740 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts-2:17.2.7-925.ge81df420.el9.noarch 2024-05-06T12:02:53.740 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-05-06T12:02:53.740 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-05-06T12:02:53.740 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-06T12:02:53.740 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-06T12:02:53.740 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:02:53.740 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-06T12:02:53.906 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:00:58 ago on Mon 06 May 2024 12:01:55 PM UTC. 2024-05-06T12:02:53.924 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-06T12:02:53.986 INFO:teuthology.orchestra.run.smithi190.stdout:Package ceph-mgr-2:17.2.7-925.ge81df420.el9.x86_64 is already installed. 2024-05-06T12:02:54.028 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-05-06T12:02:54.029 INFO:teuthology.orchestra.run.smithi190.stdout:Nothing to do. 2024-05-06T12:02:54.029 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-05-06T12:02:54.095 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-mgr-dashboard 2024-05-06T12:02:54.426 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:02 ago on Mon 06 May 2024 12:01:52 PM UTC. 2024-05-06T12:02:54.547 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-06T12:02:54.548 INFO:teuthology.orchestra.run.smithi092.stdout:====================================================================================== 2024-05-06T12:02:54.548 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-05-06T12:02:54.548 INFO:teuthology.orchestra.run.smithi092.stdout:====================================================================================== 2024-05-06T12:02:54.548 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-06T12:02:54.548 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-925.ge81df420.el9 ceph-noarch 7.4 M 2024-05-06T12:02:54.548 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-05-06T12:02:54.548 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-06T12:02:54.548 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-06T12:02:54.548 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-06T12:02:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-06T12:02:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-06T12:02:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-06T12:02:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-06T12:02:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-06T12:02:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-06T12:02:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-06T12:02:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-06T12:02:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:02:54.549 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-06T12:02:54.549 INFO:teuthology.orchestra.run.smithi092.stdout:====================================================================================== 2024-05-06T12:02:54.549 INFO:teuthology.orchestra.run.smithi092.stdout:Install 12 Packages 2024-05-06T12:02:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:02:54.550 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 38 M 2024-05-06T12:02:54.550 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 209 M 2024-05-06T12:02:54.550 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-06T12:02:54.599 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:00:59 ago on Mon 06 May 2024 12:01:55 PM UTC. 2024-05-06T12:02:54.719 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-05-06T12:02:54.720 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:02:54.720 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2024-05-06T12:02:54.720 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:02:54.720 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-05-06T12:02:54.720 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard noarch 2:17.2.7-925.ge81df420.el9 ceph-noarch 3.4 M 2024-05-06T12:02:54.720 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-05-06T12:02:54.720 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards noarch 2:17.2.7-925.ge81df420.el9 ceph-noarch 24 k 2024-05-06T12:02:54.720 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-prometheus-alerts noarch 2:17.2.7-925.ge81df420.el9 ceph-noarch 15 k 2024-05-06T12:02:54.720 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-05-06T12:02:54.720 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-06T12:02:54.721 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-06T12:02:54.721 INFO:teuthology.orchestra.run.smithi190.stdout:Installing weak dependencies: 2024-05-06T12:02:54.721 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-05-06T12:02:54.721 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:02:54.721 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-05-06T12:02:54.721 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:02:54.721 INFO:teuthology.orchestra.run.smithi190.stdout:Install 7 Packages 2024-05-06T12:02:54.721 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:02:54.721 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 3.7 M 2024-05-06T12:02:54.722 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 80 M 2024-05-06T12:02:54.722 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-05-06T12:02:55.079 INFO:teuthology.orchestra.run.smithi092.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 195 kB 00:00 2024-05-06T12:02:55.093 INFO:teuthology.orchestra.run.smithi190.stdout:(1/7): ceph-prometheus-alerts-17.2.7-925.ge81df 71 kB/s | 15 kB 00:00 2024-05-06T12:02:55.110 INFO:teuthology.orchestra.run.smithi190.stdout:(2/7): ceph-grafana-dashboards-17.2.7-925.ge81d 110 kB/s | 24 kB 00:00 2024-05-06T12:02:55.171 INFO:teuthology.orchestra.run.smithi092.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 3.4 MB/s | 803 kB 00:00 2024-05-06T12:02:55.196 INFO:teuthology.orchestra.run.smithi092.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 280 kB/s | 33 kB 00:00 2024-05-06T12:02:55.238 INFO:teuthology.orchestra.run.smithi092.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 435 kB/s | 18 kB 00:00 2024-05-06T12:02:55.280 INFO:teuthology.orchestra.run.smithi092.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 847 kB/s | 35 kB 00:00 2024-05-06T12:02:55.411 INFO:teuthology.orchestra.run.smithi190.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 29 kB/s | 9.0 kB 00:00 2024-05-06T12:02:55.456 INFO:teuthology.orchestra.run.smithi092.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 11 MB/s | 3.0 MB 00:00 2024-05-06T12:02:55.506 INFO:teuthology.orchestra.run.smithi092.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-05-06T12:02:55.586 INFO:teuthology.orchestra.run.smithi190.stdout:(4/7): python3-jwt-2.4.0-1.el9.noarch.rpm 86 kB/s | 41 kB 00:00 2024-05-06T12:02:55.611 INFO:teuthology.orchestra.run.smithi190.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 154 kB/s | 31 kB 00:00 2024-05-06T12:02:55.681 INFO:teuthology.orchestra.run.smithi092.stdout:(8/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 5.1 MB 00:00 2024-05-06T12:02:55.773 INFO:teuthology.orchestra.run.smithi092.stdout:(9/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 9.6 MB/s | 4.7 MB 00:00 2024-05-06T12:02:55.798 INFO:teuthology.orchestra.run.smithi092.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.1 MB/s | 250 kB 00:00 2024-05-06T12:02:55.903 INFO:teuthology.orchestra.run.smithi190.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 595 kB/s | 188 kB 00:00 2024-05-06T12:02:55.982 INFO:teuthology.orchestra.run.smithi092.stdout:(11/12): ceph-mgr-diskprediction-local-17.2.7-9 7.1 MB/s | 7.4 MB 00:01 2024-05-06T12:02:56.003 INFO:teuthology.orchestra.run.smithi190.stdout:(7/7): ceph-mgr-dashboard-17.2.7-925.ge81df420. 3.0 MB/s | 3.4 MB 00:01 2024-05-06T12:02:56.006 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-05-06T12:02:56.007 INFO:teuthology.orchestra.run.smithi190.stdout:Total 2.8 MB/s | 3.7 MB 00:01 2024-05-06T12:02:56.007 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-05-06T12:02:56.015 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-05-06T12:02:56.015 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-05-06T12:02:56.075 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-05-06T12:02:56.076 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-05-06T12:02:56.369 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-05-06T12:02:56.413 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-05-06T12:02:56.415 INFO:teuthology.orchestra.run.smithi092.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-05-06T12:02:56.416 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-06T12:02:56.417 INFO:teuthology.orchestra.run.smithi092.stdout:Total 20 MB/s | 38 MB 00:01 2024-05-06T12:02:56.451 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-05-06T12:02:56.502 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-05-06T12:02:56.533 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-05-06T12:02:56.562 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-925.ge81df420.el9.no 5/7 2024-05-06T12:02:56.766 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-06T12:02:56.792 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-06T12:02:56.792 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-06T12:02:57.194 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-06T12:02:57.195 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-06T12:02:57.559 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-925.ge81df420.el9.n 6/7 2024-05-06T12:02:57.591 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-925.ge81df420.el9.noarch 7/7 2024-05-06T12:02:57.943 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-06T12:02:58.003 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-05-06T12:02:58.047 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-05-06T12:02:58.148 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-05-06T12:02:58.189 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-05-06T12:02:58.367 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-05-06T12:02:58.397 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-05-06T12:02:58.473 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-05-06T12:02:58.624 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-925.ge81df420.el9.noarch 7/7 2024-05-06T12:02:58.624 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-925.ge81df420.el9.n 1/7 2024-05-06T12:02:58.624 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-925.ge81df420.el9.noarch 2/7 2024-05-06T12:02:58.624 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-925.ge81df420.el9.no 3/7 2024-05-06T12:02:58.625 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-05-06T12:02:58.625 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-05-06T12:02:58.625 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-05-06T12:02:58.878 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-05-06T12:02:58.878 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:02:58.878 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-05-06T12:02:58.878 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards-2:17.2.7-925.ge81df420.el9.noarch 2024-05-06T12:02:58.878 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard-2:17.2.7-925.ge81df420.el9.noarch 2024-05-06T12:02:58.878 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-prometheus-alerts-2:17.2.7-925.ge81df420.el9.noarch 2024-05-06T12:02:58.878 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-05-06T12:02:58.878 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-05-06T12:02:58.878 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-06T12:02:58.879 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-06T12:02:58.879 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:02:58.879 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-05-06T12:02:59.098 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-06T12:02:59.107 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-05-06T12:02:59.219 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-05-06T12:02:59.599 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:01:04 ago on Mon 06 May 2024 12:01:55 PM UTC. 2024-05-06T12:02:59.721 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-05-06T12:02:59.722 INFO:teuthology.orchestra.run.smithi190.stdout:====================================================================================== 2024-05-06T12:02:59.722 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2024-05-06T12:02:59.722 INFO:teuthology.orchestra.run.smithi190.stdout:====================================================================================== 2024-05-06T12:02:59.722 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-05-06T12:02:59.722 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-925.ge81df420.el9 ceph-noarch 7.4 M 2024-05-06T12:02:59.722 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-05-06T12:02:59.723 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-06T12:02:59.723 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-06T12:02:59.723 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-06T12:02:59.723 INFO:teuthology.orchestra.run.smithi190.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-06T12:02:59.723 INFO:teuthology.orchestra.run.smithi190.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-06T12:02:59.723 INFO:teuthology.orchestra.run.smithi190.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-06T12:02:59.723 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-06T12:02:59.723 INFO:teuthology.orchestra.run.smithi190.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-06T12:02:59.723 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-06T12:02:59.723 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-06T12:02:59.723 INFO:teuthology.orchestra.run.smithi190.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-06T12:02:59.723 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:02:59.723 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-05-06T12:02:59.723 INFO:teuthology.orchestra.run.smithi190.stdout:====================================================================================== 2024-05-06T12:02:59.724 INFO:teuthology.orchestra.run.smithi190.stdout:Install 12 Packages 2024-05-06T12:02:59.724 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:02:59.724 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 38 M 2024-05-06T12:02:59.724 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 209 M 2024-05-06T12:02:59.724 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-05-06T12:03:00.394 INFO:teuthology.orchestra.run.smithi190.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 195 kB 00:00 2024-05-06T12:03:00.423 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-05-06T12:03:00.444 INFO:teuthology.orchestra.run.smithi190.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 4.1 MB/s | 803 kB 00:00 2024-05-06T12:03:00.478 INFO:teuthology.orchestra.run.smithi190.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 392 kB/s | 33 kB 00:00 2024-05-06T12:03:00.520 INFO:teuthology.orchestra.run.smithi190.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 436 kB/s | 18 kB 00:00 2024-05-06T12:03:00.562 INFO:teuthology.orchestra.run.smithi190.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 847 kB/s | 35 kB 00:00 2024-05-06T12:03:00.770 INFO:teuthology.orchestra.run.smithi190.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 9.2 MB/s | 3.0 MB 00:00 2024-05-06T12:03:00.821 INFO:teuthology.orchestra.run.smithi190.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-05-06T12:03:00.904 INFO:teuthology.orchestra.run.smithi190.stdout:(8/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 14 MB/s | 4.7 MB 00:00 2024-05-06T12:03:00.954 INFO:teuthology.orchestra.run.smithi190.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 4.9 MB/s | 250 kB 00:00 2024-05-06T12:03:01.080 INFO:teuthology.orchestra.run.smithi190.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 5.1 MB 00:00 2024-05-06T12:03:01.441 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-05-06T12:03:01.446 INFO:teuthology.orchestra.run.smithi190.stdout:(11/12): ceph-mgr-diskprediction-local-17.2.7-9 6.2 MB/s | 7.4 MB 00:01 2024-05-06T12:03:01.472 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-925.ge81df4 12/12 2024-05-06T12:03:01.605 INFO:teuthology.orchestra.run.smithi190.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-05-06T12:03:01.606 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-05-06T12:03:01.606 INFO:teuthology.orchestra.run.smithi190.stdout:Total 20 MB/s | 38 MB 00:01 2024-05-06T12:03:01.951 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-05-06T12:03:01.978 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-05-06T12:03:01.978 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-05-06T12:03:02.382 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-05-06T12:03:02.383 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-05-06T12:03:03.173 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-05-06T12:03:03.237 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-05-06T12:03:03.280 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-05-06T12:03:03.379 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-05-06T12:03:03.415 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-05-06T12:03:03.592 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-05-06T12:03:03.630 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-05-06T12:03:03.706 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-05-06T12:03:04.042 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-925.ge81df4 12/12 2024-05-06T12:03:04.042 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-925.ge81df4 1/12 2024-05-06T12:03:04.042 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-05-06T12:03:04.042 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-05-06T12:03:04.042 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-05-06T12:03:04.043 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-05-06T12:03:04.043 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-05-06T12:03:04.043 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-05-06T12:03:04.043 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-05-06T12:03:04.043 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-05-06T12:03:04.043 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-05-06T12:03:04.043 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-05-06T12:03:04.344 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-05-06T12:03:04.431 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-05-06T12:03:04.432 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:04.432 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-06T12:03:04.432 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local-2:17.2.7-925.ge81df420.el9.noarch 2024-05-06T12:03:04.432 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-06T12:03:04.432 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-06T12:03:04.432 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-06T12:03:04.432 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-06T12:03:04.432 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-06T12:03:04.432 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-06T12:03:04.432 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-06T12:03:04.432 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-06T12:03:04.432 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-06T12:03:04.432 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-06T12:03:04.432 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-06T12:03:04.433 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:04.433 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-06T12:03:04.451 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-05-06T12:03:04.810 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr-rook 2024-05-06T12:03:05.324 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:13 ago on Mon 06 May 2024 12:01:52 PM UTC. 2024-05-06T12:03:05.444 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-06T12:03:05.445 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:03:05.446 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-05-06T12:03:05.446 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:03:05.446 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-06T12:03:05.446 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook noarch 2:17.2.7-925.ge81df420.el9 ceph-noarch 47 k 2024-05-06T12:03:05.446 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-05-06T12:03:05.446 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-06T12:03:05.446 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-06T12:03:05.446 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-06T12:03:05.446 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-06T12:03:05.446 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-06T12:03:05.446 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-06T12:03:05.446 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-06T12:03:05.447 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-06T12:03:05.447 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-06T12:03:05.447 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-06T12:03:05.447 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-06T12:03:05.447 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-06T12:03:05.447 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:05.447 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-06T12:03:05.447 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:03:05.447 INFO:teuthology.orchestra.run.smithi092.stdout:Install 13 Packages 2024-05-06T12:03:05.447 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:05.448 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 2.2 M 2024-05-06T12:03:05.448 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 26 M 2024-05-06T12:03:05.448 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-06T12:03:05.656 INFO:teuthology.orchestra.run.smithi092.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 256 kB/s | 26 kB 00:00 2024-05-06T12:03:05.673 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-05-06T12:03:05.681 INFO:teuthology.orchestra.run.smithi092.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 153 kB/s | 19 kB 00:00 2024-05-06T12:03:05.748 INFO:teuthology.orchestra.run.smithi092.stdout:(3/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.4 MB/s | 159 kB 00:00 2024-05-06T12:03:05.799 INFO:teuthology.orchestra.run.smithi092.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.9 MB/s | 222 kB 00:00 2024-05-06T12:03:05.849 INFO:teuthology.orchestra.run.smithi092.stdout:(5/13): ceph-mgr-rook-17.2.7-925.ge81df420.el9. 162 kB/s | 47 kB 00:00 2024-05-06T12:03:05.874 INFO:teuthology.orchestra.run.smithi092.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 712 kB/s | 54 kB 00:00 2024-05-06T12:03:05.900 INFO:teuthology.orchestra.run.smithi092.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.8 MB/s | 279 kB 00:00 2024-05-06T12:03:06.050 INFO:teuthology.orchestra.run.smithi092.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 85 kB/s | 15 kB 00:00 2024-05-06T12:03:06.075 INFO:teuthology.orchestra.run.smithi092.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 142 kB/s | 32 kB 00:00 2024-05-06T12:03:06.217 INFO:teuthology.orchestra.run.smithi092.stdout:(10/13): python3-google-auth-2.29.0-1.el9.noarc 648 kB/s | 206 kB 00:00 2024-05-06T12:03:06.492 INFO:teuthology.orchestra.run.smithi092.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 326 kB/s | 90 kB 00:00 2024-05-06T12:03:06.517 INFO:teuthology.orchestra.run.smithi092.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 134 kB/s | 59 kB 00:00 2024-05-06T12:03:06.617 INFO:teuthology.orchestra.run.smithi092.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.8 MB/s | 1.0 MB 00:00 2024-05-06T12:03:06.618 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-06T12:03:06.618 INFO:teuthology.orchestra.run.smithi092.stdout:Total 1.9 MB/s | 2.2 MB 00:01 2024-05-06T12:03:06.682 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-06T12:03:06.697 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-06T12:03:06.697 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-06T12:03:06.705 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-05-06T12:03:06.737 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-925.ge81df4 12/12 2024-05-06T12:03:06.779 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-06T12:03:06.779 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-06T12:03:07.345 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-06T12:03:07.512 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-05-06T12:03:07.568 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-05-06T12:03:07.623 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-05-06T12:03:07.653 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-05-06T12:03:07.694 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-05-06T12:03:07.787 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-05-06T12:03:07.896 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-05-06T12:03:07.946 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-05-06T12:03:08.656 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-05-06T12:03:08.710 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-05-06T12:03:08.745 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-05-06T12:03:08.789 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-05-06T12:03:08.814 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-rook-2:17.2.7-925.ge81df420.el9.noarch 13/13 2024-05-06T12:03:09.270 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-925.ge81df4 12/12 2024-05-06T12:03:09.270 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-925.ge81df4 1/12 2024-05-06T12:03:09.271 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-05-06T12:03:09.271 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-05-06T12:03:09.271 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-05-06T12:03:09.271 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-05-06T12:03:09.271 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-05-06T12:03:09.271 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-05-06T12:03:09.271 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-05-06T12:03:09.271 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-05-06T12:03:09.271 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-05-06T12:03:09.271 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-05-06T12:03:09.529 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-925.ge81df420.el9.noarch 13/13 2024-05-06T12:03:09.529 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-rook-2:17.2.7-925.ge81df420.el9.noarch 1/13 2024-05-06T12:03:09.529 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-05-06T12:03:09.529 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-05-06T12:03:09.529 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-05-06T12:03:09.529 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-05-06T12:03:09.529 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-05-06T12:03:09.529 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-05-06T12:03:09.529 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-05-06T12:03:09.529 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-05-06T12:03:09.530 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-05-06T12:03:09.530 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-05-06T12:03:09.530 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-05-06T12:03:09.669 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-05-06T12:03:09.669 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:09.669 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-05-06T12:03:09.669 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local-2:17.2.7-925.ge81df420.el9.noarch 2024-05-06T12:03:09.669 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-06T12:03:09.670 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-06T12:03:09.670 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-06T12:03:09.670 INFO:teuthology.orchestra.run.smithi190.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-06T12:03:09.670 INFO:teuthology.orchestra.run.smithi190.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-06T12:03:09.670 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-06T12:03:09.670 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-06T12:03:09.670 INFO:teuthology.orchestra.run.smithi190.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-06T12:03:09.670 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-06T12:03:09.670 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-06T12:03:09.670 INFO:teuthology.orchestra.run.smithi190.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-06T12:03:09.670 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:09.670 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-05-06T12:03:09.836 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-05-06T12:03:09.836 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:09.837 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-06T12:03:09.837 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook-2:17.2.7-925.ge81df420.el9.noarch 2024-05-06T12:03:09.837 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-06T12:03:09.837 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-06T12:03:09.837 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-06T12:03:09.837 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-06T12:03:09.837 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-06T12:03:09.837 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-06T12:03:09.837 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-06T12:03:09.837 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-06T12:03:09.837 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-06T12:03:09.837 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-06T12:03:09.838 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-06T12:03:09.838 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-06T12:03:09.838 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:09.838 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-06T12:03:10.083 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-mgr-rook 2024-05-06T12:03:10.117 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr-cephadm 2024-05-06T12:03:10.608 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:01:15 ago on Mon 06 May 2024 12:01:55 PM UTC. 2024-05-06T12:03:10.645 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:18 ago on Mon 06 May 2024 12:01:52 PM UTC. 2024-05-06T12:03:10.731 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-05-06T12:03:10.732 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:03:10.732 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2024-05-06T12:03:10.732 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:03:10.732 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-05-06T12:03:10.732 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook noarch 2:17.2.7-925.ge81df420.el9 ceph-noarch 47 k 2024-05-06T12:03:10.733 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-05-06T12:03:10.733 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-06T12:03:10.733 INFO:teuthology.orchestra.run.smithi190.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-06T12:03:10.733 INFO:teuthology.orchestra.run.smithi190.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-06T12:03:10.733 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-06T12:03:10.733 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-06T12:03:10.733 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-06T12:03:10.733 INFO:teuthology.orchestra.run.smithi190.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-06T12:03:10.733 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-06T12:03:10.733 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-06T12:03:10.733 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-06T12:03:10.733 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-06T12:03:10.733 INFO:teuthology.orchestra.run.smithi190.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-06T12:03:10.733 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:10.734 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-05-06T12:03:10.734 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:03:10.734 INFO:teuthology.orchestra.run.smithi190.stdout:Install 13 Packages 2024-05-06T12:03:10.734 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:10.734 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 2.2 M 2024-05-06T12:03:10.734 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 26 M 2024-05-06T12:03:10.735 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-05-06T12:03:10.763 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-06T12:03:10.764 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:03:10.764 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-05-06T12:03:10.764 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:03:10.765 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-06T12:03:10.765 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm noarch 2:17.2.7-925.ge81df420.el9 ceph-noarch 123 k 2024-05-06T12:03:10.765 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-05-06T12:03:10.765 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-06T12:03:10.765 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-06T12:03:10.765 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-06T12:03:10.765 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-06T12:03:10.765 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-06T12:03:10.765 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:10.765 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-06T12:03:10.765 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:03:10.765 INFO:teuthology.orchestra.run.smithi092.stdout:Install 6 Packages 2024-05-06T12:03:10.765 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:10.766 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 7.0 M 2024-05-06T12:03:10.766 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 33 M 2024-05-06T12:03:10.766 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-06T12:03:11.054 INFO:teuthology.orchestra.run.smithi190.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 244 kB/s | 26 kB 00:00 2024-05-06T12:03:11.079 INFO:teuthology.orchestra.run.smithi190.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 147 kB/s | 19 kB 00:00 2024-05-06T12:03:11.146 INFO:teuthology.orchestra.run.smithi190.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.4 MB/s | 222 kB 00:00 2024-05-06T12:03:11.153 INFO:teuthology.orchestra.run.smithi092.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.3 MB/s | 248 kB 00:00 2024-05-06T12:03:11.178 INFO:teuthology.orchestra.run.smithi092.stdout:(2/6): ceph-mgr-cephadm-17.2.7-925.ge81df420.el 592 kB/s | 123 kB 00:00 2024-05-06T12:03:11.188 INFO:teuthology.orchestra.run.smithi190.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-05-06T12:03:11.205 INFO:teuthology.orchestra.run.smithi190.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.6 MB/s | 279 kB 00:00 2024-05-06T12:03:11.230 INFO:teuthology.orchestra.run.smithi190.stdout:(6/13): ceph-mgr-rook-17.2.7-925.ge81df420.el9. 169 kB/s | 47 kB 00:00 2024-05-06T12:03:11.264 INFO:teuthology.orchestra.run.smithi190.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 708 kB/s | 54 kB 00:00 2024-05-06T12:03:11.315 INFO:teuthology.orchestra.run.smithi190.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 179 kB/s | 15 kB 00:00 2024-05-06T12:03:11.339 INFO:teuthology.orchestra.run.smithi190.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 240 kB/s | 32 kB 00:00 2024-05-06T12:03:11.379 INFO:teuthology.orchestra.run.smithi092.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-05-06T12:03:11.389 INFO:teuthology.orchestra.run.smithi190.stdout:(10/13): python3-google-auth-2.29.0-1.el9.noarc 1.6 MB/s | 206 kB 00:00 2024-05-06T12:03:11.404 INFO:teuthology.orchestra.run.smithi092.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 256 kB/s | 58 kB 00:00 2024-05-06T12:03:11.415 INFO:teuthology.orchestra.run.smithi190.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 786 kB/s | 59 kB 00:00 2024-05-06T12:03:11.465 INFO:teuthology.orchestra.run.smithi190.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-05-06T12:03:11.490 INFO:teuthology.orchestra.run.smithi190.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 5.8 MB/s | 1.0 MB 00:00 2024-05-06T12:03:11.491 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-05-06T12:03:11.492 INFO:teuthology.orchestra.run.smithi190.stdout:Total 2.9 MB/s | 2.2 MB 00:00 2024-05-06T12:03:11.538 INFO:teuthology.orchestra.run.smithi092.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.4 MB/s | 548 kB 00:00 2024-05-06T12:03:11.552 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-05-06T12:03:11.563 INFO:teuthology.orchestra.run.smithi092.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 278 kB/s | 51 kB 00:00 2024-05-06T12:03:11.565 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-06T12:03:11.565 INFO:teuthology.orchestra.run.smithi092.stdout:Total 8.7 MB/s | 7.0 MB 00:00 2024-05-06T12:03:11.566 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-05-06T12:03:11.566 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-05-06T12:03:11.635 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-06T12:03:11.646 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-06T12:03:11.646 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-06T12:03:11.648 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-05-06T12:03:11.648 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-05-06T12:03:11.741 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-06T12:03:11.742 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-06T12:03:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-06T12:03:12.229 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-05-06T12:03:12.234 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-05-06T12:03:12.289 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-05-06T12:03:12.404 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-05-06T12:03:12.459 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-05-06T12:03:12.519 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-05-06T12:03:12.551 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-05-06T12:03:12.609 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-05-06T12:03:12.685 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-05-06T12:03:12.804 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-05-06T12:03:12.810 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-05-06T12:03:12.854 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-05-06T12:03:12.894 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-05-06T12:03:12.949 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-05-06T12:03:12.977 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-925.ge81df420.el9.noarch 6/6 2024-05-06T12:03:13.563 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-05-06T12:03:13.635 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-05-06T12:03:13.666 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-05-06T12:03:13.712 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-05-06T12:03:13.733 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-925.ge81df420.el9.noarch 6/6 2024-05-06T12:03:13.733 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-925.ge81df420.el9.noarch 1/6 2024-05-06T12:03:13.733 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-05-06T12:03:13.733 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-05-06T12:03:13.734 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-05-06T12:03:13.734 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-05-06T12:03:13.736 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-rook-2:17.2.7-925.ge81df420.el9.noarch 13/13 2024-05-06T12:03:13.984 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-05-06T12:03:13.984 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:13.985 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-06T12:03:13.985 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm-2:17.2.7-925.ge81df420.el9.noarch 2024-05-06T12:03:13.985 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-06T12:03:13.985 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-06T12:03:13.985 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-06T12:03:13.985 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-06T12:03:13.985 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-06T12:03:13.985 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:13.985 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-06T12:03:14.236 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-fuse 2024-05-06T12:03:14.448 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-925.ge81df420.el9.noarch 13/13 2024-05-06T12:03:14.448 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-rook-2:17.2.7-925.ge81df420.el9.noarch 1/13 2024-05-06T12:03:14.448 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-05-06T12:03:14.448 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-05-06T12:03:14.448 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-05-06T12:03:14.448 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-05-06T12:03:14.448 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-05-06T12:03:14.448 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-05-06T12:03:14.448 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-05-06T12:03:14.448 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-05-06T12:03:14.449 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-05-06T12:03:14.449 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-05-06T12:03:14.449 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-05-06T12:03:14.744 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:22 ago on Mon 06 May 2024 12:01:52 PM UTC. 2024-05-06T12:03:14.756 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-05-06T12:03:14.756 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:14.756 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-05-06T12:03:14.756 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook-2:17.2.7-925.ge81df420.el9.noarch 2024-05-06T12:03:14.757 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-06T12:03:14.757 INFO:teuthology.orchestra.run.smithi190.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-06T12:03:14.757 INFO:teuthology.orchestra.run.smithi190.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-06T12:03:14.757 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-06T12:03:14.757 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-06T12:03:14.757 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-06T12:03:14.757 INFO:teuthology.orchestra.run.smithi190.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-06T12:03:14.757 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-06T12:03:14.757 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-06T12:03:14.757 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-06T12:03:14.757 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-06T12:03:14.757 INFO:teuthology.orchestra.run.smithi190.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-06T12:03:14.757 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:14.757 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-05-06T12:03:14.861 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-06T12:03:14.862 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:03:14.862 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-05-06T12:03:14.862 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:03:14.862 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-06T12:03:14.862 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse x86_64 2:17.2.7-925.ge81df420.el9 ceph 813 k 2024-05-06T12:03:14.862 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-05-06T12:03:14.862 INFO:teuthology.orchestra.run.smithi092.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-06T12:03:14.862 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:14.862 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-06T12:03:14.862 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:03:14.862 INFO:teuthology.orchestra.run.smithi092.stdout:Install 2 Packages 2024-05-06T12:03:14.862 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:14.863 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 893 k 2024-05-06T12:03:14.863 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 2.6 M 2024-05-06T12:03:14.863 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-06T12:03:15.012 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-mgr-cephadm 2024-05-06T12:03:15.207 INFO:teuthology.orchestra.run.smithi092.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 403 kB/s | 80 kB 00:00 2024-05-06T12:03:15.340 INFO:teuthology.orchestra.run.smithi092.stdout:(2/2): ceph-fuse-17.2.7-925.ge81df420.el9.x86_6 2.4 MB/s | 813 kB 00:00 2024-05-06T12:03:15.341 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-06T12:03:15.341 INFO:teuthology.orchestra.run.smithi092.stdout:Total 1.8 MB/s | 893 kB 00:00 2024-05-06T12:03:15.352 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-06T12:03:15.373 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-06T12:03:15.373 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-06T12:03:15.449 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-06T12:03:15.449 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-06T12:03:15.528 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:01:20 ago on Mon 06 May 2024 12:01:55 PM UTC. 2024-05-06T12:03:15.636 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-06T12:03:15.648 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-05-06T12:03:15.648 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:03:15.648 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2024-05-06T12:03:15.649 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:03:15.649 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-05-06T12:03:15.649 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-cephadm noarch 2:17.2.7-925.ge81df420.el9 ceph-noarch 123 k 2024-05-06T12:03:15.649 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-05-06T12:03:15.649 INFO:teuthology.orchestra.run.smithi190.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-06T12:03:15.649 INFO:teuthology.orchestra.run.smithi190.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-06T12:03:15.649 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-06T12:03:15.649 INFO:teuthology.orchestra.run.smithi190.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-06T12:03:15.649 INFO:teuthology.orchestra.run.smithi190.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-06T12:03:15.649 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:15.649 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-05-06T12:03:15.649 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:03:15.649 INFO:teuthology.orchestra.run.smithi190.stdout:Install 6 Packages 2024-05-06T12:03:15.649 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:15.650 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 7.0 M 2024-05-06T12:03:15.650 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 33 M 2024-05-06T12:03:15.650 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-05-06T12:03:15.770 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-05-06T12:03:15.829 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-fuse-2:17.2.7-925.ge81df420.el9.x86_64 2/2 2024-05-06T12:03:16.067 INFO:teuthology.orchestra.run.smithi190.stdout:(1/6): ceph-mgr-cephadm-17.2.7-925.ge81df420.el 565 kB/s | 123 kB 00:00 2024-05-06T12:03:16.409 INFO:teuthology.orchestra.run.smithi190.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.6 MB/s | 548 kB 00:00 2024-05-06T12:03:16.434 INFO:teuthology.orchestra.run.smithi190.stdout:(3/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 423 kB/s | 248 kB 00:00 2024-05-06T12:03:16.450 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-fuse-2:17.2.7-925.ge81df420.el9.x86_64 2/2 2024-05-06T12:03:16.451 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-fuse-2:17.2.7-925.ge81df420.el9.x86_64 1/2 2024-05-06T12:03:16.476 INFO:teuthology.orchestra.run.smithi190.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 859 kB/s | 58 kB 00:00 2024-05-06T12:03:16.626 INFO:teuthology.orchestra.run.smithi190.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 266 kB/s | 51 kB 00:00 2024-05-06T12:03:16.702 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-05-06T12:03:16.702 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:16.702 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-06T12:03:16.702 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse-2:17.2.7-925.ge81df420.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-05-06T12:03:16.702 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:16.702 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-06T12:03:16.963 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install librados-devel 2024-05-06T12:03:17.485 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:25 ago on Mon 06 May 2024 12:01:52 PM UTC. 2024-05-06T12:03:17.611 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-06T12:03:17.612 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:03:17.612 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repo Size 2024-05-06T12:03:17.613 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:03:17.613 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-06T12:03:17.613 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel x86_64 2:17.2.7-925.ge81df420.el9 ceph 126 k 2024-05-06T12:03:17.613 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:17.613 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-06T12:03:17.613 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:03:17.613 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-05-06T12:03:17.613 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:17.613 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 126 k 2024-05-06T12:03:17.613 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 448 k 2024-05-06T12:03:17.613 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-06T12:03:17.929 INFO:teuthology.orchestra.run.smithi092.stdout:librados-devel-17.2.7-925.ge81df420.el9.x86_64. 400 kB/s | 126 kB 00:00 2024-05-06T12:03:17.929 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-06T12:03:17.929 INFO:teuthology.orchestra.run.smithi092.stdout:Total 399 kB/s | 126 kB 00:00 2024-05-06T12:03:17.930 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-06T12:03:17.935 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-06T12:03:17.935 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-06T12:03:17.996 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-06T12:03:17.996 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-06T12:03:18.145 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-06T12:03:18.249 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librados-devel-2:17.2.7-925.ge81df420.el9.x86_64 1/1 2024-05-06T12:03:18.614 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados-devel-2:17.2.7-925.ge81df420.el9.x86_64 1/1 2024-05-06T12:03:18.841 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados-devel-2:17.2.7-925.ge81df420.el9.x86_64 1/1 2024-05-06T12:03:18.841 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:18.841 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-06T12:03:18.841 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:03:18.841 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:18.841 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-06T12:03:19.080 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install libcephfs2 2024-05-06T12:03:19.535 INFO:teuthology.orchestra.run.smithi190.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 1.6 MB/s | 6.0 MB 00:03 2024-05-06T12:03:19.538 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-05-06T12:03:19.538 INFO:teuthology.orchestra.run.smithi190.stdout:Total 1.8 MB/s | 7.0 MB 00:03 2024-05-06T12:03:19.603 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:27 ago on Mon 06 May 2024 12:01:52 PM UTC. 2024-05-06T12:03:19.610 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-05-06T12:03:19.622 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-05-06T12:03:19.622 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-05-06T12:03:19.683 INFO:teuthology.orchestra.run.smithi092.stdout:Package libcephfs2-2:17.2.7-925.ge81df420.el9.x86_64 is already installed. 2024-05-06T12:03:19.719 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-05-06T12:03:19.719 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-05-06T12:03:19.724 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-06T12:03:19.725 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-05-06T12:03:19.725 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-06T12:03:19.806 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install libcephfs-devel 2024-05-06T12:03:20.021 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-05-06T12:03:20.191 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-05-06T12:03:20.236 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-05-06T12:03:20.326 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:28 ago on Mon 06 May 2024 12:01:52 PM UTC. 2024-05-06T12:03:20.445 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-06T12:03:20.445 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:03:20.445 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repo Size 2024-05-06T12:03:20.446 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:03:20.446 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-06T12:03:20.446 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel x86_64 2:17.2.7-925.ge81df420.el9 ceph 25 k 2024-05-06T12:03:20.446 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:20.446 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-06T12:03:20.446 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:03:20.446 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-05-06T12:03:20.446 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:20.447 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 25 k 2024-05-06T12:03:20.447 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 110 k 2024-05-06T12:03:20.447 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-06T12:03:20.700 INFO:teuthology.orchestra.run.smithi092.stdout:libcephfs-devel-17.2.7-925.ge81df420.el9.x86_64 99 kB/s | 25 kB 00:00 2024-05-06T12:03:20.700 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-06T12:03:20.700 INFO:teuthology.orchestra.run.smithi092.stdout:Total 98 kB/s | 25 kB 00:00 2024-05-06T12:03:20.701 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-06T12:03:20.704 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-06T12:03:20.704 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-06T12:03:20.706 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-06T12:03:20.706 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-06T12:03:20.760 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-05-06T12:03:20.792 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-06T12:03:20.855 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-05-06T12:03:20.894 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs-devel-2:17.2.7-925.ge81df420.el9.x86_64 1/1 2024-05-06T12:03:20.902 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-05-06T12:03:20.940 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-925.ge81df420.el9.noarch 6/6 2024-05-06T12:03:21.239 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-925.ge81df420.el9.x86_64 1/1 2024-05-06T12:03:21.449 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs-devel-2:17.2.7-925.ge81df420.el9.x86_64 1/1 2024-05-06T12:03:21.449 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:21.449 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-06T12:03:21.449 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:03:21.449 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:21.449 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-06T12:03:21.622 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install librados2 2024-05-06T12:03:21.644 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-925.ge81df420.el9.noarch 6/6 2024-05-06T12:03:21.645 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-925.ge81df420.el9.noarch 1/6 2024-05-06T12:03:21.645 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-05-06T12:03:21.645 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-05-06T12:03:21.645 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-05-06T12:03:21.645 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-05-06T12:03:21.880 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-05-06T12:03:21.880 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:21.880 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-05-06T12:03:21.880 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-cephadm-2:17.2.7-925.ge81df420.el9.noarch 2024-05-06T12:03:21.880 INFO:teuthology.orchestra.run.smithi190.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-06T12:03:21.881 INFO:teuthology.orchestra.run.smithi190.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-06T12:03:21.881 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-06T12:03:21.881 INFO:teuthology.orchestra.run.smithi190.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-06T12:03:21.881 INFO:teuthology.orchestra.run.smithi190.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-06T12:03:21.881 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:21.881 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-05-06T12:03:22.123 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-fuse 2024-05-06T12:03:22.129 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:30 ago on Mon 06 May 2024 12:01:52 PM UTC. 2024-05-06T12:03:22.207 INFO:teuthology.orchestra.run.smithi092.stdout:Package librados2-2:17.2.7-925.ge81df420.el9.x86_64 is already installed. 2024-05-06T12:03:22.248 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-06T12:03:22.249 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-05-06T12:03:22.249 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-06T12:03:22.327 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install librbd1 2024-05-06T12:03:22.629 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:01:27 ago on Mon 06 May 2024 12:01:55 PM UTC. 2024-05-06T12:03:22.747 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-05-06T12:03:22.748 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:03:22.748 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2024-05-06T12:03:22.748 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:03:22.749 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-05-06T12:03:22.749 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse x86_64 2:17.2.7-925.ge81df420.el9 ceph 813 k 2024-05-06T12:03:22.749 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-05-06T12:03:22.749 INFO:teuthology.orchestra.run.smithi190.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-06T12:03:22.749 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:22.749 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-05-06T12:03:22.749 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:03:22.749 INFO:teuthology.orchestra.run.smithi190.stdout:Install 2 Packages 2024-05-06T12:03:22.749 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:22.749 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 893 k 2024-05-06T12:03:22.749 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 2.6 M 2024-05-06T12:03:22.749 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-05-06T12:03:22.837 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:30 ago on Mon 06 May 2024 12:01:52 PM UTC. 2024-05-06T12:03:22.915 INFO:teuthology.orchestra.run.smithi092.stdout:Package librbd1-2:17.2.7-925.ge81df420.el9.x86_64 is already installed. 2024-05-06T12:03:22.956 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-06T12:03:22.957 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-05-06T12:03:22.957 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-06T12:03:23.028 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install python3-rados 2024-05-06T12:03:23.144 INFO:teuthology.orchestra.run.smithi190.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 607 kB/s | 80 kB 00:00 2024-05-06T12:03:23.311 INFO:teuthology.orchestra.run.smithi190.stdout:(2/2): ceph-fuse-17.2.7-925.ge81df420.el9.x86_6 2.7 MB/s | 813 kB 00:00 2024-05-06T12:03:23.311 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-05-06T12:03:23.311 INFO:teuthology.orchestra.run.smithi190.stdout:Total 1.6 MB/s | 893 kB 00:00 2024-05-06T12:03:23.322 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-05-06T12:03:23.344 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-05-06T12:03:23.344 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-05-06T12:03:23.420 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-05-06T12:03:23.420 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-05-06T12:03:23.541 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:31 ago on Mon 06 May 2024 12:01:52 PM UTC. 2024-05-06T12:03:23.612 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-05-06T12:03:23.620 INFO:teuthology.orchestra.run.smithi092.stdout:Package python3-rados-2:17.2.7-925.ge81df420.el9.x86_64 is already installed. 2024-05-06T12:03:23.660 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-06T12:03:23.661 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-05-06T12:03:23.661 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-06T12:03:23.728 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install python3-rgw 2024-05-06T12:03:23.747 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-05-06T12:03:23.816 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-fuse-2:17.2.7-925.ge81df420.el9.x86_64 2/2 2024-05-06T12:03:24.245 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:32 ago on Mon 06 May 2024 12:01:52 PM UTC. 2024-05-06T12:03:24.325 INFO:teuthology.orchestra.run.smithi092.stdout:Package python3-rgw-2:17.2.7-925.ge81df420.el9.x86_64 is already installed. 2024-05-06T12:03:24.365 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-06T12:03:24.366 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-05-06T12:03:24.366 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-06T12:03:24.410 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-fuse-2:17.2.7-925.ge81df420.el9.x86_64 2/2 2024-05-06T12:03:24.410 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-fuse-2:17.2.7-925.ge81df420.el9.x86_64 1/2 2024-05-06T12:03:24.439 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install python3-cephfs 2024-05-06T12:03:24.699 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-05-06T12:03:24.699 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:24.699 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-05-06T12:03:24.699 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse-2:17.2.7-925.ge81df420.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-05-06T12:03:24.699 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:24.699 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-05-06T12:03:24.939 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install librados-devel 2024-05-06T12:03:24.952 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:32 ago on Mon 06 May 2024 12:01:52 PM UTC. 2024-05-06T12:03:25.032 INFO:teuthology.orchestra.run.smithi092.stdout:Package python3-cephfs-2:17.2.7-925.ge81df420.el9.x86_64 is already installed. 2024-05-06T12:03:25.073 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-06T12:03:25.074 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-05-06T12:03:25.074 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-06T12:03:25.147 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install python3-rbd 2024-05-06T12:03:25.459 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:01:30 ago on Mon 06 May 2024 12:01:55 PM UTC. 2024-05-06T12:03:25.579 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-05-06T12:03:25.580 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:03:25.580 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repo Size 2024-05-06T12:03:25.580 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:03:25.581 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-05-06T12:03:25.581 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel x86_64 2:17.2.7-925.ge81df420.el9 ceph 126 k 2024-05-06T12:03:25.581 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:25.581 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-05-06T12:03:25.581 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:03:25.581 INFO:teuthology.orchestra.run.smithi190.stdout:Install 1 Package 2024-05-06T12:03:25.581 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:25.581 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 126 k 2024-05-06T12:03:25.581 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 448 k 2024-05-06T12:03:25.581 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-05-06T12:03:25.660 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:33 ago on Mon 06 May 2024 12:01:52 PM UTC. 2024-05-06T12:03:25.740 INFO:teuthology.orchestra.run.smithi092.stdout:Package python3-rbd-2:17.2.7-925.ge81df420.el9.x86_64 is already installed. 2024-05-06T12:03:25.780 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-06T12:03:25.781 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-05-06T12:03:25.781 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-06T12:03:25.815 INFO:teuthology.orchestra.run.smithi190.stdout:librados-devel-17.2.7-925.ge81df420.el9.x86_64. 539 kB/s | 126 kB 00:00 2024-05-06T12:03:25.816 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-05-06T12:03:25.816 INFO:teuthology.orchestra.run.smithi190.stdout:Total 535 kB/s | 126 kB 00:00 2024-05-06T12:03:25.816 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-05-06T12:03:25.822 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-05-06T12:03:25.822 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-05-06T12:03:25.855 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install rbd-fuse 2024-05-06T12:03:25.883 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-05-06T12:03:25.884 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-05-06T12:03:26.041 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-05-06T12:03:26.168 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librados-devel-2:17.2.7-925.ge81df420.el9.x86_64 1/1 2024-05-06T12:03:26.367 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:34 ago on Mon 06 May 2024 12:01:52 PM UTC. 2024-05-06T12:03:26.485 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-06T12:03:26.486 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:03:26.486 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2024-05-06T12:03:26.486 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:03:26.486 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-06T12:03:26.487 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse x86_64 2:17.2.7-925.ge81df420.el9 ceph 82 k 2024-05-06T12:03:26.487 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:26.487 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-06T12:03:26.487 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:03:26.487 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-05-06T12:03:26.487 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:26.487 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 82 k 2024-05-06T12:03:26.487 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 222 k 2024-05-06T12:03:26.487 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-06T12:03:26.555 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados-devel-2:17.2.7-925.ge81df420.el9.x86_64 1/1 2024-05-06T12:03:26.788 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados-devel-2:17.2.7-925.ge81df420.el9.x86_64 1/1 2024-05-06T12:03:26.788 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:26.788 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-05-06T12:03:26.788 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:03:26.788 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:26.788 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-05-06T12:03:26.819 INFO:teuthology.orchestra.run.smithi092.stdout:rbd-fuse-17.2.7-925.ge81df420.el9.x86_64.rpm 246 kB/s | 82 kB 00:00 2024-05-06T12:03:26.819 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-06T12:03:26.820 INFO:teuthology.orchestra.run.smithi092.stdout:Total 245 kB/s | 82 kB 00:00 2024-05-06T12:03:26.820 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-06T12:03:26.826 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-06T12:03:26.826 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-06T12:03:26.888 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-06T12:03:26.888 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-06T12:03:27.015 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install libcephfs2 2024-05-06T12:03:27.029 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-06T12:03:27.130 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-fuse-2:17.2.7-925.ge81df420.el9.x86_64 1/1 2024-05-06T12:03:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-fuse-2:17.2.7-925.ge81df420.el9.x86_64 1/1 2024-05-06T12:03:27.540 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:01:32 ago on Mon 06 May 2024 12:01:55 PM UTC. 2024-05-06T12:03:27.621 INFO:teuthology.orchestra.run.smithi190.stdout:Package libcephfs2-2:17.2.7-925.ge81df420.el9.x86_64 is already installed. 2024-05-06T12:03:27.662 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-05-06T12:03:27.663 INFO:teuthology.orchestra.run.smithi190.stdout:Nothing to do. 2024-05-06T12:03:27.663 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-05-06T12:03:27.696 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-fuse-2:17.2.7-925.ge81df420.el9.x86_64 1/1 2024-05-06T12:03:27.696 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:27.696 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-06T12:03:27.696 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:03:27.696 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:27.696 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-06T12:03:27.740 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install libcephfs-devel 2024-05-06T12:03:27.952 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install rbd-mirror 2024-05-06T12:03:28.258 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:01:33 ago on Mon 06 May 2024 12:01:55 PM UTC. 2024-05-06T12:03:28.379 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-05-06T12:03:28.379 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:03:28.379 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repo Size 2024-05-06T12:03:28.380 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:03:28.380 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-05-06T12:03:28.380 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel x86_64 2:17.2.7-925.ge81df420.el9 ceph 25 k 2024-05-06T12:03:28.380 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:28.380 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-05-06T12:03:28.380 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:03:28.380 INFO:teuthology.orchestra.run.smithi190.stdout:Install 1 Package 2024-05-06T12:03:28.380 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:28.380 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 25 k 2024-05-06T12:03:28.380 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 110 k 2024-05-06T12:03:28.380 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-05-06T12:03:28.480 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:36 ago on Mon 06 May 2024 12:01:52 PM UTC. 2024-05-06T12:03:28.543 INFO:teuthology.orchestra.run.smithi190.stdout:libcephfs-devel-17.2.7-925.ge81df420.el9.x86_64 153 kB/s | 25 kB 00:00 2024-05-06T12:03:28.544 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-05-06T12:03:28.544 INFO:teuthology.orchestra.run.smithi190.stdout:Total 152 kB/s | 25 kB 00:00 2024-05-06T12:03:28.544 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-05-06T12:03:28.547 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-05-06T12:03:28.547 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-05-06T12:03:28.549 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-05-06T12:03:28.549 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-05-06T12:03:28.603 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-06T12:03:28.603 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:03:28.604 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repo Size 2024-05-06T12:03:28.604 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:03:28.604 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-06T12:03:28.604 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror x86_64 2:17.2.7-925.ge81df420.el9 ceph 2.9 M 2024-05-06T12:03:28.604 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:28.604 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-06T12:03:28.604 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:03:28.604 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-05-06T12:03:28.604 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:28.605 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 2.9 M 2024-05-06T12:03:28.605 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 12 M 2024-05-06T12:03:28.605 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-06T12:03:28.635 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-05-06T12:03:28.738 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephfs-devel-2:17.2.7-925.ge81df420.el9.x86_64 1/1 2024-05-06T12:03:29.065 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-925.ge81df420.el9.x86_64 1/1 2024-05-06T12:03:29.262 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs-devel-2:17.2.7-925.ge81df420.el9.x86_64 1/1 2024-05-06T12:03:29.262 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:29.262 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-05-06T12:03:29.262 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:03:29.262 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:29.262 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-05-06T12:03:29.432 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install librados2 2024-05-06T12:03:29.755 INFO:teuthology.orchestra.run.smithi092.stdout:rbd-mirror-17.2.7-925.ge81df420.el9.x86_64.rpm 2.6 MB/s | 2.9 MB 00:01 2024-05-06T12:03:29.756 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-06T12:03:29.756 INFO:teuthology.orchestra.run.smithi092.stdout:Total 2.5 MB/s | 2.9 MB 00:01 2024-05-06T12:03:29.756 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-06T12:03:29.765 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-06T12:03:29.765 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-06T12:03:29.843 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-06T12:03:29.843 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-06T12:03:29.942 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:01:34 ago on Mon 06 May 2024 12:01:55 PM UTC. 2024-05-06T12:03:30.020 INFO:teuthology.orchestra.run.smithi190.stdout:Package librados2-2:17.2.7-925.ge81df420.el9.x86_64 is already installed. 2024-05-06T12:03:30.063 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-05-06T12:03:30.064 INFO:teuthology.orchestra.run.smithi190.stdout:Nothing to do. 2024-05-06T12:03:30.064 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-05-06T12:03:30.119 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install librbd1 2024-05-06T12:03:30.249 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-06T12:03:30.314 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-mirror-2:17.2.7-925.ge81df420.el9.x86_64 1/1 2024-05-06T12:03:30.342 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-mirror-2:17.2.7-925.ge81df420.el9.x86_64 1/1 2024-05-06T12:03:30.342 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-06T12:03:30.342 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-06T12:03:30.343 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-06T12:03:30.343 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-06T12:03:30.343 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:30.630 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:01:35 ago on Mon 06 May 2024 12:01:55 PM UTC. 2024-05-06T12:03:30.709 INFO:teuthology.orchestra.run.smithi190.stdout:Package librbd1-2:17.2.7-925.ge81df420.el9.x86_64 is already installed. 2024-05-06T12:03:30.750 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-05-06T12:03:30.751 INFO:teuthology.orchestra.run.smithi190.stdout:Nothing to do. 2024-05-06T12:03:30.751 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-05-06T12:03:30.819 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install python3-rados 2024-05-06T12:03:31.339 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:01:36 ago on Mon 06 May 2024 12:01:55 PM UTC. 2024-05-06T12:03:31.380 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-mirror-2:17.2.7-925.ge81df420.el9.x86_64 1/1 2024-05-06T12:03:31.380 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:31.380 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-06T12:03:31.380 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:03:31.381 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:31.381 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-06T12:03:31.418 INFO:teuthology.orchestra.run.smithi190.stdout:Package python3-rados-2:17.2.7-925.ge81df420.el9.x86_64 is already installed. 2024-05-06T12:03:31.460 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-05-06T12:03:31.461 INFO:teuthology.orchestra.run.smithi190.stdout:Nothing to do. 2024-05-06T12:03:31.461 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-05-06T12:03:31.528 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install python3-rgw 2024-05-06T12:03:31.602 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install rbd-nbd 2024-05-06T12:03:32.047 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:01:37 ago on Mon 06 May 2024 12:01:55 PM UTC. 2024-05-06T12:03:32.120 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:40 ago on Mon 06 May 2024 12:01:52 PM UTC. 2024-05-06T12:03:32.126 INFO:teuthology.orchestra.run.smithi190.stdout:Package python3-rgw-2:17.2.7-925.ge81df420.el9.x86_64 is already installed. 2024-05-06T12:03:32.166 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-05-06T12:03:32.167 INFO:teuthology.orchestra.run.smithi190.stdout:Nothing to do. 2024-05-06T12:03:32.168 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-05-06T12:03:32.241 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-06T12:03:32.242 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:03:32.242 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2024-05-06T12:03:32.242 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:03:32.242 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-06T12:03:32.242 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd x86_64 2:17.2.7-925.ge81df420.el9 ceph 165 k 2024-05-06T12:03:32.242 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:32.242 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-06T12:03:32.242 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-06T12:03:32.242 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-05-06T12:03:32.242 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:32.242 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 165 k 2024-05-06T12:03:32.242 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 477 k 2024-05-06T12:03:32.243 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-06T12:03:32.248 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install python3-cephfs 2024-05-06T12:03:32.558 INFO:teuthology.orchestra.run.smithi092.stdout:rbd-nbd-17.2.7-925.ge81df420.el9.x86_64.rpm 524 kB/s | 165 kB 00:00 2024-05-06T12:03:32.559 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-06T12:03:32.559 INFO:teuthology.orchestra.run.smithi092.stdout:Total 521 kB/s | 165 kB 00:00 2024-05-06T12:03:32.559 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-06T12:03:32.567 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-06T12:03:32.567 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-06T12:03:32.629 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-06T12:03:32.629 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-06T12:03:32.779 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-06T12:03:32.782 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:01:37 ago on Mon 06 May 2024 12:01:55 PM UTC. 2024-05-06T12:03:32.862 INFO:teuthology.orchestra.run.smithi190.stdout:Package python3-cephfs-2:17.2.7-925.ge81df420.el9.x86_64 is already installed. 2024-05-06T12:03:32.877 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-nbd-2:17.2.7-925.ge81df420.el9.x86_64 1/1 2024-05-06T12:03:32.903 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-05-06T12:03:32.904 INFO:teuthology.orchestra.run.smithi190.stdout:Nothing to do. 2024-05-06T12:03:32.904 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-05-06T12:03:32.972 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install python3-rbd 2024-05-06T12:03:33.255 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-nbd-2:17.2.7-925.ge81df420.el9.x86_64 1/1 2024-05-06T12:03:33.481 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-nbd-2:17.2.7-925.ge81df420.el9.x86_64 1/1 2024-05-06T12:03:33.481 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:33.481 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-06T12:03:33.481 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:03:33.481 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:03:33.482 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-06T12:03:33.493 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:01:38 ago on Mon 06 May 2024 12:01:55 PM UTC. 2024-05-06T12:03:33.573 INFO:teuthology.orchestra.run.smithi190.stdout:Package python3-rbd-2:17.2.7-925.ge81df420.el9.x86_64 is already installed. 2024-05-06T12:03:33.613 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-05-06T12:03:33.614 INFO:teuthology.orchestra.run.smithi190.stdout:Nothing to do. 2024-05-06T12:03:33.614 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-05-06T12:03:33.685 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install rbd-fuse 2024-05-06T12:03:33.727 DEBUG:teuthology.parallel:result is None 2024-05-06T12:03:34.197 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:01:39 ago on Mon 06 May 2024 12:01:55 PM UTC. 2024-05-06T12:03:34.316 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-05-06T12:03:34.317 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:03:34.317 INFO:teuthology.orchestra.run.smithi190.stdout: Package Architecture Version Repository Size 2024-05-06T12:03:34.317 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:03:34.318 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-05-06T12:03:34.318 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse x86_64 2:17.2.7-925.ge81df420.el9 ceph 82 k 2024-05-06T12:03:34.318 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:34.318 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-05-06T12:03:34.318 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:03:34.318 INFO:teuthology.orchestra.run.smithi190.stdout:Install 1 Package 2024-05-06T12:03:34.318 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:34.318 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 82 k 2024-05-06T12:03:34.318 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 222 k 2024-05-06T12:03:34.318 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-05-06T12:03:34.521 INFO:teuthology.orchestra.run.smithi190.stdout:rbd-fuse-17.2.7-925.ge81df420.el9.x86_64.rpm 403 kB/s | 82 kB 00:00 2024-05-06T12:03:34.521 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-05-06T12:03:34.522 INFO:teuthology.orchestra.run.smithi190.stdout:Total 400 kB/s | 82 kB 00:00 2024-05-06T12:03:34.522 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-05-06T12:03:34.528 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-05-06T12:03:34.528 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-05-06T12:03:34.591 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-05-06T12:03:34.592 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-05-06T12:03:34.734 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-05-06T12:03:34.841 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-fuse-2:17.2.7-925.ge81df420.el9.x86_64 1/1 2024-05-06T12:03:35.170 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: rbd-fuse-2:17.2.7-925.ge81df420.el9.x86_64 1/1 2024-05-06T12:03:35.420 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-fuse-2:17.2.7-925.ge81df420.el9.x86_64 1/1 2024-05-06T12:03:35.420 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:35.420 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-05-06T12:03:35.420 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:03:35.421 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:35.421 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-05-06T12:03:35.689 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install rbd-mirror 2024-05-06T12:03:36.218 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:01:41 ago on Mon 06 May 2024 12:01:55 PM UTC. 2024-05-06T12:03:36.341 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-05-06T12:03:36.341 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:03:36.342 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repo Size 2024-05-06T12:03:36.342 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:03:36.342 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-05-06T12:03:36.342 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror x86_64 2:17.2.7-925.ge81df420.el9 ceph 2.9 M 2024-05-06T12:03:36.342 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:36.342 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-05-06T12:03:36.342 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:03:36.342 INFO:teuthology.orchestra.run.smithi190.stdout:Install 1 Package 2024-05-06T12:03:36.342 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:36.342 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 2.9 M 2024-05-06T12:03:36.342 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 12 M 2024-05-06T12:03:36.342 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-05-06T12:03:37.420 INFO:teuthology.orchestra.run.smithi190.stdout:rbd-mirror-17.2.7-925.ge81df420.el9.x86_64.rpm 2.7 MB/s | 2.9 MB 00:01 2024-05-06T12:03:37.420 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-05-06T12:03:37.421 INFO:teuthology.orchestra.run.smithi190.stdout:Total 2.7 MB/s | 2.9 MB 00:01 2024-05-06T12:03:37.421 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-05-06T12:03:37.430 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-05-06T12:03:37.430 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-05-06T12:03:37.510 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-05-06T12:03:37.511 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-05-06T12:03:37.918 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-05-06T12:03:37.979 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-mirror-2:17.2.7-925.ge81df420.el9.x86_64 1/1 2024-05-06T12:03:38.003 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: rbd-mirror-2:17.2.7-925.ge81df420.el9.x86_64 1/1 2024-05-06T12:03:38.003 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-06T12:03:38.003 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-06T12:03:38.004 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-06T12:03:38.004 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-06T12:03:38.004 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:39.029 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-mirror-2:17.2.7-925.ge81df420.el9.x86_64 1/1 2024-05-06T12:03:39.029 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:39.029 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-05-06T12:03:39.029 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:03:39.029 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:39.029 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-05-06T12:03:39.274 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install rbd-nbd 2024-05-06T12:03:39.797 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:01:44 ago on Mon 06 May 2024 12:01:55 PM UTC. 2024-05-06T12:03:39.920 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-05-06T12:03:39.921 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:03:39.921 INFO:teuthology.orchestra.run.smithi190.stdout: Package Architecture Version Repository Size 2024-05-06T12:03:39.921 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:03:39.921 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-05-06T12:03:39.921 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd x86_64 2:17.2.7-925.ge81df420.el9 ceph 165 k 2024-05-06T12:03:39.922 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:39.922 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-05-06T12:03:39.922 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-06T12:03:39.922 INFO:teuthology.orchestra.run.smithi190.stdout:Install 1 Package 2024-05-06T12:03:39.922 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:39.922 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 165 k 2024-05-06T12:03:39.922 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 477 k 2024-05-06T12:03:39.922 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-05-06T12:03:40.140 INFO:teuthology.orchestra.run.smithi190.stdout:rbd-nbd-17.2.7-925.ge81df420.el9.x86_64.rpm 757 kB/s | 165 kB 00:00 2024-05-06T12:03:40.141 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-05-06T12:03:40.141 INFO:teuthology.orchestra.run.smithi190.stdout:Total 752 kB/s | 165 kB 00:00 2024-05-06T12:03:40.141 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-05-06T12:03:40.149 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-05-06T12:03:40.149 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-05-06T12:03:40.211 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-05-06T12:03:40.211 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-05-06T12:03:40.358 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-05-06T12:03:40.469 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-nbd-2:17.2.7-925.ge81df420.el9.x86_64 1/1 2024-05-06T12:03:40.847 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: rbd-nbd-2:17.2.7-925.ge81df420.el9.x86_64 1/1 2024-05-06T12:03:41.121 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-nbd-2:17.2.7-925.ge81df420.el9.x86_64 1/1 2024-05-06T12:03:41.121 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:41.121 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-05-06T12:03:41.121 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd-2:17.2.7-925.ge81df420.el9.x86_64 2024-05-06T12:03:41.121 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:03:41.121 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-05-06T12:03:41.360 DEBUG:teuthology.parallel:result is None 2024-05-06T12:03:41.361 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-06T12:03:41.361 INFO:teuthology.packaging:ref: None 2024-05-06T12:03:41.361 INFO:teuthology.packaging:tag: None 2024-05-06T12:03:41.361 INFO:teuthology.packaging:branch: quincy 2024-05-06T12:03:41.361 INFO:teuthology.packaging:sha1: 70ce9539b62a15b9de429c1f0e91a1e012b099aa 2024-05-06T12:03:41.362 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-05-06T12:03:41.626 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-06T12:03:41.652 INFO:teuthology.orchestra.run.smithi092.stdout:17.2.7-925.ge81df420.el9 2024-05-06T12:03:41.653 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-925.ge81df420.el9 2024-05-06T12:03:41.653 INFO:teuthology.task.install:The correct ceph version 17.2.7-925.ge81df420 is installed. 2024-05-06T12:03:41.657 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-06T12:03:41.657 INFO:teuthology.packaging:ref: None 2024-05-06T12:03:41.657 INFO:teuthology.packaging:tag: None 2024-05-06T12:03:41.657 INFO:teuthology.packaging:branch: quincy 2024-05-06T12:03:41.657 INFO:teuthology.packaging:sha1: 70ce9539b62a15b9de429c1f0e91a1e012b099aa 2024-05-06T12:03:41.658 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-05-06T12:03:42.109 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-06T12:03:42.131 INFO:teuthology.orchestra.run.smithi190.stdout:17.2.7-925.ge81df420.el9 2024-05-06T12:03:42.131 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-925.ge81df420.el9 2024-05-06T12:03:42.132 INFO:teuthology.task.install:The correct ceph version 17.2.7-925.ge81df420 is installed. 2024-05-06T12:03:42.134 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-06T12:03:42.135 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-06T12:03:42.135 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-06T12:03:42.163 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-06T12:03:42.164 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-06T12:03:42.196 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-06T12:03:42.196 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-06T12:03:42.196 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/daemon-helper 2024-05-06T12:03:42.231 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-06T12:03:42.294 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-06T12:03:42.295 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/daemon-helper 2024-05-06T12:03:42.317 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-06T12:03:42.382 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-06T12:03:42.382 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-06T12:03:42.382 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-06T12:03:42.405 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-06T12:03:42.469 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-06T12:03:42.469 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-06T12:03:42.492 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-06T12:03:42.555 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-06T12:03:42.556 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-06T12:03:42.556 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/stdin-killer 2024-05-06T12:03:42.580 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-06T12:03:42.649 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-06T12:03:42.649 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/stdin-killer 2024-05-06T12:03:42.672 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-06T12:03:42.736 INFO:teuthology.run_tasks:Running task print... 2024-05-06T12:03:42.745 INFO:teuthology.task.print:**** done install task... 2024-05-06T12:03:42.746 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-06T12:03:42.847 INFO:tasks.cephadm:Config: {'cephadm_branch': 'quincy', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}}, 'image': 'quay.ceph.io/ceph-ci/ceph:quincy', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': '70ce9539b62a15b9de429c1f0e91a1e012b099aa'} 2024-05-06T12:03:42.848 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:quincy 2024-05-06T12:03:42.848 INFO:tasks.cephadm:Cluster fsid is aee1d912-0ba0-11ef-bc97-c7b262605968 2024-05-06T12:03:42.848 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-06T12:03:42.848 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-05-06T12:03:42.848 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi092': '172.21.15.92', 'mon.smithi190': '172.21.15.190'} 2024-05-06T12:03:42.848 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-06T12:03:42.849 DEBUG:teuthology.orchestra.run.smithi092:> sudo hostname $(hostname -s) 2024-05-06T12:03:42.873 DEBUG:teuthology.orchestra.run.smithi190:> sudo hostname $(hostname -s) 2024-05-06T12:03:42.897 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref quincy)... 2024-05-06T12:03:42.898 DEBUG:teuthology.orchestra.run.smithi092:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-06T12:03:43.201 INFO:teuthology.orchestra.run.smithi092.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 May 6 12:03 /home/ubuntu/cephtest/cephadm 2024-05-06T12:03:43.201 DEBUG:teuthology.orchestra.run.smithi190:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-06T12:03:43.490 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 May 6 12:03 /home/ubuntu/cephtest/cephadm 2024-05-06T12:03:43.490 DEBUG:teuthology.orchestra.run.smithi092:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-06T12:03:43.507 DEBUG:teuthology.orchestra.run.smithi190:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-06T12:03:43.533 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:quincy on all hosts... 2024-05-06T12:03:43.533 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-05-06T12:03:43.549 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-05-06T12:03:44.943 INFO:teuthology.orchestra.run.smithi092.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-05-06T12:03:45.120 INFO:teuthology.orchestra.run.smithi190.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-05-06T12:04:35.685 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-05-06T12:04:35.685 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_version": "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)", 2024-05-06T12:04:35.685 INFO:teuthology.orchestra.run.smithi190.stdout: "image_id": "96d5ae7fd4b440c05b2389bfd474e9e1f4dd7545543250f9a951bd3ff2478bbf", 2024-05-06T12:04:35.685 INFO:teuthology.orchestra.run.smithi190.stdout: "repo_digests": [ 2024-05-06T12:04:35.685 INFO:teuthology.orchestra.run.smithi190.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:1a0b7fd3cefd2c5baddf682493f92b4e27596f3c0ff27b02fdccd2891f56859f" 2024-05-06T12:04:35.686 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-05-06T12:04:35.686 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-05-06T12:04:42.277 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:04:42.277 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_version": "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)", 2024-05-06T12:04:42.277 INFO:teuthology.orchestra.run.smithi092.stdout: "image_id": "96d5ae7fd4b440c05b2389bfd474e9e1f4dd7545543250f9a951bd3ff2478bbf", 2024-05-06T12:04:42.277 INFO:teuthology.orchestra.run.smithi092.stdout: "repo_digests": [ 2024-05-06T12:04:42.277 INFO:teuthology.orchestra.run.smithi092.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:1a0b7fd3cefd2c5baddf682493f92b4e27596f3c0ff27b02fdccd2891f56859f" 2024-05-06T12:04:42.277 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-05-06T12:04:42.277 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:04:42.295 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph 2024-05-06T12:04:42.352 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph 2024-05-06T12:04:42.384 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 777 /etc/ceph 2024-05-06T12:04:42.407 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 777 /etc/ceph 2024-05-06T12:04:42.451 INFO:tasks.cephadm:Writing seed config... 2024-05-06T12:04:42.452 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-05-06T12:04:42.452 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-05-06T12:04:42.452 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-05-06T12:04:42.452 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-05-06T12:04:42.452 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-05-06T12:04:42.453 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-06T12:04:42.453 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-06T12:04:42.453 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-05-06T12:04:42.453 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-05-06T12:04:42.453 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-06T12:04:42.453 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-06T12:04:42.453 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-05-06T12:04:42.453 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-06T12:04:42.453 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-06T12:04:42.453 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-06T12:04:42.453 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-06T12:04:42.453 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-06T12:04:42.453 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-06T12:04:42.453 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-06T12:04:42.454 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-06T12:04:42.454 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-06T12:04:42.454 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-06T12:04:42.454 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-06T12:04:42.454 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-05-06T12:04:42.454 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-06T12:04:42.454 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-06T12:04:42.454 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-06T12:04:42.454 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-06T12:04:42.454 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-06T12:04:42.454 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-06T12:04:42.454 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-06T12:04:42.454 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-06T12:04:42.454 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-06T12:04:42.454 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-06T12:04:42.455 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-06T12:04:42.455 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-06T12:04:42.455 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-06T12:04:42.455 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-06T12:04:42.455 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-06T12:04:42.455 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-06T12:04:42.455 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-06T12:04:42.455 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-06T12:04:42.456 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-06T12:04:42.456 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-06T12:04:42.470 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = aee1d912-0ba0-11ef-bc97-c7b262605968 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-06T12:04:42.471 DEBUG:teuthology.orchestra.run.smithi092:mon.smithi092> sudo journalctl -f -n 0 -u ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@mon.smithi092.service 2024-05-06T12:04:42.513 INFO:tasks.cephadm:Bootstrapping... 2024-05-06T12:04:42.513 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy -v bootstrap --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.92 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-06T12:04:42.711 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-06T12:04:42.711 INFO:teuthology.orchestra.run.smithi092.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:quincy', '-v', 'bootstrap', '--fsid', 'aee1d912-0ba0-11ef-bc97-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.92', '--skip-admin-label'] 2024-05-06T12:04:42.736 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-06T12:04:42.737 INFO:teuthology.orchestra.run.smithi092.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-06T12:04:42.737 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying podman|docker is present... 2024-05-06T12:04:42.760 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-06T12:04:42.760 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying lvm2 is present... 2024-05-06T12:04:42.760 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying time synchronization is in place... 2024-05-06T12:04:42.768 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-06T12:04:42.768 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-06T12:04:42.774 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-06T12:04:42.774 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout inactive 2024-05-06T12:04:42.780 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout enabled 2024-05-06T12:04:42.787 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout active 2024-05-06T12:04:42.787 INFO:teuthology.orchestra.run.smithi092.stdout:Unit chronyd.service is enabled and running 2024-05-06T12:04:42.787 INFO:teuthology.orchestra.run.smithi092.stdout:Repeating the final host check... 2024-05-06T12:04:42.810 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-06T12:04:42.810 INFO:teuthology.orchestra.run.smithi092.stdout:podman (/bin/podman) version 4.9.4 is present 2024-05-06T12:04:42.810 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl is present 2024-05-06T12:04:42.810 INFO:teuthology.orchestra.run.smithi092.stdout:lvcreate is present 2024-05-06T12:04:42.816 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-06T12:04:42.817 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-06T12:04:42.823 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-06T12:04:42.823 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout inactive 2024-05-06T12:04:42.829 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout enabled 2024-05-06T12:04:42.836 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout active 2024-05-06T12:04:42.836 INFO:teuthology.orchestra.run.smithi092.stdout:Unit chronyd.service is enabled and running 2024-05-06T12:04:42.836 INFO:teuthology.orchestra.run.smithi092.stdout:Host looks OK 2024-05-06T12:04:42.836 INFO:teuthology.orchestra.run.smithi092.stdout:Cluster fsid: aee1d912-0ba0-11ef-bc97-c7b262605968 2024-05-06T12:04:42.836 INFO:teuthology.orchestra.run.smithi092.stdout:Acquiring lock 139951018603424 on /run/cephadm/aee1d912-0ba0-11ef-bc97-c7b262605968.lock 2024-05-06T12:04:42.836 INFO:teuthology.orchestra.run.smithi092.stdout:Lock 139951018603424 acquired on /run/cephadm/aee1d912-0ba0-11ef-bc97-c7b262605968.lock 2024-05-06T12:04:42.837 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying IP 172.21.15.92 port 3300 ... 2024-05-06T12:04:42.837 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying IP 172.21.15.92 port 6789 ... 2024-05-06T12:04:42.837 INFO:teuthology.orchestra.run.smithi092.stdout:Base mon IP(s) is [172.21.15.92:3300, 172.21.15.92:6789], mon addrv is [v2:172.21.15.92:3300,v1:172.21.15.92:6789] 2024-05-06T12:04:42.841 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.92 metric 100 2024-05-06T12:04:42.841 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.92 metric 100 2024-05-06T12:04:42.844 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-06T12:04:42.844 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-05-06T12:04:42.844 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-05-06T12:04:42.847 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-06T12:04:42.847 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-06T12:04:42.847 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-06T12:04:42.847 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-05-06T12:04:42.847 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout inet6 fe80::519e:3ffc:201e:e942/64 scope link noprefixroute 2024-05-06T12:04:42.848 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-06T12:04:42.848 INFO:teuthology.orchestra.run.smithi092.stdout:Mon IP `172.21.15.92` is in CIDR network `172.21.0.0/20` 2024-05-06T12:04:42.848 INFO:teuthology.orchestra.run.smithi092.stdout:Mon IP `172.21.15.92` is in CIDR network `172.21.0.0/20` 2024-05-06T12:04:42.848 INFO:teuthology.orchestra.run.smithi092.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-06T12:04:42.848 INFO:teuthology.orchestra.run.smithi092.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-06T12:04:42.848 INFO:teuthology.orchestra.run.smithi092.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-05-06T12:04:43.335 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stdout 96d5ae7fd4b440c05b2389bfd474e9e1f4dd7545543250f9a951bd3ff2478bbf 2024-05-06T12:04:43.335 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:quincy... 2024-05-06T12:04:43.335 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stderr Getting image source signatures 2024-05-06T12:04:43.335 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stderr Copying blob sha256:d347531dbfb1d868848a4af113d98d0a9b3f508d90935f533dcb0c8321230427 2024-05-06T12:04:43.335 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stderr Copying blob sha256:e2f2fd2adca92139da4b4b907fa90a6775d3eb4eddaa977c42ef37c66eea70bd 2024-05-06T12:04:43.335 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stderr Copying config sha256:96d5ae7fd4b440c05b2389bfd474e9e1f4dd7545543250f9a951bd3ff2478bbf 2024-05-06T12:04:43.335 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-06T12:04:44.290 INFO:teuthology.orchestra.run.smithi092.stdout:ceph: stdout ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable) 2024-05-06T12:04:44.290 INFO:teuthology.orchestra.run.smithi092.stdout:Ceph version: ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable) 2024-05-06T12:04:44.290 INFO:teuthology.orchestra.run.smithi092.stdout:Extracting ceph user uid/gid from container image... 2024-05-06T12:04:45.238 INFO:teuthology.orchestra.run.smithi092.stdout:stat: stdout 167 167 2024-05-06T12:04:45.238 INFO:teuthology.orchestra.run.smithi092.stdout:Creating initial keys... 2024-05-06T12:04:46.150 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph-authtool: stdout AQBdxzhmJWG5IRAAnCiaTkdqrgHPNBut8M2Z7A== 2024-05-06T12:04:46.949 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph-authtool: stdout AQBexzhm/CuoGxAArrdT0igClaP3Jfi5zHIG/w== 2024-05-06T12:04:47.913 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph-authtool: stdout AQBfxzhmAbAAERAAfA5HEHZxOIlv+HoXw4cBcQ== 2024-05-06T12:04:47.914 INFO:teuthology.orchestra.run.smithi092.stdout:Creating initial monmap... 2024-05-06T12:04:48.818 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-06T12:04:48.818 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-05-06T12:04:48.818 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to aee1d912-0ba0-11ef-bc97-c7b262605968 2024-05-06T12:04:48.818 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-06T12:04:48.819 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool for smithi092 [v2:172.21.15.92:3300,v1:172.21.15.92:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-06T12:04:48.819 INFO:teuthology.orchestra.run.smithi092.stdout:setting min_mon_release = octopus 2024-05-06T12:04:48.819 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/monmaptool: set fsid to aee1d912-0ba0-11ef-bc97-c7b262605968 2024-05-06T12:04:48.819 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-06T12:04:48.819 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:04:48.819 INFO:teuthology.orchestra.run.smithi092.stdout:Creating mon... 2024-05-06T12:04:49.865 INFO:teuthology.orchestra.run.smithi092.stdout:create mon.smithi092 on 2024-05-06T12:04:50.055 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-06T12:04:50.206 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-06T12:04:50.352 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-aee1d912-0ba0-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-aee1d912-0ba0-11ef-bc97-c7b262605968.target. 2024-05-06T12:04:50.352 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-aee1d912-0ba0-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-aee1d912-0ba0-11ef-bc97-c7b262605968.target. 2024-05-06T12:04:50.545 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@mon.smithi092 2024-05-06T12:04:50.545 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Failed to reset failed state of unit ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@mon.smithi092.service: Unit ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@mon.smithi092.service not loaded. 2024-05-06T12:04:50.710 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-aee1d912-0ba0-11ef-bc97-c7b262605968.target.wants/ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@mon.smithi092.service → /etc/systemd/system/ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@.service. 2024-05-06T12:04:51.297 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-06T12:04:51.297 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout disabled 2024-05-06T12:04:51.306 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-06T12:04:51.306 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout inactive 2024-05-06T12:04:51.306 INFO:teuthology.orchestra.run.smithi092.stdout:firewalld.service is not enabled 2024-05-06T12:04:51.306 INFO:teuthology.orchestra.run.smithi092.stdout:Not possible to enable service . firewalld.service is not available 2024-05-06T12:04:51.306 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for mon to start... 2024-05-06T12:04:51.306 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for mon... 2024-05-06T12:04:52.422 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:52 smithi092 ceph-mon[26584]: from='client.? 172.21.15.92:0/3254455779' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-06T12:04:52.675 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout cluster: 2024-05-06T12:04:52.675 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout id: aee1d912-0ba0-11ef-bc97-c7b262605968 2024-05-06T12:04:52.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-06T12:04:52.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-05-06T12:04:52.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout services: 2024-05-06T12:04:52.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi092 (age 0.850927s) 2024-05-06T12:04:52.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-06T12:04:52.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-06T12:04:52.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-05-06T12:04:52.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout data: 2024-05-06T12:04:52.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-06T12:04:52.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-06T12:04:52.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-06T12:04:52.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout pgs: 2024-05-06T12:04:52.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-05-06T12:04:52.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.995+0000 7f546ffc0700 1 Processor -- start 2024-05-06T12:04:52.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.995+0000 7f546ffc0700 1 -- start start 2024-05-06T12:04:52.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.995+0000 7f546ffc0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5468101740 0x7f5468101b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:04:52.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.995+0000 7f546ffc0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5468102090 con 0x7f5468101740 2024-05-06T12:04:52.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.995+0000 7f546dd5c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5468101740 0x7f5468101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:04:52.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.995+0000 7f546dd5c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5468101740 0x7f5468101b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38748/0 (socket says 172.21.15.92:38748) 2024-05-06T12:04:52.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.995+0000 7f546dd5c700 1 -- 172.21.15.92:0/3596899134 learned_addr learned my addr 172.21.15.92:0/3596899134 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:04:52.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.996+0000 7f546dd5c700 1 -- 172.21.15.92:0/3596899134 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54681021d0 con 0x7f5468101740 2024-05-06T12:04:52.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.996+0000 7f546dd5c700 1 --2- 172.21.15.92:0/3596899134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5468101740 0x7f5468101b50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f54640098e0 tx=0x7f5464009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1355931e612cf5b server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:04:52.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.996+0000 7f546cd5a700 1 -- 172.21.15.92:0/3596899134 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f546400b860 con 0x7f5468101740 2024-05-06T12:04:52.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.996+0000 7f546cd5a700 1 -- 172.21.15.92:0/3596899134 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f546400b9c0 con 0x7f5468101740 2024-05-06T12:04:52.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.996+0000 7f546cd5a700 1 -- 172.21.15.92:0/3596899134 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f546400bc90 con 0x7f5468101740 2024-05-06T12:04:52.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.996+0000 7f546ffc0700 1 -- 172.21.15.92:0/3596899134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5468101740 msgr2=0x7f5468101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:04:52.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.996+0000 7f546ffc0700 1 --2- 172.21.15.92:0/3596899134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5468101740 0x7f5468101b50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f54640098e0 tx=0x7f5464009250 comp rx=0 tx=0).stop 2024-05-06T12:04:52.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.997+0000 7f546ffc0700 1 -- 172.21.15.92:0/3596899134 shutdown_connections 2024-05-06T12:04:52.678 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.997+0000 7f546ffc0700 1 --2- 172.21.15.92:0/3596899134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5468101740 0x7f5468101b50 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:04:52.678 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.997+0000 7f546ffc0700 1 -- 172.21.15.92:0/3596899134 >> 172.21.15.92:0/3596899134 conn(0x7f54680fcec0 msgr2=0x7f54680ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:04:52.678 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.997+0000 7f546ffc0700 1 -- 172.21.15.92:0/3596899134 shutdown_connections 2024-05-06T12:04:52.678 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.997+0000 7f546ffc0700 1 -- 172.21.15.92:0/3596899134 wait complete. 2024-05-06T12:04:52.678 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.997+0000 7f546ffc0700 1 Processor -- start 2024-05-06T12:04:52.678 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.997+0000 7f546ffc0700 1 -- start start 2024-05-06T12:04:52.678 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.997+0000 7f546ffc0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5468101740 0x7f546810fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:04:52.678 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.997+0000 7f546ffc0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5468110220 con 0x7f5468101740 2024-05-06T12:04:52.678 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.997+0000 7f546dd5c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5468101740 0x7f546810fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:04:52.678 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.998+0000 7f546dd5c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5468101740 0x7f546810fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38758/0 (socket says 172.21.15.92:38758) 2024-05-06T12:04:52.678 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.998+0000 7f546dd5c700 1 -- 172.21.15.92:0/3254455779 learned_addr learned my addr 172.21.15.92:0/3254455779 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:04:52.678 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.998+0000 7f546dd5c700 1 -- 172.21.15.92:0/3254455779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54640094f0 con 0x7f5468101740 2024-05-06T12:04:52.679 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.998+0000 7f546dd5c700 1 --2- 172.21.15.92:0/3254455779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5468101740 0x7f546810fce0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5464003080 tx=0x7f5464016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:04:52.679 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.998+0000 7f545e7fc700 1 -- 172.21.15.92:0/3254455779 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f546401c5c0 con 0x7f5468101740 2024-05-06T12:04:52.679 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.998+0000 7f545e7fc700 1 -- 172.21.15.92:0/3254455779 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f546401c720 con 0x7f5468101740 2024-05-06T12:04:52.679 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.998+0000 7f545e7fc700 1 -- 172.21.15.92:0/3254455779 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f546401ca30 con 0x7f5468101740 2024-05-06T12:04:52.679 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.998+0000 7f546ffc0700 1 -- 172.21.15.92:0/3254455779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5468110420 con 0x7f5468101740 2024-05-06T12:04:52.679 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.998+0000 7f546ffc0700 1 -- 172.21.15.92:0/3254455779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5468110f90 con 0x7f5468101740 2024-05-06T12:04:52.679 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.998+0000 7f545e7fc700 1 -- 172.21.15.92:0/3254455779 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f546401cc60 con 0x7f5468101740 2024-05-06T12:04:52.679 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.998+0000 7f545e7fc700 1 -- 172.21.15.92:0/3254455779 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f546401ac80 con 0x7f5468101740 2024-05-06T12:04:52.679 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:51.999+0000 7f546ffc0700 1 -- 172.21.15.92:0/3254455779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f546805f5c0 con 0x7f5468101740 2024-05-06T12:04:52.679 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:52.001+0000 7f545e7fc700 1 -- 172.21.15.92:0/3254455779 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f546401a450 con 0x7f5468101740 2024-05-06T12:04:52.679 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:52.069+0000 7f546ffc0700 1 -- 172.21.15.92:0/3254455779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f54681111d0 con 0x7f5468101740 2024-05-06T12:04:52.679 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:52.070+0000 7f545e7fc700 1 -- 172.21.15.92:0/3254455779 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7f5464025440 con 0x7f5468101740 2024-05-06T12:04:52.679 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:52.070+0000 7f546ffc0700 1 -- 172.21.15.92:0/3254455779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5468101740 msgr2=0x7f546810fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:04:52.680 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:52.070+0000 7f546ffc0700 1 --2- 172.21.15.92:0/3254455779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5468101740 0x7f546810fce0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5464003080 tx=0x7f5464016000 comp rx=0 tx=0).stop 2024-05-06T12:04:52.680 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:52.070+0000 7f546ffc0700 1 -- 172.21.15.92:0/3254455779 shutdown_connections 2024-05-06T12:04:52.680 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:52.070+0000 7f546ffc0700 1 --2- 172.21.15.92:0/3254455779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5468101740 0x7f546810fce0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:04:52.680 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:52.070+0000 7f546ffc0700 1 -- 172.21.15.92:0/3254455779 >> 172.21.15.92:0/3254455779 conn(0x7f54680fcec0 msgr2=0x7f54681064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:04:52.680 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:52.071+0000 7f546ffc0700 1 -- 172.21.15.92:0/3254455779 shutdown_connections 2024-05-06T12:04:52.680 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:52.071+0000 7f546ffc0700 1 -- 172.21.15.92:0/3254455779 wait complete. 2024-05-06T12:04:52.680 INFO:teuthology.orchestra.run.smithi092.stdout:mon is available 2024-05-06T12:04:52.680 INFO:teuthology.orchestra.run.smithi092.stdout:Assimilating anything we can from ceph.conf... 2024-05-06T12:04:53.461 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:53 smithi092 ceph-mon[26584]: from='client.? 172.21.15.92:0/1931830245' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-06T12:04:53.461 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:53 smithi092 ceph-mon[26584]: from='client.? 172.21.15.92:0/1931830245' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-06T12:04:53.747 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-05-06T12:04:53.747 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout [global] 2024-05-06T12:04:53.747 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout fsid = aee1d912-0ba0-11ef-bc97-c7b262605968 2024-05-06T12:04:53.747 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.92:3300,v1:172.21.15.92:6789] 2024-05-06T12:04:53.747 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-06T12:04:53.747 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-06T12:04:53.747 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-06T12:04:53.747 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-06T12:04:53.747 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-05-06T12:04:53.747 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-06T12:04:53.747 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-06T12:04:53.748 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-05-06T12:04:53.748 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout [osd] 2024-05-06T12:04:53.748 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-06T12:04:53.749 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-06T12:04:53.749 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-06T12:04:53.749 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-06T12:04:53.749 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-06T12:04:53.750 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.083+0000 7fb314ab1700 1 Processor -- start 2024-05-06T12:04:53.750 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.083+0000 7fb314ab1700 1 -- start start 2024-05-06T12:04:53.750 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.083+0000 7fb314ab1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb310102880 0x7fb310102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:04:53.750 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.083+0000 7fb314ab1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3101031d0 con 0x7fb310102880 2024-05-06T12:04:53.750 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.083+0000 7fb30e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb310102880 0x7fb310102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:04:53.750 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.083+0000 7fb30e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb310102880 0x7fb310102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57538/0 (socket says 172.21.15.92:57538) 2024-05-06T12:04:53.750 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.083+0000 7fb30e59c700 1 -- 172.21.15.92:0/3234523048 learned_addr learned my addr 172.21.15.92:0/3234523048 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:04:53.750 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.084+0000 7fb30e59c700 1 -- 172.21.15.92:0/3234523048 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb310103310 con 0x7fb310102880 2024-05-06T12:04:53.750 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.084+0000 7fb30e59c700 1 --2- 172.21.15.92:0/3234523048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb310102880 0x7fb310102c90 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fb3000098e0 tx=0x7fb300009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c0a7033126f3837 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:04:53.750 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.084+0000 7fb30d59a700 1 -- 172.21.15.92:0/3234523048 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb30000b860 con 0x7fb310102880 2024-05-06T12:04:53.750 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.084+0000 7fb30d59a700 1 -- 172.21.15.92:0/3234523048 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30000b9c0 con 0x7fb310102880 2024-05-06T12:04:53.750 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.084+0000 7fb30d59a700 1 -- 172.21.15.92:0/3234523048 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb30000bc90 con 0x7fb310102880 2024-05-06T12:04:53.750 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.084+0000 7fb314ab1700 1 -- 172.21.15.92:0/3234523048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb310102880 msgr2=0x7fb310102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:04:53.751 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.084+0000 7fb314ab1700 1 --2- 172.21.15.92:0/3234523048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb310102880 0x7fb310102c90 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fb3000098e0 tx=0x7fb300009250 comp rx=0 tx=0).stop 2024-05-06T12:04:53.751 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.085+0000 7fb314ab1700 1 -- 172.21.15.92:0/3234523048 shutdown_connections 2024-05-06T12:04:53.751 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.085+0000 7fb314ab1700 1 --2- 172.21.15.92:0/3234523048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb310102880 0x7fb310102c90 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:04:53.751 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.085+0000 7fb314ab1700 1 -- 172.21.15.92:0/3234523048 >> 172.21.15.92:0/3234523048 conn(0x7fb3100fe060 msgr2=0x7fb310100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:04:53.751 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.085+0000 7fb314ab1700 1 -- 172.21.15.92:0/3234523048 shutdown_connections 2024-05-06T12:04:53.751 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.085+0000 7fb314ab1700 1 -- 172.21.15.92:0/3234523048 wait complete. 2024-05-06T12:04:53.751 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.085+0000 7fb314ab1700 1 Processor -- start 2024-05-06T12:04:53.751 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.085+0000 7fb314ab1700 1 -- start start 2024-05-06T12:04:53.751 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.085+0000 7fb314ab1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb310102880 0x7fb310110e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:04:53.751 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.085+0000 7fb314ab1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3101113a0 con 0x7fb310102880 2024-05-06T12:04:53.751 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.085+0000 7fb30e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb310102880 0x7fb310110e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:04:53.751 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.085+0000 7fb30e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb310102880 0x7fb310110e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57548/0 (socket says 172.21.15.92:57548) 2024-05-06T12:04:53.751 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.085+0000 7fb30e59c700 1 -- 172.21.15.92:0/1931830245 learned_addr learned my addr 172.21.15.92:0/1931830245 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:04:53.751 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.085+0000 7fb30e59c700 1 -- 172.21.15.92:0/1931830245 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3000094f0 con 0x7fb310102880 2024-05-06T12:04:53.752 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.086+0000 7fb30e59c700 1 --2- 172.21.15.92:0/1931830245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb310102880 0x7fb310110e60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fb300003080 tx=0x7fb300016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:04:53.752 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.086+0000 7fb2feffd700 1 -- 172.21.15.92:0/1931830245 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb30001c5c0 con 0x7fb310102880 2024-05-06T12:04:53.752 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.086+0000 7fb2feffd700 1 -- 172.21.15.92:0/1931830245 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30001c720 con 0x7fb310102880 2024-05-06T12:04:53.752 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.086+0000 7fb314ab1700 1 -- 172.21.15.92:0/1931830245 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3101115a0 con 0x7fb310102880 2024-05-06T12:04:53.752 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.086+0000 7fb2feffd700 1 -- 172.21.15.92:0/1931830245 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb30001ca30 con 0x7fb310102880 2024-05-06T12:04:53.752 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.086+0000 7fb314ab1700 1 -- 172.21.15.92:0/1931830245 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb310111fd0 con 0x7fb310102880 2024-05-06T12:04:53.752 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.086+0000 7fb2feffd700 1 -- 172.21.15.92:0/1931830245 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fb30001ccb0 con 0x7fb310102880 2024-05-06T12:04:53.752 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.086+0000 7fb2feffd700 1 -- 172.21.15.92:0/1931830245 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fb30001ac90 con 0x7fb310102880 2024-05-06T12:04:53.752 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.087+0000 7fb314ab1700 1 -- 172.21.15.92:0/1931830245 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb31005f5c0 con 0x7fb310102880 2024-05-06T12:04:53.752 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.089+0000 7fb2feffd700 1 -- 172.21.15.92:0/1931830245 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7fb30001a450 con 0x7fb310102880 2024-05-06T12:04:53.752 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.155+0000 7fb314ab1700 1 -- 172.21.15.92:0/1931830245 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fb31010a850 con 0x7fb310102880 2024-05-06T12:04:53.752 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.158+0000 7fb2feffd700 1 -- 172.21.15.92:0/1931830245 <== mon.0 v2:172.21.15.92:3300/0 7 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fb30001b870 con 0x7fb310102880 2024-05-06T12:04:53.753 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.158+0000 7fb2feffd700 1 -- 172.21.15.92:0/1931830245 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fb300024610 con 0x7fb310102880 2024-05-06T12:04:53.753 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.160+0000 7fb314ab1700 1 -- 172.21.15.92:0/1931830245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb310102880 msgr2=0x7fb310110e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:04:53.753 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.160+0000 7fb314ab1700 1 --2- 172.21.15.92:0/1931830245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb310102880 0x7fb310110e60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fb300003080 tx=0x7fb300016000 comp rx=0 tx=0).stop 2024-05-06T12:04:53.753 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.160+0000 7fb314ab1700 1 -- 172.21.15.92:0/1931830245 shutdown_connections 2024-05-06T12:04:53.753 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.160+0000 7fb314ab1700 1 --2- 172.21.15.92:0/1931830245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb310102880 0x7fb310110e60 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:04:53.753 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.160+0000 7fb314ab1700 1 -- 172.21.15.92:0/1931830245 >> 172.21.15.92:0/1931830245 conn(0x7fb3100fe060 msgr2=0x7fb310105250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:04:53.753 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.160+0000 7fb314ab1700 1 -- 172.21.15.92:0/1931830245 shutdown_connections 2024-05-06T12:04:53.753 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:53.160+0000 7fb314ab1700 1 -- 172.21.15.92:0/1931830245 wait complete. 2024-05-06T12:04:53.753 INFO:teuthology.orchestra.run.smithi092.stdout:Generating new minimal ceph.conf... 2024-05-06T12:04:54.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:54 smithi092 ceph-mon[26584]: from='client.? 172.21.15.92:0/1223991634' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:04:54.825 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.150+0000 7f2a585b6700 1 Processor -- start 2024-05-06T12:04:54.825 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.150+0000 7f2a585b6700 1 -- start start 2024-05-06T12:04:54.825 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.151+0000 7f2a585b6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a50101d60 0x7f2a50102170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:04:54.825 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.151+0000 7f2a585b6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a501026b0 con 0x7f2a50101d60 2024-05-06T12:04:54.826 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.151+0000 7f2a56352700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a50101d60 0x7f2a50102170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:04:54.826 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.151+0000 7f2a56352700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a50101d60 0x7f2a50102170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57550/0 (socket says 172.21.15.92:57550) 2024-05-06T12:04:54.826 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.151+0000 7f2a56352700 1 -- 172.21.15.92:0/311577161 learned_addr learned my addr 172.21.15.92:0/311577161 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:04:54.826 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.151+0000 7f2a56352700 1 -- 172.21.15.92:0/311577161 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a501027f0 con 0x7f2a50101d60 2024-05-06T12:04:54.826 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.151+0000 7f2a56352700 1 --2- 172.21.15.92:0/311577161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a50101d60 0x7f2a50102170 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2a400098e0 tx=0x7f2a40009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5501fcd099c818 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:04:54.826 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.151+0000 7f2a55350700 1 -- 172.21.15.92:0/311577161 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2a4000b860 con 0x7f2a50101d60 2024-05-06T12:04:54.826 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.151+0000 7f2a55350700 1 -- 172.21.15.92:0/311577161 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f2a4000be50 con 0x7f2a50101d60 2024-05-06T12:04:54.826 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.152+0000 7f2a55350700 1 -- 172.21.15.92:0/311577161 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2a40010380 con 0x7f2a50101d60 2024-05-06T12:04:54.826 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.152+0000 7f2a585b6700 1 -- 172.21.15.92:0/311577161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a50101d60 msgr2=0x7f2a50102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:04:54.826 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.152+0000 7f2a585b6700 1 --2- 172.21.15.92:0/311577161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a50101d60 0x7f2a50102170 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2a400098e0 tx=0x7f2a40009250 comp rx=0 tx=0).stop 2024-05-06T12:04:54.826 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.152+0000 7f2a585b6700 1 -- 172.21.15.92:0/311577161 shutdown_connections 2024-05-06T12:04:54.826 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.152+0000 7f2a585b6700 1 --2- 172.21.15.92:0/311577161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a50101d60 0x7f2a50102170 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:04:54.827 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.152+0000 7f2a585b6700 1 -- 172.21.15.92:0/311577161 >> 172.21.15.92:0/311577161 conn(0x7f2a500fc6f0 msgr2=0x7f2a500feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:04:54.827 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.152+0000 7f2a585b6700 1 -- 172.21.15.92:0/311577161 shutdown_connections 2024-05-06T12:04:54.827 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.152+0000 7f2a585b6700 1 -- 172.21.15.92:0/311577161 wait complete. 2024-05-06T12:04:54.827 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.153+0000 7f2a585b6700 1 Processor -- start 2024-05-06T12:04:54.827 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.153+0000 7f2a585b6700 1 -- start start 2024-05-06T12:04:54.827 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.153+0000 7f2a585b6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a50101d60 0x7f2a50101560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:04:54.827 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.153+0000 7f2a585b6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a501142e0 con 0x7f2a50101d60 2024-05-06T12:04:54.827 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.153+0000 7f2a56352700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a50101d60 0x7f2a50101560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:04:54.827 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.153+0000 7f2a56352700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a50101d60 0x7f2a50101560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57562/0 (socket says 172.21.15.92:57562) 2024-05-06T12:04:54.827 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.153+0000 7f2a56352700 1 -- 172.21.15.92:0/1223991634 learned_addr learned my addr 172.21.15.92:0/1223991634 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:04:54.827 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.153+0000 7f2a56352700 1 -- 172.21.15.92:0/1223991634 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a400094f0 con 0x7f2a50101d60 2024-05-06T12:04:54.827 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.153+0000 7f2a56352700 1 --2- 172.21.15.92:0/1223991634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a50101d60 0x7f2a50101560 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2a40000c00 tx=0x7f2a40009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:04:54.827 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.153+0000 7f2a46ffd700 1 -- 172.21.15.92:0/1223991634 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2a400107c0 con 0x7f2a50101d60 2024-05-06T12:04:54.827 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.154+0000 7f2a46ffd700 1 -- 172.21.15.92:0/1223991634 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f2a40010db0 con 0x7f2a50101d60 2024-05-06T12:04:54.828 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.154+0000 7f2a46ffd700 1 -- 172.21.15.92:0/1223991634 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2a4001ad00 con 0x7f2a50101d60 2024-05-06T12:04:54.828 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.154+0000 7f2a585b6700 1 -- 172.21.15.92:0/1223991634 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a50114420 con 0x7f2a50101d60 2024-05-06T12:04:54.828 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.154+0000 7f2a585b6700 1 -- 172.21.15.92:0/1223991634 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a501148b0 con 0x7f2a50101d60 2024-05-06T12:04:54.828 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.154+0000 7f2a46ffd700 1 -- 172.21.15.92:0/1223991634 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f2a40024440 con 0x7f2a50101d60 2024-05-06T12:04:54.828 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.154+0000 7f2a46ffd700 1 -- 172.21.15.92:0/1223991634 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f2a40019710 con 0x7f2a50101d60 2024-05-06T12:04:54.828 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.155+0000 7f2a585b6700 1 -- 172.21.15.92:0/1223991634 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a5004bed0 con 0x7f2a50101d60 2024-05-06T12:04:54.828 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.157+0000 7f2a46ffd700 1 -- 172.21.15.92:0/1223991634 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f2a4001a530 con 0x7f2a50101d60 2024-05-06T12:04:54.828 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.222+0000 7f2a585b6700 1 -- 172.21.15.92:0/1223991634 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f2a50103660 con 0x7f2a50101d60 2024-05-06T12:04:54.828 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.222+0000 7f2a46ffd700 1 -- 172.21.15.92:0/1223991634 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f2a50103660 con 0x7f2a50101d60 2024-05-06T12:04:54.828 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.223+0000 7f2a585b6700 1 -- 172.21.15.92:0/1223991634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a50101d60 msgr2=0x7f2a50101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:04:54.828 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.223+0000 7f2a585b6700 1 --2- 172.21.15.92:0/1223991634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a50101d60 0x7f2a50101560 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2a40000c00 tx=0x7f2a40009250 comp rx=0 tx=0).stop 2024-05-06T12:04:54.828 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.223+0000 7f2a585b6700 1 -- 172.21.15.92:0/1223991634 shutdown_connections 2024-05-06T12:04:54.828 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.223+0000 7f2a585b6700 1 --2- 172.21.15.92:0/1223991634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a50101d60 0x7f2a50101560 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:04:54.828 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.223+0000 7f2a585b6700 1 -- 172.21.15.92:0/1223991634 >> 172.21.15.92:0/1223991634 conn(0x7f2a500fc6f0 msgr2=0x7f2a500fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:04:54.829 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.223+0000 7f2a585b6700 1 -- 172.21.15.92:0/1223991634 shutdown_connections 2024-05-06T12:04:54.829 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:54.223+0000 7f2a585b6700 1 -- 172.21.15.92:0/1223991634 wait complete. 2024-05-06T12:04:54.829 INFO:teuthology.orchestra.run.smithi092.stdout:Restarting the monitor... 2024-05-06T12:04:55.097 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:54 smithi092 systemd[1]: Stopping Ceph mon.smithi092 for aee1d912-0ba0-11ef-bc97-c7b262605968... 2024-05-06T12:04:55.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:55 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi092[26580]: 2024-05-06T12:04:55.093+0000 7f9048709700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi092 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-06T12:04:55.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:55 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi092[26580]: 2024-05-06T12:04:55.093+0000 7f9048709700 -1 mon.smithi092@0(leader) e1 *** Got Signal Terminated *** 2024-05-06T12:04:55.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:55 smithi092 podman[26831]: 2024-05-06 12:04:55.25827389 +0000 UTC m=+0.275865855 container died da97777082e82a457a250ac526b0c37eac8c59bc36d5cf9394cdfeef41e40e2f (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi092, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=quincy-e81df42, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5) 2024-05-06T12:04:55.863 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:55 smithi092 podman[26831]: 2024-05-06 12:04:55.618727051 +0000 UTC m=+0.636319015 container cleanup da97777082e82a457a250ac526b0c37eac8c59bc36d5cf9394cdfeef41e40e2f (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi092, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=quincy-e81df42, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS) 2024-05-06T12:04:55.863 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:55 smithi092 bash[26831]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi092 2024-05-06T12:04:56.216 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:55 smithi092 podman[26845]: 2024-05-06 12:04:55.861311079 +0000 UTC m=+0.605945706 container remove da97777082e82a457a250ac526b0c37eac8c59bc36d5cf9394cdfeef41e40e2f (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi092, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, RELEASE=quincy-e81df42, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430) 2024-05-06T12:04:56.217 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 systemd[1]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@mon.smithi092.service: Deactivated successfully. 2024-05-06T12:04:56.217 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 systemd[1]: Stopped Ceph mon.smithi092 for aee1d912-0ba0-11ef-bc97-c7b262605968. 2024-05-06T12:04:56.217 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 systemd[1]: Starting Ceph mon.smithi092 for aee1d912-0ba0-11ef-bc97-c7b262605968... 2024-05-06T12:04:56.468 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 podman[26935]: 2024-05-06 12:04:56.117849962 +0000 UTC m=+0.017767985 image pull 96d5ae7fd4b440c05b2389bfd474e9e1f4dd7545543250f9a951bd3ff2478bbf quay.ceph.io/ceph-ci/ceph:quincy 2024-05-06T12:04:56.468 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 podman[26935]: 2024-05-06 12:04:56.22080152 +0000 UTC m=+0.120719535 container create df5b322e971909a283945f048d107c3733ddc2ad3fefc6ae30f7b0fafadef4c6 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi092, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=quincy-e81df42, maintainer=Guillaume Abrioux ) 2024-05-06T12:04:56.469 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 podman[26935]: 2024-05-06 12:04:56.439766039 +0000 UTC m=+0.339684056 container init df5b322e971909a283945f048d107c3733ddc2ad3fefc6ae30f7b0fafadef4c6 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi092, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=quincy-e81df42, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-06T12:04:56.469 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 podman[26935]: 2024-05-06 12:04:56.442657359 +0000 UTC m=+0.342575378 container start df5b322e971909a283945f048d107c3733ddc2ad3fefc6ae30f7b0fafadef4c6 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi092, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, RELEASE=quincy-e81df42, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-06T12:04:56.469 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: set uid:gid to 167:167 (ceph:ceph) 2024-05-06T12:04:56.469 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable), process ceph-mon, pid 2 2024-05-06T12:04:56.469 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: pidfile_write: ignore empty --pid-file 2024-05-06T12:04:56.469 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: load: jerasure load: lrc 2024-05-06T12:04:56.538 INFO:teuthology.orchestra.run.smithi092.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-06T12:04:56.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: RocksDB version: 6.15.5 2024-05-06T12:04:56.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-05-06T12:04:56.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Compile date May 4 2024 2024-05-06T12:04:56.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: DB SUMMARY 2024-05-06T12:04:56.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: DB Session ID: E1GEX9X7DVSXWQ6DYXDC 2024-05-06T12:04:56.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: CURRENT file: CURRENT 2024-05-06T12:04:56.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: IDENTITY file: IDENTITY 2024-05-06T12:04:56.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-05-06T12:04:56.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi092/store.db dir, Total Num: 1, files: 000008.sst 2024-05-06T12:04:56.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi092/store.db: 000010.log size: 98164 ; 2024-05-06T12:04:56.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.error_if_exists: 0 2024-05-06T12:04:56.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.create_if_missing: 0 2024-05-06T12:04:56.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.paranoid_checks: 1 2024-05-06T12:04:56.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-06T12:04:56.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.env: 0x564ff857eb00 2024-05-06T12:04:56.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.fs: Posix File System 2024-05-06T12:04:56.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.info_log: 0x564ff92f7620 2024-05-06T12:04:56.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.max_file_opening_threads: 16 2024-05-06T12:04:56.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.statistics: (nil) 2024-05-06T12:04:56.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.use_fsync: 0 2024-05-06T12:04:56.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.max_log_file_size: 0 2024-05-06T12:04:56.725 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-06T12:04:56.725 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-06T12:04:56.725 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.keep_log_file_num: 1000 2024-05-06T12:04:56.725 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.recycle_log_file_num: 0 2024-05-06T12:04:56.725 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.allow_fallocate: 1 2024-05-06T12:04:56.725 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.allow_mmap_reads: 0 2024-05-06T12:04:56.725 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.allow_mmap_writes: 0 2024-05-06T12:04:56.725 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.use_direct_reads: 0 2024-05-06T12:04:56.725 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-06T12:04:56.725 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.create_missing_column_families: 0 2024-05-06T12:04:56.725 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.db_log_dir: 2024-05-06T12:04:56.725 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi092/store.db 2024-05-06T12:04:56.725 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-06T12:04:56.725 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-06T12:04:56.726 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-06T12:04:56.726 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-06T12:04:56.726 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-06T12:04:56.726 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-06T12:04:56.726 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.advise_random_on_open: 1 2024-05-06T12:04:56.726 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.db_write_buffer_size: 0 2024-05-06T12:04:56.726 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.write_buffer_manager: 0x564ff9feaa20 2024-05-06T12:04:56.726 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-06T12:04:56.726 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-05-06T12:04:56.726 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-06T12:04:56.726 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-06T12:04:56.726 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.rate_limiter: (nil) 2024-05-06T12:04:56.726 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-06T12:04:56.726 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.wal_recovery_mode: 2 2024-05-06T12:04:56.727 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.enable_thread_tracking: 0 2024-05-06T12:04:56.727 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.enable_pipelined_write: 0 2024-05-06T12:04:56.727 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.unordered_write: 0 2024-05-06T12:04:56.727 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-06T12:04:56.727 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-06T12:04:56.727 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-06T12:04:56.727 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-06T12:04:56.727 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.row_cache: None 2024-05-06T12:04:56.727 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.wal_filter: None 2024-05-06T12:04:56.727 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-06T12:04:56.727 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.allow_ingest_behind: 0 2024-05-06T12:04:56.727 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.preserve_deletes: 0 2024-05-06T12:04:56.727 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.two_write_queues: 0 2024-05-06T12:04:56.727 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.manual_wal_flush: 0 2024-05-06T12:04:56.727 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.atomic_flush: 0 2024-05-06T12:04:56.728 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-06T12:04:56.728 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-06T12:04:56.728 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-06T12:04:56.728 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.log_readahead_size: 0 2024-05-06T12:04:56.728 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-06T12:04:56.728 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.best_efforts_recovery: 0 2024-05-06T12:04:56.728 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-06T12:04:56.728 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-06T12:04:56.728 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.allow_data_in_errors: 0 2024-05-06T12:04:56.728 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.db_host_id: __hostname__ 2024-05-06T12:04:56.728 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.max_background_jobs: 2 2024-05-06T12:04:56.728 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.max_background_compactions: -1 2024-05-06T12:04:56.728 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.max_subcompactions: 1 2024-05-06T12:04:56.728 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-06T12:04:56.728 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-06T12:04:56.730 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-06T12:04:56.730 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.max_total_wal_size: 0 2024-05-06T12:04:56.730 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-06T12:04:56.730 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-06T12:04:56.730 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-06T12:04:56.730 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-06T12:04:56.730 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.max_open_files: -1 2024-05-06T12:04:56.730 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.bytes_per_sync: 0 2024-05-06T12:04:56.730 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-06T12:04:56.730 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-06T12:04:56.731 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.compaction_readahead_size: 0 2024-05-06T12:04:56.731 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.max_background_flushes: -1 2024-05-06T12:04:56.731 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Compression algorithms supported: 2024-05-06T12:04:56.731 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-06T12:04:56.731 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: kZSTD supported: 0 2024-05-06T12:04:56.731 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: kXpressCompression supported: 0 2024-05-06T12:04:56.731 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: kLZ4HCCompression supported: 1 2024-05-06T12:04:56.731 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: kZlibCompression supported: 1 2024-05-06T12:04:56.731 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: kSnappyCompression supported: 1 2024-05-06T12:04:56.731 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: kLZ4Compression supported: 1 2024-05-06T12:04:56.731 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: kBZip2Compression supported: 0 2024-05-06T12:04:56.731 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-06T12:04:56.731 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi092/store.db/MANIFEST-000009 2024-05-06T12:04:56.731 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-05-06T12:04:56.731 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-06T12:04:56.732 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.merge_operator: 2024-05-06T12:04:56.732 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.compaction_filter: None 2024-05-06T12:04:56.732 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.compaction_filter_factory: None 2024-05-06T12:04:56.732 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.sst_partitioner_factory: None 2024-05-06T12:04:56.732 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-06T12:04:56.732 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-06T12:04:56.732 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x564ff92caf58) 2024-05-06T12:04:56.732 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: cache_index_and_filter_blocks: 1 2024-05-06T12:04:56.732 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-06T12:04:56.732 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-06T12:04:56.732 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: pin_top_level_index_and_filter: 1 2024-05-06T12:04:56.732 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: index_type: 0 2024-05-06T12:04:56.732 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: data_block_index_type: 0 2024-05-06T12:04:56.732 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: index_shortening: 1 2024-05-06T12:04:56.733 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-06T12:04:56.733 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: hash_index_allow_collision: 1 2024-05-06T12:04:56.733 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: checksum: 1 2024-05-06T12:04:56.733 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: no_block_cache: 0 2024-05-06T12:04:56.733 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: block_cache: 0x564ff92fd090 2024-05-06T12:04:56.733 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: block_cache_name: BinnedLRUCache 2024-05-06T12:04:56.733 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: block_cache_options: 2024-05-06T12:04:56.733 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: capacity : 536870912 2024-05-06T12:04:56.733 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: num_shard_bits : 4 2024-05-06T12:04:56.733 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: strict_capacity_limit : 0 2024-05-06T12:04:56.733 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: high_pri_pool_ratio: 0.000 2024-05-06T12:04:56.733 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: block_cache_compressed: (nil) 2024-05-06T12:04:56.733 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: persistent_cache: (nil) 2024-05-06T12:04:56.733 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: block_size: 4096 2024-05-06T12:04:56.733 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: block_size_deviation: 10 2024-05-06T12:04:56.733 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: block_restart_interval: 16 2024-05-06T12:04:56.734 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: index_block_restart_interval: 1 2024-05-06T12:04:56.734 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: metadata_block_size: 4096 2024-05-06T12:04:56.734 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: partition_filters: 0 2024-05-06T12:04:56.734 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: use_delta_encoding: 1 2024-05-06T12:04:56.734 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-05-06T12:04:56.734 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: whole_key_filtering: 1 2024-05-06T12:04:56.734 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: verify_compression: 0 2024-05-06T12:04:56.734 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: read_amp_bytes_per_bit: 0 2024-05-06T12:04:56.734 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: format_version: 4 2024-05-06T12:04:56.734 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: enable_index_compression: 1 2024-05-06T12:04:56.734 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: block_align: 0 2024-05-06T12:04:56.734 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.write_buffer_size: 33554432 2024-05-06T12:04:56.734 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.max_write_buffer_number: 2 2024-05-06T12:04:56.734 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.compression: NoCompression 2024-05-06T12:04:56.735 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.bottommost_compression: Disabled 2024-05-06T12:04:56.735 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.prefix_extractor: nullptr 2024-05-06T12:04:56.735 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-06T12:04:56.735 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.num_levels: 7 2024-05-06T12:04:56.735 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-06T12:04:56.735 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-06T12:04:56.735 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-06T12:04:56.735 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-06T12:04:56.735 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-06T12:04:56.735 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-06T12:04:56.735 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-06T12:04:56.735 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-06T12:04:56.735 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-06T12:04:56.735 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-06T12:04:56.735 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-06T12:04:56.736 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.compression_opts.level: 32767 2024-05-06T12:04:56.736 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.compression_opts.strategy: 0 2024-05-06T12:04:56.736 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-06T12:04:56.736 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-06T12:04:56.736 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-06T12:04:56.736 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.compression_opts.enabled: false 2024-05-06T12:04:56.736 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-06T12:04:56.736 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-06T12:04:56.736 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-06T12:04:56.736 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.target_file_size_base: 67108864 2024-05-06T12:04:56.736 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-06T12:04:56.736 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-06T12:04:56.736 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-06T12:04:56.736 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-06T12:04:56.736 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-06T12:04:56.737 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-06T12:04:56.737 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-06T12:04:56.737 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-06T12:04:56.737 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-06T12:04:56.737 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-06T12:04:56.737 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-06T12:04:56.737 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-06T12:04:56.737 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-06T12:04:56.737 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.arena_block_size: 4194304 2024-05-06T12:04:56.737 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-06T12:04:56.737 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-06T12:04:56.737 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-05-06T12:04:56.737 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.disable_auto_compactions: 0 2024-05-06T12:04:56.737 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-06T12:04:56.737 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-06T12:04:56.738 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-06T12:04:56.738 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-06T12:04:56.738 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-06T12:04:56.738 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-06T12:04:56.738 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-06T12:04:56.738 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-06T12:04:56.738 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-06T12:04:56.738 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-06T12:04:56.738 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.table_properties_collectors: 2024-05-06T12:04:56.738 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.inplace_update_support: 0 2024-05-06T12:04:56.738 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-06T12:04:56.738 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-06T12:04:56.738 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-06T12:04:56.738 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-06T12:04:56.739 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.bloom_locality: 0 2024-05-06T12:04:56.739 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.max_successive_merges: 0 2024-05-06T12:04:56.739 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-06T12:04:56.739 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.paranoid_file_checks: 0 2024-05-06T12:04:56.739 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.force_consistency_checks: 1 2024-05-06T12:04:56.739 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.report_bg_io_stats: 0 2024-05-06T12:04:56.739 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.ttl: 2592000 2024-05-06T12:04:56.739 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-06T12:04:56.739 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.enable_blob_files: false 2024-05-06T12:04:56.739 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.min_blob_size: 0 2024-05-06T12:04:56.739 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.blob_file_size: 268435456 2024-05-06T12:04:56.739 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-06T12:04:56.739 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-06T12:04:56.739 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-06T12:04:56.739 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi092/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-06T12:04:56.740 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-05-06T12:04:56.740 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-05-06T12:04:56.740 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714997096468622, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-05-06T12:04:56.740 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-05-06T12:04:56.740 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-05-06T12:04:56.740 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714997096469772, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 93370, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 91482, "index_size": 248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13576, "raw_average_key_size": 49, "raw_value_size": 84046, "raw_average_value_size": 307, "num_data_blocks": 11, "num_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1714997096, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "bfe465a0-d978-484c-93f8-8fdf204d8d29", "db_session_id": "E1GEX9X7DVSXWQ6DYXDC"}} 2024-05-06T12:04:56.740 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-05-06T12:04:56.740 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714997096470188, "job": 1, "event": "recovery_finished"} 2024-05-06T12:04:56.740 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi092/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-06T12:04:56.740 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x564ff9291180 2024-05-06T12:04:56.740 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: DB pointer 0x564ff9366000 2024-05-06T12:04:56.740 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-05-06T12:04:56.740 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-05-06T12:04:56.740 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: ** DB Stats ** 2024-05-06T12:04:56.741 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-06T12:04:56.741 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-06T12:04:56.741 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-06T12:04:56.741 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-06T12:04:56.741 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-06T12:04:56.741 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-05-06T12:04:56.741 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-06T12:04:56.741 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: 2024-05-06T12:04:56.741 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: ** Compaction Stats [default] ** 2024-05-06T12:04:56.741 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-05-06T12:04:56.741 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-06T12:04:56.741 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: L0 2/0 92.98 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 118.6 0.00 0.00 1 0.001 0 0 2024-05-06T12:04:56.741 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Sum 2/0 92.98 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 118.6 0.00 0.00 1 0.001 0 0 2024-05-06T12:04:56.741 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 118.6 0.00 0.00 1 0.001 0 0 2024-05-06T12:04:56.741 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: 2024-05-06T12:04:56.741 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: ** Compaction Stats [default] ** 2024-05-06T12:04:56.742 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-05-06T12:04:56.742 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-06T12:04:56.742 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 118.6 0.00 0.00 1 0.001 0 0 2024-05-06T12:04:56.742 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-06T12:04:56.742 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-06T12:04:56.742 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-06T12:04:56.742 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-06T12:04:56.742 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-06T12:04:56.742 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-06T12:04:56.742 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Cumulative compaction: 0.00 GB write, 21.80 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-06T12:04:56.742 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Interval compaction: 0.00 GB write, 21.80 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-06T12:04:56.742 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-06T12:04:56.742 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: 2024-05-06T12:04:56.742 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-06T12:04:56.743 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: 2024-05-06T12:04:56.743 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: ** Compaction Stats [default] ** 2024-05-06T12:04:56.743 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-05-06T12:04:56.743 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-06T12:04:56.743 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: L0 2/0 92.98 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 118.6 0.00 0.00 1 0.001 0 0 2024-05-06T12:04:56.743 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Sum 2/0 92.98 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 118.6 0.00 0.00 1 0.001 0 0 2024-05-06T12:04:56.743 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-05-06T12:04:56.743 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: 2024-05-06T12:04:56.743 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: ** Compaction Stats [default] ** 2024-05-06T12:04:56.743 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-05-06T12:04:56.743 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-06T12:04:56.743 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 118.6 0.00 0.00 1 0.001 0 0 2024-05-06T12:04:56.743 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-06T12:04:56.743 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-06T12:04:56.743 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-06T12:04:56.744 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-06T12:04:56.744 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-06T12:04:56.744 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-06T12:04:56.744 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Cumulative compaction: 0.00 GB write, 21.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-06T12:04:56.744 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-06T12:04:56.744 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-06T12:04:56.744 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: 2024-05-06T12:04:56.744 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-06T12:04:56.744 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: starting mon.smithi092 rank 0 at public addrs [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] at bind addrs [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi092 fsid aee1d912-0ba0-11ef-bc97-c7b262605968 2024-05-06T12:04:56.744 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: mon.smithi092@-1(???) e1 preinit fsid aee1d912-0ba0-11ef-bc97-c7b262605968 2024-05-06T12:04:56.744 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: mon.smithi092@-1(???).mds e1 new map 2024-05-06T12:04:56.744 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: mon.smithi092@-1(???).mds e1 print_map 2024-05-06T12:04:56.744 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: e1 2024-05-06T12:04:56.744 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-06T12:04:56.744 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:04:56.745 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: legacy client fscid: -1 2024-05-06T12:04:56.745 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: 2024-05-06T12:04:56.745 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: No filesystems configured 2024-05-06T12:04:56.745 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: mon.smithi092@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-06T12:04:56.745 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: mon.smithi092@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-06T12:04:56.745 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: mon.smithi092@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-06T12:04:56.745 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: mon.smithi092@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-06T12:04:56.745 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: mon.smithi092@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-06T12:04:56.745 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 bash[26935]: df5b322e971909a283945f048d107c3733ddc2ad3fefc6ae30f7b0fafadef4c6 2024-05-06T12:04:56.745 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: mon.smithi092 is new leader, mons smithi092 in quorum (ranks 0) 2024-05-06T12:04:56.745 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: monmap e1: 1 mons at {smithi092=[v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-06T12:04:56.745 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: fsmap 2024-05-06T12:04:56.745 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: osdmap e1: 0 total, 0 up, 0 in 2024-05-06T12:04:56.745 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 ceph-mon[26954]: mgrmap e1: no daemons active 2024-05-06T12:04:56.745 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:56 smithi092 systemd[1]: Started Ceph mon.smithi092 for aee1d912-0ba0-11ef-bc97-c7b262605968. 2024-05-06T12:04:57.530 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.923+0000 7f8228752700 1 Processor -- start 2024-05-06T12:04:57.530 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.923+0000 7f8228752700 1 -- start start 2024-05-06T12:04:57.530 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.923+0000 7f8228752700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8220076c50 0x7f8220105400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:04:57.530 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.923+0000 7f8228752700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8220105940 con 0x7f8220076c50 2024-05-06T12:04:57.530 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.923+0000 7f82264ee700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8220076c50 0x7f8220105400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:04:57.530 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.924+0000 7f82264ee700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8220076c50 0x7f8220105400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57570/0 (socket says 172.21.15.92:57570) 2024-05-06T12:04:57.530 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.924+0000 7f82264ee700 1 -- 172.21.15.92:0/2895910460 learned_addr learned my addr 172.21.15.92:0/2895910460 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:04:57.530 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.924+0000 7f82264ee700 1 -- 172.21.15.92:0/2895910460 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8220105a80 con 0x7f8220076c50 2024-05-06T12:04:57.530 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.924+0000 7f82264ee700 1 --2- 172.21.15.92:0/2895910460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8220076c50 0x7f8220105400 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f821c0098e0 tx=0x7f821c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6fd86ecc1576277 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:04:57.530 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.924+0000 7f82254ec700 1 -- 172.21.15.92:0/2895910460 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f821c00b860 con 0x7f8220076c50 2024-05-06T12:04:57.531 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.924+0000 7f82254ec700 1 -- 172.21.15.92:0/2895910460 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f821c00be50 con 0x7f8220076c50 2024-05-06T12:04:57.531 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.924+0000 7f82254ec700 1 -- 172.21.15.92:0/2895910460 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f821c010380 con 0x7f8220076c50 2024-05-06T12:04:57.531 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.925+0000 7f8228752700 1 -- 172.21.15.92:0/2895910460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8220076c50 msgr2=0x7f8220105400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:04:57.531 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.925+0000 7f8228752700 1 --2- 172.21.15.92:0/2895910460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8220076c50 0x7f8220105400 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f821c0098e0 tx=0x7f821c009250 comp rx=0 tx=0).stop 2024-05-06T12:04:57.531 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.925+0000 7f8228752700 1 -- 172.21.15.92:0/2895910460 shutdown_connections 2024-05-06T12:04:57.531 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.925+0000 7f8228752700 1 --2- 172.21.15.92:0/2895910460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8220076c50 0x7f8220105400 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:04:57.531 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.925+0000 7f8228752700 1 -- 172.21.15.92:0/2895910460 >> 172.21.15.92:0/2895910460 conn(0x7f82200fcef0 msgr2=0x7f82200ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:04:57.531 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.925+0000 7f8228752700 1 -- 172.21.15.92:0/2895910460 shutdown_connections 2024-05-06T12:04:57.531 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.925+0000 7f8228752700 1 -- 172.21.15.92:0/2895910460 wait complete. 2024-05-06T12:04:57.531 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.925+0000 7f8228752700 1 Processor -- start 2024-05-06T12:04:57.531 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.925+0000 7f8228752700 1 -- start start 2024-05-06T12:04:57.531 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.925+0000 7f8228752700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8220076c50 0x7f8220102d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:04:57.531 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.925+0000 7f8228752700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8220103290 con 0x7f8220076c50 2024-05-06T12:04:57.531 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.926+0000 7f82264ee700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8220076c50 0x7f8220102d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:04:57.532 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.926+0000 7f82264ee700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8220076c50 0x7f8220102d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57576/0 (socket says 172.21.15.92:57576) 2024-05-06T12:04:57.532 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.926+0000 7f82264ee700 1 -- 172.21.15.92:0/1236296931 learned_addr learned my addr 172.21.15.92:0/1236296931 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:04:57.532 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.926+0000 7f82264ee700 1 -- 172.21.15.92:0/1236296931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821c0094f0 con 0x7f8220076c50 2024-05-06T12:04:57.532 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.926+0000 7f82264ee700 1 --2- 172.21.15.92:0/1236296931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8220076c50 0x7f8220102d50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f821c003050 tx=0x7f821c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:04:57.532 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.926+0000 7f8216ffd700 1 -- 172.21.15.92:0/1236296931 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f821c010760 con 0x7f8220076c50 2024-05-06T12:04:57.532 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.926+0000 7f8216ffd700 1 -- 172.21.15.92:0/1236296931 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f821c010d50 con 0x7f8220076c50 2024-05-06T12:04:57.532 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.926+0000 7f8228752700 1 -- 172.21.15.92:0/1236296931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8220101300 con 0x7f8220076c50 2024-05-06T12:04:57.532 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.926+0000 7f8216ffd700 1 -- 172.21.15.92:0/1236296931 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f821c01ae20 con 0x7f8220076c50 2024-05-06T12:04:57.532 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.926+0000 7f8228752700 1 -- 172.21.15.92:0/1236296931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82201017f0 con 0x7f8220076c50 2024-05-06T12:04:57.532 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.926+0000 7f8216ffd700 1 -- 172.21.15.92:0/1236296931 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f821c02e440 con 0x7f8220076c50 2024-05-06T12:04:57.532 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.926+0000 7f8216ffd700 1 -- 172.21.15.92:0/1236296931 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f821c024740 con 0x7f8220076c50 2024-05-06T12:04:57.532 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.927+0000 7f8228752700 1 -- 172.21.15.92:0/1236296931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f822004bed0 con 0x7f8220076c50 2024-05-06T12:04:57.532 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.929+0000 7f8216ffd700 1 -- 172.21.15.92:0/1236296931 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f821c029070 con 0x7f8220076c50 2024-05-06T12:04:57.532 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:56.994+0000 7f8228752700 1 -- 172.21.15.92:0/1236296931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f8220101a30 con 0x7f8220076c50 2024-05-06T12:04:57.533 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:57.000+0000 7f8216ffd700 1 -- 172.21.15.92:0/1236296931 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f821c006d70 con 0x7f8220076c50 2024-05-06T12:04:57.533 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:57.000+0000 7f8228752700 1 -- 172.21.15.92:0/1236296931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8220076c50 msgr2=0x7f8220102d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:04:57.533 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:57.000+0000 7f8228752700 1 --2- 172.21.15.92:0/1236296931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8220076c50 0x7f8220102d50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f821c003050 tx=0x7f821c009250 comp rx=0 tx=0).stop 2024-05-06T12:04:57.533 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:57.000+0000 7f8228752700 1 -- 172.21.15.92:0/1236296931 shutdown_connections 2024-05-06T12:04:57.533 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:57.000+0000 7f8228752700 1 --2- 172.21.15.92:0/1236296931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8220076c50 0x7f8220102d50 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:04:57.533 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:57.000+0000 7f8228752700 1 -- 172.21.15.92:0/1236296931 >> 172.21.15.92:0/1236296931 conn(0x7f82200fcef0 msgr2=0x7f82200fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:04:57.533 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:57.001+0000 7f8228752700 1 -- 172.21.15.92:0/1236296931 shutdown_connections 2024-05-06T12:04:57.533 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:57.001+0000 7f8228752700 1 -- 172.21.15.92:0/1236296931 wait complete. 2024-05-06T12:04:57.533 INFO:teuthology.orchestra.run.smithi092.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-06T12:04:57.533 INFO:teuthology.orchestra.run.smithi092.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-06T12:04:57.533 INFO:teuthology.orchestra.run.smithi092.stdout:Creating mgr... 2024-05-06T12:04:57.533 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying port 9283 ... 2024-05-06T12:04:57.700 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@mgr.smithi092.gyzeqb 2024-05-06T12:04:57.700 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Failed to reset failed state of unit ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@mgr.smithi092.gyzeqb.service: Unit ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@mgr.smithi092.gyzeqb.service not loaded. 2024-05-06T12:04:57.849 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-aee1d912-0ba0-11ef-bc97-c7b262605968.target.wants/ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@mgr.smithi092.gyzeqb.service → /etc/systemd/system/ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@.service. 2024-05-06T12:04:58.293 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:58 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/1236296931' entity='client.admin' 2024-05-06T12:04:58.406 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-06T12:04:58.406 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout disabled 2024-05-06T12:04:58.414 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-06T12:04:58.414 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout inactive 2024-05-06T12:04:58.414 INFO:teuthology.orchestra.run.smithi092.stdout:firewalld.service is not enabled 2024-05-06T12:04:58.414 INFO:teuthology.orchestra.run.smithi092.stdout:Not possible to enable service . firewalld.service is not available 2024-05-06T12:04:58.423 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-06T12:04:58.423 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout disabled 2024-05-06T12:04:58.432 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-06T12:04:58.432 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout inactive 2024-05-06T12:04:58.432 INFO:teuthology.orchestra.run.smithi092.stdout:firewalld.service is not enabled 2024-05-06T12:04:58.432 INFO:teuthology.orchestra.run.smithi092.stdout:Not possible to open ports <[9283]>. firewalld.service is not available 2024-05-06T12:04:58.433 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for mgr to start... 2024-05-06T12:04:58.433 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for mgr... 2024-05-06T12:04:59.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:04:59 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/3565964097' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-06T12:04:59.579 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-05-06T12:04:59.579 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-05-06T12:04:59.579 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "fsid": "aee1d912-0ba0-11ef-bc97-c7b262605968", 2024-05-06T12:04:59.579 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "health": { 2024-05-06T12:04:59.579 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-06T12:04:59.579 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-06T12:04:59.579 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-06T12:04:59.579 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-05-06T12:04:59.580 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-06T12:04:59.580 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-06T12:04:59.580 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 0 2024-05-06T12:04:59.580 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-05-06T12:04:59.580 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-06T12:04:59.580 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "smithi092" 2024-05-06T12:04:59.580 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-05-06T12:04:59.580 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-06T12:04:59.580 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-06T12:04:59.580 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-06T12:04:59.580 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-05-06T12:04:59.580 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-06T12:04:59.580 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-05-06T12:04:59.580 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-06T12:04:59.580 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-06T12:04:59.581 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-06T12:04:59.581 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-06T12:04:59.581 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-06T12:04:59.589 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-06T12:04:59.589 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-06T12:04:59.589 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-06T12:04:59.589 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-05-06T12:04:59.589 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-06T12:04:59.589 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-06T12:04:59.589 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-06T12:04:59.589 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-06T12:04:59.589 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-06T12:04:59.590 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-06T12:04:59.590 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-06T12:04:59.590 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-06T12:04:59.590 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-06T12:04:59.590 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-05-06T12:04:59.590 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-06T12:04:59.590 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-06T12:04:59.590 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-06T12:04:59.590 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-06T12:04:59.590 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-05-06T12:04:59.590 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-06T12:04:59.590 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-06T12:04:59.590 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-06T12:04:59.590 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-06T12:04:59.591 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-06T12:04:59.591 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-06T12:04:59.591 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "restful" 2024-05-06T12:04:59.591 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-05-06T12:04:59.591 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-06T12:04:59.591 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-05-06T12:04:59.591 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-06T12:04:59.591 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-06T12:04:59.591 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "modified": "2024-05-06T12:04:51.221370+0000", 2024-05-06T12:04:59.591 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-06T12:04:59.591 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-05-06T12:04:59.591 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-06T12:04:59.591 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-05-06T12:04:59.591 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.889+0000 7f43c8ba3700 1 Processor -- start 2024-05-06T12:04:59.592 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.889+0000 7f43c8ba3700 1 -- start start 2024-05-06T12:04:59.592 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.889+0000 7f43c8ba3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43c4102870 0x7f43c4102c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:04:59.592 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.889+0000 7f43c8ba3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c41031c0 con 0x7f43c4102870 2024-05-06T12:04:59.592 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.889+0000 7f43c259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43c4102870 0x7f43c4102c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:04:59.592 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.889+0000 7f43c259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43c4102870 0x7f43c4102c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57590/0 (socket says 172.21.15.92:57590) 2024-05-06T12:04:59.592 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.889+0000 7f43c259c700 1 -- 172.21.15.92:0/1923705332 learned_addr learned my addr 172.21.15.92:0/1923705332 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:04:59.592 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.890+0000 7f43c259c700 1 -- 172.21.15.92:0/1923705332 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c4103300 con 0x7f43c4102870 2024-05-06T12:04:59.592 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.890+0000 7f43c259c700 1 --2- 172.21.15.92:0/1923705332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43c4102870 0x7f43c4102c80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f43b40098e0 tx=0x7f43b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49fbd5a0c11e3461 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:04:59.592 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.890+0000 7f43c159a700 1 -- 172.21.15.92:0/1923705332 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f43b400b860 con 0x7f43c4102870 2024-05-06T12:04:59.592 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.890+0000 7f43c159a700 1 -- 172.21.15.92:0/1923705332 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f43b400be50 con 0x7f43c4102870 2024-05-06T12:04:59.592 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.890+0000 7f43c159a700 1 -- 172.21.15.92:0/1923705332 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f43b4010380 con 0x7f43c4102870 2024-05-06T12:04:59.592 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.891+0000 7f43c8ba3700 1 -- 172.21.15.92:0/1923705332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43c4102870 msgr2=0x7f43c4102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:04:59.592 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.891+0000 7f43c8ba3700 1 --2- 172.21.15.92:0/1923705332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43c4102870 0x7f43c4102c80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f43b40098e0 tx=0x7f43b4009250 comp rx=0 tx=0).stop 2024-05-06T12:04:59.593 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.891+0000 7f43c8ba3700 1 -- 172.21.15.92:0/1923705332 shutdown_connections 2024-05-06T12:04:59.593 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.891+0000 7f43c8ba3700 1 --2- 172.21.15.92:0/1923705332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43c4102870 0x7f43c4102c80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:04:59.593 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.891+0000 7f43c8ba3700 1 -- 172.21.15.92:0/1923705332 >> 172.21.15.92:0/1923705332 conn(0x7f43c40fe030 msgr2=0x7f43c4100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:04:59.593 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.891+0000 7f43c8ba3700 1 -- 172.21.15.92:0/1923705332 shutdown_connections 2024-05-06T12:04:59.593 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.891+0000 7f43c8ba3700 1 -- 172.21.15.92:0/1923705332 wait complete. 2024-05-06T12:04:59.593 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.891+0000 7f43c8ba3700 1 Processor -- start 2024-05-06T12:04:59.593 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.891+0000 7f43c8ba3700 1 -- start start 2024-05-06T12:04:59.593 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.891+0000 7f43c8ba3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43c4102870 0x7f43c4111590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:04:59.593 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.891+0000 7f43c8ba3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c4111ad0 con 0x7f43c4102870 2024-05-06T12:04:59.593 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.892+0000 7f43c259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43c4102870 0x7f43c4111590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:04:59.593 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.892+0000 7f43c259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43c4102870 0x7f43c4111590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57602/0 (socket says 172.21.15.92:57602) 2024-05-06T12:04:59.593 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.892+0000 7f43c259c700 1 -- 172.21.15.92:0/3565964097 learned_addr learned my addr 172.21.15.92:0/3565964097 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:04:59.593 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.892+0000 7f43c259c700 1 -- 172.21.15.92:0/3565964097 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b40094f0 con 0x7f43c4102870 2024-05-06T12:04:59.593 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.892+0000 7f43c259c700 1 --2- 172.21.15.92:0/3565964097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43c4102870 0x7f43c4111590 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f43b4000c00 tx=0x7f43b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:04:59.594 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.892+0000 7f43b2ffd700 1 -- 172.21.15.92:0/3565964097 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f43b40107c0 con 0x7f43c4102870 2024-05-06T12:04:59.594 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.892+0000 7f43b2ffd700 1 -- 172.21.15.92:0/3565964097 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f43b4010db0 con 0x7f43c4102870 2024-05-06T12:04:59.594 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.892+0000 7f43c8ba3700 1 -- 172.21.15.92:0/3565964097 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c4111cd0 con 0x7f43c4102870 2024-05-06T12:04:59.594 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.892+0000 7f43b2ffd700 1 -- 172.21.15.92:0/3565964097 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f43b401ad00 con 0x7f43c4102870 2024-05-06T12:04:59.594 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.892+0000 7f43c8ba3700 1 -- 172.21.15.92:0/3565964097 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c4112840 con 0x7f43c4102870 2024-05-06T12:04:59.594 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.893+0000 7f43b2ffd700 1 -- 172.21.15.92:0/3565964097 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f43b4024440 con 0x7f43c4102870 2024-05-06T12:04:59.594 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.893+0000 7f43b2ffd700 1 -- 172.21.15.92:0/3565964097 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f43b4019710 con 0x7f43c4102870 2024-05-06T12:04:59.594 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.893+0000 7f43c8ba3700 1 -- 172.21.15.92:0/3565964097 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c405f5c0 con 0x7f43c4102870 2024-05-06T12:04:59.594 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.895+0000 7f43b2ffd700 1 -- 172.21.15.92:0/3565964097 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f43b4042790 con 0x7f43c4102870 2024-05-06T12:04:59.594 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.965+0000 7f43c8ba3700 1 -- 172.21.15.92:0/3565964097 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f43c404bed0 con 0x7f43c4102870 2024-05-06T12:04:59.594 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.966+0000 7f43b2ffd700 1 -- 172.21.15.92:0/3565964097 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f43b4031020 con 0x7f43c4102870 2024-05-06T12:04:59.594 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.967+0000 7f43c8ba3700 1 -- 172.21.15.92:0/3565964097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43c4102870 msgr2=0x7f43c4111590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:04:59.594 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.967+0000 7f43c8ba3700 1 --2- 172.21.15.92:0/3565964097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43c4102870 0x7f43c4111590 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f43b4000c00 tx=0x7f43b4009250 comp rx=0 tx=0).stop 2024-05-06T12:04:59.595 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.967+0000 7f43c8ba3700 1 -- 172.21.15.92:0/3565964097 shutdown_connections 2024-05-06T12:04:59.595 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.967+0000 7f43c8ba3700 1 --2- 172.21.15.92:0/3565964097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43c4102870 0x7f43c4111590 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:04:59.595 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.967+0000 7f43c8ba3700 1 -- 172.21.15.92:0/3565964097 >> 172.21.15.92:0/3565964097 conn(0x7f43c40fe030 msgr2=0x7f43c40feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:04:59.595 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.967+0000 7f43c8ba3700 1 -- 172.21.15.92:0/3565964097 shutdown_connections 2024-05-06T12:04:59.595 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:04:58.967+0000 7f43c8ba3700 1 -- 172.21.15.92:0/3565964097 wait complete. 2024-05-06T12:04:59.595 INFO:teuthology.orchestra.run.smithi092.stdout:mgr not available, waiting (1/15)... 2024-05-06T12:05:02.268 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:02 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/3282687548' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-06T12:05:02.639 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-05-06T12:05:02.640 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-05-06T12:05:02.640 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "fsid": "aee1d912-0ba0-11ef-bc97-c7b262605968", 2024-05-06T12:05:02.640 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "health": { 2024-05-06T12:05:02.640 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-06T12:05:02.640 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-06T12:05:02.640 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-06T12:05:02.640 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-05-06T12:05:02.640 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-06T12:05:02.640 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-06T12:05:02.640 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 0 2024-05-06T12:05:02.640 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-05-06T12:05:02.641 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-06T12:05:02.641 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "smithi092" 2024-05-06T12:05:02.641 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-05-06T12:05:02.641 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-05-06T12:05:02.641 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-06T12:05:02.641 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-06T12:05:02.641 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-05-06T12:05:02.641 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-06T12:05:02.641 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-05-06T12:05:02.642 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-06T12:05:02.642 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-06T12:05:02.642 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-06T12:05:02.642 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-06T12:05:02.642 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-06T12:05:02.642 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-06T12:05:02.642 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-06T12:05:02.642 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-06T12:05:02.642 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-05-06T12:05:02.643 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-06T12:05:02.643 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-06T12:05:02.643 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-06T12:05:02.643 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-06T12:05:02.643 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-06T12:05:02.643 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-06T12:05:02.643 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-06T12:05:02.643 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-06T12:05:02.643 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-06T12:05:02.644 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-05-06T12:05:02.644 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-06T12:05:02.644 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-06T12:05:02.644 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-06T12:05:02.644 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-06T12:05:02.644 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-05-06T12:05:02.644 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-06T12:05:02.644 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-06T12:05:02.644 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-06T12:05:02.644 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-06T12:05:02.644 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-06T12:05:02.644 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-06T12:05:02.644 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "restful" 2024-05-06T12:05:02.645 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-05-06T12:05:02.645 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-06T12:05:02.645 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-05-06T12:05:02.645 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-06T12:05:02.645 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-06T12:05:02.645 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "modified": "2024-05-06T12:04:51.221370+0000", 2024-05-06T12:05:02.645 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-06T12:05:02.645 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-05-06T12:05:02.645 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-06T12:05:02.645 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-05-06T12:05:02.645 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:01.997+0000 7f63a19c5700 1 Processor -- start 2024-05-06T12:05:02.645 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:01.997+0000 7f63a19c5700 1 -- start start 2024-05-06T12:05:02.645 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:01.997+0000 7f63a19c5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f639c102880 0x7f639c102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:02.645 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:01.997+0000 7f63a19c5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c1031d0 con 0x7f639c102880 2024-05-06T12:05:02.646 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:01.997+0000 7f639affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f639c102880 0x7f639c102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:02.646 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:01.997+0000 7f639affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f639c102880 0x7f639c102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57618/0 (socket says 172.21.15.92:57618) 2024-05-06T12:05:02.646 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:01.997+0000 7f639affd700 1 -- 172.21.15.92:0/2213117449 learned_addr learned my addr 172.21.15.92:0/2213117449 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:02.646 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:01.997+0000 7f639affd700 1 -- 172.21.15.92:0/2213117449 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f639c103310 con 0x7f639c102880 2024-05-06T12:05:02.646 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:01.997+0000 7f639affd700 1 --2- 172.21.15.92:0/2213117449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f639c102880 0x7f639c102c90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f638c0098e0 tx=0x7f638c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e992e063afaf67 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:02.646 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:01.998+0000 7f6399ffb700 1 -- 172.21.15.92:0/2213117449 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f638c00b860 con 0x7f639c102880 2024-05-06T12:05:02.646 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:01.998+0000 7f6399ffb700 1 -- 172.21.15.92:0/2213117449 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f638c00be50 con 0x7f639c102880 2024-05-06T12:05:02.646 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:01.998+0000 7f6399ffb700 1 -- 172.21.15.92:0/2213117449 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f638c010380 con 0x7f639c102880 2024-05-06T12:05:02.646 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:01.998+0000 7f63a19c5700 1 -- 172.21.15.92:0/2213117449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f639c102880 msgr2=0x7f639c102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:02.646 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:01.998+0000 7f63a19c5700 1 --2- 172.21.15.92:0/2213117449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f639c102880 0x7f639c102c90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f638c0098e0 tx=0x7f638c009250 comp rx=0 tx=0).stop 2024-05-06T12:05:02.646 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:01.998+0000 7f63a19c5700 1 -- 172.21.15.92:0/2213117449 shutdown_connections 2024-05-06T12:05:02.646 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:01.998+0000 7f63a19c5700 1 --2- 172.21.15.92:0/2213117449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f639c102880 0x7f639c102c90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:02.646 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:01.998+0000 7f63a19c5700 1 -- 172.21.15.92:0/2213117449 >> 172.21.15.92:0/2213117449 conn(0x7f639c0fe060 msgr2=0x7f639c100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:02.647 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:01.998+0000 7f63a19c5700 1 -- 172.21.15.92:0/2213117449 shutdown_connections 2024-05-06T12:05:02.647 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:01.998+0000 7f63a19c5700 1 -- 172.21.15.92:0/2213117449 wait complete. 2024-05-06T12:05:02.647 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:01.999+0000 7f63a19c5700 1 Processor -- start 2024-05-06T12:05:02.647 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:01.999+0000 7f63a19c5700 1 -- start start 2024-05-06T12:05:02.647 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:01.999+0000 7f63a19c5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f639c102880 0x7f639c1115f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:02.647 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:01.999+0000 7f63a19c5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c111b30 con 0x7f639c102880 2024-05-06T12:05:02.647 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:01.999+0000 7f639affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f639c102880 0x7f639c1115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:02.647 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:01.999+0000 7f639affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f639c102880 0x7f639c1115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57630/0 (socket says 172.21.15.92:57630) 2024-05-06T12:05:02.647 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:01.999+0000 7f639affd700 1 -- 172.21.15.92:0/3282687548 learned_addr learned my addr 172.21.15.92:0/3282687548 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:02.647 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:01.999+0000 7f639affd700 1 -- 172.21.15.92:0/3282687548 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f638c0094f0 con 0x7f639c102880 2024-05-06T12:05:02.647 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:02.000+0000 7f639affd700 1 --2- 172.21.15.92:0/3282687548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f639c102880 0x7f639c1115f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f638c00b440 tx=0x7f638c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:02.647 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:02.000+0000 7f63837fe700 1 -- 172.21.15.92:0/3282687548 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f638c010760 con 0x7f639c102880 2024-05-06T12:05:02.648 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:02.000+0000 7f63837fe700 1 -- 172.21.15.92:0/3282687548 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f638c010d50 con 0x7f639c102880 2024-05-06T12:05:02.648 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:02.000+0000 7f63a19c5700 1 -- 172.21.15.92:0/3282687548 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f639c111d30 con 0x7f639c102880 2024-05-06T12:05:02.648 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:02.000+0000 7f63a19c5700 1 -- 172.21.15.92:0/3282687548 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f639c1128a0 con 0x7f639c102880 2024-05-06T12:05:02.648 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:02.000+0000 7f63837fe700 1 -- 172.21.15.92:0/3282687548 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f638c01ae20 con 0x7f639c102880 2024-05-06T12:05:02.648 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:02.000+0000 7f63837fe700 1 -- 172.21.15.92:0/3282687548 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f638c02e440 con 0x7f639c102880 2024-05-06T12:05:02.648 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:02.000+0000 7f63837fe700 1 -- 172.21.15.92:0/3282687548 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f638c024760 con 0x7f639c102880 2024-05-06T12:05:02.648 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:02.001+0000 7f63a19c5700 1 -- 172.21.15.92:0/3282687548 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f639c05f5c0 con 0x7f639c102880 2024-05-06T12:05:02.648 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:02.003+0000 7f63837fe700 1 -- 172.21.15.92:0/3282687548 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f638c029070 con 0x7f639c102880 2024-05-06T12:05:02.648 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:02.075+0000 7f63a19c5700 1 -- 172.21.15.92:0/3282687548 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f639c104650 con 0x7f639c102880 2024-05-06T12:05:02.649 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:02.075+0000 7f63837fe700 1 -- 172.21.15.92:0/3282687548 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f638c01a6a0 con 0x7f639c102880 2024-05-06T12:05:02.649 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:02.076+0000 7f63a19c5700 1 -- 172.21.15.92:0/3282687548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f639c102880 msgr2=0x7f639c1115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:02.649 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:02.076+0000 7f63a19c5700 1 --2- 172.21.15.92:0/3282687548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f639c102880 0x7f639c1115f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f638c00b440 tx=0x7f638c009250 comp rx=0 tx=0).stop 2024-05-06T12:05:02.649 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:02.076+0000 7f63a19c5700 1 -- 172.21.15.92:0/3282687548 shutdown_connections 2024-05-06T12:05:02.649 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:02.076+0000 7f63a19c5700 1 --2- 172.21.15.92:0/3282687548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f639c102880 0x7f639c1115f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:02.649 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:02.076+0000 7f63a19c5700 1 -- 172.21.15.92:0/3282687548 >> 172.21.15.92:0/3282687548 conn(0x7f639c0fe060 msgr2=0x7f639c0feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:02.649 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:02.076+0000 7f63a19c5700 1 -- 172.21.15.92:0/3282687548 shutdown_connections 2024-05-06T12:05:02.649 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:02.076+0000 7f63a19c5700 1 -- 172.21.15.92:0/3282687548 wait complete. 2024-05-06T12:05:02.649 INFO:teuthology.orchestra.run.smithi092.stdout:mgr not available, waiting (2/15)... 2024-05-06T12:05:04.865 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:04 smithi092 ceph-mon[26954]: Activating manager daemon smithi092.gyzeqb 2024-05-06T12:05:04.865 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:04 smithi092 ceph-mon[26954]: mgrmap e2: smithi092.gyzeqb(active, starting, since 0.0028137s) 2024-05-06T12:05:04.865 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:04 smithi092 ceph-mon[26954]: from='mgr.14100 172.21.15.92:0/1332524907' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-06T12:05:04.866 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:04 smithi092 ceph-mon[26954]: from='mgr.14100 172.21.15.92:0/1332524907' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-06T12:05:04.866 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:04 smithi092 ceph-mon[26954]: from='mgr.14100 172.21.15.92:0/1332524907' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-06T12:05:04.866 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:04 smithi092 ceph-mon[26954]: from='mgr.14100 172.21.15.92:0/1332524907' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi092"}]: dispatch 2024-05-06T12:05:04.866 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:04 smithi092 ceph-mon[26954]: from='mgr.14100 172.21.15.92:0/1332524907' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mgr metadata", "who": "smithi092.gyzeqb", "id": "smithi092.gyzeqb"}]: dispatch 2024-05-06T12:05:04.866 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:04 smithi092 ceph-mon[26954]: Manager daemon smithi092.gyzeqb is now available 2024-05-06T12:05:04.866 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:04 smithi092 ceph-mon[26954]: from='mgr.14100 172.21.15.92:0/1332524907' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi092.gyzeqb/mirror_snapshot_schedule"}]: dispatch 2024-05-06T12:05:04.866 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:04 smithi092 ceph-mon[26954]: from='mgr.14100 172.21.15.92:0/1332524907' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi092.gyzeqb/trash_purge_schedule"}]: dispatch 2024-05-06T12:05:04.866 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:04 smithi092 ceph-mon[26954]: from='mgr.14100 172.21.15.92:0/1332524907' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:05:04.866 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:04 smithi092 ceph-mon[26954]: from='mgr.14100 172.21.15.92:0/1332524907' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:05:04.866 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:04 smithi092 ceph-mon[26954]: from='mgr.14100 172.21.15.92:0/1332524907' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:05:05.723 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-05-06T12:05:05.724 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-05-06T12:05:05.724 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "fsid": "aee1d912-0ba0-11ef-bc97-c7b262605968", 2024-05-06T12:05:05.724 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "health": { 2024-05-06T12:05:05.724 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-06T12:05:05.724 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-06T12:05:05.724 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-06T12:05:05.724 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-05-06T12:05:05.724 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-06T12:05:05.724 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-06T12:05:05.724 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 0 2024-05-06T12:05:05.724 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-05-06T12:05:05.724 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-06T12:05:05.724 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "smithi092" 2024-05-06T12:05:05.725 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-05-06T12:05:05.725 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-05-06T12:05:05.725 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-06T12:05:05.725 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-06T12:05:05.725 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-05-06T12:05:05.725 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-06T12:05:05.725 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-05-06T12:05:05.725 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-06T12:05:05.725 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-06T12:05:05.725 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-06T12:05:05.725 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-06T12:05:05.725 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-06T12:05:05.733 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-06T12:05:05.733 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-06T12:05:05.733 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-06T12:05:05.733 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-05-06T12:05:05.733 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-06T12:05:05.733 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-06T12:05:05.733 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-06T12:05:05.733 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-06T12:05:05.733 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-06T12:05:05.734 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-06T12:05:05.734 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-06T12:05:05.734 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-06T12:05:05.734 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-06T12:05:05.734 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-05-06T12:05:05.734 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-06T12:05:05.734 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-06T12:05:05.734 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-06T12:05:05.734 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-06T12:05:05.734 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-05-06T12:05:05.734 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-06T12:05:05.734 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-06T12:05:05.734 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-06T12:05:05.734 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-06T12:05:05.734 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-06T12:05:05.735 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-06T12:05:05.735 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "restful" 2024-05-06T12:05:05.735 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-05-06T12:05:05.735 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-06T12:05:05.735 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-05-06T12:05:05.735 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-06T12:05:05.735 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-06T12:05:05.735 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "modified": "2024-05-06T12:04:51.221370+0000", 2024-05-06T12:05:05.735 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-06T12:05:05.735 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-05-06T12:05:05.735 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-06T12:05:05.735 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-05-06T12:05:05.735 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.048+0000 7f0db5e77700 1 Processor -- start 2024-05-06T12:05:05.735 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.048+0000 7f0db5e77700 1 -- start start 2024-05-06T12:05:05.736 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.048+0000 7f0db5e77700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0db0102880 0x7f0db0102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:05.736 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.048+0000 7f0db5e77700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db01031d0 con 0x7f0db0102880 2024-05-06T12:05:05.736 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.048+0000 7f0daf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0db0102880 0x7f0db0102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:05.736 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.048+0000 7f0daf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0db0102880 0x7f0db0102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37674/0 (socket says 172.21.15.92:37674) 2024-05-06T12:05:05.736 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.048+0000 7f0daf7fe700 1 -- 172.21.15.92:0/1850577491 learned_addr learned my addr 172.21.15.92:0/1850577491 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:05.736 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.048+0000 7f0daf7fe700 1 -- 172.21.15.92:0/1850577491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db0103310 con 0x7f0db0102880 2024-05-06T12:05:05.736 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.048+0000 7f0daf7fe700 1 --2- 172.21.15.92:0/1850577491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0db0102880 0x7f0db0102c90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0da80098e0 tx=0x7f0da8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83fccb1d0304e050 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:05.736 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.049+0000 7f0dae7fc700 1 -- 172.21.15.92:0/1850577491 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0da800b860 con 0x7f0db0102880 2024-05-06T12:05:05.736 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.049+0000 7f0dae7fc700 1 -- 172.21.15.92:0/1850577491 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f0da800be50 con 0x7f0db0102880 2024-05-06T12:05:05.736 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.049+0000 7f0dae7fc700 1 -- 172.21.15.92:0/1850577491 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0da8010380 con 0x7f0db0102880 2024-05-06T12:05:05.736 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.049+0000 7f0db5e77700 1 -- 172.21.15.92:0/1850577491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0db0102880 msgr2=0x7f0db0102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:05.736 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.049+0000 7f0db5e77700 1 --2- 172.21.15.92:0/1850577491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0db0102880 0x7f0db0102c90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0da80098e0 tx=0x7f0da8009250 comp rx=0 tx=0).stop 2024-05-06T12:05:05.736 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.049+0000 7f0db5e77700 1 -- 172.21.15.92:0/1850577491 shutdown_connections 2024-05-06T12:05:05.736 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.049+0000 7f0db5e77700 1 --2- 172.21.15.92:0/1850577491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0db0102880 0x7f0db0102c90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:05.736 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.049+0000 7f0db5e77700 1 -- 172.21.15.92:0/1850577491 >> 172.21.15.92:0/1850577491 conn(0x7f0db00fe060 msgr2=0x7f0db0100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:05.737 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.049+0000 7f0db5e77700 1 -- 172.21.15.92:0/1850577491 shutdown_connections 2024-05-06T12:05:05.737 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.049+0000 7f0db5e77700 1 -- 172.21.15.92:0/1850577491 wait complete. 2024-05-06T12:05:05.737 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.050+0000 7f0db5e77700 1 Processor -- start 2024-05-06T12:05:05.737 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.050+0000 7f0db5e77700 1 -- start start 2024-05-06T12:05:05.737 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.050+0000 7f0db5e77700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0db0102880 0x7f0db01115f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:05.737 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.050+0000 7f0db5e77700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db0111b30 con 0x7f0db0102880 2024-05-06T12:05:05.737 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.050+0000 7f0daf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0db0102880 0x7f0db01115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:05.737 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.050+0000 7f0daf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0db0102880 0x7f0db01115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37678/0 (socket says 172.21.15.92:37678) 2024-05-06T12:05:05.737 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.050+0000 7f0daf7fe700 1 -- 172.21.15.92:0/2291612521 learned_addr learned my addr 172.21.15.92:0/2291612521 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:05.737 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.050+0000 7f0daf7fe700 1 -- 172.21.15.92:0/2291612521 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da80094f0 con 0x7f0db0102880 2024-05-06T12:05:05.737 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.051+0000 7f0daf7fe700 1 --2- 172.21.15.92:0/2291612521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0db0102880 0x7f0db01115f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0da8000c00 tx=0x7f0da8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:05.737 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.051+0000 7f0da7fff700 1 -- 172.21.15.92:0/2291612521 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0da80107c0 con 0x7f0db0102880 2024-05-06T12:05:05.737 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.051+0000 7f0da7fff700 1 -- 172.21.15.92:0/2291612521 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f0da8010db0 con 0x7f0db0102880 2024-05-06T12:05:05.737 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.051+0000 7f0db5e77700 1 -- 172.21.15.92:0/2291612521 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db0111d30 con 0x7f0db0102880 2024-05-06T12:05:05.738 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.051+0000 7f0da7fff700 1 -- 172.21.15.92:0/2291612521 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0da801ad00 con 0x7f0db0102880 2024-05-06T12:05:05.738 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.051+0000 7f0db5e77700 1 -- 172.21.15.92:0/2291612521 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0db01128a0 con 0x7f0db0102880 2024-05-06T12:05:05.738 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.052+0000 7f0da7fff700 1 -- 172.21.15.92:0/2291612521 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 2) v1 ==== 43132+0+0 (secure 0 0 0) 0x7f0da8010920 con 0x7f0db0102880 2024-05-06T12:05:05.738 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.052+0000 7f0da7fff700 1 -- 172.21.15.92:0/2291612521 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f0da804a240 con 0x7f0db0102880 2024-05-06T12:05:05.738 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.052+0000 7f0db5e77700 1 -- 172.21.15.92:0/2291612521 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0db005f5c0 con 0x7f0db0102880 2024-05-06T12:05:05.738 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.054+0000 7f0da7fff700 1 -- 172.21.15.92:0/2291612521 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f0da8010bd0 con 0x7f0db0102880 2024-05-06T12:05:05.738 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.122+0000 7f0db5e77700 1 -- 172.21.15.92:0/2291612521 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f0db0104650 con 0x7f0db0102880 2024-05-06T12:05:05.738 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.122+0000 7f0da7fff700 1 -- 172.21.15.92:0/2291612521 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f0da8048030 con 0x7f0db0102880 2024-05-06T12:05:05.738 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.123+0000 7f0db5e77700 1 -- 172.21.15.92:0/2291612521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0db0102880 msgr2=0x7f0db01115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:05.738 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.123+0000 7f0db5e77700 1 --2- 172.21.15.92:0/2291612521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0db0102880 0x7f0db01115f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0da8000c00 tx=0x7f0da8009250 comp rx=0 tx=0).stop 2024-05-06T12:05:05.738 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.123+0000 7f0db5e77700 1 -- 172.21.15.92:0/2291612521 shutdown_connections 2024-05-06T12:05:05.738 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.123+0000 7f0db5e77700 1 --2- 172.21.15.92:0/2291612521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0db0102880 0x7f0db01115f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:05.738 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.123+0000 7f0db5e77700 1 -- 172.21.15.92:0/2291612521 >> 172.21.15.92:0/2291612521 conn(0x7f0db00fe060 msgr2=0x7f0db00feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:05.738 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.123+0000 7f0db5e77700 1 -- 172.21.15.92:0/2291612521 shutdown_connections 2024-05-06T12:05:05.738 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:05.123+0000 7f0db5e77700 1 -- 172.21.15.92:0/2291612521 wait complete. 2024-05-06T12:05:05.739 INFO:teuthology.orchestra.run.smithi092.stdout:mgr not available, waiting (3/15)... 2024-05-06T12:05:05.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:05 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/2291612521' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-06T12:05:05.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:05 smithi092 ceph-mon[26954]: mgrmap e3: smithi092.gyzeqb(active, since 1.00667s) 2024-05-06T12:05:08.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:07 smithi092 ceph-mon[26954]: mgrmap e4: smithi092.gyzeqb(active, since 2s) 2024-05-06T12:05:11.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:11 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/1803985967' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-06T12:05:16.744 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-05-06T12:05:16.744 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-05-06T12:05:16.744 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "fsid": "aee1d912-0ba0-11ef-bc97-c7b262605968", 2024-05-06T12:05:16.744 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "health": { 2024-05-06T12:05:16.744 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-06T12:05:16.744 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-06T12:05:16.744 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-06T12:05:16.744 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-05-06T12:05:16.745 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-06T12:05:16.745 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-06T12:05:16.745 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 0 2024-05-06T12:05:16.745 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-05-06T12:05:16.745 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-06T12:05:16.745 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "smithi092" 2024-05-06T12:05:16.745 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-05-06T12:05:16.745 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-05-06T12:05:16.745 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-06T12:05:16.745 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-06T12:05:16.745 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-05-06T12:05:16.745 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-06T12:05:16.745 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-05-06T12:05:16.745 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-06T12:05:16.745 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-06T12:05:16.746 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-06T12:05:16.746 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-06T12:05:16.746 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-06T12:05:16.755 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-06T12:05:16.755 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-06T12:05:16.755 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-06T12:05:16.755 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-05-06T12:05:16.755 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-06T12:05:16.755 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-06T12:05:16.755 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-06T12:05:16.755 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-06T12:05:16.755 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-06T12:05:16.755 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-06T12:05:16.755 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-06T12:05:16.755 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-06T12:05:16.755 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-06T12:05:16.756 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-05-06T12:05:16.756 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-06T12:05:16.756 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-06T12:05:16.756 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-06T12:05:16.756 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-06T12:05:16.756 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-05-06T12:05:16.756 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-06T12:05:16.756 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-06T12:05:16.756 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-06T12:05:16.756 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-06T12:05:16.756 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-06T12:05:16.756 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-06T12:05:16.756 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "restful" 2024-05-06T12:05:16.756 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-05-06T12:05:16.756 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-06T12:05:16.757 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-05-06T12:05:16.757 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-06T12:05:16.757 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-06T12:05:16.757 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "modified": "2024-05-06T12:04:51.221370+0000", 2024-05-06T12:05:16.757 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-06T12:05:16.757 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-05-06T12:05:16.757 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-06T12:05:16.757 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-05-06T12:05:16.757 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.771+0000 7fb2d44f2700 1 Processor -- start 2024-05-06T12:05:16.757 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.771+0000 7fb2d44f2700 1 -- start start 2024-05-06T12:05:16.757 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.771+0000 7fb2d44f2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2cc078cc0 0x7fb2cc077270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:16.757 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.771+0000 7fb2d44f2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2cc0790d0 con 0x7fb2cc078cc0 2024-05-06T12:05:16.757 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.771+0000 7fb2d228e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2cc078cc0 0x7fb2cc077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:16.757 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.771+0000 7fb2d228e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2cc078cc0 0x7fb2cc077270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37682/0 (socket says 172.21.15.92:37682) 2024-05-06T12:05:16.757 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.771+0000 7fb2d228e700 1 -- 172.21.15.92:0/2327582979 learned_addr learned my addr 172.21.15.92:0/2327582979 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:16.758 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.772+0000 7fb2d228e700 1 -- 172.21.15.92:0/2327582979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2cc0777b0 con 0x7fb2cc078cc0 2024-05-06T12:05:16.758 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.772+0000 7fb2d228e700 1 --2- 172.21.15.92:0/2327582979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2cc078cc0 0x7fb2cc077270 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4009e50 tx=0x7fb2c4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a917fec428673c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:16.758 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.772+0000 7fb2d128c700 1 -- 172.21.15.92:0/2327582979 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb2c40092e0 con 0x7fb2cc078cc0 2024-05-06T12:05:16.758 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.772+0000 7fb2d128c700 1 -- 172.21.15.92:0/2327582979 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fb2c4012430 con 0x7fb2cc078cc0 2024-05-06T12:05:16.758 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.772+0000 7fb2d128c700 1 -- 172.21.15.92:0/2327582979 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb2c40075c0 con 0x7fb2cc078cc0 2024-05-06T12:05:16.758 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.772+0000 7fb2d44f2700 1 -- 172.21.15.92:0/2327582979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2cc078cc0 msgr2=0x7fb2cc077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:16.758 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.773+0000 7fb2d44f2700 1 --2- 172.21.15.92:0/2327582979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2cc078cc0 0x7fb2cc077270 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4009e50 tx=0x7fb2c4009840 comp rx=0 tx=0).stop 2024-05-06T12:05:16.758 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.773+0000 7fb2d44f2700 1 -- 172.21.15.92:0/2327582979 shutdown_connections 2024-05-06T12:05:16.758 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.773+0000 7fb2d44f2700 1 --2- 172.21.15.92:0/2327582979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2cc078cc0 0x7fb2cc077270 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:16.758 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.773+0000 7fb2d44f2700 1 -- 172.21.15.92:0/2327582979 >> 172.21.15.92:0/2327582979 conn(0x7fb2cc0fe060 msgr2=0x7fb2cc100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:16.758 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.773+0000 7fb2d44f2700 1 -- 172.21.15.92:0/2327582979 shutdown_connections 2024-05-06T12:05:16.758 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.773+0000 7fb2d44f2700 1 -- 172.21.15.92:0/2327582979 wait complete. 2024-05-06T12:05:16.758 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.773+0000 7fb2d44f2700 1 Processor -- start 2024-05-06T12:05:16.758 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.773+0000 7fb2d44f2700 1 -- start start 2024-05-06T12:05:16.759 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.773+0000 7fb2d44f2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2cc078cc0 0x7fb2cc113890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:16.759 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.773+0000 7fb2d44f2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2cc113dd0 con 0x7fb2cc078cc0 2024-05-06T12:05:16.759 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.773+0000 7fb2d228e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2cc078cc0 0x7fb2cc113890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:16.759 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.773+0000 7fb2d228e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2cc078cc0 0x7fb2cc113890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37684/0 (socket says 172.21.15.92:37684) 2024-05-06T12:05:16.759 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.773+0000 7fb2d228e700 1 -- 172.21.15.92:0/1803985967 learned_addr learned my addr 172.21.15.92:0/1803985967 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:16.759 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.774+0000 7fb2d228e700 1 -- 172.21.15.92:0/1803985967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c4009ae0 con 0x7fb2cc078cc0 2024-05-06T12:05:16.759 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.774+0000 7fb2d228e700 1 --2- 172.21.15.92:0/1803985967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2cc078cc0 0x7fb2cc113890 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4009fd0 tx=0x7fb2c4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:16.759 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.774+0000 7fb2caffd700 1 -- 172.21.15.92:0/1803985967 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb2c40092e0 con 0x7fb2cc078cc0 2024-05-06T12:05:16.759 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.774+0000 7fb2caffd700 1 -- 172.21.15.92:0/1803985967 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fb2c4007d10 con 0x7fb2cc078cc0 2024-05-06T12:05:16.759 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.774+0000 7fb2d44f2700 1 -- 172.21.15.92:0/1803985967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2cc113fd0 con 0x7fb2cc078cc0 2024-05-06T12:05:16.759 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.774+0000 7fb2caffd700 1 -- 172.21.15.92:0/1803985967 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb2c401c8c0 con 0x7fb2cc078cc0 2024-05-06T12:05:16.759 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.774+0000 7fb2d44f2700 1 -- 172.21.15.92:0/1803985967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2cc114b40 con 0x7fb2cc078cc0 2024-05-06T12:05:16.759 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.775+0000 7fb2caffd700 1 -- 172.21.15.92:0/1803985967 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 43365+0+0 (secure 0 0 0) 0x7fb2c4019070 con 0x7fb2cc078cc0 2024-05-06T12:05:16.759 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.775+0000 7fb2d44f2700 1 -- 172.21.15.92:0/1803985967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2cc05f5c0 con 0x7fb2cc078cc0 2024-05-06T12:05:16.760 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.775+0000 7fb2caffd700 1 --2- 172.21.15.92:0/1803985967 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7fb2b4036850 0x7fb2b4038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:16.760 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.775+0000 7fb2caffd700 1 -- 172.21.15.92:0/1803985967 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fb2c4049d80 con 0x7fb2cc078cc0 2024-05-06T12:05:16.760 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.775+0000 7fb2d1a8d700 1 --2- 172.21.15.92:0/1803985967 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7fb2b4036850 0x7fb2b4038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:16.760 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.775+0000 7fb2d1a8d700 1 --2- 172.21.15.92:0/1803985967 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7fb2b4036850 0x7fb2b4038d00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0009990 tx=0x7fb2c0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:16.760 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:10.780+0000 7fb2caffd700 1 -- 172.21.15.92:0/1803985967 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb2c4011570 con 0x7fb2cc078cc0 2024-05-06T12:05:16.760 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:11.048+0000 7fb2d44f2700 1 -- 172.21.15.92:0/1803985967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb2cc114ec0 con 0x7fb2cc078cc0 2024-05-06T12:05:16.760 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:11.049+0000 7fb2caffd700 1 -- 172.21.15.92:0/1803985967 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7fb2c4020020 con 0x7fb2cc078cc0 2024-05-06T12:05:16.760 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:11.051+0000 7fb2d44f2700 1 -- 172.21.15.92:0/1803985967 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7fb2b4036850 msgr2=0x7fb2b4038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:16.760 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:11.051+0000 7fb2d44f2700 1 --2- 172.21.15.92:0/1803985967 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7fb2b4036850 0x7fb2b4038d00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0009990 tx=0x7fb2c0006e40 comp rx=0 tx=0).stop 2024-05-06T12:05:16.760 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:11.051+0000 7fb2d44f2700 1 -- 172.21.15.92:0/1803985967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2cc078cc0 msgr2=0x7fb2cc113890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:16.760 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:11.051+0000 7fb2d44f2700 1 --2- 172.21.15.92:0/1803985967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2cc078cc0 0x7fb2cc113890 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4009fd0 tx=0x7fb2c4009840 comp rx=0 tx=0).stop 2024-05-06T12:05:16.760 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:11.051+0000 7fb2d44f2700 1 -- 172.21.15.92:0/1803985967 shutdown_connections 2024-05-06T12:05:16.760 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:11.051+0000 7fb2d44f2700 1 --2- 172.21.15.92:0/1803985967 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7fb2b4036850 0x7fb2b4038d00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:16.760 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:11.051+0000 7fb2d44f2700 1 --2- 172.21.15.92:0/1803985967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2cc078cc0 0x7fb2cc113890 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:16.761 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:11.051+0000 7fb2d44f2700 1 -- 172.21.15.92:0/1803985967 >> 172.21.15.92:0/1803985967 conn(0x7fb2cc0fe060 msgr2=0x7fb2cc0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:16.761 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:11.051+0000 7fb2d44f2700 1 -- 172.21.15.92:0/1803985967 shutdown_connections 2024-05-06T12:05:16.761 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:11.051+0000 7fb2d44f2700 1 -- 172.21.15.92:0/1803985967 wait complete. 2024-05-06T12:05:16.761 INFO:teuthology.orchestra.run.smithi092.stdout:mgr is available 2024-05-06T12:05:18.721 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:18 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/2258651113' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-06T12:05:18.790 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-05-06T12:05:18.790 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout [global] 2024-05-06T12:05:18.791 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout fsid = aee1d912-0ba0-11ef-bc97-c7b262605968 2024-05-06T12:05:18.791 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-06T12:05:18.791 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-06T12:05:18.791 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-06T12:05:18.791 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-06T12:05:18.791 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-05-06T12:05:18.791 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-06T12:05:18.791 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-06T12:05:18.791 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-05-06T12:05:18.791 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout [osd] 2024-05-06T12:05:18.791 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-06T12:05:18.791 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-06T12:05:18.791 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-06T12:05:18.791 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-06T12:05:18.791 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-06T12:05:18.792 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.012+0000 7f5677dc5700 1 Processor -- start 2024-05-06T12:05:18.792 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.013+0000 7f5677dc5700 1 -- start start 2024-05-06T12:05:18.792 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.013+0000 7f5677dc5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5670078c80 0x7f5670077230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:18.792 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.013+0000 7f5677dc5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5670079090 con 0x7f5670078c80 2024-05-06T12:05:18.792 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.013+0000 7f5675b61700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5670078c80 0x7f5670077230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:18.792 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.013+0000 7f5675b61700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5670078c80 0x7f5670077230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45108/0 (socket says 172.21.15.92:45108) 2024-05-06T12:05:18.792 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.013+0000 7f5675b61700 1 -- 172.21.15.92:0/2211584309 learned_addr learned my addr 172.21.15.92:0/2211584309 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:18.792 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.013+0000 7f5675b61700 1 -- 172.21.15.92:0/2211584309 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5670077770 con 0x7f5670078c80 2024-05-06T12:05:18.792 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.013+0000 7f5675b61700 1 --2- 172.21.15.92:0/2211584309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5670078c80 0x7f5670077230 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f56600098e0 tx=0x7f5660009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8eeecbbb83873d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:18.792 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.014+0000 7f5674b5f700 1 -- 172.21.15.92:0/2211584309 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f566000b860 con 0x7f5670078c80 2024-05-06T12:05:18.792 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.014+0000 7f5674b5f700 1 -- 172.21.15.92:0/2211584309 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f566000be50 con 0x7f5670078c80 2024-05-06T12:05:18.792 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.014+0000 7f5674b5f700 1 -- 172.21.15.92:0/2211584309 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5660010380 con 0x7f5670078c80 2024-05-06T12:05:18.792 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.014+0000 7f5677dc5700 1 -- 172.21.15.92:0/2211584309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5670078c80 msgr2=0x7f5670077230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:18.793 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.014+0000 7f5677dc5700 1 --2- 172.21.15.92:0/2211584309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5670078c80 0x7f5670077230 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f56600098e0 tx=0x7f5660009250 comp rx=0 tx=0).stop 2024-05-06T12:05:18.793 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.014+0000 7f5677dc5700 1 -- 172.21.15.92:0/2211584309 shutdown_connections 2024-05-06T12:05:18.793 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.014+0000 7f5677dc5700 1 --2- 172.21.15.92:0/2211584309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5670078c80 0x7f5670077230 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:18.793 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.014+0000 7f5677dc5700 1 -- 172.21.15.92:0/2211584309 >> 172.21.15.92:0/2211584309 conn(0x7f56700fe060 msgr2=0x7f5670100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:18.793 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.014+0000 7f5677dc5700 1 -- 172.21.15.92:0/2211584309 shutdown_connections 2024-05-06T12:05:18.793 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.014+0000 7f5677dc5700 1 -- 172.21.15.92:0/2211584309 wait complete. 2024-05-06T12:05:18.793 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.015+0000 7f5677dc5700 1 Processor -- start 2024-05-06T12:05:18.793 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.015+0000 7f5677dc5700 1 -- start start 2024-05-06T12:05:18.793 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.015+0000 7f5677dc5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5670078c80 0x7f5670113810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:18.793 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.015+0000 7f5677dc5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5670113d50 con 0x7f5670078c80 2024-05-06T12:05:18.793 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.015+0000 7f5675b61700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5670078c80 0x7f5670113810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:18.793 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.015+0000 7f5675b61700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5670078c80 0x7f5670113810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45110/0 (socket says 172.21.15.92:45110) 2024-05-06T12:05:18.793 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.015+0000 7f5675b61700 1 -- 172.21.15.92:0/2258651113 learned_addr learned my addr 172.21.15.92:0/2258651113 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:18.793 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.015+0000 7f5675b61700 1 -- 172.21.15.92:0/2258651113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56600094f0 con 0x7f5670078c80 2024-05-06T12:05:18.793 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.016+0000 7f5675b61700 1 --2- 172.21.15.92:0/2258651113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5670078c80 0x7f5670113810 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5660006b00 tx=0x7f5660009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:18.794 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.016+0000 7f56667fc700 1 -- 172.21.15.92:0/2258651113 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f56600107c0 con 0x7f5670078c80 2024-05-06T12:05:18.794 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.016+0000 7f5677dc5700 1 -- 172.21.15.92:0/2258651113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5670113f50 con 0x7f5670078c80 2024-05-06T12:05:18.794 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.016+0000 7f56667fc700 1 -- 172.21.15.92:0/2258651113 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f5660010db0 con 0x7f5670078c80 2024-05-06T12:05:18.794 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.016+0000 7f56667fc700 1 -- 172.21.15.92:0/2258651113 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f566001acf0 con 0x7f5670078c80 2024-05-06T12:05:18.794 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.016+0000 7f5677dc5700 1 -- 172.21.15.92:0/2258651113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5670114ac0 con 0x7f5670078c80 2024-05-06T12:05:18.794 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.017+0000 7f56667fc700 1 -- 172.21.15.92:0/2258651113 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 43365+0+0 (secure 0 0 0) 0x7f56600193c0 con 0x7f5670078c80 2024-05-06T12:05:18.794 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.017+0000 7f5677dc5700 1 -- 172.21.15.92:0/2258651113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f567005f5c0 con 0x7f5670078c80 2024-05-06T12:05:18.794 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.017+0000 7f56667fc700 1 --2- 172.21.15.92:0/2258651113 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7f565c036800 0x7f565c038cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:18.794 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.017+0000 7f56667fc700 1 -- 172.21.15.92:0/2258651113 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f566004a760 con 0x7f5670078c80 2024-05-06T12:05:18.794 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.017+0000 7f5675360700 1 --2- 172.21.15.92:0/2258651113 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7f565c036800 0x7f565c038cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:18.794 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.017+0000 7f5675360700 1 --2- 172.21.15.92:0/2258651113 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7f565c036800 0x7f565c038cb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f566c009990 tx=0x7f566c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:18.794 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.022+0000 7f56667fc700 1 -- 172.21.15.92:0/2258651113 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5660019670 con 0x7f5670078c80 2024-05-06T12:05:18.794 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.207+0000 7f5677dc5700 1 -- 172.21.15.92:0/2258651113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f567004bed0 con 0x7f5670078c80 2024-05-06T12:05:18.795 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.208+0000 7f56667fc700 1 -- 172.21.15.92:0/2258651113 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f5660018b40 con 0x7f5670078c80 2024-05-06T12:05:18.795 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.210+0000 7f5677dc5700 1 -- 172.21.15.92:0/2258651113 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7f565c036800 msgr2=0x7f565c038cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:18.795 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.210+0000 7f5677dc5700 1 --2- 172.21.15.92:0/2258651113 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7f565c036800 0x7f565c038cb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f566c009990 tx=0x7f566c006e40 comp rx=0 tx=0).stop 2024-05-06T12:05:18.795 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.210+0000 7f5677dc5700 1 -- 172.21.15.92:0/2258651113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5670078c80 msgr2=0x7f5670113810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:18.795 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.210+0000 7f5677dc5700 1 --2- 172.21.15.92:0/2258651113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5670078c80 0x7f5670113810 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5660006b00 tx=0x7f5660009250 comp rx=0 tx=0).stop 2024-05-06T12:05:18.795 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.210+0000 7f5677dc5700 1 -- 172.21.15.92:0/2258651113 shutdown_connections 2024-05-06T12:05:18.795 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.210+0000 7f5677dc5700 1 --2- 172.21.15.92:0/2258651113 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7f565c036800 0x7f565c038cb0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:18.795 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.210+0000 7f5677dc5700 1 --2- 172.21.15.92:0/2258651113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5670078c80 0x7f5670113810 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:18.795 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.210+0000 7f5677dc5700 1 -- 172.21.15.92:0/2258651113 >> 172.21.15.92:0/2258651113 conn(0x7f56700fe060 msgr2=0x7f56700febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:18.795 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.210+0000 7f5677dc5700 1 -- 172.21.15.92:0/2258651113 shutdown_connections 2024-05-06T12:05:18.795 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:18.210+0000 7f5677dc5700 1 -- 172.21.15.92:0/2258651113 wait complete. 2024-05-06T12:05:18.795 INFO:teuthology.orchestra.run.smithi092.stdout:Enabling cephadm module... 2024-05-06T12:05:19.845 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:19 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/733600205' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-06T12:05:20.120 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.238+0000 7f3244eb9700 1 Processor -- start 2024-05-06T12:05:20.120 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.239+0000 7f3244eb9700 1 -- start start 2024-05-06T12:05:20.120 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.239+0000 7f3244eb9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3240078c80 0x7f3240077230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:20.120 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.239+0000 7f3244eb9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3240079090 con 0x7f3240078c80 2024-05-06T12:05:20.120 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.239+0000 7f323e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3240078c80 0x7f3240077230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:20.120 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.239+0000 7f323e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3240078c80 0x7f3240077230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45112/0 (socket says 172.21.15.92:45112) 2024-05-06T12:05:20.120 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.239+0000 7f323e59c700 1 -- 172.21.15.92:0/4090436876 learned_addr learned my addr 172.21.15.92:0/4090436876 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:20.121 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.239+0000 7f323e59c700 1 -- 172.21.15.92:0/4090436876 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3240077770 con 0x7f3240078c80 2024-05-06T12:05:20.121 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.239+0000 7f323e59c700 1 --2- 172.21.15.92:0/4090436876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3240078c80 0x7f3240077230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f32280098e0 tx=0x7f3228009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db9508abbc87f772 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:20.121 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.240+0000 7f323d59a700 1 -- 172.21.15.92:0/4090436876 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f322800b860 con 0x7f3240078c80 2024-05-06T12:05:20.121 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.240+0000 7f323d59a700 1 -- 172.21.15.92:0/4090436876 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f322800be50 con 0x7f3240078c80 2024-05-06T12:05:20.121 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.240+0000 7f323d59a700 1 -- 172.21.15.92:0/4090436876 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3228010380 con 0x7f3240078c80 2024-05-06T12:05:20.121 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.240+0000 7f3244eb9700 1 -- 172.21.15.92:0/4090436876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3240078c80 msgr2=0x7f3240077230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:20.121 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.240+0000 7f3244eb9700 1 --2- 172.21.15.92:0/4090436876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3240078c80 0x7f3240077230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f32280098e0 tx=0x7f3228009250 comp rx=0 tx=0).stop 2024-05-06T12:05:20.121 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.240+0000 7f3244eb9700 1 -- 172.21.15.92:0/4090436876 shutdown_connections 2024-05-06T12:05:20.121 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.240+0000 7f3244eb9700 1 --2- 172.21.15.92:0/4090436876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3240078c80 0x7f3240077230 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:20.121 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.240+0000 7f3244eb9700 1 -- 172.21.15.92:0/4090436876 >> 172.21.15.92:0/4090436876 conn(0x7f32400fe060 msgr2=0x7f3240100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:20.121 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.240+0000 7f3244eb9700 1 -- 172.21.15.92:0/4090436876 shutdown_connections 2024-05-06T12:05:20.121 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.240+0000 7f3244eb9700 1 -- 172.21.15.92:0/4090436876 wait complete. 2024-05-06T12:05:20.121 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.241+0000 7f3244eb9700 1 Processor -- start 2024-05-06T12:05:20.122 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.241+0000 7f3244eb9700 1 -- start start 2024-05-06T12:05:20.122 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.241+0000 7f3244eb9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3240078c80 0x7f3240113810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:20.122 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.241+0000 7f3244eb9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3240113d50 con 0x7f3240078c80 2024-05-06T12:05:20.122 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.241+0000 7f323e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3240078c80 0x7f3240113810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:20.122 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.241+0000 7f323e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3240078c80 0x7f3240113810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45120/0 (socket says 172.21.15.92:45120) 2024-05-06T12:05:20.122 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.241+0000 7f323e59c700 1 -- 172.21.15.92:0/733600205 learned_addr learned my addr 172.21.15.92:0/733600205 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:20.122 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.241+0000 7f323e59c700 1 -- 172.21.15.92:0/733600205 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32280094f0 con 0x7f3240078c80 2024-05-06T12:05:20.122 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.242+0000 7f323e59c700 1 --2- 172.21.15.92:0/733600205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3240078c80 0x7f3240113810 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3228003050 tx=0x7f3228009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:20.122 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.242+0000 7f3236ffd700 1 -- 172.21.15.92:0/733600205 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3228010760 con 0x7f3240078c80 2024-05-06T12:05:20.122 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.242+0000 7f3236ffd700 1 -- 172.21.15.92:0/733600205 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f3228010d50 con 0x7f3240078c80 2024-05-06T12:05:20.122 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.242+0000 7f3244eb9700 1 -- 172.21.15.92:0/733600205 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3240113f50 con 0x7f3240078c80 2024-05-06T12:05:20.122 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.242+0000 7f3236ffd700 1 -- 172.21.15.92:0/733600205 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f322801ae20 con 0x7f3240078c80 2024-05-06T12:05:20.122 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.242+0000 7f3244eb9700 1 -- 172.21.15.92:0/733600205 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3240114ac0 con 0x7f3240078c80 2024-05-06T12:05:20.123 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.243+0000 7f3236ffd700 1 -- 172.21.15.92:0/733600205 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 43365+0+0 (secure 0 0 0) 0x7f32280108c0 con 0x7f3240078c80 2024-05-06T12:05:20.123 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.243+0000 7f3236ffd700 1 --2- 172.21.15.92:0/733600205 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7f322c0367e0 0x7f322c038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:20.123 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.243+0000 7f3244eb9700 1 -- 172.21.15.92:0/733600205 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f324005f5c0 con 0x7f3240078c80 2024-05-06T12:05:20.123 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.243+0000 7f3236ffd700 1 -- 172.21.15.92:0/733600205 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f322804a4e0 con 0x7f3240078c80 2024-05-06T12:05:20.123 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.243+0000 7f323dd9b700 1 --2- 172.21.15.92:0/733600205 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7f322c0367e0 0x7f322c038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:20.123 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.243+0000 7f323dd9b700 1 --2- 172.21.15.92:0/733600205 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7f322c0367e0 0x7f322c038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3230009990 tx=0x7f3230006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:20.123 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.248+0000 7f3236ffd700 1 -- 172.21.15.92:0/733600205 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f322801a430 con 0x7f3240078c80 2024-05-06T12:05:20.123 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.486+0000 7f3244eb9700 1 -- 172.21.15.92:0/733600205 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f324004bed0 con 0x7f3240078c80 2024-05-06T12:05:20.123 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.539+0000 7f3236ffd700 1 -- 172.21.15.92:0/733600205 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 5) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f322802e440 con 0x7f3240078c80 2024-05-06T12:05:20.123 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.540+0000 7f3236ffd700 1 -- 172.21.15.92:0/733600205 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f3228018350 con 0x7f3240078c80 2024-05-06T12:05:20.123 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.542+0000 7f3244eb9700 1 -- 172.21.15.92:0/733600205 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7f322c0367e0 msgr2=0x7f322c038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:20.123 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.542+0000 7f3244eb9700 1 --2- 172.21.15.92:0/733600205 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7f322c0367e0 0x7f322c038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3230009990 tx=0x7f3230006e40 comp rx=0 tx=0).stop 2024-05-06T12:05:20.123 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.542+0000 7f3244eb9700 1 -- 172.21.15.92:0/733600205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3240078c80 msgr2=0x7f3240113810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:20.123 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.542+0000 7f3244eb9700 1 --2- 172.21.15.92:0/733600205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3240078c80 0x7f3240113810 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3228003050 tx=0x7f3228009250 comp rx=0 tx=0).stop 2024-05-06T12:05:20.124 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.542+0000 7f3244eb9700 1 -- 172.21.15.92:0/733600205 shutdown_connections 2024-05-06T12:05:20.124 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.542+0000 7f3244eb9700 1 --2- 172.21.15.92:0/733600205 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7f322c0367e0 0x7f322c038c90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:20.124 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.542+0000 7f3244eb9700 1 --2- 172.21.15.92:0/733600205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3240078c80 0x7f3240113810 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:20.124 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.542+0000 7f3244eb9700 1 -- 172.21.15.92:0/733600205 >> 172.21.15.92:0/733600205 conn(0x7f32400fe060 msgr2=0x7f32400febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:20.124 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.542+0000 7f3244eb9700 1 -- 172.21.15.92:0/733600205 shutdown_connections 2024-05-06T12:05:20.124 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:19.542+0000 7f3244eb9700 1 -- 172.21.15.92:0/733600205 wait complete. 2024-05-06T12:05:20.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:20 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/733600205' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-06T12:05:20.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:20 smithi092 ceph-mon[26954]: mgrmap e5: smithi092.gyzeqb(active, since 14s) 2024-05-06T12:05:21.394 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-05-06T12:05:21.394 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-06T12:05:21.394 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-06T12:05:21.394 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "active_name": "smithi092.gyzeqb", 2024-05-06T12:05:21.395 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-06T12:05:21.395 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-05-06T12:05:21.395 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.550+0000 7f286cc28700 1 Processor -- start 2024-05-06T12:05:21.395 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.551+0000 7f286cc28700 1 -- start start 2024-05-06T12:05:21.395 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.551+0000 7f286cc28700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2868101710 0x7f2868101b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:21.395 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.551+0000 7f286cc28700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2868102060 con 0x7f2868101710 2024-05-06T12:05:21.395 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.551+0000 7f286659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2868101710 0x7f2868101b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:21.395 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.551+0000 7f286659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2868101710 0x7f2868101b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45150/0 (socket says 172.21.15.92:45150) 2024-05-06T12:05:21.395 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.551+0000 7f286659c700 1 -- 172.21.15.92:0/2982200051 learned_addr learned my addr 172.21.15.92:0/2982200051 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:21.395 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.551+0000 7f286659c700 1 -- 172.21.15.92:0/2982200051 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28681021a0 con 0x7f2868101710 2024-05-06T12:05:21.395 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.552+0000 7f286659c700 1 --2- 172.21.15.92:0/2982200051 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2868101710 0x7f2868101b20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f285800bb80 tx=0x7f285800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=690515b6f5969fde server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:21.395 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.552+0000 7f286559a700 1 -- 172.21.15.92:0/2982200051 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f285800dd00 con 0x7f2868101710 2024-05-06T12:05:21.395 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.552+0000 7f286559a700 1 -- 172.21.15.92:0/2982200051 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f285800de60 con 0x7f2868101710 2024-05-06T12:05:21.396 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.552+0000 7f286559a700 1 -- 172.21.15.92:0/2982200051 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f28580188a0 con 0x7f2868101710 2024-05-06T12:05:21.396 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.552+0000 7f286cc28700 1 -- 172.21.15.92:0/2982200051 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2868101710 msgr2=0x7f2868101b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:21.396 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.552+0000 7f286cc28700 1 --2- 172.21.15.92:0/2982200051 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2868101710 0x7f2868101b20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f285800bb80 tx=0x7f285800b4f0 comp rx=0 tx=0).stop 2024-05-06T12:05:21.396 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.552+0000 7f286cc28700 1 -- 172.21.15.92:0/2982200051 shutdown_connections 2024-05-06T12:05:21.396 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.552+0000 7f286cc28700 1 --2- 172.21.15.92:0/2982200051 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2868101710 0x7f2868101b20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:21.396 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.552+0000 7f286cc28700 1 -- 172.21.15.92:0/2982200051 >> 172.21.15.92:0/2982200051 conn(0x7f28680fced0 msgr2=0x7f28680ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:21.396 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.552+0000 7f286cc28700 1 -- 172.21.15.92:0/2982200051 shutdown_connections 2024-05-06T12:05:21.396 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.552+0000 7f286cc28700 1 -- 172.21.15.92:0/2982200051 wait complete. 2024-05-06T12:05:21.396 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.553+0000 7f286cc28700 1 Processor -- start 2024-05-06T12:05:21.396 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.553+0000 7f286cc28700 1 -- start start 2024-05-06T12:05:21.396 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.553+0000 7f286cc28700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2868101710 0x7f286810f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:21.396 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.553+0000 7f286cc28700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286810f890 con 0x7f2868101710 2024-05-06T12:05:21.396 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.553+0000 7f286659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2868101710 0x7f286810f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:21.396 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.553+0000 7f286659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2868101710 0x7f286810f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45166/0 (socket says 172.21.15.92:45166) 2024-05-06T12:05:21.397 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.553+0000 7f286659c700 1 -- 172.21.15.92:0/111281940 learned_addr learned my addr 172.21.15.92:0/111281940 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:21.397 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.554+0000 7f286659c700 1 -- 172.21.15.92:0/111281940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f285800b790 con 0x7f2868101710 2024-05-06T12:05:21.397 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.554+0000 7f286659c700 1 --2- 172.21.15.92:0/111281940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2868101710 0x7f286810f350 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2858018600 tx=0x7f285800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:21.397 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.554+0000 7f2856ffd700 1 -- 172.21.15.92:0/111281940 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2858010bd0 con 0x7f2868101710 2024-05-06T12:05:21.397 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.554+0000 7f2856ffd700 1 -- 172.21.15.92:0/111281940 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f285801d050 con 0x7f2868101710 2024-05-06T12:05:21.397 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.554+0000 7f286cc28700 1 -- 172.21.15.92:0/111281940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f286810fa90 con 0x7f2868101710 2024-05-06T12:05:21.397 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.554+0000 7f2856ffd700 1 -- 172.21.15.92:0/111281940 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2858022590 con 0x7f2868101710 2024-05-06T12:05:21.397 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.554+0000 7f286cc28700 1 -- 172.21.15.92:0/111281940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2868110600 con 0x7f2868101710 2024-05-06T12:05:21.397 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.555+0000 7f2856ffd700 1 -- 172.21.15.92:0/111281940 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f2858023a00 con 0x7f2868101710 2024-05-06T12:05:21.397 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.555+0000 7f286cc28700 1 -- 172.21.15.92:0/111281940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f286805f5c0 con 0x7f2868101710 2024-05-06T12:05:21.397 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.555+0000 7f2856ffd700 1 --2- 172.21.15.92:0/111281940 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7f2850036830 0x7f2850038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:21.398 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.555+0000 7f2856ffd700 1 -- 172.21.15.92:0/111281940 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f2858014070 con 0x7f2868101710 2024-05-06T12:05:21.398 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.555+0000 7f2865d9b700 1 -- 172.21.15.92:0/111281940 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7f2850036830 msgr2=0x7f2850038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/3656822751 2024-05-06T12:05:21.398 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.555+0000 7f2865d9b700 1 --2- 172.21.15.92:0/111281940 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7f2850036830 0x7f2850038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-06T12:05:21.398 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.560+0000 7f2856ffd700 1 -- 172.21.15.92:0/111281940 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2858022790 con 0x7f2868101710 2024-05-06T12:05:21.398 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.756+0000 7f2865d9b700 1 -- 172.21.15.92:0/111281940 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7f2850036830 msgr2=0x7f2850038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/3656822751 2024-05-06T12:05:21.398 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.756+0000 7f2865d9b700 1 --2- 172.21.15.92:0/111281940 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7f2850036830 0x7f2850038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-06T12:05:21.398 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.835+0000 7f286cc28700 1 -- 172.21.15.92:0/111281940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f2868111170 con 0x7f2868101710 2024-05-06T12:05:21.398 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.835+0000 7f2856ffd700 1 -- 172.21.15.92:0/111281940 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f2858033e60 con 0x7f2868101710 2024-05-06T12:05:21.398 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.837+0000 7f286cc28700 1 -- 172.21.15.92:0/111281940 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7f2850036830 msgr2=0x7f2850038ce0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-06T12:05:21.398 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.837+0000 7f286cc28700 1 --2- 172.21.15.92:0/111281940 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7f2850036830 0x7f2850038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:21.398 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.837+0000 7f286cc28700 1 -- 172.21.15.92:0/111281940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2868101710 msgr2=0x7f286810f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:21.398 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.837+0000 7f286cc28700 1 --2- 172.21.15.92:0/111281940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2868101710 0x7f286810f350 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2858018600 tx=0x7f285800b4f0 comp rx=0 tx=0).stop 2024-05-06T12:05:21.398 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.837+0000 7f286cc28700 1 -- 172.21.15.92:0/111281940 shutdown_connections 2024-05-06T12:05:21.399 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.837+0000 7f286cc28700 1 --2- 172.21.15.92:0/111281940 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7f2850036830 0x7f2850038ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:21.399 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.837+0000 7f286cc28700 1 --2- 172.21.15.92:0/111281940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2868101710 0x7f286810f350 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:21.399 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.837+0000 7f286cc28700 1 -- 172.21.15.92:0/111281940 >> 172.21.15.92:0/111281940 conn(0x7f28680fced0 msgr2=0x7f2868105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:21.399 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.837+0000 7f286cc28700 1 -- 172.21.15.92:0/111281940 shutdown_connections 2024-05-06T12:05:21.399 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:20.838+0000 7f286cc28700 1 -- 172.21.15.92:0/111281940 wait complete. 2024-05-06T12:05:21.399 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for the mgr to restart... 2024-05-06T12:05:21.399 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for mgr epoch 5... 2024-05-06T12:05:21.648 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:21 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/111281940' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-06T12:05:26.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:26 smithi092 ceph-mon[26954]: Active manager daemon smithi092.gyzeqb restarted 2024-05-06T12:05:26.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:26 smithi092 ceph-mon[26954]: Activating manager daemon smithi092.gyzeqb 2024-05-06T12:05:26.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:26 smithi092 ceph-mon[26954]: osdmap e2: 0 total, 0 up, 0 in 2024-05-06T12:05:26.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:26 smithi092 ceph-mon[26954]: mgrmap e6: smithi092.gyzeqb(active, starting, since 0.00322776s) 2024-05-06T12:05:26.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:26 smithi092 ceph-mon[26954]: from='mgr.14120 172.21.15.92:0/2103511002' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi092"}]: dispatch 2024-05-06T12:05:26.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:26 smithi092 ceph-mon[26954]: from='mgr.14120 172.21.15.92:0/2103511002' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mgr metadata", "who": "smithi092.gyzeqb", "id": "smithi092.gyzeqb"}]: dispatch 2024-05-06T12:05:26.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:26 smithi092 ceph-mon[26954]: from='mgr.14120 172.21.15.92:0/2103511002' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-06T12:05:26.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:26 smithi092 ceph-mon[26954]: from='mgr.14120 172.21.15.92:0/2103511002' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-06T12:05:26.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:26 smithi092 ceph-mon[26954]: from='mgr.14120 172.21.15.92:0/2103511002' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-06T12:05:26.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:26 smithi092 ceph-mon[26954]: Manager daemon smithi092.gyzeqb is now available 2024-05-06T12:05:26.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:26 smithi092 ceph-mon[26954]: from='mgr.14120 172.21.15.92:0/2103511002' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:05:26.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:26 smithi092 ceph-mon[26954]: from='mgr.14120 172.21.15.92:0/2103511002' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:05:26.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:26 smithi092 ceph-mon[26954]: from='mgr.14120 172.21.15.92:0/2103511002' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:05:26.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:26 smithi092 ceph-mon[26954]: from='mgr.14120 172.21.15.92:0/2103511002' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:05:26.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:26 smithi092 ceph-mon[26954]: from='mgr.14120 172.21.15.92:0/2103511002' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:05:26.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:26 smithi092 ceph-mon[26954]: from='mgr.14120 172.21.15.92:0/2103511002' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi092.gyzeqb/mirror_snapshot_schedule"}]: dispatch 2024-05-06T12:05:26.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:26 smithi092 ceph-mon[26954]: from='mgr.14120 172.21.15.92:0/2103511002' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi092.gyzeqb/trash_purge_schedule"}]: dispatch 2024-05-06T12:05:27.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:27 smithi092 ceph-mon[26954]: Found migration_current of "None". Setting to last migration. 2024-05-06T12:05:27.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:27 smithi092 ceph-mon[26954]: from='mgr.14120 172.21.15.92:0/2103511002' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:05:27.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:27 smithi092 ceph-mon[26954]: from='mgr.14120 172.21.15.92:0/2103511002' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:05:27.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:27 smithi092 ceph-mon[26954]: mgrmap e7: smithi092.gyzeqb(active, since 1.00569s) 2024-05-06T12:05:27.526 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-05-06T12:05:27.526 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-06T12:05:27.526 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-06T12:05:27.526 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-05-06T12:05:27.526 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.860+0000 7fbfd4c10700 1 Processor -- start 2024-05-06T12:05:27.526 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.860+0000 7fbfd4c10700 1 -- start start 2024-05-06T12:05:27.526 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.860+0000 7fbfd4c10700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfd0076bf0 0x7fbfd0105480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:27.526 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.860+0000 7fbfd4c10700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd01059c0 con 0x7fbfd0076bf0 2024-05-06T12:05:27.526 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.860+0000 7fbfce59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfd0076bf0 0x7fbfd0105480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:27.527 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.860+0000 7fbfce59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfd0076bf0 0x7fbfd0105480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45176/0 (socket says 172.21.15.92:45176) 2024-05-06T12:05:27.527 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.860+0000 7fbfce59c700 1 -- 172.21.15.92:0/3309230470 learned_addr learned my addr 172.21.15.92:0/3309230470 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:27.527 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.861+0000 7fbfce59c700 1 -- 172.21.15.92:0/3309230470 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd0105b00 con 0x7fbfd0076bf0 2024-05-06T12:05:27.527 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.861+0000 7fbfce59c700 1 --2- 172.21.15.92:0/3309230470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfd0076bf0 0x7fbfd0105480 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00098e0 tx=0x7fbfc0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fdc2f7281ef0995 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:27.527 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.861+0000 7fbfcd59a700 1 -- 172.21.15.92:0/3309230470 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbfc000b860 con 0x7fbfd0076bf0 2024-05-06T12:05:27.527 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.861+0000 7fbfcd59a700 1 -- 172.21.15.92:0/3309230470 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fbfc000be50 con 0x7fbfd0076bf0 2024-05-06T12:05:27.527 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.861+0000 7fbfcd59a700 1 -- 172.21.15.92:0/3309230470 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbfc0010380 con 0x7fbfd0076bf0 2024-05-06T12:05:27.527 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.862+0000 7fbfd4c10700 1 -- 172.21.15.92:0/3309230470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfd0076bf0 msgr2=0x7fbfd0105480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:27.527 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.862+0000 7fbfd4c10700 1 --2- 172.21.15.92:0/3309230470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfd0076bf0 0x7fbfd0105480 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00098e0 tx=0x7fbfc0009250 comp rx=0 tx=0).stop 2024-05-06T12:05:27.527 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.862+0000 7fbfd4c10700 1 -- 172.21.15.92:0/3309230470 shutdown_connections 2024-05-06T12:05:27.527 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.862+0000 7fbfd4c10700 1 --2- 172.21.15.92:0/3309230470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfd0076bf0 0x7fbfd0105480 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:27.527 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.862+0000 7fbfd4c10700 1 -- 172.21.15.92:0/3309230470 >> 172.21.15.92:0/3309230470 conn(0x7fbfd00fcef0 msgr2=0x7fbfd00ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:27.527 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.862+0000 7fbfd4c10700 1 -- 172.21.15.92:0/3309230470 shutdown_connections 2024-05-06T12:05:27.528 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.862+0000 7fbfd4c10700 1 -- 172.21.15.92:0/3309230470 wait complete. 2024-05-06T12:05:27.528 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.862+0000 7fbfd4c10700 1 Processor -- start 2024-05-06T12:05:27.528 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.862+0000 7fbfd4c10700 1 -- start start 2024-05-06T12:05:27.528 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.863+0000 7fbfd4c10700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfd0076bf0 0x7fbfd0102dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:27.528 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.863+0000 7fbfd4c10700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd0103310 con 0x7fbfd0076bf0 2024-05-06T12:05:27.528 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.863+0000 7fbfce59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfd0076bf0 0x7fbfd0102dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:27.528 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.863+0000 7fbfce59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfd0076bf0 0x7fbfd0102dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45192/0 (socket says 172.21.15.92:45192) 2024-05-06T12:05:27.528 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.863+0000 7fbfce59c700 1 -- 172.21.15.92:0/456842754 learned_addr learned my addr 172.21.15.92:0/456842754 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:27.528 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.863+0000 7fbfce59c700 1 -- 172.21.15.92:0/456842754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc00094f0 con 0x7fbfd0076bf0 2024-05-06T12:05:27.528 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.863+0000 7fbfce59c700 1 --2- 172.21.15.92:0/456842754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfd0076bf0 0x7fbfd0102dd0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0003050 tx=0x7fbfc0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:27.528 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.863+0000 7fbfbeffd700 1 -- 172.21.15.92:0/456842754 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbfc0010760 con 0x7fbfd0076bf0 2024-05-06T12:05:27.528 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.863+0000 7fbfd4c10700 1 -- 172.21.15.92:0/456842754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd0101380 con 0x7fbfd0076bf0 2024-05-06T12:05:27.528 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.863+0000 7fbfbeffd700 1 -- 172.21.15.92:0/456842754 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fbfc0010d50 con 0x7fbfd0076bf0 2024-05-06T12:05:27.528 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.863+0000 7fbfbeffd700 1 -- 172.21.15.92:0/456842754 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbfc001ae20 con 0x7fbfd0076bf0 2024-05-06T12:05:27.529 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.863+0000 7fbfd4c10700 1 -- 172.21.15.92:0/456842754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd0101870 con 0x7fbfd0076bf0 2024-05-06T12:05:27.529 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.864+0000 7fbfbeffd700 1 -- 172.21.15.92:0/456842754 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fbfc002e440 con 0x7fbfd0076bf0 2024-05-06T12:05:27.529 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.864+0000 7fbfbeffd700 1 --2- 172.21.15.92:0/456842754 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7fbfb80368a0 0x7fbfb8038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:27.529 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.864+0000 7fbfbeffd700 1 -- 172.21.15.92:0/456842754 --> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbfb8039460 con 0x7fbfb80368a0 2024-05-06T12:05:27.529 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.864+0000 7fbfbeffd700 1 -- 172.21.15.92:0/456842754 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fbfc004a4a0 con 0x7fbfd0076bf0 2024-05-06T12:05:27.529 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.864+0000 7fbfcdd9b700 1 -- 172.21.15.92:0/456842754 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7fbfb80368a0 msgr2=0x7fbfb8038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/3656822751 2024-05-06T12:05:27.529 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:21.864+0000 7fbfcdd9b700 1 --2- 172.21.15.92:0/456842754 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7fbfb80368a0 0x7fbfb8038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-06T12:05:27.529 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:22.065+0000 7fbfcdd9b700 1 -- 172.21.15.92:0/456842754 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7fbfb80368a0 msgr2=0x7fbfb8038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/3656822751 2024-05-06T12:05:27.529 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:22.065+0000 7fbfcdd9b700 1 --2- 172.21.15.92:0/456842754 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7fbfb80368a0 0x7fbfb8038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-06T12:05:27.529 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:22.466+0000 7fbfcdd9b700 1 -- 172.21.15.92:0/456842754 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7fbfb80368a0 msgr2=0x7fbfb8038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/3656822751 2024-05-06T12:05:27.529 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:22.466+0000 7fbfcdd9b700 1 --2- 172.21.15.92:0/456842754 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7fbfb80368a0 0x7fbfb8038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-06T12:05:27.529 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:23.267+0000 7fbfcdd9b700 1 -- 172.21.15.92:0/456842754 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7fbfb80368a0 msgr2=0x7fbfb8038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/3656822751 2024-05-06T12:05:27.529 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:23.267+0000 7fbfcdd9b700 1 --2- 172.21.15.92:0/456842754 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7fbfb80368a0 0x7fbfb8038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-06T12:05:27.529 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:24.868+0000 7fbfcdd9b700 1 -- 172.21.15.92:0/456842754 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7fbfb80368a0 msgr2=0x7fbfb8038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/3656822751 2024-05-06T12:05:27.530 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:24.868+0000 7fbfcdd9b700 1 --2- 172.21.15.92:0/456842754 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7fbfb80368a0 0x7fbfb8038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-06T12:05:27.530 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:25.955+0000 7fbfbeffd700 1 -- 172.21.15.92:0/456842754 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mgrmap(e 6) v1 ==== 43143+0+0 (secure 0 0 0) 0x7fbfc00243f0 con 0x7fbfd0076bf0 2024-05-06T12:05:27.530 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:25.955+0000 7fbfbeffd700 1 -- 172.21.15.92:0/456842754 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7fbfb80368a0 msgr2=0x7fbfb8038d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-06T12:05:27.530 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:25.955+0000 7fbfbeffd700 1 --2- 172.21.15.92:0/456842754 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7fbfb80368a0 0x7fbfb8038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:27.530 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:26.956+0000 7fbfbeffd700 1 -- 172.21.15.92:0/456842754 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 7) v1 ==== 43270+0+0 (secure 0 0 0) 0x7fbfc0018d40 con 0x7fbfd0076bf0 2024-05-06T12:05:27.530 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:26.957+0000 7fbfbeffd700 1 --2- 172.21.15.92:0/456842754 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7fbfb8039ff0 0x7fbfb803c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:27.530 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:26.957+0000 7fbfbeffd700 1 -- 172.21.15.92:0/456842754 --> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbfb8039460 con 0x7fbfb8039ff0 2024-05-06T12:05:27.530 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:26.957+0000 7fbfcdd9b700 1 --2- 172.21.15.92:0/456842754 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7fbfb8039ff0 0x7fbfb803c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:27.530 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:26.957+0000 7fbfcdd9b700 1 --2- 172.21.15.92:0/456842754 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7fbfb8039ff0 0x7fbfb803c3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40039d0 tx=0x7fbfc4009350 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:27.530 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:26.958+0000 7fbfbeffd700 1 -- 172.21.15.92:0/456842754 <== mgr.14120 v2:172.21.15.92:6800/3731057893 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7fbfb8039460 con 0x7fbfb8039ff0 2024-05-06T12:05:27.530 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:26.964+0000 7fbfd4c10700 1 -- 172.21.15.92:0/456842754 --> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fbfd004bed0 con 0x7fbfb8039ff0 2024-05-06T12:05:27.530 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:26.964+0000 7fbfbeffd700 1 -- 172.21.15.92:0/456842754 <== mgr.14120 v2:172.21.15.92:6800/3731057893 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fbfd004bed0 con 0x7fbfb8039ff0 2024-05-06T12:05:27.530 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:26.964+0000 7fbfd4c10700 1 -- 172.21.15.92:0/456842754 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7fbfb8039ff0 msgr2=0x7fbfb803c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:27.530 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:26.964+0000 7fbfd4c10700 1 --2- 172.21.15.92:0/456842754 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7fbfb8039ff0 0x7fbfb803c3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40039d0 tx=0x7fbfc4009350 comp rx=0 tx=0).stop 2024-05-06T12:05:27.531 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:26.964+0000 7fbfd4c10700 1 -- 172.21.15.92:0/456842754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfd0076bf0 msgr2=0x7fbfd0102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:27.531 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:26.964+0000 7fbfd4c10700 1 --2- 172.21.15.92:0/456842754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfd0076bf0 0x7fbfd0102dd0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0003050 tx=0x7fbfc0009250 comp rx=0 tx=0).stop 2024-05-06T12:05:27.531 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:26.964+0000 7fbfd4c10700 1 -- 172.21.15.92:0/456842754 shutdown_connections 2024-05-06T12:05:27.531 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:26.964+0000 7fbfd4c10700 1 --2- 172.21.15.92:0/456842754 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7fbfb8039ff0 0x7fbfb803c3d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:27.531 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:26.964+0000 7fbfd4c10700 1 --2- 172.21.15.92:0/456842754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfd0076bf0 0x7fbfd0102dd0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:27.531 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:26.964+0000 7fbfd4c10700 1 --2- 172.21.15.92:0/456842754 >> [v2:172.21.15.92:6800/3656822751,v1:172.21.15.92:6801/3656822751] conn(0x7fbfb80368a0 0x7fbfb8038d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:27.531 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:26.964+0000 7fbfd4c10700 1 -- 172.21.15.92:0/456842754 >> 172.21.15.92:0/456842754 conn(0x7fbfd00fcef0 msgr2=0x7fbfd00fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:27.531 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:26.964+0000 7fbfd4c10700 1 -- 172.21.15.92:0/456842754 shutdown_connections 2024-05-06T12:05:27.531 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:26.964+0000 7fbfd4c10700 1 -- 172.21.15.92:0/456842754 wait complete. 2024-05-06T12:05:27.531 INFO:teuthology.orchestra.run.smithi092.stdout:mgr epoch 5 is available 2024-05-06T12:05:27.531 INFO:teuthology.orchestra.run.smithi092.stdout:Setting orchestrator backend to cephadm... 2024-05-06T12:05:28.154 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:28 smithi092 ceph-mon[26954]: [06/May/2024:12:05:26] ENGINE Bus STARTING 2024-05-06T12:05:28.154 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:28 smithi092 ceph-mon[26954]: [06/May/2024:12:05:26] ENGINE Serving on https://172.21.15.92:7150 2024-05-06T12:05:28.154 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:28 smithi092 ceph-mon[26954]: [06/May/2024:12:05:26] ENGINE Bus STARTED 2024-05-06T12:05:28.154 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:28 smithi092 ceph-mon[26954]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-06T12:05:28.155 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:28 smithi092 ceph-mon[26954]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-06T12:05:28.738 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.940+0000 7f8287599700 1 Processor -- start 2024-05-06T12:05:28.739 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.940+0000 7f8287599700 1 -- start start 2024-05-06T12:05:28.739 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.940+0000 7f8287599700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8280101d40 0x7f8280102150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:28.739 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.940+0000 7f8287599700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8280102690 con 0x7f8280101d40 2024-05-06T12:05:28.739 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.940+0000 7f8285335700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8280101d40 0x7f8280102150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:28.739 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.940+0000 7f8285335700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8280101d40 0x7f8280102150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36180/0 (socket says 172.21.15.92:36180) 2024-05-06T12:05:28.739 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.940+0000 7f8285335700 1 -- 172.21.15.92:0/550981850 learned_addr learned my addr 172.21.15.92:0/550981850 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:28.739 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.941+0000 7f8285335700 1 -- 172.21.15.92:0/550981850 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82801027d0 con 0x7f8280101d40 2024-05-06T12:05:28.739 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.941+0000 7f8285335700 1 --2- 172.21.15.92:0/550981850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8280101d40 0x7f8280102150 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f827c0098e0 tx=0x7f827c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c16bd7d818bc64e server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:28.739 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.941+0000 7f8277fff700 1 -- 172.21.15.92:0/550981850 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f827c00b860 con 0x7f8280101d40 2024-05-06T12:05:28.739 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.941+0000 7f8277fff700 1 -- 172.21.15.92:0/550981850 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f827c00be50 con 0x7f8280101d40 2024-05-06T12:05:28.739 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.941+0000 7f8277fff700 1 -- 172.21.15.92:0/550981850 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f827c010380 con 0x7f8280101d40 2024-05-06T12:05:28.739 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.941+0000 7f8287599700 1 -- 172.21.15.92:0/550981850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8280101d40 msgr2=0x7f8280102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:28.739 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.941+0000 7f8287599700 1 --2- 172.21.15.92:0/550981850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8280101d40 0x7f8280102150 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f827c0098e0 tx=0x7f827c009250 comp rx=0 tx=0).stop 2024-05-06T12:05:28.740 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.941+0000 7f8287599700 1 -- 172.21.15.92:0/550981850 shutdown_connections 2024-05-06T12:05:28.740 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.941+0000 7f8287599700 1 --2- 172.21.15.92:0/550981850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8280101d40 0x7f8280102150 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:28.740 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.941+0000 7f8287599700 1 -- 172.21.15.92:0/550981850 >> 172.21.15.92:0/550981850 conn(0x7f82800fc6f0 msgr2=0x7f82800feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:28.740 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.942+0000 7f8287599700 1 -- 172.21.15.92:0/550981850 shutdown_connections 2024-05-06T12:05:28.740 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.942+0000 7f8287599700 1 -- 172.21.15.92:0/550981850 wait complete. 2024-05-06T12:05:28.740 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.942+0000 7f8287599700 1 Processor -- start 2024-05-06T12:05:28.740 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.942+0000 7f8287599700 1 -- start start 2024-05-06T12:05:28.740 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.942+0000 7f8287599700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8280101d40 0x7f8280101540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:28.740 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.942+0000 7f8287599700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82801142d0 con 0x7f8280101d40 2024-05-06T12:05:28.740 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.942+0000 7f8285335700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8280101d40 0x7f8280101540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:28.740 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.942+0000 7f8285335700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8280101d40 0x7f8280101540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36194/0 (socket says 172.21.15.92:36194) 2024-05-06T12:05:28.740 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.942+0000 7f8285335700 1 -- 172.21.15.92:0/912487331 learned_addr learned my addr 172.21.15.92:0/912487331 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:28.741 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.943+0000 7f8285335700 1 -- 172.21.15.92:0/912487331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827c0094f0 con 0x7f8280101d40 2024-05-06T12:05:28.741 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.943+0000 7f8285335700 1 --2- 172.21.15.92:0/912487331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8280101d40 0x7f8280101540 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f827c000c00 tx=0x7f827c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:28.741 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.943+0000 7f8275ffb700 1 -- 172.21.15.92:0/912487331 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f827c0107c0 con 0x7f8280101d40 2024-05-06T12:05:28.741 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.943+0000 7f8275ffb700 1 -- 172.21.15.92:0/912487331 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f827c010db0 con 0x7f8280101d40 2024-05-06T12:05:28.741 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.943+0000 7f8287599700 1 -- 172.21.15.92:0/912487331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8280114410 con 0x7f8280101d40 2024-05-06T12:05:28.741 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.943+0000 7f8275ffb700 1 -- 172.21.15.92:0/912487331 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f827c01ad00 con 0x7f8280101d40 2024-05-06T12:05:28.741 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.943+0000 7f8287599700 1 -- 172.21.15.92:0/912487331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82801148a0 con 0x7f8280101d40 2024-05-06T12:05:28.741 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.944+0000 7f8275ffb700 1 -- 172.21.15.92:0/912487331 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 43270+0+0 (secure 0 0 0) 0x7f827c0193c0 con 0x7f8280101d40 2024-05-06T12:05:28.741 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.944+0000 7f8287599700 1 -- 172.21.15.92:0/912487331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f828004bed0 con 0x7f8280101d40 2024-05-06T12:05:28.741 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.944+0000 7f8275ffb700 1 --2- 172.21.15.92:0/912487331 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f826c036720 0x7f826c038bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:28.741 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.944+0000 7f8275ffb700 1 -- 172.21.15.92:0/912487331 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f827c04a620 con 0x7f8280101d40 2024-05-06T12:05:28.741 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.944+0000 7f8284b34700 1 --2- 172.21.15.92:0/912487331 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f826c036720 0x7f826c038bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:28.742 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.945+0000 7f8284b34700 1 --2- 172.21.15.92:0/912487331 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f826c036720 0x7f826c038bd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8270009990 tx=0x7f8270006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:28.742 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:27.949+0000 7f8275ffb700 1 -- 172.21.15.92:0/912487331 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f827c022930 con 0x7f8280101d40 2024-05-06T12:05:28.742 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:28.007+0000 7f8275ffb700 1 -- 172.21.15.92:0/912487331 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f827c01a530 con 0x7f8280101d40 2024-05-06T12:05:28.742 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:28.151+0000 7f8287599700 1 -- 172.21.15.92:0/912487331 --> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f82800fd8e0 con 0x7f826c036720 2024-05-06T12:05:28.742 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:28.160+0000 7f8275ffb700 1 -- 172.21.15.92:0/912487331 <== mgr.14120 v2:172.21.15.92:6800/3731057893 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f82800fd8e0 con 0x7f826c036720 2024-05-06T12:05:28.742 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:28.162+0000 7f8287599700 1 -- 172.21.15.92:0/912487331 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f826c036720 msgr2=0x7f826c038bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:28.742 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:28.162+0000 7f8287599700 1 --2- 172.21.15.92:0/912487331 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f826c036720 0x7f826c038bd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8270009990 tx=0x7f8270006e40 comp rx=0 tx=0).stop 2024-05-06T12:05:28.742 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:28.162+0000 7f8287599700 1 -- 172.21.15.92:0/912487331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8280101d40 msgr2=0x7f8280101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:28.742 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:28.162+0000 7f8287599700 1 --2- 172.21.15.92:0/912487331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8280101d40 0x7f8280101540 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f827c000c00 tx=0x7f827c009250 comp rx=0 tx=0).stop 2024-05-06T12:05:28.742 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:28.163+0000 7f8287599700 1 -- 172.21.15.92:0/912487331 shutdown_connections 2024-05-06T12:05:28.742 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:28.163+0000 7f8287599700 1 --2- 172.21.15.92:0/912487331 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f826c036720 0x7f826c038bd0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:28.742 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:28.163+0000 7f8287599700 1 --2- 172.21.15.92:0/912487331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8280101d40 0x7f8280101540 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:28.742 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:28.163+0000 7f8287599700 1 -- 172.21.15.92:0/912487331 >> 172.21.15.92:0/912487331 conn(0x7f82800fc6f0 msgr2=0x7f82800fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:28.742 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:28.163+0000 7f8287599700 1 -- 172.21.15.92:0/912487331 shutdown_connections 2024-05-06T12:05:28.743 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:28.163+0000 7f8287599700 1 -- 172.21.15.92:0/912487331 wait complete. 2024-05-06T12:05:29.309 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:29 smithi092 ceph-mon[26954]: mgrmap e8: smithi092.gyzeqb(active, since 2s) 2024-05-06T12:05:29.309 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:29 smithi092 ceph-mon[26954]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:05:29.309 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:29 smithi092 ceph-mon[26954]: from='mgr.14120 172.21.15.92:0/2103511002' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:05:29.309 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:29 smithi092 ceph-mon[26954]: from='mgr.14120 172.21.15.92:0/2103511002' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:05:29.881 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-06T12:05:29.881 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.110+0000 7f9907371700 1 Processor -- start 2024-05-06T12:05:29.881 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.110+0000 7f9907371700 1 -- start start 2024-05-06T12:05:29.881 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.110+0000 7f9907371700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9900102810 0x7f9900102c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:29.881 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.110+0000 7f9907371700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9900103160 con 0x7f9900102810 2024-05-06T12:05:29.881 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.110+0000 7f990510d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9900102810 0x7f9900102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:29.881 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.110+0000 7f990510d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9900102810 0x7f9900102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36198/0 (socket says 172.21.15.92:36198) 2024-05-06T12:05:29.881 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.110+0000 7f990510d700 1 -- 172.21.15.92:0/2555954411 learned_addr learned my addr 172.21.15.92:0/2555954411 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:29.881 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.110+0000 7f990510d700 1 -- 172.21.15.92:0/2555954411 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99001032a0 con 0x7f9900102810 2024-05-06T12:05:29.881 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.111+0000 7f990510d700 1 --2- 172.21.15.92:0/2555954411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9900102810 0x7f9900102c20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f98f40098e0 tx=0x7f98f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f8a30e6d4f9c05f server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:29.881 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.111+0000 7f98f3fff700 1 -- 172.21.15.92:0/2555954411 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f98f400b860 con 0x7f9900102810 2024-05-06T12:05:29.881 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.111+0000 7f98f3fff700 1 -- 172.21.15.92:0/2555954411 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f98f400be50 con 0x7f9900102810 2024-05-06T12:05:29.882 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.111+0000 7f98f3fff700 1 -- 172.21.15.92:0/2555954411 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f98f4010380 con 0x7f9900102810 2024-05-06T12:05:29.882 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.111+0000 7f9907371700 1 -- 172.21.15.92:0/2555954411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9900102810 msgr2=0x7f9900102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:29.882 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.111+0000 7f9907371700 1 --2- 172.21.15.92:0/2555954411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9900102810 0x7f9900102c20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f98f40098e0 tx=0x7f98f4009250 comp rx=0 tx=0).stop 2024-05-06T12:05:29.882 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.111+0000 7f9907371700 1 -- 172.21.15.92:0/2555954411 shutdown_connections 2024-05-06T12:05:29.882 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.111+0000 7f9907371700 1 --2- 172.21.15.92:0/2555954411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9900102810 0x7f9900102c20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:29.882 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.111+0000 7f9907371700 1 -- 172.21.15.92:0/2555954411 >> 172.21.15.92:0/2555954411 conn(0x7f99000fdfd0 msgr2=0x7f99001003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:29.882 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.111+0000 7f9907371700 1 -- 172.21.15.92:0/2555954411 shutdown_connections 2024-05-06T12:05:29.882 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.111+0000 7f9907371700 1 -- 172.21.15.92:0/2555954411 wait complete. 2024-05-06T12:05:29.882 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.112+0000 7f9907371700 1 Processor -- start 2024-05-06T12:05:29.882 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.112+0000 7f9907371700 1 -- start start 2024-05-06T12:05:29.882 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.112+0000 7f9907371700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9900102810 0x7f99001116a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:29.882 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.112+0000 7f9907371700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9900111be0 con 0x7f9900102810 2024-05-06T12:05:29.882 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.112+0000 7f990510d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9900102810 0x7f99001116a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:29.882 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.112+0000 7f990510d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9900102810 0x7f99001116a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36212/0 (socket says 172.21.15.92:36212) 2024-05-06T12:05:29.883 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.112+0000 7f990510d700 1 -- 172.21.15.92:0/2088761596 learned_addr learned my addr 172.21.15.92:0/2088761596 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:29.883 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.112+0000 7f990510d700 1 -- 172.21.15.92:0/2088761596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f40094f0 con 0x7f9900102810 2024-05-06T12:05:29.883 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.112+0000 7f990510d700 1 --2- 172.21.15.92:0/2088761596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9900102810 0x7f99001116a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f98f400ba90 tx=0x7f98f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:29.883 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.113+0000 7f98f1ffb700 1 -- 172.21.15.92:0/2088761596 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f98f40107c0 con 0x7f9900102810 2024-05-06T12:05:29.883 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.113+0000 7f98f1ffb700 1 -- 172.21.15.92:0/2088761596 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f98f4010db0 con 0x7f9900102810 2024-05-06T12:05:29.883 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.113+0000 7f9907371700 1 -- 172.21.15.92:0/2088761596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9900111de0 con 0x7f9900102810 2024-05-06T12:05:29.883 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.113+0000 7f98f1ffb700 1 -- 172.21.15.92:0/2088761596 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f98f401add0 con 0x7f9900102810 2024-05-06T12:05:29.883 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.113+0000 7f9907371700 1 -- 172.21.15.92:0/2088761596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9900112950 con 0x7f9900102810 2024-05-06T12:05:29.883 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.114+0000 7f98f1ffb700 1 -- 172.21.15.92:0/2088761596 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f98f402e440 con 0x7f9900102810 2024-05-06T12:05:29.883 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.114+0000 7f98f1ffb700 1 --2- 172.21.15.92:0/2088761596 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f98ec036850 0x7f98ec038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:29.883 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.114+0000 7f98f1ffb700 1 -- 172.21.15.92:0/2088761596 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f98f404a5b0 con 0x7f9900102810 2024-05-06T12:05:29.883 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.114+0000 7f9907371700 1 -- 172.21.15.92:0/2088761596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f990005f5c0 con 0x7f9900102810 2024-05-06T12:05:29.883 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.114+0000 7f990490c700 1 --2- 172.21.15.92:0/2088761596 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f98ec036850 0x7f98ec038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:29.883 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.114+0000 7f990490c700 1 --2- 172.21.15.92:0/2088761596 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f98ec036850 0x7f98ec038d00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f98fc009990 tx=0x7f98fc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:29.884 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.118+0000 7f98f1ffb700 1 -- 172.21.15.92:0/2088761596 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f98f4018940 con 0x7f9900102810 2024-05-06T12:05:29.884 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.306+0000 7f9907371700 1 -- 172.21.15.92:0/2088761596 --> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f99000ff200 con 0x7f98ec036850 2024-05-06T12:05:29.884 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.307+0000 7f98f1ffb700 1 -- 172.21.15.92:0/2088761596 <== mgr.14120 v2:172.21.15.92:6800/3731057893 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f99000ff200 con 0x7f98ec036850 2024-05-06T12:05:29.884 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.309+0000 7f9907371700 1 -- 172.21.15.92:0/2088761596 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f98ec036850 msgr2=0x7f98ec038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:29.884 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.309+0000 7f9907371700 1 --2- 172.21.15.92:0/2088761596 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f98ec036850 0x7f98ec038d00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f98fc009990 tx=0x7f98fc006e40 comp rx=0 tx=0).stop 2024-05-06T12:05:29.884 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.309+0000 7f9907371700 1 -- 172.21.15.92:0/2088761596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9900102810 msgr2=0x7f99001116a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:29.884 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.309+0000 7f9907371700 1 --2- 172.21.15.92:0/2088761596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9900102810 0x7f99001116a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f98f400ba90 tx=0x7f98f4009250 comp rx=0 tx=0).stop 2024-05-06T12:05:29.884 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.309+0000 7f9907371700 1 -- 172.21.15.92:0/2088761596 shutdown_connections 2024-05-06T12:05:29.884 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.309+0000 7f9907371700 1 --2- 172.21.15.92:0/2088761596 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f98ec036850 0x7f98ec038d00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:29.884 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.309+0000 7f9907371700 1 --2- 172.21.15.92:0/2088761596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9900102810 0x7f99001116a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:29.884 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.309+0000 7f9907371700 1 -- 172.21.15.92:0/2088761596 >> 172.21.15.92:0/2088761596 conn(0x7f99000fdfd0 msgr2=0x7f99000feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:29.884 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.309+0000 7f9907371700 1 -- 172.21.15.92:0/2088761596 shutdown_connections 2024-05-06T12:05:29.884 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:29.309+0000 7f9907371700 1 -- 172.21.15.92:0/2088761596 wait complete. 2024-05-06T12:05:29.884 INFO:teuthology.orchestra.run.smithi092.stdout:Generating ssh key... 2024-05-06T12:05:30.191 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:30 smithi092 ceph-mon[26954]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:05:31.221 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:31 smithi092 ceph-mon[26954]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:05:31.221 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:31 smithi092 ceph-mon[26954]: Generating ssh key... 2024-05-06T12:05:31.221 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:31 smithi092 ceph-mon[26954]: from='mgr.14120 172.21.15.92:0/2103511002' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:05:31.221 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:31 smithi092 ceph-mon[26954]: from='mgr.14120 172.21.15.92:0/2103511002' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:05:31.229 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.283+0000 7f22a9492700 1 Processor -- start 2024-05-06T12:05:31.229 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.283+0000 7f22a9492700 1 -- start start 2024-05-06T12:05:31.229 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.283+0000 7f22a9492700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a4102880 0x7f22a4102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:31.229 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.283+0000 7f22a9492700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a41031d0 con 0x7f22a4102880 2024-05-06T12:05:31.229 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.284+0000 7f22a2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a4102880 0x7f22a4102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:31.230 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.284+0000 7f22a2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a4102880 0x7f22a4102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36226/0 (socket says 172.21.15.92:36226) 2024-05-06T12:05:31.230 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.284+0000 7f22a2ffd700 1 -- 172.21.15.92:0/3371721579 learned_addr learned my addr 172.21.15.92:0/3371721579 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:31.230 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.284+0000 7f22a2ffd700 1 -- 172.21.15.92:0/3371721579 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a4103310 con 0x7f22a4102880 2024-05-06T12:05:31.230 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.284+0000 7f22a2ffd700 1 --2- 172.21.15.92:0/3371721579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a4102880 0x7f22a4102c90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f229c0098e0 tx=0x7f229c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b3ffb0dfdf80fd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:31.230 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.284+0000 7f22a1ffb700 1 -- 172.21.15.92:0/3371721579 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f229c00b860 con 0x7f22a4102880 2024-05-06T12:05:31.230 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.284+0000 7f22a1ffb700 1 -- 172.21.15.92:0/3371721579 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f229c00be50 con 0x7f22a4102880 2024-05-06T12:05:31.230 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.284+0000 7f22a1ffb700 1 -- 172.21.15.92:0/3371721579 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f229c010380 con 0x7f22a4102880 2024-05-06T12:05:31.230 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.285+0000 7f22a9492700 1 -- 172.21.15.92:0/3371721579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a4102880 msgr2=0x7f22a4102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:31.230 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.285+0000 7f22a9492700 1 --2- 172.21.15.92:0/3371721579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a4102880 0x7f22a4102c90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f229c0098e0 tx=0x7f229c009250 comp rx=0 tx=0).stop 2024-05-06T12:05:31.230 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.285+0000 7f22a9492700 1 -- 172.21.15.92:0/3371721579 shutdown_connections 2024-05-06T12:05:31.230 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.285+0000 7f22a9492700 1 --2- 172.21.15.92:0/3371721579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a4102880 0x7f22a4102c90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:31.230 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.285+0000 7f22a9492700 1 -- 172.21.15.92:0/3371721579 >> 172.21.15.92:0/3371721579 conn(0x7f22a40fe060 msgr2=0x7f22a4100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:31.230 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.285+0000 7f22a9492700 1 -- 172.21.15.92:0/3371721579 shutdown_connections 2024-05-06T12:05:31.231 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.285+0000 7f22a9492700 1 -- 172.21.15.92:0/3371721579 wait complete. 2024-05-06T12:05:31.231 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.286+0000 7f22a9492700 1 Processor -- start 2024-05-06T12:05:31.231 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.286+0000 7f22a9492700 1 -- start start 2024-05-06T12:05:31.231 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.286+0000 7f22a9492700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a4102880 0x7f22a41115f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:31.231 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.286+0000 7f22a9492700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a4111b30 con 0x7f22a4102880 2024-05-06T12:05:31.231 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.286+0000 7f22a2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a4102880 0x7f22a41115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:31.231 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.286+0000 7f22a2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a4102880 0x7f22a41115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36234/0 (socket says 172.21.15.92:36234) 2024-05-06T12:05:31.231 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.286+0000 7f22a2ffd700 1 -- 172.21.15.92:0/1961602413 learned_addr learned my addr 172.21.15.92:0/1961602413 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:31.231 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.286+0000 7f22a2ffd700 1 -- 172.21.15.92:0/1961602413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f229c0094f0 con 0x7f22a4102880 2024-05-06T12:05:31.231 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.286+0000 7f22a2ffd700 1 --2- 172.21.15.92:0/1961602413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a4102880 0x7f22a41115f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f229c000c00 tx=0x7f229c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:31.231 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.286+0000 7f228b7fe700 1 -- 172.21.15.92:0/1961602413 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f229c0107c0 con 0x7f22a4102880 2024-05-06T12:05:31.231 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.286+0000 7f228b7fe700 1 -- 172.21.15.92:0/1961602413 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f229c010db0 con 0x7f22a4102880 2024-05-06T12:05:31.231 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.286+0000 7f228b7fe700 1 -- 172.21.15.92:0/1961602413 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f229c01ad00 con 0x7f22a4102880 2024-05-06T12:05:31.231 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.286+0000 7f22a9492700 1 -- 172.21.15.92:0/1961602413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a4111d30 con 0x7f22a4102880 2024-05-06T12:05:31.232 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.287+0000 7f22a9492700 1 -- 172.21.15.92:0/1961602413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a41128a0 con 0x7f22a4102880 2024-05-06T12:05:31.232 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.287+0000 7f228b7fe700 1 -- 172.21.15.92:0/1961602413 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f229c0193c0 con 0x7f22a4102880 2024-05-06T12:05:31.232 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.287+0000 7f228b7fe700 1 --2- 172.21.15.92:0/1961602413 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f228c036850 0x7f228c038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:31.232 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.288+0000 7f228b7fe700 1 -- 172.21.15.92:0/1961602413 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f229c04a670 con 0x7f22a4102880 2024-05-06T12:05:31.232 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.288+0000 7f22a9492700 1 -- 172.21.15.92:0/1961602413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a405f5c0 con 0x7f22a4102880 2024-05-06T12:05:31.232 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.288+0000 7f22a27fc700 1 --2- 172.21.15.92:0/1961602413 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f228c036850 0x7f228c038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:31.232 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.288+0000 7f22a27fc700 1 --2- 172.21.15.92:0/1961602413 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f228c036850 0x7f228c038d00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2298009990 tx=0x7f2298006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:31.232 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.292+0000 7f228b7fe700 1 -- 172.21.15.92:0/1961602413 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f229c0196c0 con 0x7f22a4102880 2024-05-06T12:05:31.232 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.480+0000 7f22a9492700 1 -- 172.21.15.92:0/1961602413 --> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f22a40ff200 con 0x7f228c036850 2024-05-06T12:05:31.232 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.626+0000 7f228b7fe700 1 -- 172.21.15.92:0/1961602413 <== mgr.14120 v2:172.21.15.92:6800/3731057893 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f22a40ff200 con 0x7f228c036850 2024-05-06T12:05:31.232 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.629+0000 7f22a9492700 1 -- 172.21.15.92:0/1961602413 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f228c036850 msgr2=0x7f228c038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:31.232 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.629+0000 7f22a9492700 1 --2- 172.21.15.92:0/1961602413 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f228c036850 0x7f228c038d00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2298009990 tx=0x7f2298006e40 comp rx=0 tx=0).stop 2024-05-06T12:05:31.232 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.629+0000 7f22a9492700 1 -- 172.21.15.92:0/1961602413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a4102880 msgr2=0x7f22a41115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:31.232 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.629+0000 7f22a9492700 1 --2- 172.21.15.92:0/1961602413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a4102880 0x7f22a41115f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f229c000c00 tx=0x7f229c009250 comp rx=0 tx=0).stop 2024-05-06T12:05:31.232 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.629+0000 7f22a9492700 1 -- 172.21.15.92:0/1961602413 shutdown_connections 2024-05-06T12:05:31.233 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.629+0000 7f22a9492700 1 --2- 172.21.15.92:0/1961602413 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f228c036850 0x7f228c038d00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:31.233 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.629+0000 7f22a9492700 1 --2- 172.21.15.92:0/1961602413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a4102880 0x7f22a41115f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:31.233 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.629+0000 7f22a9492700 1 -- 172.21.15.92:0/1961602413 >> 172.21.15.92:0/1961602413 conn(0x7f22a40fe060 msgr2=0x7f22a40feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:31.233 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.629+0000 7f22a9492700 1 -- 172.21.15.92:0/1961602413 shutdown_connections 2024-05-06T12:05:31.233 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:30.629+0000 7f22a9492700 1 -- 172.21.15.92:0/1961602413 wait complete. 2024-05-06T12:05:32.403 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJEJ3jgs3fuPpPzaDYtNSt3dKhCPMJ5legS8fmNRb5uRtWAWoqLD0MlrIbR26GB3JxancyiuBTWG6N/Y1l0kCHFQTAH2tC25zKADOpZOvIeFMsLGa+hT55Uvjucomzfp4rD8DEMAF3T2uH2PzHa+VjWAx1q/txe8h3bz1fkCRC0v3a8IYAMNZZs1KheEXE+N0QKhx+iSjyoKDO+bAOcOMRzZzhTTOL1y17SdXHdkW6zx4El+Lq8hbiK5+GKvdEL5n9aRx9Fo8ymZoCVwSzw2zSmsNF0/rupr/IrraNCbwN+8kbo++MiY4BcAnrLIvH3SuTvvT9k1tnkBRljgs1qk8x1DcXJIdghJvjsrdVg8pA0gc3m4wHdPms0sXu/ySv5eLAikZ2hK99D7a4VC/dPkgPKMuhOXPIwZ8/NgPK78ESAWc+FZ8D65hG77clvO4Q/Ui+jV6Jz1xhW5wroEmxpSn0ldKGSyVrUqdIFW8tL4Bv24/796I0wSIfGc0jZymkDjU= ceph-aee1d912-0ba0-11ef-bc97-c7b262605968 2024-05-06T12:05:32.403 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.655+0000 7f7c3cb8a700 1 Processor -- start 2024-05-06T12:05:32.403 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.655+0000 7f7c3cb8a700 1 -- start start 2024-05-06T12:05:32.404 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.655+0000 7f7c3cb8a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c38102880 0x7f7c38102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:32.404 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.655+0000 7f7c3cb8a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c381031d0 con 0x7f7c38102880 2024-05-06T12:05:32.404 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.655+0000 7f7c3659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c38102880 0x7f7c38102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:32.404 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.655+0000 7f7c3659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c38102880 0x7f7c38102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36244/0 (socket says 172.21.15.92:36244) 2024-05-06T12:05:32.404 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.655+0000 7f7c3659c700 1 -- 172.21.15.92:0/94588972 learned_addr learned my addr 172.21.15.92:0/94588972 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:32.404 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.656+0000 7f7c3659c700 1 -- 172.21.15.92:0/94588972 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c38103310 con 0x7f7c38102880 2024-05-06T12:05:32.404 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.656+0000 7f7c3659c700 1 --2- 172.21.15.92:0/94588972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c38102880 0x7f7c38102c90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7c300098e0 tx=0x7f7c30009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=976fce1dfccd558d server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:32.404 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.656+0000 7f7c3559a700 1 -- 172.21.15.92:0/94588972 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7c3000b860 con 0x7f7c38102880 2024-05-06T12:05:32.404 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.656+0000 7f7c3559a700 1 -- 172.21.15.92:0/94588972 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f7c3000be50 con 0x7f7c38102880 2024-05-06T12:05:32.404 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.656+0000 7f7c3559a700 1 -- 172.21.15.92:0/94588972 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7c30010380 con 0x7f7c38102880 2024-05-06T12:05:32.404 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.657+0000 7f7c3cb8a700 1 -- 172.21.15.92:0/94588972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c38102880 msgr2=0x7f7c38102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:32.405 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.657+0000 7f7c3cb8a700 1 --2- 172.21.15.92:0/94588972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c38102880 0x7f7c38102c90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7c300098e0 tx=0x7f7c30009250 comp rx=0 tx=0).stop 2024-05-06T12:05:32.405 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.657+0000 7f7c3cb8a700 1 -- 172.21.15.92:0/94588972 shutdown_connections 2024-05-06T12:05:32.405 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.657+0000 7f7c3cb8a700 1 --2- 172.21.15.92:0/94588972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c38102880 0x7f7c38102c90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:32.405 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.657+0000 7f7c3cb8a700 1 -- 172.21.15.92:0/94588972 >> 172.21.15.92:0/94588972 conn(0x7f7c380fe060 msgr2=0x7f7c38100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:32.405 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.657+0000 7f7c3cb8a700 1 -- 172.21.15.92:0/94588972 shutdown_connections 2024-05-06T12:05:32.405 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.657+0000 7f7c3cb8a700 1 -- 172.21.15.92:0/94588972 wait complete. 2024-05-06T12:05:32.405 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.657+0000 7f7c3cb8a700 1 Processor -- start 2024-05-06T12:05:32.405 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.657+0000 7f7c3cb8a700 1 -- start start 2024-05-06T12:05:32.405 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.657+0000 7f7c3cb8a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c38102880 0x7f7c38078c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:32.405 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.657+0000 7f7c3cb8a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c38079160 con 0x7f7c38102880 2024-05-06T12:05:32.405 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.658+0000 7f7c3659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c38102880 0x7f7c38078c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:32.405 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.658+0000 7f7c3659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c38102880 0x7f7c38078c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36250/0 (socket says 172.21.15.92:36250) 2024-05-06T12:05:32.406 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.658+0000 7f7c3659c700 1 -- 172.21.15.92:0/4050434120 learned_addr learned my addr 172.21.15.92:0/4050434120 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:32.406 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.658+0000 7f7c3659c700 1 -- 172.21.15.92:0/4050434120 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c300094f0 con 0x7f7c38102880 2024-05-06T12:05:32.406 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.658+0000 7f7c3659c700 1 --2- 172.21.15.92:0/4050434120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c38102880 0x7f7c38078c20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000b440 tx=0x7f7c30009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:32.406 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.658+0000 7f7c26ffd700 1 -- 172.21.15.92:0/4050434120 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7c30010760 con 0x7f7c38102880 2024-05-06T12:05:32.406 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.658+0000 7f7c26ffd700 1 -- 172.21.15.92:0/4050434120 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f7c30010d50 con 0x7f7c38102880 2024-05-06T12:05:32.406 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.658+0000 7f7c3cb8a700 1 -- 172.21.15.92:0/4050434120 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c380771d0 con 0x7f7c38102880 2024-05-06T12:05:32.406 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.658+0000 7f7c26ffd700 1 -- 172.21.15.92:0/4050434120 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7c3001ae20 con 0x7f7c38102880 2024-05-06T12:05:32.406 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.658+0000 7f7c3cb8a700 1 -- 172.21.15.92:0/4050434120 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c380776c0 con 0x7f7c38102880 2024-05-06T12:05:32.406 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.659+0000 7f7c26ffd700 1 -- 172.21.15.92:0/4050434120 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f7c300108c0 con 0x7f7c38102880 2024-05-06T12:05:32.406 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.659+0000 7f7c26ffd700 1 --2- 172.21.15.92:0/4050434120 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f7c20036830 0x7f7c20038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:32.406 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.659+0000 7f7c3cb8a700 1 -- 172.21.15.92:0/4050434120 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c3804bed0 con 0x7f7c38102880 2024-05-06T12:05:32.406 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.659+0000 7f7c26ffd700 1 -- 172.21.15.92:0/4050434120 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f7c3004a650 con 0x7f7c38102880 2024-05-06T12:05:32.406 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.659+0000 7f7c35d9b700 1 --2- 172.21.15.92:0/4050434120 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f7c20036830 0x7f7c20038ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:32.407 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.660+0000 7f7c35d9b700 1 --2- 172.21.15.92:0/4050434120 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f7c20036830 0x7f7c20038ce0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c009990 tx=0x7f7c2c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:32.407 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.664+0000 7f7c26ffd700 1 -- 172.21.15.92:0/4050434120 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7c3002e440 con 0x7f7c38102880 2024-05-06T12:05:32.407 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.849+0000 7f7c3cb8a700 1 -- 172.21.15.92:0/4050434120 --> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c38077e80 con 0x7f7c20036830 2024-05-06T12:05:32.407 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.850+0000 7f7c26ffd700 1 -- 172.21.15.92:0/4050434120 <== mgr.14120 v2:172.21.15.92:6800/3731057893 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f7c38077e80 con 0x7f7c20036830 2024-05-06T12:05:32.407 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.853+0000 7f7c3cb8a700 1 -- 172.21.15.92:0/4050434120 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f7c20036830 msgr2=0x7f7c20038ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:32.407 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.853+0000 7f7c3cb8a700 1 --2- 172.21.15.92:0/4050434120 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f7c20036830 0x7f7c20038ce0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c009990 tx=0x7f7c2c006e40 comp rx=0 tx=0).stop 2024-05-06T12:05:32.407 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.853+0000 7f7c3cb8a700 1 -- 172.21.15.92:0/4050434120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c38102880 msgr2=0x7f7c38078c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:32.407 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.853+0000 7f7c3cb8a700 1 --2- 172.21.15.92:0/4050434120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c38102880 0x7f7c38078c20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000b440 tx=0x7f7c30009250 comp rx=0 tx=0).stop 2024-05-06T12:05:32.407 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.853+0000 7f7c3cb8a700 1 -- 172.21.15.92:0/4050434120 shutdown_connections 2024-05-06T12:05:32.407 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.853+0000 7f7c3cb8a700 1 --2- 172.21.15.92:0/4050434120 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f7c20036830 0x7f7c20038ce0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:32.407 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.853+0000 7f7c3cb8a700 1 --2- 172.21.15.92:0/4050434120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c38102880 0x7f7c38078c20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:32.407 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.853+0000 7f7c3cb8a700 1 -- 172.21.15.92:0/4050434120 >> 172.21.15.92:0/4050434120 conn(0x7f7c380fe060 msgr2=0x7f7c380feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:32.407 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.853+0000 7f7c3cb8a700 1 -- 172.21.15.92:0/4050434120 shutdown_connections 2024-05-06T12:05:32.408 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:31.853+0000 7f7c3cb8a700 1 -- 172.21.15.92:0/4050434120 wait complete. 2024-05-06T12:05:32.408 INFO:teuthology.orchestra.run.smithi092.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-06T12:05:32.408 INFO:teuthology.orchestra.run.smithi092.stdout:Adding key to root@localhost authorized_keys... 2024-05-06T12:05:32.408 INFO:teuthology.orchestra.run.smithi092.stdout:Adding host smithi092... 2024-05-06T12:05:32.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:32 smithi092 ceph-mon[26954]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:05:33.868 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:33 smithi092 ceph-mon[26954]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi092", "addr": "172.21.15.92", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:05:34.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:34 smithi092 ceph-mon[26954]: Deploying cephadm binary to smithi092 2024-05-06T12:05:35.402 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout Added host 'smithi092' with addr '172.21.15.92' 2024-05-06T12:05:35.402 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.834+0000 7f05bb6d5700 1 Processor -- start 2024-05-06T12:05:35.402 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.835+0000 7f05bb6d5700 1 -- start start 2024-05-06T12:05:35.402 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.835+0000 7f05bb6d5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05b4076c50 0x7f05b4105400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:35.403 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.835+0000 7f05bb6d5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b4105940 con 0x7f05b4076c50 2024-05-06T12:05:35.403 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.835+0000 7f05b9471700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05b4076c50 0x7f05b4105400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:35.403 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.835+0000 7f05b9471700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05b4076c50 0x7f05b4105400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53982/0 (socket says 172.21.15.92:53982) 2024-05-06T12:05:35.403 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.835+0000 7f05b9471700 1 -- 172.21.15.92:0/383667524 learned_addr learned my addr 172.21.15.92:0/383667524 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:35.403 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.835+0000 7f05b9471700 1 -- 172.21.15.92:0/383667524 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b4105a80 con 0x7f05b4076c50 2024-05-06T12:05:35.403 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.835+0000 7f05b9471700 1 --2- 172.21.15.92:0/383667524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05b4076c50 0x7f05b4105400 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f05a40098e0 tx=0x7f05a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=388cc76862c0308 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:35.403 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.836+0000 7f05abfff700 1 -- 172.21.15.92:0/383667524 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f05a400b860 con 0x7f05b4076c50 2024-05-06T12:05:35.403 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.836+0000 7f05abfff700 1 -- 172.21.15.92:0/383667524 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f05a400be50 con 0x7f05b4076c50 2024-05-06T12:05:35.403 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.836+0000 7f05abfff700 1 -- 172.21.15.92:0/383667524 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f05a4010380 con 0x7f05b4076c50 2024-05-06T12:05:35.403 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.836+0000 7f05bb6d5700 1 -- 172.21.15.92:0/383667524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05b4076c50 msgr2=0x7f05b4105400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:35.404 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.836+0000 7f05bb6d5700 1 --2- 172.21.15.92:0/383667524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05b4076c50 0x7f05b4105400 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f05a40098e0 tx=0x7f05a4009250 comp rx=0 tx=0).stop 2024-05-06T12:05:35.404 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.836+0000 7f05bb6d5700 1 -- 172.21.15.92:0/383667524 shutdown_connections 2024-05-06T12:05:35.404 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.836+0000 7f05bb6d5700 1 --2- 172.21.15.92:0/383667524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05b4076c50 0x7f05b4105400 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:35.404 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.836+0000 7f05bb6d5700 1 -- 172.21.15.92:0/383667524 >> 172.21.15.92:0/383667524 conn(0x7f05b40fcef0 msgr2=0x7f05b40ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:35.404 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.836+0000 7f05bb6d5700 1 -- 172.21.15.92:0/383667524 shutdown_connections 2024-05-06T12:05:35.404 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.836+0000 7f05bb6d5700 1 -- 172.21.15.92:0/383667524 wait complete. 2024-05-06T12:05:35.404 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.837+0000 7f05bb6d5700 1 Processor -- start 2024-05-06T12:05:35.404 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.837+0000 7f05bb6d5700 1 -- start start 2024-05-06T12:05:35.404 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.837+0000 7f05bb6d5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05b4076c50 0x7f05b41115e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:35.404 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.837+0000 7f05bb6d5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b4111b20 con 0x7f05b4076c50 2024-05-06T12:05:35.404 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.837+0000 7f05b9471700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05b4076c50 0x7f05b41115e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:35.404 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.837+0000 7f05b9471700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05b4076c50 0x7f05b41115e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53998/0 (socket says 172.21.15.92:53998) 2024-05-06T12:05:35.404 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.837+0000 7f05b9471700 1 -- 172.21.15.92:0/3289381075 learned_addr learned my addr 172.21.15.92:0/3289381075 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:35.404 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.837+0000 7f05b9471700 1 -- 172.21.15.92:0/3289381075 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a40094f0 con 0x7f05b4076c50 2024-05-06T12:05:35.405 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.838+0000 7f05b9471700 1 --2- 172.21.15.92:0/3289381075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05b4076c50 0x7f05b41115e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f05a4003050 tx=0x7f05a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:35.405 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.838+0000 7f05a9ffb700 1 -- 172.21.15.92:0/3289381075 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f05a4010760 con 0x7f05b4076c50 2024-05-06T12:05:35.405 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.838+0000 7f05bb6d5700 1 -- 172.21.15.92:0/3289381075 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b4111d20 con 0x7f05b4076c50 2024-05-06T12:05:35.405 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.838+0000 7f05a9ffb700 1 -- 172.21.15.92:0/3289381075 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f05a4010d50 con 0x7f05b4076c50 2024-05-06T12:05:35.405 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.838+0000 7f05a9ffb700 1 -- 172.21.15.92:0/3289381075 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f05a401ae20 con 0x7f05b4076c50 2024-05-06T12:05:35.405 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.838+0000 7f05bb6d5700 1 -- 172.21.15.92:0/3289381075 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b4112890 con 0x7f05b4076c50 2024-05-06T12:05:35.405 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.839+0000 7f05a9ffb700 1 -- 172.21.15.92:0/3289381075 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f05a402e440 con 0x7f05b4076c50 2024-05-06T12:05:35.405 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.839+0000 7f05bb6d5700 1 -- 172.21.15.92:0/3289381075 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05b405f5c0 con 0x7f05b4076c50 2024-05-06T12:05:35.405 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.839+0000 7f05a9ffb700 1 --2- 172.21.15.92:0/3289381075 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f05a0036830 0x7f05a0038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:35.405 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.839+0000 7f05a9ffb700 1 -- 172.21.15.92:0/3289381075 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f05a404a4f0 con 0x7f05b4076c50 2024-05-06T12:05:35.405 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.839+0000 7f05b8c70700 1 --2- 172.21.15.92:0/3289381075 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f05a0036830 0x7f05a0038ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:35.405 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.839+0000 7f05b8c70700 1 --2- 172.21.15.92:0/3289381075 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f05a0036830 0x7f05a0038ce0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009990 tx=0x7f05b0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:35.405 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:32.843+0000 7f05a9ffb700 1 -- 172.21.15.92:0/3289381075 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f05a4018b40 con 0x7f05b4076c50 2024-05-06T12:05:35.406 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:33.034+0000 7f05bb6d5700 1 -- 172.21.15.92:0/3289381075 --> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi092", "addr": "172.21.15.92", "target": ["mon-mgr", ""]}) v1 -- 0x7f05b40fe0b0 con 0x7f05a0036830 2024-05-06T12:05:35.406 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:34.816+0000 7f05a9ffb700 1 -- 172.21.15.92:0/3289381075 <== mgr.14120 v2:172.21.15.92:6800/3731057893 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f05b40fe0b0 con 0x7f05a0036830 2024-05-06T12:05:35.406 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:34.819+0000 7f05bb6d5700 1 -- 172.21.15.92:0/3289381075 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f05a0036830 msgr2=0x7f05a0038ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:35.406 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:34.819+0000 7f05bb6d5700 1 --2- 172.21.15.92:0/3289381075 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f05a0036830 0x7f05a0038ce0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009990 tx=0x7f05b0006e40 comp rx=0 tx=0).stop 2024-05-06T12:05:35.406 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:34.819+0000 7f05bb6d5700 1 -- 172.21.15.92:0/3289381075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05b4076c50 msgr2=0x7f05b41115e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:35.406 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:34.819+0000 7f05bb6d5700 1 --2- 172.21.15.92:0/3289381075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05b4076c50 0x7f05b41115e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f05a4003050 tx=0x7f05a4009250 comp rx=0 tx=0).stop 2024-05-06T12:05:35.406 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:34.819+0000 7f05bb6d5700 1 -- 172.21.15.92:0/3289381075 shutdown_connections 2024-05-06T12:05:35.406 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:34.819+0000 7f05bb6d5700 1 --2- 172.21.15.92:0/3289381075 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f05a0036830 0x7f05a0038ce0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:35.406 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:34.819+0000 7f05bb6d5700 1 --2- 172.21.15.92:0/3289381075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05b4076c50 0x7f05b41115e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:35.406 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:34.819+0000 7f05bb6d5700 1 -- 172.21.15.92:0/3289381075 >> 172.21.15.92:0/3289381075 conn(0x7f05b40fcef0 msgr2=0x7f05b40fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:35.406 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:34.819+0000 7f05bb6d5700 1 -- 172.21.15.92:0/3289381075 shutdown_connections 2024-05-06T12:05:35.406 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:34.819+0000 7f05bb6d5700 1 -- 172.21.15.92:0/3289381075 wait complete. 2024-05-06T12:05:35.406 INFO:teuthology.orchestra.run.smithi092.stdout:Deploying mon service with default placement... 2024-05-06T12:05:36.010 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:35 smithi092 ceph-mon[26954]: from='mgr.14120 172.21.15.92:0/2103511002' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:05:36.028 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:35 smithi092 ceph-mon[26954]: Added host smithi092 2024-05-06T12:05:36.028 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:35 smithi092 ceph-mon[26954]: from='mgr.14120 172.21.15.92:0/2103511002' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:05:36.654 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-06T12:05:36.654 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.836+0000 7f5a0257a700 1 Processor -- start 2024-05-06T12:05:36.654 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.836+0000 7f5a0257a700 1 -- start start 2024-05-06T12:05:36.654 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.837+0000 7f5a0257a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59fc102c90 0x7f59fc1030a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:36.654 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.837+0000 7f5a0257a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc103670 con 0x7f59fc102c90 2024-05-06T12:05:36.654 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.837+0000 7f59fbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59fc102c90 0x7f59fc1030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:36.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.837+0000 7f59fbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59fc102c90 0x7f59fc1030a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54006/0 (socket says 172.21.15.92:54006) 2024-05-06T12:05:36.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.837+0000 7f59fbfff700 1 -- 172.21.15.92:0/3072055685 learned_addr learned my addr 172.21.15.92:0/3072055685 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:36.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.837+0000 7f59fbfff700 1 -- 172.21.15.92:0/3072055685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59fc1037b0 con 0x7f59fc102c90 2024-05-06T12:05:36.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.837+0000 7f59fbfff700 1 --2- 172.21.15.92:0/3072055685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59fc102c90 0x7f59fc1030a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0098e0 tx=0x7f59ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=340decb12c25b6db server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:36.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.837+0000 7f59faffd700 1 -- 172.21.15.92:0/3072055685 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f59ec00b860 con 0x7f59fc102c90 2024-05-06T12:05:36.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.837+0000 7f59faffd700 1 -- 172.21.15.92:0/3072055685 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f59ec00be50 con 0x7f59fc102c90 2024-05-06T12:05:36.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.837+0000 7f59faffd700 1 -- 172.21.15.92:0/3072055685 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f59ec010380 con 0x7f59fc102c90 2024-05-06T12:05:36.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.838+0000 7f5a0257a700 1 -- 172.21.15.92:0/3072055685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59fc102c90 msgr2=0x7f59fc1030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:36.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.838+0000 7f5a0257a700 1 --2- 172.21.15.92:0/3072055685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59fc102c90 0x7f59fc1030a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0098e0 tx=0x7f59ec009250 comp rx=0 tx=0).stop 2024-05-06T12:05:36.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.838+0000 7f5a0257a700 1 -- 172.21.15.92:0/3072055685 shutdown_connections 2024-05-06T12:05:36.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.838+0000 7f5a0257a700 1 --2- 172.21.15.92:0/3072055685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59fc102c90 0x7f59fc1030a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:36.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.838+0000 7f5a0257a700 1 -- 172.21.15.92:0/3072055685 >> 172.21.15.92:0/3072055685 conn(0x7f59fc0fc6f0 msgr2=0x7f59fc0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:36.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.838+0000 7f5a0257a700 1 -- 172.21.15.92:0/3072055685 shutdown_connections 2024-05-06T12:05:36.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.838+0000 7f5a0257a700 1 -- 172.21.15.92:0/3072055685 wait complete. 2024-05-06T12:05:36.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.838+0000 7f5a0257a700 1 Processor -- start 2024-05-06T12:05:36.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.839+0000 7f5a0257a700 1 -- start start 2024-05-06T12:05:36.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.839+0000 7f5a0257a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59fc102c90 0x7f59fc1137d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:36.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.839+0000 7f5a0257a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc113d10 con 0x7f59fc102c90 2024-05-06T12:05:36.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.839+0000 7f59fbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59fc102c90 0x7f59fc1137d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:36.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.839+0000 7f59fbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59fc102c90 0x7f59fc1137d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54018/0 (socket says 172.21.15.92:54018) 2024-05-06T12:05:36.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.839+0000 7f59fbfff700 1 -- 172.21.15.92:0/808036368 learned_addr learned my addr 172.21.15.92:0/808036368 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:36.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.839+0000 7f59fbfff700 1 -- 172.21.15.92:0/808036368 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ec0094f0 con 0x7f59fc102c90 2024-05-06T12:05:36.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.839+0000 7f59fbfff700 1 --2- 172.21.15.92:0/808036368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59fc102c90 0x7f59fc1137d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f59ec000c00 tx=0x7f59ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:36.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.839+0000 7f59f8ff9700 1 -- 172.21.15.92:0/808036368 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f59ec0107c0 con 0x7f59fc102c90 2024-05-06T12:05:36.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.839+0000 7f59f8ff9700 1 -- 172.21.15.92:0/808036368 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f59ec010db0 con 0x7f59fc102c90 2024-05-06T12:05:36.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.839+0000 7f5a0257a700 1 -- 172.21.15.92:0/808036368 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59fc113f10 con 0x7f59fc102c90 2024-05-06T12:05:36.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.839+0000 7f59f8ff9700 1 -- 172.21.15.92:0/808036368 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f59ec01ad00 con 0x7f59fc102c90 2024-05-06T12:05:36.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.839+0000 7f5a0257a700 1 -- 172.21.15.92:0/808036368 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59fc114a80 con 0x7f59fc102c90 2024-05-06T12:05:36.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.840+0000 7f59f8ff9700 1 -- 172.21.15.92:0/808036368 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f59ec0193c0 con 0x7f59fc102c90 2024-05-06T12:05:36.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.840+0000 7f59f8ff9700 1 --2- 172.21.15.92:0/808036368 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f59e4036850 0x7f59e4038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:36.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.840+0000 7f5a0257a700 1 -- 172.21.15.92:0/808036368 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59fc05f5c0 con 0x7f59fc102c90 2024-05-06T12:05:36.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.840+0000 7f59f8ff9700 1 -- 172.21.15.92:0/808036368 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f59ec04a740 con 0x7f59fc102c90 2024-05-06T12:05:36.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.840+0000 7f59fb7fe700 1 --2- 172.21.15.92:0/808036368 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f59e4036850 0x7f59e4038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:36.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.841+0000 7f59fb7fe700 1 --2- 172.21.15.92:0/808036368 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f59e4036850 0x7f59e4038d00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f59f0009990 tx=0x7f59f0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:36.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:35.845+0000 7f59f8ff9700 1 -- 172.21.15.92:0/808036368 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f59ec0196c0 con 0x7f59fc102c90 2024-05-06T12:05:36.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:36.047+0000 7f5a0257a700 1 -- 172.21.15.92:0/808036368 --> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f59fc0fd8b0 con 0x7f59e4036850 2024-05-06T12:05:36.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:36.055+0000 7f59f8ff9700 1 -- 172.21.15.92:0/808036368 <== mgr.14120 v2:172.21.15.92:6800/3731057893 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f59fc0fd8b0 con 0x7f59e4036850 2024-05-06T12:05:36.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:36.057+0000 7f5a0257a700 1 -- 172.21.15.92:0/808036368 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f59e4036850 msgr2=0x7f59e4038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:36.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:36.057+0000 7f5a0257a700 1 --2- 172.21.15.92:0/808036368 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f59e4036850 0x7f59e4038d00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f59f0009990 tx=0x7f59f0006e40 comp rx=0 tx=0).stop 2024-05-06T12:05:36.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:36.057+0000 7f5a0257a700 1 -- 172.21.15.92:0/808036368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59fc102c90 msgr2=0x7f59fc1137d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:36.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:36.057+0000 7f5a0257a700 1 --2- 172.21.15.92:0/808036368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59fc102c90 0x7f59fc1137d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f59ec000c00 tx=0x7f59ec009250 comp rx=0 tx=0).stop 2024-05-06T12:05:36.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:36.057+0000 7f5a0257a700 1 -- 172.21.15.92:0/808036368 shutdown_connections 2024-05-06T12:05:36.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:36.057+0000 7f5a0257a700 1 --2- 172.21.15.92:0/808036368 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f59e4036850 0x7f59e4038d00 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:36.658 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:36.057+0000 7f5a0257a700 1 --2- 172.21.15.92:0/808036368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59fc102c90 0x7f59fc1137d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:36.658 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:36.057+0000 7f5a0257a700 1 -- 172.21.15.92:0/808036368 >> 172.21.15.92:0/808036368 conn(0x7f59fc0fc6f0 msgr2=0x7f59fc0fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:36.658 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:36.057+0000 7f5a0257a700 1 -- 172.21.15.92:0/808036368 shutdown_connections 2024-05-06T12:05:36.658 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:36.057+0000 7f5a0257a700 1 -- 172.21.15.92:0/808036368 wait complete. 2024-05-06T12:05:36.658 INFO:teuthology.orchestra.run.smithi092.stdout:Deploying mgr service with default placement... 2024-05-06T12:05:37.217 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:37 smithi092 ceph-mon[26954]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:05:37.218 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:37 smithi092 ceph-mon[26954]: Saving service mon spec with placement count:5 2024-05-06T12:05:37.218 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:37 smithi092 ceph-mon[26954]: from='mgr.14120 172.21.15.92:0/2103511002' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:05:37.863 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-06T12:05:37.863 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.053+0000 7ff0613a3700 1 Processor -- start 2024-05-06T12:05:37.863 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.053+0000 7ff0613a3700 1 -- start start 2024-05-06T12:05:37.863 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.053+0000 7ff0613a3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff05c076c10 0x7ff05c105460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:37.863 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.053+0000 7ff0613a3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05c1059a0 con 0x7ff05c076c10 2024-05-06T12:05:37.863 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.053+0000 7ff05affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff05c076c10 0x7ff05c105460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:37.863 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.053+0000 7ff05affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff05c076c10 0x7ff05c105460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54028/0 (socket says 172.21.15.92:54028) 2024-05-06T12:05:37.863 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.053+0000 7ff05affd700 1 -- 172.21.15.92:0/1036575839 learned_addr learned my addr 172.21.15.92:0/1036575839 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:37.863 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.053+0000 7ff05affd700 1 -- 172.21.15.92:0/1036575839 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff05c105ae0 con 0x7ff05c076c10 2024-05-06T12:05:37.864 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.054+0000 7ff05affd700 1 --2- 172.21.15.92:0/1036575839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff05c076c10 0x7ff05c105460 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff04c0098e0 tx=0x7ff04c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b412f67bc1b5044 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:37.864 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.058+0000 7ff05a7fc700 1 -- 172.21.15.92:0/1036575839 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff04c00b860 con 0x7ff05c076c10 2024-05-06T12:05:37.864 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.058+0000 7ff05a7fc700 1 -- 172.21.15.92:0/1036575839 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ff04c00be50 con 0x7ff05c076c10 2024-05-06T12:05:37.864 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.058+0000 7ff05a7fc700 1 -- 172.21.15.92:0/1036575839 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff04c010380 con 0x7ff05c076c10 2024-05-06T12:05:37.864 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.059+0000 7ff0613a3700 1 -- 172.21.15.92:0/1036575839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff05c076c10 msgr2=0x7ff05c105460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:37.864 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.059+0000 7ff0613a3700 1 --2- 172.21.15.92:0/1036575839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff05c076c10 0x7ff05c105460 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff04c0098e0 tx=0x7ff04c009250 comp rx=0 tx=0).stop 2024-05-06T12:05:37.865 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.059+0000 7ff0613a3700 1 -- 172.21.15.92:0/1036575839 shutdown_connections 2024-05-06T12:05:37.865 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.059+0000 7ff0613a3700 1 --2- 172.21.15.92:0/1036575839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff05c076c10 0x7ff05c105460 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:37.865 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.059+0000 7ff0613a3700 1 -- 172.21.15.92:0/1036575839 >> 172.21.15.92:0/1036575839 conn(0x7ff05c0fcef0 msgr2=0x7ff05c0ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:37.865 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.059+0000 7ff0613a3700 1 -- 172.21.15.92:0/1036575839 shutdown_connections 2024-05-06T12:05:37.865 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.059+0000 7ff0613a3700 1 -- 172.21.15.92:0/1036575839 wait complete. 2024-05-06T12:05:37.865 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.059+0000 7ff0613a3700 1 Processor -- start 2024-05-06T12:05:37.865 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.059+0000 7ff0613a3700 1 -- start start 2024-05-06T12:05:37.865 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.059+0000 7ff0613a3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff05c076c10 0x7ff05c1115b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:37.866 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.059+0000 7ff0613a3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05c111af0 con 0x7ff05c076c10 2024-05-06T12:05:37.866 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.059+0000 7ff05affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff05c076c10 0x7ff05c1115b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:37.866 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.059+0000 7ff05affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff05c076c10 0x7ff05c1115b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54036/0 (socket says 172.21.15.92:54036) 2024-05-06T12:05:37.866 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.059+0000 7ff05affd700 1 -- 172.21.15.92:0/2664374946 learned_addr learned my addr 172.21.15.92:0/2664374946 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:37.866 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.060+0000 7ff05affd700 1 -- 172.21.15.92:0/2664374946 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04c0094f0 con 0x7ff05c076c10 2024-05-06T12:05:37.866 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.060+0000 7ff05affd700 1 --2- 172.21.15.92:0/2664374946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff05c076c10 0x7ff05c1115b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff04c003050 tx=0x7ff04c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:37.866 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.060+0000 7ff0537fe700 1 -- 172.21.15.92:0/2664374946 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff04c010760 con 0x7ff05c076c10 2024-05-06T12:05:37.866 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.060+0000 7ff0537fe700 1 -- 172.21.15.92:0/2664374946 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ff04c010d50 con 0x7ff05c076c10 2024-05-06T12:05:37.866 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.060+0000 7ff0537fe700 1 -- 172.21.15.92:0/2664374946 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff04c01adb0 con 0x7ff05c076c10 2024-05-06T12:05:37.866 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.060+0000 7ff0613a3700 1 -- 172.21.15.92:0/2664374946 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff05c111d50 con 0x7ff05c076c10 2024-05-06T12:05:37.866 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.060+0000 7ff0613a3700 1 -- 172.21.15.92:0/2664374946 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff05c112860 con 0x7ff05c076c10 2024-05-06T12:05:37.866 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.061+0000 7ff0537fe700 1 -- 172.21.15.92:0/2664374946 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7ff04c02e440 con 0x7ff05c076c10 2024-05-06T12:05:37.867 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.061+0000 7ff0537fe700 1 --2- 172.21.15.92:0/2664374946 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7ff048036850 0x7ff048038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:37.867 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.061+0000 7ff0537fe700 1 -- 172.21.15.92:0/2664374946 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7ff04c04a470 con 0x7ff05c076c10 2024-05-06T12:05:37.867 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.061+0000 7ff053fff700 1 --2- 172.21.15.92:0/2664374946 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7ff048036850 0x7ff048038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:37.867 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.061+0000 7ff0613a3700 1 -- 172.21.15.92:0/2664374946 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff05c05f5c0 con 0x7ff05c076c10 2024-05-06T12:05:37.867 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.061+0000 7ff053fff700 1 --2- 172.21.15.92:0/2664374946 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7ff048036850 0x7ff048038d00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff044009990 tx=0x7ff044006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:37.867 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.066+0000 7ff0537fe700 1 -- 172.21.15.92:0/2664374946 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff04c018940 con 0x7ff05c076c10 2024-05-06T12:05:37.867 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.264+0000 7ff0613a3700 1 -- 172.21.15.92:0/2664374946 --> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7ff05c0fe0b0 con 0x7ff048036850 2024-05-06T12:05:37.868 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.271+0000 7ff0537fe700 1 -- 172.21.15.92:0/2664374946 <== mgr.14120 v2:172.21.15.92:6800/3731057893 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7ff05c0fe0b0 con 0x7ff048036850 2024-05-06T12:05:37.868 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.273+0000 7ff0613a3700 1 -- 172.21.15.92:0/2664374946 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7ff048036850 msgr2=0x7ff048038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:37.868 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.273+0000 7ff0613a3700 1 --2- 172.21.15.92:0/2664374946 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7ff048036850 0x7ff048038d00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff044009990 tx=0x7ff044006e40 comp rx=0 tx=0).stop 2024-05-06T12:05:37.868 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.273+0000 7ff0613a3700 1 -- 172.21.15.92:0/2664374946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff05c076c10 msgr2=0x7ff05c1115b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:37.868 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.273+0000 7ff0613a3700 1 --2- 172.21.15.92:0/2664374946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff05c076c10 0x7ff05c1115b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff04c003050 tx=0x7ff04c009250 comp rx=0 tx=0).stop 2024-05-06T12:05:37.868 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.273+0000 7ff0613a3700 1 -- 172.21.15.92:0/2664374946 shutdown_connections 2024-05-06T12:05:37.868 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.273+0000 7ff0613a3700 1 --2- 172.21.15.92:0/2664374946 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7ff048036850 0x7ff048038d00 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:37.868 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.273+0000 7ff0613a3700 1 --2- 172.21.15.92:0/2664374946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff05c076c10 0x7ff05c1115b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:37.868 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.273+0000 7ff0613a3700 1 -- 172.21.15.92:0/2664374946 >> 172.21.15.92:0/2664374946 conn(0x7ff05c0fcef0 msgr2=0x7ff05c0fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:37.868 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.274+0000 7ff0613a3700 1 -- 172.21.15.92:0/2664374946 shutdown_connections 2024-05-06T12:05:37.868 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:37.274+0000 7ff0613a3700 1 -- 172.21.15.92:0/2664374946 wait complete. 2024-05-06T12:05:37.868 INFO:teuthology.orchestra.run.smithi092.stdout:Deploying crash service with default placement... 2024-05-06T12:05:38.567 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:38 smithi092 ceph-mon[26954]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:05:38.567 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:38 smithi092 ceph-mon[26954]: Saving service mgr spec with placement count:2 2024-05-06T12:05:38.567 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:38 smithi092 ceph-mon[26954]: from='mgr.14120 172.21.15.92:0/2103511002' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:05:39.765 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-05-06T12:05:39.765 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.029+0000 7fd602b50700 1 Processor -- start 2024-05-06T12:05:39.765 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.029+0000 7fd602b50700 1 -- start start 2024-05-06T12:05:39.765 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.029+0000 7fd602b50700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5fc102810 0x7fd5fc102c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:39.765 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.029+0000 7fd602b50700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc103160 con 0x7fd5fc102810 2024-05-06T12:05:39.765 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.029+0000 7fd6008ec700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5fc102810 0x7fd5fc102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:39.766 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.029+0000 7fd6008ec700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5fc102810 0x7fd5fc102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54046/0 (socket says 172.21.15.92:54046) 2024-05-06T12:05:39.766 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.029+0000 7fd6008ec700 1 -- 172.21.15.92:0/1263815538 learned_addr learned my addr 172.21.15.92:0/1263815538 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:39.766 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.029+0000 7fd6008ec700 1 -- 172.21.15.92:0/1263815538 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5fc1032a0 con 0x7fd5fc102810 2024-05-06T12:05:39.766 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.029+0000 7fd6008ec700 1 --2- 172.21.15.92:0/1263815538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5fc102810 0x7fd5fc102c20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd5f00098e0 tx=0x7fd5f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=658758759b6562da server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:39.766 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.030+0000 7fd5fb7fe700 1 -- 172.21.15.92:0/1263815538 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd5f000b860 con 0x7fd5fc102810 2024-05-06T12:05:39.766 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.030+0000 7fd5fb7fe700 1 -- 172.21.15.92:0/1263815538 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd5f000be90 con 0x7fd5fc102810 2024-05-06T12:05:39.766 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.030+0000 7fd5fb7fe700 1 -- 172.21.15.92:0/1263815538 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd5f00103c0 con 0x7fd5fc102810 2024-05-06T12:05:39.766 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.030+0000 7fd602b50700 1 -- 172.21.15.92:0/1263815538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5fc102810 msgr2=0x7fd5fc102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:39.766 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.030+0000 7fd602b50700 1 --2- 172.21.15.92:0/1263815538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5fc102810 0x7fd5fc102c20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd5f00098e0 tx=0x7fd5f0009250 comp rx=0 tx=0).stop 2024-05-06T12:05:39.766 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.030+0000 7fd602b50700 1 -- 172.21.15.92:0/1263815538 shutdown_connections 2024-05-06T12:05:39.766 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.030+0000 7fd602b50700 1 --2- 172.21.15.92:0/1263815538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5fc102810 0x7fd5fc102c20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:39.766 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.030+0000 7fd602b50700 1 -- 172.21.15.92:0/1263815538 >> 172.21.15.92:0/1263815538 conn(0x7fd5fc0fdfd0 msgr2=0x7fd5fc1003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:39.766 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.030+0000 7fd602b50700 1 -- 172.21.15.92:0/1263815538 shutdown_connections 2024-05-06T12:05:39.767 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.030+0000 7fd602b50700 1 -- 172.21.15.92:0/1263815538 wait complete. 2024-05-06T12:05:39.767 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.031+0000 7fd602b50700 1 Processor -- start 2024-05-06T12:05:39.767 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.031+0000 7fd602b50700 1 -- start start 2024-05-06T12:05:39.767 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.031+0000 7fd602b50700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5fc102810 0x7fd5fc1116c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:39.767 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.031+0000 7fd602b50700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc111c00 con 0x7fd5fc102810 2024-05-06T12:05:39.767 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.031+0000 7fd6008ec700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5fc102810 0x7fd5fc1116c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:39.767 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.031+0000 7fd6008ec700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5fc102810 0x7fd5fc1116c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54050/0 (socket says 172.21.15.92:54050) 2024-05-06T12:05:39.767 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.031+0000 7fd6008ec700 1 -- 172.21.15.92:0/3165220640 learned_addr learned my addr 172.21.15.92:0/3165220640 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:39.767 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.032+0000 7fd6008ec700 1 -- 172.21.15.92:0/3165220640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f00094f0 con 0x7fd5fc102810 2024-05-06T12:05:39.767 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.032+0000 7fd6008ec700 1 --2- 172.21.15.92:0/3165220640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5fc102810 0x7fd5fc1116c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0006b00 tx=0x7fd5f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:39.767 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.032+0000 7fd5f97fa700 1 -- 172.21.15.92:0/3165220640 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd5f0010840 con 0x7fd5fc102810 2024-05-06T12:05:39.767 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.032+0000 7fd5f97fa700 1 -- 172.21.15.92:0/3165220640 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd5f0010e70 con 0x7fd5fc102810 2024-05-06T12:05:39.767 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.032+0000 7fd602b50700 1 -- 172.21.15.92:0/3165220640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5fc111e00 con 0x7fd5fc102810 2024-05-06T12:05:39.768 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.032+0000 7fd5f97fa700 1 -- 172.21.15.92:0/3165220640 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd5f001ae70 con 0x7fd5fc102810 2024-05-06T12:05:39.768 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.032+0000 7fd602b50700 1 -- 172.21.15.92:0/3165220640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5fc112970 con 0x7fd5fc102810 2024-05-06T12:05:39.768 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.033+0000 7fd5f97fa700 1 -- 172.21.15.92:0/3165220640 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fd5f00244d0 con 0x7fd5fc102810 2024-05-06T12:05:39.768 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.033+0000 7fd5f97fa700 1 --2- 172.21.15.92:0/3165220640 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7fd5ec036850 0x7fd5ec038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:39.768 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.033+0000 7fd5f97fa700 1 -- 172.21.15.92:0/3165220640 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fd5f004acc0 con 0x7fd5fc102810 2024-05-06T12:05:39.768 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.033+0000 7fd602b50700 1 -- 172.21.15.92:0/3165220640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5fc05f5c0 con 0x7fd5fc102810 2024-05-06T12:05:39.768 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.033+0000 7fd5fbfff700 1 --2- 172.21.15.92:0/3165220640 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7fd5ec036850 0x7fd5ec038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:39.768 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.033+0000 7fd5fbfff700 1 --2- 172.21.15.92:0/3165220640 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7fd5ec036850 0x7fd5ec038d00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8009990 tx=0x7fd5e8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:39.768 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.038+0000 7fd5f97fa700 1 -- 172.21.15.92:0/3165220640 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd5f002e440 con 0x7fd5fc102810 2024-05-06T12:05:39.768 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.249+0000 7fd602b50700 1 -- 172.21.15.92:0/3165220640 --> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5fc0ff230 con 0x7fd5ec036850 2024-05-06T12:05:39.768 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.254+0000 7fd5f97fa700 1 -- 172.21.15.92:0/3165220640 <== mgr.14120 v2:172.21.15.92:6800/3731057893 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fd5fc0ff230 con 0x7fd5ec036850 2024-05-06T12:05:39.768 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.256+0000 7fd602b50700 1 -- 172.21.15.92:0/3165220640 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7fd5ec036850 msgr2=0x7fd5ec038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:39.768 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.256+0000 7fd602b50700 1 --2- 172.21.15.92:0/3165220640 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7fd5ec036850 0x7fd5ec038d00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8009990 tx=0x7fd5e8006e40 comp rx=0 tx=0).stop 2024-05-06T12:05:39.768 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.256+0000 7fd602b50700 1 -- 172.21.15.92:0/3165220640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5fc102810 msgr2=0x7fd5fc1116c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:39.769 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.256+0000 7fd602b50700 1 --2- 172.21.15.92:0/3165220640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5fc102810 0x7fd5fc1116c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0006b00 tx=0x7fd5f0009250 comp rx=0 tx=0).stop 2024-05-06T12:05:39.769 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.256+0000 7fd602b50700 1 -- 172.21.15.92:0/3165220640 shutdown_connections 2024-05-06T12:05:39.769 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.256+0000 7fd602b50700 1 --2- 172.21.15.92:0/3165220640 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7fd5ec036850 0x7fd5ec038d00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:39.769 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.256+0000 7fd602b50700 1 --2- 172.21.15.92:0/3165220640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5fc102810 0x7fd5fc1116c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:39.769 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.256+0000 7fd602b50700 1 -- 172.21.15.92:0/3165220640 >> 172.21.15.92:0/3165220640 conn(0x7fd5fc0fdfd0 msgr2=0x7fd5fc0feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:39.769 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.256+0000 7fd602b50700 1 -- 172.21.15.92:0/3165220640 shutdown_connections 2024-05-06T12:05:39.769 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:39.256+0000 7fd602b50700 1 -- 172.21.15.92:0/3165220640 wait complete. 2024-05-06T12:05:39.769 INFO:teuthology.orchestra.run.smithi092.stdout:Deploying prometheus service with default placement... 2024-05-06T12:05:39.875 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:39 smithi092 ceph-mon[26954]: from='mgr.14120 172.21.15.92:0/2103511002' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:05:39.875 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:39 smithi092 ceph-mon[26954]: from='mgr.14120 172.21.15.92:0/2103511002' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:05:39.875 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:39 smithi092 ceph-mon[26954]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:05:39.875 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:39 smithi092 ceph-mon[26954]: Saving service crash spec with placement * 2024-05-06T12:05:39.875 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:39 smithi092 ceph-mon[26954]: from='mgr.14120 172.21.15.92:0/2103511002' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:05:41.677 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:41 smithi092 ceph-mon[26954]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:05:41.677 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:41 smithi092 ceph-mon[26954]: Saving service prometheus spec with placement count:1 2024-05-06T12:05:41.677 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:41 smithi092 ceph-mon[26954]: from='mgr.14120 172.21.15.92:0/2103511002' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:05:41.677 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:41 smithi092 ceph-mon[26954]: from='mgr.14120 172.21.15.92:0/2103511002' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:05:41.684 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-05-06T12:05:41.684 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.180+0000 7fe775daf700 1 Processor -- start 2024-05-06T12:05:41.684 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.180+0000 7fe775daf700 1 -- start start 2024-05-06T12:05:41.684 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.180+0000 7fe775daf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe770101750 0x7fe770101b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:41.684 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.180+0000 7fe775daf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7701020a0 con 0x7fe770101750 2024-05-06T12:05:41.685 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.180+0000 7fe76f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe770101750 0x7fe770101b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:41.685 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.180+0000 7fe76f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe770101750 0x7fe770101b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54054/0 (socket says 172.21.15.92:54054) 2024-05-06T12:05:41.685 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.180+0000 7fe76f7fe700 1 -- 172.21.15.92:0/607067000 learned_addr learned my addr 172.21.15.92:0/607067000 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:41.685 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.180+0000 7fe76f7fe700 1 -- 172.21.15.92:0/607067000 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7701021e0 con 0x7fe770101750 2024-05-06T12:05:41.685 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.181+0000 7fe76f7fe700 1 --2- 172.21.15.92:0/607067000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe770101750 0x7fe770101b60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe76000bb80 tx=0x7fe76000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de088cda0e8ffae7 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:41.685 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.181+0000 7fe76e7fc700 1 -- 172.21.15.92:0/607067000 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe76000dd00 con 0x7fe770101750 2024-05-06T12:05:41.685 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.181+0000 7fe76e7fc700 1 -- 172.21.15.92:0/607067000 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe76000de60 con 0x7fe770101750 2024-05-06T12:05:41.685 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.181+0000 7fe76e7fc700 1 -- 172.21.15.92:0/607067000 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe760018920 con 0x7fe770101750 2024-05-06T12:05:41.685 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.181+0000 7fe775daf700 1 -- 172.21.15.92:0/607067000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe770101750 msgr2=0x7fe770101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:41.685 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.181+0000 7fe775daf700 1 --2- 172.21.15.92:0/607067000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe770101750 0x7fe770101b60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe76000bb80 tx=0x7fe76000b4f0 comp rx=0 tx=0).stop 2024-05-06T12:05:41.685 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.181+0000 7fe775daf700 1 -- 172.21.15.92:0/607067000 shutdown_connections 2024-05-06T12:05:41.685 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.181+0000 7fe775daf700 1 --2- 172.21.15.92:0/607067000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe770101750 0x7fe770101b60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:41.685 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.181+0000 7fe775daf700 1 -- 172.21.15.92:0/607067000 >> 172.21.15.92:0/607067000 conn(0x7fe7700fcef0 msgr2=0x7fe7700ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:41.686 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.181+0000 7fe775daf700 1 -- 172.21.15.92:0/607067000 shutdown_connections 2024-05-06T12:05:41.686 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.181+0000 7fe775daf700 1 -- 172.21.15.92:0/607067000 wait complete. 2024-05-06T12:05:41.686 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.182+0000 7fe775daf700 1 Processor -- start 2024-05-06T12:05:41.686 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.182+0000 7fe775daf700 1 -- start start 2024-05-06T12:05:41.686 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.182+0000 7fe775daf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe770101750 0x7fe77010f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:41.686 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.182+0000 7fe775daf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77010f8f0 con 0x7fe770101750 2024-05-06T12:05:41.686 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.182+0000 7fe76f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe770101750 0x7fe77010f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:41.686 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.182+0000 7fe76f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe770101750 0x7fe77010f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54066/0 (socket says 172.21.15.92:54066) 2024-05-06T12:05:41.686 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.182+0000 7fe76f7fe700 1 -- 172.21.15.92:0/3973396978 learned_addr learned my addr 172.21.15.92:0/3973396978 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:41.686 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.183+0000 7fe76f7fe700 1 -- 172.21.15.92:0/3973396978 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe76000b790 con 0x7fe770101750 2024-05-06T12:05:41.686 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.183+0000 7fe76f7fe700 1 --2- 172.21.15.92:0/3973396978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe770101750 0x7fe77010f3b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe760006b00 tx=0x7fe76000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:41.686 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.183+0000 7fe757fff700 1 -- 172.21.15.92:0/3973396978 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe760010bd0 con 0x7fe770101750 2024-05-06T12:05:41.686 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.183+0000 7fe775daf700 1 -- 172.21.15.92:0/3973396978 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe77010faf0 con 0x7fe770101750 2024-05-06T12:05:41.687 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.183+0000 7fe757fff700 1 -- 172.21.15.92:0/3973396978 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe76001d050 con 0x7fe770101750 2024-05-06T12:05:41.687 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.183+0000 7fe757fff700 1 -- 172.21.15.92:0/3973396978 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe760022590 con 0x7fe770101750 2024-05-06T12:05:41.687 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.183+0000 7fe775daf700 1 -- 172.21.15.92:0/3973396978 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe770110660 con 0x7fe770101750 2024-05-06T12:05:41.687 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.184+0000 7fe757fff700 1 -- 172.21.15.92:0/3973396978 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fe760022810 con 0x7fe770101750 2024-05-06T12:05:41.687 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.184+0000 7fe757fff700 1 --2- 172.21.15.92:0/3973396978 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7fe758036850 0x7fe758038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:41.687 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.184+0000 7fe775daf700 1 -- 172.21.15.92:0/3973396978 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe77005f5c0 con 0x7fe770101750 2024-05-06T12:05:41.687 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.184+0000 7fe757fff700 1 -- 172.21.15.92:0/3973396978 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fe760014070 con 0x7fe770101750 2024-05-06T12:05:41.687 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.184+0000 7fe76effd700 1 --2- 172.21.15.92:0/3973396978 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7fe758036850 0x7fe758038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:41.687 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.184+0000 7fe76effd700 1 --2- 172.21.15.92:0/3973396978 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7fe758036850 0x7fe758038d00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe764009990 tx=0x7fe764006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:41.687 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.188+0000 7fe757fff700 1 -- 172.21.15.92:0/3973396978 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe76000e360 con 0x7fe770101750 2024-05-06T12:05:41.687 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.408+0000 7fe775daf700 1 -- 172.21.15.92:0/3973396978 --> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7fe770110a10 con 0x7fe758036850 2024-05-06T12:05:41.687 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.412+0000 7fe757fff700 1 -- 172.21.15.92:0/3973396978 <== mgr.14120 v2:172.21.15.92:6800/3731057893 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7fe770110a10 con 0x7fe758036850 2024-05-06T12:05:41.687 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.414+0000 7fe775daf700 1 -- 172.21.15.92:0/3973396978 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7fe758036850 msgr2=0x7fe758038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:41.687 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.414+0000 7fe775daf700 1 --2- 172.21.15.92:0/3973396978 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7fe758036850 0x7fe758038d00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe764009990 tx=0x7fe764006e40 comp rx=0 tx=0).stop 2024-05-06T12:05:41.687 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.414+0000 7fe775daf700 1 -- 172.21.15.92:0/3973396978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe770101750 msgr2=0x7fe77010f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:41.687 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.414+0000 7fe775daf700 1 --2- 172.21.15.92:0/3973396978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe770101750 0x7fe77010f3b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe760006b00 tx=0x7fe76000b4f0 comp rx=0 tx=0).stop 2024-05-06T12:05:41.688 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.414+0000 7fe775daf700 1 -- 172.21.15.92:0/3973396978 shutdown_connections 2024-05-06T12:05:41.688 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.414+0000 7fe775daf700 1 --2- 172.21.15.92:0/3973396978 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7fe758036850 0x7fe758038d00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:41.688 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.414+0000 7fe775daf700 1 --2- 172.21.15.92:0/3973396978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe770101750 0x7fe77010f3b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:41.688 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.414+0000 7fe775daf700 1 -- 172.21.15.92:0/3973396978 >> 172.21.15.92:0/3973396978 conn(0x7fe7700fcef0 msgr2=0x7fe770105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:41.688 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.415+0000 7fe775daf700 1 -- 172.21.15.92:0/3973396978 shutdown_connections 2024-05-06T12:05:41.688 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:40.415+0000 7fe775daf700 1 -- 172.21.15.92:0/3973396978 wait complete. 2024-05-06T12:05:41.688 INFO:teuthology.orchestra.run.smithi092.stdout:Deploying grafana service with default placement... 2024-05-06T12:05:42.950 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-05-06T12:05:42.950 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.162+0000 7f4a5427e700 1 Processor -- start 2024-05-06T12:05:42.950 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.162+0000 7f4a5427e700 1 -- start start 2024-05-06T12:05:42.950 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.163+0000 7f4a5427e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a4c076b70 0x7f4a4c105500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:42.950 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.163+0000 7f4a5427e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c105a40 con 0x7f4a4c076b70 2024-05-06T12:05:42.950 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.163+0000 7f4a5201a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a4c076b70 0x7f4a4c105500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:42.950 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.163+0000 7f4a5201a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a4c076b70 0x7f4a4c105500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54080/0 (socket says 172.21.15.92:54080) 2024-05-06T12:05:42.950 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.163+0000 7f4a5201a700 1 -- 172.21.15.92:0/4160339919 learned_addr learned my addr 172.21.15.92:0/4160339919 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:42.951 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.164+0000 7f4a5201a700 1 -- 172.21.15.92:0/4160339919 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4c105b80 con 0x7f4a4c076b70 2024-05-06T12:05:42.951 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.164+0000 7f4a5201a700 1 --2- 172.21.15.92:0/4160339919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a4c076b70 0x7f4a4c105500 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4a44009e50 tx=0x7f4a44009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa4ce833b51c118 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:42.951 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.164+0000 7f4a51018700 1 -- 172.21.15.92:0/4160339919 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4a440092e0 con 0x7f4a4c076b70 2024-05-06T12:05:42.951 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.164+0000 7f4a51018700 1 -- 172.21.15.92:0/4160339919 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a44012430 con 0x7f4a4c076b70 2024-05-06T12:05:42.951 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.164+0000 7f4a51018700 1 -- 172.21.15.92:0/4160339919 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4a440075a0 con 0x7f4a4c076b70 2024-05-06T12:05:42.951 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.165+0000 7f4a5427e700 1 -- 172.21.15.92:0/4160339919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a4c076b70 msgr2=0x7f4a4c105500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:42.951 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.165+0000 7f4a5427e700 1 --2- 172.21.15.92:0/4160339919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a4c076b70 0x7f4a4c105500 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4a44009e50 tx=0x7f4a44009840 comp rx=0 tx=0).stop 2024-05-06T12:05:42.951 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.165+0000 7f4a5427e700 1 -- 172.21.15.92:0/4160339919 shutdown_connections 2024-05-06T12:05:42.951 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.165+0000 7f4a5427e700 1 --2- 172.21.15.92:0/4160339919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a4c076b70 0x7f4a4c105500 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:42.951 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.165+0000 7f4a5427e700 1 -- 172.21.15.92:0/4160339919 >> 172.21.15.92:0/4160339919 conn(0x7f4a4c0fcef0 msgr2=0x7f4a4c0ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:42.951 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.165+0000 7f4a5427e700 1 -- 172.21.15.92:0/4160339919 shutdown_connections 2024-05-06T12:05:42.951 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.165+0000 7f4a5427e700 1 -- 172.21.15.92:0/4160339919 wait complete. 2024-05-06T12:05:42.952 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.165+0000 7f4a5427e700 1 Processor -- start 2024-05-06T12:05:42.952 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.165+0000 7f4a5427e700 1 -- start start 2024-05-06T12:05:42.952 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.165+0000 7f4a5427e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a4c076b70 0x7f4a4c1116a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:42.952 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.165+0000 7f4a5427e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c111be0 con 0x7f4a4c076b70 2024-05-06T12:05:42.952 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.166+0000 7f4a5201a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a4c076b70 0x7f4a4c1116a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:42.952 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.166+0000 7f4a5201a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a4c076b70 0x7f4a4c1116a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54086/0 (socket says 172.21.15.92:54086) 2024-05-06T12:05:42.952 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.166+0000 7f4a5201a700 1 -- 172.21.15.92:0/1390745166 learned_addr learned my addr 172.21.15.92:0/1390745166 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:42.952 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.166+0000 7f4a5201a700 1 -- 172.21.15.92:0/1390745166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a44009ae0 con 0x7f4a4c076b70 2024-05-06T12:05:42.952 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.166+0000 7f4a5201a700 1 --2- 172.21.15.92:0/1390745166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a4c076b70 0x7f4a4c1116a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4a44012f20 tx=0x7f4a44009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:42.952 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.166+0000 7f4a4affd700 1 -- 172.21.15.92:0/1390745166 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4a440092e0 con 0x7f4a4c076b70 2024-05-06T12:05:42.952 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.166+0000 7f4a5427e700 1 -- 172.21.15.92:0/1390745166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a4c111de0 con 0x7f4a4c076b70 2024-05-06T12:05:42.952 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.166+0000 7f4a4affd700 1 -- 172.21.15.92:0/1390745166 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a44007d30 con 0x7f4a4c076b70 2024-05-06T12:05:42.952 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.166+0000 7f4a5427e700 1 -- 172.21.15.92:0/1390745166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a4c112950 con 0x7f4a4c076b70 2024-05-06T12:05:42.953 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.166+0000 7f4a4affd700 1 -- 172.21.15.92:0/1390745166 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4a44025a00 con 0x7f4a4c076b70 2024-05-06T12:05:42.953 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.167+0000 7f4a4affd700 1 -- 172.21.15.92:0/1390745166 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f4a44019050 con 0x7f4a4c076b70 2024-05-06T12:05:42.953 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.167+0000 7f4a5427e700 1 -- 172.21.15.92:0/1390745166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a4c05f5c0 con 0x7f4a4c076b70 2024-05-06T12:05:42.953 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.167+0000 7f4a4affd700 1 --2- 172.21.15.92:0/1390745166 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f4a34036850 0x7f4a34038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:42.953 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.167+0000 7f4a4affd700 1 -- 172.21.15.92:0/1390745166 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f4a4404a030 con 0x7f4a4c076b70 2024-05-06T12:05:42.953 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.167+0000 7f4a51819700 1 --2- 172.21.15.92:0/1390745166 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f4a34036850 0x7f4a34038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:42.953 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.168+0000 7f4a51819700 1 --2- 172.21.15.92:0/1390745166 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f4a34036850 0x7f4a34038d00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4a40009990 tx=0x7f4a40006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:42.953 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.174+0000 7f4a4affd700 1 -- 172.21.15.92:0/1390745166 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4a4402d9a0 con 0x7f4a4c076b70 2024-05-06T12:05:42.953 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.387+0000 7f4a5427e700 1 -- 172.21.15.92:0/1390745166 --> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a4c0fe120 con 0x7f4a34036850 2024-05-06T12:05:42.953 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.392+0000 7f4a4affd700 1 -- 172.21.15.92:0/1390745166 <== mgr.14120 v2:172.21.15.92:6800/3731057893 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f4a4c0fe120 con 0x7f4a34036850 2024-05-06T12:05:42.953 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.394+0000 7f4a5427e700 1 -- 172.21.15.92:0/1390745166 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f4a34036850 msgr2=0x7f4a34038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:42.953 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.394+0000 7f4a5427e700 1 --2- 172.21.15.92:0/1390745166 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f4a34036850 0x7f4a34038d00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4a40009990 tx=0x7f4a40006e40 comp rx=0 tx=0).stop 2024-05-06T12:05:42.953 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.394+0000 7f4a5427e700 1 -- 172.21.15.92:0/1390745166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a4c076b70 msgr2=0x7f4a4c1116a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:42.954 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.394+0000 7f4a5427e700 1 --2- 172.21.15.92:0/1390745166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a4c076b70 0x7f4a4c1116a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4a44012f20 tx=0x7f4a44009840 comp rx=0 tx=0).stop 2024-05-06T12:05:42.954 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.394+0000 7f4a5427e700 1 -- 172.21.15.92:0/1390745166 shutdown_connections 2024-05-06T12:05:42.954 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.394+0000 7f4a5427e700 1 --2- 172.21.15.92:0/1390745166 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f4a34036850 0x7f4a34038d00 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:42.954 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.394+0000 7f4a5427e700 1 --2- 172.21.15.92:0/1390745166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a4c076b70 0x7f4a4c1116a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:42.954 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.394+0000 7f4a5427e700 1 -- 172.21.15.92:0/1390745166 >> 172.21.15.92:0/1390745166 conn(0x7f4a4c0fcef0 msgr2=0x7f4a4c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:42.954 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.394+0000 7f4a5427e700 1 -- 172.21.15.92:0/1390745166 shutdown_connections 2024-05-06T12:05:42.954 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:42.394+0000 7f4a5427e700 1 -- 172.21.15.92:0/1390745166 wait complete. 2024-05-06T12:05:42.954 INFO:teuthology.orchestra.run.smithi092.stdout:Deploying node-exporter service with default placement... 2024-05-06T12:05:43.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:43 smithi092 ceph-mon[26954]: from='mgr.14120 172.21.15.92:0/2103511002' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:05:43.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:43 smithi092 ceph-mon[26954]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:05:43.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:43 smithi092 ceph-mon[26954]: Saving service grafana spec with placement count:1 2024-05-06T12:05:43.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:43 smithi092 ceph-mon[26954]: from='mgr.14120 172.21.15.92:0/2103511002' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:05:44.761 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:44 smithi092 ceph-mon[26954]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:05:44.762 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:44 smithi092 ceph-mon[26954]: Saving service node-exporter spec with placement * 2024-05-06T12:05:44.762 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:44 smithi092 ceph-mon[26954]: from='mgr.14120 172.21.15.92:0/2103511002' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:05:45.164 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-05-06T12:05:45.164 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.406+0000 7f86ae12a700 1 Processor -- start 2024-05-06T12:05:45.164 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.406+0000 7f86ae12a700 1 -- start start 2024-05-06T12:05:45.164 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.406+0000 7f86ae12a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86a8101730 0x7f86a8101b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:45.164 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.406+0000 7f86ae12a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a8102080 con 0x7f86a8101730 2024-05-06T12:05:45.164 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.406+0000 7f86a77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86a8101730 0x7f86a8101b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:45.164 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.406+0000 7f86a77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86a8101730 0x7f86a8101b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51730/0 (socket says 172.21.15.92:51730) 2024-05-06T12:05:45.164 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.406+0000 7f86a77fe700 1 -- 172.21.15.92:0/3640683998 learned_addr learned my addr 172.21.15.92:0/3640683998 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:45.164 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.407+0000 7f86a77fe700 1 -- 172.21.15.92:0/3640683998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a81021c0 con 0x7f86a8101730 2024-05-06T12:05:45.164 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.407+0000 7f86a77fe700 1 --2- 172.21.15.92:0/3640683998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86a8101730 0x7f86a8101b40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f86980098e0 tx=0x7f8698009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcac80ebcfd9dd89 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:45.165 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.407+0000 7f86a67fc700 1 -- 172.21.15.92:0/3640683998 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f869800b860 con 0x7f86a8101730 2024-05-06T12:05:45.165 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.407+0000 7f86a67fc700 1 -- 172.21.15.92:0/3640683998 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f869800be90 con 0x7f86a8101730 2024-05-06T12:05:45.165 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.407+0000 7f86a67fc700 1 -- 172.21.15.92:0/3640683998 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f86980103c0 con 0x7f86a8101730 2024-05-06T12:05:45.165 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.408+0000 7f86ae12a700 1 -- 172.21.15.92:0/3640683998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86a8101730 msgr2=0x7f86a8101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:45.165 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.408+0000 7f86ae12a700 1 --2- 172.21.15.92:0/3640683998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86a8101730 0x7f86a8101b40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f86980098e0 tx=0x7f8698009250 comp rx=0 tx=0).stop 2024-05-06T12:05:45.165 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.408+0000 7f86ae12a700 1 -- 172.21.15.92:0/3640683998 shutdown_connections 2024-05-06T12:05:45.165 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.408+0000 7f86ae12a700 1 --2- 172.21.15.92:0/3640683998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86a8101730 0x7f86a8101b40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:45.165 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.408+0000 7f86ae12a700 1 -- 172.21.15.92:0/3640683998 >> 172.21.15.92:0/3640683998 conn(0x7f86a80fcf30 msgr2=0x7f86a80ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:45.165 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.408+0000 7f86ae12a700 1 -- 172.21.15.92:0/3640683998 shutdown_connections 2024-05-06T12:05:45.165 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.408+0000 7f86ae12a700 1 -- 172.21.15.92:0/3640683998 wait complete. 2024-05-06T12:05:45.165 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.408+0000 7f86ae12a700 1 Processor -- start 2024-05-06T12:05:45.165 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.409+0000 7f86ae12a700 1 -- start start 2024-05-06T12:05:45.165 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.409+0000 7f86ae12a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86a8101730 0x7f86a810f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:45.165 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.409+0000 7f86ae12a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a810f8d0 con 0x7f86a8101730 2024-05-06T12:05:45.166 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.409+0000 7f86a77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86a8101730 0x7f86a810f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:45.166 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.409+0000 7f86a77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86a8101730 0x7f86a810f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51736/0 (socket says 172.21.15.92:51736) 2024-05-06T12:05:45.166 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.409+0000 7f86a77fe700 1 -- 172.21.15.92:0/3834345686 learned_addr learned my addr 172.21.15.92:0/3834345686 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:45.166 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.409+0000 7f86a77fe700 1 -- 172.21.15.92:0/3834345686 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86980094f0 con 0x7f86a8101730 2024-05-06T12:05:45.166 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.409+0000 7f86a77fe700 1 --2- 172.21.15.92:0/3834345686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86a8101730 0x7f86a810f390 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8698011680 tx=0x7f8698009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:45.166 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.410+0000 7f868ffff700 1 -- 172.21.15.92:0/3834345686 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8698010890 con 0x7f86a8101730 2024-05-06T12:05:45.166 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.410+0000 7f868ffff700 1 -- 172.21.15.92:0/3834345686 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8698010ec0 con 0x7f86a8101730 2024-05-06T12:05:45.166 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.410+0000 7f86ae12a700 1 -- 172.21.15.92:0/3834345686 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86a810fad0 con 0x7f86a8101730 2024-05-06T12:05:45.166 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.410+0000 7f868ffff700 1 -- 172.21.15.92:0/3834345686 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f86980243f0 con 0x7f86a8101730 2024-05-06T12:05:45.166 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.410+0000 7f86ae12a700 1 -- 172.21.15.92:0/3834345686 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a8110640 con 0x7f86a8101730 2024-05-06T12:05:45.166 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.411+0000 7f868ffff700 1 -- 172.21.15.92:0/3834345686 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f869802d3f0 con 0x7f86a8101730 2024-05-06T12:05:45.166 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.411+0000 7f86ae12a700 1 -- 172.21.15.92:0/3834345686 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86a805f5c0 con 0x7f86a8101730 2024-05-06T12:05:45.166 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.411+0000 7f868ffff700 1 --2- 172.21.15.92:0/3834345686 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f8690038a90 0x7f869003af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:45.166 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.411+0000 7f868ffff700 1 -- 172.21.15.92:0/3834345686 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f869804a250 con 0x7f86a8101730 2024-05-06T12:05:45.167 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.411+0000 7f86a6ffd700 1 --2- 172.21.15.92:0/3834345686 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f8690038a90 0x7f869003af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:45.167 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.412+0000 7f86a6ffd700 1 --2- 172.21.15.92:0/3834345686 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f8690038a90 0x7f869003af40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f869c009990 tx=0x7f869c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:45.167 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.418+0000 7f868ffff700 1 -- 172.21.15.92:0/3834345686 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8698006d70 con 0x7f86a8101730 2024-05-06T12:05:45.167 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.630+0000 7f86ae12a700 1 -- 172.21.15.92:0/3834345686 --> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f86a81109d0 con 0x7f8690038a90 2024-05-06T12:05:45.167 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.634+0000 7f868ffff700 1 -- 172.21.15.92:0/3834345686 <== mgr.14120 v2:172.21.15.92:6800/3731057893 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f86a81109d0 con 0x7f8690038a90 2024-05-06T12:05:45.167 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.636+0000 7f86ae12a700 1 -- 172.21.15.92:0/3834345686 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f8690038a90 msgr2=0x7f869003af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:45.167 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.636+0000 7f86ae12a700 1 --2- 172.21.15.92:0/3834345686 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f8690038a90 0x7f869003af40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f869c009990 tx=0x7f869c006e40 comp rx=0 tx=0).stop 2024-05-06T12:05:45.167 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.636+0000 7f86ae12a700 1 -- 172.21.15.92:0/3834345686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86a8101730 msgr2=0x7f86a810f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:45.167 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.636+0000 7f86ae12a700 1 --2- 172.21.15.92:0/3834345686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86a8101730 0x7f86a810f390 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8698011680 tx=0x7f8698009250 comp rx=0 tx=0).stop 2024-05-06T12:05:45.167 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.636+0000 7f86ae12a700 1 -- 172.21.15.92:0/3834345686 shutdown_connections 2024-05-06T12:05:45.167 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.636+0000 7f86ae12a700 1 --2- 172.21.15.92:0/3834345686 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f8690038a90 0x7f869003af40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:45.167 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.636+0000 7f86ae12a700 1 --2- 172.21.15.92:0/3834345686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86a8101730 0x7f86a810f390 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:45.167 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.636+0000 7f86ae12a700 1 -- 172.21.15.92:0/3834345686 >> 172.21.15.92:0/3834345686 conn(0x7f86a80fcf30 msgr2=0x7f86a8105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:45.167 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.636+0000 7f86ae12a700 1 -- 172.21.15.92:0/3834345686 shutdown_connections 2024-05-06T12:05:45.168 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:43.636+0000 7f86ae12a700 1 -- 172.21.15.92:0/3834345686 wait complete. 2024-05-06T12:05:45.168 INFO:teuthology.orchestra.run.smithi092.stdout:Deploying alertmanager service with default placement... 2024-05-06T12:05:46.329 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-05-06T12:05:46.329 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.594+0000 7f3af90bd700 1 Processor -- start 2024-05-06T12:05:46.329 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.594+0000 7f3af90bd700 1 -- start start 2024-05-06T12:05:46.329 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.594+0000 7f3af90bd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3af4102810 0x7f3af4102c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:46.329 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.594+0000 7f3af90bd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af4103160 con 0x7f3af4102810 2024-05-06T12:05:46.329 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.594+0000 7f3af2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3af4102810 0x7f3af4102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:46.329 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.594+0000 7f3af2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3af4102810 0x7f3af4102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51744/0 (socket says 172.21.15.92:51744) 2024-05-06T12:05:46.330 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.594+0000 7f3af2d9d700 1 -- 172.21.15.92:0/3294182154 learned_addr learned my addr 172.21.15.92:0/3294182154 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:46.330 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.594+0000 7f3af2d9d700 1 -- 172.21.15.92:0/3294182154 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af41032a0 con 0x7f3af4102810 2024-05-06T12:05:46.330 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.594+0000 7f3af2d9d700 1 --2- 172.21.15.92:0/3294182154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3af4102810 0x7f3af4102c20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3ae80098e0 tx=0x7f3ae8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970a2218d8630cb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:46.330 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.595+0000 7f3af1d9b700 1 -- 172.21.15.92:0/3294182154 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ae800b860 con 0x7f3af4102810 2024-05-06T12:05:46.330 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.595+0000 7f3af1d9b700 1 -- 172.21.15.92:0/3294182154 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ae800be90 con 0x7f3af4102810 2024-05-06T12:05:46.330 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.595+0000 7f3af1d9b700 1 -- 172.21.15.92:0/3294182154 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ae80103c0 con 0x7f3af4102810 2024-05-06T12:05:46.330 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.595+0000 7f3af90bd700 1 -- 172.21.15.92:0/3294182154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3af4102810 msgr2=0x7f3af4102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:46.330 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.595+0000 7f3af90bd700 1 --2- 172.21.15.92:0/3294182154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3af4102810 0x7f3af4102c20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3ae80098e0 tx=0x7f3ae8009250 comp rx=0 tx=0).stop 2024-05-06T12:05:46.330 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.595+0000 7f3af90bd700 1 -- 172.21.15.92:0/3294182154 shutdown_connections 2024-05-06T12:05:46.330 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.595+0000 7f3af90bd700 1 --2- 172.21.15.92:0/3294182154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3af4102810 0x7f3af4102c20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:46.330 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.595+0000 7f3af90bd700 1 -- 172.21.15.92:0/3294182154 >> 172.21.15.92:0/3294182154 conn(0x7f3af40fdfd0 msgr2=0x7f3af41003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:46.330 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.595+0000 7f3af90bd700 1 -- 172.21.15.92:0/3294182154 shutdown_connections 2024-05-06T12:05:46.331 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.595+0000 7f3af90bd700 1 -- 172.21.15.92:0/3294182154 wait complete. 2024-05-06T12:05:46.331 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.596+0000 7f3af90bd700 1 Processor -- start 2024-05-06T12:05:46.331 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.596+0000 7f3af90bd700 1 -- start start 2024-05-06T12:05:46.331 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.596+0000 7f3af90bd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3af4102810 0x7f3af41116c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:46.331 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.596+0000 7f3af90bd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af4111c00 con 0x7f3af4102810 2024-05-06T12:05:46.331 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.596+0000 7f3af2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3af4102810 0x7f3af41116c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:46.331 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.596+0000 7f3af2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3af4102810 0x7f3af41116c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51756/0 (socket says 172.21.15.92:51756) 2024-05-06T12:05:46.331 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.596+0000 7f3af2d9d700 1 -- 172.21.15.92:0/2706490108 learned_addr learned my addr 172.21.15.92:0/2706490108 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:46.331 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.597+0000 7f3af2d9d700 1 -- 172.21.15.92:0/2706490108 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae80094f0 con 0x7f3af4102810 2024-05-06T12:05:46.331 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.597+0000 7f3af2d9d700 1 --2- 172.21.15.92:0/2706490108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3af4102810 0x7f3af41116c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8011c10 tx=0x7f3ae8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:46.331 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.597+0000 7f3adb7fe700 1 -- 172.21.15.92:0/2706490108 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ae8010890 con 0x7f3af4102810 2024-05-06T12:05:46.331 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.597+0000 7f3adb7fe700 1 -- 172.21.15.92:0/2706490108 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ae8010ec0 con 0x7f3af4102810 2024-05-06T12:05:46.332 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.597+0000 7f3af90bd700 1 -- 172.21.15.92:0/2706490108 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3af4111e00 con 0x7f3af4102810 2024-05-06T12:05:46.332 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.597+0000 7f3adb7fe700 1 -- 172.21.15.92:0/2706490108 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ae80243f0 con 0x7f3af4102810 2024-05-06T12:05:46.332 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.597+0000 7f3af90bd700 1 -- 172.21.15.92:0/2706490108 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af4112970 con 0x7f3af4102810 2024-05-06T12:05:46.332 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.598+0000 7f3adb7fe700 1 -- 172.21.15.92:0/2706490108 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f3ae802d3f0 con 0x7f3af4102810 2024-05-06T12:05:46.332 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.598+0000 7f3af90bd700 1 -- 172.21.15.92:0/2706490108 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3af405f5c0 con 0x7f3af4102810 2024-05-06T12:05:46.332 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.598+0000 7f3adb7fe700 1 --2- 172.21.15.92:0/2706490108 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f3adc036850 0x7f3adc038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:46.332 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.598+0000 7f3adb7fe700 1 -- 172.21.15.92:0/2706490108 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f3ae804a3b0 con 0x7f3af4102810 2024-05-06T12:05:46.332 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.598+0000 7f3af259c700 1 --2- 172.21.15.92:0/2706490108 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f3adc036850 0x7f3adc038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:46.332 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.599+0000 7f3af259c700 1 --2- 172.21.15.92:0/2706490108 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f3adc036850 0x7f3adc038d00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4009990 tx=0x7f3ae4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:46.332 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.603+0000 7f3adb7fe700 1 -- 172.21.15.92:0/2706490108 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3ae802b8e0 con 0x7f3af4102810 2024-05-06T12:05:46.332 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.801+0000 7f3af90bd700 1 -- 172.21.15.92:0/2706490108 --> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7f3af40ff230 con 0x7f3adc036850 2024-05-06T12:05:46.332 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.805+0000 7f3adb7fe700 1 -- 172.21.15.92:0/2706490108 <== mgr.14120 v2:172.21.15.92:6800/3731057893 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f3af40ff230 con 0x7f3adc036850 2024-05-06T12:05:46.333 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.807+0000 7f3af90bd700 1 -- 172.21.15.92:0/2706490108 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f3adc036850 msgr2=0x7f3adc038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:46.333 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.807+0000 7f3af90bd700 1 --2- 172.21.15.92:0/2706490108 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f3adc036850 0x7f3adc038d00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4009990 tx=0x7f3ae4006e40 comp rx=0 tx=0).stop 2024-05-06T12:05:46.333 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.807+0000 7f3af90bd700 1 -- 172.21.15.92:0/2706490108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3af4102810 msgr2=0x7f3af41116c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:46.333 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.807+0000 7f3af90bd700 1 --2- 172.21.15.92:0/2706490108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3af4102810 0x7f3af41116c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8011c10 tx=0x7f3ae8009250 comp rx=0 tx=0).stop 2024-05-06T12:05:46.333 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.807+0000 7f3af90bd700 1 -- 172.21.15.92:0/2706490108 shutdown_connections 2024-05-06T12:05:46.333 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.807+0000 7f3af90bd700 1 --2- 172.21.15.92:0/2706490108 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f3adc036850 0x7f3adc038d00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:46.333 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.807+0000 7f3af90bd700 1 --2- 172.21.15.92:0/2706490108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3af4102810 0x7f3af41116c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:46.333 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.807+0000 7f3af90bd700 1 -- 172.21.15.92:0/2706490108 >> 172.21.15.92:0/2706490108 conn(0x7f3af40fdfd0 msgr2=0x7f3af40feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:46.333 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.808+0000 7f3af90bd700 1 -- 172.21.15.92:0/2706490108 shutdown_connections 2024-05-06T12:05:46.333 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:45.808+0000 7f3af90bd700 1 -- 172.21.15.92:0/2706490108 wait complete. 2024-05-06T12:05:46.908 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:46 smithi092 ceph-mon[26954]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:05:46.908 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:46 smithi092 ceph-mon[26954]: Saving service alertmanager spec with placement count:1 2024-05-06T12:05:46.908 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:46 smithi092 ceph-mon[26954]: from='mgr.14120 172.21.15.92:0/2103511002' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:05:46.909 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:46 smithi092 ceph-mon[26954]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:05:47.476 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.710+0000 7f1a02b60700 1 Processor -- start 2024-05-06T12:05:47.476 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.710+0000 7f1a02b60700 1 -- start start 2024-05-06T12:05:47.476 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.711+0000 7f1a02b60700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19fc078ca0 0x7f19fc077250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:47.477 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.711+0000 7f1a02b60700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19fc0790b0 con 0x7f19fc078ca0 2024-05-06T12:05:47.477 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.711+0000 7f1a008fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19fc078ca0 0x7f19fc077250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:47.477 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.711+0000 7f1a008fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19fc078ca0 0x7f19fc077250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51758/0 (socket says 172.21.15.92:51758) 2024-05-06T12:05:47.477 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.711+0000 7f1a008fc700 1 -- 172.21.15.92:0/3936918903 learned_addr learned my addr 172.21.15.92:0/3936918903 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:47.477 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.711+0000 7f1a008fc700 1 -- 172.21.15.92:0/3936918903 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19fc077790 con 0x7f19fc078ca0 2024-05-06T12:05:47.477 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.711+0000 7f1a008fc700 1 --2- 172.21.15.92:0/3936918903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19fc078ca0 0x7f19fc077250 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f19f00098e0 tx=0x7f19f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65eda3eb9194955f server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:47.477 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.711+0000 7f19fb7fe700 1 -- 172.21.15.92:0/3936918903 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f19f000b860 con 0x7f19fc078ca0 2024-05-06T12:05:47.477 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.711+0000 7f19fb7fe700 1 -- 172.21.15.92:0/3936918903 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f19f000be90 con 0x7f19fc078ca0 2024-05-06T12:05:47.477 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.711+0000 7f19fb7fe700 1 -- 172.21.15.92:0/3936918903 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f19f00103c0 con 0x7f19fc078ca0 2024-05-06T12:05:47.477 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.712+0000 7f1a02b60700 1 -- 172.21.15.92:0/3936918903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19fc078ca0 msgr2=0x7f19fc077250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:47.477 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.712+0000 7f1a02b60700 1 --2- 172.21.15.92:0/3936918903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19fc078ca0 0x7f19fc077250 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f19f00098e0 tx=0x7f19f0009250 comp rx=0 tx=0).stop 2024-05-06T12:05:47.477 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.712+0000 7f1a02b60700 1 -- 172.21.15.92:0/3936918903 shutdown_connections 2024-05-06T12:05:47.477 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.712+0000 7f1a02b60700 1 --2- 172.21.15.92:0/3936918903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19fc078ca0 0x7f19fc077250 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:47.478 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.712+0000 7f1a02b60700 1 -- 172.21.15.92:0/3936918903 >> 172.21.15.92:0/3936918903 conn(0x7f19fc0fe060 msgr2=0x7f19fc100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:47.478 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.712+0000 7f1a02b60700 1 -- 172.21.15.92:0/3936918903 shutdown_connections 2024-05-06T12:05:47.478 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.712+0000 7f1a02b60700 1 -- 172.21.15.92:0/3936918903 wait complete. 2024-05-06T12:05:47.478 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.713+0000 7f1a02b60700 1 Processor -- start 2024-05-06T12:05:47.478 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.713+0000 7f1a02b60700 1 -- start start 2024-05-06T12:05:47.478 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.713+0000 7f1a02b60700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19fc078ca0 0x7f19fc113830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:47.478 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.713+0000 7f1a02b60700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19fc113d70 con 0x7f19fc078ca0 2024-05-06T12:05:47.478 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.713+0000 7f1a008fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19fc078ca0 0x7f19fc113830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:47.478 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.713+0000 7f1a008fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19fc078ca0 0x7f19fc113830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51760/0 (socket says 172.21.15.92:51760) 2024-05-06T12:05:47.478 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.713+0000 7f1a008fc700 1 -- 172.21.15.92:0/577149323 learned_addr learned my addr 172.21.15.92:0/577149323 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:47.478 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.713+0000 7f1a008fc700 1 -- 172.21.15.92:0/577149323 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f00094f0 con 0x7f19fc078ca0 2024-05-06T12:05:47.478 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.713+0000 7f1a008fc700 1 --2- 172.21.15.92:0/577149323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19fc078ca0 0x7f19fc113830 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f19f0006d40 tx=0x7f19f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:47.478 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.713+0000 7f19f97fa700 1 -- 172.21.15.92:0/577149323 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f19f0010840 con 0x7f19fc078ca0 2024-05-06T12:05:47.479 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.714+0000 7f19f97fa700 1 -- 172.21.15.92:0/577149323 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f19f0010e70 con 0x7f19fc078ca0 2024-05-06T12:05:47.479 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.714+0000 7f19f97fa700 1 -- 172.21.15.92:0/577149323 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f19f0024ea0 con 0x7f19fc078ca0 2024-05-06T12:05:47.479 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.714+0000 7f1a02b60700 1 -- 172.21.15.92:0/577149323 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19fc113f70 con 0x7f19fc078ca0 2024-05-06T12:05:47.479 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.714+0000 7f1a02b60700 1 -- 172.21.15.92:0/577149323 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19fc114ae0 con 0x7f19fc078ca0 2024-05-06T12:05:47.479 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.714+0000 7f19f97fa700 1 -- 172.21.15.92:0/577149323 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f19f001a4c0 con 0x7f19fc078ca0 2024-05-06T12:05:47.479 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.714+0000 7f19f97fa700 1 --2- 172.21.15.92:0/577149323 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f19ec036850 0x7f19ec038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:47.479 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.715+0000 7f19f97fa700 1 -- 172.21.15.92:0/577149323 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f19f004a880 con 0x7f19fc078ca0 2024-05-06T12:05:47.479 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.715+0000 7f1a02b60700 1 -- 172.21.15.92:0/577149323 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19fc05f5c0 con 0x7f19fc078ca0 2024-05-06T12:05:47.479 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.715+0000 7f19fbfff700 1 --2- 172.21.15.92:0/577149323 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f19ec036850 0x7f19ec038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:47.479 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.715+0000 7f19fbfff700 1 --2- 172.21.15.92:0/577149323 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f19ec036850 0x7f19ec038d00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f19e8009990 tx=0x7f19e8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:47.479 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.719+0000 7f19f97fa700 1 -- 172.21.15.92:0/577149323 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f19f0023d20 con 0x7f19fc078ca0 2024-05-06T12:05:47.479 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.905+0000 7f1a02b60700 1 -- 172.21.15.92:0/577149323 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f19fc04bed0 con 0x7f19fc078ca0 2024-05-06T12:05:47.479 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.911+0000 7f19f97fa700 1 -- 172.21.15.92:0/577149323 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f19f002ee00 con 0x7f19fc078ca0 2024-05-06T12:05:47.480 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.913+0000 7f1a02b60700 1 -- 172.21.15.92:0/577149323 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f19ec036850 msgr2=0x7f19ec038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:47.480 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.913+0000 7f1a02b60700 1 --2- 172.21.15.92:0/577149323 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f19ec036850 0x7f19ec038d00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f19e8009990 tx=0x7f19e8006e40 comp rx=0 tx=0).stop 2024-05-06T12:05:47.480 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.913+0000 7f1a02b60700 1 -- 172.21.15.92:0/577149323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19fc078ca0 msgr2=0x7f19fc113830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:47.480 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.913+0000 7f1a02b60700 1 --2- 172.21.15.92:0/577149323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19fc078ca0 0x7f19fc113830 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f19f0006d40 tx=0x7f19f0009250 comp rx=0 tx=0).stop 2024-05-06T12:05:47.480 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.913+0000 7f1a02b60700 1 -- 172.21.15.92:0/577149323 shutdown_connections 2024-05-06T12:05:47.480 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.913+0000 7f1a02b60700 1 --2- 172.21.15.92:0/577149323 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f19ec036850 0x7f19ec038d00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:47.480 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.913+0000 7f1a02b60700 1 --2- 172.21.15.92:0/577149323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19fc078ca0 0x7f19fc113830 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:47.480 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.913+0000 7f1a02b60700 1 -- 172.21.15.92:0/577149323 >> 172.21.15.92:0/577149323 conn(0x7f19fc0fe060 msgr2=0x7f19fc0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:47.480 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.913+0000 7f1a02b60700 1 -- 172.21.15.92:0/577149323 shutdown_connections 2024-05-06T12:05:47.480 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:46.913+0000 7f1a02b60700 1 -- 172.21.15.92:0/577149323 wait complete. 2024-05-06T12:05:48.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:47 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/577149323' entity='client.admin' 2024-05-06T12:05:48.700 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.892+0000 7f567782d700 1 Processor -- start 2024-05-06T12:05:48.700 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.892+0000 7f567782d700 1 -- start start 2024-05-06T12:05:48.701 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.892+0000 7f567782d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5670100cd0 0x7f56701010e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:48.701 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.892+0000 7f567782d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5670101620 con 0x7f5670100cd0 2024-05-06T12:05:48.701 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.892+0000 7f56755c9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5670100cd0 0x7f56701010e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:48.701 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.892+0000 7f56755c9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5670100cd0 0x7f56701010e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51774/0 (socket says 172.21.15.92:51774) 2024-05-06T12:05:48.701 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.892+0000 7f56755c9700 1 -- 172.21.15.92:0/443794636 learned_addr learned my addr 172.21.15.92:0/443794636 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:48.701 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.893+0000 7f56755c9700 1 -- 172.21.15.92:0/443794636 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5670101760 con 0x7f5670100cd0 2024-05-06T12:05:48.701 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.893+0000 7f56755c9700 1 --2- 172.21.15.92:0/443794636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5670100cd0 0x7f56701010e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f566c0098e0 tx=0x7f566c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b111d8f9d1336916 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:48.701 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.893+0000 7f5667fff700 1 -- 172.21.15.92:0/443794636 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f566c00b860 con 0x7f5670100cd0 2024-05-06T12:05:48.701 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.893+0000 7f5667fff700 1 -- 172.21.15.92:0/443794636 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f566c00be90 con 0x7f5670100cd0 2024-05-06T12:05:48.701 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.893+0000 7f5667fff700 1 -- 172.21.15.92:0/443794636 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f566c0103c0 con 0x7f5670100cd0 2024-05-06T12:05:48.701 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.893+0000 7f567782d700 1 -- 172.21.15.92:0/443794636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5670100cd0 msgr2=0x7f56701010e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:48.701 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.893+0000 7f567782d700 1 --2- 172.21.15.92:0/443794636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5670100cd0 0x7f56701010e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f566c0098e0 tx=0x7f566c009250 comp rx=0 tx=0).stop 2024-05-06T12:05:48.701 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.894+0000 7f567782d700 1 -- 172.21.15.92:0/443794636 shutdown_connections 2024-05-06T12:05:48.702 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.894+0000 7f567782d700 1 --2- 172.21.15.92:0/443794636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5670100cd0 0x7f56701010e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:48.702 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.894+0000 7f567782d700 1 -- 172.21.15.92:0/443794636 >> 172.21.15.92:0/443794636 conn(0x7f56700fc6f0 msgr2=0x7f56700feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:48.702 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.894+0000 7f567782d700 1 -- 172.21.15.92:0/443794636 shutdown_connections 2024-05-06T12:05:48.702 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.894+0000 7f567782d700 1 -- 172.21.15.92:0/443794636 wait complete. 2024-05-06T12:05:48.702 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.894+0000 7f567782d700 1 Processor -- start 2024-05-06T12:05:48.702 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.894+0000 7f567782d700 1 -- start start 2024-05-06T12:05:48.702 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.894+0000 7f567782d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5670100cd0 0x7f56701115d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:48.702 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.894+0000 7f567782d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5670111b10 con 0x7f5670100cd0 2024-05-06T12:05:48.702 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.894+0000 7f56755c9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5670100cd0 0x7f56701115d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:48.702 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.894+0000 7f56755c9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5670100cd0 0x7f56701115d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51790/0 (socket says 172.21.15.92:51790) 2024-05-06T12:05:48.702 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.894+0000 7f56755c9700 1 -- 172.21.15.92:0/2507406569 learned_addr learned my addr 172.21.15.92:0/2507406569 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:48.702 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.895+0000 7f56755c9700 1 -- 172.21.15.92:0/2507406569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f566c0094f0 con 0x7f5670100cd0 2024-05-06T12:05:48.702 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.895+0000 7f56755c9700 1 --2- 172.21.15.92:0/2507406569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5670100cd0 0x7f56701115d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f566c011680 tx=0x7f566c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:48.702 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.895+0000 7f5665ffb700 1 -- 172.21.15.92:0/2507406569 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f566c010890 con 0x7f5670100cd0 2024-05-06T12:05:48.703 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.895+0000 7f5665ffb700 1 -- 172.21.15.92:0/2507406569 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f566c010ec0 con 0x7f5670100cd0 2024-05-06T12:05:48.703 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.895+0000 7f5665ffb700 1 -- 172.21.15.92:0/2507406569 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f566c0243f0 con 0x7f5670100cd0 2024-05-06T12:05:48.703 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.895+0000 7f567782d700 1 -- 172.21.15.92:0/2507406569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5670111d10 con 0x7f5670100cd0 2024-05-06T12:05:48.703 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.895+0000 7f567782d700 1 -- 172.21.15.92:0/2507406569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5670112880 con 0x7f5670100cd0 2024-05-06T12:05:48.703 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.896+0000 7f5665ffb700 1 -- 172.21.15.92:0/2507406569 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f566c0109f0 con 0x7f5670100cd0 2024-05-06T12:05:48.703 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.896+0000 7f5665ffb700 1 --2- 172.21.15.92:0/2507406569 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f565c0368a0 0x7f565c038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:48.703 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.896+0000 7f5665ffb700 1 -- 172.21.15.92:0/2507406569 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f566c04a200 con 0x7f5670100cd0 2024-05-06T12:05:48.703 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.896+0000 7f567782d700 1 -- 172.21.15.92:0/2507406569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f567005f5c0 con 0x7f5670100cd0 2024-05-06T12:05:48.703 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.896+0000 7f5674dc8700 1 --2- 172.21.15.92:0/2507406569 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f565c0368a0 0x7f565c038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:48.704 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.897+0000 7f5674dc8700 1 --2- 172.21.15.92:0/2507406569 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f565c0368a0 0x7f565c038d50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5660009990 tx=0x7f5660006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:48.704 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:47.901+0000 7f5665ffb700 1 -- 172.21.15.92:0/2507406569 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f566c01a430 con 0x7f5670100cd0 2024-05-06T12:05:48.704 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:48.087+0000 7f567782d700 1 -- 172.21.15.92:0/2507406569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f567004bed0 con 0x7f5670100cd0 2024-05-06T12:05:48.704 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:48.093+0000 7f5665ffb700 1 -- 172.21.15.92:0/2507406569 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f566c02b2f0 con 0x7f5670100cd0 2024-05-06T12:05:48.704 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:48.095+0000 7f567782d700 1 -- 172.21.15.92:0/2507406569 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f565c0368a0 msgr2=0x7f565c038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:48.704 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:48.095+0000 7f567782d700 1 --2- 172.21.15.92:0/2507406569 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f565c0368a0 0x7f565c038d50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5660009990 tx=0x7f5660006e40 comp rx=0 tx=0).stop 2024-05-06T12:05:48.704 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:48.095+0000 7f567782d700 1 -- 172.21.15.92:0/2507406569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5670100cd0 msgr2=0x7f56701115d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:48.704 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:48.095+0000 7f567782d700 1 --2- 172.21.15.92:0/2507406569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5670100cd0 0x7f56701115d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f566c011680 tx=0x7f566c009250 comp rx=0 tx=0).stop 2024-05-06T12:05:48.704 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:48.095+0000 7f567782d700 1 -- 172.21.15.92:0/2507406569 shutdown_connections 2024-05-06T12:05:48.704 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:48.095+0000 7f567782d700 1 --2- 172.21.15.92:0/2507406569 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f565c0368a0 0x7f565c038d50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:48.704 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:48.095+0000 7f567782d700 1 --2- 172.21.15.92:0/2507406569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5670100cd0 0x7f56701115d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:48.704 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:48.095+0000 7f567782d700 1 -- 172.21.15.92:0/2507406569 >> 172.21.15.92:0/2507406569 conn(0x7f56700fc6f0 msgr2=0x7f56700fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:48.704 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:48.095+0000 7f567782d700 1 -- 172.21.15.92:0/2507406569 shutdown_connections 2024-05-06T12:05:48.704 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:48.095+0000 7f567782d700 1 -- 172.21.15.92:0/2507406569 wait complete. 2024-05-06T12:05:48.705 INFO:teuthology.orchestra.run.smithi092.stdout:Enabling the dashboard module... 2024-05-06T12:05:49.376 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:49 smithi092 ceph-mon[26954]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:05:49.376 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:49 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/2507406569' entity='client.admin' 2024-05-06T12:05:50.357 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:50 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/3679320516' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-06T12:05:50.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.126+0000 7f2d87c57700 1 Processor -- start 2024-05-06T12:05:50.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.127+0000 7f2d87c57700 1 -- start start 2024-05-06T12:05:50.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.127+0000 7f2d87c57700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d80078c60 0x7f2d80077210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:50.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.127+0000 7f2d87c57700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d80079070 con 0x7f2d80078c60 2024-05-06T12:05:50.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.127+0000 7f2d859f3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d80078c60 0x7f2d80077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:50.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.127+0000 7f2d859f3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d80078c60 0x7f2d80077210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51804/0 (socket says 172.21.15.92:51804) 2024-05-06T12:05:50.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.127+0000 7f2d859f3700 1 -- 172.21.15.92:0/561627008 learned_addr learned my addr 172.21.15.92:0/561627008 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:50.658 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.127+0000 7f2d859f3700 1 -- 172.21.15.92:0/561627008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d80077750 con 0x7f2d80078c60 2024-05-06T12:05:50.658 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.128+0000 7f2d859f3700 1 --2- 172.21.15.92:0/561627008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d80078c60 0x7f2d80077210 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2d700098e0 tx=0x7f2d70009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973d2a8784740858 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:50.658 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.128+0000 7f2d849f1700 1 -- 172.21.15.92:0/561627008 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2d7000b860 con 0x7f2d80078c60 2024-05-06T12:05:50.658 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.128+0000 7f2d849f1700 1 -- 172.21.15.92:0/561627008 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d7000be90 con 0x7f2d80078c60 2024-05-06T12:05:50.658 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.128+0000 7f2d849f1700 1 -- 172.21.15.92:0/561627008 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2d700103c0 con 0x7f2d80078c60 2024-05-06T12:05:50.658 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.128+0000 7f2d87c57700 1 -- 172.21.15.92:0/561627008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d80078c60 msgr2=0x7f2d80077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:50.658 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.128+0000 7f2d87c57700 1 --2- 172.21.15.92:0/561627008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d80078c60 0x7f2d80077210 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2d700098e0 tx=0x7f2d70009250 comp rx=0 tx=0).stop 2024-05-06T12:05:50.658 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.128+0000 7f2d87c57700 1 -- 172.21.15.92:0/561627008 shutdown_connections 2024-05-06T12:05:50.658 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.128+0000 7f2d87c57700 1 --2- 172.21.15.92:0/561627008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d80078c60 0x7f2d80077210 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:50.658 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.128+0000 7f2d87c57700 1 -- 172.21.15.92:0/561627008 >> 172.21.15.92:0/561627008 conn(0x7f2d800fe060 msgr2=0x7f2d80100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:50.658 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.129+0000 7f2d87c57700 1 -- 172.21.15.92:0/561627008 shutdown_connections 2024-05-06T12:05:50.658 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.129+0000 7f2d87c57700 1 -- 172.21.15.92:0/561627008 wait complete. 2024-05-06T12:05:50.658 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.129+0000 7f2d87c57700 1 Processor -- start 2024-05-06T12:05:50.658 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.129+0000 7f2d87c57700 1 -- start start 2024-05-06T12:05:50.659 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.129+0000 7f2d87c57700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d80078c60 0x7f2d80113840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:50.659 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.129+0000 7f2d87c57700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d80113d80 con 0x7f2d80078c60 2024-05-06T12:05:50.659 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.129+0000 7f2d859f3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d80078c60 0x7f2d80113840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:50.659 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.129+0000 7f2d859f3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d80078c60 0x7f2d80113840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51812/0 (socket says 172.21.15.92:51812) 2024-05-06T12:05:50.659 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.129+0000 7f2d859f3700 1 -- 172.21.15.92:0/3679320516 learned_addr learned my addr 172.21.15.92:0/3679320516 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:50.659 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.129+0000 7f2d859f3700 1 -- 172.21.15.92:0/3679320516 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d700094f0 con 0x7f2d80078c60 2024-05-06T12:05:50.659 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.130+0000 7f2d859f3700 1 --2- 172.21.15.92:0/3679320516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d80078c60 0x7f2d80113840 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2d70006d40 tx=0x7f2d70009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:50.659 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.130+0000 7f2d767fc700 1 -- 172.21.15.92:0/3679320516 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2d70010840 con 0x7f2d80078c60 2024-05-06T12:05:50.659 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.130+0000 7f2d767fc700 1 -- 172.21.15.92:0/3679320516 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d70010e70 con 0x7f2d80078c60 2024-05-06T12:05:50.659 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.130+0000 7f2d87c57700 1 -- 172.21.15.92:0/3679320516 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d80113f80 con 0x7f2d80078c60 2024-05-06T12:05:50.659 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.130+0000 7f2d767fc700 1 -- 172.21.15.92:0/3679320516 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2d70024ea0 con 0x7f2d80078c60 2024-05-06T12:05:50.659 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.130+0000 7f2d87c57700 1 -- 172.21.15.92:0/3679320516 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d80114af0 con 0x7f2d80078c60 2024-05-06T12:05:50.659 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.131+0000 7f2d767fc700 1 -- 172.21.15.92:0/3679320516 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f2d7001a510 con 0x7f2d80078c60 2024-05-06T12:05:50.659 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.131+0000 7f2d87c57700 1 -- 172.21.15.92:0/3679320516 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d8005f5c0 con 0x7f2d80078c60 2024-05-06T12:05:50.660 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.131+0000 7f2d767fc700 1 --2- 172.21.15.92:0/3679320516 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f2d6c036800 0x7f2d6c038cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:50.660 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.131+0000 7f2d767fc700 1 -- 172.21.15.92:0/3679320516 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f2d7004b7b0 con 0x7f2d80078c60 2024-05-06T12:05:50.660 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.131+0000 7f2d851f2700 1 --2- 172.21.15.92:0/3679320516 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f2d6c036800 0x7f2d6c038cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:50.660 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.131+0000 7f2d851f2700 1 --2- 172.21.15.92:0/3679320516 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f2d6c036800 0x7f2d6c038cb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c009990 tx=0x7f2d7c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:50.660 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.135+0000 7f2d767fc700 1 -- 172.21.15.92:0/3679320516 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2d70018940 con 0x7f2d80078c60 2024-05-06T12:05:50.660 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:49.373+0000 7f2d87c57700 1 -- 172.21.15.92:0/3679320516 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f2d8004bed0 con 0x7f2d80078c60 2024-05-06T12:05:50.660 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:50.100+0000 7f2d767fc700 1 -- 172.21.15.92:0/3679320516 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 9) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f2d70024780 con 0x7f2d80078c60 2024-05-06T12:05:50.660 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:50.101+0000 7f2d767fc700 1 -- 172.21.15.92:0/3679320516 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f2d70018350 con 0x7f2d80078c60 2024-05-06T12:05:50.660 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:50.104+0000 7f2d87c57700 1 -- 172.21.15.92:0/3679320516 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f2d6c036800 msgr2=0x7f2d6c038cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:50.660 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:50.104+0000 7f2d87c57700 1 --2- 172.21.15.92:0/3679320516 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f2d6c036800 0x7f2d6c038cb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c009990 tx=0x7f2d7c006e40 comp rx=0 tx=0).stop 2024-05-06T12:05:50.660 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:50.104+0000 7f2d87c57700 1 -- 172.21.15.92:0/3679320516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d80078c60 msgr2=0x7f2d80113840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:50.660 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:50.104+0000 7f2d87c57700 1 --2- 172.21.15.92:0/3679320516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d80078c60 0x7f2d80113840 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2d70006d40 tx=0x7f2d70009250 comp rx=0 tx=0).stop 2024-05-06T12:05:50.660 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:50.104+0000 7f2d87c57700 1 -- 172.21.15.92:0/3679320516 shutdown_connections 2024-05-06T12:05:50.660 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:50.104+0000 7f2d87c57700 1 --2- 172.21.15.92:0/3679320516 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f2d6c036800 0x7f2d6c038cb0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:50.661 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:50.104+0000 7f2d87c57700 1 --2- 172.21.15.92:0/3679320516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d80078c60 0x7f2d80113840 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:50.661 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:50.104+0000 7f2d87c57700 1 -- 172.21.15.92:0/3679320516 >> 172.21.15.92:0/3679320516 conn(0x7f2d800fe060 msgr2=0x7f2d800febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:50.661 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:50.105+0000 7f2d87c57700 1 -- 172.21.15.92:0/3679320516 shutdown_connections 2024-05-06T12:05:50.661 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:50.105+0000 7f2d87c57700 1 -- 172.21.15.92:0/3679320516 wait complete. 2024-05-06T12:05:51.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:51 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/3679320516' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-06T12:05:51.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:51 smithi092 ceph-mon[26954]: mgrmap e9: smithi092.gyzeqb(active, since 24s) 2024-05-06T12:05:51.890 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-05-06T12:05:51.890 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-06T12:05:51.890 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-06T12:05:51.890 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "active_name": "smithi092.gyzeqb", 2024-05-06T12:05:51.891 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-06T12:05:51.891 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-05-06T12:05:51.891 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.055+0000 7f62505c6700 1 Processor -- start 2024-05-06T12:05:51.891 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.055+0000 7f62505c6700 1 -- start start 2024-05-06T12:05:51.891 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.055+0000 7f62505c6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6248102c70 0x7f6248103080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:51.891 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.055+0000 7f62505c6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6248103650 con 0x7f6248102c70 2024-05-06T12:05:51.891 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.056+0000 7f624e362700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6248102c70 0x7f6248103080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:51.891 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.056+0000 7f624e362700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6248102c70 0x7f6248103080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51830/0 (socket says 172.21.15.92:51830) 2024-05-06T12:05:51.891 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.056+0000 7f624e362700 1 -- 172.21.15.92:0/2714116630 learned_addr learned my addr 172.21.15.92:0/2714116630 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:51.891 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.056+0000 7f624e362700 1 -- 172.21.15.92:0/2714116630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6248103790 con 0x7f6248102c70 2024-05-06T12:05:51.891 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.056+0000 7f624e362700 1 --2- 172.21.15.92:0/2714116630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6248102c70 0x7f6248103080 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f62440098e0 tx=0x7f6244009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e91149fd37df9b56 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:51.891 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.056+0000 7f624d360700 1 -- 172.21.15.92:0/2714116630 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f624400b860 con 0x7f6248102c70 2024-05-06T12:05:51.891 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.056+0000 7f624d360700 1 -- 172.21.15.92:0/2714116630 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f624400be90 con 0x7f6248102c70 2024-05-06T12:05:51.892 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.056+0000 7f624d360700 1 -- 172.21.15.92:0/2714116630 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f62440103c0 con 0x7f6248102c70 2024-05-06T12:05:51.892 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.057+0000 7f62505c6700 1 -- 172.21.15.92:0/2714116630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6248102c70 msgr2=0x7f6248103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:51.892 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.057+0000 7f62505c6700 1 --2- 172.21.15.92:0/2714116630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6248102c70 0x7f6248103080 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f62440098e0 tx=0x7f6244009250 comp rx=0 tx=0).stop 2024-05-06T12:05:51.892 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.057+0000 7f62505c6700 1 -- 172.21.15.92:0/2714116630 shutdown_connections 2024-05-06T12:05:51.892 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.057+0000 7f62505c6700 1 --2- 172.21.15.92:0/2714116630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6248102c70 0x7f6248103080 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:51.892 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.057+0000 7f62505c6700 1 -- 172.21.15.92:0/2714116630 >> 172.21.15.92:0/2714116630 conn(0x7f62480fc6b0 msgr2=0x7f62480feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:51.892 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.057+0000 7f62505c6700 1 -- 172.21.15.92:0/2714116630 shutdown_connections 2024-05-06T12:05:51.892 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.057+0000 7f62505c6700 1 -- 172.21.15.92:0/2714116630 wait complete. 2024-05-06T12:05:51.892 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.057+0000 7f62505c6700 1 Processor -- start 2024-05-06T12:05:51.892 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.057+0000 7f62505c6700 1 -- start start 2024-05-06T12:05:51.892 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.058+0000 7f62505c6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6248102c70 0x7f6248113800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:51.892 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.058+0000 7f62505c6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6248113d40 con 0x7f6248102c70 2024-05-06T12:05:51.892 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.058+0000 7f624e362700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6248102c70 0x7f6248113800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:51.892 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.058+0000 7f624e362700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6248102c70 0x7f6248113800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51840/0 (socket says 172.21.15.92:51840) 2024-05-06T12:05:51.892 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.058+0000 7f624e362700 1 -- 172.21.15.92:0/755845432 learned_addr learned my addr 172.21.15.92:0/755845432 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:51.893 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.058+0000 7f624e362700 1 -- 172.21.15.92:0/755845432 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62440094f0 con 0x7f6248102c70 2024-05-06T12:05:51.893 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.058+0000 7f624e362700 1 --2- 172.21.15.92:0/755845432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6248102c70 0x7f6248113800 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f62440094c0 tx=0x7f6244009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:51.893 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.058+0000 7f623effd700 1 -- 172.21.15.92:0/755845432 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6244010890 con 0x7f6248102c70 2024-05-06T12:05:51.893 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.058+0000 7f62505c6700 1 -- 172.21.15.92:0/755845432 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6248113f40 con 0x7f6248102c70 2024-05-06T12:05:51.893 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.058+0000 7f623effd700 1 -- 172.21.15.92:0/755845432 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6244010ec0 con 0x7f6248102c70 2024-05-06T12:05:51.893 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.058+0000 7f623effd700 1 -- 172.21.15.92:0/755845432 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f62440243f0 con 0x7f6248102c70 2024-05-06T12:05:51.893 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.058+0000 7f62505c6700 1 -- 172.21.15.92:0/755845432 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6248114ab0 con 0x7f6248102c70 2024-05-06T12:05:51.893 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.059+0000 7f62505c6700 1 -- 172.21.15.92:0/755845432 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f624805f5c0 con 0x7f6248102c70 2024-05-06T12:05:51.893 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.059+0000 7f623effd700 1 -- 172.21.15.92:0/755845432 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f624402d3f0 con 0x7f6248102c70 2024-05-06T12:05:51.893 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.059+0000 7f623effd700 1 --2- 172.21.15.92:0/755845432 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f62340368a0 0x7f6234038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:51.893 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.059+0000 7f623effd700 1 -- 172.21.15.92:0/755845432 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f624404a320 con 0x7f6248102c70 2024-05-06T12:05:51.893 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.059+0000 7f624db61700 1 -- 172.21.15.92:0/755845432 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f62340368a0 msgr2=0x7f6234038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/3731057893 2024-05-06T12:05:51.893 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.060+0000 7f624db61700 1 --2- 172.21.15.92:0/755845432 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f62340368a0 0x7f6234038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-06T12:05:51.894 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.064+0000 7f623effd700 1 -- 172.21.15.92:0/755845432 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f624402b8e0 con 0x7f6248102c70 2024-05-06T12:05:51.894 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.261+0000 7f624db61700 1 -- 172.21.15.92:0/755845432 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f62340368a0 msgr2=0x7f6234038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/3731057893 2024-05-06T12:05:51.894 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.261+0000 7f624db61700 1 --2- 172.21.15.92:0/755845432 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f62340368a0 0x7f6234038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-06T12:05:51.894 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.316+0000 7f62505c6700 1 -- 172.21.15.92:0/755845432 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f6248114df0 con 0x7f6248102c70 2024-05-06T12:05:51.894 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.318+0000 7f623effd700 1 -- 172.21.15.92:0/755845432 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f624402b2f0 con 0x7f6248102c70 2024-05-06T12:05:51.894 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.320+0000 7f62505c6700 1 -- 172.21.15.92:0/755845432 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f62340368a0 msgr2=0x7f6234038d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-06T12:05:51.894 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.320+0000 7f62505c6700 1 --2- 172.21.15.92:0/755845432 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f62340368a0 0x7f6234038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:51.894 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.320+0000 7f62505c6700 1 -- 172.21.15.92:0/755845432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6248102c70 msgr2=0x7f6248113800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:51.894 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.320+0000 7f62505c6700 1 --2- 172.21.15.92:0/755845432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6248102c70 0x7f6248113800 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f62440094c0 tx=0x7f6244009250 comp rx=0 tx=0).stop 2024-05-06T12:05:51.894 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.321+0000 7f62505c6700 1 -- 172.21.15.92:0/755845432 shutdown_connections 2024-05-06T12:05:51.894 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.321+0000 7f62505c6700 1 --2- 172.21.15.92:0/755845432 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7f62340368a0 0x7f6234038d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:51.894 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.321+0000 7f62505c6700 1 --2- 172.21.15.92:0/755845432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6248102c70 0x7f6248113800 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f62440094c0 tx=0x7f6244009250 comp rx=0 tx=0).stop 2024-05-06T12:05:51.894 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.321+0000 7f62505c6700 1 -- 172.21.15.92:0/755845432 >> 172.21.15.92:0/755845432 conn(0x7f62480fc6b0 msgr2=0x7f62480fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:51.894 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.321+0000 7f62505c6700 1 -- 172.21.15.92:0/755845432 shutdown_connections 2024-05-06T12:05:51.895 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:51.321+0000 7f62505c6700 1 -- 172.21.15.92:0/755845432 wait complete. 2024-05-06T12:05:51.895 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for the mgr to restart... 2024-05-06T12:05:51.895 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for mgr epoch 9... 2024-05-06T12:05:52.277 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:52 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/755845432' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-06T12:05:56.631 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:56 smithi092 ceph-mon[26954]: Active manager daemon smithi092.gyzeqb restarted 2024-05-06T12:05:56.631 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:56 smithi092 ceph-mon[26954]: Activating manager daemon smithi092.gyzeqb 2024-05-06T12:05:56.631 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:56 smithi092 ceph-mon[26954]: osdmap e3: 0 total, 0 up, 0 in 2024-05-06T12:05:56.631 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:56 smithi092 ceph-mon[26954]: mgrmap e10: smithi092.gyzeqb(active, starting, since 0.00324165s) 2024-05-06T12:05:56.631 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:56 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi092"}]: dispatch 2024-05-06T12:05:56.632 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:56 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mgr metadata", "who": "smithi092.gyzeqb", "id": "smithi092.gyzeqb"}]: dispatch 2024-05-06T12:05:56.632 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:56 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-06T12:05:56.632 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:56 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-06T12:05:56.632 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:56 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-06T12:05:56.632 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:56 smithi092 ceph-mon[26954]: Manager daemon smithi092.gyzeqb is now available 2024-05-06T12:05:56.632 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:56 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:05:56.632 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:56 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:05:56.632 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:56 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi092.gyzeqb/mirror_snapshot_schedule"}]: dispatch 2024-05-06T12:05:56.632 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:56 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi092.gyzeqb/trash_purge_schedule"}]: dispatch 2024-05-06T12:05:58.284 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-05-06T12:05:58.284 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-06T12:05:58.284 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-06T12:05:58.284 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-05-06T12:05:58.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.346+0000 7fda59fca700 1 Processor -- start 2024-05-06T12:05:58.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.346+0000 7fda59fca700 1 -- start start 2024-05-06T12:05:58.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.346+0000 7fda59fca700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda540712d0 0x7fda54107870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:58.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.346+0000 7fda59fca700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda54103260 con 0x7fda540712d0 2024-05-06T12:05:58.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.346+0000 7fda537fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda540712d0 0x7fda54107870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:58.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.346+0000 7fda537fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda540712d0 0x7fda54107870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51844/0 (socket says 172.21.15.92:51844) 2024-05-06T12:05:58.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.346+0000 7fda537fe700 1 -- 172.21.15.92:0/2166442406 learned_addr learned my addr 172.21.15.92:0/2166442406 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:58.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.346+0000 7fda537fe700 1 -- 172.21.15.92:0/2166442406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda54107db0 con 0x7fda540712d0 2024-05-06T12:05:58.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.347+0000 7fda537fe700 1 --2- 172.21.15.92:0/2166442406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda540712d0 0x7fda54107870 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fda440098e0 tx=0x7fda44009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc576fbc6f9d6b server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:58.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.347+0000 7fda527fc700 1 -- 172.21.15.92:0/2166442406 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fda4400b860 con 0x7fda540712d0 2024-05-06T12:05:58.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.347+0000 7fda527fc700 1 -- 172.21.15.92:0/2166442406 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda4400be90 con 0x7fda540712d0 2024-05-06T12:05:58.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.347+0000 7fda527fc700 1 -- 172.21.15.92:0/2166442406 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fda440103c0 con 0x7fda540712d0 2024-05-06T12:05:58.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.347+0000 7fda59fca700 1 -- 172.21.15.92:0/2166442406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda540712d0 msgr2=0x7fda54107870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:58.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.347+0000 7fda59fca700 1 --2- 172.21.15.92:0/2166442406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda540712d0 0x7fda54107870 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fda440098e0 tx=0x7fda44009250 comp rx=0 tx=0).stop 2024-05-06T12:05:58.286 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.348+0000 7fda59fca700 1 -- 172.21.15.92:0/2166442406 shutdown_connections 2024-05-06T12:05:58.286 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.348+0000 7fda59fca700 1 --2- 172.21.15.92:0/2166442406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda540712d0 0x7fda54107870 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:58.286 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.348+0000 7fda59fca700 1 -- 172.21.15.92:0/2166442406 >> 172.21.15.92:0/2166442406 conn(0x7fda540fcea0 msgr2=0x7fda540ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:58.286 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.348+0000 7fda59fca700 1 -- 172.21.15.92:0/2166442406 shutdown_connections 2024-05-06T12:05:58.286 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.348+0000 7fda59fca700 1 -- 172.21.15.92:0/2166442406 wait complete. 2024-05-06T12:05:58.286 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.348+0000 7fda59fca700 1 Processor -- start 2024-05-06T12:05:58.286 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.348+0000 7fda59fca700 1 -- start start 2024-05-06T12:05:58.286 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.348+0000 7fda59fca700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda540712d0 0x7fda54113870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:58.286 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.349+0000 7fda59fca700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda54113db0 con 0x7fda540712d0 2024-05-06T12:05:58.286 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.349+0000 7fda537fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda540712d0 0x7fda54113870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:58.286 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.349+0000 7fda537fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda540712d0 0x7fda54113870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51858/0 (socket says 172.21.15.92:51858) 2024-05-06T12:05:58.287 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.349+0000 7fda537fe700 1 -- 172.21.15.92:0/2009608810 learned_addr learned my addr 172.21.15.92:0/2009608810 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:58.287 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.349+0000 7fda537fe700 1 -- 172.21.15.92:0/2009608810 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda440094f0 con 0x7fda540712d0 2024-05-06T12:05:58.287 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.349+0000 7fda537fe700 1 --2- 172.21.15.92:0/2009608810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda540712d0 0x7fda54113870 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fda440094c0 tx=0x7fda44009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:58.287 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.349+0000 7fda3bfff700 1 -- 172.21.15.92:0/2009608810 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fda44011920 con 0x7fda540712d0 2024-05-06T12:05:58.287 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.349+0000 7fda3bfff700 1 -- 172.21.15.92:0/2009608810 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda44010d60 con 0x7fda540712d0 2024-05-06T12:05:58.287 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.349+0000 7fda3bfff700 1 -- 172.21.15.92:0/2009608810 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fda4401ae70 con 0x7fda540712d0 2024-05-06T12:05:58.287 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.349+0000 7fda59fca700 1 -- 172.21.15.92:0/2009608810 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda54113fb0 con 0x7fda540712d0 2024-05-06T12:05:58.287 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.349+0000 7fda59fca700 1 -- 172.21.15.92:0/2009608810 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda54114b20 con 0x7fda540712d0 2024-05-06T12:05:58.287 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.350+0000 7fda3bfff700 1 -- 172.21.15.92:0/2009608810 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 43389+0+0 (secure 0 0 0) 0x7fda4402e440 con 0x7fda540712d0 2024-05-06T12:05:58.287 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.350+0000 7fda3bfff700 1 --2- 172.21.15.92:0/2009608810 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7fda3c0368a0 0x7fda3c038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:58.287 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.350+0000 7fda3bfff700 1 -- 172.21.15.92:0/2009608810 --> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fda3c039460 con 0x7fda3c0368a0 2024-05-06T12:05:58.287 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.350+0000 7fda3bfff700 1 -- 172.21.15.92:0/2009608810 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fda4404a6b0 con 0x7fda540712d0 2024-05-06T12:05:58.287 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.350+0000 7fda52ffd700 1 -- 172.21.15.92:0/2009608810 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7fda3c0368a0 msgr2=0x7fda3c038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/3731057893 2024-05-06T12:05:58.287 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.350+0000 7fda52ffd700 1 --2- 172.21.15.92:0/2009608810 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7fda3c0368a0 0x7fda3c038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-06T12:05:58.288 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.551+0000 7fda52ffd700 1 -- 172.21.15.92:0/2009608810 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7fda3c0368a0 msgr2=0x7fda3c038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/3731057893 2024-05-06T12:05:58.288 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.551+0000 7fda52ffd700 1 --2- 172.21.15.92:0/2009608810 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7fda3c0368a0 0x7fda3c038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-06T12:05:58.288 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.952+0000 7fda52ffd700 1 -- 172.21.15.92:0/2009608810 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7fda3c0368a0 msgr2=0x7fda3c038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/3731057893 2024-05-06T12:05:58.288 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:52.952+0000 7fda52ffd700 1 --2- 172.21.15.92:0/2009608810 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7fda3c0368a0 0x7fda3c038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-06T12:05:58.288 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:53.753+0000 7fda52ffd700 1 -- 172.21.15.92:0/2009608810 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7fda3c0368a0 msgr2=0x7fda3c038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/3731057893 2024-05-06T12:05:58.288 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:53.753+0000 7fda52ffd700 1 --2- 172.21.15.92:0/2009608810 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7fda3c0368a0 0x7fda3c038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-06T12:05:58.288 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:55.354+0000 7fda52ffd700 1 -- 172.21.15.92:0/2009608810 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7fda3c0368a0 msgr2=0x7fda3c038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/3731057893 2024-05-06T12:05:58.288 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:55.354+0000 7fda52ffd700 1 --2- 172.21.15.92:0/2009608810 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7fda3c0368a0 0x7fda3c038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-06T12:05:58.288 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:56.547+0000 7fda3bfff700 1 -- 172.21.15.92:0/2009608810 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mgrmap(e 10) v1 ==== 43156+0+0 (secure 0 0 0) 0x7fda4401a430 con 0x7fda540712d0 2024-05-06T12:05:58.288 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:56.548+0000 7fda3bfff700 1 -- 172.21.15.92:0/2009608810 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7fda3c0368a0 msgr2=0x7fda3c038d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-06T12:05:58.288 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:56.548+0000 7fda3bfff700 1 --2- 172.21.15.92:0/2009608810 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7fda3c0368a0 0x7fda3c038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:58.288 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:57.549+0000 7fda3bfff700 1 -- 172.21.15.92:0/2009608810 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 11) v1 ==== 43283+0+0 (secure 0 0 0) 0x7fda4401a750 con 0x7fda540712d0 2024-05-06T12:05:58.288 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:57.550+0000 7fda3bfff700 1 --2- 172.21.15.92:0/2009608810 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7fda3c03a1d0 0x7fda3c03c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:58.288 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:57.550+0000 7fda3bfff700 1 -- 172.21.15.92:0/2009608810 --> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fda3c039460 con 0x7fda3c03a1d0 2024-05-06T12:05:58.289 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:57.550+0000 7fda52ffd700 1 --2- 172.21.15.92:0/2009608810 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7fda3c03a1d0 0x7fda3c03c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:58.289 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:57.551+0000 7fda52ffd700 1 --2- 172.21.15.92:0/2009608810 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7fda3c03a1d0 0x7fda3c03c5b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fda480039d0 tx=0x7fda48009350 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:58.289 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:57.551+0000 7fda3bfff700 1 -- 172.21.15.92:0/2009608810 <== mgr.14162 v2:172.21.15.92:6800/543287293 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7fda3c039460 con 0x7fda3c03a1d0 2024-05-06T12:05:58.289 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:57.557+0000 7fda59fca700 1 -- 172.21.15.92:0/2009608810 --> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fda54114440 con 0x7fda3c03a1d0 2024-05-06T12:05:58.289 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:57.557+0000 7fda3bfff700 1 -- 172.21.15.92:0/2009608810 <== mgr.14162 v2:172.21.15.92:6800/543287293 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fda54114440 con 0x7fda3c03a1d0 2024-05-06T12:05:58.289 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:57.558+0000 7fda59fca700 1 -- 172.21.15.92:0/2009608810 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7fda3c03a1d0 msgr2=0x7fda3c03c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:58.289 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:57.558+0000 7fda59fca700 1 --2- 172.21.15.92:0/2009608810 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7fda3c03a1d0 0x7fda3c03c5b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fda480039d0 tx=0x7fda48009350 comp rx=0 tx=0).stop 2024-05-06T12:05:58.289 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:57.558+0000 7fda59fca700 1 -- 172.21.15.92:0/2009608810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda540712d0 msgr2=0x7fda54113870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:58.289 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:57.558+0000 7fda59fca700 1 --2- 172.21.15.92:0/2009608810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda540712d0 0x7fda54113870 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fda440094c0 tx=0x7fda44009250 comp rx=0 tx=0).stop 2024-05-06T12:05:58.289 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:57.558+0000 7fda59fca700 1 -- 172.21.15.92:0/2009608810 shutdown_connections 2024-05-06T12:05:58.289 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:57.558+0000 7fda59fca700 1 --2- 172.21.15.92:0/2009608810 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7fda3c03a1d0 0x7fda3c03c5b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:58.289 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:57.558+0000 7fda59fca700 1 --2- 172.21.15.92:0/2009608810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda540712d0 0x7fda54113870 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:58.289 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:57.558+0000 7fda59fca700 1 --2- 172.21.15.92:0/2009608810 >> [v2:172.21.15.92:6800/3731057893,v1:172.21.15.92:6801/3731057893] conn(0x7fda3c0368a0 0x7fda3c038d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:58.289 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:57.558+0000 7fda59fca700 1 -- 172.21.15.92:0/2009608810 >> 172.21.15.92:0/2009608810 conn(0x7fda540fcea0 msgr2=0x7fda540fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:58.290 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:57.558+0000 7fda59fca700 1 -- 172.21.15.92:0/2009608810 shutdown_connections 2024-05-06T12:05:58.290 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:57.558+0000 7fda59fca700 1 -- 172.21.15.92:0/2009608810 wait complete. 2024-05-06T12:05:58.290 INFO:teuthology.orchestra.run.smithi092.stdout:mgr epoch 9 is available 2024-05-06T12:05:58.290 INFO:teuthology.orchestra.run.smithi092.stdout:Generating a dashboard self-signed certificate... 2024-05-06T12:05:58.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:58 smithi092 ceph-mon[26954]: [06/May/2024:12:05:57] ENGINE Bus STARTING 2024-05-06T12:05:58.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:58 smithi092 ceph-mon[26954]: [06/May/2024:12:05:57] ENGINE Serving on https://172.21.15.92:7150 2024-05-06T12:05:58.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:58 smithi092 ceph-mon[26954]: [06/May/2024:12:05:57] ENGINE Bus STARTED 2024-05-06T12:05:58.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:58 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:05:58.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:58 smithi092 ceph-mon[26954]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-06T12:05:58.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:58 smithi092 ceph-mon[26954]: mgrmap e11: smithi092.gyzeqb(active, since 1.00609s) 2024-05-06T12:05:58.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:58 smithi092 ceph-mon[26954]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-06T12:05:58.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:58 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:05:59.917 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:59 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:05:59.917 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:59 smithi092 ceph-mon[26954]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:05:59.917 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:59 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:05:59.917 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:59 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:05:59.917 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:05:59 smithi092 ceph-mon[26954]: mgrmap e12: smithi092.gyzeqb(active, since 2s) 2024-05-06T12:05:59.924 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-06T12:05:59.924 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.894+0000 7f9363847700 1 Processor -- start 2024-05-06T12:05:59.924 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.894+0000 7f9363847700 1 -- start start 2024-05-06T12:05:59.925 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.894+0000 7f9363847700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f935c102850 0x7f935c102c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:59.925 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.894+0000 7f9363847700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c1031a0 con 0x7f935c102850 2024-05-06T12:05:59.925 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.894+0000 7f93615e3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f935c102850 0x7f935c102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:59.925 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.894+0000 7f93615e3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f935c102850 0x7f935c102c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59114/0 (socket says 172.21.15.92:59114) 2024-05-06T12:05:59.925 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.894+0000 7f93615e3700 1 -- 172.21.15.92:0/3748450650 learned_addr learned my addr 172.21.15.92:0/3748450650 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:59.925 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.895+0000 7f93615e3700 1 -- 172.21.15.92:0/3748450650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935c1032e0 con 0x7f935c102850 2024-05-06T12:05:59.925 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.895+0000 7f93615e3700 1 --2- 172.21.15.92:0/3748450650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f935c102850 0x7f935c102c60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f93580098e0 tx=0x7f9358009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13cc610a4ebcc27f server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:59.925 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.895+0000 7f9353fff700 1 -- 172.21.15.92:0/3748450650 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f935800b860 con 0x7f935c102850 2024-05-06T12:05:59.925 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.895+0000 7f9353fff700 1 -- 172.21.15.92:0/3748450650 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f935800be90 con 0x7f935c102850 2024-05-06T12:05:59.925 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.895+0000 7f9353fff700 1 -- 172.21.15.92:0/3748450650 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f93580103c0 con 0x7f935c102850 2024-05-06T12:05:59.925 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.896+0000 7f9363847700 1 -- 172.21.15.92:0/3748450650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f935c102850 msgr2=0x7f935c102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:59.925 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.896+0000 7f9363847700 1 --2- 172.21.15.92:0/3748450650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f935c102850 0x7f935c102c60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f93580098e0 tx=0x7f9358009250 comp rx=0 tx=0).stop 2024-05-06T12:05:59.925 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.896+0000 7f9363847700 1 -- 172.21.15.92:0/3748450650 shutdown_connections 2024-05-06T12:05:59.925 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.896+0000 7f9363847700 1 --2- 172.21.15.92:0/3748450650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f935c102850 0x7f935c102c60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:59.926 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.896+0000 7f9363847700 1 -- 172.21.15.92:0/3748450650 >> 172.21.15.92:0/3748450650 conn(0x7f935c0fdff0 msgr2=0x7f935c100400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:59.926 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.896+0000 7f9363847700 1 -- 172.21.15.92:0/3748450650 shutdown_connections 2024-05-06T12:05:59.926 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.896+0000 7f9363847700 1 -- 172.21.15.92:0/3748450650 wait complete. 2024-05-06T12:05:59.926 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.896+0000 7f9363847700 1 Processor -- start 2024-05-06T12:05:59.926 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.896+0000 7f9363847700 1 -- start start 2024-05-06T12:05:59.926 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.897+0000 7f9363847700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f935c102850 0x7f935c111610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:59.926 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.897+0000 7f9363847700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c111b50 con 0x7f935c102850 2024-05-06T12:05:59.926 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.897+0000 7f93615e3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f935c102850 0x7f935c111610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:59.926 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.897+0000 7f93615e3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f935c102850 0x7f935c111610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59122/0 (socket says 172.21.15.92:59122) 2024-05-06T12:05:59.926 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.897+0000 7f93615e3700 1 -- 172.21.15.92:0/2224248833 learned_addr learned my addr 172.21.15.92:0/2224248833 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:05:59.926 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.897+0000 7f93615e3700 1 -- 172.21.15.92:0/2224248833 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93580094f0 con 0x7f935c102850 2024-05-06T12:05:59.926 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.897+0000 7f93615e3700 1 --2- 172.21.15.92:0/2224248833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f935c102850 0x7f935c111610 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f93580090b0 tx=0x7f9358009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:59.926 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.897+0000 7f9351ffb700 1 -- 172.21.15.92:0/2224248833 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9358010840 con 0x7f935c102850 2024-05-06T12:05:59.926 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.897+0000 7f9351ffb700 1 -- 172.21.15.92:0/2224248833 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9358010e70 con 0x7f935c102850 2024-05-06T12:05:59.927 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.897+0000 7f9363847700 1 -- 172.21.15.92:0/2224248833 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f935c111d50 con 0x7f935c102850 2024-05-06T12:05:59.927 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.897+0000 7f9351ffb700 1 -- 172.21.15.92:0/2224248833 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9358024e40 con 0x7f935c102850 2024-05-06T12:05:59.927 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.898+0000 7f9363847700 1 -- 172.21.15.92:0/2224248833 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f935c1128c0 con 0x7f935c102850 2024-05-06T12:05:59.927 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.898+0000 7f9351ffb700 1 -- 172.21.15.92:0/2224248833 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 43283+0+0 (secure 0 0 0) 0x7f935801a4e0 con 0x7f935c102850 2024-05-06T12:05:59.927 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.898+0000 7f9363847700 1 -- 172.21.15.92:0/2224248833 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f935c05f5c0 con 0x7f935c102850 2024-05-06T12:05:59.927 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.899+0000 7f9351ffb700 1 --2- 172.21.15.92:0/2224248833 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f9348036770 0x7f9348038c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:05:59.927 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.899+0000 7f9351ffb700 1 -- 172.21.15.92:0/2224248833 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f935804a700 con 0x7f935c102850 2024-05-06T12:05:59.927 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.899+0000 7f9360de2700 1 --2- 172.21.15.92:0/2224248833 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f9348036770 0x7f9348038c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:05:59.927 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.899+0000 7f9360de2700 1 --2- 172.21.15.92:0/2224248833 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f9348036770 0x7f9348038c20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f934c009990 tx=0x7f934c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:05:59.927 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:58.907+0000 7f9351ffb700 1 -- 172.21.15.92:0/2224248833 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9358018940 con 0x7f935c102850 2024-05-06T12:05:59.927 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:59.107+0000 7f9363847700 1 -- 172.21.15.92:0/2224248833 --> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f935c0ff160 con 0x7f9348036770 2024-05-06T12:05:59.927 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:59.249+0000 7f9351ffb700 1 -- 172.21.15.92:0/2224248833 <== mgr.14162 v2:172.21.15.92:6800/543287293 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f935c0ff160 con 0x7f9348036770 2024-05-06T12:05:59.927 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:59.251+0000 7f9363847700 1 -- 172.21.15.92:0/2224248833 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f9348036770 msgr2=0x7f9348038c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:59.928 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:59.251+0000 7f9363847700 1 --2- 172.21.15.92:0/2224248833 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f9348036770 0x7f9348038c20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f934c009990 tx=0x7f934c006e40 comp rx=0 tx=0).stop 2024-05-06T12:05:59.928 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:59.251+0000 7f9363847700 1 -- 172.21.15.92:0/2224248833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f935c102850 msgr2=0x7f935c111610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:05:59.928 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:59.251+0000 7f9363847700 1 --2- 172.21.15.92:0/2224248833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f935c102850 0x7f935c111610 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f93580090b0 tx=0x7f9358009250 comp rx=0 tx=0).stop 2024-05-06T12:05:59.928 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:59.251+0000 7f9363847700 1 -- 172.21.15.92:0/2224248833 shutdown_connections 2024-05-06T12:05:59.928 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:59.251+0000 7f9363847700 1 --2- 172.21.15.92:0/2224248833 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f9348036770 0x7f9348038c20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:59.928 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:59.251+0000 7f9363847700 1 --2- 172.21.15.92:0/2224248833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f935c102850 0x7f935c111610 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:05:59.928 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:59.251+0000 7f9363847700 1 -- 172.21.15.92:0/2224248833 >> 172.21.15.92:0/2224248833 conn(0x7f935c0fdff0 msgr2=0x7f935c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:05:59.928 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:59.251+0000 7f9363847700 1 -- 172.21.15.92:0/2224248833 shutdown_connections 2024-05-06T12:05:59.928 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:05:59.251+0000 7f9363847700 1 -- 172.21.15.92:0/2224248833 wait complete. 2024-05-06T12:05:59.928 INFO:teuthology.orchestra.run.smithi092.stdout:Creating initial admin user... 2024-05-06T12:06:01.719 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$y9a4BmnvKQVExF2fb24sxelyZGLohpqwungVVsh9TWF66WiTA3LE2", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714997160, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-06T12:06:01.719 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.316+0000 7f6827bab700 1 Processor -- start 2024-05-06T12:06:01.719 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.317+0000 7f6827bab700 1 -- start start 2024-05-06T12:06:01.719 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.317+0000 7f6827bab700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6820102880 0x7f6820102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:01.719 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.317+0000 7f6827bab700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68201031d0 con 0x7f6820102880 2024-05-06T12:06:01.720 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.317+0000 7f6825947700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6820102880 0x7f6820102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:01.720 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.317+0000 7f6825947700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6820102880 0x7f6820102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59130/0 (socket says 172.21.15.92:59130) 2024-05-06T12:06:01.720 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.317+0000 7f6825947700 1 -- 172.21.15.92:0/429553370 learned_addr learned my addr 172.21.15.92:0/429553370 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:06:01.720 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.317+0000 7f6825947700 1 -- 172.21.15.92:0/429553370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6820103310 con 0x7f6820102880 2024-05-06T12:06:01.720 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.318+0000 7f6825947700 1 --2- 172.21.15.92:0/429553370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6820102880 0x7f6820102c90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f68100098e0 tx=0x7f6810009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb74dcb99cc4014 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:01.720 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.318+0000 7f6824945700 1 -- 172.21.15.92:0/429553370 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f681000b860 con 0x7f6820102880 2024-05-06T12:06:01.720 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.318+0000 7f6824945700 1 -- 172.21.15.92:0/429553370 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f681000be90 con 0x7f6820102880 2024-05-06T12:06:01.720 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.318+0000 7f6824945700 1 -- 172.21.15.92:0/429553370 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f68100103c0 con 0x7f6820102880 2024-05-06T12:06:01.720 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.318+0000 7f6827bab700 1 -- 172.21.15.92:0/429553370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6820102880 msgr2=0x7f6820102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:01.720 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.318+0000 7f6827bab700 1 --2- 172.21.15.92:0/429553370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6820102880 0x7f6820102c90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f68100098e0 tx=0x7f6810009250 comp rx=0 tx=0).stop 2024-05-06T12:06:01.720 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.318+0000 7f6827bab700 1 -- 172.21.15.92:0/429553370 shutdown_connections 2024-05-06T12:06:01.720 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.318+0000 7f6827bab700 1 --2- 172.21.15.92:0/429553370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6820102880 0x7f6820102c90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:01.720 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.318+0000 7f6827bab700 1 -- 172.21.15.92:0/429553370 >> 172.21.15.92:0/429553370 conn(0x7f68200fe060 msgr2=0x7f6820100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:01.720 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.318+0000 7f6827bab700 1 -- 172.21.15.92:0/429553370 shutdown_connections 2024-05-06T12:06:01.721 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.318+0000 7f6827bab700 1 -- 172.21.15.92:0/429553370 wait complete. 2024-05-06T12:06:01.721 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.319+0000 7f6827bab700 1 Processor -- start 2024-05-06T12:06:01.721 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.319+0000 7f6827bab700 1 -- start start 2024-05-06T12:06:01.721 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.319+0000 7f6827bab700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6820102880 0x7f68201115c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:01.721 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.319+0000 7f6827bab700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6820111b00 con 0x7f6820102880 2024-05-06T12:06:01.721 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.319+0000 7f6825947700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6820102880 0x7f68201115c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:01.721 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.319+0000 7f6825947700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6820102880 0x7f68201115c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59142/0 (socket says 172.21.15.92:59142) 2024-05-06T12:06:01.721 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.319+0000 7f6825947700 1 -- 172.21.15.92:0/1633774880 learned_addr learned my addr 172.21.15.92:0/1633774880 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:06:01.721 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.319+0000 7f6825947700 1 -- 172.21.15.92:0/1633774880 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68100094f0 con 0x7f6820102880 2024-05-06T12:06:01.721 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.320+0000 7f6825947700 1 --2- 172.21.15.92:0/1633774880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6820102880 0x7f68201115c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6810011680 tx=0x7f6810009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:01.721 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.320+0000 7f68167fc700 1 -- 172.21.15.92:0/1633774880 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6810010890 con 0x7f6820102880 2024-05-06T12:06:01.721 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.320+0000 7f68167fc700 1 -- 172.21.15.92:0/1633774880 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6810010ec0 con 0x7f6820102880 2024-05-06T12:06:01.721 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.320+0000 7f6827bab700 1 -- 172.21.15.92:0/1633774880 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6820111d00 con 0x7f6820102880 2024-05-06T12:06:01.721 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.320+0000 7f68167fc700 1 -- 172.21.15.92:0/1633774880 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f68100243f0 con 0x7f6820102880 2024-05-06T12:06:01.721 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.320+0000 7f6827bab700 1 -- 172.21.15.92:0/1633774880 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6820112870 con 0x7f6820102880 2024-05-06T12:06:01.721 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.320+0000 7f68167fc700 1 -- 172.21.15.92:0/1633774880 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f681002d3f0 con 0x7f6820102880 2024-05-06T12:06:01.722 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.321+0000 7f68167fc700 1 --2- 172.21.15.92:0/1633774880 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f680c0368f0 0x7f680c038da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:01.722 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.321+0000 7f68167fc700 1 -- 172.21.15.92:0/1633774880 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f681004a320 con 0x7f6820102880 2024-05-06T12:06:01.722 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.321+0000 7f6825146700 1 --2- 172.21.15.92:0/1633774880 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f680c0368f0 0x7f680c038da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:01.722 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.321+0000 7f6827bab700 1 -- 172.21.15.92:0/1633774880 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f682005f5c0 con 0x7f6820102880 2024-05-06T12:06:01.722 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.321+0000 7f6825146700 1 --2- 172.21.15.92:0/1633774880 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f680c0368f0 0x7f680c038da0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f681c009990 tx=0x7f681c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:01.722 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.325+0000 7f68167fc700 1 -- 172.21.15.92:0/1633774880 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f681002b8e0 con 0x7f6820102880 2024-05-06T12:06:01.722 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.537+0000 7f6827bab700 1 -- 172.21.15.92:0/1633774880 --> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f682004bed0 con 0x7f680c0368f0 2024-05-06T12:06:01.722 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.757+0000 7f68167fc700 1 -- 172.21.15.92:0/1633774880 <== mgr.14162 v2:172.21.15.92:6800/543287293 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f682004bed0 con 0x7f680c0368f0 2024-05-06T12:06:01.722 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.759+0000 7f6827bab700 1 -- 172.21.15.92:0/1633774880 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f680c0368f0 msgr2=0x7f680c038da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:01.722 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.759+0000 7f6827bab700 1 --2- 172.21.15.92:0/1633774880 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f680c0368f0 0x7f680c038da0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f681c009990 tx=0x7f681c006e40 comp rx=0 tx=0).stop 2024-05-06T12:06:01.722 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.759+0000 7f6827bab700 1 -- 172.21.15.92:0/1633774880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6820102880 msgr2=0x7f68201115c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:01.722 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.759+0000 7f6827bab700 1 --2- 172.21.15.92:0/1633774880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6820102880 0x7f68201115c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6810011680 tx=0x7f6810009250 comp rx=0 tx=0).stop 2024-05-06T12:06:01.722 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.759+0000 7f6827bab700 1 -- 172.21.15.92:0/1633774880 shutdown_connections 2024-05-06T12:06:01.723 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.759+0000 7f6827bab700 1 --2- 172.21.15.92:0/1633774880 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f680c0368f0 0x7f680c038da0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:01.723 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.759+0000 7f6827bab700 1 --2- 172.21.15.92:0/1633774880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6820102880 0x7f68201115c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:01.723 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.759+0000 7f6827bab700 1 -- 172.21.15.92:0/1633774880 >> 172.21.15.92:0/1633774880 conn(0x7f68200fe060 msgr2=0x7f68200feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:01.723 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.759+0000 7f6827bab700 1 -- 172.21.15.92:0/1633774880 shutdown_connections 2024-05-06T12:06:01.723 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:00.759+0000 7f6827bab700 1 -- 172.21.15.92:0/1633774880 wait complete. 2024-05-06T12:06:01.723 INFO:teuthology.orchestra.run.smithi092.stdout:Fetching dashboard port number... 2024-05-06T12:06:01.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:01 smithi092 ceph-mon[26954]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:06:01.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:01 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:02.960 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:02 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/2893760436' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-06T12:06:02.968 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 8443 2024-05-06T12:06:02.968 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.183+0000 7f1c5e938700 1 Processor -- start 2024-05-06T12:06:02.968 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.183+0000 7f1c5e938700 1 -- start start 2024-05-06T12:06:02.968 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.183+0000 7f1c5e938700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c58102880 0x7f1c58102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:02.968 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.183+0000 7f1c5e938700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c581031d0 con 0x7f1c58102880 2024-05-06T12:06:02.968 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.183+0000 7f1c57fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c58102880 0x7f1c58102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:02.968 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.183+0000 7f1c57fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c58102880 0x7f1c58102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59146/0 (socket says 172.21.15.92:59146) 2024-05-06T12:06:02.968 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.183+0000 7f1c57fff700 1 -- 172.21.15.92:0/4287710682 learned_addr learned my addr 172.21.15.92:0/4287710682 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:06:02.968 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.183+0000 7f1c57fff700 1 -- 172.21.15.92:0/4287710682 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c58103310 con 0x7f1c58102880 2024-05-06T12:06:02.968 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.183+0000 7f1c57fff700 1 --2- 172.21.15.92:0/4287710682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c58102880 0x7f1c58102c90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1c480098e0 tx=0x7f1c48009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b01cc8e7a33d31c server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:02.968 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.184+0000 7f1c56ffd700 1 -- 172.21.15.92:0/4287710682 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c4800b860 con 0x7f1c58102880 2024-05-06T12:06:02.968 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.184+0000 7f1c56ffd700 1 -- 172.21.15.92:0/4287710682 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c4800be90 con 0x7f1c58102880 2024-05-06T12:06:02.969 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.184+0000 7f1c56ffd700 1 -- 172.21.15.92:0/4287710682 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c480103c0 con 0x7f1c58102880 2024-05-06T12:06:02.969 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.184+0000 7f1c5e938700 1 -- 172.21.15.92:0/4287710682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c58102880 msgr2=0x7f1c58102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:02.969 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.184+0000 7f1c5e938700 1 --2- 172.21.15.92:0/4287710682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c58102880 0x7f1c58102c90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1c480098e0 tx=0x7f1c48009250 comp rx=0 tx=0).stop 2024-05-06T12:06:02.969 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.184+0000 7f1c5e938700 1 -- 172.21.15.92:0/4287710682 shutdown_connections 2024-05-06T12:06:02.969 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.184+0000 7f1c5e938700 1 --2- 172.21.15.92:0/4287710682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c58102880 0x7f1c58102c90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:02.969 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.184+0000 7f1c5e938700 1 -- 172.21.15.92:0/4287710682 >> 172.21.15.92:0/4287710682 conn(0x7f1c580fe060 msgr2=0x7f1c58100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:02.969 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.184+0000 7f1c5e938700 1 -- 172.21.15.92:0/4287710682 shutdown_connections 2024-05-06T12:06:02.969 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.184+0000 7f1c5e938700 1 -- 172.21.15.92:0/4287710682 wait complete. 2024-05-06T12:06:02.969 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.185+0000 7f1c5e938700 1 Processor -- start 2024-05-06T12:06:02.969 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.185+0000 7f1c5e938700 1 -- start start 2024-05-06T12:06:02.969 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.185+0000 7f1c5e938700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c58102880 0x7f1c58078c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:02.969 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.185+0000 7f1c5e938700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c58079160 con 0x7f1c58102880 2024-05-06T12:06:02.969 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.185+0000 7f1c57fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c58102880 0x7f1c58078c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:02.969 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.185+0000 7f1c57fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c58102880 0x7f1c58078c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59162/0 (socket says 172.21.15.92:59162) 2024-05-06T12:06:02.970 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.185+0000 7f1c57fff700 1 -- 172.21.15.92:0/2893760436 learned_addr learned my addr 172.21.15.92:0/2893760436 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:06:02.970 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.186+0000 7f1c57fff700 1 -- 172.21.15.92:0/2893760436 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c480094f0 con 0x7f1c58102880 2024-05-06T12:06:02.970 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.186+0000 7f1c57fff700 1 --2- 172.21.15.92:0/2893760436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c58102880 0x7f1c58078c20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1c48011680 tx=0x7f1c48009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:02.970 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.186+0000 7f1c54ff9700 1 -- 172.21.15.92:0/2893760436 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c48010890 con 0x7f1c58102880 2024-05-06T12:06:02.970 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.186+0000 7f1c5e938700 1 -- 172.21.15.92:0/2893760436 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c580771d0 con 0x7f1c58102880 2024-05-06T12:06:02.970 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.186+0000 7f1c54ff9700 1 -- 172.21.15.92:0/2893760436 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c48010ec0 con 0x7f1c58102880 2024-05-06T12:06:02.970 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.186+0000 7f1c54ff9700 1 -- 172.21.15.92:0/2893760436 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c480243f0 con 0x7f1c58102880 2024-05-06T12:06:02.970 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.186+0000 7f1c5e938700 1 -- 172.21.15.92:0/2893760436 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c580776c0 con 0x7f1c58102880 2024-05-06T12:06:02.970 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.187+0000 7f1c54ff9700 1 -- 172.21.15.92:0/2893760436 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f1c4802d3f0 con 0x7f1c58102880 2024-05-06T12:06:02.970 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.187+0000 7f1c54ff9700 1 --2- 172.21.15.92:0/2893760436 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f1c400368a0 0x7f1c40038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:02.970 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.187+0000 7f1c5e938700 1 -- 172.21.15.92:0/2893760436 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c5804bed0 con 0x7f1c58102880 2024-05-06T12:06:02.970 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.187+0000 7f1c54ff9700 1 -- 172.21.15.92:0/2893760436 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f1c4804a370 con 0x7f1c58102880 2024-05-06T12:06:02.971 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.187+0000 7f1c577fe700 1 --2- 172.21.15.92:0/2893760436 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f1c400368a0 0x7f1c40038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:02.971 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.188+0000 7f1c577fe700 1 --2- 172.21.15.92:0/2893760436 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f1c400368a0 0x7f1c40038d50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c009990 tx=0x7f1c4c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:02.971 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.192+0000 7f1c54ff9700 1 -- 172.21.15.92:0/2893760436 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1c4802b8e0 con 0x7f1c58102880 2024-05-06T12:06:02.971 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.376+0000 7f1c5e938700 1 -- 172.21.15.92:0/2893760436 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f1c58077b40 con 0x7f1c58102880 2024-05-06T12:06:02.971 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.376+0000 7f1c54ff9700 1 -- 172.21.15.92:0/2893760436 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f1c4802c350 con 0x7f1c58102880 2024-05-06T12:06:02.971 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.379+0000 7f1c5e938700 1 -- 172.21.15.92:0/2893760436 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f1c400368a0 msgr2=0x7f1c40038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:02.971 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.379+0000 7f1c5e938700 1 --2- 172.21.15.92:0/2893760436 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f1c400368a0 0x7f1c40038d50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c009990 tx=0x7f1c4c006e40 comp rx=0 tx=0).stop 2024-05-06T12:06:02.971 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.379+0000 7f1c5e938700 1 -- 172.21.15.92:0/2893760436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c58102880 msgr2=0x7f1c58078c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:02.971 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.379+0000 7f1c5e938700 1 --2- 172.21.15.92:0/2893760436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c58102880 0x7f1c58078c20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1c48011680 tx=0x7f1c48009250 comp rx=0 tx=0).stop 2024-05-06T12:06:02.971 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.379+0000 7f1c5e938700 1 -- 172.21.15.92:0/2893760436 shutdown_connections 2024-05-06T12:06:02.971 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.379+0000 7f1c5e938700 1 --2- 172.21.15.92:0/2893760436 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f1c400368a0 0x7f1c40038d50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:02.971 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.379+0000 7f1c5e938700 1 --2- 172.21.15.92:0/2893760436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c58102880 0x7f1c58078c20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:02.971 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.379+0000 7f1c5e938700 1 -- 172.21.15.92:0/2893760436 >> 172.21.15.92:0/2893760436 conn(0x7f1c580fe060 msgr2=0x7f1c580feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:02.971 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.379+0000 7f1c5e938700 1 -- 172.21.15.92:0/2893760436 shutdown_connections 2024-05-06T12:06:02.972 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:02.379+0000 7f1c5e938700 1 -- 172.21.15.92:0/2893760436 wait complete. 2024-05-06T12:06:02.974 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-06T12:06:02.974 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout disabled 2024-05-06T12:06:02.982 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-06T12:06:02.982 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout inactive 2024-05-06T12:06:02.982 INFO:teuthology.orchestra.run.smithi092.stdout:firewalld.service is not enabled 2024-05-06T12:06:02.982 INFO:teuthology.orchestra.run.smithi092.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-06T12:06:02.984 INFO:teuthology.orchestra.run.smithi092.stdout:Ceph Dashboard is now available at: 2024-05-06T12:06:02.984 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:06:02.984 INFO:teuthology.orchestra.run.smithi092.stdout: URL: https://smithi092:8443/ 2024-05-06T12:06:02.984 INFO:teuthology.orchestra.run.smithi092.stdout: User: admin 2024-05-06T12:06:02.984 INFO:teuthology.orchestra.run.smithi092.stdout: Password: 0q3c2k8uvp 2024-05-06T12:06:02.984 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:06:02.984 INFO:teuthology.orchestra.run.smithi092.stdout:Saving cluster configuration to /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config directory 2024-05-06T12:06:02.985 INFO:teuthology.orchestra.run.smithi092.stdout:Enabling autotune for osd_memory_target 2024-05-06T12:06:03.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:03 smithi092 ceph-mon[26954]: mgrmap e13: smithi092.gyzeqb(active, since 6s) 2024-05-06T12:06:04.215 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.433+0000 7efd6fdc0700 1 Processor -- start 2024-05-06T12:06:04.216 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.433+0000 7efd6fdc0700 1 -- start start 2024-05-06T12:06:04.216 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.433+0000 7efd6fdc0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd68076bf0 0x7efd68105460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:04.216 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.433+0000 7efd6fdc0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd681059a0 con 0x7efd68076bf0 2024-05-06T12:06:04.216 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.433+0000 7efd6db5c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd68076bf0 0x7efd68105460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:04.216 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.433+0000 7efd6db5c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd68076bf0 0x7efd68105460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53332/0 (socket says 172.21.15.92:53332) 2024-05-06T12:06:04.216 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.434+0000 7efd6db5c700 1 -- 172.21.15.92:0/144972904 learned_addr learned my addr 172.21.15.92:0/144972904 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:06:04.216 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.434+0000 7efd6db5c700 1 -- 172.21.15.92:0/144972904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd68105ae0 con 0x7efd68076bf0 2024-05-06T12:06:04.216 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.434+0000 7efd6db5c700 1 --2- 172.21.15.92:0/144972904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd68076bf0 0x7efd68105460 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efd64009e50 tx=0x7efd64009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed60ff27c09ebd8b server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:04.216 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.434+0000 7efd6cb5a700 1 -- 172.21.15.92:0/144972904 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efd640092e0 con 0x7efd68076bf0 2024-05-06T12:06:04.216 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.434+0000 7efd6cb5a700 1 -- 172.21.15.92:0/144972904 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd64012430 con 0x7efd68076bf0 2024-05-06T12:06:04.216 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.434+0000 7efd6cb5a700 1 -- 172.21.15.92:0/144972904 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efd640075a0 con 0x7efd68076bf0 2024-05-06T12:06:04.216 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.435+0000 7efd6fdc0700 1 -- 172.21.15.92:0/144972904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd68076bf0 msgr2=0x7efd68105460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:04.216 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.435+0000 7efd6fdc0700 1 --2- 172.21.15.92:0/144972904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd68076bf0 0x7efd68105460 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efd64009e50 tx=0x7efd64009840 comp rx=0 tx=0).stop 2024-05-06T12:06:04.216 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.435+0000 7efd6fdc0700 1 -- 172.21.15.92:0/144972904 shutdown_connections 2024-05-06T12:06:04.217 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.435+0000 7efd6fdc0700 1 --2- 172.21.15.92:0/144972904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd68076bf0 0x7efd68105460 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:04.217 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.435+0000 7efd6fdc0700 1 -- 172.21.15.92:0/144972904 >> 172.21.15.92:0/144972904 conn(0x7efd680fcef0 msgr2=0x7efd680ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:04.217 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.435+0000 7efd6fdc0700 1 -- 172.21.15.92:0/144972904 shutdown_connections 2024-05-06T12:06:04.217 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.435+0000 7efd6fdc0700 1 -- 172.21.15.92:0/144972904 wait complete. 2024-05-06T12:06:04.217 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.435+0000 7efd6fdc0700 1 Processor -- start 2024-05-06T12:06:04.217 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.435+0000 7efd6fdc0700 1 -- start start 2024-05-06T12:06:04.217 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.435+0000 7efd6fdc0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd68076bf0 0x7efd68102db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:04.217 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.435+0000 7efd6fdc0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd681032f0 con 0x7efd68076bf0 2024-05-06T12:06:04.217 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.436+0000 7efd6db5c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd68076bf0 0x7efd68102db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:04.217 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.436+0000 7efd6db5c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd68076bf0 0x7efd68102db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53344/0 (socket says 172.21.15.92:53344) 2024-05-06T12:06:04.217 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.436+0000 7efd6db5c700 1 -- 172.21.15.92:0/2272989062 learned_addr learned my addr 172.21.15.92:0/2272989062 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:06:04.217 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.436+0000 7efd6db5c700 1 -- 172.21.15.92:0/2272989062 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd64009ae0 con 0x7efd68076bf0 2024-05-06T12:06:04.217 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.436+0000 7efd6db5c700 1 --2- 172.21.15.92:0/2272989062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd68076bf0 0x7efd68102db0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efd64012f20 tx=0x7efd64009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:04.217 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.436+0000 7efd5e7fc700 1 -- 172.21.15.92:0/2272989062 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efd640092e0 con 0x7efd68076bf0 2024-05-06T12:06:04.218 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.436+0000 7efd5e7fc700 1 -- 172.21.15.92:0/2272989062 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd64007d30 con 0x7efd68076bf0 2024-05-06T12:06:04.218 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.436+0000 7efd5e7fc700 1 -- 172.21.15.92:0/2272989062 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efd64025a00 con 0x7efd68076bf0 2024-05-06T12:06:04.218 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.436+0000 7efd6fdc0700 1 -- 172.21.15.92:0/2272989062 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd68101360 con 0x7efd68076bf0 2024-05-06T12:06:04.218 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.436+0000 7efd6fdc0700 1 -- 172.21.15.92:0/2272989062 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd68101850 con 0x7efd68076bf0 2024-05-06T12:06:04.218 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.437+0000 7efd5e7fc700 1 -- 172.21.15.92:0/2272989062 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7efd64019050 con 0x7efd68076bf0 2024-05-06T12:06:04.218 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.437+0000 7efd5e7fc700 1 --2- 172.21.15.92:0/2272989062 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7efd54036990 0x7efd54038e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:04.218 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.437+0000 7efd6fdc0700 1 -- 172.21.15.92:0/2272989062 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd6804bed0 con 0x7efd68076bf0 2024-05-06T12:06:04.218 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.437+0000 7efd5e7fc700 1 -- 172.21.15.92:0/2272989062 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7efd6404a160 con 0x7efd68076bf0 2024-05-06T12:06:04.218 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.437+0000 7efd6d35b700 1 --2- 172.21.15.92:0/2272989062 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7efd54036990 0x7efd54038e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:04.218 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.438+0000 7efd6d35b700 1 --2- 172.21.15.92:0/2272989062 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7efd54036990 0x7efd54038e40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efd58009990 tx=0x7efd58006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:04.218 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.442+0000 7efd5e7fc700 1 -- 172.21.15.92:0/2272989062 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7efd64020080 con 0x7efd68076bf0 2024-05-06T12:06:04.218 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.627+0000 7efd6fdc0700 1 -- 172.21.15.92:0/2272989062 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7efd68101d90 con 0x7efd68076bf0 2024-05-06T12:06:04.218 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.627+0000 7efd5e7fc700 1 -- 172.21.15.92:0/2272989062 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7efd64017020 con 0x7efd68076bf0 2024-05-06T12:06:04.219 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.630+0000 7efd6fdc0700 1 -- 172.21.15.92:0/2272989062 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7efd54036990 msgr2=0x7efd54038e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:04.219 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.630+0000 7efd6fdc0700 1 --2- 172.21.15.92:0/2272989062 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7efd54036990 0x7efd54038e40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efd58009990 tx=0x7efd58006e40 comp rx=0 tx=0).stop 2024-05-06T12:06:04.219 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.630+0000 7efd6fdc0700 1 -- 172.21.15.92:0/2272989062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd68076bf0 msgr2=0x7efd68102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:04.219 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.630+0000 7efd6fdc0700 1 --2- 172.21.15.92:0/2272989062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd68076bf0 0x7efd68102db0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efd64012f20 tx=0x7efd64009840 comp rx=0 tx=0).stop 2024-05-06T12:06:04.219 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.630+0000 7efd6fdc0700 1 -- 172.21.15.92:0/2272989062 shutdown_connections 2024-05-06T12:06:04.219 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.630+0000 7efd6fdc0700 1 --2- 172.21.15.92:0/2272989062 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7efd54036990 0x7efd54038e40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:04.219 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.630+0000 7efd6fdc0700 1 --2- 172.21.15.92:0/2272989062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd68076bf0 0x7efd68102db0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:04.219 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.630+0000 7efd6fdc0700 1 -- 172.21.15.92:0/2272989062 >> 172.21.15.92:0/2272989062 conn(0x7efd680fcef0 msgr2=0x7efd680fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:04.219 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.630+0000 7efd6fdc0700 1 -- 172.21.15.92:0/2272989062 shutdown_connections 2024-05-06T12:06:04.219 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:03.630+0000 7efd6fdc0700 1 -- 172.21.15.92:0/2272989062 wait complete. 2024-05-06T12:06:05.445 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.637+0000 7f3f43fa1700 1 Processor -- start 2024-05-06T12:06:05.445 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.638+0000 7f3f43fa1700 1 -- start start 2024-05-06T12:06:05.445 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.638+0000 7f3f43fa1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f3c101740 0x7f3f3c101b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:05.445 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.638+0000 7f3f43fa1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c102090 con 0x7f3f3c101740 2024-05-06T12:06:05.445 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.638+0000 7f3f41d3d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f3c101740 0x7f3f3c101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:05.446 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.638+0000 7f3f41d3d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f3c101740 0x7f3f3c101b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53348/0 (socket says 172.21.15.92:53348) 2024-05-06T12:06:05.446 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.638+0000 7f3f41d3d700 1 -- 172.21.15.92:0/505507696 learned_addr learned my addr 172.21.15.92:0/505507696 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:06:05.451 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.638+0000 7f3f41d3d700 1 -- 172.21.15.92:0/505507696 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f3c1021d0 con 0x7f3f3c101740 2024-05-06T12:06:05.451 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.638+0000 7f3f41d3d700 1 --2- 172.21.15.92:0/505507696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f3c101740 0x7f3f3c101b50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c0098e0 tx=0x7f3f2c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26389095fe8f1f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:05.451 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.638+0000 7f3f40d3b700 1 -- 172.21.15.92:0/505507696 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3f2c00b860 con 0x7f3f3c101740 2024-05-06T12:06:05.451 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.639+0000 7f3f40d3b700 1 -- 172.21.15.92:0/505507696 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f2c00be90 con 0x7f3f3c101740 2024-05-06T12:06:05.451 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.639+0000 7f3f40d3b700 1 -- 172.21.15.92:0/505507696 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3f2c0103c0 con 0x7f3f3c101740 2024-05-06T12:06:05.451 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.639+0000 7f3f43fa1700 1 -- 172.21.15.92:0/505507696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f3c101740 msgr2=0x7f3f3c101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:05.451 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.639+0000 7f3f43fa1700 1 --2- 172.21.15.92:0/505507696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f3c101740 0x7f3f3c101b50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c0098e0 tx=0x7f3f2c009250 comp rx=0 tx=0).stop 2024-05-06T12:06:05.452 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.639+0000 7f3f43fa1700 1 -- 172.21.15.92:0/505507696 shutdown_connections 2024-05-06T12:06:05.452 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.639+0000 7f3f43fa1700 1 --2- 172.21.15.92:0/505507696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f3c101740 0x7f3f3c101b50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:05.452 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.639+0000 7f3f43fa1700 1 -- 172.21.15.92:0/505507696 >> 172.21.15.92:0/505507696 conn(0x7f3f3c0fcec0 msgr2=0x7f3f3c0ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:05.452 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.639+0000 7f3f43fa1700 1 -- 172.21.15.92:0/505507696 shutdown_connections 2024-05-06T12:06:05.452 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.639+0000 7f3f43fa1700 1 -- 172.21.15.92:0/505507696 wait complete. 2024-05-06T12:06:05.452 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.640+0000 7f3f43fa1700 1 Processor -- start 2024-05-06T12:06:05.452 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.640+0000 7f3f43fa1700 1 -- start start 2024-05-06T12:06:05.452 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.640+0000 7f3f43fa1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f3c101740 0x7f3f3c1104f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:05.452 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.640+0000 7f3f43fa1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c110a30 con 0x7f3f3c101740 2024-05-06T12:06:05.452 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.640+0000 7f3f41d3d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f3c101740 0x7f3f3c1104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:05.452 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.640+0000 7f3f41d3d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f3c101740 0x7f3f3c1104f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53352/0 (socket says 172.21.15.92:53352) 2024-05-06T12:06:05.452 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.640+0000 7f3f41d3d700 1 -- 172.21.15.92:0/1996460583 learned_addr learned my addr 172.21.15.92:0/1996460583 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:06:05.452 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.640+0000 7f3f41d3d700 1 -- 172.21.15.92:0/1996460583 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f2c0094f0 con 0x7f3f3c101740 2024-05-06T12:06:05.453 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.640+0000 7f3f41d3d700 1 --2- 172.21.15.92:0/1996460583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f3c101740 0x7f3f3c1104f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c0090b0 tx=0x7f3f2c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:05.453 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.640+0000 7f3f327fc700 1 -- 172.21.15.92:0/1996460583 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3f2c010840 con 0x7f3f3c101740 2024-05-06T12:06:05.453 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.640+0000 7f3f327fc700 1 -- 172.21.15.92:0/1996460583 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f2c010e70 con 0x7f3f3c101740 2024-05-06T12:06:05.453 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.640+0000 7f3f43fa1700 1 -- 172.21.15.92:0/1996460583 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f3c110c30 con 0x7f3f3c101740 2024-05-06T12:06:05.453 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.640+0000 7f3f327fc700 1 -- 172.21.15.92:0/1996460583 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3f2c024e40 con 0x7f3f3c101740 2024-05-06T12:06:05.453 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.640+0000 7f3f43fa1700 1 -- 172.21.15.92:0/1996460583 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f3c1117a0 con 0x7f3f3c101740 2024-05-06T12:06:05.453 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.641+0000 7f3f327fc700 1 -- 172.21.15.92:0/1996460583 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f3f2c01a530 con 0x7f3f3c101740 2024-05-06T12:06:05.453 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.641+0000 7f3f43fa1700 1 -- 172.21.15.92:0/1996460583 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f3c05f5c0 con 0x7f3f3c101740 2024-05-06T12:06:05.453 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.641+0000 7f3f327fc700 1 --2- 172.21.15.92:0/1996460583 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f3f280368f0 0x7f3f28038da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:05.453 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.642+0000 7f3f327fc700 1 -- 172.21.15.92:0/1996460583 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f3f2c04ab60 con 0x7f3f3c101740 2024-05-06T12:06:05.453 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.642+0000 7f3f4153c700 1 --2- 172.21.15.92:0/1996460583 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f3f280368f0 0x7f3f28038da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:05.453 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.642+0000 7f3f4153c700 1 --2- 172.21.15.92:0/1996460583 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f3f280368f0 0x7f3f28038da0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3f38009990 tx=0x7f3f38006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:05.453 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.646+0000 7f3f327fc700 1 -- 172.21.15.92:0/1996460583 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3f2c023d20 con 0x7f3f3c101740 2024-05-06T12:06:05.453 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.935+0000 7f3f43fa1700 1 -- 172.21.15.92:0/1996460583 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f3f3c04bed0 con 0x7f3f3c101740 2024-05-06T12:06:05.453 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.937+0000 7f3f327fc700 1 -- 172.21.15.92:0/1996460583 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v33)=0 set mgr/dashboard/cluster/status v33) v1 ==== 153+0+0 (secure 0 0 0) 0x7f3f2c02d3a0 con 0x7f3f3c101740 2024-05-06T12:06:05.454 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.939+0000 7f3f43fa1700 1 -- 172.21.15.92:0/1996460583 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f3f280368f0 msgr2=0x7f3f28038da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:05.454 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.939+0000 7f3f43fa1700 1 --2- 172.21.15.92:0/1996460583 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f3f280368f0 0x7f3f28038da0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3f38009990 tx=0x7f3f38006e40 comp rx=0 tx=0).stop 2024-05-06T12:06:05.454 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.939+0000 7f3f43fa1700 1 -- 172.21.15.92:0/1996460583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f3c101740 msgr2=0x7f3f3c1104f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:05.454 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.939+0000 7f3f43fa1700 1 --2- 172.21.15.92:0/1996460583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f3c101740 0x7f3f3c1104f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c0090b0 tx=0x7f3f2c009250 comp rx=0 tx=0).stop 2024-05-06T12:06:05.454 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.939+0000 7f3f43fa1700 1 -- 172.21.15.92:0/1996460583 shutdown_connections 2024-05-06T12:06:05.454 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.939+0000 7f3f43fa1700 1 --2- 172.21.15.92:0/1996460583 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f3f280368f0 0x7f3f28038da0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:05.454 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.939+0000 7f3f43fa1700 1 --2- 172.21.15.92:0/1996460583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f3c101740 0x7f3f3c1104f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:05.454 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.939+0000 7f3f43fa1700 1 -- 172.21.15.92:0/1996460583 >> 172.21.15.92:0/1996460583 conn(0x7f3f3c0fcec0 msgr2=0x7f3f3c0fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:05.454 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.939+0000 7f3f43fa1700 1 -- 172.21.15.92:0/1996460583 shutdown_connections 2024-05-06T12:06:05.454 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-05-06T12:06:04.939+0000 7f3f43fa1700 1 -- 172.21.15.92:0/1996460583 wait complete. 2024-05-06T12:06:05.454 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-06T12:06:05.454 INFO:teuthology.orchestra.run.smithi092.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-06T12:06:05.454 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:06:05.455 INFO:teuthology.orchestra.run.smithi092.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-06T12:06:05.455 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:06:05.455 INFO:teuthology.orchestra.run.smithi092.stdout:Or, if you are only running a single cluster on this host: 2024-05-06T12:06:05.455 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:06:05.455 INFO:teuthology.orchestra.run.smithi092.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-06T12:06:05.455 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:06:05.455 INFO:teuthology.orchestra.run.smithi092.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-06T12:06:05.455 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:06:05.455 INFO:teuthology.orchestra.run.smithi092.stdout: ceph telemetry on 2024-05-06T12:06:05.455 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:06:05.455 INFO:teuthology.orchestra.run.smithi092.stdout:For more information see: 2024-05-06T12:06:05.455 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:06:05.455 INFO:teuthology.orchestra.run.smithi092.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-05-06T12:06:05.455 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:06:05.455 INFO:teuthology.orchestra.run.smithi092.stdout:Bootstrap complete. 2024-05-06T12:06:05.480 INFO:tasks.cephadm:Fetching config... 2024-05-06T12:06:05.480 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-06T12:06:05.480 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-06T12:06:05.535 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-06T12:06:05.536 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-06T12:06:05.536 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-06T12:06:05.591 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-06T12:06:05.591 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-06T12:06:05.591 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/keyring of=/dev/stdout 2024-05-06T12:06:05.654 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-06T12:06:05.655 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-06T12:06:05.655 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-06T12:06:05.709 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-06T12:06:05.710 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJEJ3jgs3fuPpPzaDYtNSt3dKhCPMJ5legS8fmNRb5uRtWAWoqLD0MlrIbR26GB3JxancyiuBTWG6N/Y1l0kCHFQTAH2tC25zKADOpZOvIeFMsLGa+hT55Uvjucomzfp4rD8DEMAF3T2uH2PzHa+VjWAx1q/txe8h3bz1fkCRC0v3a8IYAMNZZs1KheEXE+N0QKhx+iSjyoKDO+bAOcOMRzZzhTTOL1y17SdXHdkW6zx4El+Lq8hbiK5+GKvdEL5n9aRx9Fo8ymZoCVwSzw2zSmsNF0/rupr/IrraNCbwN+8kbo++MiY4BcAnrLIvH3SuTvvT9k1tnkBRljgs1qk8x1DcXJIdghJvjsrdVg8pA0gc3m4wHdPms0sXu/ySv5eLAikZ2hK99D7a4VC/dPkgPKMuhOXPIwZ8/NgPK78ESAWc+FZ8D65hG77clvO4Q/Ui+jV6Jz1xhW5wroEmxpSn0ldKGSyVrUqdIFW8tL4Bv24/796I0wSIfGc0jZymkDjU= ceph-aee1d912-0ba0-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-06T12:06:05.787 INFO:teuthology.orchestra.run.smithi092.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJEJ3jgs3fuPpPzaDYtNSt3dKhCPMJ5legS8fmNRb5uRtWAWoqLD0MlrIbR26GB3JxancyiuBTWG6N/Y1l0kCHFQTAH2tC25zKADOpZOvIeFMsLGa+hT55Uvjucomzfp4rD8DEMAF3T2uH2PzHa+VjWAx1q/txe8h3bz1fkCRC0v3a8IYAMNZZs1KheEXE+N0QKhx+iSjyoKDO+bAOcOMRzZzhTTOL1y17SdXHdkW6zx4El+Lq8hbiK5+GKvdEL5n9aRx9Fo8ymZoCVwSzw2zSmsNF0/rupr/IrraNCbwN+8kbo++MiY4BcAnrLIvH3SuTvvT9k1tnkBRljgs1qk8x1DcXJIdghJvjsrdVg8pA0gc3m4wHdPms0sXu/ySv5eLAikZ2hK99D7a4VC/dPkgPKMuhOXPIwZ8/NgPK78ESAWc+FZ8D65hG77clvO4Q/Ui+jV6Jz1xhW5wroEmxpSn0ldKGSyVrUqdIFW8tL4Bv24/796I0wSIfGc0jZymkDjU= ceph-aee1d912-0ba0-11ef-bc97-c7b262605968 2024-05-06T12:06:05.798 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJEJ3jgs3fuPpPzaDYtNSt3dKhCPMJ5legS8fmNRb5uRtWAWoqLD0MlrIbR26GB3JxancyiuBTWG6N/Y1l0kCHFQTAH2tC25zKADOpZOvIeFMsLGa+hT55Uvjucomzfp4rD8DEMAF3T2uH2PzHa+VjWAx1q/txe8h3bz1fkCRC0v3a8IYAMNZZs1KheEXE+N0QKhx+iSjyoKDO+bAOcOMRzZzhTTOL1y17SdXHdkW6zx4El+Lq8hbiK5+GKvdEL5n9aRx9Fo8ymZoCVwSzw2zSmsNF0/rupr/IrraNCbwN+8kbo++MiY4BcAnrLIvH3SuTvvT9k1tnkBRljgs1qk8x1DcXJIdghJvjsrdVg8pA0gc3m4wHdPms0sXu/ySv5eLAikZ2hK99D7a4VC/dPkgPKMuhOXPIwZ8/NgPK78ESAWc+FZ8D65hG77clvO4Q/Ui+jV6Jz1xhW5wroEmxpSn0ldKGSyVrUqdIFW8tL4Bv24/796I0wSIfGc0jZymkDjU= ceph-aee1d912-0ba0-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-06T12:06:05.836 INFO:teuthology.orchestra.run.smithi190.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJEJ3jgs3fuPpPzaDYtNSt3dKhCPMJ5legS8fmNRb5uRtWAWoqLD0MlrIbR26GB3JxancyiuBTWG6N/Y1l0kCHFQTAH2tC25zKADOpZOvIeFMsLGa+hT55Uvjucomzfp4rD8DEMAF3T2uH2PzHa+VjWAx1q/txe8h3bz1fkCRC0v3a8IYAMNZZs1KheEXE+N0QKhx+iSjyoKDO+bAOcOMRzZzhTTOL1y17SdXHdkW6zx4El+Lq8hbiK5+GKvdEL5n9aRx9Fo8ymZoCVwSzw2zSmsNF0/rupr/IrraNCbwN+8kbo++MiY4BcAnrLIvH3SuTvvT9k1tnkBRljgs1qk8x1DcXJIdghJvjsrdVg8pA0gc3m4wHdPms0sXu/ySv5eLAikZ2hK99D7a4VC/dPkgPKMuhOXPIwZ8/NgPK78ESAWc+FZ8D65hG77clvO4Q/Ui+jV6Jz1xhW5wroEmxpSn0ldKGSyVrUqdIFW8tL4Bv24/796I0wSIfGc0jZymkDjU= ceph-aee1d912-0ba0-11ef-bc97-c7b262605968 2024-05-06T12:06:05.848 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-06T12:06:06.096 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:06:06.218 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:05 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/1996460583' entity='client.admin' 2024-05-06T12:06:08.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.685+0000 7f0ab16b6700 1 -- 172.21.15.92:0/650633429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0aac0fd020 msgr2=0x7f0aac0fd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:08.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.685+0000 7f0ab16b6700 1 --2- 172.21.15.92:0/650633429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0aac0fd020 0x7f0aac0fd430 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4009c60 tx=0x7f0aa4004d10 comp rx=0 tx=0).stop 2024-05-06T12:06:08.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.685+0000 7f0ab16b6700 1 -- 172.21.15.92:0/650633429 shutdown_connections 2024-05-06T12:06:08.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.685+0000 7f0ab16b6700 1 --2- 172.21.15.92:0/650633429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0aac0fd020 0x7f0aac0fd430 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:08.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.685+0000 7f0ab16b6700 1 -- 172.21.15.92:0/650633429 >> 172.21.15.92:0/650633429 conn(0x7f0aac0f70a0 msgr2=0x7f0aac0f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:08.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.685+0000 7f0ab16b6700 1 -- 172.21.15.92:0/650633429 shutdown_connections 2024-05-06T12:06:08.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.685+0000 7f0ab16b6700 1 -- 172.21.15.92:0/650633429 wait complete. 2024-05-06T12:06:08.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.686+0000 7f0ab16b6700 1 Processor -- start 2024-05-06T12:06:08.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.686+0000 7f0ab16b6700 1 -- start start 2024-05-06T12:06:08.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.686+0000 7f0ab16b6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0aac0fd020 0x7f0aac0f9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:08.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.686+0000 7f0ab16b6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac0f9570 con 0x7f0aac0fd020 2024-05-06T12:06:08.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.686+0000 7f0aaaffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0aac0fd020 0x7f0aac0f9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:08.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.686+0000 7f0aaaffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0aac0fd020 0x7f0aac0f9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53380/0 (socket says 172.21.15.92:53380) 2024-05-06T12:06:08.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.686+0000 7f0aaaffd700 1 -- 172.21.15.92:0/1873680689 learned_addr learned my addr 172.21.15.92:0/1873680689 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:06:08.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.686+0000 7f0aaaffd700 1 -- 172.21.15.92:0/1873680689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa4009940 con 0x7f0aac0fd020 2024-05-06T12:06:08.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.686+0000 7f0aaaffd700 1 --2- 172.21.15.92:0/1873680689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0aac0fd020 0x7f0aac0f9030 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4009500 tx=0x7f0aa4004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:08.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.686+0000 7f0aa37fe700 1 -- 172.21.15.92:0/1873680689 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0aa4007c80 con 0x7f0aac0fd020 2024-05-06T12:06:08.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.687+0000 7f0aa37fe700 1 -- 172.21.15.92:0/1873680689 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0aa401d910 con 0x7f0aac0fd020 2024-05-06T12:06:08.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.687+0000 7f0aa37fe700 1 -- 172.21.15.92:0/1873680689 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0aa40129d0 con 0x7f0aac0fd020 2024-05-06T12:06:08.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.687+0000 7f0ab16b6700 1 -- 172.21.15.92:0/1873680689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aac0f7640 con 0x7f0aac0fd020 2024-05-06T12:06:08.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.687+0000 7f0ab16b6700 1 -- 172.21.15.92:0/1873680689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aac0f7b30 con 0x7f0aac0fd020 2024-05-06T12:06:08.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.688+0000 7f0aa37fe700 1 -- 172.21.15.92:0/1873680689 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f0aa4019040 con 0x7f0aac0fd020 2024-05-06T12:06:08.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.688+0000 7f0aa37fe700 1 --2- 172.21.15.92:0/1873680689 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f0a98036880 0x7f0a98038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:08.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.688+0000 7f0aa37fe700 1 -- 172.21.15.92:0/1873680689 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f0aa404a460 con 0x7f0aac0fd020 2024-05-06T12:06:08.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.688+0000 7f0ab16b6700 1 -- 172.21.15.92:0/1873680689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aac05f5c0 con 0x7f0aac0fd020 2024-05-06T12:06:08.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.688+0000 7f0aaa7fc700 1 --2- 172.21.15.92:0/1873680689 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f0a98036880 0x7f0a98038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:08.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.688+0000 7f0aaa7fc700 1 --2- 172.21.15.92:0/1873680689 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f0a98036880 0x7f0a98038d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0a94009990 tx=0x7f0a94006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:08.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.693+0000 7f0aa37fe700 1 -- 172.21.15.92:0/1873680689 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0aa4017070 con 0x7f0aac0fd020 2024-05-06T12:06:08.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.890+0000 7f0ab16b6700 1 -- 172.21.15.92:0/1873680689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f0aac101780 con 0x7f0aac0fd020 2024-05-06T12:06:08.893 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:08 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:08.894 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:08 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:08.894 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:08 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi092", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:06:08.894 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:08 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:08.894 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:08 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi092", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-06T12:06:08.894 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:08 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi092", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-05-06T12:06:08.894 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:08 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:06:08.894 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:08 smithi092 ceph-mon[26954]: Deploying daemon crash.smithi092 on smithi092 2024-05-06T12:06:08.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.896+0000 7f0aa37fe700 1 -- 172.21.15.92:0/1873680689 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f0aa4021030 con 0x7f0aac0fd020 2024-05-06T12:06:08.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.899+0000 7f0ab16b6700 1 -- 172.21.15.92:0/1873680689 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f0a98036880 msgr2=0x7f0a98038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:08.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.899+0000 7f0ab16b6700 1 --2- 172.21.15.92:0/1873680689 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f0a98036880 0x7f0a98038d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0a94009990 tx=0x7f0a94006e40 comp rx=0 tx=0).stop 2024-05-06T12:06:08.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.899+0000 7f0ab16b6700 1 -- 172.21.15.92:0/1873680689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0aac0fd020 msgr2=0x7f0aac0f9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:08.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.899+0000 7f0ab16b6700 1 --2- 172.21.15.92:0/1873680689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0aac0fd020 0x7f0aac0f9030 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4009500 tx=0x7f0aa4004d10 comp rx=0 tx=0).stop 2024-05-06T12:06:08.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.899+0000 7f0ab16b6700 1 -- 172.21.15.92:0/1873680689 shutdown_connections 2024-05-06T12:06:08.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.899+0000 7f0ab16b6700 1 --2- 172.21.15.92:0/1873680689 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f0a98036880 0x7f0a98038d30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:08.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.899+0000 7f0ab16b6700 1 --2- 172.21.15.92:0/1873680689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0aac0fd020 0x7f0aac0f9030 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:08.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.899+0000 7f0ab16b6700 1 -- 172.21.15.92:0/1873680689 >> 172.21.15.92:0/1873680689 conn(0x7f0aac0f70a0 msgr2=0x7f0aac0658e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:08.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.899+0000 7f0ab16b6700 1 -- 172.21.15.92:0/1873680689 shutdown_connections 2024-05-06T12:06:08.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:08.899+0000 7f0ab16b6700 1 -- 172.21.15.92:0/1873680689 wait complete. 2024-05-06T12:06:09.514 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-06T12:06:09.514 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-06T12:06:10.144 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:09 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/1873680689' entity='client.admin' 2024-05-06T12:06:10.148 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:06:11.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.719+0000 7f0c40faa700 1 -- 172.21.15.92:0/977937905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c3c0fca30 msgr2=0x7f0c3c0fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:11.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.719+0000 7f0c40faa700 1 --2- 172.21.15.92:0/977937905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c3c0fca30 0x7f0c3c0fce40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c0048c0 tx=0x7f0c2c009e30 comp rx=0 tx=0).stop 2024-05-06T12:06:11.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.719+0000 7f0c40faa700 1 -- 172.21.15.92:0/977937905 shutdown_connections 2024-05-06T12:06:11.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.719+0000 7f0c40faa700 1 --2- 172.21.15.92:0/977937905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c3c0fca30 0x7f0c3c0fce40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:11.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.719+0000 7f0c40faa700 1 -- 172.21.15.92:0/977937905 >> 172.21.15.92:0/977937905 conn(0x7f0c3c0f8170 msgr2=0x7f0c3c0fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:11.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.720+0000 7f0c40faa700 1 -- 172.21.15.92:0/977937905 shutdown_connections 2024-05-06T12:06:11.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.720+0000 7f0c40faa700 1 -- 172.21.15.92:0/977937905 wait complete. 2024-05-06T12:06:11.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.720+0000 7f0c40faa700 1 Processor -- start 2024-05-06T12:06:11.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.720+0000 7f0c40faa700 1 -- start start 2024-05-06T12:06:11.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.720+0000 7f0c40faa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c3c0fca30 0x7f0c3c071d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:11.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.720+0000 7f0c40faa700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c072240 con 0x7f0c3c0fca30 2024-05-06T12:06:11.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.720+0000 7f0c3a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c3c0fca30 0x7f0c3c071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:11.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.720+0000 7f0c3a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c3c0fca30 0x7f0c3c071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53402/0 (socket says 172.21.15.92:53402) 2024-05-06T12:06:11.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.720+0000 7f0c3a59c700 1 -- 172.21.15.92:0/2230495655 learned_addr learned my addr 172.21.15.92:0/2230495655 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:06:11.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.721+0000 7f0c3a59c700 1 -- 172.21.15.92:0/2230495655 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2c005040 con 0x7f0c3c0fca30 2024-05-06T12:06:11.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.721+0000 7f0c3a59c700 1 --2- 172.21.15.92:0/2230495655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c3c0fca30 0x7f0c3c071d00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00b900 tx=0x7f0c2c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:11.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.721+0000 7f0c2affd700 1 -- 172.21.15.92:0/2230495655 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0c2c009620 con 0x7f0c3c0fca30 2024-05-06T12:06:11.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.721+0000 7f0c40faa700 1 -- 172.21.15.92:0/2230495655 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c3c070310 con 0x7f0c3c0fca30 2024-05-06T12:06:11.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.721+0000 7f0c2affd700 1 -- 172.21.15.92:0/2230495655 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c2c009780 con 0x7f0c3c0fca30 2024-05-06T12:06:11.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.721+0000 7f0c2affd700 1 -- 172.21.15.92:0/2230495655 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0c2c01c590 con 0x7f0c3c0fca30 2024-05-06T12:06:11.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.721+0000 7f0c40faa700 1 -- 172.21.15.92:0/2230495655 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c3c070800 con 0x7f0c3c0fca30 2024-05-06T12:06:11.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.722+0000 7f0c2affd700 1 -- 172.21.15.92:0/2230495655 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f0c2c01c810 con 0x7f0c3c0fca30 2024-05-06T12:06:11.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.722+0000 7f0c40faa700 1 -- 172.21.15.92:0/2230495655 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c3c05f5c0 con 0x7f0c3c0fca30 2024-05-06T12:06:11.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.722+0000 7f0c2affd700 1 --2- 172.21.15.92:0/2230495655 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f0c24036880 0x7f0c24038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:11.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.722+0000 7f0c2affd700 1 -- 172.21.15.92:0/2230495655 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f0c2c049a00 con 0x7f0c3c0fca30 2024-05-06T12:06:11.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.722+0000 7f0c39d9b700 1 --2- 172.21.15.92:0/2230495655 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f0c24036880 0x7f0c24038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:11.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.722+0000 7f0c39d9b700 1 --2- 172.21.15.92:0/2230495655 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f0c24036880 0x7f0c24038d30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009990 tx=0x7f0c30006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:11.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.727+0000 7f0c2affd700 1 -- 172.21.15.92:0/2230495655 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0c2c025400 con 0x7f0c3c0fca30 2024-05-06T12:06:11.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.923+0000 7f0c40faa700 1 -- 172.21.15.92:0/2230495655 --> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c3c106480 con 0x7f0c24036880 2024-05-06T12:06:11.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.926+0000 7f0c2affd700 1 -- 172.21.15.92:0/2230495655 <== mgr.14162 v2:172.21.15.92:6800/543287293 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0c3c106480 con 0x7f0c24036880 2024-05-06T12:06:11.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.928+0000 7f0c40faa700 1 -- 172.21.15.92:0/2230495655 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f0c24036880 msgr2=0x7f0c24038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:11.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.928+0000 7f0c40faa700 1 --2- 172.21.15.92:0/2230495655 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f0c24036880 0x7f0c24038d30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009990 tx=0x7f0c30006e40 comp rx=0 tx=0).stop 2024-05-06T12:06:11.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.928+0000 7f0c40faa700 1 -- 172.21.15.92:0/2230495655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c3c0fca30 msgr2=0x7f0c3c071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:11.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.928+0000 7f0c40faa700 1 --2- 172.21.15.92:0/2230495655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c3c0fca30 0x7f0c3c071d00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00b900 tx=0x7f0c2c009e30 comp rx=0 tx=0).stop 2024-05-06T12:06:11.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.928+0000 7f0c40faa700 1 -- 172.21.15.92:0/2230495655 shutdown_connections 2024-05-06T12:06:11.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.928+0000 7f0c40faa700 1 --2- 172.21.15.92:0/2230495655 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f0c24036880 0x7f0c24038d30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:11.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.928+0000 7f0c40faa700 1 --2- 172.21.15.92:0/2230495655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c3c0fca30 0x7f0c3c071d00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:11.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.928+0000 7f0c40faa700 1 -- 172.21.15.92:0/2230495655 >> 172.21.15.92:0/2230495655 conn(0x7f0c3c0f8170 msgr2=0x7f0c3c103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:11.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.928+0000 7f0c40faa700 1 -- 172.21.15.92:0/2230495655 shutdown_connections 2024-05-06T12:06:11.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:11.928+0000 7f0c40faa700 1 -- 172.21.15.92:0/2230495655 wait complete. 2024-05-06T12:06:13.059 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:12 smithi092 ceph-mon[26954]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:06:13.059 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:12 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:13.399 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi190 2024-05-06T12:06:13.399 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-06T12:06:13.400 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/etc/ceph/ceph.conf 2024-05-06T12:06:13.418 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-06T12:06:13.418 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-06T12:06:13.479 INFO:tasks.cephadm:Adding host smithi190 to orchestrator... 2024-05-06T12:06:13.479 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph orch host add smithi190 2024-05-06T12:06:13.927 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:06:15.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:14 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:15.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:14 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:15.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:14 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:15.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:14 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:15.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:14 smithi092 ceph-mon[26954]: Deploying daemon node-exporter.smithi092 on smithi092 2024-05-06T12:06:15.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:15.452+0000 7f49334e8700 1 -- 172.21.15.92:0/1365823322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f492c0fb940 msgr2=0x7f492c0fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:15.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:15.452+0000 7f49334e8700 1 --2- 172.21.15.92:0/1365823322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f492c0fb940 0x7f492c0fbd50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f49280048c0 tx=0x7f4928009e30 comp rx=0 tx=0).stop 2024-05-06T12:06:15.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:15.452+0000 7f49334e8700 1 -- 172.21.15.92:0/1365823322 shutdown_connections 2024-05-06T12:06:15.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:15.452+0000 7f49334e8700 1 --2- 172.21.15.92:0/1365823322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f492c0fb940 0x7f492c0fbd50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:15.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:15.452+0000 7f49334e8700 1 -- 172.21.15.92:0/1365823322 >> 172.21.15.92:0/1365823322 conn(0x7f492c0f70a0 msgr2=0x7f492c0f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:15.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:15.452+0000 7f49334e8700 1 -- 172.21.15.92:0/1365823322 shutdown_connections 2024-05-06T12:06:15.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:15.452+0000 7f49334e8700 1 -- 172.21.15.92:0/1365823322 wait complete. 2024-05-06T12:06:15.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:15.453+0000 7f49334e8700 1 Processor -- start 2024-05-06T12:06:15.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:15.453+0000 7f49334e8700 1 -- start start 2024-05-06T12:06:15.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:15.453+0000 7f49334e8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f492c0fb940 0x7f492c10d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:15.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:15.453+0000 7f49334e8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c10d5c0 con 0x7f492c0fb940 2024-05-06T12:06:15.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:15.453+0000 7f4931284700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f492c0fb940 0x7f492c10d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:15.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:15.453+0000 7f4931284700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f492c0fb940 0x7f492c10d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45980/0 (socket says 172.21.15.92:45980) 2024-05-06T12:06:15.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:15.453+0000 7f4931284700 1 -- 172.21.15.92:0/1280566685 learned_addr learned my addr 172.21.15.92:0/1280566685 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:06:15.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:15.454+0000 7f4931284700 1 -- 172.21.15.92:0/1280566685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4928005040 con 0x7f492c0fb940 2024-05-06T12:06:15.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:15.454+0000 7f4931284700 1 --2- 172.21.15.92:0/1280566685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f492c0fb940 0x7f492c10d080 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f49280096b0 tx=0x7f4928009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:15.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:15.454+0000 7f491dffb700 1 -- 172.21.15.92:0/1280566685 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f492800bed0 con 0x7f492c0fb940 2024-05-06T12:06:15.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:15.454+0000 7f491dffb700 1 -- 172.21.15.92:0/1280566685 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f492800b5c0 con 0x7f492c0fb940 2024-05-06T12:06:15.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:15.454+0000 7f49334e8700 1 -- 172.21.15.92:0/1280566685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f492c10d7c0 con 0x7f492c0fb940 2024-05-06T12:06:15.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:15.454+0000 7f491dffb700 1 -- 172.21.15.92:0/1280566685 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f492801c5b0 con 0x7f492c0fb940 2024-05-06T12:06:15.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:15.454+0000 7f49334e8700 1 -- 172.21.15.92:0/1280566685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f492c10e330 con 0x7f492c0fb940 2024-05-06T12:06:15.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:15.455+0000 7f491dffb700 1 -- 172.21.15.92:0/1280566685 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f492801c710 con 0x7f492c0fb940 2024-05-06T12:06:15.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:15.455+0000 7f491dffb700 1 --2- 172.21.15.92:0/1280566685 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f49180368d0 0x7f4918038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:15.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:15.455+0000 7f49334e8700 1 -- 172.21.15.92:0/1280566685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f492c101e50 con 0x7f492c0fb940 2024-05-06T12:06:15.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:15.455+0000 7f491dffb700 1 -- 172.21.15.92:0/1280566685 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f49280499c0 con 0x7f492c0fb940 2024-05-06T12:06:15.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:15.455+0000 7f4930a83700 1 --2- 172.21.15.92:0/1280566685 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f49180368d0 0x7f4918038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:15.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:15.456+0000 7f4930a83700 1 --2- 172.21.15.92:0/1280566685 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f49180368d0 0x7f4918038d80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4920009990 tx=0x7f4920006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:15.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:15.460+0000 7f491dffb700 1 -- 172.21.15.92:0/1280566685 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4928023300 con 0x7f492c0fb940 2024-05-06T12:06:15.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:15.663+0000 7f49334e8700 1 -- 172.21.15.92:0/1280566685 --> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi190", "target": ["mon-mgr", ""]}) v1 -- 0x7f492c066450 con 0x7f49180368d0 2024-05-06T12:06:16.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:16 smithi092 ceph-mon[26954]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi190", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:06:17.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:17.646+0000 7f491dffb700 1 -- 172.21.15.92:0/1280566685 <== mgr.14162 v2:172.21.15.92:6800/543287293 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f492c066450 con 0x7f49180368d0 2024-05-06T12:06:17.649 INFO:teuthology.orchestra.run.smithi092.stdout:Added host 'smithi190' with addr '172.21.15.190' 2024-05-06T12:06:17.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:17.648+0000 7f49334e8700 1 -- 172.21.15.92:0/1280566685 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f49180368d0 msgr2=0x7f4918038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:17.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:17.648+0000 7f49334e8700 1 --2- 172.21.15.92:0/1280566685 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f49180368d0 0x7f4918038d80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4920009990 tx=0x7f4920006e40 comp rx=0 tx=0).stop 2024-05-06T12:06:17.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:17.648+0000 7f49334e8700 1 -- 172.21.15.92:0/1280566685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f492c0fb940 msgr2=0x7f492c10d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:17.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:17.648+0000 7f49334e8700 1 --2- 172.21.15.92:0/1280566685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f492c0fb940 0x7f492c10d080 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f49280096b0 tx=0x7f4928009e30 comp rx=0 tx=0).stop 2024-05-06T12:06:17.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:17.649+0000 7f49334e8700 1 -- 172.21.15.92:0/1280566685 shutdown_connections 2024-05-06T12:06:17.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:17.649+0000 7f49334e8700 1 --2- 172.21.15.92:0/1280566685 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f49180368d0 0x7f4918038d80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:17.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:17.649+0000 7f49334e8700 1 --2- 172.21.15.92:0/1280566685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f492c0fb940 0x7f492c10d080 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:17.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:17.649+0000 7f49334e8700 1 -- 172.21.15.92:0/1280566685 >> 172.21.15.92:0/1280566685 conn(0x7f492c0f70a0 msgr2=0x7f492c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:17.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:17.649+0000 7f49334e8700 1 -- 172.21.15.92:0/1280566685 shutdown_connections 2024-05-06T12:06:17.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:17.649+0000 7f49334e8700 1 -- 172.21.15.92:0/1280566685 wait complete. 2024-05-06T12:06:17.661 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:17 smithi092 ceph-mon[26954]: Deploying cephadm binary to smithi190 2024-05-06T12:06:17.661 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:17 smithi092 ceph-mon[26954]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:06:17.661 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:17 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:18.895 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-06T12:06:18.916 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:18 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:18.916 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:18 smithi092 ceph-mon[26954]: Added host smithi190 2024-05-06T12:06:18.916 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:18 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:18.916 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:18 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:18.916 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:18 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:18.916 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:18 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:19.181 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:06:19.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:19 smithi092 ceph-mon[26954]: Deploying daemon alertmanager.smithi092 on smithi092 2024-05-06T12:06:19.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:19 smithi092 ceph-mon[26954]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:06:21.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.116+0000 7fbcb213d700 1 -- 172.21.15.92:0/270550272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcac0fca40 msgr2=0x7fbcac0fce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:21.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.116+0000 7fbcb213d700 1 --2- 172.21.15.92:0/270550272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcac0fca40 0x7fbcac0fce50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c0048c0 tx=0x7fbc9c009e30 comp rx=0 tx=0).stop 2024-05-06T12:06:21.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.116+0000 7fbcb213d700 1 -- 172.21.15.92:0/270550272 shutdown_connections 2024-05-06T12:06:21.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.116+0000 7fbcb213d700 1 --2- 172.21.15.92:0/270550272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcac0fca40 0x7fbcac0fce50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:21.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.116+0000 7fbcb213d700 1 -- 172.21.15.92:0/270550272 >> 172.21.15.92:0/270550272 conn(0x7fbcac0f81a0 msgr2=0x7fbcac0fa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:21.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.116+0000 7fbcb213d700 1 -- 172.21.15.92:0/270550272 shutdown_connections 2024-05-06T12:06:21.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.116+0000 7fbcb213d700 1 -- 172.21.15.92:0/270550272 wait complete. 2024-05-06T12:06:21.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.117+0000 7fbcb213d700 1 Processor -- start 2024-05-06T12:06:21.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.117+0000 7fbcb213d700 1 -- start start 2024-05-06T12:06:21.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.117+0000 7fbcb213d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcac0fca40 0x7fbcac10d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:21.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.117+0000 7fbcb213d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac10d5d0 con 0x7fbcac0fca40 2024-05-06T12:06:21.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.117+0000 7fbcab7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcac0fca40 0x7fbcac10d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:21.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.117+0000 7fbcab7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcac0fca40 0x7fbcac10d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46008/0 (socket says 172.21.15.92:46008) 2024-05-06T12:06:21.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.117+0000 7fbcab7fe700 1 -- 172.21.15.92:0/1334971808 learned_addr learned my addr 172.21.15.92:0/1334971808 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:06:21.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.117+0000 7fbcab7fe700 1 -- 172.21.15.92:0/1334971808 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc9c005040 con 0x7fbcac0fca40 2024-05-06T12:06:21.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.118+0000 7fbcab7fe700 1 --2- 172.21.15.92:0/1334971808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcac0fca40 0x7fbcac10d090 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c0096b0 tx=0x7fbc9c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:21.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.118+0000 7fbc93fff700 1 -- 172.21.15.92:0/1334971808 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc9c00bed0 con 0x7fbcac0fca40 2024-05-06T12:06:21.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.118+0000 7fbc93fff700 1 -- 172.21.15.92:0/1334971808 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc9c00b5c0 con 0x7fbcac0fca40 2024-05-06T12:06:21.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.118+0000 7fbcb213d700 1 -- 172.21.15.92:0/1334971808 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcac10d7d0 con 0x7fbcac0fca40 2024-05-06T12:06:21.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.118+0000 7fbc93fff700 1 -- 172.21.15.92:0/1334971808 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc9c01c5b0 con 0x7fbcac0fca40 2024-05-06T12:06:21.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.118+0000 7fbcb213d700 1 -- 172.21.15.92:0/1334971808 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcac10e340 con 0x7fbcac0fca40 2024-05-06T12:06:21.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.118+0000 7fbc93fff700 1 -- 172.21.15.92:0/1334971808 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7fbc9c01c820 con 0x7fbcac0fca40 2024-05-06T12:06:21.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.119+0000 7fbc93fff700 1 --2- 172.21.15.92:0/1334971808 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7fbc94036880 0x7fbc94038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:21.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.119+0000 7fbc93fff700 1 -- 172.21.15.92:0/1334971808 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fbc9c049a40 con 0x7fbcac0fca40 2024-05-06T12:06:21.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.119+0000 7fbcb213d700 1 -- 172.21.15.92:0/1334971808 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcac102f50 con 0x7fbcac0fca40 2024-05-06T12:06:21.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.119+0000 7fbcaaffd700 1 --2- 172.21.15.92:0/1334971808 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7fbc94036880 0x7fbc94038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:21.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.119+0000 7fbcaaffd700 1 --2- 172.21.15.92:0/1334971808 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7fbc94036880 0x7fbc94038d30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbca0009990 tx=0x7fbca0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:21.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.124+0000 7fbc93fff700 1 -- 172.21.15.92:0/1334971808 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fbc9c023300 con 0x7fbcac0fca40 2024-05-06T12:06:21.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.334+0000 7fbcb213d700 1 -- 172.21.15.92:0/1334971808 --> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbcac102930 con 0x7fbc94036880 2024-05-06T12:06:21.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.335+0000 7fbc93fff700 1 -- 172.21.15.92:0/1334971808 <== mgr.14162 v2:172.21.15.92:6800/543287293 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7fbca0009590 con 0x7fbc94036880 2024-05-06T12:06:21.338 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:06:21.338 INFO:teuthology.orchestra.run.smithi092.stdout:[{"addr": "172.21.15.92", "hostname": "smithi092", "labels": [], "status": ""}, {"addr": "172.21.15.190", "hostname": "smithi190", "labels": [], "status": ""}] 2024-05-06T12:06:21.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.337+0000 7fbcb213d700 1 -- 172.21.15.92:0/1334971808 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7fbc94036880 msgr2=0x7fbc94038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:21.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.337+0000 7fbcb213d700 1 --2- 172.21.15.92:0/1334971808 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7fbc94036880 0x7fbc94038d30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbca0009990 tx=0x7fbca0006e40 comp rx=0 tx=0).stop 2024-05-06T12:06:21.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.337+0000 7fbcb213d700 1 -- 172.21.15.92:0/1334971808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcac0fca40 msgr2=0x7fbcac10d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:21.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.337+0000 7fbcb213d700 1 --2- 172.21.15.92:0/1334971808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcac0fca40 0x7fbcac10d090 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c0096b0 tx=0x7fbc9c009e30 comp rx=0 tx=0).stop 2024-05-06T12:06:21.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.337+0000 7fbcb213d700 1 -- 172.21.15.92:0/1334971808 shutdown_connections 2024-05-06T12:06:21.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.337+0000 7fbcb213d700 1 --2- 172.21.15.92:0/1334971808 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7fbc94036880 0x7fbc94038d30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:21.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.337+0000 7fbcb213d700 1 --2- 172.21.15.92:0/1334971808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcac0fca40 0x7fbcac10d090 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:21.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.337+0000 7fbcb213d700 1 -- 172.21.15.92:0/1334971808 >> 172.21.15.92:0/1334971808 conn(0x7fbcac0f81a0 msgr2=0x7fbcac103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:21.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.337+0000 7fbcb213d700 1 -- 172.21.15.92:0/1334971808 shutdown_connections 2024-05-06T12:06:21.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:21.337+0000 7fbcb213d700 1 -- 172.21.15.92:0/1334971808 wait complete. 2024-05-06T12:06:21.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:21 smithi092 ceph-mon[26954]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:06:21.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:21 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:22.263 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-06T12:06:22.263 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph osd crush tunables default 2024-05-06T12:06:22.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:22 smithi092 ceph-mon[26954]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-06T12:06:23.538 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:06:23.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:23 smithi092 ceph-mon[26954]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:06:25.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:25 smithi092 ceph-mon[26954]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:06:27.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:27.611+0000 7f3c78871700 1 -- 172.21.15.92:0/2515356266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c74065d30 msgr2=0x7f3c740ff630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:27.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:27.611+0000 7f3c78871700 1 --2- 172.21.15.92:0/2515356266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c74065d30 0x7f3c740ff630 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3c640048c0 tx=0x7f3c64009e30 comp rx=0 tx=0).stop 2024-05-06T12:06:27.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:27.612+0000 7f3c78871700 1 -- 172.21.15.92:0/2515356266 shutdown_connections 2024-05-06T12:06:27.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:27.612+0000 7f3c78871700 1 --2- 172.21.15.92:0/2515356266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c74065d30 0x7f3c740ff630 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:27.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:27.612+0000 7f3c78871700 1 -- 172.21.15.92:0/2515356266 >> 172.21.15.92:0/2515356266 conn(0x7f3c740f70a0 msgr2=0x7f3c740f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:27.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:27.612+0000 7f3c78871700 1 -- 172.21.15.92:0/2515356266 shutdown_connections 2024-05-06T12:06:27.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:27.612+0000 7f3c78871700 1 -- 172.21.15.92:0/2515356266 wait complete. 2024-05-06T12:06:27.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:27.612+0000 7f3c78871700 1 Processor -- start 2024-05-06T12:06:27.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:27.612+0000 7f3c78871700 1 -- start start 2024-05-06T12:06:27.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:27.612+0000 7f3c78871700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c74065d30 0x7f3c740fcf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:27.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:27.612+0000 7f3c78871700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c740fd4c0 con 0x7f3c74065d30 2024-05-06T12:06:27.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:27.613+0000 7f3c7259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c74065d30 0x7f3c740fcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:27.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:27.613+0000 7f3c7259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c74065d30 0x7f3c740fcf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49660/0 (socket says 172.21.15.92:49660) 2024-05-06T12:06:27.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:27.613+0000 7f3c7259c700 1 -- 172.21.15.92:0/1110224241 learned_addr learned my addr 172.21.15.92:0/1110224241 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:06:27.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:27.613+0000 7f3c7259c700 1 -- 172.21.15.92:0/1110224241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c64005040 con 0x7f3c74065d30 2024-05-06T12:06:27.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:27.613+0000 7f3c7259c700 1 --2- 172.21.15.92:0/1110224241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c74065d30 0x7f3c740fcf80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3c6400b900 tx=0x7f3c64009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:27.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:27.613+0000 7f3c62ffd700 1 -- 172.21.15.92:0/1110224241 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3c6400b5c0 con 0x7f3c74065d30 2024-05-06T12:06:27.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:27.613+0000 7f3c62ffd700 1 -- 172.21.15.92:0/1110224241 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c6400be70 con 0x7f3c74065d30 2024-05-06T12:06:27.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:27.613+0000 7f3c78871700 1 -- 172.21.15.92:0/1110224241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c740fb590 con 0x7f3c74065d30 2024-05-06T12:06:27.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:27.613+0000 7f3c62ffd700 1 -- 172.21.15.92:0/1110224241 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3c6401c590 con 0x7f3c74065d30 2024-05-06T12:06:27.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:27.614+0000 7f3c78871700 1 -- 172.21.15.92:0/1110224241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c740fba80 con 0x7f3c74065d30 2024-05-06T12:06:27.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:27.614+0000 7f3c62ffd700 1 -- 172.21.15.92:0/1110224241 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f3c6401c800 con 0x7f3c74065d30 2024-05-06T12:06:27.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:27.615+0000 7f3c78871700 1 -- 172.21.15.92:0/1110224241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c7405f5c0 con 0x7f3c74065d30 2024-05-06T12:06:27.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:27.615+0000 7f3c62ffd700 1 --2- 172.21.15.92:0/1110224241 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f3c5c0368d0 0x7f3c5c038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:27.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:27.615+0000 7f3c62ffd700 1 -- 172.21.15.92:0/1110224241 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f3c64049380 con 0x7f3c74065d30 2024-05-06T12:06:27.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:27.615+0000 7f3c71d9b700 1 --2- 172.21.15.92:0/1110224241 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f3c5c0368d0 0x7f3c5c038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:27.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:27.615+0000 7f3c71d9b700 1 --2- 172.21.15.92:0/1110224241 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f3c5c0368d0 0x7f3c5c038d80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009990 tx=0x7f3c68006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:27.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:27.619+0000 7f3c62ffd700 1 -- 172.21.15.92:0/1110224241 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3c64011550 con 0x7f3c74065d30 2024-05-06T12:06:27.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:27.832+0000 7f3c78871700 1 -- 172.21.15.92:0/1110224241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f3c74106410 con 0x7f3c74065d30 2024-05-06T12:06:27.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:27 smithi092 ceph-mon[26954]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:06:28.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:28.663+0000 7f3c62ffd700 1 -- 172.21.15.92:0/1110224241 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f3c64021070 con 0x7f3c74065d30 2024-05-06T12:06:28.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:28.665+0000 7f3c78871700 1 -- 172.21.15.92:0/1110224241 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f3c5c0368d0 msgr2=0x7f3c5c038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:28.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:28.665+0000 7f3c78871700 1 --2- 172.21.15.92:0/1110224241 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f3c5c0368d0 0x7f3c5c038d80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009990 tx=0x7f3c68006e40 comp rx=0 tx=0).stop 2024-05-06T12:06:28.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:28.665+0000 7f3c78871700 1 -- 172.21.15.92:0/1110224241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c74065d30 msgr2=0x7f3c740fcf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:28.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:28.665+0000 7f3c78871700 1 --2- 172.21.15.92:0/1110224241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c74065d30 0x7f3c740fcf80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3c6400b900 tx=0x7f3c64009e30 comp rx=0 tx=0).stop 2024-05-06T12:06:28.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:28.665+0000 7f3c78871700 1 -- 172.21.15.92:0/1110224241 shutdown_connections 2024-05-06T12:06:28.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:28.665+0000 7f3c78871700 1 --2- 172.21.15.92:0/1110224241 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f3c5c0368d0 0x7f3c5c038d80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:28.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:28.665+0000 7f3c78871700 1 --2- 172.21.15.92:0/1110224241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c74065d30 0x7f3c740fcf80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:28.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:28.665+0000 7f3c78871700 1 -- 172.21.15.92:0/1110224241 >> 172.21.15.92:0/1110224241 conn(0x7f3c740f70a0 msgr2=0x7f3c74103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:28.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:28.665+0000 7f3c78871700 1 -- 172.21.15.92:0/1110224241 shutdown_connections 2024-05-06T12:06:28.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:06:28.665+0000 7f3c78871700 1 -- 172.21.15.92:0/1110224241 wait complete. 2024-05-06T12:06:28.669 INFO:teuthology.orchestra.run.smithi092.stderr:adjusted tunables profile to default 2024-05-06T12:06:28.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:28 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/1110224241' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-06T12:06:28.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:28 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:28.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:28 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:28.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:28 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:28.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:28 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:28.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:28 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:28.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:28 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:28.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:28 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-05-06T12:06:28.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:28 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:29.278 INFO:tasks.cephadm:Adding mon.smithi092 on smithi092 2024-05-06T12:06:29.278 INFO:tasks.cephadm:Adding mon.smithi190 on smithi190 2024-05-06T12:06:29.279 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph orch apply mon '2;smithi092:172.21.15.92=smithi092;smithi190:172.21.15.190=smithi190' 2024-05-06T12:06:29.525 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-06T12:06:29.574 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-06T12:06:29.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:29 smithi092 ceph-mon[26954]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:06:29.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:29 smithi092 ceph-mon[26954]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-05-06T12:06:29.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:29 smithi092 ceph-mon[26954]: Deploying daemon grafana.smithi092 on smithi092 2024-05-06T12:06:29.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:29 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/1110224241' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-06T12:06:29.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:29 smithi092 ceph-mon[26954]: osdmap e4: 0 total, 0 up, 0 in 2024-05-06T12:06:31.034 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.030+0000 7fb4a5765700 1 -- 172.21.15.190:0/641568398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4a00fca30 msgr2=0x7fb4a00fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:31.034 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.030+0000 7fb4a5765700 1 --2- 172.21.15.190:0/641568398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4a00fca30 0x7fb4a00fce40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb490009c60 tx=0x7fb490004d10 comp rx=0 tx=0).stop 2024-05-06T12:06:31.034 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.030+0000 7fb4a5765700 1 -- 172.21.15.190:0/641568398 shutdown_connections 2024-05-06T12:06:31.034 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.030+0000 7fb4a5765700 1 --2- 172.21.15.190:0/641568398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4a00fca30 0x7fb4a00fce40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:31.034 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.030+0000 7fb4a5765700 1 -- 172.21.15.190:0/641568398 >> 172.21.15.190:0/641568398 conn(0x7fb4a00f8190 msgr2=0x7fb4a00fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:31.034 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.030+0000 7fb4a5765700 1 -- 172.21.15.190:0/641568398 shutdown_connections 2024-05-06T12:06:31.034 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.030+0000 7fb4a5765700 1 -- 172.21.15.190:0/641568398 wait complete. 2024-05-06T12:06:31.034 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.031+0000 7fb4a5765700 1 Processor -- start 2024-05-06T12:06:31.035 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.031+0000 7fb4a5765700 1 -- start start 2024-05-06T12:06:31.035 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.031+0000 7fb4a5765700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4a00fca30 0x7fb4a010d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:31.035 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.031+0000 7fb4a5765700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a010d5c0 con 0x7fb4a00fca30 2024-05-06T12:06:31.035 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.031+0000 7fb49effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4a00fca30 0x7fb4a010d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:31.035 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.031+0000 7fb49effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4a00fca30 0x7fb4a010d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.190:50400/0 (socket says 172.21.15.190:50400) 2024-05-06T12:06:31.035 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.031+0000 7fb49effd700 1 -- 172.21.15.190:0/3322821596 learned_addr learned my addr 172.21.15.190:0/3322821596 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-06T12:06:31.035 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.032+0000 7fb49effd700 1 -- 172.21.15.190:0/3322821596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb490009940 con 0x7fb4a00fca30 2024-05-06T12:06:31.035 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.032+0000 7fb49effd700 1 --2- 172.21.15.190:0/3322821596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4a00fca30 0x7fb4a010d080 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb490009500 tx=0x7fb490004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:31.036 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.032+0000 7fb4977fe700 1 -- 172.21.15.190:0/3322821596 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb490007c80 con 0x7fb4a00fca30 2024-05-06T12:06:31.036 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.032+0000 7fb4977fe700 1 -- 172.21.15.190:0/3322821596 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb49001d910 con 0x7fb4a00fca30 2024-05-06T12:06:31.036 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.032+0000 7fb4a5765700 1 -- 172.21.15.190:0/3322821596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4a010d7c0 con 0x7fb4a00fca30 2024-05-06T12:06:31.036 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.032+0000 7fb4977fe700 1 -- 172.21.15.190:0/3322821596 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb4900129d0 con 0x7fb4a00fca30 2024-05-06T12:06:31.036 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.032+0000 7fb4a5765700 1 -- 172.21.15.190:0/3322821596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a010e330 con 0x7fb4a00fca30 2024-05-06T12:06:31.037 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.033+0000 7fb4977fe700 1 -- 172.21.15.190:0/3322821596 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7fb490019040 con 0x7fb4a00fca30 2024-05-06T12:06:31.037 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.033+0000 7fb4a5765700 1 -- 172.21.15.190:0/3322821596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4a0102f40 con 0x7fb4a00fca30 2024-05-06T12:06:31.037 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.033+0000 7fb4977fe700 1 --2- 172.21.15.190:0/3322821596 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7fb48c0368d0 0x7fb48c038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:31.037 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.033+0000 7fb4977fe700 1 -- 172.21.15.190:0/3322821596 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fb49004a540 con 0x7fb4a00fca30 2024-05-06T12:06:31.037 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.033+0000 7fb49e7fc700 1 --2- 172.21.15.190:0/3322821596 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7fb48c0368d0 0x7fb48c038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:31.037 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.034+0000 7fb49e7fc700 1 --2- 172.21.15.190:0/3322821596 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7fb48c0368d0 0x7fb48c038d80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb488009990 tx=0x7fb488006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:31.040 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.038+0000 7fb4977fe700 1 -- 172.21.15.190:0/3322821596 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb490024310 con 0x7fb4a00fca30 2024-05-06T12:06:31.235 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.232+0000 7fb4a5765700 1 -- 172.21.15.190:0/3322821596 --> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi092:172.21.15.92=smithi092;smithi190:172.21.15.190=smithi190", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4a010e570 con 0x7fb48c0368d0 2024-05-06T12:06:31.240 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.238+0000 7fb4977fe700 1 -- 172.21.15.190:0/3322821596 <== mgr.14162 v2:172.21.15.92:6800/543287293 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fb4a010e570 con 0x7fb48c0368d0 2024-05-06T12:06:31.240 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled mon update... 2024-05-06T12:06:31.242 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.240+0000 7fb4a5765700 1 -- 172.21.15.190:0/3322821596 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7fb48c0368d0 msgr2=0x7fb48c038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:31.242 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.240+0000 7fb4a5765700 1 --2- 172.21.15.190:0/3322821596 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7fb48c0368d0 0x7fb48c038d80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb488009990 tx=0x7fb488006e40 comp rx=0 tx=0).stop 2024-05-06T12:06:31.242 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.240+0000 7fb4a5765700 1 -- 172.21.15.190:0/3322821596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4a00fca30 msgr2=0x7fb4a010d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:31.242 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.240+0000 7fb4a5765700 1 --2- 172.21.15.190:0/3322821596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4a00fca30 0x7fb4a010d080 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb490009500 tx=0x7fb490004d10 comp rx=0 tx=0).stop 2024-05-06T12:06:31.242 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.240+0000 7fb4a5765700 1 -- 172.21.15.190:0/3322821596 shutdown_connections 2024-05-06T12:06:31.243 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.240+0000 7fb4a5765700 1 --2- 172.21.15.190:0/3322821596 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7fb48c0368d0 0x7fb48c038d80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:31.243 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.240+0000 7fb4a5765700 1 --2- 172.21.15.190:0/3322821596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4a00fca30 0x7fb4a010d080 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:31.243 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.240+0000 7fb4a5765700 1 -- 172.21.15.190:0/3322821596 >> 172.21.15.190:0/3322821596 conn(0x7fb4a00f8190 msgr2=0x7fb4a0103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:31.243 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.240+0000 7fb4a5765700 1 -- 172.21.15.190:0/3322821596 shutdown_connections 2024-05-06T12:06:31.243 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:31.240+0000 7fb4a5765700 1 -- 172.21.15.190:0/3322821596 wait complete. 2024-05-06T12:06:31.819 DEBUG:teuthology.orchestra.run.smithi190:mon.smithi190> sudo journalctl -f -n 0 -u ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@mon.smithi190.service 2024-05-06T12:06:31.821 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-06T12:06:31.822 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-06T12:06:31.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:31 smithi092 ceph-mon[26954]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:06:31.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:31 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:31.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:31 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:32.093 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-06T12:06:32.140 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-06T12:06:32.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:32 smithi092 ceph-mon[26954]: from='client.14194 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi092:172.21.15.92=smithi092;smithi190:172.21.15.190=smithi190", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:06:32.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:32 smithi092 ceph-mon[26954]: Saving service mon spec with placement smithi092:172.21.15.92=smithi092;smithi190:172.21.15.190=smithi190;count:2 2024-05-06T12:06:33.459 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.454+0000 7f569328a700 1 -- 172.21.15.190:0/3857248583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f568c0fb940 msgr2=0x7f568c0fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:33.459 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.454+0000 7f569328a700 1 --2- 172.21.15.190:0/3857248583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f568c0fb940 0x7f568c0fbd50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f56880048c0 tx=0x7f5688009e30 comp rx=0 tx=0).stop 2024-05-06T12:06:33.459 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.454+0000 7f569328a700 1 -- 172.21.15.190:0/3857248583 shutdown_connections 2024-05-06T12:06:33.459 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.454+0000 7f569328a700 1 --2- 172.21.15.190:0/3857248583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f568c0fb940 0x7f568c0fbd50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:33.459 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.454+0000 7f569328a700 1 -- 172.21.15.190:0/3857248583 >> 172.21.15.190:0/3857248583 conn(0x7f568c0f70a0 msgr2=0x7f568c0f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:33.459 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.454+0000 7f569328a700 1 -- 172.21.15.190:0/3857248583 shutdown_connections 2024-05-06T12:06:33.459 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.454+0000 7f569328a700 1 -- 172.21.15.190:0/3857248583 wait complete. 2024-05-06T12:06:33.460 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.454+0000 7f569328a700 1 Processor -- start 2024-05-06T12:06:33.460 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.454+0000 7f569328a700 1 -- start start 2024-05-06T12:06:33.460 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.455+0000 7f569328a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f568c0fb940 0x7f568c10d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:33.460 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.455+0000 7f569328a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c10d5a0 con 0x7f568c0fb940 2024-05-06T12:06:33.460 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.455+0000 7f5691026700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f568c0fb940 0x7f568c10d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:33.460 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.455+0000 7f5691026700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f568c0fb940 0x7f568c10d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.190:50416/0 (socket says 172.21.15.190:50416) 2024-05-06T12:06:33.460 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.455+0000 7f5691026700 1 -- 172.21.15.190:0/2168895209 learned_addr learned my addr 172.21.15.190:0/2168895209 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-06T12:06:33.460 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.455+0000 7f5691026700 1 -- 172.21.15.190:0/2168895209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5688005040 con 0x7f568c0fb940 2024-05-06T12:06:33.460 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.455+0000 7f5691026700 1 --2- 172.21.15.190:0/2168895209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f568c0fb940 0x7f568c10d060 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f56880096b0 tx=0x7f5688009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:33.460 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.455+0000 7f5681ffb700 1 -- 172.21.15.190:0/2168895209 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f568800bed0 con 0x7f568c0fb940 2024-05-06T12:06:33.460 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.455+0000 7f5681ffb700 1 -- 172.21.15.190:0/2168895209 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f568800b5c0 con 0x7f568c0fb940 2024-05-06T12:06:33.460 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.455+0000 7f5681ffb700 1 -- 172.21.15.190:0/2168895209 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f568801c5b0 con 0x7f568c0fb940 2024-05-06T12:06:33.460 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.456+0000 7f569328a700 1 -- 172.21.15.190:0/2168895209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f568c10d7a0 con 0x7f568c0fb940 2024-05-06T12:06:33.460 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.456+0000 7f569328a700 1 -- 172.21.15.190:0/2168895209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f568c10e310 con 0x7f568c0fb940 2024-05-06T12:06:33.461 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.457+0000 7f5681ffb700 1 -- 172.21.15.190:0/2168895209 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f5688013a80 con 0x7f568c0fb940 2024-05-06T12:06:33.462 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.457+0000 7f569328a700 1 -- 172.21.15.190:0/2168895209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f568c101e50 con 0x7f568c0fb940 2024-05-06T12:06:33.462 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.457+0000 7f5681ffb700 1 --2- 172.21.15.190:0/2168895209 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f5678036880 0x7f5678038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:33.462 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.457+0000 7f5681ffb700 1 -- 172.21.15.190:0/2168895209 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f5688049a00 con 0x7f568c0fb940 2024-05-06T12:06:33.462 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.457+0000 7f5690825700 1 --2- 172.21.15.190:0/2168895209 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f5678036880 0x7f5678038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:33.462 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.457+0000 7f5690825700 1 --2- 172.21.15.190:0/2168895209 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f5678036880 0x7f5678038d30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f567c009990 tx=0x7f567c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:33.466 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.463+0000 7f5681ffb700 1 -- 172.21.15.190:0/2168895209 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5688011960 con 0x7f568c0fb940 2024-05-06T12:06:33.731 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.728+0000 7f569328a700 1 -- 172.21.15.190:0/2168895209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f568c10e550 con 0x7f568c0fb940 2024-05-06T12:06:33.732 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.728+0000 7f5681ffb700 1 -- 172.21.15.190:0/2168895209 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f5688011370 con 0x7f568c0fb940 2024-05-06T12:06:33.732 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:06:33.732 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":1,"fsid":"aee1d912-0ba0-11ef-bc97-c7b262605968","modified":"2024-05-06T12:04:48.255830Z","created":"2024-05-06T12:04:48.255830Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi092","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-06T12:06:33.734 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.730+0000 7f569328a700 1 -- 172.21.15.190:0/2168895209 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f5678036880 msgr2=0x7f5678038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:33.734 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.730+0000 7f569328a700 1 --2- 172.21.15.190:0/2168895209 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f5678036880 0x7f5678038d30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f567c009990 tx=0x7f567c006e40 comp rx=0 tx=0).stop 2024-05-06T12:06:33.734 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.730+0000 7f569328a700 1 -- 172.21.15.190:0/2168895209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f568c0fb940 msgr2=0x7f568c10d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:33.734 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.730+0000 7f569328a700 1 --2- 172.21.15.190:0/2168895209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f568c0fb940 0x7f568c10d060 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f56880096b0 tx=0x7f5688009e30 comp rx=0 tx=0).stop 2024-05-06T12:06:33.734 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.730+0000 7f569328a700 1 -- 172.21.15.190:0/2168895209 shutdown_connections 2024-05-06T12:06:33.734 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.730+0000 7f569328a700 1 --2- 172.21.15.190:0/2168895209 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f5678036880 0x7f5678038d30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:33.734 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.730+0000 7f569328a700 1 --2- 172.21.15.190:0/2168895209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f568c0fb940 0x7f568c10d060 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:33.734 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.730+0000 7f569328a700 1 -- 172.21.15.190:0/2168895209 >> 172.21.15.190:0/2168895209 conn(0x7f568c0f70a0 msgr2=0x7f568c065ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:33.735 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.730+0000 7f569328a700 1 -- 172.21.15.190:0/2168895209 shutdown_connections 2024-05-06T12:06:33.735 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:33.730+0000 7f569328a700 1 -- 172.21.15.190:0/2168895209 wait complete. 2024-05-06T12:06:33.735 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 1 2024-05-06T12:06:33.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:33 smithi092 ceph-mon[26954]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:06:34.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:34 smithi092 ceph-mon[26954]: from='client.? 172.21.15.190:0/2168895209' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-06T12:06:35.264 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-06T12:06:35.264 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-06T12:06:35.512 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-06T12:06:35.559 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-06T12:06:35.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:35 smithi092 ceph-mon[26954]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:06:36.970 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:36.965+0000 7fce5302c700 1 -- 172.21.15.190:0/1378430280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce4c0fca60 msgr2=0x7fce4c0fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:36.970 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:36.965+0000 7fce5302c700 1 --2- 172.21.15.190:0/1378430280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce4c0fca60 0x7fce4c0fce70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fce400048c0 tx=0x7fce40009e30 comp rx=0 tx=0).stop 2024-05-06T12:06:36.970 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:36.965+0000 7fce5302c700 1 -- 172.21.15.190:0/1378430280 shutdown_connections 2024-05-06T12:06:36.970 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:36.965+0000 7fce5302c700 1 --2- 172.21.15.190:0/1378430280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce4c0fca60 0x7fce4c0fce70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:36.970 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:36.965+0000 7fce5302c700 1 -- 172.21.15.190:0/1378430280 >> 172.21.15.190:0/1378430280 conn(0x7fce4c0f81c0 msgr2=0x7fce4c0fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:36.970 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:36.965+0000 7fce5302c700 1 -- 172.21.15.190:0/1378430280 shutdown_connections 2024-05-06T12:06:36.971 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:36.966+0000 7fce5302c700 1 -- 172.21.15.190:0/1378430280 wait complete. 2024-05-06T12:06:36.971 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:36.966+0000 7fce5302c700 1 Processor -- start 2024-05-06T12:06:36.971 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:36.966+0000 7fce5302c700 1 -- start start 2024-05-06T12:06:36.971 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:36.966+0000 7fce5302c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce4c0fca60 0x7fce4c10d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:36.971 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:36.966+0000 7fce5302c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c10d5f0 con 0x7fce4c0fca60 2024-05-06T12:06:36.971 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:36.966+0000 7fce50dc8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce4c0fca60 0x7fce4c10d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:36.971 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:36.966+0000 7fce50dc8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce4c0fca60 0x7fce4c10d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.190:60904/0 (socket says 172.21.15.190:60904) 2024-05-06T12:06:36.971 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:36.966+0000 7fce50dc8700 1 -- 172.21.15.190:0/2763245915 learned_addr learned my addr 172.21.15.190:0/2763245915 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-06T12:06:36.971 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:36.967+0000 7fce50dc8700 1 -- 172.21.15.190:0/2763245915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce40005040 con 0x7fce4c0fca60 2024-05-06T12:06:36.971 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:36.967+0000 7fce50dc8700 1 --2- 172.21.15.190:0/2763245915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce4c0fca60 0x7fce4c10d0b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fce400096b0 tx=0x7fce40009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:36.972 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:36.967+0000 7fce497fa700 1 -- 172.21.15.190:0/2763245915 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fce4000bed0 con 0x7fce4c0fca60 2024-05-06T12:06:36.972 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:36.967+0000 7fce497fa700 1 -- 172.21.15.190:0/2763245915 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce4000b5c0 con 0x7fce4c0fca60 2024-05-06T12:06:36.973 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:36.967+0000 7fce5302c700 1 -- 172.21.15.190:0/2763245915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce4c10d7f0 con 0x7fce4c0fca60 2024-05-06T12:06:36.973 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:36.967+0000 7fce497fa700 1 -- 172.21.15.190:0/2763245915 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fce4001c5b0 con 0x7fce4c0fca60 2024-05-06T12:06:36.973 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:36.967+0000 7fce5302c700 1 -- 172.21.15.190:0/2763245915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce4c10e360 con 0x7fce4c0fca60 2024-05-06T12:06:36.973 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:36.968+0000 7fce497fa700 1 -- 172.21.15.190:0/2763245915 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7fce40013420 con 0x7fce4c0fca60 2024-05-06T12:06:36.973 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:36.968+0000 7fce5302c700 1 -- 172.21.15.190:0/2763245915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce4c102f70 con 0x7fce4c0fca60 2024-05-06T12:06:36.973 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:36.968+0000 7fce497fa700 1 --2- 172.21.15.190:0/2763245915 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7fce3c036880 0x7fce3c038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:36.973 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:36.968+0000 7fce497fa700 1 -- 172.21.15.190:0/2763245915 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fce40049b10 con 0x7fce4c0fca60 2024-05-06T12:06:36.973 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:36.968+0000 7fce4bfff700 1 --2- 172.21.15.190:0/2763245915 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7fce3c036880 0x7fce3c038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:36.973 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:36.969+0000 7fce4bfff700 1 --2- 172.21.15.190:0/2763245915 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7fce3c036880 0x7fce3c038d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fce38009990 tx=0x7fce38006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:36.977 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:36.973+0000 7fce497fa700 1 -- 172.21.15.190:0/2763245915 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fce40025400 con 0x7fce4c0fca60 2024-05-06T12:06:37.238 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:37.233+0000 7fce5302c700 1 -- 172.21.15.190:0/2763245915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fce4c10ed70 con 0x7fce4c0fca60 2024-05-06T12:06:37.238 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:37.233+0000 7fce497fa700 1 -- 172.21.15.190:0/2763245915 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fce40004d30 con 0x7fce4c0fca60 2024-05-06T12:06:37.238 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:06:37.238 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":1,"fsid":"aee1d912-0ba0-11ef-bc97-c7b262605968","modified":"2024-05-06T12:04:48.255830Z","created":"2024-05-06T12:04:48.255830Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi092","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-06T12:06:37.239 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:37.235+0000 7fce5302c700 1 -- 172.21.15.190:0/2763245915 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7fce3c036880 msgr2=0x7fce3c038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:37.239 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:37.235+0000 7fce5302c700 1 --2- 172.21.15.190:0/2763245915 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7fce3c036880 0x7fce3c038d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fce38009990 tx=0x7fce38006e40 comp rx=0 tx=0).stop 2024-05-06T12:06:37.239 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:37.235+0000 7fce5302c700 1 -- 172.21.15.190:0/2763245915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce4c0fca60 msgr2=0x7fce4c10d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:37.239 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:37.235+0000 7fce5302c700 1 --2- 172.21.15.190:0/2763245915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce4c0fca60 0x7fce4c10d0b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fce400096b0 tx=0x7fce40009e30 comp rx=0 tx=0).stop 2024-05-06T12:06:37.239 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:37.236+0000 7fce5302c700 1 -- 172.21.15.190:0/2763245915 shutdown_connections 2024-05-06T12:06:37.239 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:37.236+0000 7fce5302c700 1 --2- 172.21.15.190:0/2763245915 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7fce3c036880 0x7fce3c038d30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:37.239 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:37.236+0000 7fce5302c700 1 --2- 172.21.15.190:0/2763245915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce4c0fca60 0x7fce4c10d0b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:37.240 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:37.236+0000 7fce5302c700 1 -- 172.21.15.190:0/2763245915 >> 172.21.15.190:0/2763245915 conn(0x7fce4c0f81c0 msgr2=0x7fce4c103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:37.240 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:37.236+0000 7fce5302c700 1 -- 172.21.15.190:0/2763245915 shutdown_connections 2024-05-06T12:06:37.240 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:37.236+0000 7fce5302c700 1 -- 172.21.15.190:0/2763245915 wait complete. 2024-05-06T12:06:37.240 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 1 2024-05-06T12:06:37.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:37 smithi092 ceph-mon[26954]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:06:37.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:37 smithi092 ceph-mon[26954]: from='client.? 172.21.15.190:0/2763245915' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-06T12:06:38.852 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-06T12:06:38.853 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-06T12:06:39.092 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-06T12:06:39.140 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-06T12:06:39.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:39 smithi092 ceph-mon[26954]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:06:40.574 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.571+0000 7f9652766700 1 -- 172.21.15.190:0/2693690506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f964c0fae40 msgr2=0x7f964c0fb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:40.574 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.571+0000 7f9652766700 1 --2- 172.21.15.190:0/2693690506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f964c0fae40 0x7f964c0fb250 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f963c0048c0 tx=0x7f963c009e30 comp rx=0 tx=0).stop 2024-05-06T12:06:40.574 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.571+0000 7f9652766700 1 -- 172.21.15.190:0/2693690506 shutdown_connections 2024-05-06T12:06:40.574 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.571+0000 7f9652766700 1 --2- 172.21.15.190:0/2693690506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f964c0fae40 0x7f964c0fb250 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:40.574 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.571+0000 7f9652766700 1 -- 172.21.15.190:0/2693690506 >> 172.21.15.190:0/2693690506 conn(0x7f964c0f6820 msgr2=0x7f964c0f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:40.574 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.571+0000 7f9652766700 1 -- 172.21.15.190:0/2693690506 shutdown_connections 2024-05-06T12:06:40.574 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.571+0000 7f9652766700 1 -- 172.21.15.190:0/2693690506 wait complete. 2024-05-06T12:06:40.576 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.572+0000 7f9652766700 1 Processor -- start 2024-05-06T12:06:40.576 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.572+0000 7f9652766700 1 -- start start 2024-05-06T12:06:40.576 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.572+0000 7f9652766700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f964c0fae40 0x7f964c10d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:40.576 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.572+0000 7f9652766700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964c10d800 con 0x7f964c0fae40 2024-05-06T12:06:40.576 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.572+0000 7f964bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f964c0fae40 0x7f964c10d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:40.576 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.572+0000 7f964bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f964c0fae40 0x7f964c10d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.190:60926/0 (socket says 172.21.15.190:60926) 2024-05-06T12:06:40.576 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.572+0000 7f964bfff700 1 -- 172.21.15.190:0/997653530 learned_addr learned my addr 172.21.15.190:0/997653530 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-06T12:06:40.576 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.573+0000 7f964bfff700 1 -- 172.21.15.190:0/997653530 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f963c005040 con 0x7f964c0fae40 2024-05-06T12:06:40.576 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.573+0000 7f964bfff700 1 --2- 172.21.15.190:0/997653530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f964c0fae40 0x7f964c10d2c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f963c0096b0 tx=0x7f963c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:40.576 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.573+0000 7f9648ff9700 1 -- 172.21.15.190:0/997653530 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f963c00bed0 con 0x7f964c0fae40 2024-05-06T12:06:40.576 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.573+0000 7f9652766700 1 -- 172.21.15.190:0/997653530 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f964c10da00 con 0x7f964c0fae40 2024-05-06T12:06:40.576 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.573+0000 7f9648ff9700 1 -- 172.21.15.190:0/997653530 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f963c00b5c0 con 0x7f964c0fae40 2024-05-06T12:06:40.576 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.573+0000 7f9652766700 1 -- 172.21.15.190:0/997653530 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f964c10e570 con 0x7f964c0fae40 2024-05-06T12:06:40.577 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.573+0000 7f9648ff9700 1 -- 172.21.15.190:0/997653530 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f963c025560 con 0x7f964c0fae40 2024-05-06T12:06:40.577 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.574+0000 7f9648ff9700 1 -- 172.21.15.190:0/997653530 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f963c004d30 con 0x7f964c0fae40 2024-05-06T12:06:40.578 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.574+0000 7f9652766700 1 -- 172.21.15.190:0/997653530 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f964c103170 con 0x7f964c0fae40 2024-05-06T12:06:40.578 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.574+0000 7f9648ff9700 1 --2- 172.21.15.190:0/997653530 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f9634036880 0x7f9634038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:40.578 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.574+0000 7f9648ff9700 1 -- 172.21.15.190:0/997653530 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f963c049af0 con 0x7f964c0fae40 2024-05-06T12:06:40.578 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.574+0000 7f964b7fe700 1 --2- 172.21.15.190:0/997653530 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f9634036880 0x7f9634038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:40.578 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.575+0000 7f964b7fe700 1 --2- 172.21.15.190:0/997653530 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f9634036880 0x7f9634038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9640009990 tx=0x7f9640006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:40.581 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.579+0000 7f9648ff9700 1 -- 172.21.15.190:0/997653530 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f963c011940 con 0x7f964c0fae40 2024-05-06T12:06:40.846 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.844+0000 7f9652766700 1 -- 172.21.15.190:0/997653530 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f964c0008d0 con 0x7f964c0fae40 2024-05-06T12:06:40.847 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.844+0000 7f9648ff9700 1 -- 172.21.15.190:0/997653530 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f963c011370 con 0x7f964c0fae40 2024-05-06T12:06:40.847 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:06:40.847 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":1,"fsid":"aee1d912-0ba0-11ef-bc97-c7b262605968","modified":"2024-05-06T12:04:48.255830Z","created":"2024-05-06T12:04:48.255830Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi092","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-06T12:06:40.850 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.847+0000 7f9652766700 1 -- 172.21.15.190:0/997653530 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f9634036880 msgr2=0x7f9634038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:40.850 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.847+0000 7f9652766700 1 --2- 172.21.15.190:0/997653530 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f9634036880 0x7f9634038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9640009990 tx=0x7f9640006e40 comp rx=0 tx=0).stop 2024-05-06T12:06:40.850 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.847+0000 7f9652766700 1 -- 172.21.15.190:0/997653530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f964c0fae40 msgr2=0x7f964c10d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:40.850 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.847+0000 7f9652766700 1 --2- 172.21.15.190:0/997653530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f964c0fae40 0x7f964c10d2c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f963c0096b0 tx=0x7f963c009e30 comp rx=0 tx=0).stop 2024-05-06T12:06:40.850 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.847+0000 7f9652766700 1 -- 172.21.15.190:0/997653530 shutdown_connections 2024-05-06T12:06:40.850 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.847+0000 7f9652766700 1 --2- 172.21.15.190:0/997653530 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f9634036880 0x7f9634038d30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:40.850 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.847+0000 7f9652766700 1 --2- 172.21.15.190:0/997653530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f964c0fae40 0x7f964c10d2c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:40.850 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.847+0000 7f9652766700 1 -- 172.21.15.190:0/997653530 >> 172.21.15.190:0/997653530 conn(0x7f964c0f6820 msgr2=0x7f964c103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:40.850 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.847+0000 7f9652766700 1 -- 172.21.15.190:0/997653530 shutdown_connections 2024-05-06T12:06:40.851 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:40.847+0000 7f9652766700 1 -- 172.21.15.190:0/997653530 wait complete. 2024-05-06T12:06:40.851 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 1 2024-05-06T12:06:41.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:41 smithi092 ceph-mon[26954]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:06:41.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:41 smithi092 ceph-mon[26954]: from='client.? 172.21.15.190:0/997653530' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-06T12:06:42.474 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-06T12:06:42.494 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-06T12:06:42.742 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-06T12:06:42.789 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-06T12:06:43.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:43 smithi092 ceph-mon[26954]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:06:44.219 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.216+0000 7f3cad2bd700 1 -- 172.21.15.190:0/1239230733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ca80fca30 msgr2=0x7f3ca80fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:44.219 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.216+0000 7f3cad2bd700 1 --2- 172.21.15.190:0/1239230733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ca80fca30 0x7f3ca80fce40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3c980048c0 tx=0x7f3c98009e30 comp rx=0 tx=0).stop 2024-05-06T12:06:44.219 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.216+0000 7f3cad2bd700 1 -- 172.21.15.190:0/1239230733 shutdown_connections 2024-05-06T12:06:44.219 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.216+0000 7f3cad2bd700 1 --2- 172.21.15.190:0/1239230733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ca80fca30 0x7f3ca80fce40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:44.220 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.216+0000 7f3cad2bd700 1 -- 172.21.15.190:0/1239230733 >> 172.21.15.190:0/1239230733 conn(0x7f3ca80f8170 msgr2=0x7f3ca80fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:44.220 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.216+0000 7f3cad2bd700 1 -- 172.21.15.190:0/1239230733 shutdown_connections 2024-05-06T12:06:44.220 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.216+0000 7f3cad2bd700 1 -- 172.21.15.190:0/1239230733 wait complete. 2024-05-06T12:06:44.220 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.216+0000 7f3cad2bd700 1 Processor -- start 2024-05-06T12:06:44.220 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.216+0000 7f3cad2bd700 1 -- start start 2024-05-06T12:06:44.220 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.217+0000 7f3cad2bd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ca80fca30 0x7f3ca810d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:44.220 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.217+0000 7f3cad2bd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca810d5c0 con 0x7f3ca80fca30 2024-05-06T12:06:44.220 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.217+0000 7f3ca6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ca80fca30 0x7f3ca810d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:44.220 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.217+0000 7f3ca6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ca80fca30 0x7f3ca810d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.190:60950/0 (socket says 172.21.15.190:60950) 2024-05-06T12:06:44.220 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.217+0000 7f3ca6ffd700 1 -- 172.21.15.190:0/1656228620 learned_addr learned my addr 172.21.15.190:0/1656228620 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-06T12:06:44.221 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.217+0000 7f3ca6ffd700 1 -- 172.21.15.190:0/1656228620 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c98005040 con 0x7f3ca80fca30 2024-05-06T12:06:44.222 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.217+0000 7f3ca6ffd700 1 --2- 172.21.15.190:0/1656228620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ca80fca30 0x7f3ca810d080 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3c980096b0 tx=0x7f3c98009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:44.222 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.218+0000 7f3c8f7fe700 1 -- 172.21.15.190:0/1656228620 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3c9800bed0 con 0x7f3ca80fca30 2024-05-06T12:06:44.222 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.218+0000 7f3cad2bd700 1 -- 172.21.15.190:0/1656228620 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ca810d7c0 con 0x7f3ca80fca30 2024-05-06T12:06:44.222 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.218+0000 7f3cad2bd700 1 -- 172.21.15.190:0/1656228620 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca810e330 con 0x7f3ca80fca30 2024-05-06T12:06:44.222 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.218+0000 7f3c8f7fe700 1 -- 172.21.15.190:0/1656228620 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c9800b5c0 con 0x7f3ca80fca30 2024-05-06T12:06:44.223 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.218+0000 7f3c8f7fe700 1 -- 172.21.15.190:0/1656228620 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3c98025400 con 0x7f3ca80fca30 2024-05-06T12:06:44.223 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.219+0000 7f3c8f7fe700 1 -- 172.21.15.190:0/1656228620 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f3c98025560 con 0x7f3ca80fca30 2024-05-06T12:06:44.223 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.219+0000 7f3c8f7fe700 1 --2- 172.21.15.190:0/1656228620 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f3c900368d0 0x7f3c90038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:44.223 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.219+0000 7f3cad2bd700 1 -- 172.21.15.190:0/1656228620 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c940052e0 con 0x7f3ca80fca30 2024-05-06T12:06:44.223 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.219+0000 7f3c8f7fe700 1 -- 172.21.15.190:0/1656228620 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f3c98049a50 con 0x7f3ca80fca30 2024-05-06T12:06:44.223 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.219+0000 7f3ca67fc700 1 --2- 172.21.15.190:0/1656228620 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f3c900368d0 0x7f3c90038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:44.223 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.220+0000 7f3ca67fc700 1 --2- 172.21.15.190:0/1656228620 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f3c900368d0 0x7f3c90038d80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c009990 tx=0x7f3c9c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:44.226 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.224+0000 7f3c8f7fe700 1 -- 172.21.15.190:0/1656228620 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3c9801a2f0 con 0x7f3ca80fca30 2024-05-06T12:06:44.488 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.486+0000 7f3cad2bd700 1 -- 172.21.15.190:0/1656228620 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3c94005590 con 0x7f3ca80fca30 2024-05-06T12:06:44.489 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.486+0000 7f3c8f7fe700 1 -- 172.21.15.190:0/1656228620 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f3c98024b00 con 0x7f3ca80fca30 2024-05-06T12:06:44.489 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:06:44.489 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":1,"fsid":"aee1d912-0ba0-11ef-bc97-c7b262605968","modified":"2024-05-06T12:04:48.255830Z","created":"2024-05-06T12:04:48.255830Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi092","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-06T12:06:44.491 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.488+0000 7f3cad2bd700 1 -- 172.21.15.190:0/1656228620 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f3c900368d0 msgr2=0x7f3c90038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:44.491 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.488+0000 7f3cad2bd700 1 --2- 172.21.15.190:0/1656228620 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f3c900368d0 0x7f3c90038d80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c009990 tx=0x7f3c9c006e40 comp rx=0 tx=0).stop 2024-05-06T12:06:44.491 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.489+0000 7f3cad2bd700 1 -- 172.21.15.190:0/1656228620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ca80fca30 msgr2=0x7f3ca810d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:44.491 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.489+0000 7f3cad2bd700 1 --2- 172.21.15.190:0/1656228620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ca80fca30 0x7f3ca810d080 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3c980096b0 tx=0x7f3c98009e30 comp rx=0 tx=0).stop 2024-05-06T12:06:44.491 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.489+0000 7f3cad2bd700 1 -- 172.21.15.190:0/1656228620 shutdown_connections 2024-05-06T12:06:44.491 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.489+0000 7f3cad2bd700 1 --2- 172.21.15.190:0/1656228620 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f3c900368d0 0x7f3c90038d80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:44.491 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.489+0000 7f3cad2bd700 1 --2- 172.21.15.190:0/1656228620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ca80fca30 0x7f3ca810d080 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:44.491 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.489+0000 7f3cad2bd700 1 -- 172.21.15.190:0/1656228620 >> 172.21.15.190:0/1656228620 conn(0x7f3ca80f8170 msgr2=0x7f3ca8103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:44.492 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.489+0000 7f3cad2bd700 1 -- 172.21.15.190:0/1656228620 shutdown_connections 2024-05-06T12:06:44.492 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:44.489+0000 7f3cad2bd700 1 -- 172.21.15.190:0/1656228620 wait complete. 2024-05-06T12:06:44.492 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 1 2024-05-06T12:06:44.904 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:44 smithi092 ceph-mon[26954]: from='client.? 172.21.15.190:0/1656228620' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-06T12:06:46.068 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-06T12:06:46.068 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-06T12:06:46.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:46 smithi092 ceph-mon[26954]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:06:46.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:46 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:46.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:46 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:46.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:46 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:46.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:46 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:46.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:46 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:46.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:46 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:46.316 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-06T12:06:46.363 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-06T12:06:47.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:47 smithi092 ceph-mon[26954]: Deploying daemon prometheus.smithi092 on smithi092 2024-05-06T12:06:47.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:47 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:47.658 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.654+0000 7f24e0193700 1 -- 172.21.15.190:0/100997851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24d80fca30 msgr2=0x7f24d80fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:47.659 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.654+0000 7f24e0193700 1 --2- 172.21.15.190:0/100997851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24d80fca30 0x7f24d80fce40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f24c80048c0 tx=0x7f24c8009e30 comp rx=0 tx=0).stop 2024-05-06T12:06:47.659 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.654+0000 7f24e0193700 1 -- 172.21.15.190:0/100997851 shutdown_connections 2024-05-06T12:06:47.659 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.654+0000 7f24e0193700 1 --2- 172.21.15.190:0/100997851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24d80fca30 0x7f24d80fce40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:47.659 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.654+0000 7f24e0193700 1 -- 172.21.15.190:0/100997851 >> 172.21.15.190:0/100997851 conn(0x7f24d80f8170 msgr2=0x7f24d80fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:47.659 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.655+0000 7f24e0193700 1 -- 172.21.15.190:0/100997851 shutdown_connections 2024-05-06T12:06:47.659 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.655+0000 7f24e0193700 1 -- 172.21.15.190:0/100997851 wait complete. 2024-05-06T12:06:47.659 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.655+0000 7f24e0193700 1 Processor -- start 2024-05-06T12:06:47.659 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.655+0000 7f24e0193700 1 -- start start 2024-05-06T12:06:47.659 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.655+0000 7f24e0193700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24d80fca30 0x7f24d810d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:47.659 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.655+0000 7f24e0193700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d810d5f0 con 0x7f24d80fca30 2024-05-06T12:06:47.659 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.656+0000 7f24ddf2f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24d80fca30 0x7f24d810d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:47.659 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.656+0000 7f24ddf2f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24d80fca30 0x7f24d810d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.190:60302/0 (socket says 172.21.15.190:60302) 2024-05-06T12:06:47.659 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.656+0000 7f24ddf2f700 1 -- 172.21.15.190:0/407609021 learned_addr learned my addr 172.21.15.190:0/407609021 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-06T12:06:47.660 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.656+0000 7f24ddf2f700 1 -- 172.21.15.190:0/407609021 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c8005040 con 0x7f24d80fca30 2024-05-06T12:06:47.660 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.656+0000 7f24ddf2f700 1 --2- 172.21.15.190:0/407609021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24d80fca30 0x7f24d810d0b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f24c80096b0 tx=0x7f24c8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:47.660 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.656+0000 7f24ce7fc700 1 -- 172.21.15.190:0/407609021 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f24c800bed0 con 0x7f24d80fca30 2024-05-06T12:06:47.660 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.656+0000 7f24ce7fc700 1 -- 172.21.15.190:0/407609021 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f24c800b5c0 con 0x7f24d80fca30 2024-05-06T12:06:47.660 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.656+0000 7f24e0193700 1 -- 172.21.15.190:0/407609021 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d810d7f0 con 0x7f24d80fca30 2024-05-06T12:06:47.660 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.656+0000 7f24ce7fc700 1 -- 172.21.15.190:0/407609021 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f24c801c5b0 con 0x7f24d80fca30 2024-05-06T12:06:47.660 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.656+0000 7f24e0193700 1 -- 172.21.15.190:0/407609021 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d810e360 con 0x7f24d80fca30 2024-05-06T12:06:47.661 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.657+0000 7f24ce7fc700 1 -- 172.21.15.190:0/407609021 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f24c8013420 con 0x7f24d80fca30 2024-05-06T12:06:47.661 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.657+0000 7f24e0193700 1 -- 172.21.15.190:0/407609021 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d8102f40 con 0x7f24d80fca30 2024-05-06T12:06:47.661 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.657+0000 7f24ce7fc700 1 --2- 172.21.15.190:0/407609021 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f24c4036880 0x7f24c4038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:47.661 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.657+0000 7f24ce7fc700 1 -- 172.21.15.190:0/407609021 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f24c8049ac0 con 0x7f24d80fca30 2024-05-06T12:06:47.661 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.657+0000 7f24dd72e700 1 --2- 172.21.15.190:0/407609021 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f24c4036880 0x7f24c4038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:47.661 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.658+0000 7f24dd72e700 1 --2- 172.21.15.190:0/407609021 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f24c4036880 0x7f24c4038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f24d4009990 tx=0x7f24d4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:47.664 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.662+0000 7f24ce7fc700 1 -- 172.21.15.190:0/407609021 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f24c8004d30 con 0x7f24d80fca30 2024-05-06T12:06:47.925 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.923+0000 7f24e0193700 1 -- 172.21.15.190:0/407609021 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f24d810ed70 con 0x7f24d80fca30 2024-05-06T12:06:47.925 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.923+0000 7f24ce7fc700 1 -- 172.21.15.190:0/407609021 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f24c8011370 con 0x7f24d80fca30 2024-05-06T12:06:47.926 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:06:47.926 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":1,"fsid":"aee1d912-0ba0-11ef-bc97-c7b262605968","modified":"2024-05-06T12:04:48.255830Z","created":"2024-05-06T12:04:48.255830Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi092","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-06T12:06:47.928 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.925+0000 7f24e0193700 1 -- 172.21.15.190:0/407609021 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f24c4036880 msgr2=0x7f24c4038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:47.928 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.925+0000 7f24e0193700 1 --2- 172.21.15.190:0/407609021 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f24c4036880 0x7f24c4038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f24d4009990 tx=0x7f24d4006e40 comp rx=0 tx=0).stop 2024-05-06T12:06:47.928 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.925+0000 7f24e0193700 1 -- 172.21.15.190:0/407609021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24d80fca30 msgr2=0x7f24d810d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:47.928 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.925+0000 7f24e0193700 1 --2- 172.21.15.190:0/407609021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24d80fca30 0x7f24d810d0b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f24c80096b0 tx=0x7f24c8009e30 comp rx=0 tx=0).stop 2024-05-06T12:06:47.928 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.925+0000 7f24e0193700 1 -- 172.21.15.190:0/407609021 shutdown_connections 2024-05-06T12:06:47.928 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.925+0000 7f24e0193700 1 --2- 172.21.15.190:0/407609021 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f24c4036880 0x7f24c4038d30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:47.928 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.925+0000 7f24e0193700 1 --2- 172.21.15.190:0/407609021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24d80fca30 0x7f24d810d0b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:47.928 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.925+0000 7f24e0193700 1 -- 172.21.15.190:0/407609021 >> 172.21.15.190:0/407609021 conn(0x7f24d80f8170 msgr2=0x7f24d8103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:47.928 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.925+0000 7f24e0193700 1 -- 172.21.15.190:0/407609021 shutdown_connections 2024-05-06T12:06:47.928 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:47.925+0000 7f24e0193700 1 -- 172.21.15.190:0/407609021 wait complete. 2024-05-06T12:06:47.929 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 1 2024-05-06T12:06:48.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:48 smithi092 ceph-mon[26954]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:06:48.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:48 smithi092 ceph-mon[26954]: from='client.? 172.21.15.190:0/407609021' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-06T12:06:49.483 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-06T12:06:49.483 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-06T12:06:49.732 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-06T12:06:49.780 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-06T12:06:50.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:50 smithi092 ceph-mon[26954]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:06:51.129 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.125+0000 7fc6a4860700 1 -- 172.21.15.190:0/4019610379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a00fbdb0 msgr2=0x7fc6a00fc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:51.129 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.125+0000 7fc6a4860700 1 --2- 172.21.15.190:0/4019610379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a00fbdb0 0x7fc6a00fc1c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc6900048c0 tx=0x7fc690009e30 comp rx=0 tx=0).stop 2024-05-06T12:06:51.129 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.125+0000 7fc6a4860700 1 -- 172.21.15.190:0/4019610379 shutdown_connections 2024-05-06T12:06:51.130 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.125+0000 7fc6a4860700 1 --2- 172.21.15.190:0/4019610379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a00fbdb0 0x7fc6a00fc1c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:51.130 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.125+0000 7fc6a4860700 1 -- 172.21.15.190:0/4019610379 >> 172.21.15.190:0/4019610379 conn(0x7fc6a00f6820 msgr2=0x7fc6a00f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:51.130 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.125+0000 7fc6a4860700 1 -- 172.21.15.190:0/4019610379 shutdown_connections 2024-05-06T12:06:51.130 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.125+0000 7fc6a4860700 1 -- 172.21.15.190:0/4019610379 wait complete. 2024-05-06T12:06:51.130 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.125+0000 7fc6a4860700 1 Processor -- start 2024-05-06T12:06:51.130 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.126+0000 7fc6a4860700 1 -- start start 2024-05-06T12:06:51.130 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.126+0000 7fc6a4860700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a00fbdb0 0x7fc6a0113680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:51.130 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.126+0000 7fc6a4860700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a0113bc0 con 0x7fc6a00fbdb0 2024-05-06T12:06:51.130 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.126+0000 7fc69e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a00fbdb0 0x7fc6a0113680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:51.130 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.126+0000 7fc69e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a00fbdb0 0x7fc6a0113680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.190:60324/0 (socket says 172.21.15.190:60324) 2024-05-06T12:06:51.130 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.126+0000 7fc69e59c700 1 -- 172.21.15.190:0/4175395659 learned_addr learned my addr 172.21.15.190:0/4175395659 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-06T12:06:51.130 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.126+0000 7fc69e59c700 1 -- 172.21.15.190:0/4175395659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc690005040 con 0x7fc6a00fbdb0 2024-05-06T12:06:51.131 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.126+0000 7fc69e59c700 1 --2- 172.21.15.190:0/4175395659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a00fbdb0 0x7fc6a0113680 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc690009680 tx=0x7fc690009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:51.131 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.126+0000 7fc68effd700 1 -- 172.21.15.190:0/4175395659 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc69000b5c0 con 0x7fc6a00fbdb0 2024-05-06T12:06:51.131 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.127+0000 7fc68effd700 1 -- 172.21.15.190:0/4175395659 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc69000be70 con 0x7fc6a00fbdb0 2024-05-06T12:06:51.131 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.127+0000 7fc68effd700 1 -- 172.21.15.190:0/4175395659 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6900135a0 con 0x7fc6a00fbdb0 2024-05-06T12:06:51.131 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.127+0000 7fc6a4860700 1 -- 172.21.15.190:0/4175395659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6a0113dc0 con 0x7fc6a00fbdb0 2024-05-06T12:06:51.131 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.127+0000 7fc6a4860700 1 -- 172.21.15.190:0/4175395659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6a0114930 con 0x7fc6a00fbdb0 2024-05-06T12:06:51.132 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.127+0000 7fc68effd700 1 -- 172.21.15.190:0/4175395659 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7fc69001c430 con 0x7fc6a00fbdb0 2024-05-06T12:06:51.132 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.127+0000 7fc6a4860700 1 -- 172.21.15.190:0/4175395659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6a0100d20 con 0x7fc6a00fbdb0 2024-05-06T12:06:51.132 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.128+0000 7fc68effd700 1 --2- 172.21.15.190:0/4175395659 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7fc688036880 0x7fc688038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:51.132 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.128+0000 7fc68effd700 1 -- 172.21.15.190:0/4175395659 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fc690049430 con 0x7fc6a00fbdb0 2024-05-06T12:06:51.132 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.128+0000 7fc69dd9b700 1 --2- 172.21.15.190:0/4175395659 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7fc688036880 0x7fc688038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:51.132 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.128+0000 7fc69dd9b700 1 --2- 172.21.15.190:0/4175395659 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7fc688036880 0x7fc688038d30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc694009990 tx=0x7fc694006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:51.135 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.132+0000 7fc68effd700 1 -- 172.21.15.190:0/4175395659 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc6900117e0 con 0x7fc6a00fbdb0 2024-05-06T12:06:51.395 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.393+0000 7fc6a4860700 1 -- 172.21.15.190:0/4175395659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc6a0115340 con 0x7fc6a00fbdb0 2024-05-06T12:06:51.396 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.393+0000 7fc68effd700 1 -- 172.21.15.190:0/4175395659 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fc690021070 con 0x7fc6a00fbdb0 2024-05-06T12:06:51.396 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:06:51.396 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":1,"fsid":"aee1d912-0ba0-11ef-bc97-c7b262605968","modified":"2024-05-06T12:04:48.255830Z","created":"2024-05-06T12:04:48.255830Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi092","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-06T12:06:51.398 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.395+0000 7fc6a4860700 1 -- 172.21.15.190:0/4175395659 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7fc688036880 msgr2=0x7fc688038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:51.398 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.395+0000 7fc6a4860700 1 --2- 172.21.15.190:0/4175395659 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7fc688036880 0x7fc688038d30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc694009990 tx=0x7fc694006e40 comp rx=0 tx=0).stop 2024-05-06T12:06:51.398 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.395+0000 7fc6a4860700 1 -- 172.21.15.190:0/4175395659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a00fbdb0 msgr2=0x7fc6a0113680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:51.398 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.395+0000 7fc6a4860700 1 --2- 172.21.15.190:0/4175395659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a00fbdb0 0x7fc6a0113680 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc690009680 tx=0x7fc690009e30 comp rx=0 tx=0).stop 2024-05-06T12:06:51.398 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.395+0000 7fc6a4860700 1 -- 172.21.15.190:0/4175395659 shutdown_connections 2024-05-06T12:06:51.398 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.395+0000 7fc6a4860700 1 --2- 172.21.15.190:0/4175395659 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7fc688036880 0x7fc688038d30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:51.398 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.395+0000 7fc6a4860700 1 --2- 172.21.15.190:0/4175395659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a00fbdb0 0x7fc6a0113680 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:51.398 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.395+0000 7fc6a4860700 1 -- 172.21.15.190:0/4175395659 >> 172.21.15.190:0/4175395659 conn(0x7fc6a00f6820 msgr2=0x7fc6a010aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:51.399 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.395+0000 7fc6a4860700 1 -- 172.21.15.190:0/4175395659 shutdown_connections 2024-05-06T12:06:51.399 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:51.396+0000 7fc6a4860700 1 -- 172.21.15.190:0/4175395659 wait complete. 2024-05-06T12:06:51.399 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 1 2024-05-06T12:06:52.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:52 smithi092 ceph-mon[26954]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:06:52.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:52 smithi092 ceph-mon[26954]: from='client.? 172.21.15.190:0/4175395659' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-06T12:06:53.105 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-06T12:06:53.105 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-06T12:06:53.355 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-06T12:06:53.403 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-06T12:06:54.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:54 smithi092 ceph-mon[26954]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:06:54.830 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:54.824+0000 7f28f045c700 1 -- 172.21.15.190:0/21585778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28e80fca20 msgr2=0x7f28e80fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:54.830 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:54.824+0000 7f28f045c700 1 --2- 172.21.15.190:0/21585778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28e80fca20 0x7f28e80fce30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f28d80048c0 tx=0x7f28d8009e30 comp rx=0 tx=0).stop 2024-05-06T12:06:54.831 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:54.824+0000 7f28f045c700 1 -- 172.21.15.190:0/21585778 shutdown_connections 2024-05-06T12:06:54.831 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:54.824+0000 7f28f045c700 1 --2- 172.21.15.190:0/21585778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28e80fca20 0x7f28e80fce30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:54.831 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:54.824+0000 7f28f045c700 1 -- 172.21.15.190:0/21585778 >> 172.21.15.190:0/21585778 conn(0x7f28e80f81a0 msgr2=0x7f28e80fa5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:54.831 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:54.824+0000 7f28f045c700 1 -- 172.21.15.190:0/21585778 shutdown_connections 2024-05-06T12:06:54.831 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:54.824+0000 7f28f045c700 1 -- 172.21.15.190:0/21585778 wait complete. 2024-05-06T12:06:54.831 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:54.824+0000 7f28f045c700 1 Processor -- start 2024-05-06T12:06:54.831 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:54.824+0000 7f28f045c700 1 -- start start 2024-05-06T12:06:54.832 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:54.825+0000 7f28f045c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28e80fca20 0x7f28e810d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:54.832 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:54.825+0000 7f28f045c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e810d5e0 con 0x7f28e80fca20 2024-05-06T12:06:54.832 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:54.825+0000 7f28ee1f8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28e80fca20 0x7f28e810d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:54.832 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:54.825+0000 7f28ee1f8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28e80fca20 0x7f28e810d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.190:60326/0 (socket says 172.21.15.190:60326) 2024-05-06T12:06:54.832 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:54.825+0000 7f28ee1f8700 1 -- 172.21.15.190:0/2018790453 learned_addr learned my addr 172.21.15.190:0/2018790453 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-06T12:06:54.832 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:54.825+0000 7f28ee1f8700 1 -- 172.21.15.190:0/2018790453 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d8005040 con 0x7f28e80fca20 2024-05-06T12:06:54.832 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:54.825+0000 7f28ee1f8700 1 --2- 172.21.15.190:0/2018790453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28e80fca20 0x7f28e810d0a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f28d8009680 tx=0x7f28d8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:54.833 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:54.825+0000 7f28deffd700 1 -- 172.21.15.190:0/2018790453 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f28d800b5c0 con 0x7f28e80fca20 2024-05-06T12:06:54.833 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:54.826+0000 7f28f045c700 1 -- 172.21.15.190:0/2018790453 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28e810d7e0 con 0x7f28e80fca20 2024-05-06T12:06:54.833 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:54.826+0000 7f28deffd700 1 -- 172.21.15.190:0/2018790453 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28d800be70 con 0x7f28e80fca20 2024-05-06T12:06:54.833 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:54.826+0000 7f28deffd700 1 -- 172.21.15.190:0/2018790453 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f28d80135a0 con 0x7f28e80fca20 2024-05-06T12:06:54.833 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:54.826+0000 7f28f045c700 1 -- 172.21.15.190:0/2018790453 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e810e350 con 0x7f28e80fca20 2024-05-06T12:06:54.833 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:54.827+0000 7f28f045c700 1 -- 172.21.15.190:0/2018790453 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28e8102f30 con 0x7f28e80fca20 2024-05-06T12:06:54.833 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:54.827+0000 7f28deffd700 1 -- 172.21.15.190:0/2018790453 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f28d801ca90 con 0x7f28e80fca20 2024-05-06T12:06:54.833 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:54.827+0000 7f28deffd700 1 --2- 172.21.15.190:0/2018790453 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f28d4036880 0x7f28d4038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:54.834 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:54.827+0000 7f28deffd700 1 -- 172.21.15.190:0/2018790453 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f28d80493a0 con 0x7f28e80fca20 2024-05-06T12:06:54.834 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:54.827+0000 7f28ed9f7700 1 --2- 172.21.15.190:0/2018790453 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f28d4036880 0x7f28d4038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:54.834 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:54.827+0000 7f28ed9f7700 1 --2- 172.21.15.190:0/2018790453 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f28d4036880 0x7f28d4038d30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f28e4009990 tx=0x7f28e4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:54.834 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:54.832+0000 7f28deffd700 1 -- 172.21.15.190:0/2018790453 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f28d8011960 con 0x7f28e80fca20 2024-05-06T12:06:55.094 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:55.092+0000 7f28f045c700 1 -- 172.21.15.190:0/2018790453 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f28e810ed60 con 0x7f28e80fca20 2024-05-06T12:06:55.095 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:55.092+0000 7f28deffd700 1 -- 172.21.15.190:0/2018790453 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f28d8021070 con 0x7f28e80fca20 2024-05-06T12:06:55.095 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:06:55.095 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":1,"fsid":"aee1d912-0ba0-11ef-bc97-c7b262605968","modified":"2024-05-06T12:04:48.255830Z","created":"2024-05-06T12:04:48.255830Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi092","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-06T12:06:55.097 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:55.095+0000 7f28f045c700 1 -- 172.21.15.190:0/2018790453 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f28d4036880 msgr2=0x7f28d4038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:55.097 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:55.095+0000 7f28f045c700 1 --2- 172.21.15.190:0/2018790453 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f28d4036880 0x7f28d4038d30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f28e4009990 tx=0x7f28e4006e40 comp rx=0 tx=0).stop 2024-05-06T12:06:55.097 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:55.095+0000 7f28f045c700 1 -- 172.21.15.190:0/2018790453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28e80fca20 msgr2=0x7f28e810d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:55.098 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:55.095+0000 7f28f045c700 1 --2- 172.21.15.190:0/2018790453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28e80fca20 0x7f28e810d0a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f28d8009680 tx=0x7f28d8009e30 comp rx=0 tx=0).stop 2024-05-06T12:06:55.098 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:55.095+0000 7f28f045c700 1 -- 172.21.15.190:0/2018790453 shutdown_connections 2024-05-06T12:06:55.098 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:55.095+0000 7f28f045c700 1 --2- 172.21.15.190:0/2018790453 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f28d4036880 0x7f28d4038d30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:55.098 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:55.095+0000 7f28f045c700 1 --2- 172.21.15.190:0/2018790453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28e80fca20 0x7f28e810d0a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:55.098 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:55.095+0000 7f28f045c700 1 -- 172.21.15.190:0/2018790453 >> 172.21.15.190:0/2018790453 conn(0x7f28e80f81a0 msgr2=0x7f28e8103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:55.098 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:55.095+0000 7f28f045c700 1 -- 172.21.15.190:0/2018790453 shutdown_connections 2024-05-06T12:06:55.098 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:55.095+0000 7f28f045c700 1 -- 172.21.15.190:0/2018790453 wait complete. 2024-05-06T12:06:55.098 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 1 2024-05-06T12:06:55.248 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:55 smithi092 ceph-mon[26954]: from='client.? 172.21.15.190:0/2018790453' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-06T12:06:56.198 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:56 smithi092 ceph-mon[26954]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:06:56.665 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-06T12:06:56.666 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-06T12:06:56.912 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-06T12:06:56.960 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-06T12:06:57.698 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:57 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:57.698 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:57 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:57.698 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:57 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:57.698 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:57 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-05-06T12:06:57.699 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:57 smithi092 ceph-mon[26954]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:06:57.699 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:57 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:06:58.470 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.465+0000 7f901bd6f700 1 -- 172.21.15.190:0/3676899868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90140fb930 msgr2=0x7f90140fbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:58.471 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.465+0000 7f901bd6f700 1 --2- 172.21.15.190:0/3676899868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90140fb930 0x7f90140fbd40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f90040048c0 tx=0x7f9004009e30 comp rx=0 tx=0).stop 2024-05-06T12:06:58.471 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.465+0000 7f901bd6f700 1 -- 172.21.15.190:0/3676899868 shutdown_connections 2024-05-06T12:06:58.471 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.465+0000 7f901bd6f700 1 --2- 172.21.15.190:0/3676899868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90140fb930 0x7f90140fbd40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:58.471 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.465+0000 7f901bd6f700 1 -- 172.21.15.190:0/3676899868 >> 172.21.15.190:0/3676899868 conn(0x7f90140f7070 msgr2=0x7f90140f94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:58.471 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.465+0000 7f901bd6f700 1 -- 172.21.15.190:0/3676899868 shutdown_connections 2024-05-06T12:06:58.471 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.465+0000 7f901bd6f700 1 -- 172.21.15.190:0/3676899868 wait complete. 2024-05-06T12:06:58.471 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.466+0000 7f901bd6f700 1 Processor -- start 2024-05-06T12:06:58.471 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.466+0000 7f901bd6f700 1 -- start start 2024-05-06T12:06:58.471 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.466+0000 7f901bd6f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90140fb930 0x7f9014066820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:58.471 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.466+0000 7f901bd6f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901410dbc0 con 0x7f90140fb930 2024-05-06T12:06:58.471 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.466+0000 7f9019b0b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90140fb930 0x7f9014066820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:06:58.471 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.466+0000 7f9019b0b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90140fb930 0x7f9014066820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.190:60342/0 (socket says 172.21.15.190:60342) 2024-05-06T12:06:58.471 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.466+0000 7f9019b0b700 1 -- 172.21.15.190:0/4211164400 learned_addr learned my addr 172.21.15.190:0/4211164400 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-06T12:06:58.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.467+0000 7f9019b0b700 1 -- 172.21.15.190:0/4211164400 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9004005040 con 0x7f90140fb930 2024-05-06T12:06:58.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.467+0000 7f9019b0b700 1 --2- 172.21.15.190:0/4211164400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90140fb930 0x7f9014066820 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f90040096b0 tx=0x7f9004009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:06:58.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.467+0000 7f900a7fc700 1 -- 172.21.15.190:0/4211164400 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f900400bed0 con 0x7f90140fb930 2024-05-06T12:06:58.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.467+0000 7f900a7fc700 1 -- 172.21.15.190:0/4211164400 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f900400b5c0 con 0x7f90140fb930 2024-05-06T12:06:58.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.467+0000 7f900a7fc700 1 -- 172.21.15.190:0/4211164400 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f900401c5b0 con 0x7f90140fb930 2024-05-06T12:06:58.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.467+0000 7f901bd6f700 1 -- 172.21.15.190:0/4211164400 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90141121d0 con 0x7f90140fb930 2024-05-06T12:06:58.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.467+0000 7f901bd6f700 1 -- 172.21.15.190:0/4211164400 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9014112600 con 0x7f90140fb930 2024-05-06T12:06:58.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.468+0000 7f900a7fc700 1 -- 172.21.15.190:0/4211164400 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) v1 ==== 43446+0+0 (secure 0 0 0) 0x7f9004013420 con 0x7f90140fb930 2024-05-06T12:06:58.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.468+0000 7f901bd6f700 1 -- 172.21.15.190:0/4211164400 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90141058f0 con 0x7f90140fb930 2024-05-06T12:06:58.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.468+0000 7f900a7fc700 1 --2- 172.21.15.190:0/4211164400 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f90000368d0 0x7f9000038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:06:58.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.468+0000 7f900a7fc700 1 -- 172.21.15.190:0/4211164400 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f9004049b30 con 0x7f90140fb930 2024-05-06T12:06:58.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.468+0000 7f901930a700 1 -- 172.21.15.190:0/4211164400 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f90000368d0 msgr2=0x7f9000038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/543287293 2024-05-06T12:06:58.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.468+0000 7f901930a700 1 --2- 172.21.15.190:0/4211164400 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f90000368d0 0x7f9000038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-06T12:06:58.475 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.473+0000 7f900a7fc700 1 -- 172.21.15.190:0/4211164400 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9004025400 con 0x7f90140fb930 2024-05-06T12:06:58.671 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.669+0000 7f901930a700 1 -- 172.21.15.190:0/4211164400 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f90000368d0 msgr2=0x7f9000038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/543287293 2024-05-06T12:06:58.671 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.669+0000 7f901930a700 1 --2- 172.21.15.190:0/4211164400 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f90000368d0 0x7f9000038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-06T12:06:58.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:58 smithi092 ceph-mon[26954]: from='mgr.14162 172.21.15.92:0/1165313948' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-05-06T12:06:58.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:58 smithi092 ceph-mon[26954]: mgrmap e14: smithi092.gyzeqb(active, since 60s) 2024-05-06T12:06:58.734 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.732+0000 7f901bd6f700 1 -- 172.21.15.190:0/4211164400 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f901405f5c0 con 0x7f90140fb930 2024-05-06T12:06:58.734 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.732+0000 7f900a7fc700 1 -- 172.21.15.190:0/4211164400 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f9004004d30 con 0x7f90140fb930 2024-05-06T12:06:58.735 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:06:58.735 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":1,"fsid":"aee1d912-0ba0-11ef-bc97-c7b262605968","modified":"2024-05-06T12:04:48.255830Z","created":"2024-05-06T12:04:48.255830Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi092","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-06T12:06:58.737 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.734+0000 7f901bd6f700 1 -- 172.21.15.190:0/4211164400 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f90000368d0 msgr2=0x7f9000038d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-06T12:06:58.737 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.734+0000 7f901bd6f700 1 --2- 172.21.15.190:0/4211164400 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f90000368d0 0x7f9000038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:58.737 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.734+0000 7f901bd6f700 1 -- 172.21.15.190:0/4211164400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90140fb930 msgr2=0x7f9014066820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:06:58.737 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.734+0000 7f901bd6f700 1 --2- 172.21.15.190:0/4211164400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90140fb930 0x7f9014066820 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f90040096b0 tx=0x7f9004009e30 comp rx=0 tx=0).stop 2024-05-06T12:06:58.737 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.734+0000 7f901bd6f700 1 -- 172.21.15.190:0/4211164400 shutdown_connections 2024-05-06T12:06:58.737 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.734+0000 7f901bd6f700 1 --2- 172.21.15.190:0/4211164400 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f90000368d0 0x7f9000038d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:58.737 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.734+0000 7f901bd6f700 1 --2- 172.21.15.190:0/4211164400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90140fb930 0x7f9014066820 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:06:58.737 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.734+0000 7f901bd6f700 1 -- 172.21.15.190:0/4211164400 >> 172.21.15.190:0/4211164400 conn(0x7f90140f7070 msgr2=0x7f9014102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:06:58.737 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.734+0000 7f901bd6f700 1 -- 172.21.15.190:0/4211164400 shutdown_connections 2024-05-06T12:06:58.738 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:06:58.734+0000 7f901bd6f700 1 -- 172.21.15.190:0/4211164400 wait complete. 2024-05-06T12:06:58.738 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 1 2024-05-06T12:06:59.647 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:06:59 smithi092 ceph-mon[26954]: from='client.? 172.21.15.190:0/4211164400' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-06T12:07:00.271 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-06T12:07:00.272 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-06T12:07:00.521 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-06T12:07:00.569 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-06T12:07:02.001 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:01.997+0000 7f3e744cc700 1 -- 172.21.15.190:0/271580558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e6c0fb930 msgr2=0x7f3e6c0fbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:07:02.002 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:01.997+0000 7f3e744cc700 1 --2- 172.21.15.190:0/271580558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e6c0fb930 0x7f3e6c0fbd40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3e680048c0 tx=0x7f3e68009e30 comp rx=0 tx=0).stop 2024-05-06T12:07:02.002 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:01.997+0000 7f3e744cc700 1 -- 172.21.15.190:0/271580558 shutdown_connections 2024-05-06T12:07:02.002 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:01.997+0000 7f3e744cc700 1 --2- 172.21.15.190:0/271580558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e6c0fb930 0x7f3e6c0fbd40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:07:02.002 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:01.997+0000 7f3e744cc700 1 -- 172.21.15.190:0/271580558 >> 172.21.15.190:0/271580558 conn(0x7f3e6c0f7070 msgr2=0x7f3e6c0f94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:07:02.002 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:01.997+0000 7f3e744cc700 1 -- 172.21.15.190:0/271580558 shutdown_connections 2024-05-06T12:07:02.002 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:01.997+0000 7f3e744cc700 1 -- 172.21.15.190:0/271580558 wait complete. 2024-05-06T12:07:02.002 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:01.998+0000 7f3e744cc700 1 Processor -- start 2024-05-06T12:07:02.002 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:01.998+0000 7f3e744cc700 1 -- start start 2024-05-06T12:07:02.002 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:01.998+0000 7f3e744cc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e6c0fb930 0x7f3e6c066880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:07:02.002 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:01.998+0000 7f3e744cc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c10fd70 con 0x7f3e6c0fb930 2024-05-06T12:07:02.002 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:01.998+0000 7f3e72268700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e6c0fb930 0x7f3e6c066880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:07:02.002 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:01.998+0000 7f3e72268700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e6c0fb930 0x7f3e6c066880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.190:60372/0 (socket says 172.21.15.190:60372) 2024-05-06T12:07:02.003 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:01.998+0000 7f3e72268700 1 -- 172.21.15.190:0/2687512014 learned_addr learned my addr 172.21.15.190:0/2687512014 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-06T12:07:02.003 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:01.998+0000 7f3e72268700 1 -- 172.21.15.190:0/2687512014 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e68005040 con 0x7f3e6c0fb930 2024-05-06T12:07:02.003 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:01.998+0000 7f3e72268700 1 --2- 172.21.15.190:0/2687512014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e6c0fb930 0x7f3e6c066880 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3e680096b0 tx=0x7f3e68009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:07:02.003 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:01.999+0000 7f3e62ffd700 1 -- 172.21.15.190:0/2687512014 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e6800bed0 con 0x7f3e6c0fb930 2024-05-06T12:07:02.003 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:01.999+0000 7f3e744cc700 1 -- 172.21.15.190:0/2687512014 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e6c10ff10 con 0x7f3e6c0fb930 2024-05-06T12:07:02.003 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:01.999+0000 7f3e62ffd700 1 -- 172.21.15.190:0/2687512014 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e6800b5c0 con 0x7f3e6c0fb930 2024-05-06T12:07:02.003 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:01.999+0000 7f3e62ffd700 1 -- 172.21.15.190:0/2687512014 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e6801c5b0 con 0x7f3e6c0fb930 2024-05-06T12:07:02.003 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:01.999+0000 7f3e744cc700 1 -- 172.21.15.190:0/2687512014 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e6c1103a0 con 0x7f3e6c0fb930 2024-05-06T12:07:02.004 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:01.999+0000 7f3e62ffd700 1 -- 172.21.15.190:0/2687512014 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) v1 ==== 43446+0+0 (secure 0 0 0) 0x7f3e68013420 con 0x7f3e6c0fb930 2024-05-06T12:07:02.004 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:01.999+0000 7f3e744cc700 1 -- 172.21.15.190:0/2687512014 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e6c1058f0 con 0x7f3e6c0fb930 2024-05-06T12:07:02.004 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:02.000+0000 7f3e62ffd700 1 --2- 172.21.15.190:0/2687512014 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f3e580368d0 0x7f3e58038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:07:02.004 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:02.000+0000 7f3e62ffd700 1 -- 172.21.15.190:0/2687512014 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f3e68049a60 con 0x7f3e6c0fb930 2024-05-06T12:07:02.004 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:02.000+0000 7f3e71a67700 1 -- 172.21.15.190:0/2687512014 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f3e580368d0 msgr2=0x7f3e58038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/543287293 2024-05-06T12:07:02.004 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:02.000+0000 7f3e71a67700 1 --2- 172.21.15.190:0/2687512014 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f3e580368d0 0x7f3e58038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-06T12:07:02.007 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:02.005+0000 7f3e62ffd700 1 -- 172.21.15.190:0/2687512014 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3e68025400 con 0x7f3e6c0fb930 2024-05-06T12:07:02.203 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:02.201+0000 7f3e71a67700 1 -- 172.21.15.190:0/2687512014 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f3e580368d0 msgr2=0x7f3e58038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/543287293 2024-05-06T12:07:02.203 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:02.201+0000 7f3e71a67700 1 --2- 172.21.15.190:0/2687512014 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f3e580368d0 0x7f3e58038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-06T12:07:02.268 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:02.265+0000 7f3e744cc700 1 -- 172.21.15.190:0/2687512014 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3e6c05f5c0 con 0x7f3e6c0fb930 2024-05-06T12:07:02.268 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:02.266+0000 7f3e62ffd700 1 -- 172.21.15.190:0/2687512014 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f3e68004d30 con 0x7f3e6c0fb930 2024-05-06T12:07:02.269 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:07:02.269 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":1,"fsid":"aee1d912-0ba0-11ef-bc97-c7b262605968","modified":"2024-05-06T12:04:48.255830Z","created":"2024-05-06T12:04:48.255830Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi092","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-06T12:07:02.270 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:02.268+0000 7f3e744cc700 1 -- 172.21.15.190:0/2687512014 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f3e580368d0 msgr2=0x7f3e58038d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-06T12:07:02.271 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:02.268+0000 7f3e744cc700 1 --2- 172.21.15.190:0/2687512014 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f3e580368d0 0x7f3e58038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:07:02.271 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:02.268+0000 7f3e744cc700 1 -- 172.21.15.190:0/2687512014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e6c0fb930 msgr2=0x7f3e6c066880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:07:02.271 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:02.268+0000 7f3e744cc700 1 --2- 172.21.15.190:0/2687512014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e6c0fb930 0x7f3e6c066880 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3e680096b0 tx=0x7f3e68009e30 comp rx=0 tx=0).stop 2024-05-06T12:07:02.271 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:02.268+0000 7f3e744cc700 1 -- 172.21.15.190:0/2687512014 shutdown_connections 2024-05-06T12:07:02.271 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:02.268+0000 7f3e744cc700 1 --2- 172.21.15.190:0/2687512014 >> [v2:172.21.15.92:6800/543287293,v1:172.21.15.92:6801/543287293] conn(0x7f3e580368d0 0x7f3e58038d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:07:02.271 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:02.268+0000 7f3e744cc700 1 --2- 172.21.15.190:0/2687512014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e6c0fb930 0x7f3e6c066880 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:07:02.271 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:02.268+0000 7f3e744cc700 1 -- 172.21.15.190:0/2687512014 >> 172.21.15.190:0/2687512014 conn(0x7f3e6c0f7070 msgr2=0x7f3e6c102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:07:02.271 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:02.268+0000 7f3e744cc700 1 -- 172.21.15.190:0/2687512014 shutdown_connections 2024-05-06T12:07:02.271 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:02.268+0000 7f3e744cc700 1 -- 172.21.15.190:0/2687512014 wait complete. 2024-05-06T12:07:02.271 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 1 2024-05-06T12:07:02.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:02 smithi092 ceph-mon[26954]: from='client.? 172.21.15.190:0/2687512014' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-06T12:07:03.827 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-06T12:07:03.827 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-06T12:07:03.889 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:03 smithi092 ceph-mon[26954]: Active manager daemon smithi092.gyzeqb restarted 2024-05-06T12:07:03.889 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:03 smithi092 ceph-mon[26954]: Activating manager daemon smithi092.gyzeqb 2024-05-06T12:07:03.889 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:03 smithi092 ceph-mon[26954]: osdmap e5: 0 total, 0 up, 0 in 2024-05-06T12:07:03.889 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:03 smithi092 ceph-mon[26954]: mgrmap e15: smithi092.gyzeqb(active, starting, since 0.00592224s) 2024-05-06T12:07:03.889 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:03 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi092"}]: dispatch 2024-05-06T12:07:03.889 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:03 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mgr metadata", "who": "smithi092.gyzeqb", "id": "smithi092.gyzeqb"}]: dispatch 2024-05-06T12:07:03.889 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:03 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-06T12:07:03.889 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:03 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-06T12:07:03.889 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:03 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-06T12:07:03.889 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:03 smithi092 ceph-mon[26954]: Manager daemon smithi092.gyzeqb is now available 2024-05-06T12:07:03.890 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:03 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:07:03.890 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:03 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:07:03.890 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:03 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:03.890 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:03 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:07:04.081 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-06T12:07:04.129 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-06T12:07:04.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:04 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi092.gyzeqb/mirror_snapshot_schedule"}]: dispatch 2024-05-06T12:07:04.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:04 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi092.gyzeqb/trash_purge_schedule"}]: dispatch 2024-05-06T12:07:04.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:04 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:04.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:04 smithi092 ceph-mon[26954]: mgrmap e16: smithi092.gyzeqb(active, since 1.0056s) 2024-05-06T12:07:04.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:04 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:05.651 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.647+0000 7f5a2895d700 1 -- 172.21.15.190:0/862925198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a240fb940 msgr2=0x7f5a240fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:07:05.651 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.647+0000 7f5a2895d700 1 --2- 172.21.15.190:0/862925198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a240fb940 0x7f5a240fbd50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5a140048c0 tx=0x7f5a14009e30 comp rx=0 tx=0).stop 2024-05-06T12:07:05.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.647+0000 7f5a2895d700 1 -- 172.21.15.190:0/862925198 shutdown_connections 2024-05-06T12:07:05.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.647+0000 7f5a2895d700 1 --2- 172.21.15.190:0/862925198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a240fb940 0x7f5a240fbd50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:07:05.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.647+0000 7f5a2895d700 1 -- 172.21.15.190:0/862925198 >> 172.21.15.190:0/862925198 conn(0x7f5a240f70a0 msgr2=0x7f5a240f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:07:05.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.647+0000 7f5a2895d700 1 -- 172.21.15.190:0/862925198 shutdown_connections 2024-05-06T12:07:05.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.647+0000 7f5a2895d700 1 -- 172.21.15.190:0/862925198 wait complete. 2024-05-06T12:07:05.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.648+0000 7f5a2895d700 1 Processor -- start 2024-05-06T12:07:05.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.648+0000 7f5a2895d700 1 -- start start 2024-05-06T12:07:05.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.648+0000 7f5a2895d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a240fb940 0x7f5a2410d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:07:05.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.648+0000 7f5a2895d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2410d5e0 con 0x7f5a240fb940 2024-05-06T12:07:05.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.648+0000 7f5a2259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a240fb940 0x7f5a2410d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:07:05.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.648+0000 7f5a2259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a240fb940 0x7f5a2410d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.190:57490/0 (socket says 172.21.15.190:57490) 2024-05-06T12:07:05.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.648+0000 7f5a2259c700 1 -- 172.21.15.190:0/3511577527 learned_addr learned my addr 172.21.15.190:0/3511577527 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-06T12:07:05.653 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.648+0000 7f5a2259c700 1 -- 172.21.15.190:0/3511577527 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a14005040 con 0x7f5a240fb940 2024-05-06T12:07:05.653 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.649+0000 7f5a2259c700 1 --2- 172.21.15.190:0/3511577527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a240fb940 0x7f5a2410d0a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5a14009680 tx=0x7f5a14009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:07:05.653 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.649+0000 7f5a12ffd700 1 -- 172.21.15.190:0/3511577527 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a1400b5c0 con 0x7f5a240fb940 2024-05-06T12:07:05.653 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.649+0000 7f5a12ffd700 1 -- 172.21.15.190:0/3511577527 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a1400be70 con 0x7f5a240fb940 2024-05-06T12:07:05.653 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.649+0000 7f5a2895d700 1 -- 172.21.15.190:0/3511577527 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a2410d7e0 con 0x7f5a240fb940 2024-05-06T12:07:05.653 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.649+0000 7f5a12ffd700 1 -- 172.21.15.190:0/3511577527 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a140135a0 con 0x7f5a240fb940 2024-05-06T12:07:05.653 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.649+0000 7f5a2895d700 1 -- 172.21.15.190:0/3511577527 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a2410e350 con 0x7f5a240fb940 2024-05-06T12:07:05.654 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.650+0000 7f5a12ffd700 1 -- 172.21.15.190:0/3511577527 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) v1 ==== 43297+0+0 (secure 0 0 0) 0x7f5a14013860 con 0x7f5a240fb940 2024-05-06T12:07:05.654 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.650+0000 7f5a2895d700 1 -- 172.21.15.190:0/3511577527 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a24101e50 con 0x7f5a240fb940 2024-05-06T12:07:05.654 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.650+0000 7f5a12ffd700 1 --2- 172.21.15.190:0/3511577527 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f5a0c036710 0x7f5a0c038bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:07:05.654 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.650+0000 7f5a12ffd700 1 -- 172.21.15.190:0/3511577527 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f5a140493b0 con 0x7f5a240fb940 2024-05-06T12:07:05.654 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.650+0000 7f5a21d9b700 1 --2- 172.21.15.190:0/3511577527 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f5a0c036710 0x7f5a0c038bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:07:05.654 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.650+0000 7f5a21d9b700 1 --2- 172.21.15.190:0/3511577527 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f5a0c036710 0x7f5a0c038bc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5a18009990 tx=0x7f5a18006e40 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:07:05.657 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.655+0000 7f5a12ffd700 1 -- 172.21.15.190:0/3511577527 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5a14011370 con 0x7f5a240fb940 2024-05-06T12:07:05.937 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.934+0000 7f5a2895d700 1 -- 172.21.15.190:0/3511577527 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5a2410e590 con 0x7f5a240fb940 2024-05-06T12:07:05.937 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.935+0000 7f5a12ffd700 1 -- 172.21.15.190:0/3511577527 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f5a14024cf0 con 0x7f5a240fb940 2024-05-06T12:07:05.938 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:07:05.938 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":1,"fsid":"aee1d912-0ba0-11ef-bc97-c7b262605968","modified":"2024-05-06T12:04:48.255830Z","created":"2024-05-06T12:04:48.255830Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi092","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-06T12:07:05.940 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.937+0000 7f5a2895d700 1 -- 172.21.15.190:0/3511577527 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f5a0c036710 msgr2=0x7f5a0c038bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:07:05.940 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.937+0000 7f5a2895d700 1 --2- 172.21.15.190:0/3511577527 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f5a0c036710 0x7f5a0c038bc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5a18009990 tx=0x7f5a18006e40 comp rx=0 tx=0).stop 2024-05-06T12:07:05.940 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.937+0000 7f5a2895d700 1 -- 172.21.15.190:0/3511577527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a240fb940 msgr2=0x7f5a2410d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:07:05.940 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.937+0000 7f5a2895d700 1 --2- 172.21.15.190:0/3511577527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a240fb940 0x7f5a2410d0a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5a14009680 tx=0x7f5a14009e30 comp rx=0 tx=0).stop 2024-05-06T12:07:05.940 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.937+0000 7f5a2895d700 1 -- 172.21.15.190:0/3511577527 shutdown_connections 2024-05-06T12:07:05.940 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.937+0000 7f5a2895d700 1 --2- 172.21.15.190:0/3511577527 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f5a0c036710 0x7f5a0c038bc0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:07:05.940 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.937+0000 7f5a2895d700 1 --2- 172.21.15.190:0/3511577527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a240fb940 0x7f5a2410d0a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:07:05.940 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.937+0000 7f5a2895d700 1 -- 172.21.15.190:0/3511577527 >> 172.21.15.190:0/3511577527 conn(0x7f5a240f70a0 msgr2=0x7f5a24065050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:07:05.940 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.937+0000 7f5a2895d700 1 -- 172.21.15.190:0/3511577527 shutdown_connections 2024-05-06T12:07:05.941 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:05.937+0000 7f5a2895d700 1 -- 172.21.15.190:0/3511577527 wait complete. 2024-05-06T12:07:05.941 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 1 2024-05-06T12:07:06.645 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:06 smithi092 ceph-mon[26954]: [06/May/2024:12:07:05] ENGINE Bus STARTING 2024-05-06T12:07:06.646 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:06 smithi092 ceph-mon[26954]: [06/May/2024:12:07:05] ENGINE Serving on https://172.21.15.92:7150 2024-05-06T12:07:06.646 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:06 smithi092 ceph-mon[26954]: [06/May/2024:12:07:05] ENGINE Bus STARTED 2024-05-06T12:07:06.646 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:06 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:06.646 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:06 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:06.646 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:06 smithi092 ceph-mon[26954]: from='client.? 172.21.15.190:0/3511577527' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-06T12:07:07.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:07 smithi092 ceph-mon[26954]: mgrmap e17: smithi092.gyzeqb(active, since 2s) 2024-05-06T12:07:07.975 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-06T12:07:07.975 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-06T12:07:08.254 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-06T12:07:08.311 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-06T12:07:09.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:08 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:09.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:08 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:09.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:08 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:09.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:08 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:09.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:08 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi092", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:07:10.287 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.283+0000 7fdfef34b700 1 -- 172.21.15.190:0/3164023413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfe80fca20 msgr2=0x7fdfe80fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:07:10.288 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.283+0000 7fdfef34b700 1 --2- 172.21.15.190:0/3164023413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfe80fca20 0x7fdfe80fce30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40048c0 tx=0x7fdfe4009e30 comp rx=0 tx=0).stop 2024-05-06T12:07:10.288 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.283+0000 7fdfef34b700 1 -- 172.21.15.190:0/3164023413 shutdown_connections 2024-05-06T12:07:10.288 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.283+0000 7fdfef34b700 1 --2- 172.21.15.190:0/3164023413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfe80fca20 0x7fdfe80fce30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:07:10.288 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.283+0000 7fdfef34b700 1 -- 172.21.15.190:0/3164023413 >> 172.21.15.190:0/3164023413 conn(0x7fdfe80f81a0 msgr2=0x7fdfe80fa5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:07:10.288 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.283+0000 7fdfef34b700 1 -- 172.21.15.190:0/3164023413 shutdown_connections 2024-05-06T12:07:10.288 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.284+0000 7fdfef34b700 1 -- 172.21.15.190:0/3164023413 wait complete. 2024-05-06T12:07:10.288 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.284+0000 7fdfef34b700 1 Processor -- start 2024-05-06T12:07:10.288 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.284+0000 7fdfef34b700 1 -- start start 2024-05-06T12:07:10.288 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.284+0000 7fdfef34b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfe80fca20 0x7fdfe810d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:07:10.288 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.284+0000 7fdfef34b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe810d5e0 con 0x7fdfe80fca20 2024-05-06T12:07:10.289 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.284+0000 7fdfed0e7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfe80fca20 0x7fdfe810d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:07:10.289 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.284+0000 7fdfed0e7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfe80fca20 0x7fdfe810d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.190:57498/0 (socket says 172.21.15.190:57498) 2024-05-06T12:07:10.289 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.284+0000 7fdfed0e7700 1 -- 172.21.15.190:0/1791042549 learned_addr learned my addr 172.21.15.190:0/1791042549 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-06T12:07:10.290 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.285+0000 7fdfed0e7700 1 -- 172.21.15.190:0/1791042549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe4005040 con 0x7fdfe80fca20 2024-05-06T12:07:10.290 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.285+0000 7fdfed0e7700 1 --2- 172.21.15.190:0/1791042549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfe80fca20 0x7fdfe810d0a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40096b0 tx=0x7fdfe4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:07:10.290 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.285+0000 7fdfddffb700 1 -- 172.21.15.190:0/1791042549 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdfe400bed0 con 0x7fdfe80fca20 2024-05-06T12:07:10.290 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.285+0000 7fdfddffb700 1 -- 172.21.15.190:0/1791042549 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdfe400b5c0 con 0x7fdfe80fca20 2024-05-06T12:07:10.291 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.285+0000 7fdfef34b700 1 -- 172.21.15.190:0/1791042549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfe810d7e0 con 0x7fdfe80fca20 2024-05-06T12:07:10.291 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.285+0000 7fdfddffb700 1 -- 172.21.15.190:0/1791042549 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdfe401c5b0 con 0x7fdfe80fca20 2024-05-06T12:07:10.291 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.285+0000 7fdfef34b700 1 -- 172.21.15.190:0/1791042549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe810e350 con 0x7fdfe80fca20 2024-05-06T12:07:10.291 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.286+0000 7fdfddffb700 1 -- 172.21.15.190:0/1791042549 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) v1 ==== 43489+0+0 (secure 0 0 0) 0x7fdfe401c820 con 0x7fdfe80fca20 2024-05-06T12:07:10.291 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.286+0000 7fdfef34b700 1 -- 172.21.15.190:0/1791042549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfe8102f30 con 0x7fdfe80fca20 2024-05-06T12:07:10.291 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.286+0000 7fdfddffb700 1 --2- 172.21.15.190:0/1791042549 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fdfd4036970 0x7fdfd4038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:07:10.291 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.287+0000 7fdfddffb700 1 -- 172.21.15.190:0/1791042549 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fdfe4049c40 con 0x7fdfe80fca20 2024-05-06T12:07:10.291 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.287+0000 7fdfec8e6700 1 --2- 172.21.15.190:0/1791042549 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fdfd4036970 0x7fdfd4038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:07:10.291 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.287+0000 7fdfec8e6700 1 --2- 172.21.15.190:0/1791042549 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fdfd4036970 0x7fdfd4038e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8009990 tx=0x7fdfd8006e40 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:07:10.294 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.291+0000 7fdfddffb700 1 -- 172.21.15.190:0/1791042549 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fdfe4023300 con 0x7fdfe80fca20 2024-05-06T12:07:10.569 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.566+0000 7fdfef34b700 1 -- 172.21.15.190:0/1791042549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fdfe810ed60 con 0x7fdfe80fca20 2024-05-06T12:07:10.570 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.566+0000 7fdfddffb700 1 -- 172.21.15.190:0/1791042549 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fdfe4011b60 con 0x7fdfe80fca20 2024-05-06T12:07:10.570 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:07:10.570 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":1,"fsid":"aee1d912-0ba0-11ef-bc97-c7b262605968","modified":"2024-05-06T12:04:48.255830Z","created":"2024-05-06T12:04:48.255830Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi092","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-06T12:07:10.572 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.569+0000 7fdfef34b700 1 -- 172.21.15.190:0/1791042549 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fdfd4036970 msgr2=0x7fdfd4038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:07:10.572 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.569+0000 7fdfef34b700 1 --2- 172.21.15.190:0/1791042549 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fdfd4036970 0x7fdfd4038e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8009990 tx=0x7fdfd8006e40 comp rx=0 tx=0).stop 2024-05-06T12:07:10.572 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.569+0000 7fdfef34b700 1 -- 172.21.15.190:0/1791042549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfe80fca20 msgr2=0x7fdfe810d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:07:10.572 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.569+0000 7fdfef34b700 1 --2- 172.21.15.190:0/1791042549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfe80fca20 0x7fdfe810d0a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40096b0 tx=0x7fdfe4009e30 comp rx=0 tx=0).stop 2024-05-06T12:07:10.572 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.569+0000 7fdfef34b700 1 -- 172.21.15.190:0/1791042549 shutdown_connections 2024-05-06T12:07:10.573 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.569+0000 7fdfef34b700 1 --2- 172.21.15.190:0/1791042549 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fdfd4036970 0x7fdfd4038e20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:07:10.573 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.569+0000 7fdfef34b700 1 --2- 172.21.15.190:0/1791042549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfe80fca20 0x7fdfe810d0a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:07:10.573 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.569+0000 7fdfef34b700 1 -- 172.21.15.190:0/1791042549 >> 172.21.15.190:0/1791042549 conn(0x7fdfe80f81a0 msgr2=0x7fdfe8103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:07:10.573 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.569+0000 7fdfef34b700 1 -- 172.21.15.190:0/1791042549 shutdown_connections 2024-05-06T12:07:10.573 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:10.569+0000 7fdfef34b700 1 -- 172.21.15.190:0/1791042549 wait complete. 2024-05-06T12:07:10.573 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 1 2024-05-06T12:07:11.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:10 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:11.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:10 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:11.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:10 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:11.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:10 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:11.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:10 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi190", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:07:11.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:10 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:07:11.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:10 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:07:11.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:10 smithi092 ceph-mon[26954]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-06T12:07:11.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:10 smithi092 ceph-mon[26954]: Updating smithi190:/etc/ceph/ceph.conf 2024-05-06T12:07:11.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:10 smithi092 ceph-mon[26954]: Updating smithi190:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-06T12:07:11.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:10 smithi092 ceph-mon[26954]: Updating smithi092:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-06T12:07:11.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:10 smithi092 ceph-mon[26954]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-05-06T12:07:11.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:10 smithi092 ceph-mon[26954]: from='client.? 172.21.15.190:0/1791042549' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-06T12:07:11.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:10 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:11.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:10 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:12.167 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-06T12:07:12.167 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-06T12:07:12.415 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-06T12:07:12.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:12 smithi092 ceph-mon[26954]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-05-06T12:07:12.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:12 smithi092 ceph-mon[26954]: Updating smithi190:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-06T12:07:12.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:12 smithi092 ceph-mon[26954]: Updating smithi092:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-06T12:07:12.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:12 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:12.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:12 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:12.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:12 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:12.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:12 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi190", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-06T12:07:12.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:12 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi190", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-05-06T12:07:12.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:12 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:07:12.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:12 smithi092 ceph-mon[26954]: Deploying daemon crash.smithi190 on smithi190 2024-05-06T12:07:14.108 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:13.987+0000 7ff63bcd9700 1 -- 172.21.15.190:0/4206313394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff634102aa0 msgr2=0x7ff634102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:07:14.108 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:13.987+0000 7ff63bcd9700 1 --2- 172.21.15.190:0/4206313394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff634102aa0 0x7ff634102e70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff6300048c0 tx=0x7ff630009e30 comp rx=0 tx=0).stop 2024-05-06T12:07:14.109 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:13.987+0000 7ff63bcd9700 1 -- 172.21.15.190:0/4206313394 shutdown_connections 2024-05-06T12:07:14.109 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:13.987+0000 7ff63bcd9700 1 --2- 172.21.15.190:0/4206313394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff634102aa0 0x7ff634102e70 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:07:14.109 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:13.987+0000 7ff63bcd9700 1 -- 172.21.15.190:0/4206313394 >> 172.21.15.190:0/4206313394 conn(0x7ff6340f85e0 msgr2=0x7ff6340fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:07:14.109 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:13.987+0000 7ff63bcd9700 1 -- 172.21.15.190:0/4206313394 shutdown_connections 2024-05-06T12:07:14.109 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:13.987+0000 7ff63bcd9700 1 -- 172.21.15.190:0/4206313394 wait complete. 2024-05-06T12:07:14.109 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:13.988+0000 7ff63bcd9700 1 Processor -- start 2024-05-06T12:07:14.109 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:13.988+0000 7ff63bcd9700 1 -- start start 2024-05-06T12:07:14.109 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:13.988+0000 7ff63bcd9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff634102aa0 0x7ff634113100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:07:14.109 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:13.988+0000 7ff63bcd9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff634113640 con 0x7ff634102aa0 2024-05-06T12:07:14.110 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:13.988+0000 7ff639a75700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff634102aa0 0x7ff634113100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:07:14.110 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:13.988+0000 7ff639a75700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff634102aa0 0x7ff634113100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.190:57508/0 (socket says 172.21.15.190:57508) 2024-05-06T12:07:14.110 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:13.988+0000 7ff639a75700 1 -- 172.21.15.190:0/2131536253 learned_addr learned my addr 172.21.15.190:0/2131536253 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-06T12:07:14.110 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:13.988+0000 7ff639a75700 1 -- 172.21.15.190:0/2131536253 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff630005040 con 0x7ff634102aa0 2024-05-06T12:07:14.110 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:13.988+0000 7ff639a75700 1 --2- 172.21.15.190:0/2131536253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff634102aa0 0x7ff634113100 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff63000b820 tx=0x7ff630009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:07:14.110 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:13.989+0000 7ff62a7fc700 1 -- 172.21.15.190:0/2131536253 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff6300093c0 con 0x7ff634102aa0 2024-05-06T12:07:14.111 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:13.989+0000 7ff63bcd9700 1 -- 172.21.15.190:0/2131536253 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff634113840 con 0x7ff634102aa0 2024-05-06T12:07:14.111 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:13.989+0000 7ff62a7fc700 1 -- 172.21.15.190:0/2131536253 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff630013420 con 0x7ff634102aa0 2024-05-06T12:07:14.111 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:13.989+0000 7ff62a7fc700 1 -- 172.21.15.190:0/2131536253 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff63001c5b0 con 0x7ff634102aa0 2024-05-06T12:07:14.111 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:13.989+0000 7ff63bcd9700 1 -- 172.21.15.190:0/2131536253 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff634113d30 con 0x7ff634102aa0 2024-05-06T12:07:14.111 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:13.989+0000 7ff63bcd9700 1 -- 172.21.15.190:0/2131536253 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6340fe1a0 con 0x7ff634102aa0 2024-05-06T12:07:14.111 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:13.989+0000 7ff62a7fc700 1 -- 172.21.15.190:0/2131536253 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) v1 ==== 43489+0+0 (secure 0 0 0) 0x7ff630013a90 con 0x7ff634102aa0 2024-05-06T12:07:14.111 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:13.990+0000 7ff62a7fc700 1 --2- 172.21.15.190:0/2131536253 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7ff620036970 0x7ff620038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:07:14.111 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:13.990+0000 7ff62a7fc700 1 -- 172.21.15.190:0/2131536253 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7ff630049b30 con 0x7ff634102aa0 2024-05-06T12:07:14.111 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:13.990+0000 7ff639274700 1 --2- 172.21.15.190:0/2131536253 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7ff620036970 0x7ff620038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:07:14.112 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:13.990+0000 7ff639274700 1 --2- 172.21.15.190:0/2131536253 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7ff620036970 0x7ff620038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff624009990 tx=0x7ff624006e40 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:07:14.112 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:13.995+0000 7ff62a7fc700 1 -- 172.21.15.190:0/2131536253 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff6300098a0 con 0x7ff634102aa0 2024-05-06T12:07:14.272 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:14.268+0000 7ff63bcd9700 1 -- 172.21.15.190:0/2131536253 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff63405f5c0 con 0x7ff634102aa0 2024-05-06T12:07:14.272 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:14.269+0000 7ff62a7fc700 1 -- 172.21.15.190:0/2131536253 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7ff630011370 con 0x7ff634102aa0 2024-05-06T12:07:14.273 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:07:14.273 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":1,"fsid":"aee1d912-0ba0-11ef-bc97-c7b262605968","modified":"2024-05-06T12:04:48.255830Z","created":"2024-05-06T12:04:48.255830Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi092","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-06T12:07:14.276 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:14.272+0000 7ff63bcd9700 1 -- 172.21.15.190:0/2131536253 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7ff620036970 msgr2=0x7ff620038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:07:14.276 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:14.272+0000 7ff63bcd9700 1 --2- 172.21.15.190:0/2131536253 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7ff620036970 0x7ff620038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff624009990 tx=0x7ff624006e40 comp rx=0 tx=0).stop 2024-05-06T12:07:14.276 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:14.272+0000 7ff63bcd9700 1 -- 172.21.15.190:0/2131536253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff634102aa0 msgr2=0x7ff634113100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:07:14.276 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:14.272+0000 7ff63bcd9700 1 --2- 172.21.15.190:0/2131536253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff634102aa0 0x7ff634113100 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff63000b820 tx=0x7ff630009e30 comp rx=0 tx=0).stop 2024-05-06T12:07:14.276 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:14.272+0000 7ff63bcd9700 1 -- 172.21.15.190:0/2131536253 shutdown_connections 2024-05-06T12:07:14.276 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:14.272+0000 7ff63bcd9700 1 --2- 172.21.15.190:0/2131536253 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7ff620036970 0x7ff620038e20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:07:14.276 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:14.272+0000 7ff63bcd9700 1 --2- 172.21.15.190:0/2131536253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff634102aa0 0x7ff634113100 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:07:14.276 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:14.272+0000 7ff63bcd9700 1 -- 172.21.15.190:0/2131536253 >> 172.21.15.190:0/2131536253 conn(0x7ff6340f85e0 msgr2=0x7ff6340f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:07:14.276 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:14.273+0000 7ff63bcd9700 1 -- 172.21.15.190:0/2131536253 shutdown_connections 2024-05-06T12:07:14.276 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:14.273+0000 7ff63bcd9700 1 -- 172.21.15.190:0/2131536253 wait complete. 2024-05-06T12:07:14.277 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 1 2024-05-06T12:07:14.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:14 smithi092 ceph-mon[26954]: from='client.? 172.21.15.190:0/2131536253' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-06T12:07:14.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:14 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:15.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:15 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:15.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:15 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:15.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:15 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:15.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:15 smithi092 ceph-mon[26954]: Deploying daemon node-exporter.smithi190 on smithi190 2024-05-06T12:07:15.928 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-06T12:07:15.928 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-06T12:07:16.179 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-06T12:07:19.067 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.047+0000 7f8783a70700 1 -- 172.21.15.190:0/1538728859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f877c101990 msgr2=0x7f877c101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:07:19.067 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.047+0000 7f8783a70700 1 --2- 172.21.15.190:0/1538728859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f877c101990 0x7f877c101d60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8774009c60 tx=0x7f8774004d10 comp rx=0 tx=0).stop 2024-05-06T12:07:19.067 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.047+0000 7f8783a70700 1 -- 172.21.15.190:0/1538728859 shutdown_connections 2024-05-06T12:07:19.067 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.047+0000 7f8783a70700 1 --2- 172.21.15.190:0/1538728859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f877c101990 0x7f877c101d60 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:07:19.068 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.047+0000 7f8783a70700 1 -- 172.21.15.190:0/1538728859 >> 172.21.15.190:0/1538728859 conn(0x7f877c0f74f0 msgr2=0x7f877c0f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:07:19.068 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.047+0000 7f8783a70700 1 -- 172.21.15.190:0/1538728859 shutdown_connections 2024-05-06T12:07:19.068 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.047+0000 7f8783a70700 1 -- 172.21.15.190:0/1538728859 wait complete. 2024-05-06T12:07:19.068 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.048+0000 7f8783a70700 1 Processor -- start 2024-05-06T12:07:19.068 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.048+0000 7f8783a70700 1 -- start start 2024-05-06T12:07:19.068 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.048+0000 7f8783a70700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f877c101990 0x7f877c0f7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:07:19.069 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.048+0000 7f8783a70700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877c0f7ed0 con 0x7f877c101990 2024-05-06T12:07:19.069 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.048+0000 7f878180c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f877c101990 0x7f877c0f7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:07:19.069 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.048+0000 7f878180c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f877c101990 0x7f877c0f7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.190:52514/0 (socket says 172.21.15.190:52514) 2024-05-06T12:07:19.069 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.048+0000 7f878180c700 1 -- 172.21.15.190:0/4265447135 learned_addr learned my addr 172.21.15.190:0/4265447135 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-06T12:07:19.069 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.048+0000 7f878180c700 1 -- 172.21.15.190:0/4265447135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8774009940 con 0x7f877c101990 2024-05-06T12:07:19.070 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.049+0000 7f878180c700 1 --2- 172.21.15.190:0/4265447135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f877c101990 0x7f877c0f7990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f877400b740 tx=0x7f8774004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:07:19.070 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.049+0000 7f87727fc700 1 -- 172.21.15.190:0/4265447135 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8774007b30 con 0x7f877c101990 2024-05-06T12:07:19.070 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.049+0000 7f8783a70700 1 -- 172.21.15.190:0/4265447135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f877c0f80d0 con 0x7f877c101990 2024-05-06T12:07:19.070 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.049+0000 7f87727fc700 1 -- 172.21.15.190:0/4265447135 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f877401d910 con 0x7f877c101990 2024-05-06T12:07:19.070 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.049+0000 7f87727fc700 1 -- 172.21.15.190:0/4265447135 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f87740128b0 con 0x7f877c101990 2024-05-06T12:07:19.070 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.049+0000 7f8783a70700 1 -- 172.21.15.190:0/4265447135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f877c0f85c0 con 0x7f877c101990 2024-05-06T12:07:19.071 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.050+0000 7f87727fc700 1 -- 172.21.15.190:0/4265447135 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) v1 ==== 43489+0+0 (secure 0 0 0) 0x7f8774019040 con 0x7f877c101990 2024-05-06T12:07:19.071 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.050+0000 7f8783a70700 1 -- 172.21.15.190:0/4265447135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f877c0f87a0 con 0x7f877c101990 2024-05-06T12:07:19.071 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.050+0000 7f87727fc700 1 --2- 172.21.15.190:0/4265447135 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f8768036970 0x7f8768038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:07:19.071 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.050+0000 7f87727fc700 1 -- 172.21.15.190:0/4265447135 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f877404a4b0 con 0x7f877c101990 2024-05-06T12:07:19.071 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.050+0000 7f878100b700 1 --2- 172.21.15.190:0/4265447135 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f8768036970 0x7f8768038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:07:19.072 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.051+0000 7f878100b700 1 --2- 172.21.15.190:0/4265447135 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f8768036970 0x7f8768038e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f876c009990 tx=0x7f876c006e40 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:07:19.072 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.055+0000 7f87727fc700 1 -- 172.21.15.190:0/4265447135 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f877402eb40 con 0x7f877c101990 2024-05-06T12:07:19.336 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.334+0000 7f8783a70700 1 -- 172.21.15.190:0/4265447135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f877c04bed0 con 0x7f877c101990 2024-05-06T12:07:19.337 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.335+0000 7f87727fc700 1 -- 172.21.15.190:0/4265447135 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f8774017020 con 0x7f877c101990 2024-05-06T12:07:19.337 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:07:19.337 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":1,"fsid":"aee1d912-0ba0-11ef-bc97-c7b262605968","modified":"2024-05-06T12:04:48.255830Z","created":"2024-05-06T12:04:48.255830Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi092","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-06T12:07:19.339 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.337+0000 7f8783a70700 1 -- 172.21.15.190:0/4265447135 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f8768036970 msgr2=0x7f8768038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:07:19.340 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.337+0000 7f8783a70700 1 --2- 172.21.15.190:0/4265447135 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f8768036970 0x7f8768038e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f876c009990 tx=0x7f876c006e40 comp rx=0 tx=0).stop 2024-05-06T12:07:19.340 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.337+0000 7f8783a70700 1 -- 172.21.15.190:0/4265447135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f877c101990 msgr2=0x7f877c0f7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:07:19.340 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.337+0000 7f8783a70700 1 --2- 172.21.15.190:0/4265447135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f877c101990 0x7f877c0f7990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f877400b740 tx=0x7f8774004d10 comp rx=0 tx=0).stop 2024-05-06T12:07:19.340 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.337+0000 7f8783a70700 1 -- 172.21.15.190:0/4265447135 shutdown_connections 2024-05-06T12:07:19.340 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.337+0000 7f8783a70700 1 --2- 172.21.15.190:0/4265447135 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f8768036970 0x7f8768038e20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:07:19.340 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.337+0000 7f8783a70700 1 --2- 172.21.15.190:0/4265447135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f877c101990 0x7f877c0f7990 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:07:19.340 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.337+0000 7f8783a70700 1 -- 172.21.15.190:0/4265447135 >> 172.21.15.190:0/4265447135 conn(0x7f877c0f74f0 msgr2=0x7f877c065ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:07:19.340 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.337+0000 7f8783a70700 1 -- 172.21.15.190:0/4265447135 shutdown_connections 2024-05-06T12:07:19.340 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:19.337+0000 7f8783a70700 1 -- 172.21.15.190:0/4265447135 wait complete. 2024-05-06T12:07:19.341 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 1 2024-05-06T12:07:19.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:19 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:19.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:19 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:19.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:19 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:19.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:19 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:19.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:19 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi190.kczxkh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-06T12:07:19.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:19 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi190.kczxkh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-06T12:07:19.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:19 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-06T12:07:19.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:19 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:07:19.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:19 smithi092 ceph-mon[26954]: Deploying daemon mgr.smithi190.kczxkh on smithi190 2024-05-06T12:07:19.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:19 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:19.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:19 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:07:19.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:19 smithi092 ceph-mon[26954]: from='client.? 172.21.15.190:0/4265447135' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-06T12:07:22.091 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-06T12:07:22.091 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-06T12:07:22.868 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-06T12:07:22.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:22 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:22.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:22 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:22.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:22 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:22.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:22 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:22.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:22 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-06T12:07:22.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:22 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:07:22.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:22 smithi092 ceph-mon[26954]: Deploying daemon mon.smithi190 on smithi190 2024-05-06T12:07:25.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:24 smithi092 ceph-mon[26954]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:07:25.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:24 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:25.914 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:25.910+0000 7f0b5abac700 1 -- 172.21.15.190:0/1537591417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b54102bb0 msgr2=0x7f0b54102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:07:25.914 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:25.910+0000 7f0b5abac700 1 --2- 172.21.15.190:0/1537591417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b54102bb0 0x7f0b54102f80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0b480048c0 tx=0x7f0b48009e30 comp rx=0 tx=0).stop 2024-05-06T12:07:25.915 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:25.910+0000 7f0b5abac700 1 -- 172.21.15.190:0/1537591417 shutdown_connections 2024-05-06T12:07:25.915 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:25.910+0000 7f0b5abac700 1 --2- 172.21.15.190:0/1537591417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b54102bb0 0x7f0b54102f80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:07:25.915 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:25.910+0000 7f0b5abac700 1 -- 172.21.15.190:0/1537591417 >> 172.21.15.190:0/1537591417 conn(0x7f0b540f85c0 msgr2=0x7f0b540fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:07:25.915 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:25.911+0000 7f0b5abac700 1 -- 172.21.15.190:0/1537591417 shutdown_connections 2024-05-06T12:07:25.915 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:25.911+0000 7f0b5abac700 1 -- 172.21.15.190:0/1537591417 wait complete. 2024-05-06T12:07:25.915 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:25.911+0000 7f0b5abac700 1 Processor -- start 2024-05-06T12:07:25.915 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:25.911+0000 7f0b5abac700 1 -- start start 2024-05-06T12:07:25.915 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:25.912+0000 7f0b5abac700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b54102bb0 0x7f0b54070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:07:25.915 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:25.912+0000 7f0b5abac700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b54072260 con 0x7f0b54102bb0 2024-05-06T12:07:25.915 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:25.912+0000 7f0b58948700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b54102bb0 0x7f0b54070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:07:25.915 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:25.912+0000 7f0b58948700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b54102bb0 0x7f0b54070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.190:36456/0 (socket says 172.21.15.190:36456) 2024-05-06T12:07:25.915 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:25.912+0000 7f0b58948700 1 -- 172.21.15.190:0/1578280549 learned_addr learned my addr 172.21.15.190:0/1578280549 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-06T12:07:25.915 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:25.912+0000 7f0b58948700 1 -- 172.21.15.190:0/1578280549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b48005040 con 0x7f0b54102bb0 2024-05-06T12:07:25.915 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:25.912+0000 7f0b58948700 1 --2- 172.21.15.190:0/1578280549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b54102bb0 0x7f0b54070260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0b4800b820 tx=0x7f0b48009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:07:25.917 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:25.912+0000 7f0b517fa700 1 -- 172.21.15.190:0/1578280549 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0b480093c0 con 0x7f0b54102bb0 2024-05-06T12:07:25.917 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:25.912+0000 7f0b5abac700 1 -- 172.21.15.190:0/1578280549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b54070800 con 0x7f0b54102bb0 2024-05-06T12:07:25.917 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:25.913+0000 7f0b517fa700 1 -- 172.21.15.190:0/1578280549 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b48013420 con 0x7f0b54102bb0 2024-05-06T12:07:25.917 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:25.913+0000 7f0b517fa700 1 -- 172.21.15.190:0/1578280549 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0b4801c5b0 con 0x7f0b54102bb0 2024-05-06T12:07:25.917 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:25.913+0000 7f0b5abac700 1 -- 172.21.15.190:0/1578280549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b54070cf0 con 0x7f0b54102bb0 2024-05-06T12:07:25.918 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:25.914+0000 7f0b3effd700 1 -- 172.21.15.190:0/1578280549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b340052b0 con 0x7f0b54102bb0 2024-05-06T12:07:25.918 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:25.914+0000 7f0b517fa700 1 -- 172.21.15.190:0/1578280549 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) v1 ==== 43489+0+0 (secure 0 0 0) 0x7f0b48013a90 con 0x7f0b54102bb0 2024-05-06T12:07:25.918 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:25.914+0000 7f0b517fa700 1 --2- 172.21.15.190:0/1578280549 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f0b440369c0 0x7f0b44038e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:07:25.918 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:25.914+0000 7f0b517fa700 1 -- 172.21.15.190:0/1578280549 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f0b48023300 con 0x7f0b54102bb0 2024-05-06T12:07:25.918 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:25.915+0000 7f0b53fff700 1 --2- 172.21.15.190:0/1578280549 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f0b440369c0 0x7f0b44038e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:07:25.918 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:25.915+0000 7f0b53fff700 1 --2- 172.21.15.190:0/1578280549 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f0b440369c0 0x7f0b44038e70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0b40009990 tx=0x7f0b40006e40 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:07:25.922 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:25.919+0000 7f0b517fa700 1 -- 172.21.15.190:0/1578280549 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0b48011960 con 0x7f0b54102bb0 2024-05-06T12:07:26.211 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:26.208+0000 7f0b3effd700 1 -- 172.21.15.190:0/1578280549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0b34005830 con 0x7f0b54102bb0 2024-05-06T12:07:26.211 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:26.209+0000 7f0b517fa700 1 -- 172.21.15.190:0/1578280549 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f0b48023560 con 0x7f0b54102bb0 2024-05-06T12:07:26.212 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:07:26.212 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":1,"fsid":"aee1d912-0ba0-11ef-bc97-c7b262605968","modified":"2024-05-06T12:04:48.255830Z","created":"2024-05-06T12:04:48.255830Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi092","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-06T12:07:26.213 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:26.211+0000 7f0b3effd700 1 -- 172.21.15.190:0/1578280549 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f0b440369c0 msgr2=0x7f0b44038e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:07:26.213 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:26.211+0000 7f0b3effd700 1 --2- 172.21.15.190:0/1578280549 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f0b440369c0 0x7f0b44038e70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0b40009990 tx=0x7f0b40006e40 comp rx=0 tx=0).stop 2024-05-06T12:07:26.214 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:26.211+0000 7f0b3effd700 1 -- 172.21.15.190:0/1578280549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b54102bb0 msgr2=0x7f0b54070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:07:26.214 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:26.211+0000 7f0b3effd700 1 --2- 172.21.15.190:0/1578280549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b54102bb0 0x7f0b54070260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0b4800b820 tx=0x7f0b48009e30 comp rx=0 tx=0).stop 2024-05-06T12:07:26.214 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:26.211+0000 7f0b3effd700 1 -- 172.21.15.190:0/1578280549 shutdown_connections 2024-05-06T12:07:26.214 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:26.211+0000 7f0b3effd700 1 --2- 172.21.15.190:0/1578280549 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f0b440369c0 0x7f0b44038e70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:07:26.214 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:26.211+0000 7f0b3effd700 1 --2- 172.21.15.190:0/1578280549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b54102bb0 0x7f0b54070260 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:07:26.214 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:26.211+0000 7f0b3effd700 1 -- 172.21.15.190:0/1578280549 >> 172.21.15.190:0/1578280549 conn(0x7f0b540f85c0 msgr2=0x7f0b540f9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:07:26.214 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:26.211+0000 7f0b3effd700 1 -- 172.21.15.190:0/1578280549 shutdown_connections 2024-05-06T12:07:26.214 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:26.211+0000 7f0b3effd700 1 -- 172.21.15.190:0/1578280549 wait complete. 2024-05-06T12:07:26.217 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 1 2024-05-06T12:07:28.119 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-06T12:07:28.120 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-06T12:07:28.369 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi190/config 2024-05-06T12:07:31.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:31 smithi092 ceph-mon[26954]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:07:31.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:31 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi092"}]: dispatch 2024-05-06T12:07:31.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:31 smithi092 ceph-mon[26954]: mon.smithi092 calling monitor election 2024-05-06T12:07:31.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:31 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi190"}]: dispatch 2024-05-06T12:07:31.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:31 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi190"}]: dispatch 2024-05-06T12:07:31.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:31 smithi092 ceph-mon[26954]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:07:31.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:31 smithi092 ceph-mon[26954]: from='mgr.? 172.21.15.190:0/4034373643' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi190.kczxkh/crt"}]: dispatch 2024-05-06T12:07:31.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:31 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi190"}]: dispatch 2024-05-06T12:07:31.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:31 smithi092 ceph-mon[26954]: mon.smithi190 calling monitor election 2024-05-06T12:07:31.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:31 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi190"}]: dispatch 2024-05-06T12:07:31.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:31 smithi092 ceph-mon[26954]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:07:31.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:31 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi190"}]: dispatch 2024-05-06T12:07:31.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:31 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi190"}]: dispatch 2024-05-06T12:07:31.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:31 smithi092 ceph-mon[26954]: mon.smithi092 is new leader, mons smithi092,smithi190 in quorum (ranks 0,1) 2024-05-06T12:07:31.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:31 smithi092 ceph-mon[26954]: monmap e2: 2 mons at {smithi092=[v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0],smithi190=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-06T12:07:31.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:31 smithi092 ceph-mon[26954]: fsmap 2024-05-06T12:07:31.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:31 smithi092 ceph-mon[26954]: osdmap e5: 0 total, 0 up, 0 in 2024-05-06T12:07:31.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:31 smithi092 ceph-mon[26954]: mgrmap e17: smithi092.gyzeqb(active, since 27s) 2024-05-06T12:07:31.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:31 smithi092 ceph-mon[26954]: overall HEALTH_OK 2024-05-06T12:07:31.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:31 smithi092 ceph-mon[26954]: Standby manager daemon smithi190.kczxkh started 2024-05-06T12:07:31.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:31 smithi092 ceph-mon[26954]: from='mgr.? 172.21.15.190:0/4034373643' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-06T12:07:31.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:31 smithi092 ceph-mon[26954]: from='mgr.? 172.21.15.190:0/4034373643' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi190.kczxkh/key"}]: dispatch 2024-05-06T12:07:31.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:31 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:31.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:31 smithi092 ceph-mon[26954]: from='mgr.? 172.21.15.190:0/4034373643' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-06T12:07:31.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:31 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:31.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:31 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:31.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:31 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:31.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:31 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:31.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:31 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:32.720 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:32 smithi190 ceph-mon[33798]: mgrmap e18: smithi092.gyzeqb(active, since 27s), standbys: smithi190.kczxkh 2024-05-06T12:07:32.720 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:32 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mgr metadata", "who": "smithi190.kczxkh", "id": "smithi190.kczxkh"}]: dispatch 2024-05-06T12:07:32.720 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:32 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:07:32.721 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:32 smithi190 ceph-mon[33798]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:07:32.721 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:32 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi190"}]: dispatch 2024-05-06T12:07:32.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:32 smithi092 ceph-mon[26954]: mgrmap e18: smithi092.gyzeqb(active, since 27s), standbys: smithi190.kczxkh 2024-05-06T12:07:32.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:32 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mgr metadata", "who": "smithi190.kczxkh", "id": "smithi190.kczxkh"}]: dispatch 2024-05-06T12:07:32.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:32 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:07:32.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:32 smithi092 ceph-mon[26954]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:07:32.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:32 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi190"}]: dispatch 2024-05-06T12:07:32.800 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.797+0000 7fbcaeb70700 1 -- 172.21.15.190:0/299810278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc98003580 msgr2=0x7fbc98005a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:07:32.801 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.797+0000 7fbcaeb70700 1 --2- 172.21.15.190:0/299810278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc98003580 0x7fbc98005a00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbca80655d0 tx=0x7fbc9c00b040 comp rx=0 tx=0).stop 2024-05-06T12:07:32.801 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.797+0000 7fbcaeb70700 1 -- 172.21.15.190:0/299810278 shutdown_connections 2024-05-06T12:07:32.801 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.797+0000 7fbcaeb70700 1 --2- 172.21.15.190:0/299810278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc98003580 0x7fbc98005a00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:07:32.801 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.797+0000 7fbcaeb70700 1 -- 172.21.15.190:0/299810278 >> 172.21.15.190:0/299810278 conn(0x7fbca8072080 msgr2=0x7fbca8072480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:07:32.801 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.797+0000 7fbcaeb70700 1 -- 172.21.15.190:0/299810278 shutdown_connections 2024-05-06T12:07:32.801 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.797+0000 7fbcaeb70700 1 -- 172.21.15.190:0/299810278 wait complete. 2024-05-06T12:07:32.801 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.797+0000 7fbcaeb70700 1 Processor -- start 2024-05-06T12:07:32.801 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.797+0000 7fbcaeb70700 1 -- start start 2024-05-06T12:07:32.801 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.798+0000 7fbcaeb70700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc98003580 0x7fbca81198f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:07:32.801 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.798+0000 7fbcaeb70700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbca8119e30 0x7fbca8113870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:07:32.801 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.798+0000 7fbcaeb70700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca811a3f0 con 0x7fbca8119e30 2024-05-06T12:07:32.801 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.798+0000 7fbcaeb70700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca8113db0 con 0x7fbc98003580 2024-05-06T12:07:32.802 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.798+0000 7fbcac90c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc98003580 0x7fbca81198f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:07:32.802 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.798+0000 7fbcac90c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc98003580 0x7fbca81198f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:56370/0 (socket says 172.21.15.190:56370) 2024-05-06T12:07:32.802 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.798+0000 7fbcac90c700 1 -- 172.21.15.190:0/272940816 learned_addr learned my addr 172.21.15.190:0/272940816 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-06T12:07:32.802 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.798+0000 7fbcac90c700 1 -- 172.21.15.190:0/272940816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc98003580 msgr2=0x7fbca81198f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-06T12:07:32.802 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.798+0000 7fbcac90c700 1 -- 172.21.15.190:0/272940816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc98003580 msgr2=0x7fbca81198f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-06T12:07:32.802 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.798+0000 7fbcac90c700 1 --2- 172.21.15.190:0/272940816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc98003580 0x7fbca81198f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-05-06T12:07:32.802 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.798+0000 7fbcac90c700 1 --2- 172.21.15.190:0/272940816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc98003580 0x7fbca81198f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-06T12:07:32.802 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.798+0000 7fbca7fff700 1 --2- 172.21.15.190:0/272940816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbca8119e30 0x7fbca8113870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:07:32.804 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.798+0000 7fbca7fff700 1 -- 172.21.15.190:0/272940816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc98003580 msgr2=0x7fbca81198f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-06T12:07:32.804 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.798+0000 7fbca7fff700 1 --2- 172.21.15.190:0/272940816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc98003580 0x7fbca81198f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:07:32.804 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.798+0000 7fbca7fff700 1 -- 172.21.15.190:0/272940816 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc9c005040 con 0x7fbca8119e30 2024-05-06T12:07:32.804 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.799+0000 7fbca7fff700 1 --2- 172.21.15.190:0/272940816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbca8119e30 0x7fbca8113870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbc9400b790 tx=0x7fbc940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:07:32.804 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.799+0000 7fbca57fa700 1 -- 172.21.15.190:0/272940816 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc94011810 con 0x7fbca8119e30 2024-05-06T12:07:32.804 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.799+0000 7fbcaeb70700 1 -- 172.21.15.190:0/272940816 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca8114090 con 0x7fbca8119e30 2024-05-06T12:07:32.804 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.799+0000 7fbca57fa700 1 -- 172.21.15.190:0/272940816 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc94011e40 con 0x7fbca8119e30 2024-05-06T12:07:32.804 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.799+0000 7fbca57fa700 1 -- 172.21.15.190:0/272940816 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc9400f2e0 con 0x7fbca8119e30 2024-05-06T12:07:32.804 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.799+0000 7fbcaeb70700 1 -- 172.21.15.190:0/272940816 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca81145d0 con 0x7fbca8119e30 2024-05-06T12:07:32.804 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.800+0000 7fbcaeb70700 1 -- 172.21.15.190:0/272940816 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbca810d6f0 con 0x7fbca8119e30 2024-05-06T12:07:32.805 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.800+0000 7fbca57fa700 1 -- 172.21.15.190:0/272940816 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7fbc94011970 con 0x7fbca8119e30 2024-05-06T12:07:32.805 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.801+0000 7fbca57fa700 1 --2- 172.21.15.190:0/272940816 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fbc8c069360 0x7fbc8c06b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:07:32.805 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.801+0000 7fbca57fa700 1 -- 172.21.15.190:0/272940816 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fbc94086460 con 0x7fbca8119e30 2024-05-06T12:07:32.805 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.801+0000 7fbcac90c700 1 --2- 172.21.15.190:0/272940816 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fbc8c069360 0x7fbc8c06b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:07:32.805 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.801+0000 7fbcac90c700 1 --2- 172.21.15.190:0/272940816 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fbc8c069360 0x7fbc8c06b810 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c007c60 tx=0x7fbc9c018040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:07:32.808 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:32.806+0000 7fbca57fa700 1 -- 172.21.15.190:0/272940816 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fbc940571e0 con 0x7fbca8119e30 2024-05-06T12:07:33.077 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:33.074+0000 7fbcaeb70700 1 -- 172.21.15.190:0/272940816 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbca805f5c0 con 0x7fbca8119e30 2024-05-06T12:07:33.077 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:33.075+0000 7fbca57fa700 1 -- 172.21.15.190:0/272940816 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1017 (secure 0 0 0) 0x7fbc94015020 con 0x7fbca8119e30 2024-05-06T12:07:33.078 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-06T12:07:33.078 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":2,"fsid":"aee1d912-0ba0-11ef-bc97-c7b262605968","modified":"2024-05-06T12:07:26.387383Z","created":"2024-05-06T12:04:48.255830Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi092","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi190","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-06T12:07:33.079 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:33.077+0000 7fbcaeb70700 1 -- 172.21.15.190:0/272940816 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fbc8c069360 msgr2=0x7fbc8c06b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:07:33.079 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:33.077+0000 7fbcaeb70700 1 --2- 172.21.15.190:0/272940816 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fbc8c069360 0x7fbc8c06b810 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c007c60 tx=0x7fbc9c018040 comp rx=0 tx=0).stop 2024-05-06T12:07:33.079 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:33.077+0000 7fbcaeb70700 1 -- 172.21.15.190:0/272940816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbca8119e30 msgr2=0x7fbca8113870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:07:33.080 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:33.077+0000 7fbcaeb70700 1 --2- 172.21.15.190:0/272940816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbca8119e30 0x7fbca8113870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbc9400b790 tx=0x7fbc940091a0 comp rx=0 tx=0).stop 2024-05-06T12:07:33.080 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:33.077+0000 7fbcaeb70700 1 -- 172.21.15.190:0/272940816 shutdown_connections 2024-05-06T12:07:33.080 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:33.077+0000 7fbcaeb70700 1 --2- 172.21.15.190:0/272940816 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fbc8c069360 0x7fbc8c06b810 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:07:33.080 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:33.077+0000 7fbcaeb70700 1 --2- 172.21.15.190:0/272940816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc98003580 0x7fbca81198f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:07:33.080 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:33.077+0000 7fbcaeb70700 1 --2- 172.21.15.190:0/272940816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbca8119e30 0x7fbca8113870 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:07:33.080 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:33.077+0000 7fbcaeb70700 1 -- 172.21.15.190:0/272940816 >> 172.21.15.190:0/272940816 conn(0x7fbca8072080 msgr2=0x7fbca80fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:07:33.080 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:33.077+0000 7fbcaeb70700 1 -- 172.21.15.190:0/272940816 shutdown_connections 2024-05-06T12:07:33.080 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:07:33.077+0000 7fbcaeb70700 1 -- 172.21.15.190:0/272940816 wait complete. 2024-05-06T12:07:33.080 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 2 2024-05-06T12:07:33.687 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:33 smithi190 ceph-mon[33798]: from='client.? 172.21.15.190:0/272940816' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-06T12:07:33.711 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-06T12:07:33.711 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph config generate-minimal-conf 2024-05-06T12:07:33.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:33 smithi092 ceph-mon[26954]: from='client.? 172.21.15.190:0/272940816' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-06T12:07:33.993 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:07:35.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:34 smithi190 ceph-mon[33798]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:07:35.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:34 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:35.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:34 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:35.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:34 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:07:35.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:34 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:07:35.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:34 smithi190 ceph-mon[33798]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-06T12:07:35.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:34 smithi190 ceph-mon[33798]: Updating smithi190:/etc/ceph/ceph.conf 2024-05-06T12:07:35.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:34 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:07:35.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:34 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:35.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:34 smithi190 ceph-mon[33798]: Updating smithi092:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-06T12:07:35.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:34 smithi190 ceph-mon[33798]: Updating smithi190:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-06T12:07:35.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:34 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:35.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:34 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:35.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:34 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:35.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:34 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:35.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:34 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:35.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:34 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-06T12:07:35.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:34 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-06T12:07:35.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:34 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:07:35.169 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:34 smithi092 ceph-mon[26954]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:07:35.169 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:34 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:35.169 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:34 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:35.169 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:34 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:07:35.169 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:34 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:07:35.170 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:34 smithi092 ceph-mon[26954]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-06T12:07:35.170 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:34 smithi092 ceph-mon[26954]: Updating smithi190:/etc/ceph/ceph.conf 2024-05-06T12:07:35.170 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:34 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:07:35.170 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:34 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:35.170 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:34 smithi092 ceph-mon[26954]: Updating smithi092:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-06T12:07:35.170 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:34 smithi092 ceph-mon[26954]: Updating smithi190:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-06T12:07:35.170 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:34 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:35.171 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:34 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:35.171 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:34 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:35.171 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:34 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:35.171 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:34 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:35.171 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:34 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-06T12:07:35.171 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:34 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-06T12:07:35.171 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:34 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:07:36.161 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:35 smithi092 ceph-mon[26954]: Reconfiguring mon.smithi092 (unknown last config time)... 2024-05-06T12:07:36.162 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:35 smithi092 ceph-mon[26954]: Reconfiguring daemon mon.smithi092 on smithi092 2024-05-06T12:07:36.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:35 smithi190 ceph-mon[33798]: Reconfiguring mon.smithi092 (unknown last config time)... 2024-05-06T12:07:36.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:35 smithi190 ceph-mon[33798]: Reconfiguring daemon mon.smithi092 on smithi092 2024-05-06T12:07:36.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.615+0000 7f7c9fecc700 1 -- 172.21.15.92:0/1677654715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c98102e50 msgr2=0x7f7c98103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:07:36.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.615+0000 7f7c9fecc700 1 --2- 172.21.15.92:0/1677654715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c98102e50 0x7f7c98103220 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7c94004940 tx=0x7f7c94009e30 comp rx=0 tx=0).stop 2024-05-06T12:07:36.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.615+0000 7f7c9fecc700 1 -- 172.21.15.92:0/1677654715 shutdown_connections 2024-05-06T12:07:36.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.615+0000 7f7c9fecc700 1 --2- 172.21.15.92:0/1677654715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c98065640 0x7f7c98065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:07:36.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.615+0000 7f7c9fecc700 1 --2- 172.21.15.92:0/1677654715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c98102e50 0x7f7c98103220 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:07:36.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.615+0000 7f7c9fecc700 1 -- 172.21.15.92:0/1677654715 >> 172.21.15.92:0/1677654715 conn(0x7f7c98071d00 msgr2=0x7f7c98072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:07:36.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.615+0000 7f7c9fecc700 1 -- 172.21.15.92:0/1677654715 shutdown_connections 2024-05-06T12:07:36.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.615+0000 7f7c9fecc700 1 -- 172.21.15.92:0/1677654715 wait complete. 2024-05-06T12:07:36.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.616+0000 7f7c9fecc700 1 Processor -- start 2024-05-06T12:07:36.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.616+0000 7f7c9fecc700 1 -- start start 2024-05-06T12:07:36.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.616+0000 7f7c9fecc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c98065640 0x7f7c98113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:07:36.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.616+0000 7f7c9fecc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c98102e50 0x7f7c981140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:07:36.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.616+0000 7f7c9fecc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c981147c0 con 0x7f7c98102e50 2024-05-06T12:07:36.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.616+0000 7f7c9fecc700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9810db20 con 0x7f7c98065640 2024-05-06T12:07:36.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.616+0000 7f7c9d467700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c98102e50 0x7f7c981140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:07:36.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.616+0000 7f7c9d467700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c98102e50 0x7f7c981140e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38452/0 (socket says 172.21.15.92:38452) 2024-05-06T12:07:36.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.616+0000 7f7c9d467700 1 -- 172.21.15.92:0/2036632693 learned_addr learned my addr 172.21.15.92:0/2036632693 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:07:36.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.616+0000 7f7c9dc68700 1 --2- 172.21.15.92:0/2036632693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c98065640 0x7f7c98113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:07:36.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.616+0000 7f7c9d467700 1 -- 172.21.15.92:0/2036632693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c98065640 msgr2=0x7f7c98113ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:07:36.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.616+0000 7f7c9d467700 1 --2- 172.21.15.92:0/2036632693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c98065640 0x7f7c98113ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:07:36.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.616+0000 7f7c9d467700 1 -- 172.21.15.92:0/2036632693 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c94005040 con 0x7f7c98102e50 2024-05-06T12:07:36.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.616+0000 7f7c9d467700 1 --2- 172.21.15.92:0/2036632693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c98102e50 0x7f7c981140e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7c8800b790 tx=0x7f7c88009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:07:36.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.617+0000 7f7c8e7fc700 1 -- 172.21.15.92:0/2036632693 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c8800f770 con 0x7f7c98102e50 2024-05-06T12:07:36.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.617+0000 7f7c9fecc700 1 -- 172.21.15.92:0/2036632693 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c9810de00 con 0x7f7c98102e50 2024-05-06T12:07:36.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.617+0000 7f7c8e7fc700 1 -- 172.21.15.92:0/2036632693 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c8800fda0 con 0x7f7c98102e50 2024-05-06T12:07:36.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.617+0000 7f7c8e7fc700 1 -- 172.21.15.92:0/2036632693 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c8800d310 con 0x7f7c98102e50 2024-05-06T12:07:36.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.617+0000 7f7c9fecc700 1 -- 172.21.15.92:0/2036632693 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c9810e3c0 con 0x7f7c98102e50 2024-05-06T12:07:36.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.618+0000 7f7c9fecc700 1 -- 172.21.15.92:0/2036632693 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c980fe540 con 0x7f7c98102e50 2024-05-06T12:07:36.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.618+0000 7f7c8e7fc700 1 -- 172.21.15.92:0/2036632693 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7f7c88010420 con 0x7f7c98102e50 2024-05-06T12:07:36.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.619+0000 7f7c8e7fc700 1 --2- 172.21.15.92:0/2036632693 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f7c84069210 0x7f7c8406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:07:36.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.619+0000 7f7c8e7fc700 1 -- 172.21.15.92:0/2036632693 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f7c88086140 con 0x7f7c98102e50 2024-05-06T12:07:36.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.619+0000 7f7c9dc68700 1 --2- 172.21.15.92:0/2036632693 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f7c84069210 0x7f7c8406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:07:36.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.619+0000 7f7c9dc68700 1 --2- 172.21.15.92:0/2036632693 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f7c84069210 0x7f7c8406b6c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7c94004f90 tx=0x7f7c94009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:07:36.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.623+0000 7f7c8e7fc700 1 -- 172.21.15.92:0/2036632693 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7c88056f40 con 0x7f7c98102e50 2024-05-06T12:07:36.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.829+0000 7f7c9fecc700 1 -- 172.21.15.92:0/2036632693 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f7c9805f5c0 con 0x7f7c98102e50 2024-05-06T12:07:36.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.829+0000 7f7c8e7fc700 1 -- 172.21.15.92:0/2036632693 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+225 (secure 0 0 0) 0x7f7c88056ad0 con 0x7f7c98102e50 2024-05-06T12:07:36.832 INFO:teuthology.orchestra.run.smithi092.stdout:# minimal ceph.conf for aee1d912-0ba0-11ef-bc97-c7b262605968 2024-05-06T12:07:36.833 INFO:teuthology.orchestra.run.smithi092.stdout:[global] 2024-05-06T12:07:36.833 INFO:teuthology.orchestra.run.smithi092.stdout: fsid = aee1d912-0ba0-11ef-bc97-c7b262605968 2024-05-06T12:07:36.833 INFO:teuthology.orchestra.run.smithi092.stdout: mon_host = [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-05-06T12:07:36.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.831+0000 7f7c9fecc700 1 -- 172.21.15.92:0/2036632693 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f7c84069210 msgr2=0x7f7c8406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:07:36.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.831+0000 7f7c9fecc700 1 --2- 172.21.15.92:0/2036632693 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f7c84069210 0x7f7c8406b6c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7c94004f90 tx=0x7f7c94009e30 comp rx=0 tx=0).stop 2024-05-06T12:07:36.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.831+0000 7f7c9fecc700 1 -- 172.21.15.92:0/2036632693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c98102e50 msgr2=0x7f7c981140e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:07:36.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.831+0000 7f7c9fecc700 1 --2- 172.21.15.92:0/2036632693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c98102e50 0x7f7c981140e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7c8800b790 tx=0x7f7c88009250 comp rx=0 tx=0).stop 2024-05-06T12:07:36.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.831+0000 7f7c9fecc700 1 -- 172.21.15.92:0/2036632693 shutdown_connections 2024-05-06T12:07:36.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.831+0000 7f7c9fecc700 1 --2- 172.21.15.92:0/2036632693 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f7c84069210 0x7f7c8406b6c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:07:36.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.831+0000 7f7c9fecc700 1 --2- 172.21.15.92:0/2036632693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c98065640 0x7f7c98113ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:07:36.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.831+0000 7f7c9fecc700 1 --2- 172.21.15.92:0/2036632693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c98102e50 0x7f7c981140e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:07:36.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.831+0000 7f7c9fecc700 1 -- 172.21.15.92:0/2036632693 >> 172.21.15.92:0/2036632693 conn(0x7f7c98071d00 msgr2=0x7f7c980fa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:07:36.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.831+0000 7f7c9fecc700 1 -- 172.21.15.92:0/2036632693 shutdown_connections 2024-05-06T12:07:36.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:36.831+0000 7f7c9fecc700 1 -- 172.21.15.92:0/2036632693 wait complete. 2024-05-06T12:07:37.521 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:37 smithi092 ceph-mon[26954]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:07:37.522 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:37 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:37.522 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:37 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:37.522 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:37 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi092.gyzeqb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-06T12:07:37.522 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:37 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-06T12:07:37.522 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:37 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:07:37.522 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:37 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/2036632693' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:07:37.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:37 smithi190 ceph-mon[33798]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:07:37.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:37 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:37.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:37 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:37.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:37 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi092.gyzeqb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-06T12:07:37.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:37 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-06T12:07:37.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:37 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:07:37.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:37 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/2036632693' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:07:37.978 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-06T12:07:37.978 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-06T12:07:37.978 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.conf 2024-05-06T12:07:38.043 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-06T12:07:38.044 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-06T12:07:38.106 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-06T12:07:38.107 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.conf 2024-05-06T12:07:38.139 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-06T12:07:38.139 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-06T12:07:38.206 INFO:tasks.cephadm:Deploying OSDs... 2024-05-06T12:07:38.206 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-06T12:07:38.207 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/scratch_devs of=/dev/stdout 2024-05-06T12:07:38.221 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-06T12:07:38.222 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_1 2024-05-06T12:07:38.276 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-06T12:07:38.276 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-06T12:07:38.276 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-06T12:07:38.276 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-06T12:07:38.277 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-05-06T12:07:38.277 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-05-06 12:06:55.664006272 +0000 2024-05-06T12:07:38.277 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-05-06 12:01:09.175796389 +0000 2024-05-06T12:07:38.277 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-05-06 12:01:09.175796389 +0000 2024-05-06T12:07:38.277 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-05-06 12:01:09.175796389 +0000 2024-05-06T12:07:38.277 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-06T12:07:38.338 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-05-06T12:07:38.339 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-05-06T12:07:38.339 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000251134 s, 2.0 MB/s 2024-05-06T12:07:38.340 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-06T12:07:38.382 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:38 smithi190 ceph-mon[33798]: Reconfiguring mgr.smithi092.gyzeqb (unknown last config time)... 2024-05-06T12:07:38.382 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:38 smithi190 ceph-mon[33798]: Reconfiguring daemon mgr.smithi092.gyzeqb on smithi092 2024-05-06T12:07:38.395 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_2 2024-05-06T12:07:38.449 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-06T12:07:38.449 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-06T12:07:38.449 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-06T12:07:38.449 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-06T12:07:38.449 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-05-06T12:07:38.449 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-05-06 12:06:55.664006272 +0000 2024-05-06T12:07:38.449 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-05-06 12:01:09.524786121 +0000 2024-05-06T12:07:38.450 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-05-06 12:01:09.524786121 +0000 2024-05-06T12:07:38.450 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-05-06 12:01:09.524786121 +0000 2024-05-06T12:07:38.450 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-06T12:07:38.510 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:38 smithi092 ceph-mon[26954]: Reconfiguring mgr.smithi092.gyzeqb (unknown last config time)... 2024-05-06T12:07:38.510 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:38 smithi092 ceph-mon[26954]: Reconfiguring daemon mgr.smithi092.gyzeqb on smithi092 2024-05-06T12:07:38.513 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-05-06T12:07:38.513 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-05-06T12:07:38.513 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000185236 s, 2.8 MB/s 2024-05-06T12:07:38.514 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-06T12:07:38.571 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_3 2024-05-06T12:07:38.625 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-06T12:07:38.625 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-06T12:07:38.625 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-05-06T12:07:38.625 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-06T12:07:38.625 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-05-06T12:07:38.626 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-05-06 12:06:55.665006244 +0000 2024-05-06T12:07:38.626 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-05-06 12:01:09.864776118 +0000 2024-05-06T12:07:38.626 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-05-06 12:01:09.864776118 +0000 2024-05-06T12:07:38.626 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-05-06 12:01:09.864776118 +0000 2024-05-06T12:07:38.626 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-06T12:07:38.688 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-05-06T12:07:38.688 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-05-06T12:07:38.688 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000200584 s, 2.6 MB/s 2024-05-06T12:07:38.689 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-06T12:07:38.745 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_4 2024-05-06T12:07:38.800 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-06T12:07:38.883 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-06T12:07:38.883 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-05-06T12:07:38.883 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-06T12:07:38.883 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-05-06T12:07:38.884 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-05-06 12:06:55.665006244 +0000 2024-05-06T12:07:38.884 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-05-06 12:01:10.226765468 +0000 2024-05-06T12:07:38.884 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-05-06 12:01:10.226765468 +0000 2024-05-06T12:07:38.884 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-05-06 12:01:10.226765468 +0000 2024-05-06T12:07:38.884 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-06T12:07:38.907 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-05-06T12:07:38.907 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-05-06T12:07:38.907 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000195242 s, 2.6 MB/s 2024-05-06T12:07:38.908 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-06T12:07:38.964 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-06T12:07:38.964 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/scratch_devs of=/dev/stdout 2024-05-06T12:07:38.979 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-06T12:07:38.980 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_1 2024-05-06T12:07:39.035 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-06T12:07:39.035 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-06T12:07:39.035 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-06T12:07:39.035 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-06T12:07:39.035 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-05-06T12:07:39.035 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-05-06 12:07:25.642547368 +0000 2024-05-06T12:07:39.035 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-05-06 12:01:08.975473723 +0000 2024-05-06T12:07:39.035 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-05-06 12:01:08.975473723 +0000 2024-05-06T12:07:39.036 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-05-06 12:01:08.975473723 +0000 2024-05-06T12:07:39.036 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-06T12:07:39.097 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-05-06T12:07:39.098 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-05-06T12:07:39.098 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000208117 s, 2.5 MB/s 2024-05-06T12:07:39.099 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-06T12:07:39.154 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_2 2024-05-06T12:07:39.209 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-06T12:07:39.210 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-06T12:07:39.210 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-06T12:07:39.210 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-06T12:07:39.210 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-05-06T12:07:39.210 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-05-06 12:07:25.643547340 +0000 2024-05-06T12:07:39.210 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-05-06 12:01:09.315463868 +0000 2024-05-06T12:07:39.210 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-05-06 12:01:09.315463868 +0000 2024-05-06T12:07:39.210 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-05-06 12:01:09.315463868 +0000 2024-05-06T12:07:39.210 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-06T12:07:39.271 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-05-06T12:07:39.271 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-05-06T12:07:39.271 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000190425 s, 2.7 MB/s 2024-05-06T12:07:39.272 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-06T12:07:39.328 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_3 2024-05-06T12:07:39.383 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-06T12:07:39.383 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-06T12:07:39.383 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-05-06T12:07:39.383 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-06T12:07:39.383 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-05-06T12:07:39.383 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-05-06 12:07:25.643547340 +0000 2024-05-06T12:07:39.383 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-05-06 12:01:09.641454418 +0000 2024-05-06T12:07:39.383 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-05-06 12:01:09.641454418 +0000 2024-05-06T12:07:39.384 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-05-06 12:01:09.641454418 +0000 2024-05-06T12:07:39.384 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-06T12:07:39.445 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-05-06T12:07:39.445 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-05-06T12:07:39.445 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000181722 s, 2.8 MB/s 2024-05-06T12:07:39.446 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-06T12:07:39.501 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_4 2024-05-06T12:07:39.557 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-06T12:07:39.557 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-06T12:07:39.557 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-06T12:07:39.557 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-06T12:07:39.557 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-05-06T12:07:39.557 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-05-06 12:07:25.643547340 +0000 2024-05-06T12:07:39.557 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-05-06 12:01:09.977444678 +0000 2024-05-06T12:07:39.558 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-05-06 12:01:09.977444678 +0000 2024-05-06T12:07:39.558 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-05-06 12:01:09.977444678 +0000 2024-05-06T12:07:39.558 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-06T12:07:39.619 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-05-06T12:07:39.619 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-05-06T12:07:39.619 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000176608 s, 2.9 MB/s 2024-05-06T12:07:39.620 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-06T12:07:39.675 INFO:tasks.cephadm:Deploying osd.0 on smithi092 with /dev/vg_nvme/lv_4... 2024-05-06T12:07:39.675 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-06T12:07:39.698 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:39 smithi092 ceph-mon[26954]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:07:39.698 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:39 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:39.698 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:39 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:39.698 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:39 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi092", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-06T12:07:39.698 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:39 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:07:39.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:39 smithi190 ceph-mon[33798]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:07:39.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:39 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:39.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:39 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:39.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:39 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi092", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-06T12:07:39.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:39 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:07:39.933 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:07:40.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:40 smithi092 ceph-mon[26954]: Reconfiguring crash.smithi092 (monmap changed)... 2024-05-06T12:07:40.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:40 smithi092 ceph-mon[26954]: Reconfiguring daemon crash.smithi092 on smithi092 2024-05-06T12:07:40.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:40 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:40.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:40 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:40.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:40 smithi190 ceph-mon[33798]: Reconfiguring crash.smithi092 (monmap changed)... 2024-05-06T12:07:40.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:40 smithi190 ceph-mon[33798]: Reconfiguring daemon crash.smithi092 on smithi092 2024-05-06T12:07:40.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:40 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:40.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:40 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:41.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:41 smithi092 ceph-mon[26954]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:07:41.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:41 smithi092 ceph-mon[26954]: Reconfiguring alertmanager.smithi092 (dependencies changed)... 2024-05-06T12:07:41.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:41 smithi092 ceph-mon[26954]: Reconfiguring daemon alertmanager.smithi092 on smithi092 2024-05-06T12:07:41.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:41 smithi190 ceph-mon[33798]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:07:41.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:41 smithi190 ceph-mon[33798]: Reconfiguring alertmanager.smithi092 (dependencies changed)... 2024-05-06T12:07:41.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:41 smithi190 ceph-mon[33798]: Reconfiguring daemon alertmanager.smithi092 on smithi092 2024-05-06T12:07:43.665 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:43 smithi092 ceph-mon[26954]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:07:43.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:43 smithi190 ceph-mon[33798]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:07:44.369 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-06T12:07:44.369 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-06T12:07:44.370 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-05-06T12:07:44.370 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-05-06T12:07:44.370 INFO:teuthology.orchestra.run.smithi092.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0116307 s, 902 MB/s 2024-05-06T12:07:44.370 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-05-06T12:07:45.435 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_4 2024-05-06T12:07:45.692 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:45 smithi092 ceph-mon[26954]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:07:45.757 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:07:45.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:45 smithi190 ceph-mon[33798]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:07:47.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:47 smithi092 ceph-mon[26954]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:07:47.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:47 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:47.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:47 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:47.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:47 smithi190 ceph-mon[33798]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:07:47.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:47 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:47.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:47 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:48.673 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:48 smithi092 ceph-mon[26954]: Reconfiguring grafana.smithi092 (dependencies changed)... 2024-05-06T12:07:48.673 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:48 smithi092 ceph-mon[26954]: Reconfiguring daemon grafana.smithi092 on smithi092 2024-05-06T12:07:48.810 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:48 smithi190 ceph-mon[33798]: Reconfiguring grafana.smithi092 (dependencies changed)... 2024-05-06T12:07:48.810 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:48 smithi190 ceph-mon[33798]: Reconfiguring daemon grafana.smithi092 on smithi092 2024-05-06T12:07:49.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.444+0000 7fab6d358700 1 -- 172.21.15.92:0/1749814110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab680fbcb0 msgr2=0x7fab680fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:07:49.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.444+0000 7fab6d358700 1 --2- 172.21.15.92:0/1749814110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab680fbcb0 0x7fab680fc0c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fab50004940 tx=0x7fab50009e30 comp rx=0 tx=0).stop 2024-05-06T12:07:49.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.444+0000 7fab6d358700 1 -- 172.21.15.92:0/1749814110 shutdown_connections 2024-05-06T12:07:49.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.444+0000 7fab6d358700 1 --2- 172.21.15.92:0/1749814110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab680fcf60 0x7fab680fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:07:49.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.444+0000 7fab6d358700 1 --2- 172.21.15.92:0/1749814110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab680fbcb0 0x7fab680fc0c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:07:49.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.444+0000 7fab6d358700 1 -- 172.21.15.92:0/1749814110 >> 172.21.15.92:0/1749814110 conn(0x7fab680f7430 msgr2=0x7fab680f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:07:49.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.444+0000 7fab6d358700 1 -- 172.21.15.92:0/1749814110 shutdown_connections 2024-05-06T12:07:49.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.444+0000 7fab6d358700 1 -- 172.21.15.92:0/1749814110 wait complete. 2024-05-06T12:07:49.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.445+0000 7fab6d358700 1 Processor -- start 2024-05-06T12:07:49.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.445+0000 7fab6d358700 1 -- start start 2024-05-06T12:07:49.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.445+0000 7fab6d358700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab680fbcb0 0x7fab680ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:07:49.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.445+0000 7fab6d358700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab680fcf60 0x7fab680fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:07:49.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.445+0000 7fab6d358700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab681001f0 con 0x7fab680fcf60 2024-05-06T12:07:49.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.445+0000 7fab6d358700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab680fe7e0 con 0x7fab680fbcb0 2024-05-06T12:07:49.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.445+0000 7fab667fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab680fcf60 0x7fab680fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:07:49.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.445+0000 7fab667fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab680fcf60 0x7fab680fe2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54670/0 (socket says 172.21.15.92:54670) 2024-05-06T12:07:49.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.445+0000 7fab667fc700 1 -- 172.21.15.92:0/811802570 learned_addr learned my addr 172.21.15.92:0/811802570 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:07:49.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.445+0000 7fab66ffd700 1 --2- 172.21.15.92:0/811802570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab680fbcb0 0x7fab680ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:07:49.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.445+0000 7fab667fc700 1 -- 172.21.15.92:0/811802570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab680fbcb0 msgr2=0x7fab680ffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:07:49.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.446+0000 7fab667fc700 1 --2- 172.21.15.92:0/811802570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab680fbcb0 0x7fab680ffbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:07:49.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.446+0000 7fab667fc700 1 -- 172.21.15.92:0/811802570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab50005040 con 0x7fab680fcf60 2024-05-06T12:07:49.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.446+0000 7fab667fc700 1 --2- 172.21.15.92:0/811802570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab680fcf60 0x7fab680fe2a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fab58002a50 tx=0x7fab580046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:07:49.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.446+0000 7fab5f7fe700 1 -- 172.21.15.92:0/811802570 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab58007cc0 con 0x7fab680fcf60 2024-05-06T12:07:49.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.446+0000 7fab5f7fe700 1 -- 172.21.15.92:0/811802570 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab5800f040 con 0x7fab680fcf60 2024-05-06T12:07:49.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.446+0000 7fab5f7fe700 1 -- 172.21.15.92:0/811802570 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab5800da30 con 0x7fab680fcf60 2024-05-06T12:07:49.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.446+0000 7fab6d358700 1 -- 172.21.15.92:0/811802570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab680fea90 con 0x7fab680fcf60 2024-05-06T12:07:49.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.446+0000 7fab6d358700 1 -- 172.21.15.92:0/811802570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab680ff050 con 0x7fab680fcf60 2024-05-06T12:07:49.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.447+0000 7fab6d358700 1 -- 172.21.15.92:0/811802570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab68101af0 con 0x7fab680fcf60 2024-05-06T12:07:49.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.447+0000 7fab5f7fe700 1 -- 172.21.15.92:0/811802570 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7fab5800dbb0 con 0x7fab680fcf60 2024-05-06T12:07:49.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.448+0000 7fab5f7fe700 1 --2- 172.21.15.92:0/811802570 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fab54069210 0x7fab5406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:07:49.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.448+0000 7fab5f7fe700 1 -- 172.21.15.92:0/811802570 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fab58085e30 con 0x7fab680fcf60 2024-05-06T12:07:49.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.448+0000 7fab66ffd700 1 --2- 172.21.15.92:0/811802570 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fab54069210 0x7fab5406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:07:49.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.448+0000 7fab66ffd700 1 --2- 172.21.15.92:0/811802570 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fab54069210 0x7fab5406b6c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fab5000b970 tx=0x7fab50009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:07:49.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.452+0000 7fab5f7fe700 1 -- 172.21.15.92:0/811802570 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fab58056c50 con 0x7fab680fcf60 2024-05-06T12:07:49.618 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:49 smithi092 ceph-mon[26954]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:07:49.618 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:49 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:07:49.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:07:49.655+0000 7fab6d358700 1 -- 172.21.15.92:0/811802570 --> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fab6805e3a0 con 0x7fab54069210 2024-05-06T12:07:49.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:49 smithi190 ceph-mon[33798]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:07:49.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:49 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:07:50.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:50 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-06T12:07:50.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:50 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-06T12:07:50.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:50 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:07:50.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:50 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-06T12:07:50.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:50 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-06T12:07:50.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:50 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:07:51.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:51 smithi092 ceph-mon[26954]: from='client.14244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:07:51.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:51 smithi092 ceph-mon[26954]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:07:51.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:51 smithi190 ceph-mon[33798]: from='client.14244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:07:51.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:51 smithi190 ceph-mon[33798]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:07:53.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:53 smithi092 ceph-mon[26954]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:07:53.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:53 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:53.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:53 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:53.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:53 smithi190 ceph-mon[33798]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:07:53.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:53 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:53.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:53 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:07:54.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:54 smithi092 ceph-mon[26954]: Reconfiguring prometheus.smithi092 (dependencies changed)... 2024-05-06T12:07:54.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:54 smithi092 ceph-mon[26954]: Reconfiguring daemon prometheus.smithi092 on smithi092 2024-05-06T12:07:54.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:54 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/735192585' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b276435-4698-4017-9535-4be41a4e1379"}]: dispatch 2024-05-06T12:07:54.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:54 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/735192585' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1b276435-4698-4017-9535-4be41a4e1379"}]': finished 2024-05-06T12:07:54.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:54 smithi092 ceph-mon[26954]: osdmap e6: 1 total, 0 up, 1 in 2024-05-06T12:07:54.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:54 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-06T12:07:54.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:54 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/1302568917' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-06T12:07:54.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:54 smithi190 ceph-mon[33798]: Reconfiguring prometheus.smithi092 (dependencies changed)... 2024-05-06T12:07:54.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:54 smithi190 ceph-mon[33798]: Reconfiguring daemon prometheus.smithi092 on smithi092 2024-05-06T12:07:54.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:54 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/735192585' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b276435-4698-4017-9535-4be41a4e1379"}]: dispatch 2024-05-06T12:07:54.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:54 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/735192585' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1b276435-4698-4017-9535-4be41a4e1379"}]': finished 2024-05-06T12:07:54.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:54 smithi190 ceph-mon[33798]: osdmap e6: 1 total, 0 up, 1 in 2024-05-06T12:07:54.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:54 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-06T12:07:54.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:54 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/1302568917' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-06T12:07:55.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:55 smithi092 ceph-mon[26954]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:07:55.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:55 smithi190 ceph-mon[33798]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:07:57.679 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:57 smithi092 ceph-mon[26954]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:07:57.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:57 smithi190 ceph-mon[33798]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:07:59.537 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:07:59 smithi092 ceph-mon[26954]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:07:59.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:07:59 smithi190 ceph-mon[33798]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:08:01.599 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:01 smithi092 ceph-mon[26954]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:08:01.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:01 smithi190 ceph-mon[33798]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:08:03.503 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:03 smithi092 ceph-mon[26954]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:08:03.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:03 smithi190 ceph-mon[33798]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:08:04.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:04 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:04.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:04 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:04.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:04 smithi190 ceph-mon[33798]: Reconfiguring crash.smithi190 (monmap changed)... 2024-05-06T12:08:04.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:04 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi190", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-06T12:08:04.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:04 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:08:04.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:04 smithi190 ceph-mon[33798]: Reconfiguring daemon crash.smithi190 on smithi190 2024-05-06T12:08:04.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:04 smithi190 ceph-mon[33798]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:08:04.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:04 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:08:04.862 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:04 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:04.862 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:04 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:04.862 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:04 smithi092 ceph-mon[26954]: Reconfiguring crash.smithi190 (monmap changed)... 2024-05-06T12:08:04.863 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:04 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi190", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-06T12:08:04.863 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:04 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:08:04.863 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:04 smithi092 ceph-mon[26954]: Reconfiguring daemon crash.smithi190 on smithi190 2024-05-06T12:08:04.863 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:04 smithi092 ceph-mon[26954]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:08:04.863 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:04 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:08:05.719 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:05 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-06T12:08:05.719 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:05 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:08:05.719 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:05 smithi190 ceph-mon[33798]: Deploying daemon osd.0 on smithi092 2024-05-06T12:08:05.719 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:05 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:05.719 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:05 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:05.720 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:05 smithi190 ceph-mon[33798]: Reconfiguring mgr.smithi190.kczxkh (monmap changed)... 2024-05-06T12:08:05.720 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:05 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi190.kczxkh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-06T12:08:05.720 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:05 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-06T12:08:05.720 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:05 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:08:05.720 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:05 smithi190 ceph-mon[33798]: Reconfiguring daemon mgr.smithi190.kczxkh on smithi190 2024-05-06T12:08:05.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:05 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-06T12:08:05.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:05 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:08:05.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:05 smithi092 ceph-mon[26954]: Deploying daemon osd.0 on smithi092 2024-05-06T12:08:05.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:05 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:05.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:05 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:05.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:05 smithi092 ceph-mon[26954]: Reconfiguring mgr.smithi190.kczxkh (monmap changed)... 2024-05-06T12:08:05.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:05 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi190.kczxkh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-06T12:08:05.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:05 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-06T12:08:05.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:05 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:08:05.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:05 smithi092 ceph-mon[26954]: Reconfiguring daemon mgr.smithi190.kczxkh on smithi190 2024-05-06T12:08:07.673 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:07 smithi092 ceph-mon[26954]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:08:07.673 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:07 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:07.674 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:07 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:07.674 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:07 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-06T12:08:07.674 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:07 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-06T12:08:07.674 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:07 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:08:07.762 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:07 smithi190 ceph-mon[33798]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:08:07.762 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:07 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:07.762 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:07 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:07.762 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:07 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-06T12:08:07.762 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:07 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-06T12:08:07.762 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:07 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:08:08.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:08 smithi092 ceph-mon[26954]: Reconfiguring mon.smithi190 (monmap changed)... 2024-05-06T12:08:08.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:08 smithi092 ceph-mon[26954]: Reconfiguring daemon mon.smithi190 on smithi190 2024-05-06T12:08:08.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:08 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:08.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:08 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:08.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:08 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-06T12:08:08.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:08 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi092:9093"}]: dispatch 2024-05-06T12:08:08.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:08 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:08.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:08 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-06T12:08:08.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:08 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi092:3000"}]: dispatch 2024-05-06T12:08:08.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:08 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:08.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:08 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-06T12:08:08.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:08 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi092:9095"}]: dispatch 2024-05-06T12:08:08.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:08 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:08.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:08 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:08:08.806 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:08 smithi190 ceph-mon[33798]: Reconfiguring mon.smithi190 (monmap changed)... 2024-05-06T12:08:08.806 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:08 smithi190 ceph-mon[33798]: Reconfiguring daemon mon.smithi190 on smithi190 2024-05-06T12:08:08.806 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:08 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:08.806 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:08 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:08.806 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:08 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-06T12:08:08.807 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:08 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi092:9093"}]: dispatch 2024-05-06T12:08:08.807 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:08 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:08.807 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:08 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-06T12:08:08.807 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:08 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi092:3000"}]: dispatch 2024-05-06T12:08:08.807 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:08 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:08.807 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:08 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-06T12:08:08.807 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:08 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi092:9095"}]: dispatch 2024-05-06T12:08:08.807 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:08 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:08.807 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:08 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:08:09.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:09 smithi092 ceph-mon[26954]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:08:09.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:09 smithi092 ceph-mon[26954]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-06T12:08:09.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:09 smithi092 ceph-mon[26954]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi092:9093"}]: dispatch 2024-05-06T12:08:09.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:09 smithi092 ceph-mon[26954]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-06T12:08:09.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:09 smithi092 ceph-mon[26954]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi092:3000"}]: dispatch 2024-05-06T12:08:09.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:09 smithi092 ceph-mon[26954]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-06T12:08:09.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:09 smithi092 ceph-mon[26954]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi092:9095"}]: dispatch 2024-05-06T12:08:09.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:09 smithi190 ceph-mon[33798]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:08:09.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:09 smithi190 ceph-mon[33798]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-06T12:08:09.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:09 smithi190 ceph-mon[33798]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi092:9093"}]: dispatch 2024-05-06T12:08:09.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:09 smithi190 ceph-mon[33798]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-06T12:08:09.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:09 smithi190 ceph-mon[33798]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi092:3000"}]: dispatch 2024-05-06T12:08:09.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:09 smithi190 ceph-mon[33798]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-06T12:08:09.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:09 smithi190 ceph-mon[33798]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi092:9095"}]: dispatch 2024-05-06T12:08:11.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:10 smithi190 ceph-mon[33798]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:08:11.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:10 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:11.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:10 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:11.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:10 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:11.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:10 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:11.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:10 smithi092 ceph-mon[26954]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:08:11.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:10 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:11.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:10 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:11.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:10 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:11.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:10 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:13.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:12 smithi190 ceph-mon[33798]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:08:13.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:12 smithi092 ceph-mon[26954]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:08:14.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:14.373+0000 7fab5f7fe700 1 -- 172.21.15.92:0/811802570 <== mgr.14210 v2:172.21.15.92:6800/3045905943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fab6805e3a0 con 0x7fab54069210 2024-05-06T12:08:14.376 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 0 on host 'smithi092' 2024-05-06T12:08:14.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:14.375+0000 7fab6d358700 1 -- 172.21.15.92:0/811802570 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fab54069210 msgr2=0x7fab5406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:08:14.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:14.375+0000 7fab6d358700 1 --2- 172.21.15.92:0/811802570 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fab54069210 0x7fab5406b6c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fab5000b970 tx=0x7fab50009e30 comp rx=0 tx=0).stop 2024-05-06T12:08:14.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:14.375+0000 7fab6d358700 1 -- 172.21.15.92:0/811802570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab680fcf60 msgr2=0x7fab680fe2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:08:14.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:14.375+0000 7fab6d358700 1 --2- 172.21.15.92:0/811802570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab680fcf60 0x7fab680fe2a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fab58002a50 tx=0x7fab580046b0 comp rx=0 tx=0).stop 2024-05-06T12:08:14.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:14.375+0000 7fab6d358700 1 -- 172.21.15.92:0/811802570 shutdown_connections 2024-05-06T12:08:14.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:14.375+0000 7fab6d358700 1 --2- 172.21.15.92:0/811802570 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fab54069210 0x7fab5406b6c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:08:14.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:14.375+0000 7fab6d358700 1 --2- 172.21.15.92:0/811802570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab680fbcb0 0x7fab680ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:08:14.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:14.375+0000 7fab6d358700 1 --2- 172.21.15.92:0/811802570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab680fcf60 0x7fab680fe2a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:08:14.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:14.375+0000 7fab6d358700 1 -- 172.21.15.92:0/811802570 >> 172.21.15.92:0/811802570 conn(0x7fab680f7430 msgr2=0x7fab68066790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:08:14.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:14.375+0000 7fab6d358700 1 -- 172.21.15.92:0/811802570 shutdown_connections 2024-05-06T12:08:14.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:14.375+0000 7fab6d358700 1 -- 172.21.15.92:0/811802570 wait complete. 2024-05-06T12:08:15.286 DEBUG:teuthology.orchestra.run.smithi092:osd.0> sudo journalctl -f -n 0 -u ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@osd.0.service 2024-05-06T12:08:15.289 INFO:tasks.cephadm:Deploying osd.1 on smithi092 with /dev/vg_nvme/lv_3... 2024-05-06T12:08:15.289 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-06T12:08:15.573 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:15 smithi092 ceph-mon[26954]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:08:15.574 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:15 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:15.574 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:15 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:15.574 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:15 smithi092 ceph-mon[26954]: from='osd.0 [v2:172.21.15.92:6802/3575291733,v1:172.21.15.92:6803/3575291733]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-06T12:08:15.846 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:08:15.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:15 smithi190 ceph-mon[33798]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:08:15.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:15 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:15.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:15 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:15.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:15 smithi190 ceph-mon[33798]: from='osd.0 [v2:172.21.15.92:6802/3575291733,v1:172.21.15.92:6803/3575291733]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-06T12:08:16.688 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:16 smithi092 ceph-mon[26954]: from='osd.0 [v2:172.21.15.92:6802/3575291733,v1:172.21.15.92:6803/3575291733]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-06T12:08:16.689 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:16 smithi092 ceph-mon[26954]: osdmap e7: 1 total, 0 up, 1 in 2024-05-06T12:08:16.689 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:16 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-06T12:08:16.689 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:16 smithi092 ceph-mon[26954]: from='osd.0 [v2:172.21.15.92:6802/3575291733,v1:172.21.15.92:6803/3575291733]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-06T12:08:16.689 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:16 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:16.689 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:16 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:16.689 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:08:16 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0[45502]: 2024-05-06T12:08:16.371+0000 7efde1b06700 -1 osd.0 0 waiting for initial osdmap 2024-05-06T12:08:16.689 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:08:16 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0[45502]: 2024-05-06T12:08:16.374+0000 7efddc100700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-06T12:08:16.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:16 smithi190 ceph-mon[33798]: from='osd.0 [v2:172.21.15.92:6802/3575291733,v1:172.21.15.92:6803/3575291733]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-06T12:08:16.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:16 smithi190 ceph-mon[33798]: osdmap e7: 1 total, 0 up, 1 in 2024-05-06T12:08:16.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:16 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-06T12:08:16.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:16 smithi190 ceph-mon[33798]: from='osd.0 [v2:172.21.15.92:6802/3575291733,v1:172.21.15.92:6803/3575291733]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-06T12:08:16.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:16 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:16.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:16 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:17.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:17 smithi092 ceph-mon[26954]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:08:17.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:17 smithi092 ceph-mon[26954]: from='osd.0 [v2:172.21.15.92:6802/3575291733,v1:172.21.15.92:6803/3575291733]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-06T12:08:17.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:17 smithi092 ceph-mon[26954]: osdmap e8: 1 total, 0 up, 1 in 2024-05-06T12:08:17.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:17 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-06T12:08:17.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:17 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-06T12:08:17.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:17 smithi190 ceph-mon[33798]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:08:17.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:17 smithi190 ceph-mon[33798]: from='osd.0 [v2:172.21.15.92:6802/3575291733,v1:172.21.15.92:6803/3575291733]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-06T12:08:17.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:17 smithi190 ceph-mon[33798]: osdmap e8: 1 total, 0 up, 1 in 2024-05-06T12:08:17.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:17 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-06T12:08:17.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:17 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-06T12:08:17.984 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-06T12:08:17.985 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-06T12:08:17.985 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-05-06T12:08:17.985 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-05-06T12:08:17.985 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0137617 s, 762 MB/s 2024-05-06T12:08:17.985 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-05-06T12:08:18.656 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:18 smithi092 ceph-mon[26954]: purged_snaps scrub starts 2024-05-06T12:08:18.657 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:18 smithi092 ceph-mon[26954]: purged_snaps scrub ok 2024-05-06T12:08:18.657 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:18 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-06T12:08:18.657 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:18 smithi092 ceph-mon[26954]: osd.0 [v2:172.21.15.92:6802/3575291733,v1:172.21.15.92:6803/3575291733] boot 2024-05-06T12:08:18.657 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:18 smithi092 ceph-mon[26954]: osdmap e9: 1 total, 1 up, 1 in 2024-05-06T12:08:18.657 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:18 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-06T12:08:18.806 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:18 smithi190 ceph-mon[33798]: purged_snaps scrub starts 2024-05-06T12:08:18.806 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:18 smithi190 ceph-mon[33798]: purged_snaps scrub ok 2024-05-06T12:08:18.806 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:18 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-06T12:08:18.806 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:18 smithi190 ceph-mon[33798]: osd.0 [v2:172.21.15.92:6802/3575291733,v1:172.21.15.92:6803/3575291733] boot 2024-05-06T12:08:18.806 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:18 smithi190 ceph-mon[33798]: osdmap e9: 1 total, 1 up, 1 in 2024-05-06T12:08:18.806 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:18 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-06T12:08:18.951 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_3 2024-05-06T12:08:19.615 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:19 smithi092 ceph-mon[26954]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:08:19.615 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:19 smithi092 ceph-mon[26954]: osdmap e10: 1 total, 1 up, 1 in 2024-05-06T12:08:19.615 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:19 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:08:19.800 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:08:19.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:19 smithi190 ceph-mon[33798]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-06T12:08:19.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:19 smithi190 ceph-mon[33798]: osdmap e10: 1 total, 1 up, 1 in 2024-05-06T12:08:19.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:19 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:08:21.439 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:21 smithi092 ceph-mon[26954]: pgmap v36: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-06T12:08:21.439 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:21 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:21.439 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:21 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:21.439 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:21 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:08:21.439 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:21 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:21.439 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:21 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:08:21.439 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:21 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:08:21.439 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:21 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:21.440 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:21 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:08:21.720 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:21 smithi190 ceph-mon[33798]: pgmap v36: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-06T12:08:21.721 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:21 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:21.721 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:21 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:21.721 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:21 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:08:21.721 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:21 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:21.721 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:21 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:08:21.721 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:21 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:08:21.721 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:21 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:21.721 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:21 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:08:22.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.010+0000 7fef15e58700 1 -- 172.21.15.92:0/298808085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef100fdff0 msgr2=0x7fef100fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:08:22.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.010+0000 7fef15e58700 1 --2- 172.21.15.92:0/298808085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef100fdff0 0x7fef100fe460 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7feef8004940 tx=0x7feef8009e30 comp rx=0 tx=0).stop 2024-05-06T12:08:22.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.010+0000 7fef15e58700 1 -- 172.21.15.92:0/298808085 shutdown_connections 2024-05-06T12:08:22.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.010+0000 7fef15e58700 1 --2- 172.21.15.92:0/298808085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef100fdff0 0x7fef100fe460 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:08:22.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.010+0000 7fef15e58700 1 --2- 172.21.15.92:0/298808085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef100fce80 0x7fef100fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:08:22.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.010+0000 7fef15e58700 1 -- 172.21.15.92:0/298808085 >> 172.21.15.92:0/298808085 conn(0x7fef100f8540 msgr2=0x7fef100fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:08:22.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.010+0000 7fef15e58700 1 -- 172.21.15.92:0/298808085 shutdown_connections 2024-05-06T12:08:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.010+0000 7fef15e58700 1 -- 172.21.15.92:0/298808085 wait complete. 2024-05-06T12:08:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.010+0000 7fef15e58700 1 Processor -- start 2024-05-06T12:08:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.011+0000 7fef15e58700 1 -- start start 2024-05-06T12:08:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.011+0000 7fef15e58700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef100fce80 0x7fef10071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:08:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.011+0000 7fef15e58700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef100fdff0 0x7fef100702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:08:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.011+0000 7fef15e58700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef10070860 con 0x7fef100fdff0 2024-05-06T12:08:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.011+0000 7fef15e58700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef100709d0 con 0x7fef100fce80 2024-05-06T12:08:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.011+0000 7fef0effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef100fdff0 0x7fef100702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:08:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.011+0000 7fef0effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef100fdff0 0x7fef100702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55478/0 (socket says 172.21.15.92:55478) 2024-05-06T12:08:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.011+0000 7fef0effd700 1 -- 172.21.15.92:0/3881091243 learned_addr learned my addr 172.21.15.92:0/3881091243 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:08:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.011+0000 7fef0f7fe700 1 --2- 172.21.15.92:0/3881091243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef100fce80 0x7fef10071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:08:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.011+0000 7fef0effd700 1 -- 172.21.15.92:0/3881091243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef100fce80 msgr2=0x7fef10071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:08:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.011+0000 7fef0effd700 1 --2- 172.21.15.92:0/3881091243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef100fce80 0x7fef10071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:08:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.011+0000 7fef0effd700 1 -- 172.21.15.92:0/3881091243 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef8005040 con 0x7fef100fdff0 2024-05-06T12:08:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.011+0000 7fef0effd700 1 --2- 172.21.15.92:0/3881091243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef100fdff0 0x7fef100702b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7feef800b3b0 tx=0x7feef8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:08:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.012+0000 7fef07fff700 1 -- 172.21.15.92:0/3881091243 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feef800e040 con 0x7fef100fdff0 2024-05-06T12:08:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.012+0000 7fef07fff700 1 -- 172.21.15.92:0/3881091243 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feef800bcc0 con 0x7fef100fdff0 2024-05-06T12:08:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.012+0000 7fef15e58700 1 -- 172.21.15.92:0/3881091243 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef10070c50 con 0x7fef100fdff0 2024-05-06T12:08:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.012+0000 7fef07fff700 1 -- 172.21.15.92:0/3881091243 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feef801c630 con 0x7fef100fdff0 2024-05-06T12:08:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.012+0000 7fef15e58700 1 -- 172.21.15.92:0/3881091243 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef10115a10 con 0x7fef100fdff0 2024-05-06T12:08:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.013+0000 7fef15e58700 1 -- 172.21.15.92:0/3881091243 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef1005f5c0 con 0x7fef100fdff0 2024-05-06T12:08:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.013+0000 7fef07fff700 1 -- 172.21.15.92:0/3881091243 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7feef8013980 con 0x7fef100fdff0 2024-05-06T12:08:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.013+0000 7fef07fff700 1 --2- 172.21.15.92:0/3881091243 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7feefc069210 0x7feefc06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:08:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.013+0000 7fef07fff700 1 -- 172.21.15.92:0/3881091243 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1978+0+0 (secure 0 0 0) 0x7feef8087050 con 0x7fef100fdff0 2024-05-06T12:08:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.014+0000 7fef0f7fe700 1 --2- 172.21.15.92:0/3881091243 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7feefc069210 0x7feefc06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:08:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.014+0000 7fef0f7fe700 1 --2- 172.21.15.92:0/3881091243 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7feefc069210 0x7feefc06b6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fef08009fd0 tx=0x7fef0800b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:08:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.018+0000 7fef07fff700 1 -- 172.21.15.92:0/3881091243 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7feef8054200 con 0x7fef100fdff0 2024-05-06T12:08:22.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:22.218+0000 7fef15e58700 1 -- 172.21.15.92:0/3881091243 --> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fef101022d0 con 0x7feefc069210 2024-05-06T12:08:22.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:22 smithi092 ceph-mon[26954]: Detected new or changed devices on smithi092 2024-05-06T12:08:22.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:22 smithi092 ceph-mon[26954]: Adjusting osd_memory_target on smithi092 to 12864M 2024-05-06T12:08:22.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:22 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-06T12:08:22.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:22 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-06T12:08:22.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:22 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:08:22.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:22 smithi190 ceph-mon[33798]: Detected new or changed devices on smithi092 2024-05-06T12:08:22.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:22 smithi190 ceph-mon[33798]: Adjusting osd_memory_target on smithi092 to 12864M 2024-05-06T12:08:22.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:22 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-06T12:08:22.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:22 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-06T12:08:22.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:22 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:08:23.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:23 smithi092 ceph-mon[26954]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-06T12:08:23.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:23 smithi092 ceph-mon[26954]: from='client.14262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:08:23.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:23 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:23.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:23 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:23.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:23 smithi190 ceph-mon[33798]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-06T12:08:23.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:23 smithi190 ceph-mon[33798]: from='client.14262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:08:23.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:23 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:23.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:23 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:25.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:25 smithi092 ceph-mon[26954]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-06T12:08:25.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:25 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/413509185' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21688119-d15f-4b31-8842-090cc539c76d"}]: dispatch 2024-05-06T12:08:25.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:25 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/413509185' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "21688119-d15f-4b31-8842-090cc539c76d"}]': finished 2024-05-06T12:08:25.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:25 smithi092 ceph-mon[26954]: osdmap e11: 2 total, 1 up, 2 in 2024-05-06T12:08:25.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:25 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-06T12:08:25.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:25 smithi190 ceph-mon[33798]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-06T12:08:25.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:25 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/413509185' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21688119-d15f-4b31-8842-090cc539c76d"}]: dispatch 2024-05-06T12:08:25.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:25 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/413509185' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "21688119-d15f-4b31-8842-090cc539c76d"}]': finished 2024-05-06T12:08:25.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:25 smithi190 ceph-mon[33798]: osdmap e11: 2 total, 1 up, 2 in 2024-05-06T12:08:25.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:25 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-06T12:08:26.717 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:26 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/4209562229' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-06T12:08:26.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:26 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/4209562229' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-06T12:08:27.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:27 smithi092 ceph-mon[26954]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-06T12:08:27.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:27 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:27.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:27 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:27.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:27 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:08:27.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:27 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:08:27.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:27 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:27.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:27 smithi190 ceph-mon[33798]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-06T12:08:27.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:27 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:27.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:27 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:27.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:27 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:08:27.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:27 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:08:27.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:27 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:29.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:29 smithi092 ceph-mon[26954]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-06T12:08:29.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:29 smithi190 ceph-mon[33798]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-06T12:08:31.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:31 smithi092 ceph-mon[26954]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-06T12:08:31.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:31 smithi190 ceph-mon[33798]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-06T12:08:32.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:32 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-06T12:08:32.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:32 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:08:32.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:32 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-06T12:08:32.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:32 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:08:33.706 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:33 smithi092 ceph-mon[26954]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-06T12:08:33.707 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:33 smithi092 ceph-mon[26954]: Deploying daemon osd.1 on smithi092 2024-05-06T12:08:33.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:33 smithi190 ceph-mon[33798]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-06T12:08:33.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:33 smithi190 ceph-mon[33798]: Deploying daemon osd.1 on smithi092 2024-05-06T12:08:34.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:34 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:08:34.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:34 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:08:35.698 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:35 smithi092 ceph-mon[26954]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-06T12:08:35.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:35 smithi190 ceph-mon[33798]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-06T12:08:37.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:37 smithi092 ceph-mon[26954]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-06T12:08:37.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:37 smithi190 ceph-mon[33798]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-06T12:08:38.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:38 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:38.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:38 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:38.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:38 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:08:38.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:38 smithi190 ceph-mon[33798]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-06T12:08:38.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:38 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:38.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:38 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:38.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:38 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:08:38.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:38 smithi092 ceph-mon[26954]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-06T12:08:40.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:40 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:40.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:40 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:40.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:40 smithi092 ceph-mon[26954]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-06T12:08:40.787 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 1 on host 'smithi092' 2024-05-06T12:08:40.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:40.784+0000 7fef07fff700 1 -- 172.21.15.92:0/3881091243 <== mgr.14210 v2:172.21.15.92:6800/3045905943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fef101022d0 con 0x7feefc069210 2024-05-06T12:08:40.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:40.786+0000 7fef15e58700 1 -- 172.21.15.92:0/3881091243 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7feefc069210 msgr2=0x7feefc06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:08:40.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:40.786+0000 7fef15e58700 1 --2- 172.21.15.92:0/3881091243 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7feefc069210 0x7feefc06b6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fef08009fd0 tx=0x7fef0800b040 comp rx=0 tx=0).stop 2024-05-06T12:08:40.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:40.786+0000 7fef15e58700 1 -- 172.21.15.92:0/3881091243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef100fdff0 msgr2=0x7fef100702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:08:40.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:40.786+0000 7fef15e58700 1 --2- 172.21.15.92:0/3881091243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef100fdff0 0x7fef100702b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7feef800b3b0 tx=0x7feef8009e30 comp rx=0 tx=0).stop 2024-05-06T12:08:40.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:40.786+0000 7fef15e58700 1 -- 172.21.15.92:0/3881091243 shutdown_connections 2024-05-06T12:08:40.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:40.786+0000 7fef15e58700 1 --2- 172.21.15.92:0/3881091243 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7feefc069210 0x7feefc06b6c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:08:40.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:40.786+0000 7fef15e58700 1 --2- 172.21.15.92:0/3881091243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef100fce80 0x7fef10071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:08:40.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:40.786+0000 7fef15e58700 1 --2- 172.21.15.92:0/3881091243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef100fdff0 0x7fef100702b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:08:40.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:40.786+0000 7fef15e58700 1 -- 172.21.15.92:0/3881091243 >> 172.21.15.92:0/3881091243 conn(0x7fef100f8540 msgr2=0x7fef100f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:08:40.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:40.787+0000 7fef15e58700 1 -- 172.21.15.92:0/3881091243 shutdown_connections 2024-05-06T12:08:40.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:40.787+0000 7fef15e58700 1 -- 172.21.15.92:0/3881091243 wait complete. 2024-05-06T12:08:40.854 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:40 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:41.205 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:40 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:41.205 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:40 smithi190 ceph-mon[33798]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-06T12:08:41.412 DEBUG:teuthology.orchestra.run.smithi092:osd.1> sudo journalctl -f -n 0 -u ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@osd.1.service 2024-05-06T12:08:41.415 INFO:tasks.cephadm:Deploying osd.2 on smithi092 with /dev/vg_nvme/lv_2... 2024-05-06T12:08:41.415 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-06T12:08:41.722 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:08:41 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1[53068]: 2024-05-06T12:08:41.604+0000 7f94bc9d8540 -1 osd.1 0 log_to_monitors true 2024-05-06T12:08:42.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:41 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:42.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:41 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:42.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:41 smithi190 ceph-mon[33798]: from='osd.1 [v2:172.21.15.92:6810/334092652,v1:172.21.15.92:6811/334092652]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-06T12:08:42.193 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:08:42.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:41 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:42.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:41 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:42.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:41 smithi092 ceph-mon[26954]: from='osd.1 [v2:172.21.15.92:6810/334092652,v1:172.21.15.92:6811/334092652]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-06T12:08:43.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:42 smithi190 ceph-mon[33798]: from='osd.1 [v2:172.21.15.92:6810/334092652,v1:172.21.15.92:6811/334092652]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-06T12:08:43.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:42 smithi190 ceph-mon[33798]: osdmap e12: 2 total, 1 up, 2 in 2024-05-06T12:08:43.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:42 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-06T12:08:43.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:42 smithi190 ceph-mon[33798]: from='osd.1 [v2:172.21.15.92:6810/334092652,v1:172.21.15.92:6811/334092652]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-06T12:08:43.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:42 smithi190 ceph-mon[33798]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-06T12:08:43.174 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:42 smithi092 ceph-mon[26954]: from='osd.1 [v2:172.21.15.92:6810/334092652,v1:172.21.15.92:6811/334092652]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-06T12:08:43.175 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:42 smithi092 ceph-mon[26954]: osdmap e12: 2 total, 1 up, 2 in 2024-05-06T12:08:43.175 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:42 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-06T12:08:43.175 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:42 smithi092 ceph-mon[26954]: from='osd.1 [v2:172.21.15.92:6810/334092652,v1:172.21.15.92:6811/334092652]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-06T12:08:43.175 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:42 smithi092 ceph-mon[26954]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-06T12:08:43.175 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:08:42 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1[53068]: 2024-05-06T12:08:42.789+0000 7f94b31a9700 -1 osd.1 0 waiting for initial osdmap 2024-05-06T12:08:43.175 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:08:42 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1[53068]: 2024-05-06T12:08:42.793+0000 7f94affa8700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-06T12:08:44.052 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:43 smithi092 ceph-mon[26954]: from='osd.1 [v2:172.21.15.92:6810/334092652,v1:172.21.15.92:6811/334092652]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-06T12:08:44.053 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:43 smithi092 ceph-mon[26954]: osdmap e13: 2 total, 1 up, 2 in 2024-05-06T12:08:44.053 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:43 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-06T12:08:44.053 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:43 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-06T12:08:44.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:43 smithi190 ceph-mon[33798]: from='osd.1 [v2:172.21.15.92:6810/334092652,v1:172.21.15.92:6811/334092652]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-06T12:08:44.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:43 smithi190 ceph-mon[33798]: osdmap e13: 2 total, 1 up, 2 in 2024-05-06T12:08:44.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:43 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-06T12:08:44.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:43 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-06T12:08:44.764 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-06T12:08:44.765 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-06T12:08:44.765 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-05-06T12:08:44.765 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-05-06T12:08:44.765 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131245 s, 799 MB/s 2024-05-06T12:08:44.765 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-05-06T12:08:44.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:44 smithi092 ceph-mon[26954]: purged_snaps scrub starts 2024-05-06T12:08:44.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:44 smithi092 ceph-mon[26954]: purged_snaps scrub ok 2024-05-06T12:08:44.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:44 smithi092 ceph-mon[26954]: osd.1 [v2:172.21.15.92:6810/334092652,v1:172.21.15.92:6811/334092652] boot 2024-05-06T12:08:44.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:44 smithi092 ceph-mon[26954]: osdmap e14: 2 total, 2 up, 2 in 2024-05-06T12:08:44.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:44 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-06T12:08:44.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:44 smithi092 ceph-mon[26954]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-06T12:08:45.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:44 smithi190 ceph-mon[33798]: purged_snaps scrub starts 2024-05-06T12:08:45.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:44 smithi190 ceph-mon[33798]: purged_snaps scrub ok 2024-05-06T12:08:45.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:44 smithi190 ceph-mon[33798]: osd.1 [v2:172.21.15.92:6810/334092652,v1:172.21.15.92:6811/334092652] boot 2024-05-06T12:08:45.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:44 smithi190 ceph-mon[33798]: osdmap e14: 2 total, 2 up, 2 in 2024-05-06T12:08:45.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:44 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-06T12:08:45.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:44 smithi190 ceph-mon[33798]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-06T12:08:45.660 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_2 2024-05-06T12:08:45.946 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:45 smithi092 ceph-mon[26954]: osdmap e15: 2 total, 2 up, 2 in 2024-05-06T12:08:46.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:45 smithi190 ceph-mon[33798]: osdmap e15: 2 total, 2 up, 2 in 2024-05-06T12:08:46.325 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:08:47.721 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:47 smithi092 ceph-mon[26954]: pgmap v54: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-06T12:08:47.721 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:47 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:47.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:47 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:47.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:47 smithi190 ceph-mon[33798]: pgmap v54: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-06T12:08:47.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:47 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:47.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:47 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:48.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.046+0000 7feededc2700 1 -- 172.21.15.92:0/1801700295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feed80fbcc0 msgr2=0x7feed80fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:08:48.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.046+0000 7feededc2700 1 --2- 172.21.15.92:0/1801700295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feed80fbcc0 0x7feed80fc0d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7feecc004940 tx=0x7feecc009e30 comp rx=0 tx=0).stop 2024-05-06T12:08:48.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.046+0000 7feededc2700 1 -- 172.21.15.92:0/1801700295 shutdown_connections 2024-05-06T12:08:48.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.046+0000 7feededc2700 1 --2- 172.21.15.92:0/1801700295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feed80fcf70 0x7feed80fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:08:48.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.046+0000 7feededc2700 1 --2- 172.21.15.92:0/1801700295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feed80fbcc0 0x7feed80fc0d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:08:48.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.046+0000 7feededc2700 1 -- 172.21.15.92:0/1801700295 >> 172.21.15.92:0/1801700295 conn(0x7feed80f7420 msgr2=0x7feed80f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:08:48.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.046+0000 7feededc2700 1 -- 172.21.15.92:0/1801700295 shutdown_connections 2024-05-06T12:08:48.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.046+0000 7feededc2700 1 -- 172.21.15.92:0/1801700295 wait complete. 2024-05-06T12:08:48.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.047+0000 7feededc2700 1 Processor -- start 2024-05-06T12:08:48.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.047+0000 7feededc2700 1 -- start start 2024-05-06T12:08:48.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.047+0000 7feededc2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feed80fbcc0 0x7feed810c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:08:48.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.047+0000 7feededc2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feed80fcf70 0x7feed810cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:08:48.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.047+0000 7feededc2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed810d480 con 0x7feed80fbcc0 2024-05-06T12:08:48.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.047+0000 7feededc2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed8111e20 con 0x7feed80fcf70 2024-05-06T12:08:48.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.047+0000 7feedcb5e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feed80fbcc0 0x7feed810c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:08:48.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.047+0000 7feedcb5e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feed80fbcc0 0x7feed810c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33570/0 (socket says 172.21.15.92:33570) 2024-05-06T12:08:48.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.047+0000 7feedcb5e700 1 -- 172.21.15.92:0/3416008406 learned_addr learned my addr 172.21.15.92:0/3416008406 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:08:48.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.048+0000 7feedcb5e700 1 -- 172.21.15.92:0/3416008406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feed80fcf70 msgr2=0x7feed810cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:08:48.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.048+0000 7feedcb5e700 1 --2- 172.21.15.92:0/3416008406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feed80fcf70 0x7feed810cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:08:48.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.048+0000 7feedcb5e700 1 -- 172.21.15.92:0/3416008406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feecc005040 con 0x7feed80fbcc0 2024-05-06T12:08:48.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.048+0000 7feedcb5e700 1 --2- 172.21.15.92:0/3416008406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feed80fbcc0 0x7feed810c8b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7feecc009fd0 tx=0x7feecc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:08:48.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.048+0000 7feed57fa700 1 -- 172.21.15.92:0/3416008406 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feecc00e040 con 0x7feed80fbcc0 2024-05-06T12:08:48.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.048+0000 7feed57fa700 1 -- 172.21.15.92:0/3416008406 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feecc00bcc0 con 0x7feed80fbcc0 2024-05-06T12:08:48.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.048+0000 7feededc2700 1 -- 172.21.15.92:0/3416008406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed8112020 con 0x7feed80fbcc0 2024-05-06T12:08:48.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.048+0000 7feed57fa700 1 -- 172.21.15.92:0/3416008406 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feecc01c630 con 0x7feed80fbcc0 2024-05-06T12:08:48.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.048+0000 7feededc2700 1 -- 172.21.15.92:0/3416008406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed8112590 con 0x7feed80fbcc0 2024-05-06T12:08:48.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.049+0000 7feededc2700 1 -- 172.21.15.92:0/3416008406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed804c710 con 0x7feed80fbcc0 2024-05-06T12:08:48.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.049+0000 7feed57fa700 1 -- 172.21.15.92:0/3416008406 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7feecc013980 con 0x7feed80fbcc0 2024-05-06T12:08:48.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.050+0000 7feed57fa700 1 --2- 172.21.15.92:0/3416008406 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7feec8069210 0x7feec806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:08:48.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.050+0000 7feed57fa700 1 -- 172.21.15.92:0/3416008406 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2410+0+0 (secure 0 0 0) 0x7feecc0877a0 con 0x7feed80fbcc0 2024-05-06T12:08:48.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.050+0000 7feed7fff700 1 --2- 172.21.15.92:0/3416008406 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7feec8069210 0x7feec806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:08:48.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.050+0000 7feed7fff700 1 --2- 172.21.15.92:0/3416008406 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7feec8069210 0x7feec806b6c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7feec4006fd0 tx=0x7feec4006db0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:08:48.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.054+0000 7feed57fa700 1 -- 172.21.15.92:0/3416008406 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7feecc0581b0 con 0x7feed80fbcc0 2024-05-06T12:08:48.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:08:48.269+0000 7feededc2700 1 -- 172.21.15.92:0/3416008406 --> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7feed81012b0 con 0x7feec8069210 2024-05-06T12:08:48.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:48 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-06T12:08:48.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:48 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-06T12:08:48.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:48 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:08:48.805 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:48 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-06T12:08:48.805 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:48 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-06T12:08:48.805 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:48 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:08:49.466 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:49 smithi092 ceph-mon[26954]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-06T12:08:49.467 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:49 smithi092 ceph-mon[26954]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:08:49.467 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:49 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:08:49.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:49 smithi190 ceph-mon[33798]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-06T12:08:49.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:49 smithi190 ceph-mon[33798]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:08:49.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:49 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:08:51.644 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:51 smithi092 ceph-mon[26954]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-06T12:08:51.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:51 smithi190 ceph-mon[33798]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-06T12:08:52.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:52 smithi092 ceph-mon[26954]: Detected new or changed devices on smithi092 2024-05-06T12:08:52.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:52 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:52.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:52 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:52.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:52 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:08:52.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:52 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:08:52.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:52 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:52.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:52 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:08:52.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:52 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:08:52.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:52 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:52.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:52 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:08:52.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:52 smithi190 ceph-mon[33798]: Detected new or changed devices on smithi092 2024-05-06T12:08:52.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:52 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:52.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:52 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:52.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:52 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:08:52.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:52 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:08:52.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:52 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:52.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:52 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:08:52.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:52 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:08:52.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:52 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:52.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:52 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:08:53.443 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:53 smithi092 ceph-mon[26954]: Adjusting osd_memory_target on smithi092 to 6432M 2024-05-06T12:08:53.443 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:53 smithi092 ceph-mon[26954]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-06T12:08:53.443 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:53 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/3414711449' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74d16dfb-b118-432d-827a-59706d69c393"}]: dispatch 2024-05-06T12:08:53.443 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:53 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/3414711449' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "74d16dfb-b118-432d-827a-59706d69c393"}]': finished 2024-05-06T12:08:53.443 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:53 smithi092 ceph-mon[26954]: osdmap e16: 3 total, 2 up, 3 in 2024-05-06T12:08:53.443 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:53 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-06T12:08:53.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:53 smithi190 ceph-mon[33798]: Adjusting osd_memory_target on smithi092 to 6432M 2024-05-06T12:08:53.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:53 smithi190 ceph-mon[33798]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-06T12:08:53.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:53 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/3414711449' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74d16dfb-b118-432d-827a-59706d69c393"}]: dispatch 2024-05-06T12:08:53.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:53 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/3414711449' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "74d16dfb-b118-432d-827a-59706d69c393"}]': finished 2024-05-06T12:08:53.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:53 smithi190 ceph-mon[33798]: osdmap e16: 3 total, 2 up, 3 in 2024-05-06T12:08:53.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:53 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-06T12:08:54.718 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:54 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/565372286' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-06T12:08:54.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:54 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/565372286' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-06T12:08:55.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:55 smithi092 ceph-mon[26954]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-06T12:08:55.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:55 smithi190 ceph-mon[33798]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-06T12:08:57.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:56 smithi092 ceph-mon[26954]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-06T12:08:57.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:56 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:57.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:56 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:57.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:56 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:08:57.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:56 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:08:57.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:56 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:57.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:56 smithi190 ceph-mon[33798]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-06T12:08:57.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:56 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:57.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:56 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:57.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:56 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:08:57.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:56 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:08:57.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:56 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:08:58.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:08:58 smithi092 ceph-mon[26954]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-06T12:08:59.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:08:58 smithi190 ceph-mon[33798]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-06T12:09:01.088 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:00 smithi092 ceph-mon[26954]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-06T12:09:01.088 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:00 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-06T12:09:01.088 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:00 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:09:01.088 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:00 smithi092 ceph-mon[26954]: Deploying daemon osd.2 on smithi092 2024-05-06T12:09:01.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:00 smithi190 ceph-mon[33798]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-06T12:09:01.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:00 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-06T12:09:01.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:00 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:09:01.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:00 smithi190 ceph-mon[33798]: Deploying daemon osd.2 on smithi092 2024-05-06T12:09:03.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:02 smithi092 ceph-mon[26954]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-06T12:09:03.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:02 smithi190 ceph-mon[33798]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-06T12:09:04.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:03 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:09:04.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:03 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:09:05.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:04 smithi092 ceph-mon[26954]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-06T12:09:05.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:04 smithi190 ceph-mon[33798]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-06T12:09:06.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:06 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:06.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:06 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:06.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:06 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:09:06.651 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:06 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:06.652 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:06 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:06.652 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:06 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:09:07.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:07 smithi092 ceph-mon[26954]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-06T12:09:07.499 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:07 smithi190 ceph-mon[33798]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-06T12:09:08.785 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 2 on host 'smithi092' 2024-05-06T12:09:08.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:09:08.781+0000 7feed57fa700 1 -- 172.21.15.92:0/3416008406 <== mgr.14210 v2:172.21.15.92:6800/3045905943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7feed81012b0 con 0x7feec8069210 2024-05-06T12:09:08.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:09:08.784+0000 7feededc2700 1 -- 172.21.15.92:0/3416008406 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7feec8069210 msgr2=0x7feec806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:09:08.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:09:08.784+0000 7feededc2700 1 --2- 172.21.15.92:0/3416008406 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7feec8069210 0x7feec806b6c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7feec4006fd0 tx=0x7feec4006db0 comp rx=0 tx=0).stop 2024-05-06T12:09:08.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:09:08.784+0000 7feededc2700 1 -- 172.21.15.92:0/3416008406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feed80fbcc0 msgr2=0x7feed810c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:09:08.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:09:08.784+0000 7feededc2700 1 --2- 172.21.15.92:0/3416008406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feed80fbcc0 0x7feed810c8b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7feecc009fd0 tx=0x7feecc009e30 comp rx=0 tx=0).stop 2024-05-06T12:09:08.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:09:08.784+0000 7feededc2700 1 -- 172.21.15.92:0/3416008406 shutdown_connections 2024-05-06T12:09:08.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:09:08.784+0000 7feededc2700 1 --2- 172.21.15.92:0/3416008406 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7feec8069210 0x7feec806b6c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:09:08.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:09:08.784+0000 7feededc2700 1 --2- 172.21.15.92:0/3416008406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feed80fbcc0 0x7feed810c8b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:09:08.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:09:08.784+0000 7feededc2700 1 --2- 172.21.15.92:0/3416008406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feed80fcf70 0x7feed810cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:09:08.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:09:08.784+0000 7feededc2700 1 -- 172.21.15.92:0/3416008406 >> 172.21.15.92:0/3416008406 conn(0x7feed80f7420 msgr2=0x7feed80f8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:09:08.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:09:08.784+0000 7feededc2700 1 -- 172.21.15.92:0/3416008406 shutdown_connections 2024-05-06T12:09:08.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:09:08.784+0000 7feededc2700 1 -- 172.21.15.92:0/3416008406 wait complete. 2024-05-06T12:09:09.367 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:09 smithi092 ceph-mon[26954]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-06T12:09:09.367 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:09 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:09.367 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:09 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:09.367 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:09 smithi092 ceph-mon[26954]: from='osd.2 [v2:172.21.15.92:6818/3428814547,v1:172.21.15.92:6819/3428814547]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-06T12:09:09.393 DEBUG:teuthology.orchestra.run.smithi092:osd.2> sudo journalctl -f -n 0 -u ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@osd.2.service 2024-05-06T12:09:09.396 INFO:tasks.cephadm:Deploying osd.3 on smithi190 with /dev/vg_nvme/lv_4... 2024-05-06T12:09:09.396 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-06T12:09:09.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:09 smithi190 ceph-mon[33798]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-06T12:09:09.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:09 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:09.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:09 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:09.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:09 smithi190 ceph-mon[33798]: from='osd.2 [v2:172.21.15.92:6818/3428814547,v1:172.21.15.92:6819/3428814547]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-06T12:09:09.649 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi190/config 2024-05-06T12:09:10.976 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:10 smithi190 ceph-mon[33798]: from='osd.2 [v2:172.21.15.92:6818/3428814547,v1:172.21.15.92:6819/3428814547]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-06T12:09:10.976 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:10 smithi190 ceph-mon[33798]: osdmap e17: 3 total, 2 up, 3 in 2024-05-06T12:09:10.976 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:10 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-06T12:09:10.976 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:10 smithi190 ceph-mon[33798]: from='osd.2 [v2:172.21.15.92:6818/3428814547,v1:172.21.15.92:6819/3428814547]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-06T12:09:10.976 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:10 smithi190 ceph-mon[33798]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-06T12:09:11.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:10 smithi092 ceph-mon[26954]: from='osd.2 [v2:172.21.15.92:6818/3428814547,v1:172.21.15.92:6819/3428814547]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-06T12:09:11.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:10 smithi092 ceph-mon[26954]: osdmap e17: 3 total, 2 up, 3 in 2024-05-06T12:09:11.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:10 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-06T12:09:11.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:10 smithi092 ceph-mon[26954]: from='osd.2 [v2:172.21.15.92:6818/3428814547,v1:172.21.15.92:6819/3428814547]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-06T12:09:11.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:10 smithi092 ceph-mon[26954]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-06T12:09:11.223 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:09:10 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2[60584]: 2024-05-06T12:09:10.771+0000 7fdc031dd700 -1 osd.2 0 waiting for initial osdmap 2024-05-06T12:09:11.223 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:09:10 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2[60584]: 2024-05-06T12:09:10.777+0000 7fdbfc7d5700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-06T12:09:11.512 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-06T12:09:11.512 INFO:teuthology.orchestra.run.smithi190.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-06T12:09:11.512 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10+0 records in 2024-05-06T12:09:11.512 INFO:teuthology.orchestra.run.smithi190.stderr:10+0 records out 2024-05-06T12:09:11.512 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113245 s, 926 MB/s 2024-05-06T12:09:11.512 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping successful for: 2024-05-06T12:09:12.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:11 smithi190 ceph-mon[33798]: from='osd.2 [v2:172.21.15.92:6818/3428814547,v1:172.21.15.92:6819/3428814547]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-06T12:09:12.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:11 smithi190 ceph-mon[33798]: osdmap e18: 3 total, 2 up, 3 in 2024-05-06T12:09:12.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:11 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-06T12:09:12.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:11 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-06T12:09:12.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:11 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-06T12:09:12.152 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi190:vg_nvme/lv_4 2024-05-06T12:09:12.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:11 smithi092 ceph-mon[26954]: from='osd.2 [v2:172.21.15.92:6818/3428814547,v1:172.21.15.92:6819/3428814547]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-06T12:09:12.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:11 smithi092 ceph-mon[26954]: osdmap e18: 3 total, 2 up, 3 in 2024-05-06T12:09:12.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:11 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-06T12:09:12.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:11 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-06T12:09:12.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:11 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-06T12:09:12.398 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi190/config 2024-05-06T12:09:12.865 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:12 smithi092 ceph-mon[26954]: purged_snaps scrub starts 2024-05-06T12:09:12.865 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:12 smithi092 ceph-mon[26954]: purged_snaps scrub ok 2024-05-06T12:09:12.865 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:12 smithi092 ceph-mon[26954]: osd.2 [v2:172.21.15.92:6818/3428814547,v1:172.21.15.92:6819/3428814547] boot 2024-05-06T12:09:12.865 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:12 smithi092 ceph-mon[26954]: osdmap e19: 3 total, 3 up, 3 in 2024-05-06T12:09:12.866 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:12 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-06T12:09:12.866 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:12 smithi092 ceph-mon[26954]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-06T12:09:12.866 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:12 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-05-06T12:09:12.866 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:12 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:12.867 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:12 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:13.042 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:12 smithi190 ceph-mon[33798]: purged_snaps scrub starts 2024-05-06T12:09:13.042 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:12 smithi190 ceph-mon[33798]: purged_snaps scrub ok 2024-05-06T12:09:13.042 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:12 smithi190 ceph-mon[33798]: osd.2 [v2:172.21.15.92:6818/3428814547,v1:172.21.15.92:6819/3428814547] boot 2024-05-06T12:09:13.043 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:12 smithi190 ceph-mon[33798]: osdmap e19: 3 total, 3 up, 3 in 2024-05-06T12:09:13.043 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:12 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-06T12:09:13.043 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:12 smithi190 ceph-mon[33798]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-06T12:09:13.043 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:12 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-05-06T12:09:13.043 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:12 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:13.043 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:12 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:13.813 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.810+0000 7ff667a0a700 1 -- 172.21.15.190:0/1972631147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6600fbcc0 msgr2=0x7ff6600fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:09:13.813 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.810+0000 7ff667a0a700 1 --2- 172.21.15.190:0/1972631147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6600fbcc0 0x7ff6600fc0d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff65c004940 tx=0x7ff65c009e30 comp rx=0 tx=0).stop 2024-05-06T12:09:13.813 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.810+0000 7ff667a0a700 1 -- 172.21.15.190:0/1972631147 shutdown_connections 2024-05-06T12:09:13.813 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.810+0000 7ff667a0a700 1 --2- 172.21.15.190:0/1972631147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6600fcf70 0x7ff6600fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:09:13.813 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.810+0000 7ff667a0a700 1 --2- 172.21.15.190:0/1972631147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6600fbcc0 0x7ff6600fc0d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:09:13.813 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.810+0000 7ff667a0a700 1 -- 172.21.15.190:0/1972631147 >> 172.21.15.190:0/1972631147 conn(0x7ff6600f7440 msgr2=0x7ff6600f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:09:13.814 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.810+0000 7ff667a0a700 1 -- 172.21.15.190:0/1972631147 shutdown_connections 2024-05-06T12:09:13.814 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.810+0000 7ff667a0a700 1 -- 172.21.15.190:0/1972631147 wait complete. 2024-05-06T12:09:13.815 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.811+0000 7ff667a0a700 1 Processor -- start 2024-05-06T12:09:13.815 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.811+0000 7ff667a0a700 1 -- start start 2024-05-06T12:09:13.815 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.811+0000 7ff667a0a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6600fbcc0 0x7ff66010d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:09:13.815 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.811+0000 7ff667a0a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6600fcf70 0x7ff66010ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:09:13.815 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.811+0000 7ff667a0a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66010e4d0 con 0x7ff6600fcf70 2024-05-06T12:09:13.815 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.811+0000 7ff667a0a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66010e640 con 0x7ff6600fbcc0 2024-05-06T12:09:13.815 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.811+0000 7ff6657a6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6600fbcc0 0x7ff66010d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:09:13.815 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.811+0000 7ff6657a6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6600fbcc0 0x7ff66010d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:46696/0 (socket says 172.21.15.190:46696) 2024-05-06T12:09:13.815 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.811+0000 7ff6657a6700 1 -- 172.21.15.190:0/143480088 learned_addr learned my addr 172.21.15.190:0/143480088 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-06T12:09:13.816 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.811+0000 7ff664fa5700 1 --2- 172.21.15.190:0/143480088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6600fcf70 0x7ff66010ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:09:13.816 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.811+0000 7ff6657a6700 1 -- 172.21.15.190:0/143480088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6600fcf70 msgr2=0x7ff66010ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:09:13.816 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.811+0000 7ff6657a6700 1 --2- 172.21.15.190:0/143480088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6600fcf70 0x7ff66010ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:09:13.816 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.811+0000 7ff6657a6700 1 -- 172.21.15.190:0/143480088 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff65c005040 con 0x7ff6600fbcc0 2024-05-06T12:09:13.816 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.812+0000 7ff6657a6700 1 --2- 172.21.15.190:0/143480088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6600fbcc0 0x7ff66010d990 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff65c009150 tx=0x7ff65c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:09:13.816 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.812+0000 7ff655ffb700 1 -- 172.21.15.190:0/143480088 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff65c00e040 con 0x7ff6600fbcc0 2024-05-06T12:09:13.816 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.812+0000 7ff655ffb700 1 -- 172.21.15.190:0/143480088 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff65c00bcc0 con 0x7ff6600fbcc0 2024-05-06T12:09:13.816 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.812+0000 7ff667a0a700 1 -- 172.21.15.190:0/143480088 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff660112ff0 con 0x7ff6600fbcc0 2024-05-06T12:09:13.816 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.812+0000 7ff655ffb700 1 -- 172.21.15.190:0/143480088 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff65c01c630 con 0x7ff6600fbcc0 2024-05-06T12:09:13.816 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.812+0000 7ff667a0a700 1 -- 172.21.15.190:0/143480088 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff660113520 con 0x7ff6600fbcc0 2024-05-06T12:09:13.817 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.813+0000 7ff667a0a700 1 -- 172.21.15.190:0/143480088 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff660101b00 con 0x7ff6600fbcc0 2024-05-06T12:09:13.817 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.813+0000 7ff655ffb700 1 -- 172.21.15.190:0/143480088 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7ff65c013420 con 0x7ff6600fbcc0 2024-05-06T12:09:13.817 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.814+0000 7ff655ffb700 1 --2- 172.21.15.190:0/143480088 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7ff64c069210 0x7ff64c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:09:13.817 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.814+0000 7ff655ffb700 1 -- 172.21.15.190:0/143480088 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3228+0+0 (secure 0 0 0) 0x7ff65c087580 con 0x7ff6600fbcc0 2024-05-06T12:09:13.817 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.814+0000 7ff664fa5700 1 --2- 172.21.15.190:0/143480088 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7ff64c069210 0x7ff64c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:09:13.818 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.815+0000 7ff664fa5700 1 --2- 172.21.15.190:0/143480088 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7ff64c069210 0x7ff64c06b6c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff650007fd0 tx=0x7ff650009040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:09:13.821 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:13.818+0000 7ff655ffb700 1 -- 172.21.15.190:0/143480088 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff65c057ce0 con 0x7ff6600fbcc0 2024-05-06T12:09:14.011 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:14.008+0000 7ff667a0a700 1 -- 172.21.15.190:0/143480088 --> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7ff66002b340 con 0x7ff64c069210 2024-05-06T12:09:14.011 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:13 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-05-06T12:09:14.011 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:13 smithi190 ceph-mon[33798]: osdmap e20: 3 total, 3 up, 3 in 2024-05-06T12:09:14.011 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:13 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-06T12:09:14.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:13 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-05-06T12:09:14.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:13 smithi092 ceph-mon[26954]: osdmap e20: 3 total, 3 up, 3 in 2024-05-06T12:09:14.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:13 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-06T12:09:14.902 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:09:14 smithi092 sudo[64459]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-06T12:09:14.902 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:09:14 smithi092 sudo[64459]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-06T12:09:14.902 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:09:14 smithi092 sudo[64459]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-06T12:09:14.902 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:09:14 smithi092 sudo[64459]: pam_unix(sudo:session): session closed for user root 2024-05-06T12:09:14.902 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:14 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-06T12:09:14.902 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:14 smithi092 ceph-mon[26954]: osdmap e21: 3 total, 3 up, 3 in 2024-05-06T12:09:14.903 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:14 smithi092 ceph-mon[26954]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-05-06T12:09:14.903 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:14 smithi092 ceph-mon[26954]: from='client.24133 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:09:14.903 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:14 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-06T12:09:14.903 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:14 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-06T12:09:14.903 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:14 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:09:15.043 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:14 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-06T12:09:15.043 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:14 smithi190 ceph-mon[33798]: osdmap e21: 3 total, 3 up, 3 in 2024-05-06T12:09:15.044 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:14 smithi190 ceph-mon[33798]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-05-06T12:09:15.044 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:14 smithi190 ceph-mon[33798]: from='client.24133 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:09:15.044 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:14 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-06T12:09:15.044 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:14 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-06T12:09:15.044 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:14 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:09:15.172 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:09:14 smithi092 sudo[64720]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-06T12:09:15.172 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:09:14 smithi092 sudo[64720]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-06T12:09:15.172 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:09:14 smithi092 sudo[64720]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-06T12:09:15.172 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:09:14 smithi092 sudo[64720]: pam_unix(sudo:session): session closed for user root 2024-05-06T12:09:15.430 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:09:15 smithi092 sudo[64726]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-06T12:09:15.430 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:09:15 smithi092 sudo[64726]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-06T12:09:15.430 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:09:15 smithi092 sudo[64726]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-06T12:09:15.430 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:09:15 smithi092 sudo[64726]: pam_unix(sudo:session): session closed for user root 2024-05-06T12:09:15.685 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:09:15 smithi092 sudo[64730]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-06T12:09:15.685 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:09:15 smithi092 sudo[64730]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-06T12:09:15.685 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:09:15 smithi092 sudo[64730]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-06T12:09:15.685 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:09:15 smithi092 sudo[64730]: pam_unix(sudo:session): session closed for user root 2024-05-06T12:09:15.957 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:09:15 smithi092 sudo[64831]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-06T12:09:15.957 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:09:15 smithi092 sudo[64831]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-06T12:09:15.957 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:09:15 smithi092 sudo[64831]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-06T12:09:15.957 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:09:15 smithi092 sudo[64831]: pam_unix(sudo:session): session closed for user root 2024-05-06T12:09:15.957 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:15 smithi092 ceph-mon[26954]: osdmap e22: 3 total, 3 up, 3 in 2024-05-06T12:09:15.957 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:15 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:15.957 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:15 smithi092 ceph-mon[26954]: Detected new or changed devices on smithi092 2024-05-06T12:09:15.958 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:15 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:15.958 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:15 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:15.958 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:15 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:09:15.958 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:15 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:09:15.958 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:15 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:09:15.958 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:15 smithi092 ceph-mon[26954]: Adjusting osd_memory_target on smithi092 to 4288M 2024-05-06T12:09:15.958 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:15 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:15.958 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:15 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:09:15.958 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:15 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:09:15.958 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:15 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:15.958 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:15 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:09:16.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:15 smithi190 ceph-mon[33798]: osdmap e22: 3 total, 3 up, 3 in 2024-05-06T12:09:16.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:15 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:16.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:15 smithi190 ceph-mon[33798]: Detected new or changed devices on smithi092 2024-05-06T12:09:16.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:15 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:16.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:15 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:16.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:15 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:09:16.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:15 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:09:16.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:15 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:09:16.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:15 smithi190 ceph-mon[33798]: Adjusting osd_memory_target on smithi092 to 4288M 2024-05-06T12:09:16.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:15 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:16.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:15 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:09:16.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:15 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:09:16.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:15 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:16.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:15 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:09:16.217 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:16 smithi092 sudo[64926]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-06T12:09:16.218 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:16 smithi092 sudo[64926]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-06T12:09:16.218 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:16 smithi092 sudo[64926]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-06T12:09:16.218 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:16 smithi092 sudo[64926]: pam_unix(sudo:session): session closed for user root 2024-05-06T12:09:16.218 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:09:15 smithi092 sudo[64906]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-06T12:09:16.218 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:09:15 smithi092 sudo[64906]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-06T12:09:16.218 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:09:15 smithi092 sudo[64906]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-06T12:09:16.218 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:09:15 smithi092 sudo[64906]: pam_unix(sudo:session): session closed for user root 2024-05-06T12:09:16.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:16 smithi092 sudo[64943]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-06T12:09:16.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:16 smithi092 sudo[64943]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-06T12:09:16.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:16 smithi092 sudo[64943]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-06T12:09:16.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:16 smithi092 sudo[64943]: pam_unix(sudo:session): session closed for user root 2024-05-06T12:09:16.966 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:16 smithi190 sudo[37285]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-06T12:09:16.966 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:16 smithi190 sudo[37285]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-06T12:09:16.966 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:16 smithi190 sudo[37285]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-06T12:09:16.966 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:16 smithi190 sudo[37285]: pam_unix(sudo:session): session closed for user root 2024-05-06T12:09:16.980 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:16.977+0000 7ff655ffb700 1 -- 172.21.15.190:0/143480088 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7ff65c054620 con 0x7ff6600fbcc0 2024-05-06T12:09:17.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:16 smithi092 ceph-mon[26954]: pgmap v76: 1 pgs: 1 active+clean; 449 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-05-06T12:09:17.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:16 smithi092 ceph-mon[26954]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-06T12:09:17.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:16 smithi092 ceph-mon[26954]: from='client.? 172.21.15.190:0/417146884' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a9aa5ed-2502-4c73-a8ac-cb7d0259500b"}]: dispatch 2024-05-06T12:09:17.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:16 smithi092 ceph-mon[26954]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-06T12:09:17.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:16 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi092"}]: dispatch 2024-05-06T12:09:17.226 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:16 smithi092 ceph-mon[26954]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a9aa5ed-2502-4c73-a8ac-cb7d0259500b"}]: dispatch 2024-05-06T12:09:17.226 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:16 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi190"}]: dispatch 2024-05-06T12:09:17.226 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:16 smithi092 ceph-mon[26954]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a9aa5ed-2502-4c73-a8ac-cb7d0259500b"}]': finished 2024-05-06T12:09:17.226 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:16 smithi092 ceph-mon[26954]: osdmap e23: 4 total, 3 up, 4 in 2024-05-06T12:09:17.226 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:16 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi092"}]: dispatch 2024-05-06T12:09:17.226 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:16 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi190"}]: dispatch 2024-05-06T12:09:17.226 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:16 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-06T12:09:17.314 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:16 smithi190 sudo[37288]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-06T12:09:17.315 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:16 smithi190 sudo[37288]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-06T12:09:17.316 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:16 smithi190 sudo[37288]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-06T12:09:17.316 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:16 smithi190 sudo[37288]: pam_unix(sudo:session): session closed for user root 2024-05-06T12:09:17.316 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:16 smithi190 ceph-mon[33798]: pgmap v76: 1 pgs: 1 active+clean; 449 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-05-06T12:09:17.316 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:16 smithi190 ceph-mon[33798]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-06T12:09:17.316 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:16 smithi190 ceph-mon[33798]: from='client.? 172.21.15.190:0/417146884' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a9aa5ed-2502-4c73-a8ac-cb7d0259500b"}]: dispatch 2024-05-06T12:09:17.316 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:16 smithi190 ceph-mon[33798]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-06T12:09:17.317 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:16 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi092"}]: dispatch 2024-05-06T12:09:17.317 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:16 smithi190 ceph-mon[33798]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a9aa5ed-2502-4c73-a8ac-cb7d0259500b"}]: dispatch 2024-05-06T12:09:17.317 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:16 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi190"}]: dispatch 2024-05-06T12:09:17.317 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:16 smithi190 ceph-mon[33798]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a9aa5ed-2502-4c73-a8ac-cb7d0259500b"}]': finished 2024-05-06T12:09:17.317 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:16 smithi190 ceph-mon[33798]: osdmap e23: 4 total, 3 up, 4 in 2024-05-06T12:09:17.317 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:16 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi092"}]: dispatch 2024-05-06T12:09:17.317 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:16 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi190"}]: dispatch 2024-05-06T12:09:17.317 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:16 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-06T12:09:18.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:17 smithi190 ceph-mon[33798]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-06T12:09:18.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:17 smithi190 ceph-mon[33798]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-06T12:09:18.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:17 smithi190 ceph-mon[33798]: mgrmap e19: smithi092.gyzeqb(active, since 2m), standbys: smithi190.kczxkh 2024-05-06T12:09:18.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:17 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:18.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:17 smithi190 ceph-mon[33798]: from='client.? 172.21.15.190:0/2637082113' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-06T12:09:18.378 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:17 smithi092 ceph-mon[26954]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-06T12:09:18.378 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:17 smithi092 ceph-mon[26954]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-06T12:09:18.379 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:17 smithi092 ceph-mon[26954]: mgrmap e19: smithi092.gyzeqb(active, since 2m), standbys: smithi190.kczxkh 2024-05-06T12:09:18.379 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:17 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:18.379 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:17 smithi092 ceph-mon[26954]: from='client.? 172.21.15.190:0/2637082113' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-06T12:09:19.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:18 smithi190 ceph-mon[33798]: pgmap v78: 1 pgs: 1 active+clean; 449 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-05-06T12:09:19.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:18 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:09:19.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:18 smithi092 ceph-mon[26954]: pgmap v78: 1 pgs: 1 active+clean; 449 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-05-06T12:09:19.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:18 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:09:20.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:20 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:20.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:20 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:20.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:20 smithi092 ceph-mon[26954]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 5.4 KiB/s rd, 259 KiB/s wr, 17 op/s 2024-05-06T12:09:20.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:20 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:09:20.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:20 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:09:20.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:20 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:21.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:20 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:21.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:20 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:21.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:20 smithi190 ceph-mon[33798]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 5.4 KiB/s rd, 259 KiB/s wr, 17 op/s 2024-05-06T12:09:21.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:20 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:09:21.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:20 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:09:21.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:20 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:23.033 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:22 smithi190 ceph-mon[33798]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.8 KiB/s rd, 186 KiB/s wr, 12 op/s 2024-05-06T12:09:23.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:22 smithi092 ceph-mon[26954]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.8 KiB/s rd, 186 KiB/s wr, 12 op/s 2024-05-06T12:09:25.009 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:24 smithi190 ceph-mon[33798]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 145 KiB/s wr, 9 op/s 2024-05-06T12:09:25.009 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:24 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-06T12:09:25.009 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:24 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:09:25.009 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:24 smithi190 ceph-mon[33798]: Deploying daemon osd.3 on smithi190 2024-05-06T12:09:25.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:24 smithi092 ceph-mon[26954]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 145 KiB/s wr, 9 op/s 2024-05-06T12:09:25.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:24 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-06T12:09:25.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:24 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:09:25.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:24 smithi092 ceph-mon[26954]: Deploying daemon osd.3 on smithi190 2024-05-06T12:09:26.975 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:26 smithi190 ceph-mon[33798]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-05-06T12:09:27.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:26 smithi092 ceph-mon[26954]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-05-06T12:09:29.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:28 smithi190 ceph-mon[33798]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.4 KiB/s rd, 115 KiB/s wr, 7 op/s 2024-05-06T12:09:29.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:28 smithi092 ceph-mon[26954]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.4 KiB/s rd, 115 KiB/s wr, 7 op/s 2024-05-06T12:09:30.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:30 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:30.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:30 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:30.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:30 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:09:30.768 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:30 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:30.768 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:30 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:30.768 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:30 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:09:31.583 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:31 smithi190 ceph-mon[33798]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-05-06T12:09:31.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:31 smithi092 ceph-mon[26954]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-05-06T12:09:32.879 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:32.876+0000 7ff655ffb700 1 -- 172.21.15.190:0/143480088 <== mgr.14210 v2:172.21.15.92:6800/3045905943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff66002b340 con 0x7ff64c069210 2024-05-06T12:09:32.880 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 3 on host 'smithi190' 2024-05-06T12:09:32.881 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:32.878+0000 7ff667a0a700 1 -- 172.21.15.190:0/143480088 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7ff64c069210 msgr2=0x7ff64c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:09:32.882 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:32.878+0000 7ff667a0a700 1 --2- 172.21.15.190:0/143480088 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7ff64c069210 0x7ff64c06b6c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff650007fd0 tx=0x7ff650009040 comp rx=0 tx=0).stop 2024-05-06T12:09:32.882 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:32.878+0000 7ff667a0a700 1 -- 172.21.15.190:0/143480088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6600fbcc0 msgr2=0x7ff66010d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:09:32.882 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:32.878+0000 7ff667a0a700 1 --2- 172.21.15.190:0/143480088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6600fbcc0 0x7ff66010d990 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff65c009150 tx=0x7ff65c009e30 comp rx=0 tx=0).stop 2024-05-06T12:09:32.882 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:32.878+0000 7ff667a0a700 1 -- 172.21.15.190:0/143480088 shutdown_connections 2024-05-06T12:09:32.882 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:32.878+0000 7ff667a0a700 1 --2- 172.21.15.190:0/143480088 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7ff64c069210 0x7ff64c06b6c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:09:32.882 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:32.878+0000 7ff667a0a700 1 --2- 172.21.15.190:0/143480088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6600fbcc0 0x7ff66010d990 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:09:32.882 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:32.878+0000 7ff667a0a700 1 --2- 172.21.15.190:0/143480088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6600fcf70 0x7ff66010ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:09:32.882 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:32.878+0000 7ff667a0a700 1 -- 172.21.15.190:0/143480088 >> 172.21.15.190:0/143480088 conn(0x7ff6600f7440 msgr2=0x7ff660066840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:09:32.882 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:32.879+0000 7ff667a0a700 1 -- 172.21.15.190:0/143480088 shutdown_connections 2024-05-06T12:09:32.882 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:32.879+0000 7ff667a0a700 1 -- 172.21.15.190:0/143480088 wait complete. 2024-05-06T12:09:33.545 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:33 smithi190 ceph-mon[33798]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-06T12:09:33.545 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:33 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:33.545 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:33 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:33.545 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:33 smithi190 ceph-mon[33798]: from='osd.3 [v2:172.21.15.190:6800/2610168113,v1:172.21.15.190:6801/2610168113]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-06T12:09:33.545 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:33 smithi190 ceph-mon[33798]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-06T12:09:33.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:33 smithi092 ceph-mon[26954]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-06T12:09:33.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:33 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:33.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:33 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:33.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:33 smithi092 ceph-mon[26954]: from='osd.3 [v2:172.21.15.190:6800/2610168113,v1:172.21.15.190:6801/2610168113]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-06T12:09:33.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:33 smithi092 ceph-mon[26954]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-06T12:09:33.844 DEBUG:teuthology.orchestra.run.smithi190:osd.3> sudo journalctl -f -n 0 -u ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@osd.3.service 2024-05-06T12:09:33.846 INFO:tasks.cephadm:Deploying osd.4 on smithi190 with /dev/vg_nvme/lv_3... 2024-05-06T12:09:33.847 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-06T12:09:34.168 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi190/config 2024-05-06T12:09:34.558 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:34 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:09:34.558 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:34 smithi190 ceph-mon[33798]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-06T12:09:34.558 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:34 smithi190 ceph-mon[33798]: osdmap e24: 4 total, 3 up, 4 in 2024-05-06T12:09:34.558 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:34 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-06T12:09:34.558 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:34 smithi190 ceph-mon[33798]: from='osd.3 [v2:172.21.15.190:6800/2610168113,v1:172.21.15.190:6801/2610168113]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-05-06T12:09:34.558 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:34 smithi190 ceph-mon[33798]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-05-06T12:09:34.558 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:34 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:34.559 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:34 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:34.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:34 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:09:34.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:34 smithi092 ceph-mon[26954]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-06T12:09:34.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:34 smithi092 ceph-mon[26954]: osdmap e24: 4 total, 3 up, 4 in 2024-05-06T12:09:34.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:34 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-06T12:09:34.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:34 smithi092 ceph-mon[26954]: from='osd.3 [v2:172.21.15.190:6800/2610168113,v1:172.21.15.190:6801/2610168113]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-05-06T12:09:34.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:34 smithi092 ceph-mon[26954]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-05-06T12:09:34.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:34 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:34.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:34 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:35.348 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:09:34 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3[39370]: 2024-05-06T12:09:34.880+0000 7fd63ab26700 -1 osd.3 0 waiting for initial osdmap 2024-05-06T12:09:35.349 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:09:34 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3[39370]: 2024-05-06T12:09:34.886+0000 7fd637925700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-06T12:09:35.640 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:35 smithi190 ceph-mon[33798]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-06T12:09:35.640 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:35 smithi190 ceph-mon[33798]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-05-06T12:09:35.640 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:35 smithi190 ceph-mon[33798]: osdmap e25: 4 total, 3 up, 4 in 2024-05-06T12:09:35.640 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:35 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-06T12:09:35.640 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:35 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-06T12:09:35.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:35 smithi092 ceph-mon[26954]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-06T12:09:35.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:35 smithi092 ceph-mon[26954]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-05-06T12:09:35.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:35 smithi092 ceph-mon[26954]: osdmap e25: 4 total, 3 up, 4 in 2024-05-06T12:09:35.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:35 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-06T12:09:35.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:35 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-06T12:09:36.189 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-06T12:09:36.189 INFO:teuthology.orchestra.run.smithi190.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-06T12:09:36.190 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10+0 records in 2024-05-06T12:09:36.190 INFO:teuthology.orchestra.run.smithi190.stderr:10+0 records out 2024-05-06T12:09:36.190 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0122264 s, 858 MB/s 2024-05-06T12:09:36.190 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping successful for: 2024-05-06T12:09:37.163 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:36 smithi190 ceph-mon[33798]: purged_snaps scrub starts 2024-05-06T12:09:37.163 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:36 smithi190 ceph-mon[33798]: purged_snaps scrub ok 2024-05-06T12:09:37.163 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:36 smithi190 ceph-mon[33798]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-05-06T12:09:37.163 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:36 smithi190 ceph-mon[33798]: osd.3 [v2:172.21.15.190:6800/2610168113,v1:172.21.15.190:6801/2610168113] boot 2024-05-06T12:09:37.163 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:36 smithi190 ceph-mon[33798]: osdmap e26: 4 total, 4 up, 4 in 2024-05-06T12:09:37.163 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:36 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-06T12:09:37.164 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:36 smithi190 ceph-mon[33798]: osdmap e27: 4 total, 4 up, 4 in 2024-05-06T12:09:37.189 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi190:vg_nvme/lv_3 2024-05-06T12:09:37.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:36 smithi092 ceph-mon[26954]: purged_snaps scrub starts 2024-05-06T12:09:37.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:36 smithi092 ceph-mon[26954]: purged_snaps scrub ok 2024-05-06T12:09:37.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:36 smithi092 ceph-mon[26954]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-05-06T12:09:37.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:36 smithi092 ceph-mon[26954]: osd.3 [v2:172.21.15.190:6800/2610168113,v1:172.21.15.190:6801/2610168113] boot 2024-05-06T12:09:37.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:36 smithi092 ceph-mon[26954]: osdmap e26: 4 total, 4 up, 4 in 2024-05-06T12:09:37.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:36 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-06T12:09:37.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:36 smithi092 ceph-mon[26954]: osdmap e27: 4 total, 4 up, 4 in 2024-05-06T12:09:37.674 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi190/config 2024-05-06T12:09:38.805 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:38 smithi190 ceph-mon[33798]: osdmap e28: 4 total, 4 up, 4 in 2024-05-06T12:09:38.805 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:38 smithi190 ceph-mon[33798]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-06T12:09:38.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:38 smithi092 ceph-mon[26954]: osdmap e28: 4 total, 4 up, 4 in 2024-05-06T12:09:38.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:38 smithi092 ceph-mon[26954]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-06T12:09:39.710 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:39 smithi190 ceph-mon[33798]: osdmap e29: 4 total, 4 up, 4 in 2024-05-06T12:09:39.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:39 smithi092 ceph-mon[26954]: osdmap e29: 4 total, 4 up, 4 in 2024-05-06T12:09:40.180 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.176+0000 7fce7064d700 1 -- 172.21.15.190:0/3553523530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce680fed80 msgr2=0x7fce680faec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:09:40.181 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.176+0000 7fce7064d700 1 --2- 172.21.15.190:0/3553523530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce680fed80 0x7fce680faec0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fce64004940 tx=0x7fce64009e30 comp rx=0 tx=0).stop 2024-05-06T12:09:40.181 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.176+0000 7fce7064d700 1 -- 172.21.15.190:0/3553523530 shutdown_connections 2024-05-06T12:09:40.181 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.176+0000 7fce7064d700 1 --2- 172.21.15.190:0/3553523530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce680fed80 0x7fce680faec0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:09:40.181 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.176+0000 7fce7064d700 1 --2- 172.21.15.190:0/3553523530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce680fe430 0x7fce680fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:09:40.181 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.176+0000 7fce7064d700 1 -- 172.21.15.190:0/3553523530 >> 172.21.15.190:0/3553523530 conn(0x7fce680f6bd0 msgr2=0x7fce680f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:09:40.181 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.177+0000 7fce7064d700 1 -- 172.21.15.190:0/3553523530 shutdown_connections 2024-05-06T12:09:40.181 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.177+0000 7fce7064d700 1 -- 172.21.15.190:0/3553523530 wait complete. 2024-05-06T12:09:40.181 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.177+0000 7fce7064d700 1 Processor -- start 2024-05-06T12:09:40.181 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.177+0000 7fce7064d700 1 -- start start 2024-05-06T12:09:40.181 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.177+0000 7fce7064d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce680fe430 0x7fce6810b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:09:40.181 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.177+0000 7fce7064d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce680fed80 0x7fce6810bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:09:40.181 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.177+0000 7fce7064d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6810c200 con 0x7fce680fe430 2024-05-06T12:09:40.181 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.177+0000 7fce7064d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6810c370 con 0x7fce680fed80 2024-05-06T12:09:40.181 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.177+0000 7fce6dbe8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce680fed80 0x7fce6810bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:09:40.182 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.177+0000 7fce6dbe8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce680fed80 0x7fce6810bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:58566/0 (socket says 172.21.15.190:58566) 2024-05-06T12:09:40.182 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.177+0000 7fce6dbe8700 1 -- 172.21.15.190:0/3797694369 learned_addr learned my addr 172.21.15.190:0/3797694369 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-06T12:09:40.182 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.178+0000 7fce6e3e9700 1 --2- 172.21.15.190:0/3797694369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce680fe430 0x7fce6810b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:09:40.182 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.178+0000 7fce6dbe8700 1 -- 172.21.15.190:0/3797694369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce680fe430 msgr2=0x7fce6810b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:09:40.182 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.178+0000 7fce6dbe8700 1 --2- 172.21.15.190:0/3797694369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce680fe430 0x7fce6810b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:09:40.182 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.178+0000 7fce6dbe8700 1 -- 172.21.15.190:0/3797694369 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce64005040 con 0x7fce680fed80 2024-05-06T12:09:40.182 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.178+0000 7fce6dbe8700 1 --2- 172.21.15.190:0/3797694369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce680fed80 0x7fce6810bc00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fce64013870 tx=0x7fce64009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:09:40.182 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.178+0000 7fce5effd700 1 -- 172.21.15.190:0/3797694369 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce64021070 con 0x7fce680fed80 2024-05-06T12:09:40.182 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.178+0000 7fce5effd700 1 -- 172.21.15.190:0/3797694369 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce6401c400 con 0x7fce680fed80 2024-05-06T12:09:40.182 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.178+0000 7fce7064d700 1 -- 172.21.15.190:0/3797694369 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce68110d70 con 0x7fce680fed80 2024-05-06T12:09:40.182 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.178+0000 7fce5effd700 1 -- 172.21.15.190:0/3797694369 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce6400e070 con 0x7fce680fed80 2024-05-06T12:09:40.183 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.178+0000 7fce7064d700 1 -- 172.21.15.190:0/3797694369 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce681112b0 con 0x7fce680fed80 2024-05-06T12:09:40.183 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.179+0000 7fce7064d700 1 -- 172.21.15.190:0/3797694369 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce6804c710 con 0x7fce680fed80 2024-05-06T12:09:40.183 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.179+0000 7fce5effd700 1 -- 172.21.15.190:0/3797694369 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fce6401ca20 con 0x7fce680fed80 2024-05-06T12:09:40.184 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.180+0000 7fce5effd700 1 --2- 172.21.15.190:0/3797694369 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fce540692a0 0x7fce5406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:09:40.184 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.180+0000 7fce5effd700 1 -- 172.21.15.190:0/3797694369 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3770+0+0 (secure 0 0 0) 0x7fce64087f10 con 0x7fce680fed80 2024-05-06T12:09:40.184 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.180+0000 7fce6e3e9700 1 --2- 172.21.15.190:0/3797694369 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fce540692a0 0x7fce5406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:09:40.184 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.181+0000 7fce6e3e9700 1 --2- 172.21.15.190:0/3797694369 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fce540692a0 0x7fce5406b750 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fce58004600 tx=0x7fce58009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:09:40.187 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.184+0000 7fce5effd700 1 -- 172.21.15.190:0/3797694369 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fce64058370 con 0x7fce680fed80 2024-05-06T12:09:40.381 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:40.379+0000 7fce7064d700 1 -- 172.21.15.190:0/3797694369 --> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fce681114a0 con 0x7fce540692a0 2024-05-06T12:09:40.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:40 smithi092 ceph-mon[26954]: osdmap e30: 4 total, 4 up, 4 in 2024-05-06T12:09:40.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:40 smithi092 ceph-mon[26954]: Detected new or changed devices on smithi190 2024-05-06T12:09:40.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:40 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:40.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:40 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:40.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:40 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:09:40.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:40 smithi092 ceph-mon[26954]: Adjusting osd_memory_target on smithi190 to 15936M 2024-05-06T12:09:40.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:40 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:40.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:40 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:09:40.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:40 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:09:40.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:40 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:40.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:40 smithi092 ceph-mon[26954]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 146 KiB/s, 0 objects/s recovering 2024-05-06T12:09:40.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:40 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:09:40.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:40 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:09:40.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:40 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:09:40.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:40 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:40.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:40 smithi092 ceph-mon[26954]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:09:40.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:40 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-06T12:09:40.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:40 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-06T12:09:40.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:40 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:09:40.796 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:40 smithi190 ceph-mon[33798]: osdmap e30: 4 total, 4 up, 4 in 2024-05-06T12:09:40.796 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:40 smithi190 ceph-mon[33798]: Detected new or changed devices on smithi190 2024-05-06T12:09:40.797 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:40 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:40.797 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:40 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:40.797 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:40 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:09:40.797 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:40 smithi190 ceph-mon[33798]: Adjusting osd_memory_target on smithi190 to 15936M 2024-05-06T12:09:40.797 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:40 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:40.797 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:40 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:09:40.797 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:40 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:09:40.797 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:40 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:40.797 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:40 smithi190 ceph-mon[33798]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 146 KiB/s, 0 objects/s recovering 2024-05-06T12:09:40.797 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:40 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:09:40.797 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:40 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:09:40.797 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:40 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:09:40.797 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:40 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:40.798 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:40 smithi190 ceph-mon[33798]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:09:40.798 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:40 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-06T12:09:40.798 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:40 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-06T12:09:40.798 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:40 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:09:43.154 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:42 smithi190 ceph-mon[33798]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-05-06T12:09:43.154 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:42 smithi190 ceph-mon[33798]: from='client.? 172.21.15.190:0/932234737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4039e4a-c758-4632-85d6-1b7e725c0293"}]: dispatch 2024-05-06T12:09:43.154 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:42 smithi190 ceph-mon[33798]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4039e4a-c758-4632-85d6-1b7e725c0293"}]: dispatch 2024-05-06T12:09:43.154 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:42 smithi190 ceph-mon[33798]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4039e4a-c758-4632-85d6-1b7e725c0293"}]': finished 2024-05-06T12:09:43.154 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:42 smithi190 ceph-mon[33798]: osdmap e31: 5 total, 4 up, 5 in 2024-05-06T12:09:43.154 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:42 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-06T12:09:43.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:42 smithi092 ceph-mon[26954]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-05-06T12:09:43.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:42 smithi092 ceph-mon[26954]: from='client.? 172.21.15.190:0/932234737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4039e4a-c758-4632-85d6-1b7e725c0293"}]: dispatch 2024-05-06T12:09:43.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:42 smithi092 ceph-mon[26954]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4039e4a-c758-4632-85d6-1b7e725c0293"}]: dispatch 2024-05-06T12:09:43.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:42 smithi092 ceph-mon[26954]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4039e4a-c758-4632-85d6-1b7e725c0293"}]': finished 2024-05-06T12:09:43.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:42 smithi092 ceph-mon[26954]: osdmap e31: 5 total, 4 up, 5 in 2024-05-06T12:09:43.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:42 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-06T12:09:44.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:43 smithi092 ceph-mon[26954]: from='client.? 172.21.15.190:0/1022850902' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-06T12:09:44.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:43 smithi190 ceph-mon[33798]: from='client.? 172.21.15.190:0/1022850902' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-06T12:09:45.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:44 smithi092 ceph-mon[26954]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-06T12:09:45.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:44 smithi190 ceph-mon[33798]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-06T12:09:47.010 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:46 smithi190 ceph-mon[33798]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-06T12:09:47.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:46 smithi092 ceph-mon[26954]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-06T12:09:48.942 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:48 smithi190 ceph-mon[33798]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-06T12:09:48.943 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:48 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:09:49.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:48 smithi092 ceph-mon[26954]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-06T12:09:49.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:48 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:09:51.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:50 smithi190 ceph-mon[33798]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-06T12:09:51.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:50 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-06T12:09:51.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:50 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:09:51.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:50 smithi190 ceph-mon[33798]: Deploying daemon osd.4 on smithi190 2024-05-06T12:09:51.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:50 smithi092 ceph-mon[26954]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-06T12:09:51.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:50 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-06T12:09:51.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:50 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:09:51.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:50 smithi092 ceph-mon[26954]: Deploying daemon osd.4 on smithi190 2024-05-06T12:09:53.095 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:52 smithi190 ceph-mon[33798]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-06T12:09:53.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:52 smithi092 ceph-mon[26954]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-06T12:09:55.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:54 smithi190 ceph-mon[33798]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-06T12:09:55.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:54 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:09:55.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:54 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:55.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:54 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:55.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:54 smithi092 ceph-mon[26954]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-06T12:09:55.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:54 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:09:55.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:54 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:55.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:54 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:57.119 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:56 smithi190 ceph-mon[33798]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-06T12:09:57.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:56 smithi092 ceph-mon[26954]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-06T12:09:57.846 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 4 on host 'smithi190' 2024-05-06T12:09:57.847 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:57.843+0000 7fce5effd700 1 -- 172.21.15.190:0/3797694369 <== mgr.14210 v2:172.21.15.92:6800/3045905943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fce681114a0 con 0x7fce540692a0 2024-05-06T12:09:57.848 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:57.846+0000 7fce7064d700 1 -- 172.21.15.190:0/3797694369 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fce540692a0 msgr2=0x7fce5406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:09:57.848 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:57.846+0000 7fce7064d700 1 --2- 172.21.15.190:0/3797694369 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fce540692a0 0x7fce5406b750 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fce58004600 tx=0x7fce58009e30 comp rx=0 tx=0).stop 2024-05-06T12:09:57.849 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:57.846+0000 7fce7064d700 1 -- 172.21.15.190:0/3797694369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce680fed80 msgr2=0x7fce6810bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:09:57.849 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:57.846+0000 7fce7064d700 1 --2- 172.21.15.190:0/3797694369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce680fed80 0x7fce6810bc00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fce64013870 tx=0x7fce64009e30 comp rx=0 tx=0).stop 2024-05-06T12:09:57.849 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:57.846+0000 7fce7064d700 1 -- 172.21.15.190:0/3797694369 shutdown_connections 2024-05-06T12:09:57.849 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:57.846+0000 7fce7064d700 1 --2- 172.21.15.190:0/3797694369 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fce540692a0 0x7fce5406b750 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:09:57.849 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:57.846+0000 7fce7064d700 1 --2- 172.21.15.190:0/3797694369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce680fe430 0x7fce6810b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:09:57.849 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:57.846+0000 7fce7064d700 1 --2- 172.21.15.190:0/3797694369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce680fed80 0x7fce6810bc00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:09:57.849 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:57.846+0000 7fce7064d700 1 -- 172.21.15.190:0/3797694369 >> 172.21.15.190:0/3797694369 conn(0x7fce680f6bd0 msgr2=0x7fce680f8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:09:57.849 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:57.846+0000 7fce7064d700 1 -- 172.21.15.190:0/3797694369 shutdown_connections 2024-05-06T12:09:57.849 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:09:57.846+0000 7fce7064d700 1 -- 172.21.15.190:0/3797694369 wait complete. 2024-05-06T12:09:58.534 DEBUG:teuthology.orchestra.run.smithi190:osd.4> sudo journalctl -f -n 0 -u ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@osd.4.service 2024-05-06T12:09:58.537 INFO:tasks.cephadm:Deploying osd.5 on smithi190 with /dev/vg_nvme/lv_2... 2024-05-06T12:09:58.537 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-06T12:09:58.804 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:09:58 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4[46190]: 2024-05-06T12:09:58.747+0000 7f3b3f90c540 -1 osd.4 0 log_to_monitors true 2024-05-06T12:09:59.076 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:58 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:59.077 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:58 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:59.077 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:58 smithi190 ceph-mon[33798]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-06T12:09:59.077 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:58 smithi190 ceph-mon[33798]: from='osd.4 [v2:172.21.15.190:6808/1405678389,v1:172.21.15.190:6809/1405678389]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-06T12:09:59.077 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:58 smithi190 ceph-mon[33798]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-06T12:09:59.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:58 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:59.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:58 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:09:59.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:58 smithi092 ceph-mon[26954]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-06T12:09:59.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:58 smithi092 ceph-mon[26954]: from='osd.4 [v2:172.21.15.190:6808/1405678389,v1:172.21.15.190:6809/1405678389]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-06T12:09:59.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:58 smithi092 ceph-mon[26954]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-06T12:09:59.329 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi190/config 2024-05-06T12:10:00.047 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:09:59 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4[46190]: 2024-05-06T12:09:59.848+0000 7f3b360dd700 -1 osd.4 0 waiting for initial osdmap 2024-05-06T12:10:00.047 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:09:59 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4[46190]: 2024-05-06T12:09:59.855+0000 7f3b30ed8700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-06T12:10:00.047 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:59 smithi190 ceph-mon[33798]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-06T12:10:00.047 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:59 smithi190 ceph-mon[33798]: osdmap e32: 5 total, 4 up, 5 in 2024-05-06T12:10:00.047 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:59 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-06T12:10:00.047 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:59 smithi190 ceph-mon[33798]: from='osd.4 [v2:172.21.15.190:6808/1405678389,v1:172.21.15.190:6809/1405678389]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-05-06T12:10:00.047 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:09:59 smithi190 ceph-mon[33798]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-05-06T12:10:00.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:59 smithi092 ceph-mon[26954]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-06T12:10:00.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:59 smithi092 ceph-mon[26954]: osdmap e32: 5 total, 4 up, 5 in 2024-05-06T12:10:00.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:59 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-06T12:10:00.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:59 smithi092 ceph-mon[26954]: from='osd.4 [v2:172.21.15.190:6808/1405678389,v1:172.21.15.190:6809/1405678389]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-05-06T12:10:00.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:09:59 smithi092 ceph-mon[26954]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-05-06T12:10:01.068 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:00 smithi190 ceph-mon[33798]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-06T12:10:01.068 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:00 smithi190 ceph-mon[33798]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-05-06T12:10:01.068 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:00 smithi190 ceph-mon[33798]: osdmap e33: 5 total, 4 up, 5 in 2024-05-06T12:10:01.068 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:00 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-06T12:10:01.068 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:00 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-06T12:10:01.068 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:00 smithi190 ceph-mon[33798]: overall HEALTH_OK 2024-05-06T12:10:01.068 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:00 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:01.068 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:00 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:01.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:00 smithi092 ceph-mon[26954]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-06T12:10:01.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:00 smithi092 ceph-mon[26954]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-05-06T12:10:01.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:00 smithi092 ceph-mon[26954]: osdmap e33: 5 total, 4 up, 5 in 2024-05-06T12:10:01.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:00 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-06T12:10:01.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:00 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-06T12:10:01.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:00 smithi092 ceph-mon[26954]: overall HEALTH_OK 2024-05-06T12:10:01.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:00 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:01.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:00 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:01.279 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-06T12:10:01.279 INFO:teuthology.orchestra.run.smithi190.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-06T12:10:01.279 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10+0 records in 2024-05-06T12:10:01.279 INFO:teuthology.orchestra.run.smithi190.stderr:10+0 records out 2024-05-06T12:10:01.279 INFO:teuthology.orchestra.run.smithi190.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0159931 s, 656 MB/s 2024-05-06T12:10:01.279 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 2024-05-06T12:10:01.280 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping successful for: 2024-05-06T12:10:02.033 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:01 smithi190 ceph-mon[33798]: purged_snaps scrub starts 2024-05-06T12:10:02.033 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:01 smithi190 ceph-mon[33798]: purged_snaps scrub ok 2024-05-06T12:10:02.033 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:01 smithi190 ceph-mon[33798]: osd.4 [v2:172.21.15.190:6808/1405678389,v1:172.21.15.190:6809/1405678389] boot 2024-05-06T12:10:02.033 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:01 smithi190 ceph-mon[33798]: osdmap e34: 5 total, 5 up, 5 in 2024-05-06T12:10:02.033 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:01 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-06T12:10:02.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:01 smithi092 ceph-mon[26954]: purged_snaps scrub starts 2024-05-06T12:10:02.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:01 smithi092 ceph-mon[26954]: purged_snaps scrub ok 2024-05-06T12:10:02.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:01 smithi092 ceph-mon[26954]: osd.4 [v2:172.21.15.190:6808/1405678389,v1:172.21.15.190:6809/1405678389] boot 2024-05-06T12:10:02.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:01 smithi092 ceph-mon[26954]: osdmap e34: 5 total, 5 up, 5 in 2024-05-06T12:10:02.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:01 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-06T12:10:02.247 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi190:vg_nvme/lv_2 2024-05-06T12:10:02.944 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:02 smithi190 ceph-mon[33798]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-06T12:10:02.944 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:02 smithi190 ceph-mon[33798]: osdmap e35: 5 total, 5 up, 5 in 2024-05-06T12:10:03.113 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi190/config 2024-05-06T12:10:03.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:02 smithi092 ceph-mon[26954]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-06T12:10:03.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:02 smithi092 ceph-mon[26954]: osdmap e35: 5 total, 5 up, 5 in 2024-05-06T12:10:04.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:03 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:10:04.266 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:03 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:10:05.258 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.252+0000 7fd825c72700 1 -- 172.21.15.190:0/4131158845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd820066f30 msgr2=0x7fd8200ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:05.258 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.252+0000 7fd825c72700 1 --2- 172.21.15.190:0/4131158845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd820066f30 0x7fd8200ff9b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd810004940 tx=0x7fd810009e30 comp rx=0 tx=0).stop 2024-05-06T12:10:05.258 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.252+0000 7fd825c72700 1 -- 172.21.15.190:0/4131158845 shutdown_connections 2024-05-06T12:10:05.259 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.252+0000 7fd825c72700 1 --2- 172.21.15.190:0/4131158845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8200ffef0 0x7fd820102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:05.259 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.252+0000 7fd825c72700 1 --2- 172.21.15.190:0/4131158845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd820066f30 0x7fd8200ff9b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:05.259 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.252+0000 7fd825c72700 1 -- 172.21.15.190:0/4131158845 >> 172.21.15.190:0/4131158845 conn(0x7fd8200f7440 msgr2=0x7fd8200f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:10:05.259 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.252+0000 7fd825c72700 1 -- 172.21.15.190:0/4131158845 shutdown_connections 2024-05-06T12:10:05.259 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.252+0000 7fd825c72700 1 -- 172.21.15.190:0/4131158845 wait complete. 2024-05-06T12:10:05.259 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.252+0000 7fd825c72700 1 Processor -- start 2024-05-06T12:10:05.259 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.252+0000 7fd825c72700 1 -- start start 2024-05-06T12:10:05.260 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.253+0000 7fd825c72700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd820066f30 0x7fd82010d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:05.260 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.253+0000 7fd825c72700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8200ffef0 0x7fd82010df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:05.260 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.253+0000 7fd825c72700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82010e5a0 con 0x7fd8200ffef0 2024-05-06T12:10:05.260 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.253+0000 7fd825c72700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd820112ea0 con 0x7fd820066f30 2024-05-06T12:10:05.260 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.253+0000 7fd81f7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd820066f30 0x7fd82010d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:05.261 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.253+0000 7fd81f7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd820066f30 0x7fd82010d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:46178/0 (socket says 172.21.15.190:46178) 2024-05-06T12:10:05.261 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.253+0000 7fd81f7fe700 1 -- 172.21.15.190:0/3920981048 learned_addr learned my addr 172.21.15.190:0/3920981048 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-06T12:10:05.261 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.253+0000 7fd81f7fe700 1 -- 172.21.15.190:0/3920981048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8200ffef0 msgr2=0x7fd82010df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:05.261 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.253+0000 7fd81f7fe700 1 --2- 172.21.15.190:0/3920981048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8200ffef0 0x7fd82010df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:05.261 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.253+0000 7fd81f7fe700 1 -- 172.21.15.190:0/3920981048 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd810005040 con 0x7fd820066f30 2024-05-06T12:10:05.261 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.253+0000 7fd81f7fe700 1 --2- 172.21.15.190:0/3920981048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd820066f30 0x7fd82010d9d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd81000b3b0 tx=0x7fd810009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:10:05.262 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.253+0000 7fd81cff9700 1 -- 172.21.15.190:0/3920981048 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd81000e040 con 0x7fd820066f30 2024-05-06T12:10:05.262 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.253+0000 7fd81cff9700 1 -- 172.21.15.190:0/3920981048 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd81000bcc0 con 0x7fd820066f30 2024-05-06T12:10:05.262 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.253+0000 7fd81cff9700 1 -- 172.21.15.190:0/3920981048 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd81001c630 con 0x7fd820066f30 2024-05-06T12:10:05.262 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.253+0000 7fd825c72700 1 -- 172.21.15.190:0/3920981048 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8201130a0 con 0x7fd820066f30 2024-05-06T12:10:05.262 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.253+0000 7fd825c72700 1 -- 172.21.15.190:0/3920981048 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd820113610 con 0x7fd820066f30 2024-05-06T12:10:05.262 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.254+0000 7fd825c72700 1 -- 172.21.15.190:0/3920981048 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd82004c710 con 0x7fd820066f30 2024-05-06T12:10:05.263 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.255+0000 7fd81cff9700 1 -- 172.21.15.190:0/3920981048 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fd8100139f0 con 0x7fd820066f30 2024-05-06T12:10:05.263 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.255+0000 7fd81cff9700 1 --2- 172.21.15.190:0/3920981048 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fd80c0692a0 0x7fd80c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:05.263 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.255+0000 7fd81cff9700 1 -- 172.21.15.190:0/3920981048 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4202+0+0 (secure 0 0 0) 0x7fd810087ab0 con 0x7fd820066f30 2024-05-06T12:10:05.263 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.255+0000 7fd817fff700 1 --2- 172.21.15.190:0/3920981048 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fd80c0692a0 0x7fd80c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:05.263 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.256+0000 7fd817fff700 1 --2- 172.21.15.190:0/3920981048 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fd80c0692a0 0x7fd80c06b750 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd808006fd0 tx=0x7fd808006db0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:10:05.264 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.259+0000 7fd81cff9700 1 -- 172.21.15.190:0/3920981048 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd810057d60 con 0x7fd820066f30 2024-05-06T12:10:05.457 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:05.455+0000 7fd825c72700 1 -- 172.21.15.190:0/3920981048 --> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8200f8f30 con 0x7fd80c0692a0 2024-05-06T12:10:05.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:05 smithi190 ceph-mon[33798]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-06T12:10:05.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:05 smithi190 ceph-mon[33798]: Detected new or changed devices on smithi190 2024-05-06T12:10:05.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:05 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:05.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:05 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:05.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:05 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:10:05.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:05 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:10:05.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:05 smithi190 ceph-mon[33798]: Adjusting osd_memory_target on smithi190 to 7968M 2024-05-06T12:10:05.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:05 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:05.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:05 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:10:05.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:05 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:10:05.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:05 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:05.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:05 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:10:05.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:05 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:10:05.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:05 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:10:05.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:05 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:05.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:05 smithi092 ceph-mon[26954]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-06T12:10:05.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:05 smithi092 ceph-mon[26954]: Detected new or changed devices on smithi190 2024-05-06T12:10:05.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:05 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:05.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:05 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:05.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:05 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:10:05.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:05 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:10:05.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:05 smithi092 ceph-mon[26954]: Adjusting osd_memory_target on smithi190 to 7968M 2024-05-06T12:10:05.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:05 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:05.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:05 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:10:05.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:05 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:10:05.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:05 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:05.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:05 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:10:05.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:05 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:10:05.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:05 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:10:05.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:05 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:06.468 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:06 smithi190 ceph-mon[33798]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:10:06.468 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:06 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-06T12:10:06.468 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:06 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-06T12:10:06.468 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:06 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:10:06.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:06 smithi092 ceph-mon[26954]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:10:06.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:06 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-06T12:10:06.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:06 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-06T12:10:06.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:06 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:10:07.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:07 smithi190 ceph-mon[33798]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-06T12:10:07.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:07 smithi092 ceph-mon[26954]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-06T12:10:08.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:08 smithi190 ceph-mon[33798]: from='client.? 172.21.15.190:0/3356739668' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec707eee-5f24-48bb-818a-547f804fc41b"}]: dispatch 2024-05-06T12:10:08.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:08 smithi190 ceph-mon[33798]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec707eee-5f24-48bb-818a-547f804fc41b"}]: dispatch 2024-05-06T12:10:08.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:08 smithi190 ceph-mon[33798]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ec707eee-5f24-48bb-818a-547f804fc41b"}]': finished 2024-05-06T12:10:08.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:08 smithi190 ceph-mon[33798]: osdmap e36: 6 total, 5 up, 6 in 2024-05-06T12:10:08.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:08 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-06T12:10:08.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:08 smithi092 ceph-mon[26954]: from='client.? 172.21.15.190:0/3356739668' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec707eee-5f24-48bb-818a-547f804fc41b"}]: dispatch 2024-05-06T12:10:08.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:08 smithi092 ceph-mon[26954]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec707eee-5f24-48bb-818a-547f804fc41b"}]: dispatch 2024-05-06T12:10:08.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:08 smithi092 ceph-mon[26954]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ec707eee-5f24-48bb-818a-547f804fc41b"}]': finished 2024-05-06T12:10:08.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:08 smithi092 ceph-mon[26954]: osdmap e36: 6 total, 5 up, 6 in 2024-05-06T12:10:08.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:08 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-06T12:10:09.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:09 smithi190 ceph-mon[33798]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-06T12:10:09.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:09 smithi190 ceph-mon[33798]: from='client.? 172.21.15.190:0/54631870' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-06T12:10:09.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:09 smithi092 ceph-mon[26954]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-06T12:10:09.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:09 smithi092 ceph-mon[26954]: from='client.? 172.21.15.190:0/54631870' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-06T12:10:11.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:11 smithi190 ceph-mon[33798]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-06T12:10:11.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:11 smithi092 ceph-mon[26954]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-06T12:10:13.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:13 smithi190 ceph-mon[33798]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-06T12:10:13.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:13 smithi092 ceph-mon[26954]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-06T12:10:15.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:15 smithi190 ceph-mon[33798]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-06T12:10:15.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:15 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-06T12:10:15.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:15 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:10:15.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:15 smithi092 ceph-mon[26954]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-06T12:10:15.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:15 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-06T12:10:15.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:15 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:10:16.532 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:16 smithi190 ceph-mon[33798]: Deploying daemon osd.5 on smithi190 2024-05-06T12:10:16.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:16 smithi092 ceph-mon[26954]: Deploying daemon osd.5 on smithi190 2024-05-06T12:10:17.498 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:17 smithi190 ceph-mon[33798]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-06T12:10:17.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:17 smithi092 ceph-mon[26954]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-06T12:10:19.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:19 smithi190 ceph-mon[33798]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-06T12:10:19.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:19 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:10:19.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:19 smithi092 ceph-mon[26954]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-06T12:10:19.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:19 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:10:21.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:21 smithi092 ceph-mon[26954]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-06T12:10:21.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:21 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:21.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:21 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:21.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:21 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:10:21.552 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:21 smithi190 ceph-mon[33798]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-06T12:10:21.552 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:21 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:21.552 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:21 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:21.553 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:21 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:10:23.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:23 smithi092 ceph-mon[26954]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-06T12:10:23.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:23 smithi190 ceph-mon[33798]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-06T12:10:23.819 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 5 on host 'smithi190' 2024-05-06T12:10:23.819 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:23.816+0000 7fd81cff9700 1 -- 172.21.15.190:0/3920981048 <== mgr.14210 v2:172.21.15.92:6800/3045905943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd8200f8f30 con 0x7fd80c0692a0 2024-05-06T12:10:23.821 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:23.819+0000 7fd825c72700 1 -- 172.21.15.190:0/3920981048 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fd80c0692a0 msgr2=0x7fd80c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:23.821 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:23.819+0000 7fd825c72700 1 --2- 172.21.15.190:0/3920981048 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fd80c0692a0 0x7fd80c06b750 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd808006fd0 tx=0x7fd808006db0 comp rx=0 tx=0).stop 2024-05-06T12:10:23.821 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:23.819+0000 7fd825c72700 1 -- 172.21.15.190:0/3920981048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd820066f30 msgr2=0x7fd82010d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:23.822 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:23.819+0000 7fd825c72700 1 --2- 172.21.15.190:0/3920981048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd820066f30 0x7fd82010d9d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd81000b3b0 tx=0x7fd810009e30 comp rx=0 tx=0).stop 2024-05-06T12:10:23.822 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:23.819+0000 7fd825c72700 1 -- 172.21.15.190:0/3920981048 shutdown_connections 2024-05-06T12:10:23.822 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:23.819+0000 7fd825c72700 1 --2- 172.21.15.190:0/3920981048 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fd80c0692a0 0x7fd80c06b750 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:23.822 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:23.819+0000 7fd825c72700 1 --2- 172.21.15.190:0/3920981048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd820066f30 0x7fd82010d9d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:23.822 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:23.819+0000 7fd825c72700 1 --2- 172.21.15.190:0/3920981048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8200ffef0 0x7fd82010df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:23.822 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:23.819+0000 7fd825c72700 1 -- 172.21.15.190:0/3920981048 >> 172.21.15.190:0/3920981048 conn(0x7fd8200f7440 msgr2=0x7fd8200f7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:10:23.822 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:23.819+0000 7fd825c72700 1 -- 172.21.15.190:0/3920981048 shutdown_connections 2024-05-06T12:10:23.822 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:23.819+0000 7fd825c72700 1 -- 172.21.15.190:0/3920981048 wait complete. 2024-05-06T12:10:24.426 DEBUG:teuthology.orchestra.run.smithi190:osd.5> sudo journalctl -f -n 0 -u ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@osd.5.service 2024-05-06T12:10:24.431 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-05-06T12:10:24.432 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-06T12:10:24.701 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:10:25.062 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:24 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:25.062 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:24 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:25.062 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:24 smithi092 ceph-mon[26954]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-06T12:10:25.062 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:24 smithi092 ceph-mon[26954]: from='osd.5 [v2:172.21.15.190:6816/1027118727,v1:172.21.15.190:6817/1027118727]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-06T12:10:25.062 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:24 smithi092 ceph-mon[26954]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-06T12:10:25.062 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:24 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:25.062 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:24 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:25.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:24 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:25.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:24 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:25.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:24 smithi190 ceph-mon[33798]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-06T12:10:25.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:24 smithi190 ceph-mon[33798]: from='osd.5 [v2:172.21.15.190:6816/1027118727,v1:172.21.15.190:6817/1027118727]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-06T12:10:25.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:24 smithi190 ceph-mon[33798]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-06T12:10:25.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:24 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:25.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:24 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:26.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:25 smithi190 ceph-mon[33798]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-06T12:10:26.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:25 smithi190 ceph-mon[33798]: osdmap e37: 6 total, 5 up, 6 in 2024-05-06T12:10:26.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:25 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-06T12:10:26.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:25 smithi190 ceph-mon[33798]: from='osd.5 [v2:172.21.15.190:6816/1027118727,v1:172.21.15.190:6817/1027118727]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-05-06T12:10:26.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:25 smithi190 ceph-mon[33798]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-05-06T12:10:26.099 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:10:25 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[52689]: 2024-05-06T12:10:25.805+0000 7fbd7ccf2700 -1 osd.5 0 waiting for initial osdmap 2024-05-06T12:10:26.100 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:10:25 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[52689]: 2024-05-06T12:10:25.813+0000 7fbd79af1700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-06T12:10:26.146 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:25 smithi092 ceph-mon[26954]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-06T12:10:26.146 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:25 smithi092 ceph-mon[26954]: osdmap e37: 6 total, 5 up, 6 in 2024-05-06T12:10:26.146 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:25 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-06T12:10:26.146 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:25 smithi092 ceph-mon[26954]: from='osd.5 [v2:172.21.15.190:6816/1027118727,v1:172.21.15.190:6817/1027118727]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-05-06T12:10:26.146 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:25 smithi092 ceph-mon[26954]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-05-06T12:10:26.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.456+0000 7efca2f8f700 1 -- 172.21.15.92:0/4032352440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc9c0fce60 msgr2=0x7efc9c0fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:26.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.456+0000 7efca2f8f700 1 --2- 172.21.15.92:0/4032352440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc9c0fce60 0x7efc9c0fd270 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efc90004940 tx=0x7efc90009e30 comp rx=0 tx=0).stop 2024-05-06T12:10:26.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.456+0000 7efca2f8f700 1 -- 172.21.15.92:0/4032352440 shutdown_connections 2024-05-06T12:10:26.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.456+0000 7efca2f8f700 1 --2- 172.21.15.92:0/4032352440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc9c0fdfd0 0x7efc9c0fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:26.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.456+0000 7efca2f8f700 1 --2- 172.21.15.92:0/4032352440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc9c0fce60 0x7efc9c0fd270 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:26.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.456+0000 7efca2f8f700 1 -- 172.21.15.92:0/4032352440 >> 172.21.15.92:0/4032352440 conn(0x7efc9c0f8540 msgr2=0x7efc9c0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:10:26.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.456+0000 7efca2f8f700 1 -- 172.21.15.92:0/4032352440 shutdown_connections 2024-05-06T12:10:26.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.456+0000 7efca2f8f700 1 -- 172.21.15.92:0/4032352440 wait complete. 2024-05-06T12:10:26.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.456+0000 7efca2f8f700 1 Processor -- start 2024-05-06T12:10:26.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.457+0000 7efca2f8f700 1 -- start start 2024-05-06T12:10:26.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.457+0000 7efca2f8f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc9c0fce60 0x7efc9c10d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:26.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.457+0000 7efca2f8f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc9c0fdfd0 0x7efc9c10de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:26.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.457+0000 7efca2f8f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9c10e450 con 0x7efc9c0fce60 2024-05-06T12:10:26.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.457+0000 7efca2f8f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9c10e5c0 con 0x7efc9c0fdfd0 2024-05-06T12:10:26.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.457+0000 7efca0d2b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc9c0fce60 0x7efc9c10d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:26.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.457+0000 7efca0d2b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc9c0fce60 0x7efc9c10d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36736/0 (socket says 172.21.15.92:36736) 2024-05-06T12:10:26.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.457+0000 7efca0d2b700 1 -- 172.21.15.92:0/3154399956 learned_addr learned my addr 172.21.15.92:0/3154399956 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:10:26.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.457+0000 7efc9bfff700 1 --2- 172.21.15.92:0/3154399956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc9c0fdfd0 0x7efc9c10de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:26.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.457+0000 7efca0d2b700 1 -- 172.21.15.92:0/3154399956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc9c0fdfd0 msgr2=0x7efc9c10de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:26.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.457+0000 7efca0d2b700 1 --2- 172.21.15.92:0/3154399956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc9c0fdfd0 0x7efc9c10de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:26.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.457+0000 7efca0d2b700 1 -- 172.21.15.92:0/3154399956 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc90005040 con 0x7efc9c0fce60 2024-05-06T12:10:26.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.457+0000 7efca0d2b700 1 --2- 172.21.15.92:0/3154399956 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc9c0fce60 0x7efc9c10d910 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7efc90009150 tx=0x7efc90009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:10:26.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.457+0000 7efc997fa700 1 -- 172.21.15.92:0/3154399956 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc9000e040 con 0x7efc9c0fce60 2024-05-06T12:10:26.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.457+0000 7efc997fa700 1 -- 172.21.15.92:0/3154399956 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc90004290 con 0x7efc9c0fce60 2024-05-06T12:10:26.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.458+0000 7efca2f8f700 1 -- 172.21.15.92:0/3154399956 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc9c112fc0 con 0x7efc9c0fce60 2024-05-06T12:10:26.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.458+0000 7efc997fa700 1 -- 172.21.15.92:0/3154399956 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc9001c630 con 0x7efc9c0fce60 2024-05-06T12:10:26.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.458+0000 7efca2f8f700 1 -- 172.21.15.92:0/3154399956 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc9c1134f0 con 0x7efc9c0fce60 2024-05-06T12:10:26.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.459+0000 7efca2f8f700 1 -- 172.21.15.92:0/3154399956 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc9c04c710 con 0x7efc9c0fce60 2024-05-06T12:10:26.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.459+0000 7efc997fa700 1 -- 172.21.15.92:0/3154399956 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7efc9000bbf0 con 0x7efc9c0fce60 2024-05-06T12:10:26.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.459+0000 7efc997fa700 1 --2- 172.21.15.92:0/3154399956 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7efc8c0692a0 0x7efc8c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:26.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.459+0000 7efc997fa700 1 -- 172.21.15.92:0/3154399956 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4354+0+0 (secure 0 0 0) 0x7efc90087a10 con 0x7efc9c0fce60 2024-05-06T12:10:26.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.459+0000 7efc9bfff700 1 --2- 172.21.15.92:0/3154399956 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7efc8c0692a0 0x7efc8c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:26.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.460+0000 7efc9bfff700 1 --2- 172.21.15.92:0/3154399956 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7efc8c0692a0 0x7efc8c06b750 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efc88004f40 tx=0x7efc8800b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:10:26.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.464+0000 7efc997fa700 1 -- 172.21.15.92:0/3154399956 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7efc90057c20 con 0x7efc9c0fce60 2024-05-06T12:10:26.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.650+0000 7efca2f8f700 1 -- 172.21.15.92:0/3154399956 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7efc9c1058c0 con 0x7efc9c0fce60 2024-05-06T12:10:26.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.651+0000 7efc997fa700 1 -- 172.21.15.92:0/3154399956 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v38) v1 ==== 74+0+130 (secure 0 0 0) 0x7efc900577b0 con 0x7efc9c0fce60 2024-05-06T12:10:26.653 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:10:26.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.653+0000 7efca2f8f700 1 -- 172.21.15.92:0/3154399956 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7efc8c0692a0 msgr2=0x7efc8c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:26.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.653+0000 7efca2f8f700 1 --2- 172.21.15.92:0/3154399956 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7efc8c0692a0 0x7efc8c06b750 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efc88004f40 tx=0x7efc8800b040 comp rx=0 tx=0).stop 2024-05-06T12:10:26.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.653+0000 7efca2f8f700 1 -- 172.21.15.92:0/3154399956 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc9c0fce60 msgr2=0x7efc9c10d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:26.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.653+0000 7efca2f8f700 1 --2- 172.21.15.92:0/3154399956 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc9c0fce60 0x7efc9c10d910 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7efc90009150 tx=0x7efc90009e30 comp rx=0 tx=0).stop 2024-05-06T12:10:26.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.653+0000 7efca2f8f700 1 -- 172.21.15.92:0/3154399956 shutdown_connections 2024-05-06T12:10:26.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.653+0000 7efca2f8f700 1 --2- 172.21.15.92:0/3154399956 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7efc8c0692a0 0x7efc8c06b750 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:26.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.653+0000 7efca2f8f700 1 --2- 172.21.15.92:0/3154399956 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc9c0fce60 0x7efc9c10d910 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:26.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.653+0000 7efca2f8f700 1 --2- 172.21.15.92:0/3154399956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc9c0fdfd0 0x7efc9c10de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:26.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.653+0000 7efca2f8f700 1 -- 172.21.15.92:0/3154399956 >> 172.21.15.92:0/3154399956 conn(0x7efc9c0f8540 msgr2=0x7efc9c0f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:10:26.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.653+0000 7efca2f8f700 1 -- 172.21.15.92:0/3154399956 shutdown_connections 2024-05-06T12:10:26.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:26.653+0000 7efca2f8f700 1 -- 172.21.15.92:0/3154399956 wait complete. 2024-05-06T12:10:27.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:26 smithi190 ceph-mon[33798]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-05-06T12:10:27.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:26 smithi190 ceph-mon[33798]: osdmap e38: 6 total, 5 up, 6 in 2024-05-06T12:10:27.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:26 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-06T12:10:27.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:26 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-06T12:10:27.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:26 smithi190 ceph-mon[33798]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-06T12:10:27.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:26 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/3154399956' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-06T12:10:27.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:26 smithi092 ceph-mon[26954]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-05-06T12:10:27.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:26 smithi092 ceph-mon[26954]: osdmap e38: 6 total, 5 up, 6 in 2024-05-06T12:10:27.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:26 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-06T12:10:27.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:26 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-06T12:10:27.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:26 smithi092 ceph-mon[26954]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-06T12:10:27.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:26 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/3154399956' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-06T12:10:27.405 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":38,"num_osds":6,"num_up_osds":5,"osd_up_since":1714997400,"num_in_osds":6,"osd_in_since":1714997407,"num_remapped_pgs":0} 2024-05-06T12:10:27.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:27 smithi092 ceph-mon[26954]: purged_snaps scrub starts 2024-05-06T12:10:27.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:27 smithi092 ceph-mon[26954]: purged_snaps scrub ok 2024-05-06T12:10:27.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:27 smithi092 ceph-mon[26954]: osd.5 [v2:172.21.15.190:6816/1027118727,v1:172.21.15.190:6817/1027118727] boot 2024-05-06T12:10:27.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:27 smithi092 ceph-mon[26954]: osdmap e39: 6 total, 6 up, 6 in 2024-05-06T12:10:27.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:27 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-06T12:10:28.027 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:27 smithi190 ceph-mon[33798]: purged_snaps scrub starts 2024-05-06T12:10:28.027 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:27 smithi190 ceph-mon[33798]: purged_snaps scrub ok 2024-05-06T12:10:28.027 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:27 smithi190 ceph-mon[33798]: osd.5 [v2:172.21.15.190:6816/1027118727,v1:172.21.15.190:6817/1027118727] boot 2024-05-06T12:10:28.027 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:27 smithi190 ceph-mon[33798]: osdmap e39: 6 total, 6 up, 6 in 2024-05-06T12:10:28.027 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:27 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-06T12:10:28.407 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-06T12:10:28.670 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:10:29.093 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:28 smithi092 ceph-mon[26954]: osdmap e40: 6 total, 6 up, 6 in 2024-05-06T12:10:29.093 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:28 smithi092 ceph-mon[26954]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:10:29.093 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:28 smithi092 ceph-mon[26954]: Detected new or changed devices on smithi190 2024-05-06T12:10:29.093 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:28 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:29.093 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:28 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:29.093 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:28 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:10:29.093 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:28 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:10:29.093 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:28 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:10:29.093 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:28 smithi092 ceph-mon[26954]: Adjusting osd_memory_target on smithi190 to 5312M 2024-05-06T12:10:29.093 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:28 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:29.094 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:28 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:10:29.094 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:28 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:10:29.094 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:28 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:29.094 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:28 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:10:29.094 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:28 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:10:29.094 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:28 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:10:29.094 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:28 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:29.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:28 smithi190 ceph-mon[33798]: osdmap e40: 6 total, 6 up, 6 in 2024-05-06T12:10:29.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:28 smithi190 ceph-mon[33798]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:10:29.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:28 smithi190 ceph-mon[33798]: Detected new or changed devices on smithi190 2024-05-06T12:10:29.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:28 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:29.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:28 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:29.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:28 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:10:29.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:28 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:10:29.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:28 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:10:29.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:28 smithi190 ceph-mon[33798]: Adjusting osd_memory_target on smithi190 to 5312M 2024-05-06T12:10:29.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:28 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:29.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:28 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:10:29.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:28 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:10:29.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:28 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:29.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:28 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:10:29.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:28 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:10:29.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:28 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:10:29.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:28 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:10:30.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.262+0000 7f2cf4a9d700 1 -- 172.21.15.92:0/361300544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cf00fcf60 msgr2=0x7f2cf00fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:30.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.262+0000 7f2cf4a9d700 1 --2- 172.21.15.92:0/361300544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cf00fcf60 0x7f2cf00fd3d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0004940 tx=0x7f2ce0009e30 comp rx=0 tx=0).stop 2024-05-06T12:10:30.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.262+0000 7f2cf4a9d700 1 -- 172.21.15.92:0/361300544 shutdown_connections 2024-05-06T12:10:30.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.262+0000 7f2cf4a9d700 1 --2- 172.21.15.92:0/361300544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cf00fcf60 0x7f2cf00fd3d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:30.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.262+0000 7f2cf4a9d700 1 --2- 172.21.15.92:0/361300544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cf00fbcb0 0x7f2cf00fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:30.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.262+0000 7f2cf4a9d700 1 -- 172.21.15.92:0/361300544 >> 172.21.15.92:0/361300544 conn(0x7f2cf00f7430 msgr2=0x7f2cf00f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:10:30.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.262+0000 7f2cf4a9d700 1 -- 172.21.15.92:0/361300544 shutdown_connections 2024-05-06T12:10:30.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.262+0000 7f2cf4a9d700 1 -- 172.21.15.92:0/361300544 wait complete. 2024-05-06T12:10:30.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.263+0000 7f2cf4a9d700 1 Processor -- start 2024-05-06T12:10:30.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.263+0000 7f2cf4a9d700 1 -- start start 2024-05-06T12:10:30.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.263+0000 7f2cf4a9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cf00fbcb0 0x7f2cf010c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:30.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.263+0000 7f2cf4a9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cf00fcf60 0x7f2cf010ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:30.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.263+0000 7f2cf4a9d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf010d470 con 0x7f2cf00fcf60 2024-05-06T12:10:30.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.263+0000 7f2cf4a9d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf010dcc0 con 0x7f2cf00fbcb0 2024-05-06T12:10:30.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.263+0000 7f2cedd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cf00fcf60 0x7f2cf010ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:30.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.263+0000 7f2cedd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cf00fcf60 0x7f2cf010ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36752/0 (socket says 172.21.15.92:36752) 2024-05-06T12:10:30.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.263+0000 7f2cedd9b700 1 -- 172.21.15.92:0/2534748335 learned_addr learned my addr 172.21.15.92:0/2534748335 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:10:30.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.263+0000 7f2cee59c700 1 --2- 172.21.15.92:0/2534748335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cf00fbcb0 0x7f2cf010c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:30.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.263+0000 7f2cedd9b700 1 -- 172.21.15.92:0/2534748335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cf00fbcb0 msgr2=0x7f2cf010c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:30.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.263+0000 7f2cedd9b700 1 --2- 172.21.15.92:0/2534748335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cf00fbcb0 0x7f2cf010c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:30.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.263+0000 7f2cedd9b700 1 -- 172.21.15.92:0/2534748335 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce0005040 con 0x7f2cf00fcf60 2024-05-06T12:10:30.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.263+0000 7f2cedd9b700 1 --2- 172.21.15.92:0/2534748335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cf00fcf60 0x7f2cf010ce70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2ce000b3b0 tx=0x7f2ce0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:10:30.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.264+0000 7f2ce6ffd700 1 -- 172.21.15.92:0/2534748335 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ce000e040 con 0x7f2cf00fcf60 2024-05-06T12:10:30.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.264+0000 7f2ce6ffd700 1 -- 172.21.15.92:0/2534748335 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ce000bbf0 con 0x7f2cf00fcf60 2024-05-06T12:10:30.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.264+0000 7f2ce6ffd700 1 -- 172.21.15.92:0/2534748335 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ce001c630 con 0x7f2cf00fcf60 2024-05-06T12:10:30.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.264+0000 7f2cf4a9d700 1 -- 172.21.15.92:0/2534748335 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cf010df40 con 0x7f2cf00fcf60 2024-05-06T12:10:30.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.264+0000 7f2cf4a9d700 1 -- 172.21.15.92:0/2534748335 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf010e480 con 0x7f2cf00fcf60 2024-05-06T12:10:30.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.265+0000 7f2cf4a9d700 1 -- 172.21.15.92:0/2534748335 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cf005f5c0 con 0x7f2cf00fcf60 2024-05-06T12:10:30.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.265+0000 7f2ce6ffd700 1 -- 172.21.15.92:0/2534748335 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f2ce000bd60 con 0x7f2cf00fcf60 2024-05-06T12:10:30.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.265+0000 7f2ce6ffd700 1 --2- 172.21.15.92:0/2534748335 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f2cdc0692a0 0x7f2cdc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:30.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.265+0000 7f2ce6ffd700 1 -- 172.21.15.92:0/2534748335 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f2ce0087a40 con 0x7f2cf00fcf60 2024-05-06T12:10:30.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.266+0000 7f2cee59c700 1 --2- 172.21.15.92:0/2534748335 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f2cdc0692a0 0x7f2cdc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:30.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.266+0000 7f2cee59c700 1 --2- 172.21.15.92:0/2534748335 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f2cdc0692a0 0x7f2cdc06b750 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8004f40 tx=0x7f2cd800b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:10:30.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.270+0000 7f2ce6ffd700 1 -- 172.21.15.92:0/2534748335 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2ce0057be0 con 0x7f2cf00fcf60 2024-05-06T12:10:30.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.463+0000 7f2cf4a9d700 1 -- 172.21.15.92:0/2534748335 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f2cf004bed0 con 0x7f2cf00fcf60 2024-05-06T12:10:30.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.463+0000 7f2ce6ffd700 1 -- 172.21.15.92:0/2534748335 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v40) v1 ==== 74+0+130 (secure 0 0 0) 0x7f2ce0057770 con 0x7f2cf00fcf60 2024-05-06T12:10:30.466 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:10:30.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.465+0000 7f2cf4a9d700 1 -- 172.21.15.92:0/2534748335 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f2cdc0692a0 msgr2=0x7f2cdc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:30.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.465+0000 7f2cf4a9d700 1 --2- 172.21.15.92:0/2534748335 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f2cdc0692a0 0x7f2cdc06b750 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8004f40 tx=0x7f2cd800b040 comp rx=0 tx=0).stop 2024-05-06T12:10:30.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.465+0000 7f2cf4a9d700 1 -- 172.21.15.92:0/2534748335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cf00fcf60 msgr2=0x7f2cf010ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:30.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.465+0000 7f2cf4a9d700 1 --2- 172.21.15.92:0/2534748335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cf00fcf60 0x7f2cf010ce70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2ce000b3b0 tx=0x7f2ce0009e30 comp rx=0 tx=0).stop 2024-05-06T12:10:30.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.465+0000 7f2cf4a9d700 1 -- 172.21.15.92:0/2534748335 shutdown_connections 2024-05-06T12:10:30.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.465+0000 7f2cf4a9d700 1 --2- 172.21.15.92:0/2534748335 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f2cdc0692a0 0x7f2cdc06b750 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:30.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.465+0000 7f2cf4a9d700 1 --2- 172.21.15.92:0/2534748335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cf00fbcb0 0x7f2cf010c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:30.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.465+0000 7f2cf4a9d700 1 --2- 172.21.15.92:0/2534748335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cf00fcf60 0x7f2cf010ce70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:30.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.465+0000 7f2cf4a9d700 1 -- 172.21.15.92:0/2534748335 >> 172.21.15.92:0/2534748335 conn(0x7f2cf00f7430 msgr2=0x7f2cf00f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:10:30.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.465+0000 7f2cf4a9d700 1 -- 172.21.15.92:0/2534748335 shutdown_connections 2024-05-06T12:10:30.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:30.465+0000 7f2cf4a9d700 1 -- 172.21.15.92:0/2534748335 wait complete. 2024-05-06T12:10:31.114 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:30 smithi092 ceph-mon[26954]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:10:31.114 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:30 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/2534748335' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-06T12:10:31.138 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":40,"num_osds":6,"num_up_osds":6,"osd_up_since":1714997426,"num_in_osds":6,"osd_in_since":1714997407,"num_remapped_pgs":0} 2024-05-06T12:10:31.138 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-06T12:10:31.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:30 smithi190 ceph-mon[33798]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:10:31.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:30 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/2534748335' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-06T12:10:31.386 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:10:33.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.064+0000 7feec33e7700 1 -- 172.21.15.92:0/3215917467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feebc103780 msgr2=0x7feebc0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:33.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.064+0000 7feec33e7700 1 --2- 172.21.15.92:0/3215917467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feebc103780 0x7feebc0fcdf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7feeac004940 tx=0x7feeac009e30 comp rx=0 tx=0).stop 2024-05-06T12:10:33.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.064+0000 7feec33e7700 1 -- 172.21.15.92:0/3215917467 shutdown_connections 2024-05-06T12:10:33.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.064+0000 7feec33e7700 1 --2- 172.21.15.92:0/3215917467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feebc103780 0x7feebc0fcdf0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:33.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.064+0000 7feec33e7700 1 --2- 172.21.15.92:0/3215917467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feebc102de0 0x7feebc1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:33.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.064+0000 7feec33e7700 1 -- 172.21.15.92:0/3215917467 >> 172.21.15.92:0/3215917467 conn(0x7feebc0f8950 msgr2=0x7feebc0fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:10:33.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.064+0000 7feec33e7700 1 -- 172.21.15.92:0/3215917467 shutdown_connections 2024-05-06T12:10:33.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.064+0000 7feec33e7700 1 -- 172.21.15.92:0/3215917467 wait complete. 2024-05-06T12:10:33.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.065+0000 7feec33e7700 1 Processor -- start 2024-05-06T12:10:33.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.065+0000 7feec33e7700 1 -- start start 2024-05-06T12:10:33.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.065+0000 7feec33e7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feebc102de0 0x7feebc113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:33.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.065+0000 7feec33e7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feebc103780 0x7feebc1140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:33.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.065+0000 7feec33e7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc114730 con 0x7feebc102de0 2024-05-06T12:10:33.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.065+0000 7feec33e7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc10db20 con 0x7feebc103780 2024-05-06T12:10:33.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.065+0000 7feec1183700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feebc102de0 0x7feebc113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:33.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.065+0000 7feec1183700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feebc102de0 0x7feebc113ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51268/0 (socket says 172.21.15.92:51268) 2024-05-06T12:10:33.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.065+0000 7feec1183700 1 -- 172.21.15.92:0/1390758341 learned_addr learned my addr 172.21.15.92:0/1390758341 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:10:33.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.065+0000 7feec0982700 1 --2- 172.21.15.92:0/1390758341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feebc103780 0x7feebc1140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:33.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.065+0000 7feec1183700 1 -- 172.21.15.92:0/1390758341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feebc103780 msgr2=0x7feebc1140e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:33.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.065+0000 7feec1183700 1 --2- 172.21.15.92:0/1390758341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feebc103780 0x7feebc1140e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:33.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.065+0000 7feec1183700 1 -- 172.21.15.92:0/1390758341 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeac005040 con 0x7feebc102de0 2024-05-06T12:10:33.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.065+0000 7feec1183700 1 --2- 172.21.15.92:0/1390758341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feebc102de0 0x7feebc113ba0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7feeb800cb10 tx=0x7feeb8007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:10:33.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.065+0000 7feeb1ffb700 1 -- 172.21.15.92:0/1390758341 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feeb8004200 con 0x7feebc102de0 2024-05-06T12:10:33.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.065+0000 7feeb1ffb700 1 -- 172.21.15.92:0/1390758341 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feeb800acc0 con 0x7feebc102de0 2024-05-06T12:10:33.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.065+0000 7feec33e7700 1 -- 172.21.15.92:0/1390758341 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feebc10de00 con 0x7feebc102de0 2024-05-06T12:10:33.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.065+0000 7feeb1ffb700 1 -- 172.21.15.92:0/1390758341 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feeb8003d90 con 0x7feebc102de0 2024-05-06T12:10:33.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.066+0000 7feec33e7700 1 -- 172.21.15.92:0/1390758341 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feebc10e3c0 con 0x7feebc102de0 2024-05-06T12:10:33.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.067+0000 7feec33e7700 1 -- 172.21.15.92:0/1390758341 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feebc0fe530 con 0x7feebc102de0 2024-05-06T12:10:33.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.067+0000 7feeb1ffb700 1 -- 172.21.15.92:0/1390758341 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7feeb8003430 con 0x7feebc102de0 2024-05-06T12:10:33.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.067+0000 7feeb1ffb700 1 --2- 172.21.15.92:0/1390758341 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7feea80692a0 0x7feea806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:33.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.068+0000 7feeb1ffb700 1 -- 172.21.15.92:0/1390758341 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7feeb80873b0 con 0x7feebc102de0 2024-05-06T12:10:33.072 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.068+0000 7feec0982700 1 --2- 172.21.15.92:0/1390758341 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7feea80692a0 0x7feea806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:33.072 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.068+0000 7feec0982700 1 --2- 172.21.15.92:0/1390758341 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7feea80692a0 0x7feea806b750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7feeac00b3b0 tx=0x7feeac009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:10:33.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.072+0000 7feeb1ffb700 1 -- 172.21.15.92:0/1390758341 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7feeb80574b0 con 0x7feebc102de0 2024-05-06T12:10:33.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:32 smithi092 ceph-mon[26954]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:10:33.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.257+0000 7feec33e7700 1 -- 172.21.15.92:0/1390758341 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feebc109ec0 con 0x7feebc102de0 2024-05-06T12:10:33.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.258+0000 7feeb1ffb700 1 -- 172.21.15.92:0/1390758341 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11080 (secure 0 0 0) 0x7feeb8057040 con 0x7feebc102de0 2024-05-06T12:10:33.261 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:10:33.262 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":40,"fsid":"aee1d912-0ba0-11ef-bc97-c7b262605968","created":"2024-05-06T12:04:51.220909+0000","modified":"2024-05-06T12:10:27.808671+0000","last_up_change":"2024-05-06T12:10:26.803552+0000","last_in_change":"2024-05-06T12:10:07.815689+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-06T12:09:12.129668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"1b276435-4698-4017-9535-4be41a4e1379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3575291733},{"type":"v1","addr":"172.21.15.92:6803","nonce":3575291733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3575291733},{"type":"v1","addr":"172.21.15.92:6805","nonce":3575291733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3575291733},{"type":"v1","addr":"172.21.15.92:6809","nonce":3575291733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3575291733},{"type":"v1","addr":"172.21.15.92:6807","nonce":3575291733}]},"public_addr":"172.21.15.92:6803/3575291733","cluster_addr":"172.21.15.92:6805/3575291733","heartbeat_back_addr":"172.21.15.92:6809/3575291733","heartbeat_front_addr":"172.21.15.92:6807/3575291733","state":["exists","up"]},{"osd":1,"uuid":"21688119-d15f-4b31-8842-090cc539c76d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":334092652},{"type":"v1","addr":"172.21.15.92:6811","nonce":334092652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":334092652},{"type":"v1","addr":"172.21.15.92:6813","nonce":334092652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":334092652},{"type":"v1","addr":"172.21.15.92:6817","nonce":334092652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":334092652},{"type":"v1","addr":"172.21.15.92:6815","nonce":334092652}]},"public_addr":"172.21.15.92:6811/334092652","cluster_addr":"172.21.15.92:6813/334092652","heartbeat_back_addr":"172.21.15.92:6817/334092652","heartbeat_front_addr":"172.21.15.92:6815/334092652","state":["exists","up"]},{"osd":2,"uuid":"74d16dfb-b118-432d-827a-59706d69c393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3428814547},{"type":"v1","addr":"172.21.15.92:6819","nonce":3428814547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3428814547},{"type":"v1","addr":"172.21.15.92:6821","nonce":3428814547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3428814547},{"type":"v1","addr":"172.21.15.92:6825","nonce":3428814547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3428814547},{"type":"v1","addr":"172.21.15.92:6823","nonce":3428814547}]},"public_addr":"172.21.15.92:6819/3428814547","cluster_addr":"172.21.15.92:6821/3428814547","heartbeat_back_addr":"172.21.15.92:6825/3428814547","heartbeat_front_addr":"172.21.15.92:6823/3428814547","state":["exists","up"]},{"osd":3,"uuid":"5a9aa5ed-2502-4c73-a8ac-cb7d0259500b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2610168113},{"type":"v1","addr":"172.21.15.190:6801","nonce":2610168113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2610168113},{"type":"v1","addr":"172.21.15.190:6803","nonce":2610168113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2610168113},{"type":"v1","addr":"172.21.15.190:6807","nonce":2610168113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2610168113},{"type":"v1","addr":"172.21.15.190:6805","nonce":2610168113}]},"public_addr":"172.21.15.190:6801/2610168113","cluster_addr":"172.21.15.190:6803/2610168113","heartbeat_back_addr":"172.21.15.190:6807/2610168113","heartbeat_front_addr":"172.21.15.190:6805/2610168113","state":["exists","up"]},{"osd":4,"uuid":"d4039e4a-c758-4632-85d6-1b7e725c0293","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1405678389},{"type":"v1","addr":"172.21.15.190:6809","nonce":1405678389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1405678389},{"type":"v1","addr":"172.21.15.190:6811","nonce":1405678389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1405678389},{"type":"v1","addr":"172.21.15.190:6815","nonce":1405678389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1405678389},{"type":"v1","addr":"172.21.15.190:6813","nonce":1405678389}]},"public_addr":"172.21.15.190:6809/1405678389","cluster_addr":"172.21.15.190:6811/1405678389","heartbeat_back_addr":"172.21.15.190:6815/1405678389","heartbeat_front_addr":"172.21.15.190:6813/1405678389","state":["exists","up"]},{"osd":5,"uuid":"ec707eee-5f24-48bb-818a-547f804fc41b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1027118727},{"type":"v1","addr":"172.21.15.190:6817","nonce":1027118727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1027118727},{"type":"v1","addr":"172.21.15.190:6819","nonce":1027118727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1027118727},{"type":"v1","addr":"172.21.15.190:6823","nonce":1027118727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1027118727},{"type":"v1","addr":"172.21.15.190:6821","nonce":1027118727}]},"public_addr":"172.21.15.190:6817/1027118727","cluster_addr":"172.21.15.190:6819/1027118727","heartbeat_back_addr":"172.21.15.190:6823/1027118727","heartbeat_front_addr":"172.21.15.190:6821/1027118727","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:08:15.348582+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:08:42.571577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:09:10.093700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:09:34.257829+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:09:59.795522+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:10:25.131646+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:6801/543287293":"2024-05-07T12:07:03.802107+0000","172.21.15.92:6800/543287293":"2024-05-07T12:07:03.802107+0000","172.21.15.92:0/1552275072":"2024-05-07T12:07:03.802107+0000","172.21.15.92:0/2440233745":"2024-05-07T12:07:03.802107+0000","172.21.15.92:6801/3731057893":"2024-05-07T12:05:56.546947+0000","172.21.15.92:0/895013058":"2024-05-07T12:05:25.954232+0000","172.21.15.92:0/758782094":"2024-05-07T12:05:25.954232+0000","172.21.15.92:0/2210153116":"2024-05-07T12:05:56.546947+0000","172.21.15.92:0/997820112":"2024-05-07T12:05:25.954232+0000","172.21.15.92:6800/3731057893":"2024-05-07T12:05:56.546947+0000","172.21.15.92:0/1407601857":"2024-05-07T12:05:56.546947+0000","172.21.15.92:0/3693315250":"2024-05-07T12:05:56.546947+0000","172.21.15.92:6800/3656822751":"2024-05-07T12:05:25.954232+0000","172.21.15.92:0/1532977673":"2024-05-07T12:07:03.802107+0000","172.21.15.92:6801/3656822751":"2024-05-07T12:05:25.954232+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-06T12:10:33.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.260+0000 7feec33e7700 1 -- 172.21.15.92:0/1390758341 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7feea80692a0 msgr2=0x7feea806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:33.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.260+0000 7feec33e7700 1 --2- 172.21.15.92:0/1390758341 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7feea80692a0 0x7feea806b750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7feeac00b3b0 tx=0x7feeac009e30 comp rx=0 tx=0).stop 2024-05-06T12:10:33.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.260+0000 7feec33e7700 1 -- 172.21.15.92:0/1390758341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feebc102de0 msgr2=0x7feebc113ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:33.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.260+0000 7feec33e7700 1 --2- 172.21.15.92:0/1390758341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feebc102de0 0x7feebc113ba0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7feeb800cb10 tx=0x7feeb8007f90 comp rx=0 tx=0).stop 2024-05-06T12:10:33.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.261+0000 7feec33e7700 1 -- 172.21.15.92:0/1390758341 shutdown_connections 2024-05-06T12:10:33.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.261+0000 7feec33e7700 1 --2- 172.21.15.92:0/1390758341 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7feea80692a0 0x7feea806b750 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:33.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.261+0000 7feec33e7700 1 --2- 172.21.15.92:0/1390758341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feebc102de0 0x7feebc113ba0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:33.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.261+0000 7feec33e7700 1 --2- 172.21.15.92:0/1390758341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feebc103780 0x7feebc1140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:33.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.261+0000 7feec33e7700 1 -- 172.21.15.92:0/1390758341 >> 172.21.15.92:0/1390758341 conn(0x7feebc0f8950 msgr2=0x7feebc0f9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:10:33.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.261+0000 7feec33e7700 1 -- 172.21.15.92:0/1390758341 shutdown_connections 2024-05-06T12:10:33.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:33.261+0000 7feec33e7700 1 -- 172.21.15.92:0/1390758341 wait complete. 2024-05-06T12:10:33.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:32 smithi190 ceph-mon[33798]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:10:33.887 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-06T12:09:12.129668+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-05-06T12:10:33.888 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-06T12:10:34.137 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:10:34.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:33 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/1390758341' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-06T12:10:34.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:33 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:10:34.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:33 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/1390758341' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-06T12:10:34.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:33 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:10:35.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:34 smithi092 ceph-mon[26954]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:10:35.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:34 smithi190 ceph-mon[33798]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:10:35.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.694+0000 7fc60867a700 1 -- 172.21.15.92:0/286515503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc600102e50 msgr2=0x7fc600103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:35.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.694+0000 7fc60867a700 1 --2- 172.21.15.92:0/286515503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc600102e50 0x7fc600103220 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc004940 tx=0x7fc5fc009e30 comp rx=0 tx=0).stop 2024-05-06T12:10:35.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.695+0000 7fc60867a700 1 -- 172.21.15.92:0/286515503 shutdown_connections 2024-05-06T12:10:35.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.695+0000 7fc60867a700 1 --2- 172.21.15.92:0/286515503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc600065640 0x7fc600065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:35.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.695+0000 7fc60867a700 1 --2- 172.21.15.92:0/286515503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc600102e50 0x7fc600103220 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:35.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.695+0000 7fc60867a700 1 -- 172.21.15.92:0/286515503 >> 172.21.15.92:0/286515503 conn(0x7fc600071d00 msgr2=0x7fc600072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:10:35.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.695+0000 7fc60867a700 1 -- 172.21.15.92:0/286515503 shutdown_connections 2024-05-06T12:10:35.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.695+0000 7fc60867a700 1 -- 172.21.15.92:0/286515503 wait complete. 2024-05-06T12:10:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.695+0000 7fc60867a700 1 Processor -- start 2024-05-06T12:10:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.695+0000 7fc60867a700 1 -- start start 2024-05-06T12:10:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.696+0000 7fc60867a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc600065640 0x7fc600113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.696+0000 7fc60867a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc600102e50 0x7fc6001140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.696+0000 7fc60867a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc600114710 con 0x7fc600065640 2024-05-06T12:10:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.696+0000 7fc60867a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60010db00 con 0x7fc600102e50 2024-05-06T12:10:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.696+0000 7fc606416700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc600065640 0x7fc600113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.696+0000 7fc606416700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc600065640 0x7fc600113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51288/0 (socket says 172.21.15.92:51288) 2024-05-06T12:10:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.696+0000 7fc606416700 1 -- 172.21.15.92:0/863575739 learned_addr learned my addr 172.21.15.92:0/863575739 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:10:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.696+0000 7fc605c15700 1 --2- 172.21.15.92:0/863575739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc600102e50 0x7fc6001140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.696+0000 7fc606416700 1 -- 172.21.15.92:0/863575739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc600102e50 msgr2=0x7fc6001140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.696+0000 7fc606416700 1 --2- 172.21.15.92:0/863575739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc600102e50 0x7fc6001140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:35.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.696+0000 7fc606416700 1 -- 172.21.15.92:0/863575739 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5fc005040 con 0x7fc600065640 2024-05-06T12:10:35.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.696+0000 7fc606416700 1 --2- 172.21.15.92:0/863575739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc600065640 0x7fc600113b80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc00b3b0 tx=0x7fc5fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:10:35.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.696+0000 7fc5f6ffd700 1 -- 172.21.15.92:0/863575739 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5fc00e040 con 0x7fc600065640 2024-05-06T12:10:35.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.696+0000 7fc60867a700 1 -- 172.21.15.92:0/863575739 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc60010dd80 con 0x7fc600065640 2024-05-06T12:10:35.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.696+0000 7fc5f6ffd700 1 -- 172.21.15.92:0/863575739 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc5fc00bbc0 con 0x7fc600065640 2024-05-06T12:10:35.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.696+0000 7fc5f6ffd700 1 -- 172.21.15.92:0/863575739 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5fc01c3f0 con 0x7fc600065640 2024-05-06T12:10:35.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.696+0000 7fc60867a700 1 -- 172.21.15.92:0/863575739 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc60010e340 con 0x7fc600065640 2024-05-06T12:10:35.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.702+0000 7fc60867a700 1 -- 172.21.15.92:0/863575739 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc600100100 con 0x7fc600065640 2024-05-06T12:10:35.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.702+0000 7fc5f6ffd700 1 -- 172.21.15.92:0/863575739 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fc5fc00bd30 con 0x7fc600065640 2024-05-06T12:10:35.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.702+0000 7fc5f6ffd700 1 --2- 172.21.15.92:0/863575739 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fc5ec0692a0 0x7fc5ec06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:35.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.702+0000 7fc5f6ffd700 1 -- 172.21.15.92:0/863575739 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc5fc088090 con 0x7fc600065640 2024-05-06T12:10:35.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.702+0000 7fc605c15700 1 --2- 172.21.15.92:0/863575739 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fc5ec0692a0 0x7fc5ec06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:35.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.703+0000 7fc605c15700 1 --2- 172.21.15.92:0/863575739 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fc5ec0692a0 0x7fc5ec06b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0007fd0 tx=0x7fc5f0009040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:10:35.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.707+0000 7fc5f6ffd700 1 -- 172.21.15.92:0/863575739 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc5fc0581e0 con 0x7fc600065640 2024-05-06T12:10:35.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.892+0000 7fc60867a700 1 -- 172.21.15.92:0/863575739 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc60010f1d0 con 0x7fc600065640 2024-05-06T12:10:35.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.893+0000 7fc5f6ffd700 1 -- 172.21.15.92:0/863575739 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc5fc057d70 con 0x7fc600065640 2024-05-06T12:10:35.896 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2024-05-06T12:10:35.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.895+0000 7fc60867a700 1 -- 172.21.15.92:0/863575739 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fc5ec0692a0 msgr2=0x7fc5ec06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:35.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.895+0000 7fc60867a700 1 --2- 172.21.15.92:0/863575739 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fc5ec0692a0 0x7fc5ec06b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0007fd0 tx=0x7fc5f0009040 comp rx=0 tx=0).stop 2024-05-06T12:10:35.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.895+0000 7fc60867a700 1 -- 172.21.15.92:0/863575739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc600065640 msgr2=0x7fc600113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:35.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.895+0000 7fc60867a700 1 --2- 172.21.15.92:0/863575739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc600065640 0x7fc600113b80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc00b3b0 tx=0x7fc5fc009e30 comp rx=0 tx=0).stop 2024-05-06T12:10:35.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.895+0000 7fc60867a700 1 -- 172.21.15.92:0/863575739 shutdown_connections 2024-05-06T12:10:35.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.895+0000 7fc60867a700 1 --2- 172.21.15.92:0/863575739 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fc5ec0692a0 0x7fc5ec06b750 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:35.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.895+0000 7fc60867a700 1 --2- 172.21.15.92:0/863575739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc600065640 0x7fc600113b80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:35.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.895+0000 7fc60867a700 1 --2- 172.21.15.92:0/863575739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc600102e50 0x7fc6001140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:35.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.895+0000 7fc60867a700 1 -- 172.21.15.92:0/863575739 >> 172.21.15.92:0/863575739 conn(0x7fc600071d00 msgr2=0x7fc6000fb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:10:35.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.895+0000 7fc60867a700 1 -- 172.21.15.92:0/863575739 shutdown_connections 2024-05-06T12:10:35.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:35.895+0000 7fc60867a700 1 -- 172.21.15.92:0/863575739 wait complete. 2024-05-06T12:10:36.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:35 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/863575739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-06T12:10:36.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:35 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/863575739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-06T12:10:36.588 INFO:tasks.cephadm:Setting up client nodes... 2024-05-06T12:10:36.589 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-06T12:10:36.844 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:10:37.188 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:36 smithi092 ceph-mon[26954]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:10:37.249 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:36 smithi190 ceph-mon[33798]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:10:38.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.473+0000 7f5354d09700 1 -- 172.21.15.92:0/3785009158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53500fff80 msgr2=0x7f5350102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:38.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.473+0000 7f5354d09700 1 --2- 172.21.15.92:0/3785009158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53500fff80 0x7f5350102360 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5344004940 tx=0x7f5344009e30 comp rx=0 tx=0).stop 2024-05-06T12:10:38.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.473+0000 7f5354d09700 1 -- 172.21.15.92:0/3785009158 shutdown_connections 2024-05-06T12:10:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.473+0000 7f5354d09700 1 --2- 172.21.15.92:0/3785009158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53500fff80 0x7f5350102360 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.473+0000 7f5354d09700 1 --2- 172.21.15.92:0/3785009158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5350066200 0x7f53500ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.473+0000 7f5354d09700 1 -- 172.21.15.92:0/3785009158 >> 172.21.15.92:0/3785009158 conn(0x7f53500f7430 msgr2=0x7f53500f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:10:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.473+0000 7f5354d09700 1 -- 172.21.15.92:0/3785009158 shutdown_connections 2024-05-06T12:10:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.473+0000 7f5354d09700 1 -- 172.21.15.92:0/3785009158 wait complete. 2024-05-06T12:10:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.474+0000 7f5354d09700 1 Processor -- start 2024-05-06T12:10:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.474+0000 7f5354d09700 1 -- start start 2024-05-06T12:10:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.474+0000 7f5354d09700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5350066200 0x7f535010d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.474+0000 7f5354d09700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53500fff80 0x7f535010df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.474+0000 7f5354d09700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535010e590 con 0x7f5350066200 2024-05-06T12:10:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.474+0000 7f5354d09700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5350112ee0 con 0x7f53500fff80 2024-05-06T12:10:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.474+0000 7f534e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5350066200 0x7f535010d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.474+0000 7f534e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5350066200 0x7f535010d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51306/0 (socket says 172.21.15.92:51306) 2024-05-06T12:10:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.474+0000 7f534e59c700 1 -- 172.21.15.92:0/191381619 learned_addr learned my addr 172.21.15.92:0/191381619 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:10:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.474+0000 7f534dd9b700 1 --2- 172.21.15.92:0/191381619 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53500fff80 0x7f535010df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:38.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.474+0000 7f534e59c700 1 -- 172.21.15.92:0/191381619 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53500fff80 msgr2=0x7f535010df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:38.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.474+0000 7f534e59c700 1 --2- 172.21.15.92:0/191381619 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53500fff80 0x7f535010df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:38.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.474+0000 7f534e59c700 1 -- 172.21.15.92:0/191381619 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5344005040 con 0x7f5350066200 2024-05-06T12:10:38.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.475+0000 7f534e59c700 1 --2- 172.21.15.92:0/191381619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5350066200 0x7f535010d9c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f534000b790 tx=0x7f5340009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:10:38.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.475+0000 7f533effd700 1 -- 172.21.15.92:0/191381619 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f534000f6f0 con 0x7f5350066200 2024-05-06T12:10:38.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.475+0000 7f533effd700 1 -- 172.21.15.92:0/191381619 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f534000fd20 con 0x7f5350066200 2024-05-06T12:10:38.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.475+0000 7f5354d09700 1 -- 172.21.15.92:0/191381619 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5350113140 con 0x7f5350066200 2024-05-06T12:10:38.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.475+0000 7f533effd700 1 -- 172.21.15.92:0/191381619 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53400103a0 con 0x7f5350066200 2024-05-06T12:10:38.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.475+0000 7f5354d09700 1 -- 172.21.15.92:0/191381619 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53501136b0 con 0x7f5350066200 2024-05-06T12:10:38.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.476+0000 7f5354d09700 1 -- 172.21.15.92:0/191381619 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f535004c710 con 0x7f5350066200 2024-05-06T12:10:38.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.476+0000 7f533effd700 1 -- 172.21.15.92:0/191381619 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f534000f850 con 0x7f5350066200 2024-05-06T12:10:38.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.476+0000 7f533effd700 1 --2- 172.21.15.92:0/191381619 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f53380692a0 0x7f533806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:38.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.477+0000 7f534dd9b700 1 --2- 172.21.15.92:0/191381619 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f53380692a0 0x7f533806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:38.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.477+0000 7f533effd700 1 -- 172.21.15.92:0/191381619 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f5340086c30 con 0x7f5350066200 2024-05-06T12:10:38.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.477+0000 7f534dd9b700 1 --2- 172.21.15.92:0/191381619 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f53380692a0 0x7f533806b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5344004890 tx=0x7f5344009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:10:38.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.481+0000 7f533effd700 1 -- 172.21.15.92:0/191381619 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5340056d30 con 0x7f5350066200 2024-05-06T12:10:38.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.746+0000 7f5354d09700 1 -- 172.21.15.92:0/191381619 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f5350105970 con 0x7f5350066200 2024-05-06T12:10:38.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.750+0000 7f533effd700 1 -- 172.21.15.92:0/191381619 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v14) v1 ==== 170+0+59 (secure 0 0 0) 0x7f53400568c0 con 0x7f5350066200 2024-05-06T12:10:38.753 INFO:teuthology.orchestra.run.smithi092.stdout:[client.0] 2024-05-06T12:10:38.753 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQC+yDhm+8qaLBAAlO2gJhoQYwnszRWIWl8CeA== 2024-05-06T12:10:38.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.752+0000 7f5354d09700 1 -- 172.21.15.92:0/191381619 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f53380692a0 msgr2=0x7f533806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:38.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.752+0000 7f5354d09700 1 --2- 172.21.15.92:0/191381619 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f53380692a0 0x7f533806b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5344004890 tx=0x7f5344009e30 comp rx=0 tx=0).stop 2024-05-06T12:10:38.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.752+0000 7f5354d09700 1 -- 172.21.15.92:0/191381619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5350066200 msgr2=0x7f535010d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:38.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.752+0000 7f5354d09700 1 --2- 172.21.15.92:0/191381619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5350066200 0x7f535010d9c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f534000b790 tx=0x7f5340009220 comp rx=0 tx=0).stop 2024-05-06T12:10:38.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.752+0000 7f5354d09700 1 -- 172.21.15.92:0/191381619 shutdown_connections 2024-05-06T12:10:38.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.752+0000 7f5354d09700 1 --2- 172.21.15.92:0/191381619 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f53380692a0 0x7f533806b750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:38.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.752+0000 7f5354d09700 1 --2- 172.21.15.92:0/191381619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5350066200 0x7f535010d9c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:38.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.752+0000 7f5354d09700 1 --2- 172.21.15.92:0/191381619 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53500fff80 0x7f535010df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:38.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.752+0000 7f5354d09700 1 -- 172.21.15.92:0/191381619 >> 172.21.15.92:0/191381619 conn(0x7f53500f7430 msgr2=0x7f53500f7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:10:38.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.752+0000 7f5354d09700 1 -- 172.21.15.92:0/191381619 shutdown_connections 2024-05-06T12:10:38.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:38.752+0000 7f5354d09700 1 -- 172.21.15.92:0/191381619 wait complete. 2024-05-06T12:10:38.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:38 smithi092 ceph-mon[26954]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:10:38.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:38 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/191381619' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-06T12:10:38.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:38 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/191381619' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-06T12:10:39.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:38 smithi190 ceph-mon[33798]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:10:39.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:38 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/191381619' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-06T12:10:39.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:38 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/191381619' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-06T12:10:39.458 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-06T12:10:39.459 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-06T12:10:39.459 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-06T12:10:39.494 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-06T12:10:39.757 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi190/config 2024-05-06T12:10:41.053 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:40 smithi190 ceph-mon[33798]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:10:41.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:40 smithi092 ceph-mon[26954]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:10:41.255 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.251+0000 7fca477d7700 1 -- 172.21.15.190:0/3306518784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca40066e90 msgr2=0x7fca400ffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:41.256 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.251+0000 7fca477d7700 1 --2- 172.21.15.190:0/3306518784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca40066e90 0x7fca400ffa50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fca38004940 tx=0x7fca38009e30 comp rx=0 tx=0).stop 2024-05-06T12:10:41.256 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.251+0000 7fca477d7700 1 -- 172.21.15.190:0/3306518784 shutdown_connections 2024-05-06T12:10:41.256 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.251+0000 7fca477d7700 1 --2- 172.21.15.190:0/3306518784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca400fff90 0x7fca40102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:41.256 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.251+0000 7fca477d7700 1 --2- 172.21.15.190:0/3306518784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca40066e90 0x7fca400ffa50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:41.256 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.251+0000 7fca477d7700 1 -- 172.21.15.190:0/3306518784 >> 172.21.15.190:0/3306518784 conn(0x7fca400f7440 msgr2=0x7fca400f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:10:41.256 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.251+0000 7fca477d7700 1 -- 172.21.15.190:0/3306518784 shutdown_connections 2024-05-06T12:10:41.256 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.251+0000 7fca477d7700 1 -- 172.21.15.190:0/3306518784 wait complete. 2024-05-06T12:10:41.256 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.252+0000 7fca477d7700 1 Processor -- start 2024-05-06T12:10:41.256 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.252+0000 7fca477d7700 1 -- start start 2024-05-06T12:10:41.256 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.252+0000 7fca477d7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca40066e90 0x7fca4010d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:41.256 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.252+0000 7fca477d7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca400fff90 0x7fca4010ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:41.256 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.252+0000 7fca477d7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4010e560 con 0x7fca40066e90 2024-05-06T12:10:41.256 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.252+0000 7fca477d7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca40112eb0 con 0x7fca400fff90 2024-05-06T12:10:41.257 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.252+0000 7fca44d72700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca400fff90 0x7fca4010ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:41.257 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.252+0000 7fca44d72700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca400fff90 0x7fca4010ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:56576/0 (socket says 172.21.15.190:56576) 2024-05-06T12:10:41.257 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.252+0000 7fca44d72700 1 -- 172.21.15.190:0/1325080814 learned_addr learned my addr 172.21.15.190:0/1325080814 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-06T12:10:41.257 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.252+0000 7fca45573700 1 --2- 172.21.15.190:0/1325080814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca40066e90 0x7fca4010d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:41.257 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.252+0000 7fca44d72700 1 -- 172.21.15.190:0/1325080814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca40066e90 msgr2=0x7fca4010d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:41.257 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.252+0000 7fca44d72700 1 --2- 172.21.15.190:0/1325080814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca40066e90 0x7fca4010d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:41.257 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.252+0000 7fca44d72700 1 -- 172.21.15.190:0/1325080814 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca38005040 con 0x7fca400fff90 2024-05-06T12:10:41.257 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.252+0000 7fca44d72700 1 --2- 172.21.15.190:0/1325080814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca400fff90 0x7fca4010ded0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fca3000b790 tx=0x7fca30009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:10:41.257 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.253+0000 7fca35ffb700 1 -- 172.21.15.190:0/1325080814 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca3000f770 con 0x7fca400fff90 2024-05-06T12:10:41.257 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.253+0000 7fca35ffb700 1 -- 172.21.15.190:0/1325080814 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca3000fda0 con 0x7fca400fff90 2024-05-06T12:10:41.257 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.253+0000 7fca477d7700 1 -- 172.21.15.190:0/1325080814 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca40113110 con 0x7fca400fff90 2024-05-06T12:10:41.257 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.253+0000 7fca35ffb700 1 -- 172.21.15.190:0/1325080814 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca3000d310 con 0x7fca400fff90 2024-05-06T12:10:41.257 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.253+0000 7fca477d7700 1 -- 172.21.15.190:0/1325080814 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca40113680 con 0x7fca400fff90 2024-05-06T12:10:41.258 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.254+0000 7fca477d7700 1 -- 172.21.15.190:0/1325080814 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca4004c710 con 0x7fca400fff90 2024-05-06T12:10:41.258 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.254+0000 7fca35ffb700 1 -- 172.21.15.190:0/1325080814 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fca30010420 con 0x7fca400fff90 2024-05-06T12:10:41.258 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.254+0000 7fca35ffb700 1 --2- 172.21.15.190:0/1325080814 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fca2c0692b0 0x7fca2c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:41.259 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.254+0000 7fca35ffb700 1 -- 172.21.15.190:0/1325080814 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fca30086d70 con 0x7fca400fff90 2024-05-06T12:10:41.259 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.255+0000 7fca45573700 1 --2- 172.21.15.190:0/1325080814 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fca2c0692b0 0x7fca2c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:41.259 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.255+0000 7fca45573700 1 --2- 172.21.15.190:0/1325080814 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fca2c0692b0 0x7fca2c06b760 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fca38009fd0 tx=0x7fca38009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:10:41.261 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.259+0000 7fca35ffb700 1 -- 172.21.15.190:0/1325080814 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fca30056f10 con 0x7fca400fff90 2024-05-06T12:10:41.522 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.520+0000 7fca477d7700 1 -- 172.21.15.190:0/1325080814 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fca40113b30 con 0x7fca400fff90 2024-05-06T12:10:41.527 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.525+0000 7fca35ffb700 1 -- 172.21.15.190:0/1325080814 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v15) v1 ==== 170+0+59 (secure 0 0 0) 0x7fca30056aa0 con 0x7fca400fff90 2024-05-06T12:10:41.527 INFO:teuthology.orchestra.run.smithi190.stdout:[client.1] 2024-05-06T12:10:41.528 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQDByDhmuAMlHxAAb7FZUGM1hUeKXb2rsBB5ZQ== 2024-05-06T12:10:41.529 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.527+0000 7fca477d7700 1 -- 172.21.15.190:0/1325080814 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fca2c0692b0 msgr2=0x7fca2c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:41.529 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.527+0000 7fca477d7700 1 --2- 172.21.15.190:0/1325080814 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fca2c0692b0 0x7fca2c06b760 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fca38009fd0 tx=0x7fca38009e30 comp rx=0 tx=0).stop 2024-05-06T12:10:41.529 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.527+0000 7fca477d7700 1 -- 172.21.15.190:0/1325080814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca400fff90 msgr2=0x7fca4010ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:41.530 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.527+0000 7fca477d7700 1 --2- 172.21.15.190:0/1325080814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca400fff90 0x7fca4010ded0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fca3000b790 tx=0x7fca30009250 comp rx=0 tx=0).stop 2024-05-06T12:10:41.530 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.527+0000 7fca477d7700 1 -- 172.21.15.190:0/1325080814 shutdown_connections 2024-05-06T12:10:41.530 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.527+0000 7fca477d7700 1 --2- 172.21.15.190:0/1325080814 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fca2c0692b0 0x7fca2c06b760 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:41.530 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.527+0000 7fca477d7700 1 --2- 172.21.15.190:0/1325080814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca40066e90 0x7fca4010d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:41.530 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.527+0000 7fca477d7700 1 --2- 172.21.15.190:0/1325080814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca400fff90 0x7fca4010ded0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:41.530 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.527+0000 7fca477d7700 1 -- 172.21.15.190:0/1325080814 >> 172.21.15.190:0/1325080814 conn(0x7fca400f7440 msgr2=0x7fca400f7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:10:41.530 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.527+0000 7fca477d7700 1 -- 172.21.15.190:0/1325080814 shutdown_connections 2024-05-06T12:10:41.530 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:10:41.527+0000 7fca477d7700 1 -- 172.21.15.190:0/1325080814 wait complete. 2024-05-06T12:10:42.131 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-06T12:10:42.131 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-05-06T12:10:42.131 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-06T12:10:42.167 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:41 smithi190 ceph-mon[33798]: from='client.? 172.21.15.190:0/1325080814' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-06T12:10:42.167 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:41 smithi190 ceph-mon[33798]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-06T12:10:42.167 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:41 smithi190 ceph-mon[33798]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-06T12:10:42.169 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-06T12:10:42.170 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-06T12:10:42.170 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph mgr dump --format=json 2024-05-06T12:10:42.195 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:41 smithi092 ceph-mon[26954]: from='client.? 172.21.15.190:0/1325080814' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-06T12:10:42.196 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:41 smithi092 ceph-mon[26954]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-06T12:10:42.196 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:41 smithi092 ceph-mon[26954]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-06T12:10:42.430 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:10:43.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:42 smithi092 ceph-mon[26954]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:10:43.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:42 smithi190 ceph-mon[33798]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:10:44.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.089+0000 7fcf1d8f5700 1 -- 172.21.15.92:0/616702496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf18102de0 msgr2=0x7fcf181031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:44.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.089+0000 7fcf1d8f5700 1 --2- 172.21.15.92:0/616702496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf18102de0 0x7fcf181031b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcf08004940 tx=0x7fcf08009e30 comp rx=0 tx=0).stop 2024-05-06T12:10:44.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.089+0000 7fcf1d8f5700 1 -- 172.21.15.92:0/616702496 shutdown_connections 2024-05-06T12:10:44.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.089+0000 7fcf1d8f5700 1 --2- 172.21.15.92:0/616702496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf18103780 0x7fcf180fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:44.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.089+0000 7fcf1d8f5700 1 --2- 172.21.15.92:0/616702496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf18102de0 0x7fcf181031b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:44.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.089+0000 7fcf1d8f5700 1 -- 172.21.15.92:0/616702496 >> 172.21.15.92:0/616702496 conn(0x7fcf180f8990 msgr2=0x7fcf180fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:10:44.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.090+0000 7fcf1d8f5700 1 -- 172.21.15.92:0/616702496 shutdown_connections 2024-05-06T12:10:44.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.090+0000 7fcf1d8f5700 1 -- 172.21.15.92:0/616702496 wait complete. 2024-05-06T12:10:44.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.090+0000 7fcf1d8f5700 1 Processor -- start 2024-05-06T12:10:44.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.090+0000 7fcf1d8f5700 1 -- start start 2024-05-06T12:10:44.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.090+0000 7fcf1d8f5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf18102de0 0x7fcf18113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:44.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.090+0000 7fcf1d8f5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf18103780 0x7fcf18113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:44.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.090+0000 7fcf1d8f5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf18114500 con 0x7fcf18103780 2024-05-06T12:10:44.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.090+0000 7fcf1d8f5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1810db50 con 0x7fcf18102de0 2024-05-06T12:10:44.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.091+0000 7fcf167fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf18103780 0x7fcf18113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:44.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.091+0000 7fcf167fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf18103780 0x7fcf18113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39898/0 (socket says 172.21.15.92:39898) 2024-05-06T12:10:44.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.091+0000 7fcf167fc700 1 -- 172.21.15.92:0/3457634829 learned_addr learned my addr 172.21.15.92:0/3457634829 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:10:44.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.091+0000 7fcf16ffd700 1 --2- 172.21.15.92:0/3457634829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf18102de0 0x7fcf18113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:44.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.091+0000 7fcf167fc700 1 -- 172.21.15.92:0/3457634829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf18102de0 msgr2=0x7fcf18113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:44.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.091+0000 7fcf167fc700 1 --2- 172.21.15.92:0/3457634829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf18102de0 0x7fcf18113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:44.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.091+0000 7fcf167fc700 1 -- 172.21.15.92:0/3457634829 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf08005040 con 0x7fcf18103780 2024-05-06T12:10:44.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.091+0000 7fcf167fc700 1 --2- 172.21.15.92:0/3457634829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf18103780 0x7fcf18113eb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c002a50 tx=0x7fcf0c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:10:44.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.091+0000 7fceff7fe700 1 -- 172.21.15.92:0/3457634829 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf0c007cc0 con 0x7fcf18103780 2024-05-06T12:10:44.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.091+0000 7fceff7fe700 1 -- 172.21.15.92:0/3457634829 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf0c00f040 con 0x7fcf18103780 2024-05-06T12:10:44.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.091+0000 7fcf1d8f5700 1 -- 172.21.15.92:0/3457634829 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf1810de30 con 0x7fcf18103780 2024-05-06T12:10:44.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.091+0000 7fceff7fe700 1 -- 172.21.15.92:0/3457634829 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf0c00d9c0 con 0x7fcf18103780 2024-05-06T12:10:44.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.091+0000 7fcf1d8f5700 1 -- 172.21.15.92:0/3457634829 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf1810e3f0 con 0x7fcf18103780 2024-05-06T12:10:44.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.092+0000 7fcf1d8f5700 1 -- 172.21.15.92:0/3457634829 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf181000f0 con 0x7fcf18103780 2024-05-06T12:10:44.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.093+0000 7fceff7fe700 1 -- 172.21.15.92:0/3457634829 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fcf0c009730 con 0x7fcf18103780 2024-05-06T12:10:44.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.093+0000 7fceff7fe700 1 --2- 172.21.15.92:0/3457634829 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fcf000692a0 0x7fcf0006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:44.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.093+0000 7fcf16ffd700 1 --2- 172.21.15.92:0/3457634829 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fcf000692a0 0x7fcf0006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:44.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.093+0000 7fceff7fe700 1 -- 172.21.15.92:0/3457634829 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fcf0c086b30 con 0x7fcf18103780 2024-05-06T12:10:44.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.093+0000 7fcf16ffd700 1 --2- 172.21.15.92:0/3457634829 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fcf000692a0 0x7fcf0006b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcf08004f90 tx=0x7fcf08009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:10:44.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.097+0000 7fceff7fe700 1 -- 172.21.15.92:0/3457634829 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fcf0c056c30 con 0x7fcf18103780 2024-05-06T12:10:44.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.336+0000 7fcf1d8f5700 1 -- 172.21.15.92:0/3457634829 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fcf180fe530 con 0x7fcf18103780 2024-05-06T12:10:44.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.339+0000 7fceff7fe700 1 -- 172.21.15.92:0/3457634829 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+167178 (secure 0 0 0) 0x7fcf0c0567c0 con 0x7fcf18103780 2024-05-06T12:10:44.345 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:10:44.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.343+0000 7fcf1d8f5700 1 -- 172.21.15.92:0/3457634829 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fcf000692a0 msgr2=0x7fcf0006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:44.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.343+0000 7fcf1d8f5700 1 --2- 172.21.15.92:0/3457634829 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fcf000692a0 0x7fcf0006b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcf08004f90 tx=0x7fcf08009e30 comp rx=0 tx=0).stop 2024-05-06T12:10:44.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.343+0000 7fcf1d8f5700 1 -- 172.21.15.92:0/3457634829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf18103780 msgr2=0x7fcf18113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:44.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.343+0000 7fcf1d8f5700 1 --2- 172.21.15.92:0/3457634829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf18103780 0x7fcf18113eb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c002a50 tx=0x7fcf0c004600 comp rx=0 tx=0).stop 2024-05-06T12:10:44.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.343+0000 7fcf1d8f5700 1 -- 172.21.15.92:0/3457634829 shutdown_connections 2024-05-06T12:10:44.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.343+0000 7fcf1d8f5700 1 --2- 172.21.15.92:0/3457634829 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fcf000692a0 0x7fcf0006b750 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:44.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.343+0000 7fcf1d8f5700 1 --2- 172.21.15.92:0/3457634829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf18102de0 0x7fcf18113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:44.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.343+0000 7fcf1d8f5700 1 --2- 172.21.15.92:0/3457634829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf18103780 0x7fcf18113eb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:44.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.343+0000 7fcf1d8f5700 1 -- 172.21.15.92:0/3457634829 >> 172.21.15.92:0/3457634829 conn(0x7fcf180f8990 msgr2=0x7fcf18107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:10:44.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.343+0000 7fcf1d8f5700 1 -- 172.21.15.92:0/3457634829 shutdown_connections 2024-05-06T12:10:44.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:44.343+0000 7fcf1d8f5700 1 -- 172.21.15.92:0/3457634829 wait complete. 2024-05-06T12:10:45.040 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":19,"flags":0,"active_gid":14210,"active_name":"smithi092.gyzeqb","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3045905943},{"type":"v1","addr":"172.21.15.92:6801","nonce":3045905943}]},"active_addr":"172.21.15.92:6801/3045905943","active_change":"2024-05-06T12:07:03.802232+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":14230,"name":"smithi190.kczxkh","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.92:8443/","prometheus":"http://172.21.15.92:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":3417170577}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":3493498366}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":856987777}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":4121052357}]}]} 2024-05-06T12:10:45.043 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-06T12:10:45.043 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-06T12:10:45.044 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-06T12:10:45.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:44 smithi092 ceph-mon[26954]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:10:45.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:44 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/3457634829' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-06T12:10:45.339 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:10:45.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:44 smithi190 ceph-mon[33798]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:10:45.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:44 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/3457634829' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-06T12:10:46.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.863+0000 7f2a84857700 1 -- 172.21.15.92:0/404478595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a80101d00 msgr2=0x7f2a801020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:46.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.863+0000 7f2a84857700 1 --2- 172.21.15.92:0/404478595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a80101d00 0x7f2a801020d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2a70004940 tx=0x7f2a70009e30 comp rx=0 tx=0).stop 2024-05-06T12:10:46.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.863+0000 7f2a84857700 1 -- 172.21.15.92:0/404478595 shutdown_connections 2024-05-06T12:10:46.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.863+0000 7f2a84857700 1 --2- 172.21.15.92:0/404478595 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a801026a0 0x7f2a800fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:46.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.863+0000 7f2a84857700 1 --2- 172.21.15.92:0/404478595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a80101d00 0x7f2a801020d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:46.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.863+0000 7f2a84857700 1 -- 172.21.15.92:0/404478595 >> 172.21.15.92:0/404478595 conn(0x7f2a800f7870 msgr2=0x7f2a800f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:10:46.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.863+0000 7f2a84857700 1 -- 172.21.15.92:0/404478595 shutdown_connections 2024-05-06T12:10:46.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.863+0000 7f2a84857700 1 -- 172.21.15.92:0/404478595 wait complete. 2024-05-06T12:10:46.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.863+0000 7f2a84857700 1 Processor -- start 2024-05-06T12:10:46.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.864+0000 7f2a84857700 1 -- start start 2024-05-06T12:10:46.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.864+0000 7f2a84857700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a80101d00 0x7f2a80102b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:46.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.864+0000 7f2a84857700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a801026a0 0x7f2a801030a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:46.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.864+0000 7f2a84857700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a80104b00 con 0x7f2a801026a0 2024-05-06T12:10:46.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.864+0000 7f2a84857700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a801035e0 con 0x7f2a80101d00 2024-05-06T12:10:46.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.864+0000 7f2a7dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a801026a0 0x7f2a801030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:46.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.864+0000 7f2a7dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a801026a0 0x7f2a801030a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39914/0 (socket says 172.21.15.92:39914) 2024-05-06T12:10:46.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.864+0000 7f2a7dd9b700 1 -- 172.21.15.92:0/682776846 learned_addr learned my addr 172.21.15.92:0/682776846 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:10:46.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.864+0000 7f2a7dd9b700 1 -- 172.21.15.92:0/682776846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a80101d00 msgr2=0x7f2a80102b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:46.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.864+0000 7f2a7e59c700 1 --2- 172.21.15.92:0/682776846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a80101d00 0x7f2a80102b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:46.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.864+0000 7f2a7dd9b700 1 --2- 172.21.15.92:0/682776846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a80101d00 0x7f2a80102b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:46.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.864+0000 7f2a7dd9b700 1 -- 172.21.15.92:0/682776846 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a70005040 con 0x7f2a801026a0 2024-05-06T12:10:46.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.864+0000 7f2a7dd9b700 1 --2- 172.21.15.92:0/682776846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a801026a0 0x7f2a801030a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400c800 tx=0x7f2a74004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:10:46.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.864+0000 7f2a6effd700 1 -- 172.21.15.92:0/682776846 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a7400a760 con 0x7f2a801026a0 2024-05-06T12:10:46.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.864+0000 7f2a84857700 1 -- 172.21.15.92:0/682776846 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a80103890 con 0x7f2a801026a0 2024-05-06T12:10:46.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.864+0000 7f2a6effd700 1 -- 172.21.15.92:0/682776846 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a7400ad90 con 0x7f2a801026a0 2024-05-06T12:10:46.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.864+0000 7f2a6effd700 1 -- 172.21.15.92:0/682776846 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a740113a0 con 0x7f2a801026a0 2024-05-06T12:10:46.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.865+0000 7f2a84857700 1 -- 172.21.15.92:0/682776846 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a8011a0f0 con 0x7f2a801026a0 2024-05-06T12:10:46.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.865+0000 7f2a84857700 1 -- 172.21.15.92:0/682776846 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a800fd450 con 0x7f2a801026a0 2024-05-06T12:10:46.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.866+0000 7f2a6effd700 1 -- 172.21.15.92:0/682776846 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f2a74012400 con 0x7f2a801026a0 2024-05-06T12:10:46.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.866+0000 7f2a6effd700 1 --2- 172.21.15.92:0/682776846 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f2a680692a0 0x7f2a6806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:46.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.866+0000 7f2a6effd700 1 -- 172.21.15.92:0/682776846 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f2a74086ea0 con 0x7f2a801026a0 2024-05-06T12:10:46.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.866+0000 7f2a7e59c700 1 --2- 172.21.15.92:0/682776846 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f2a680692a0 0x7f2a6806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:46.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.867+0000 7f2a7e59c700 1 --2- 172.21.15.92:0/682776846 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f2a680692a0 0x7f2a6806b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2a70004f90 tx=0x7f2a70009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:10:46.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:46.871+0000 7f2a6effd700 1 -- 172.21.15.92:0/682776846 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2a74053510 con 0x7f2a801026a0 2024-05-06T12:10:47.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:47.057+0000 7f2a84857700 1 -- 172.21.15.92:0/682776846 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a8004c710 con 0x7f2a801026a0 2024-05-06T12:10:47.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:47.058+0000 7f2a6effd700 1 -- 172.21.15.92:0/682776846 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11080 (secure 0 0 0) 0x7f2a8004c710 con 0x7f2a801026a0 2024-05-06T12:10:47.062 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:10:47.062 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":40,"fsid":"aee1d912-0ba0-11ef-bc97-c7b262605968","created":"2024-05-06T12:04:51.220909+0000","modified":"2024-05-06T12:10:27.808671+0000","last_up_change":"2024-05-06T12:10:26.803552+0000","last_in_change":"2024-05-06T12:10:07.815689+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-06T12:09:12.129668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"1b276435-4698-4017-9535-4be41a4e1379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3575291733},{"type":"v1","addr":"172.21.15.92:6803","nonce":3575291733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3575291733},{"type":"v1","addr":"172.21.15.92:6805","nonce":3575291733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3575291733},{"type":"v1","addr":"172.21.15.92:6809","nonce":3575291733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3575291733},{"type":"v1","addr":"172.21.15.92:6807","nonce":3575291733}]},"public_addr":"172.21.15.92:6803/3575291733","cluster_addr":"172.21.15.92:6805/3575291733","heartbeat_back_addr":"172.21.15.92:6809/3575291733","heartbeat_front_addr":"172.21.15.92:6807/3575291733","state":["exists","up"]},{"osd":1,"uuid":"21688119-d15f-4b31-8842-090cc539c76d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":334092652},{"type":"v1","addr":"172.21.15.92:6811","nonce":334092652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":334092652},{"type":"v1","addr":"172.21.15.92:6813","nonce":334092652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":334092652},{"type":"v1","addr":"172.21.15.92:6817","nonce":334092652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":334092652},{"type":"v1","addr":"172.21.15.92:6815","nonce":334092652}]},"public_addr":"172.21.15.92:6811/334092652","cluster_addr":"172.21.15.92:6813/334092652","heartbeat_back_addr":"172.21.15.92:6817/334092652","heartbeat_front_addr":"172.21.15.92:6815/334092652","state":["exists","up"]},{"osd":2,"uuid":"74d16dfb-b118-432d-827a-59706d69c393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3428814547},{"type":"v1","addr":"172.21.15.92:6819","nonce":3428814547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3428814547},{"type":"v1","addr":"172.21.15.92:6821","nonce":3428814547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3428814547},{"type":"v1","addr":"172.21.15.92:6825","nonce":3428814547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3428814547},{"type":"v1","addr":"172.21.15.92:6823","nonce":3428814547}]},"public_addr":"172.21.15.92:6819/3428814547","cluster_addr":"172.21.15.92:6821/3428814547","heartbeat_back_addr":"172.21.15.92:6825/3428814547","heartbeat_front_addr":"172.21.15.92:6823/3428814547","state":["exists","up"]},{"osd":3,"uuid":"5a9aa5ed-2502-4c73-a8ac-cb7d0259500b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2610168113},{"type":"v1","addr":"172.21.15.190:6801","nonce":2610168113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2610168113},{"type":"v1","addr":"172.21.15.190:6803","nonce":2610168113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2610168113},{"type":"v1","addr":"172.21.15.190:6807","nonce":2610168113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2610168113},{"type":"v1","addr":"172.21.15.190:6805","nonce":2610168113}]},"public_addr":"172.21.15.190:6801/2610168113","cluster_addr":"172.21.15.190:6803/2610168113","heartbeat_back_addr":"172.21.15.190:6807/2610168113","heartbeat_front_addr":"172.21.15.190:6805/2610168113","state":["exists","up"]},{"osd":4,"uuid":"d4039e4a-c758-4632-85d6-1b7e725c0293","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1405678389},{"type":"v1","addr":"172.21.15.190:6809","nonce":1405678389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1405678389},{"type":"v1","addr":"172.21.15.190:6811","nonce":1405678389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1405678389},{"type":"v1","addr":"172.21.15.190:6815","nonce":1405678389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1405678389},{"type":"v1","addr":"172.21.15.190:6813","nonce":1405678389}]},"public_addr":"172.21.15.190:6809/1405678389","cluster_addr":"172.21.15.190:6811/1405678389","heartbeat_back_addr":"172.21.15.190:6815/1405678389","heartbeat_front_addr":"172.21.15.190:6813/1405678389","state":["exists","up"]},{"osd":5,"uuid":"ec707eee-5f24-48bb-818a-547f804fc41b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1027118727},{"type":"v1","addr":"172.21.15.190:6817","nonce":1027118727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1027118727},{"type":"v1","addr":"172.21.15.190:6819","nonce":1027118727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1027118727},{"type":"v1","addr":"172.21.15.190:6823","nonce":1027118727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1027118727},{"type":"v1","addr":"172.21.15.190:6821","nonce":1027118727}]},"public_addr":"172.21.15.190:6817/1027118727","cluster_addr":"172.21.15.190:6819/1027118727","heartbeat_back_addr":"172.21.15.190:6823/1027118727","heartbeat_front_addr":"172.21.15.190:6821/1027118727","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:08:15.348582+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:08:42.571577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:09:10.093700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:09:34.257829+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:09:59.795522+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:10:25.131646+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:6801/543287293":"2024-05-07T12:07:03.802107+0000","172.21.15.92:6800/543287293":"2024-05-07T12:07:03.802107+0000","172.21.15.92:0/1552275072":"2024-05-07T12:07:03.802107+0000","172.21.15.92:0/2440233745":"2024-05-07T12:07:03.802107+0000","172.21.15.92:6801/3731057893":"2024-05-07T12:05:56.546947+0000","172.21.15.92:0/895013058":"2024-05-07T12:05:25.954232+0000","172.21.15.92:0/758782094":"2024-05-07T12:05:25.954232+0000","172.21.15.92:0/2210153116":"2024-05-07T12:05:56.546947+0000","172.21.15.92:0/997820112":"2024-05-07T12:05:25.954232+0000","172.21.15.92:6800/3731057893":"2024-05-07T12:05:56.546947+0000","172.21.15.92:0/1407601857":"2024-05-07T12:05:56.546947+0000","172.21.15.92:0/3693315250":"2024-05-07T12:05:56.546947+0000","172.21.15.92:6800/3656822751":"2024-05-07T12:05:25.954232+0000","172.21.15.92:0/1532977673":"2024-05-07T12:07:03.802107+0000","172.21.15.92:6801/3656822751":"2024-05-07T12:05:25.954232+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-06T12:10:47.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:47.060+0000 7f2a84857700 1 -- 172.21.15.92:0/682776846 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f2a680692a0 msgr2=0x7f2a6806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:47.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:47.060+0000 7f2a84857700 1 --2- 172.21.15.92:0/682776846 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f2a680692a0 0x7f2a6806b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2a70004f90 tx=0x7f2a70009e30 comp rx=0 tx=0).stop 2024-05-06T12:10:47.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:47.060+0000 7f2a84857700 1 -- 172.21.15.92:0/682776846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a801026a0 msgr2=0x7f2a801030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:47.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:47.060+0000 7f2a84857700 1 --2- 172.21.15.92:0/682776846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a801026a0 0x7f2a801030a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400c800 tx=0x7f2a74004470 comp rx=0 tx=0).stop 2024-05-06T12:10:47.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:47.060+0000 7f2a84857700 1 -- 172.21.15.92:0/682776846 shutdown_connections 2024-05-06T12:10:47.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:47.060+0000 7f2a84857700 1 --2- 172.21.15.92:0/682776846 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f2a680692a0 0x7f2a6806b750 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:47.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:47.060+0000 7f2a84857700 1 --2- 172.21.15.92:0/682776846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a80101d00 0x7f2a80102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:47.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:47.060+0000 7f2a84857700 1 --2- 172.21.15.92:0/682776846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a801026a0 0x7f2a801030a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:47.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:47.060+0000 7f2a84857700 1 -- 172.21.15.92:0/682776846 >> 172.21.15.92:0/682776846 conn(0x7f2a800f7870 msgr2=0x7f2a800f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:10:47.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:47.060+0000 7f2a84857700 1 -- 172.21.15.92:0/682776846 shutdown_connections 2024-05-06T12:10:47.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:47.060+0000 7f2a84857700 1 -- 172.21.15.92:0/682776846 wait complete. 2024-05-06T12:10:47.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:46 smithi092 ceph-mon[26954]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:10:47.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:46 smithi190 ceph-mon[33798]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:10:47.645 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-06T12:10:47.645 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-06T12:10:47.895 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:10:48.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:47 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/682776846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-06T12:10:48.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:47 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/682776846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-06T12:10:49.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:48 smithi190 ceph-mon[33798]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:10:49.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:48 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:10:49.183 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:48 smithi092 ceph-mon[26954]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:10:49.183 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:48 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:10:49.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.501+0000 7ff524c5a700 1 -- 172.21.15.92:0/529340918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff520103780 msgr2=0x7ff5200fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:49.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.501+0000 7ff524c5a700 1 --2- 172.21.15.92:0/529340918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff520103780 0x7ff5200fcdf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff514004940 tx=0x7ff514009e30 comp rx=0 tx=0).stop 2024-05-06T12:10:49.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.501+0000 7ff524c5a700 1 -- 172.21.15.92:0/529340918 shutdown_connections 2024-05-06T12:10:49.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.501+0000 7ff524c5a700 1 --2- 172.21.15.92:0/529340918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff520103780 0x7ff5200fcdf0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:49.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.501+0000 7ff524c5a700 1 --2- 172.21.15.92:0/529340918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff520102de0 0x7ff5201031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:49.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.501+0000 7ff524c5a700 1 -- 172.21.15.92:0/529340918 >> 172.21.15.92:0/529340918 conn(0x7ff5200f8990 msgr2=0x7ff5200fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:10:49.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.501+0000 7ff524c5a700 1 -- 172.21.15.92:0/529340918 shutdown_connections 2024-05-06T12:10:49.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.501+0000 7ff524c5a700 1 -- 172.21.15.92:0/529340918 wait complete. 2024-05-06T12:10:49.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.502+0000 7ff524c5a700 1 Processor -- start 2024-05-06T12:10:49.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.502+0000 7ff524c5a700 1 -- start start 2024-05-06T12:10:49.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.502+0000 7ff524c5a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff520102de0 0x7ff520113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:49.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.502+0000 7ff524c5a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff520103780 0x7ff520113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:49.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.502+0000 7ff524c5a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff520114520 con 0x7ff520103780 2024-05-06T12:10:49.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.502+0000 7ff524c5a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52010db20 con 0x7ff520102de0 2024-05-06T12:10:49.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.502+0000 7ff51dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff520103780 0x7ff520113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:49.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.502+0000 7ff51dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff520103780 0x7ff520113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39922/0 (socket says 172.21.15.92:39922) 2024-05-06T12:10:49.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.502+0000 7ff51dd9b700 1 -- 172.21.15.92:0/254214942 learned_addr learned my addr 172.21.15.92:0/254214942 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:10:49.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.502+0000 7ff51e59c700 1 --2- 172.21.15.92:0/254214942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff520102de0 0x7ff520113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:49.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.502+0000 7ff51dd9b700 1 -- 172.21.15.92:0/254214942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff520102de0 msgr2=0x7ff520113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:49.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.502+0000 7ff51dd9b700 1 --2- 172.21.15.92:0/254214942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff520102de0 0x7ff520113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:49.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.502+0000 7ff51dd9b700 1 -- 172.21.15.92:0/254214942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff514005040 con 0x7ff520103780 2024-05-06T12:10:49.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.502+0000 7ff51e59c700 1 --2- 172.21.15.92:0/254214942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff520102de0 0x7ff520113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-06T12:10:49.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.502+0000 7ff51dd9b700 1 --2- 172.21.15.92:0/254214942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff520103780 0x7ff520113ed0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff514009100 tx=0x7ff514009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:10:49.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.502+0000 7ff50effd700 1 -- 172.21.15.92:0/254214942 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff51400e040 con 0x7ff520103780 2024-05-06T12:10:49.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.503+0000 7ff524c5a700 1 -- 172.21.15.92:0/254214942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff52010dda0 con 0x7ff520103780 2024-05-06T12:10:49.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.503+0000 7ff50effd700 1 -- 172.21.15.92:0/254214942 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff514013420 con 0x7ff520103780 2024-05-06T12:10:49.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.503+0000 7ff50effd700 1 -- 172.21.15.92:0/254214942 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff51401c610 con 0x7ff520103780 2024-05-06T12:10:49.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.503+0000 7ff524c5a700 1 -- 172.21.15.92:0/254214942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff52010e360 con 0x7ff520103780 2024-05-06T12:10:49.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.503+0000 7ff524c5a700 1 -- 172.21.15.92:0/254214942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5200fe530 con 0x7ff520103780 2024-05-06T12:10:49.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.504+0000 7ff50effd700 1 -- 172.21.15.92:0/254214942 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7ff514013590 con 0x7ff520103780 2024-05-06T12:10:49.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.504+0000 7ff50effd700 1 --2- 172.21.15.92:0/254214942 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7ff5080692a0 0x7ff50806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:49.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.504+0000 7ff50effd700 1 -- 172.21.15.92:0/254214942 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ff5140880e0 con 0x7ff520103780 2024-05-06T12:10:49.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.504+0000 7ff51e59c700 1 --2- 172.21.15.92:0/254214942 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7ff5080692a0 0x7ff50806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:49.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.505+0000 7ff51e59c700 1 --2- 172.21.15.92:0/254214942 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7ff5080692a0 0x7ff50806b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff510009fd0 tx=0x7ff510009e50 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:10:49.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.509+0000 7ff50effd700 1 -- 172.21.15.92:0/254214942 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff5140581e0 con 0x7ff520103780 2024-05-06T12:10:49.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.697+0000 7ff524c5a700 1 -- 172.21.15.92:0/254214942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff520109ec0 con 0x7ff520103780 2024-05-06T12:10:49.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.697+0000 7ff50effd700 1 -- 172.21.15.92:0/254214942 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11080 (secure 0 0 0) 0x7ff514057d70 con 0x7ff520103780 2024-05-06T12:10:49.701 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:10:49.701 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":40,"fsid":"aee1d912-0ba0-11ef-bc97-c7b262605968","created":"2024-05-06T12:04:51.220909+0000","modified":"2024-05-06T12:10:27.808671+0000","last_up_change":"2024-05-06T12:10:26.803552+0000","last_in_change":"2024-05-06T12:10:07.815689+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-06T12:09:12.129668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"1b276435-4698-4017-9535-4be41a4e1379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3575291733},{"type":"v1","addr":"172.21.15.92:6803","nonce":3575291733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3575291733},{"type":"v1","addr":"172.21.15.92:6805","nonce":3575291733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3575291733},{"type":"v1","addr":"172.21.15.92:6809","nonce":3575291733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3575291733},{"type":"v1","addr":"172.21.15.92:6807","nonce":3575291733}]},"public_addr":"172.21.15.92:6803/3575291733","cluster_addr":"172.21.15.92:6805/3575291733","heartbeat_back_addr":"172.21.15.92:6809/3575291733","heartbeat_front_addr":"172.21.15.92:6807/3575291733","state":["exists","up"]},{"osd":1,"uuid":"21688119-d15f-4b31-8842-090cc539c76d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":334092652},{"type":"v1","addr":"172.21.15.92:6811","nonce":334092652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":334092652},{"type":"v1","addr":"172.21.15.92:6813","nonce":334092652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":334092652},{"type":"v1","addr":"172.21.15.92:6817","nonce":334092652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":334092652},{"type":"v1","addr":"172.21.15.92:6815","nonce":334092652}]},"public_addr":"172.21.15.92:6811/334092652","cluster_addr":"172.21.15.92:6813/334092652","heartbeat_back_addr":"172.21.15.92:6817/334092652","heartbeat_front_addr":"172.21.15.92:6815/334092652","state":["exists","up"]},{"osd":2,"uuid":"74d16dfb-b118-432d-827a-59706d69c393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3428814547},{"type":"v1","addr":"172.21.15.92:6819","nonce":3428814547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3428814547},{"type":"v1","addr":"172.21.15.92:6821","nonce":3428814547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3428814547},{"type":"v1","addr":"172.21.15.92:6825","nonce":3428814547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3428814547},{"type":"v1","addr":"172.21.15.92:6823","nonce":3428814547}]},"public_addr":"172.21.15.92:6819/3428814547","cluster_addr":"172.21.15.92:6821/3428814547","heartbeat_back_addr":"172.21.15.92:6825/3428814547","heartbeat_front_addr":"172.21.15.92:6823/3428814547","state":["exists","up"]},{"osd":3,"uuid":"5a9aa5ed-2502-4c73-a8ac-cb7d0259500b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2610168113},{"type":"v1","addr":"172.21.15.190:6801","nonce":2610168113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2610168113},{"type":"v1","addr":"172.21.15.190:6803","nonce":2610168113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2610168113},{"type":"v1","addr":"172.21.15.190:6807","nonce":2610168113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2610168113},{"type":"v1","addr":"172.21.15.190:6805","nonce":2610168113}]},"public_addr":"172.21.15.190:6801/2610168113","cluster_addr":"172.21.15.190:6803/2610168113","heartbeat_back_addr":"172.21.15.190:6807/2610168113","heartbeat_front_addr":"172.21.15.190:6805/2610168113","state":["exists","up"]},{"osd":4,"uuid":"d4039e4a-c758-4632-85d6-1b7e725c0293","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1405678389},{"type":"v1","addr":"172.21.15.190:6809","nonce":1405678389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1405678389},{"type":"v1","addr":"172.21.15.190:6811","nonce":1405678389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1405678389},{"type":"v1","addr":"172.21.15.190:6815","nonce":1405678389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1405678389},{"type":"v1","addr":"172.21.15.190:6813","nonce":1405678389}]},"public_addr":"172.21.15.190:6809/1405678389","cluster_addr":"172.21.15.190:6811/1405678389","heartbeat_back_addr":"172.21.15.190:6815/1405678389","heartbeat_front_addr":"172.21.15.190:6813/1405678389","state":["exists","up"]},{"osd":5,"uuid":"ec707eee-5f24-48bb-818a-547f804fc41b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1027118727},{"type":"v1","addr":"172.21.15.190:6817","nonce":1027118727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1027118727},{"type":"v1","addr":"172.21.15.190:6819","nonce":1027118727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1027118727},{"type":"v1","addr":"172.21.15.190:6823","nonce":1027118727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1027118727},{"type":"v1","addr":"172.21.15.190:6821","nonce":1027118727}]},"public_addr":"172.21.15.190:6817/1027118727","cluster_addr":"172.21.15.190:6819/1027118727","heartbeat_back_addr":"172.21.15.190:6823/1027118727","heartbeat_front_addr":"172.21.15.190:6821/1027118727","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:08:15.348582+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:08:42.571577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:09:10.093700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:09:34.257829+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:09:59.795522+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:10:25.131646+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:6801/543287293":"2024-05-07T12:07:03.802107+0000","172.21.15.92:6800/543287293":"2024-05-07T12:07:03.802107+0000","172.21.15.92:0/1552275072":"2024-05-07T12:07:03.802107+0000","172.21.15.92:0/2440233745":"2024-05-07T12:07:03.802107+0000","172.21.15.92:6801/3731057893":"2024-05-07T12:05:56.546947+0000","172.21.15.92:0/895013058":"2024-05-07T12:05:25.954232+0000","172.21.15.92:0/758782094":"2024-05-07T12:05:25.954232+0000","172.21.15.92:0/2210153116":"2024-05-07T12:05:56.546947+0000","172.21.15.92:0/997820112":"2024-05-07T12:05:25.954232+0000","172.21.15.92:6800/3731057893":"2024-05-07T12:05:56.546947+0000","172.21.15.92:0/1407601857":"2024-05-07T12:05:56.546947+0000","172.21.15.92:0/3693315250":"2024-05-07T12:05:56.546947+0000","172.21.15.92:6800/3656822751":"2024-05-07T12:05:25.954232+0000","172.21.15.92:0/1532977673":"2024-05-07T12:07:03.802107+0000","172.21.15.92:6801/3656822751":"2024-05-07T12:05:25.954232+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-06T12:10:49.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.699+0000 7ff524c5a700 1 -- 172.21.15.92:0/254214942 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7ff5080692a0 msgr2=0x7ff50806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:49.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.699+0000 7ff524c5a700 1 --2- 172.21.15.92:0/254214942 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7ff5080692a0 0x7ff50806b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff510009fd0 tx=0x7ff510009e50 comp rx=0 tx=0).stop 2024-05-06T12:10:49.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.699+0000 7ff524c5a700 1 -- 172.21.15.92:0/254214942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff520103780 msgr2=0x7ff520113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:49.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.699+0000 7ff524c5a700 1 --2- 172.21.15.92:0/254214942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff520103780 0x7ff520113ed0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff514009100 tx=0x7ff514009e30 comp rx=0 tx=0).stop 2024-05-06T12:10:49.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.700+0000 7ff524c5a700 1 -- 172.21.15.92:0/254214942 shutdown_connections 2024-05-06T12:10:49.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.700+0000 7ff524c5a700 1 --2- 172.21.15.92:0/254214942 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7ff5080692a0 0x7ff50806b750 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:49.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.700+0000 7ff524c5a700 1 --2- 172.21.15.92:0/254214942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff520102de0 0x7ff520113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:49.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.700+0000 7ff524c5a700 1 --2- 172.21.15.92:0/254214942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff520103780 0x7ff520113ed0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:49.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.700+0000 7ff524c5a700 1 -- 172.21.15.92:0/254214942 >> 172.21.15.92:0/254214942 conn(0x7ff5200f8990 msgr2=0x7ff5200f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:10:49.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.700+0000 7ff524c5a700 1 -- 172.21.15.92:0/254214942 shutdown_connections 2024-05-06T12:10:49.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:49.700+0000 7ff524c5a700 1 -- 172.21.15.92:0/254214942 wait complete. 2024-05-06T12:10:50.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:49 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/254214942' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-06T12:10:50.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:49 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/254214942' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-06T12:10:50.470 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-06T12:10:50.471 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-06T12:10:50.471 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-06T12:10:50.471 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-06T12:10:50.472 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-06T12:10:50.472 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-06T12:10:50.797 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:10:50.835 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:10:51.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:50 smithi092 ceph-mon[26954]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:10:51.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:50 smithi190 ceph-mon[33798]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:10:52.874 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:10:53.110 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:52 smithi092 ceph-mon[26954]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:10:53.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.284+0000 7f525b1a2700 1 -- 172.21.15.92:0/1725062404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5254102c20 msgr2=0x7f5254102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:53.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.284+0000 7f525b1a2700 1 --2- 172.21.15.92:0/1725062404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5254102c20 0x7f5254102ff0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5250004940 tx=0x7f5250009e30 comp rx=0 tx=0).stop 2024-05-06T12:10:53.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.284+0000 7f525b1a2700 1 -- 172.21.15.92:0/1725062404 shutdown_connections 2024-05-06T12:10:53.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.284+0000 7f525b1a2700 1 --2- 172.21.15.92:0/1725062404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52541035c0 0x7f52540fcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:53.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.284+0000 7f525b1a2700 1 --2- 172.21.15.92:0/1725062404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5254102c20 0x7f5254102ff0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:53.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.284+0000 7f525b1a2700 1 -- 172.21.15.92:0/1725062404 >> 172.21.15.92:0/1725062404 conn(0x7f52540f87d0 msgr2=0x7f52540fabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:10:53.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.284+0000 7f525b1a2700 1 -- 172.21.15.92:0/1725062404 shutdown_connections 2024-05-06T12:10:53.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.284+0000 7f525b1a2700 1 -- 172.21.15.92:0/1725062404 wait complete. 2024-05-06T12:10:53.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.285+0000 7f525b1a2700 1 Processor -- start 2024-05-06T12:10:53.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.285+0000 7f525b1a2700 1 -- start start 2024-05-06T12:10:53.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.285+0000 7f525b1a2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5254102c20 0x7f52541139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:53.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.285+0000 7f525b1a2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52541035c0 0x7f5254113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:53.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.285+0000 7f525b1a2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5254114530 con 0x7f5254102c20 2024-05-06T12:10:53.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.285+0000 7f525b1a2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525410d920 con 0x7f52541035c0 2024-05-06T12:10:53.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.285+0000 7f525a1a0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5254102c20 0x7f52541139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:53.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.285+0000 7f525a1a0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5254102c20 0x7f52541139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57274/0 (socket says 172.21.15.92:57274) 2024-05-06T12:10:53.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.285+0000 7f525a1a0700 1 -- 172.21.15.92:0/530168069 learned_addr learned my addr 172.21.15.92:0/530168069 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:10:53.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.285+0000 7f525999f700 1 --2- 172.21.15.92:0/530168069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52541035c0 0x7f5254113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:53.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.285+0000 7f525a1a0700 1 -- 172.21.15.92:0/530168069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52541035c0 msgr2=0x7f5254113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:53.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.285+0000 7f525a1a0700 1 --2- 172.21.15.92:0/530168069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52541035c0 0x7f5254113ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:53.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.285+0000 7f525a1a0700 1 -- 172.21.15.92:0/530168069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5250005040 con 0x7f5254102c20 2024-05-06T12:10:53.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.285+0000 7f525a1a0700 1 --2- 172.21.15.92:0/530168069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5254102c20 0x7f52541139a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5250009150 tx=0x7f5250009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:10:53.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.285+0000 7f524affd700 1 -- 172.21.15.92:0/530168069 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f525000e040 con 0x7f5254102c20 2024-05-06T12:10:53.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.285+0000 7f524affd700 1 -- 172.21.15.92:0/530168069 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5250013420 con 0x7f5254102c20 2024-05-06T12:10:53.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.285+0000 7f525b1a2700 1 -- 172.21.15.92:0/530168069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f525410dba0 con 0x7f5254102c20 2024-05-06T12:10:53.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.285+0000 7f524affd700 1 -- 172.21.15.92:0/530168069 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f525001c630 con 0x7f5254102c20 2024-05-06T12:10:53.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.286+0000 7f525b1a2700 1 -- 172.21.15.92:0/530168069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f525410e160 con 0x7f5254102c20 2024-05-06T12:10:53.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.286+0000 7f525b1a2700 1 -- 172.21.15.92:0/530168069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f525410e790 con 0x7f5254102c20 2024-05-06T12:10:53.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.287+0000 7f524affd700 1 -- 172.21.15.92:0/530168069 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f5250013590 con 0x7f5254102c20 2024-05-06T12:10:53.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.287+0000 7f524affd700 1 --2- 172.21.15.92:0/530168069 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f52400692b0 0x7f524006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:53.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.287+0000 7f524affd700 1 -- 172.21.15.92:0/530168069 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f5250088030 con 0x7f5254102c20 2024-05-06T12:10:53.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.287+0000 7f525999f700 1 --2- 172.21.15.92:0/530168069 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f52400692b0 0x7f524006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:53.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.287+0000 7f524affd700 1 --2- 172.21.15.92:0/530168069 >> [v2:172.21.15.190:6808/1405678389,v1:172.21.15.190:6809/1405678389] conn(0x7f524006ef80 0x7f5240071390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:53.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.287+0000 7f524affd700 1 -- 172.21.15.92:0/530168069 --> [v2:172.21.15.190:6808/1405678389,v1:172.21.15.190:6809/1405678389] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5240071a40 con 0x7f524006ef80 2024-05-06T12:10:53.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.287+0000 7f524affd700 1 -- 172.21.15.92:0/530168069 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f525008a6a0 con 0x7f5254102c20 2024-05-06T12:10:53.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.288+0000 7f525999f700 1 --2- 172.21.15.92:0/530168069 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f52400692b0 0x7f524006b760 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f525410eae0 tx=0x7f5244009ee0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:10:53.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.288+0000 7f525a9a1700 1 --2- 172.21.15.92:0/530168069 >> [v2:172.21.15.190:6808/1405678389,v1:172.21.15.190:6809/1405678389] conn(0x7f524006ef80 0x7f5240071390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:53.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.288+0000 7f525a9a1700 1 --2- 172.21.15.92:0/530168069 >> [v2:172.21.15.190:6808/1405678389,v1:172.21.15.190:6809/1405678389] conn(0x7f524006ef80 0x7f5240071390 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:10:53.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.289+0000 7f524affd700 1 -- 172.21.15.92:0/530168069 <== osd.4 v2:172.21.15.190:6808/1405678389 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f5240071a40 con 0x7f524006ef80 2024-05-06T12:10:53.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.304+0000 7f525b1a2700 1 -- 172.21.15.92:0/530168069 --> [v2:172.21.15.190:6808/1405678389,v1:172.21.15.190:6809/1405678389] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f52540fa1f0 con 0x7f524006ef80 2024-05-06T12:10:53.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.304+0000 7f524affd700 1 -- 172.21.15.92:0/530168069 <== osd.4 v2:172.21.15.190:6808/1405678389 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f52540fa1f0 con 0x7f524006ef80 2024-05-06T12:10:53.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.304+0000 7f525b1a2700 1 -- 172.21.15.92:0/530168069 >> [v2:172.21.15.190:6808/1405678389,v1:172.21.15.190:6809/1405678389] conn(0x7f524006ef80 msgr2=0x7f5240071390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:53.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.304+0000 7f525b1a2700 1 --2- 172.21.15.92:0/530168069 >> [v2:172.21.15.190:6808/1405678389,v1:172.21.15.190:6809/1405678389] conn(0x7f524006ef80 0x7f5240071390 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:53.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.305+0000 7f525b1a2700 1 -- 172.21.15.92:0/530168069 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f52400692b0 msgr2=0x7f524006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:53.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.305+0000 7f525b1a2700 1 --2- 172.21.15.92:0/530168069 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f52400692b0 0x7f524006b760 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f525410eae0 tx=0x7f5244009ee0 comp rx=0 tx=0).stop 2024-05-06T12:10:53.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.305+0000 7f525b1a2700 1 -- 172.21.15.92:0/530168069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5254102c20 msgr2=0x7f52541139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:53.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.305+0000 7f525b1a2700 1 --2- 172.21.15.92:0/530168069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5254102c20 0x7f52541139a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5250009150 tx=0x7f5250009e30 comp rx=0 tx=0).stop 2024-05-06T12:10:53.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.305+0000 7f525b1a2700 1 -- 172.21.15.92:0/530168069 shutdown_connections 2024-05-06T12:10:53.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.305+0000 7f525b1a2700 1 --2- 172.21.15.92:0/530168069 >> [v2:172.21.15.190:6808/1405678389,v1:172.21.15.190:6809/1405678389] conn(0x7f524006ef80 0x7f5240071390 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:53.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.305+0000 7f525b1a2700 1 --2- 172.21.15.92:0/530168069 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f52400692b0 0x7f524006b760 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:53.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.305+0000 7f525b1a2700 1 --2- 172.21.15.92:0/530168069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5254102c20 0x7f52541139a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:53.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.305+0000 7f525b1a2700 1 --2- 172.21.15.92:0/530168069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52541035c0 0x7f5254113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:53.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.305+0000 7f525b1a2700 1 -- 172.21.15.92:0/530168069 >> 172.21.15.92:0/530168069 conn(0x7f52540f87d0 msgr2=0x7f52540f9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:10:53.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.305+0000 7f525b1a2700 1 -- 172.21.15.92:0/530168069 shutdown_connections 2024-05-06T12:10:53.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.305+0000 7f525b1a2700 1 -- 172.21.15.92:0/530168069 wait complete. 2024-05-06T12:10:53.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:52 smithi190 ceph-mon[33798]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:10:53.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.870+0000 7f16b739f700 1 -- 172.21.15.92:0/87458237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16b00fbb40 msgr2=0x7f16b00fbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:53.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.870+0000 7f16b739f700 1 --2- 172.21.15.92:0/87458237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16b00fbb40 0x7f16b00fbfb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f16a0004940 tx=0x7f16a0009e30 comp rx=0 tx=0).stop 2024-05-06T12:10:53.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.870+0000 7f16b739f700 1 -- 172.21.15.92:0/87458237 shutdown_connections 2024-05-06T12:10:53.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.870+0000 7f16b739f700 1 --2- 172.21.15.92:0/87458237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16b00fbb40 0x7f16b00fbfb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:53.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.870+0000 7f16b739f700 1 --2- 172.21.15.92:0/87458237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16b0101b00 0x7f16b0101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:53.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.870+0000 7f16b739f700 1 -- 172.21.15.92:0/87458237 >> 172.21.15.92:0/87458237 conn(0x7f16b00f7640 msgr2=0x7f16b00f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:10:53.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.870+0000 7f16b739f700 1 -- 172.21.15.92:0/87458237 shutdown_connections 2024-05-06T12:10:53.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.870+0000 7f16b739f700 1 -- 172.21.15.92:0/87458237 wait complete. 2024-05-06T12:10:53.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.871+0000 7f16b739f700 1 Processor -- start 2024-05-06T12:10:53.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.871+0000 7f16b739f700 1 -- start start 2024-05-06T12:10:53.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.871+0000 7f16b739f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16b00fbb40 0x7f16b010e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:53.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.871+0000 7f16b739f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16b0101b00 0x7f16b010ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:53.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.871+0000 7f16b739f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b010f550 con 0x7f16b0101b00 2024-05-06T12:10:53.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.871+0000 7f16b739f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b010f690 con 0x7f16b00fbb40 2024-05-06T12:10:53.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.871+0000 7f16b5b9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16b0101b00 0x7f16b010ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:53.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.871+0000 7f16b5b9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16b0101b00 0x7f16b010ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57284/0 (socket says 172.21.15.92:57284) 2024-05-06T12:10:53.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.871+0000 7f16b5b9c700 1 -- 172.21.15.92:0/3321183712 learned_addr learned my addr 172.21.15.92:0/3321183712 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:10:53.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.871+0000 7f16b639d700 1 --2- 172.21.15.92:0/3321183712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16b00fbb40 0x7f16b010e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:53.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.871+0000 7f16b5b9c700 1 -- 172.21.15.92:0/3321183712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16b00fbb40 msgr2=0x7f16b010e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:53.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.871+0000 7f16b5b9c700 1 --2- 172.21.15.92:0/3321183712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16b00fbb40 0x7f16b010e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:53.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.871+0000 7f16b5b9c700 1 -- 172.21.15.92:0/3321183712 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a0005040 con 0x7f16b0101b00 2024-05-06T12:10:53.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.871+0000 7f16b639d700 1 --2- 172.21.15.92:0/3321183712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16b00fbb40 0x7f16b010e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-06T12:10:53.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.872+0000 7f16b5b9c700 1 --2- 172.21.15.92:0/3321183712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16b0101b00 0x7f16b010ee70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f16a000b3b0 tx=0x7f16a0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:10:53.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.872+0000 7f16a6ffd700 1 -- 172.21.15.92:0/3321183712 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16a000e040 con 0x7f16b0101b00 2024-05-06T12:10:53.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.872+0000 7f16a6ffd700 1 -- 172.21.15.92:0/3321183712 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f16a000bbc0 con 0x7f16b0101b00 2024-05-06T12:10:53.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.872+0000 7f16b739f700 1 -- 172.21.15.92:0/3321183712 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16b01179a0 con 0x7f16b0101b00 2024-05-06T12:10:53.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.872+0000 7f16a6ffd700 1 -- 172.21.15.92:0/3321183712 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16a001c3f0 con 0x7f16b0101b00 2024-05-06T12:10:53.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.872+0000 7f16b739f700 1 -- 172.21.15.92:0/3321183712 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b0117f30 con 0x7f16b0101b00 2024-05-06T12:10:53.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.873+0000 7f16b739f700 1 -- 172.21.15.92:0/3321183712 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f16b00f8ed0 con 0x7f16b0101b00 2024-05-06T12:10:53.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.873+0000 7f16a6ffd700 1 -- 172.21.15.92:0/3321183712 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f16a000bd30 con 0x7f16b0101b00 2024-05-06T12:10:53.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.874+0000 7f16a6ffd700 1 --2- 172.21.15.92:0/3321183712 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f169c0692a0 0x7f169c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:53.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.874+0000 7f16b639d700 1 --2- 172.21.15.92:0/3321183712 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f169c0692a0 0x7f169c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:53.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.874+0000 7f16a6ffd700 1 -- 172.21.15.92:0/3321183712 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f16a00880f0 con 0x7f16b0101b00 2024-05-06T12:10:53.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.874+0000 7f16a6ffd700 1 --2- 172.21.15.92:0/3321183712 >> [v2:172.21.15.190:6816/1027118727,v1:172.21.15.190:6817/1027118727] conn(0x7f169c06ef70 0x7f169c071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:53.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.874+0000 7f16a6ffd700 1 -- 172.21.15.92:0/3321183712 --> [v2:172.21.15.190:6816/1027118727,v1:172.21.15.190:6817/1027118727] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f169c071a30 con 0x7f169c06ef70 2024-05-06T12:10:53.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.874+0000 7f16a6ffd700 1 -- 172.21.15.92:0/3321183712 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16a008a6e0 con 0x7f16b0101b00 2024-05-06T12:10:53.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.874+0000 7f16b639d700 1 --2- 172.21.15.92:0/3321183712 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f169c0692a0 0x7f169c06b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009b10 tx=0x7f16ac008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:10:53.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.874+0000 7f16b6b9e700 1 --2- 172.21.15.92:0/3321183712 >> [v2:172.21.15.190:6816/1027118727,v1:172.21.15.190:6817/1027118727] conn(0x7f169c06ef70 0x7f169c071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:53.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.875+0000 7f16b6b9e700 1 --2- 172.21.15.92:0/3321183712 >> [v2:172.21.15.190:6816/1027118727,v1:172.21.15.190:6817/1027118727] conn(0x7f169c06ef70 0x7f169c071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:10:53.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.875+0000 7f16a6ffd700 1 -- 172.21.15.92:0/3321183712 <== osd.5 v2:172.21.15.190:6816/1027118727 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f169c071a30 con 0x7f169c06ef70 2024-05-06T12:10:53.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.890+0000 7f16b739f700 1 -- 172.21.15.92:0/3321183712 --> [v2:172.21.15.190:6816/1027118727,v1:172.21.15.190:6817/1027118727] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f16b00fd1f0 con 0x7f169c06ef70 2024-05-06T12:10:53.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.891+0000 7f16a6ffd700 1 -- 172.21.15.92:0/3321183712 <== osd.5 v2:172.21.15.190:6816/1027118727 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f16b00fd1f0 con 0x7f169c06ef70 2024-05-06T12:10:53.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.891+0000 7f16b739f700 1 -- 172.21.15.92:0/3321183712 >> [v2:172.21.15.190:6816/1027118727,v1:172.21.15.190:6817/1027118727] conn(0x7f169c06ef70 msgr2=0x7f169c071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:53.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.891+0000 7f16b739f700 1 --2- 172.21.15.92:0/3321183712 >> [v2:172.21.15.190:6816/1027118727,v1:172.21.15.190:6817/1027118727] conn(0x7f169c06ef70 0x7f169c071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:53.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.891+0000 7f16b739f700 1 -- 172.21.15.92:0/3321183712 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f169c0692a0 msgr2=0x7f169c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:53.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.891+0000 7f16b739f700 1 --2- 172.21.15.92:0/3321183712 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f169c0692a0 0x7f169c06b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009b10 tx=0x7f16ac008040 comp rx=0 tx=0).stop 2024-05-06T12:10:53.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.891+0000 7f16b739f700 1 -- 172.21.15.92:0/3321183712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16b0101b00 msgr2=0x7f16b010ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:53.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.891+0000 7f16b739f700 1 --2- 172.21.15.92:0/3321183712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16b0101b00 0x7f16b010ee70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f16a000b3b0 tx=0x7f16a0009e30 comp rx=0 tx=0).stop 2024-05-06T12:10:53.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.891+0000 7f16b739f700 1 -- 172.21.15.92:0/3321183712 shutdown_connections 2024-05-06T12:10:53.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.891+0000 7f16b739f700 1 --2- 172.21.15.92:0/3321183712 >> [v2:172.21.15.190:6816/1027118727,v1:172.21.15.190:6817/1027118727] conn(0x7f169c06ef70 0x7f169c071380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:53.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.891+0000 7f16b739f700 1 --2- 172.21.15.92:0/3321183712 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f169c0692a0 0x7f169c06b750 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:53.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.891+0000 7f16b739f700 1 --2- 172.21.15.92:0/3321183712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16b00fbb40 0x7f16b010e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:53.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.891+0000 7f16b739f700 1 --2- 172.21.15.92:0/3321183712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16b0101b00 0x7f16b010ee70 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:53.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.891+0000 7f16b739f700 1 -- 172.21.15.92:0/3321183712 >> 172.21.15.92:0/3321183712 conn(0x7f16b00f7640 msgr2=0x7f16b00f84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:10:53.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.891+0000 7f16b739f700 1 -- 172.21.15.92:0/3321183712 shutdown_connections 2024-05-06T12:10:53.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:53.891+0000 7f16b739f700 1 -- 172.21.15.92:0/3321183712 wait complete. 2024-05-06T12:10:53.908 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:10:53.909 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:10:54.758 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:10:55.150 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:54 smithi092 ceph-mon[26954]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:10:55.175 INFO:teuthology.orchestra.run.smithi092.stdout:167503724552 2024-05-06T12:10:55.175 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-06T12:10:55.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:54 smithi190 ceph-mon[33798]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:10:57.072 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:56 smithi092 ceph-mon[26954]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:10:57.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:56 smithi190 ceph-mon[33798]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:10:57.459 INFO:teuthology.orchestra.run.smithi092.stdout:146028888077 2024-05-06T12:10:57.459 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-06T12:10:59.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:10:58 smithi190 ceph-mon[33798]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:10:59.202 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:10:58 smithi092 ceph-mon[26954]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:10:59.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.423+0000 7f2e228e4700 1 -- 172.21.15.92:0/2710203102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e1c102df0 msgr2=0x7f2e1c1031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:59.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.423+0000 7f2e228e4700 1 --2- 172.21.15.92:0/2710203102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e1c102df0 0x7f2e1c1031c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2e10004940 tx=0x7f2e10009e30 comp rx=0 tx=0).stop 2024-05-06T12:10:59.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.423+0000 7f2e228e4700 1 -- 172.21.15.92:0/2710203102 shutdown_connections 2024-05-06T12:10:59.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.423+0000 7f2e228e4700 1 --2- 172.21.15.92:0/2710203102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e1c103790 0x7f2e1c0fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:59.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.423+0000 7f2e228e4700 1 --2- 172.21.15.92:0/2710203102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e1c102df0 0x7f2e1c1031c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:59.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.423+0000 7f2e228e4700 1 -- 172.21.15.92:0/2710203102 >> 172.21.15.92:0/2710203102 conn(0x7f2e1c0f8980 msgr2=0x7f2e1c0fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:10:59.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.423+0000 7f2e228e4700 1 -- 172.21.15.92:0/2710203102 shutdown_connections 2024-05-06T12:10:59.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.423+0000 7f2e228e4700 1 -- 172.21.15.92:0/2710203102 wait complete. 2024-05-06T12:10:59.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.424+0000 7f2e228e4700 1 Processor -- start 2024-05-06T12:10:59.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.424+0000 7f2e228e4700 1 -- start start 2024-05-06T12:10:59.428 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.424+0000 7f2e228e4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e1c102df0 0x7f2e1c070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:59.428 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.424+0000 7f2e228e4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e1c103790 0x7f2e1c0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:59.428 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.424+0000 7f2e228e4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c070dc0 con 0x7f2e1c103790 2024-05-06T12:10:59.428 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.424+0000 7f2e228e4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c070f00 con 0x7f2e1c102df0 2024-05-06T12:10:59.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.424+0000 7f2e1b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e1c103790 0x7f2e1c0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:59.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.424+0000 7f2e1bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e1c102df0 0x7f2e1c070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:59.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.424+0000 7f2e1bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e1c102df0 0x7f2e1c070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.92:38278/0 (socket says 172.21.15.92:38278) 2024-05-06T12:10:59.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.424+0000 7f2e1b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e1c103790 0x7f2e1c0707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57310/0 (socket says 172.21.15.92:57310) 2024-05-06T12:10:59.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.424+0000 7f2e1bfff700 1 -- 172.21.15.92:0/2471236355 learned_addr learned my addr 172.21.15.92:0/2471236355 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:10:59.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.425+0000 7f2e1bfff700 1 -- 172.21.15.92:0/2471236355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e1c103790 msgr2=0x7f2e1c0707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:59.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.425+0000 7f2e1bfff700 1 --2- 172.21.15.92:0/2471236355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e1c103790 0x7f2e1c0707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:59.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.425+0000 7f2e1bfff700 1 -- 172.21.15.92:0/2471236355 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e10005040 con 0x7f2e1c102df0 2024-05-06T12:10:59.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.425+0000 7f2e1bfff700 1 --2- 172.21.15.92:0/2471236355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e1c102df0 0x7f2e1c070260 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2e1000b3b0 tx=0x7f2e10009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:10:59.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.425+0000 7f2e18ff9700 1 -- 172.21.15.92:0/2471236355 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e1000e040 con 0x7f2e1c102df0 2024-05-06T12:10:59.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.426+0000 7f2e18ff9700 1 -- 172.21.15.92:0/2471236355 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e10013420 con 0x7f2e1c102df0 2024-05-06T12:10:59.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.426+0000 7f2e228e4700 1 -- 172.21.15.92:0/2471236355 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e1c119e90 con 0x7f2e1c102df0 2024-05-06T12:10:59.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.426+0000 7f2e18ff9700 1 -- 172.21.15.92:0/2471236355 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e1001c630 con 0x7f2e1c102df0 2024-05-06T12:10:59.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.426+0000 7f2e228e4700 1 -- 172.21.15.92:0/2471236355 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e1c11a2b0 con 0x7f2e1c102df0 2024-05-06T12:10:59.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.427+0000 7f2e228e4700 1 -- 172.21.15.92:0/2471236355 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2e1c0fe540 con 0x7f2e1c102df0 2024-05-06T12:10:59.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.428+0000 7f2e18ff9700 1 -- 172.21.15.92:0/2471236355 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f2e1000baf0 con 0x7f2e1c102df0 2024-05-06T12:10:59.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.428+0000 7f2e18ff9700 1 --2- 172.21.15.92:0/2471236355 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f2e04069370 0x7f2e0406b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:59.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.428+0000 7f2e18ff9700 1 -- 172.21.15.92:0/2471236355 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f2e10088060 con 0x7f2e1c102df0 2024-05-06T12:10:59.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.429+0000 7f2e1b7fe700 1 --2- 172.21.15.92:0/2471236355 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f2e04069370 0x7f2e0406b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:59.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.429+0000 7f2e18ff9700 1 --2- 172.21.15.92:0/2471236355 >> [v2:172.21.15.92:6810/334092652,v1:172.21.15.92:6811/334092652] conn(0x7f2e0406f040 0x7f2e04071450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:10:59.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.429+0000 7f2e18ff9700 1 -- 172.21.15.92:0/2471236355 --> [v2:172.21.15.92:6810/334092652,v1:172.21.15.92:6811/334092652] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2e04071b00 con 0x7f2e0406f040 2024-05-06T12:10:59.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.429+0000 7f2e18ff9700 1 -- 172.21.15.92:0/2471236355 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e1008a600 con 0x7f2e1c102df0 2024-05-06T12:10:59.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.429+0000 7f2e20e81700 1 --2- 172.21.15.92:0/2471236355 >> [v2:172.21.15.92:6810/334092652,v1:172.21.15.92:6811/334092652] conn(0x7f2e0406f040 0x7f2e04071450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:10:59.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.429+0000 7f2e1b7fe700 1 --2- 172.21.15.92:0/2471236355 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f2e04069370 0x7f2e0406b820 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c071ba0 tx=0x7f2e0c0091a0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:10:59.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.430+0000 7f2e20e81700 1 --2- 172.21.15.92:0/2471236355 >> [v2:172.21.15.92:6810/334092652,v1:172.21.15.92:6811/334092652] conn(0x7f2e0406f040 0x7f2e04071450 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:10:59.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.432+0000 7f2e18ff9700 1 -- 172.21.15.92:0/2471236355 <== osd.1 v2:172.21.15.92:6810/334092652 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f2e04071b00 con 0x7f2e0406f040 2024-05-06T12:10:59.434 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:10:59.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.448+0000 7f2e228e4700 1 -- 172.21.15.92:0/2471236355 --> [v2:172.21.15.92:6810/334092652,v1:172.21.15.92:6811/334092652] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2e1c0fa260 con 0x7f2e0406f040 2024-05-06T12:10:59.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.448+0000 7f2e18ff9700 1 -- 172.21.15.92:0/2471236355 <== osd.1 v2:172.21.15.92:6810/334092652 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2e1c0fa260 con 0x7f2e0406f040 2024-05-06T12:10:59.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.449+0000 7f2e228e4700 1 -- 172.21.15.92:0/2471236355 >> [v2:172.21.15.92:6810/334092652,v1:172.21.15.92:6811/334092652] conn(0x7f2e0406f040 msgr2=0x7f2e04071450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:59.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.449+0000 7f2e228e4700 1 --2- 172.21.15.92:0/2471236355 >> [v2:172.21.15.92:6810/334092652,v1:172.21.15.92:6811/334092652] conn(0x7f2e0406f040 0x7f2e04071450 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:59.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.449+0000 7f2e228e4700 1 -- 172.21.15.92:0/2471236355 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f2e04069370 msgr2=0x7f2e0406b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:59.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.449+0000 7f2e228e4700 1 --2- 172.21.15.92:0/2471236355 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f2e04069370 0x7f2e0406b820 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c071ba0 tx=0x7f2e0c0091a0 comp rx=0 tx=0).stop 2024-05-06T12:10:59.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.449+0000 7f2e228e4700 1 -- 172.21.15.92:0/2471236355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e1c102df0 msgr2=0x7f2e1c070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:10:59.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.449+0000 7f2e228e4700 1 --2- 172.21.15.92:0/2471236355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e1c102df0 0x7f2e1c070260 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2e1000b3b0 tx=0x7f2e10009e30 comp rx=0 tx=0).stop 2024-05-06T12:10:59.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.449+0000 7f2e228e4700 1 -- 172.21.15.92:0/2471236355 shutdown_connections 2024-05-06T12:10:59.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.449+0000 7f2e228e4700 1 --2- 172.21.15.92:0/2471236355 >> [v2:172.21.15.92:6810/334092652,v1:172.21.15.92:6811/334092652] conn(0x7f2e0406f040 0x7f2e04071450 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:59.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.449+0000 7f2e228e4700 1 --2- 172.21.15.92:0/2471236355 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f2e04069370 0x7f2e0406b820 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:59.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.449+0000 7f2e228e4700 1 --2- 172.21.15.92:0/2471236355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e1c102df0 0x7f2e1c070260 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:59.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.449+0000 7f2e228e4700 1 --2- 172.21.15.92:0/2471236355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e1c103790 0x7f2e1c0707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:10:59.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.449+0000 7f2e228e4700 1 -- 172.21.15.92:0/2471236355 >> 172.21.15.92:0/2471236355 conn(0x7f2e1c0f8980 msgr2=0x7f2e1c0f9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:10:59.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.449+0000 7f2e228e4700 1 -- 172.21.15.92:0/2471236355 shutdown_connections 2024-05-06T12:10:59.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:10:59.449+0000 7f2e228e4700 1 -- 172.21.15.92:0/2471236355 wait complete. 2024-05-06T12:11:01.135 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:00 smithi092 ceph-mon[26954]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:01.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.233+0000 7f1e16050700 1 -- 172.21.15.92:0/1809883890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e10102e50 msgr2=0x7f1e10103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:01.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.233+0000 7f1e16050700 1 --2- 172.21.15.92:0/1809883890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e10102e50 0x7f1e10103220 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1e00004940 tx=0x7f1e00009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:01.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.233+0000 7f1e16050700 1 -- 172.21.15.92:0/1809883890 shutdown_connections 2024-05-06T12:11:01.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.233+0000 7f1e16050700 1 --2- 172.21.15.92:0/1809883890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e10065640 0x7f1e10065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:01.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.233+0000 7f1e16050700 1 --2- 172.21.15.92:0/1809883890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e10102e50 0x7f1e10103220 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:01.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.233+0000 7f1e16050700 1 -- 172.21.15.92:0/1809883890 >> 172.21.15.92:0/1809883890 conn(0x7f1e10071d00 msgr2=0x7f1e10072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:01.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.233+0000 7f1e16050700 1 -- 172.21.15.92:0/1809883890 shutdown_connections 2024-05-06T12:11:01.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.233+0000 7f1e16050700 1 -- 172.21.15.92:0/1809883890 wait complete. 2024-05-06T12:11:01.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.234+0000 7f1e16050700 1 Processor -- start 2024-05-06T12:11:01.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.234+0000 7f1e16050700 1 -- start start 2024-05-06T12:11:01.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.234+0000 7f1e16050700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e10065640 0x7f1e10113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:01.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.234+0000 7f1e16050700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e10102e50 0x7f1e101140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:01.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.234+0000 7f1e16050700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e10114710 con 0x7f1e10065640 2024-05-06T12:11:01.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.234+0000 7f1e16050700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1010db00 con 0x7f1e10102e50 2024-05-06T12:11:01.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.234+0000 7f1e0f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e10065640 0x7f1e10113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:01.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.234+0000 7f1e0f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e10065640 0x7f1e10113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57334/0 (socket says 172.21.15.92:57334) 2024-05-06T12:11:01.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.234+0000 7f1e0f7fe700 1 -- 172.21.15.92:0/2182231848 learned_addr learned my addr 172.21.15.92:0/2182231848 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:11:01.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.234+0000 7f1e07fff700 1 --2- 172.21.15.92:0/2182231848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e10102e50 0x7f1e101140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:01.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.234+0000 7f1e0f7fe700 1 -- 172.21.15.92:0/2182231848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e10102e50 msgr2=0x7f1e101140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:01.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.234+0000 7f1e0f7fe700 1 --2- 172.21.15.92:0/2182231848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e10102e50 0x7f1e101140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:01.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.234+0000 7f1e0f7fe700 1 -- 172.21.15.92:0/2182231848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e00005040 con 0x7f1e10065640 2024-05-06T12:11:01.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.234+0000 7f1e0f7fe700 1 --2- 172.21.15.92:0/2182231848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e10065640 0x7f1e10113b80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000b3b0 tx=0x7f1e00009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:01.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.235+0000 7f1e0cff9700 1 -- 172.21.15.92:0/2182231848 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e0000e040 con 0x7f1e10065640 2024-05-06T12:11:01.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.235+0000 7f1e0cff9700 1 -- 172.21.15.92:0/2182231848 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e0000bbc0 con 0x7f1e10065640 2024-05-06T12:11:01.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.235+0000 7f1e0cff9700 1 -- 172.21.15.92:0/2182231848 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e0001c3f0 con 0x7f1e10065640 2024-05-06T12:11:01.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.235+0000 7f1e16050700 1 -- 172.21.15.92:0/2182231848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e1010dd80 con 0x7f1e10065640 2024-05-06T12:11:01.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.235+0000 7f1e16050700 1 -- 172.21.15.92:0/2182231848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e1010e340 con 0x7f1e10065640 2024-05-06T12:11:01.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.236+0000 7f1e16050700 1 -- 172.21.15.92:0/2182231848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1e1010e920 con 0x7f1e10065640 2024-05-06T12:11:01.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.236+0000 7f1e0cff9700 1 -- 172.21.15.92:0/2182231848 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f1e0000bd30 con 0x7f1e10065640 2024-05-06T12:11:01.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.236+0000 7f1e0cff9700 1 --2- 172.21.15.92:0/2182231848 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f1dfc0692a0 0x7f1dfc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:01.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.237+0000 7f1e0cff9700 1 -- 172.21.15.92:0/2182231848 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1e00088000 con 0x7f1e10065640 2024-05-06T12:11:01.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.237+0000 7f1e07fff700 1 --2- 172.21.15.92:0/2182231848 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f1dfc0692a0 0x7f1dfc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:01.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.237+0000 7f1e0cff9700 1 --2- 172.21.15.92:0/2182231848 >> [v2:172.21.15.92:6802/3575291733,v1:172.21.15.92:6803/3575291733] conn(0x7f1dfc06ef70 0x7f1dfc071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:01.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.237+0000 7f1e0cff9700 1 -- 172.21.15.92:0/2182231848 --> [v2:172.21.15.92:6802/3575291733,v1:172.21.15.92:6803/3575291733] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1dfc071a30 con 0x7f1dfc06ef70 2024-05-06T12:11:01.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.237+0000 7f1e0cff9700 1 -- 172.21.15.92:0/2182231848 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e0008a5f0 con 0x7f1e10065640 2024-05-06T12:11:01.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.237+0000 7f1e0ffff700 1 --2- 172.21.15.92:0/2182231848 >> [v2:172.21.15.92:6802/3575291733,v1:172.21.15.92:6803/3575291733] conn(0x7f1dfc06ef70 0x7f1dfc071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:01.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.237+0000 7f1e07fff700 1 --2- 172.21.15.92:0/2182231848 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f1dfc0692a0 0x7f1dfc06b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1e1010ecc0 tx=0x7f1df8009ee0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:01.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.237+0000 7f1e0ffff700 1 --2- 172.21.15.92:0/2182231848 >> [v2:172.21.15.92:6802/3575291733,v1:172.21.15.92:6803/3575291733] conn(0x7f1dfc06ef70 0x7f1dfc071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:01.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.238+0000 7f1e0cff9700 1 -- 172.21.15.92:0/2182231848 <== osd.0 v2:172.21.15.92:6802/3575291733 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f1dfc071a30 con 0x7f1dfc06ef70 2024-05-06T12:11:01.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.267+0000 7f1e16050700 1 -- 172.21.15.92:0/2182231848 --> [v2:172.21.15.92:6802/3575291733,v1:172.21.15.92:6803/3575291733] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1e1005e660 con 0x7f1dfc06ef70 2024-05-06T12:11:01.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.267+0000 7f1e0cff9700 1 -- 172.21.15.92:0/2182231848 <== osd.0 v2:172.21.15.92:6802/3575291733 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1e1005e660 con 0x7f1dfc06ef70 2024-05-06T12:11:01.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.268+0000 7f1e16050700 1 -- 172.21.15.92:0/2182231848 >> [v2:172.21.15.92:6802/3575291733,v1:172.21.15.92:6803/3575291733] conn(0x7f1dfc06ef70 msgr2=0x7f1dfc071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:01.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.268+0000 7f1e16050700 1 --2- 172.21.15.92:0/2182231848 >> [v2:172.21.15.92:6802/3575291733,v1:172.21.15.92:6803/3575291733] conn(0x7f1dfc06ef70 0x7f1dfc071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:01.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.268+0000 7f1e16050700 1 -- 172.21.15.92:0/2182231848 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f1dfc0692a0 msgr2=0x7f1dfc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:01.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.268+0000 7f1e16050700 1 --2- 172.21.15.92:0/2182231848 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f1dfc0692a0 0x7f1dfc06b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1e1010ecc0 tx=0x7f1df8009ee0 comp rx=0 tx=0).stop 2024-05-06T12:11:01.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.268+0000 7f1e16050700 1 -- 172.21.15.92:0/2182231848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e10065640 msgr2=0x7f1e10113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:01.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.268+0000 7f1e16050700 1 --2- 172.21.15.92:0/2182231848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e10065640 0x7f1e10113b80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000b3b0 tx=0x7f1e00009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:01.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.268+0000 7f1e16050700 1 -- 172.21.15.92:0/2182231848 shutdown_connections 2024-05-06T12:11:01.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.268+0000 7f1e16050700 1 --2- 172.21.15.92:0/2182231848 >> [v2:172.21.15.92:6802/3575291733,v1:172.21.15.92:6803/3575291733] conn(0x7f1dfc06ef70 0x7f1dfc071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:01.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.268+0000 7f1e16050700 1 --2- 172.21.15.92:0/2182231848 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f1dfc0692a0 0x7f1dfc06b750 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:01.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.268+0000 7f1e16050700 1 --2- 172.21.15.92:0/2182231848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e10065640 0x7f1e10113b80 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:01.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.268+0000 7f1e16050700 1 --2- 172.21.15.92:0/2182231848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e10102e50 0x7f1e101140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:01.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.268+0000 7f1e16050700 1 -- 172.21.15.92:0/2182231848 >> 172.21.15.92:0/2182231848 conn(0x7f1e10071d00 msgr2=0x7f1e100fb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:01.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.268+0000 7f1e16050700 1 -- 172.21.15.92:0/2182231848 shutdown_connections 2024-05-06T12:11:01.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:01.268+0000 7f1e16050700 1 -- 172.21.15.92:0/2182231848 wait complete. 2024-05-06T12:11:01.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:00 smithi190 ceph-mon[33798]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:01.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.741+0000 7fc2b3555700 1 -- 172.21.15.92:0/1036799857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2ac0fdde0 msgr2=0x7fc2ac0fe1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:01.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.741+0000 7fc2b3555700 1 --2- 172.21.15.92:0/1036799857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2ac0fdde0 0x7fc2ac0fe1b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8004940 tx=0x7fc2a8009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:01.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.741+0000 7fc2b3555700 1 -- 172.21.15.92:0/1036799857 shutdown_connections 2024-05-06T12:11:01.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.741+0000 7fc2b3555700 1 --2- 172.21.15.92:0/1036799857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2ac0fe780 0x7fc2ac106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:01.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.741+0000 7fc2b3555700 1 --2- 172.21.15.92:0/1036799857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2ac0fdde0 0x7fc2ac0fe1b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:01.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.741+0000 7fc2b3555700 1 -- 172.21.15.92:0/1036799857 >> 172.21.15.92:0/1036799857 conn(0x7fc2ac0f7850 msgr2=0x7fc2ac0f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:01.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.741+0000 7fc2b3555700 1 -- 172.21.15.92:0/1036799857 shutdown_connections 2024-05-06T12:11:01.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.741+0000 7fc2b3555700 1 -- 172.21.15.92:0/1036799857 wait complete. 2024-05-06T12:11:01.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.742+0000 7fc2b3555700 1 Processor -- start 2024-05-06T12:11:01.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.742+0000 7fc2b3555700 1 -- start start 2024-05-06T12:11:01.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.742+0000 7fc2b3555700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2ac0fdde0 0x7fc2ac113ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:01.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.742+0000 7fc2b3555700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2ac0fe780 0x7fc2ac114010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:01.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.742+0000 7fc2b3555700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ac114660 con 0x7fc2ac0fe780 2024-05-06T12:11:01.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.742+0000 7fc2b3555700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ac10da50 con 0x7fc2ac0fdde0 2024-05-06T12:11:01.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.742+0000 7fc2b0af0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2ac0fe780 0x7fc2ac114010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:01.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.742+0000 7fc2b0af0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2ac0fe780 0x7fc2ac114010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57318/0 (socket says 172.21.15.92:57318) 2024-05-06T12:11:01.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.742+0000 7fc2b0af0700 1 -- 172.21.15.92:0/1068776038 learned_addr learned my addr 172.21.15.92:0/1068776038 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:11:01.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.742+0000 7fc2b12f1700 1 --2- 172.21.15.92:0/1068776038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2ac0fdde0 0x7fc2ac113ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:01.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.743+0000 7fc2b0af0700 1 -- 172.21.15.92:0/1068776038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2ac0fdde0 msgr2=0x7fc2ac113ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:01.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.743+0000 7fc2b0af0700 1 --2- 172.21.15.92:0/1068776038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2ac0fdde0 0x7fc2ac113ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:01.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.743+0000 7fc2b0af0700 1 -- 172.21.15.92:0/1068776038 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a8005040 con 0x7fc2ac0fe780 2024-05-06T12:11:01.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.743+0000 7fc2b12f1700 1 --2- 172.21.15.92:0/1068776038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2ac0fdde0 0x7fc2ac113ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-06T12:11:01.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.743+0000 7fc2b0af0700 1 --2- 172.21.15.92:0/1068776038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2ac0fe780 0x7fc2ac114010 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00eb40 tx=0x7fc29c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:01.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.743+0000 7fc2a1ffb700 1 -- 172.21.15.92:0/1068776038 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc29c007cc0 con 0x7fc2ac0fe780 2024-05-06T12:11:01.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.743+0000 7fc2b3555700 1 -- 172.21.15.92:0/1068776038 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2ac10dcd0 con 0x7fc2ac0fe780 2024-05-06T12:11:01.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.743+0000 7fc2a1ffb700 1 -- 172.21.15.92:0/1068776038 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc29c00f040 con 0x7fc2ac0fe780 2024-05-06T12:11:01.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.743+0000 7fc2a1ffb700 1 -- 172.21.15.92:0/1068776038 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc29c009620 con 0x7fc2ac0fe780 2024-05-06T12:11:01.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.743+0000 7fc2b3555700 1 -- 172.21.15.92:0/1068776038 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2ac10e210 con 0x7fc2ac0fe780 2024-05-06T12:11:01.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.744+0000 7fc2b3555700 1 -- 172.21.15.92:0/1068776038 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc2ac10e730 con 0x7fc2ac0fe780 2024-05-06T12:11:01.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.745+0000 7fc2a1ffb700 1 -- 172.21.15.92:0/1068776038 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fc29c009780 con 0x7fc2ac0fe780 2024-05-06T12:11:01.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.745+0000 7fc2a1ffb700 1 --2- 172.21.15.92:0/1068776038 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fc2980692a0 0x7fc29806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:01.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.745+0000 7fc2a1ffb700 1 -- 172.21.15.92:0/1068776038 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc29c086ad0 con 0x7fc2ac0fe780 2024-05-06T12:11:01.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.745+0000 7fc2b12f1700 1 --2- 172.21.15.92:0/1068776038 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fc2980692a0 0x7fc29806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:01.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.745+0000 7fc2a1ffb700 1 --2- 172.21.15.92:0/1068776038 >> [v2:172.21.15.190:6800/2610168113,v1:172.21.15.190:6801/2610168113] conn(0x7fc29806ef70 0x7fc298071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:01.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.745+0000 7fc2a1ffb700 1 -- 172.21.15.92:0/1068776038 --> [v2:172.21.15.190:6800/2610168113,v1:172.21.15.190:6801/2610168113] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc298071a30 con 0x7fc29806ef70 2024-05-06T12:11:01.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.745+0000 7fc2a1ffb700 1 -- 172.21.15.92:0/1068776038 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc29c08b220 con 0x7fc2ac0fe780 2024-05-06T12:11:01.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.746+0000 7fc2b12f1700 1 --2- 172.21.15.92:0/1068776038 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fc2980692a0 0x7fc29806b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8004560 tx=0x7fc2a8009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:01.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.746+0000 7fc2b1af2700 1 --2- 172.21.15.92:0/1068776038 >> [v2:172.21.15.190:6800/2610168113,v1:172.21.15.190:6801/2610168113] conn(0x7fc29806ef70 0x7fc298071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:01.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.746+0000 7fc2b1af2700 1 --2- 172.21.15.92:0/1068776038 >> [v2:172.21.15.190:6800/2610168113,v1:172.21.15.190:6801/2610168113] conn(0x7fc29806ef70 0x7fc298071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:01.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.747+0000 7fc2a1ffb700 1 -- 172.21.15.92:0/1068776038 <== osd.3 v2:172.21.15.190:6800/2610168113 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fc298071a30 con 0x7fc29806ef70 2024-05-06T12:11:01.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.763+0000 7fc2b3555700 1 -- 172.21.15.92:0/1068776038 --> [v2:172.21.15.190:6800/2610168113,v1:172.21.15.190:6801/2610168113] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc2ac10e8f0 con 0x7fc29806ef70 2024-05-06T12:11:01.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.763+0000 7fc2a1ffb700 1 -- 172.21.15.92:0/1068776038 <== osd.3 v2:172.21.15.190:6800/2610168113 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc2ac10e8f0 con 0x7fc29806ef70 2024-05-06T12:11:01.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.764+0000 7fc2b3555700 1 -- 172.21.15.92:0/1068776038 >> [v2:172.21.15.190:6800/2610168113,v1:172.21.15.190:6801/2610168113] conn(0x7fc29806ef70 msgr2=0x7fc298071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:01.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.764+0000 7fc2b3555700 1 --2- 172.21.15.92:0/1068776038 >> [v2:172.21.15.190:6800/2610168113,v1:172.21.15.190:6801/2610168113] conn(0x7fc29806ef70 0x7fc298071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:01.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.764+0000 7fc2b3555700 1 -- 172.21.15.92:0/1068776038 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fc2980692a0 msgr2=0x7fc29806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:01.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.764+0000 7fc2b3555700 1 --2- 172.21.15.92:0/1068776038 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fc2980692a0 0x7fc29806b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8004560 tx=0x7fc2a8009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:01.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.764+0000 7fc2b3555700 1 -- 172.21.15.92:0/1068776038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2ac0fe780 msgr2=0x7fc2ac114010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:01.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.764+0000 7fc2b3555700 1 --2- 172.21.15.92:0/1068776038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2ac0fe780 0x7fc2ac114010 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00eb40 tx=0x7fc29c00b040 comp rx=0 tx=0).stop 2024-05-06T12:11:01.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.764+0000 7fc2b3555700 1 -- 172.21.15.92:0/1068776038 shutdown_connections 2024-05-06T12:11:01.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.764+0000 7fc2b3555700 1 --2- 172.21.15.92:0/1068776038 >> [v2:172.21.15.190:6800/2610168113,v1:172.21.15.190:6801/2610168113] conn(0x7fc29806ef70 0x7fc298071380 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:01.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.764+0000 7fc2b3555700 1 --2- 172.21.15.92:0/1068776038 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fc2980692a0 0x7fc29806b750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:01.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.764+0000 7fc2b3555700 1 --2- 172.21.15.92:0/1068776038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2ac0fdde0 0x7fc2ac113ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.764+0000 7fc2b3555700 1 --2- 172.21.15.92:0/1068776038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2ac0fe780 0x7fc2ac114010 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.764+0000 7fc2b3555700 1 -- 172.21.15.92:0/1068776038 >> 172.21.15.92:0/1068776038 conn(0x7fc2ac0f7850 msgr2=0x7fc2ac0f8de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.764+0000 7fc2b3555700 1 -- 172.21.15.92:0/1068776038 shutdown_connections 2024-05-06T12:11:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:00.764+0000 7fc2b3555700 1 -- 172.21.15.92:0/1068776038 wait complete. 2024-05-06T12:11:02.333 INFO:teuthology.orchestra.run.smithi092.stdout:111669149715 2024-05-06T12:11:02.334 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-06T12:11:03.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:02 smithi092 ceph-mon[26954]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:03.279 INFO:teuthology.orchestra.run.smithi092.stdout:60129542173 2024-05-06T12:11:03.279 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-06T12:11:03.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:02 smithi190 ceph-mon[33798]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:04.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.077+0000 7efe0afa9700 1 -- 172.21.15.92:0/2935608172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe041037c0 msgr2=0x7efe040fb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:04.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.077+0000 7efe0afa9700 1 --2- 172.21.15.92:0/2935608172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe041037c0 0x7efe040fb3c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efdf0004940 tx=0x7efdf0009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:04.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.077+0000 7efe0afa9700 1 -- 172.21.15.92:0/2935608172 shutdown_connections 2024-05-06T12:11:04.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.077+0000 7efe0afa9700 1 --2- 172.21.15.92:0/2935608172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe041037c0 0x7efe040fb3c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:04.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.077+0000 7efe0afa9700 1 --2- 172.21.15.92:0/2935608172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe04102e20 0x7efe041031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:04.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.077+0000 7efe0afa9700 1 -- 172.21.15.92:0/2935608172 >> 172.21.15.92:0/2935608172 conn(0x7efe04072380 msgr2=0x7efe04072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:04.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.077+0000 7efe0afa9700 1 -- 172.21.15.92:0/2935608172 shutdown_connections 2024-05-06T12:11:04.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.077+0000 7efe0afa9700 1 -- 172.21.15.92:0/2935608172 wait complete. 2024-05-06T12:11:04.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.077+0000 7efe0afa9700 1 Processor -- start 2024-05-06T12:11:04.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.077+0000 7efe0afa9700 1 -- start start 2024-05-06T12:11:04.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.078+0000 7efe0afa9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe04102e20 0x7efe04113a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:04.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.078+0000 7efe0afa9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe041037c0 0x7efe04113f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:04.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.078+0000 7efe0afa9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe041145c0 con 0x7efe041037c0 2024-05-06T12:11:04.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.078+0000 7efe0afa9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0410dbe0 con 0x7efe04102e20 2024-05-06T12:11:04.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.078+0000 7efe03fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe041037c0 0x7efe04113f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:04.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.078+0000 7efe03fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe041037c0 0x7efe04113f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52862/0 (socket says 172.21.15.92:52862) 2024-05-06T12:11:04.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.078+0000 7efe03fff700 1 -- 172.21.15.92:0/1331463986 learned_addr learned my addr 172.21.15.92:0/1331463986 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:11:04.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.078+0000 7efe08d45700 1 --2- 172.21.15.92:0/1331463986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe04102e20 0x7efe04113a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:04.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.078+0000 7efe03fff700 1 -- 172.21.15.92:0/1331463986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe04102e20 msgr2=0x7efe04113a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:04.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.078+0000 7efe03fff700 1 --2- 172.21.15.92:0/1331463986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe04102e20 0x7efe04113a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:04.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.078+0000 7efe03fff700 1 -- 172.21.15.92:0/1331463986 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf0005040 con 0x7efe041037c0 2024-05-06T12:11:04.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.078+0000 7efe08d45700 1 --2- 172.21.15.92:0/1331463986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe04102e20 0x7efe04113a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-06T12:11:04.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.078+0000 7efe03fff700 1 --2- 172.21.15.92:0/1331463986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe041037c0 0x7efe04113f70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efdf0009100 tx=0x7efdf0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:04.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.078+0000 7efe017fa700 1 -- 172.21.15.92:0/1331463986 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdf000e040 con 0x7efe041037c0 2024-05-06T12:11:04.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.078+0000 7efe0afa9700 1 -- 172.21.15.92:0/1331463986 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe0410de60 con 0x7efe041037c0 2024-05-06T12:11:04.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.078+0000 7efe017fa700 1 -- 172.21.15.92:0/1331463986 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efdf0013420 con 0x7efe041037c0 2024-05-06T12:11:04.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.078+0000 7efe017fa700 1 -- 172.21.15.92:0/1331463986 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdf001c610 con 0x7efe041037c0 2024-05-06T12:11:04.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.078+0000 7efe0afa9700 1 -- 172.21.15.92:0/1331463986 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe0410e3a0 con 0x7efe041037c0 2024-05-06T12:11:04.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.079+0000 7efe0afa9700 1 -- 172.21.15.92:0/1331463986 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe04100160 con 0x7efe041037c0 2024-05-06T12:11:04.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.080+0000 7efe017fa700 1 -- 172.21.15.92:0/1331463986 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7efdf0013590 con 0x7efe041037c0 2024-05-06T12:11:04.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.080+0000 7efe017fa700 1 --2- 172.21.15.92:0/1331463986 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7efdf40692a0 0x7efdf406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:04.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.080+0000 7efe08d45700 1 --2- 172.21.15.92:0/1331463986 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7efdf40692a0 0x7efdf406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:04.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.080+0000 7efe017fa700 1 -- 172.21.15.92:0/1331463986 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7efdf00880e0 con 0x7efe041037c0 2024-05-06T12:11:04.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.081+0000 7efe08d45700 1 --2- 172.21.15.92:0/1331463986 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7efdf40692a0 0x7efdf406b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efe04065e90 tx=0x7efdf8009040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:04.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.085+0000 7efe017fa700 1 -- 172.21.15.92:0/1331463986 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7efdf0054750 con 0x7efe041037c0 2024-05-06T12:11:04.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.691+0000 7f087deb1700 1 -- 172.21.15.92:0/20031079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0878101d90 msgr2=0x7f0878102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:04.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.691+0000 7f087deb1700 1 --2- 172.21.15.92:0/20031079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0878101d90 0x7f0878102160 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0868004940 tx=0x7f0868009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:04.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.691+0000 7f087deb1700 1 -- 172.21.15.92:0/20031079 shutdown_connections 2024-05-06T12:11:04.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.691+0000 7f087deb1700 1 --2- 172.21.15.92:0/20031079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08780fbd40 0x7f08780fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:04.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.691+0000 7f087deb1700 1 --2- 172.21.15.92:0/20031079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0878101d90 0x7f0878102160 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:04.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.691+0000 7f087deb1700 1 -- 172.21.15.92:0/20031079 >> 172.21.15.92:0/20031079 conn(0x7f0878071d00 msgr2=0x7f0878072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:04.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.691+0000 7f087deb1700 1 -- 172.21.15.92:0/20031079 shutdown_connections 2024-05-06T12:11:04.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.691+0000 7f087deb1700 1 -- 172.21.15.92:0/20031079 wait complete. 2024-05-06T12:11:04.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.692+0000 7f087deb1700 1 Processor -- start 2024-05-06T12:11:04.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.692+0000 7f087deb1700 1 -- start start 2024-05-06T12:11:04.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.692+0000 7f087deb1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08780fbd40 0x7f087810eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:04.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.692+0000 7f087deb1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0878101d90 0x7f087810f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:04.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.692+0000 7f087deb1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087810f6d0 con 0x7f0878101d90 2024-05-06T12:11:04.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.692+0000 7f087deb1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087810f840 con 0x7f08780fbd40 2024-05-06T12:11:04.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.692+0000 7f0876ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0878101d90 0x7f087810f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:04.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.693+0000 7f0876ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0878101d90 0x7f087810f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52838/0 (socket says 172.21.15.92:52838) 2024-05-06T12:11:04.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.693+0000 7f0876ffd700 1 -- 172.21.15.92:0/2937923483 learned_addr learned my addr 172.21.15.92:0/2937923483 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:11:04.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.693+0000 7f0876ffd700 1 -- 172.21.15.92:0/2937923483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08780fbd40 msgr2=0x7f087810eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:04.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.693+0000 7f08777fe700 1 --2- 172.21.15.92:0/2937923483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08780fbd40 0x7f087810eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:04.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.693+0000 7f0876ffd700 1 --2- 172.21.15.92:0/2937923483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08780fbd40 0x7f087810eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:04.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.693+0000 7f0876ffd700 1 -- 172.21.15.92:0/2937923483 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0868005040 con 0x7f0878101d90 2024-05-06T12:11:04.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.693+0000 7f0876ffd700 1 --2- 172.21.15.92:0/2937923483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0878101d90 0x7f087810f080 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f086c00c830 tx=0x7f086c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:04.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.693+0000 7f085ffff700 1 -- 172.21.15.92:0/2937923483 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f086c00a760 con 0x7f0878101d90 2024-05-06T12:11:04.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.693+0000 7f087deb1700 1 -- 172.21.15.92:0/2937923483 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0878117bb0 con 0x7f0878101d90 2024-05-06T12:11:04.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.693+0000 7f085ffff700 1 -- 172.21.15.92:0/2937923483 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f086c00ad90 con 0x7f0878101d90 2024-05-06T12:11:04.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.693+0000 7f085ffff700 1 -- 172.21.15.92:0/2937923483 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f086c0113a0 con 0x7f0878101d90 2024-05-06T12:11:04.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.693+0000 7f087deb1700 1 -- 172.21.15.92:0/2937923483 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08781180e0 con 0x7f0878101d90 2024-05-06T12:11:04.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.694+0000 7f087deb1700 1 -- 172.21.15.92:0/2937923483 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0878106440 con 0x7f0878101d90 2024-05-06T12:11:04.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.695+0000 7f085ffff700 1 -- 172.21.15.92:0/2937923483 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f086c012400 con 0x7f0878101d90 2024-05-06T12:11:04.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.695+0000 7f085ffff700 1 --2- 172.21.15.92:0/2937923483 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f08600692a0 0x7f086006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:04.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.695+0000 7f085ffff700 1 -- 172.21.15.92:0/2937923483 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f086c087860 con 0x7f0878101d90 2024-05-06T12:11:04.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.695+0000 7f08777fe700 1 --2- 172.21.15.92:0/2937923483 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f08600692a0 0x7f086006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:04.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.695+0000 7f085ffff700 1 --2- 172.21.15.92:0/2937923483 >> [v2:172.21.15.92:6818/3428814547,v1:172.21.15.92:6819/3428814547] conn(0x7f086006ef70 0x7f0860071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:04.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.695+0000 7f085ffff700 1 -- 172.21.15.92:0/2937923483 --> [v2:172.21.15.92:6818/3428814547,v1:172.21.15.92:6819/3428814547] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0860071a30 con 0x7f086006ef70 2024-05-06T12:11:04.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.695+0000 7f085ffff700 1 -- 172.21.15.92:0/2937923483 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f086c087c40 con 0x7f0878101d90 2024-05-06T12:11:04.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.695+0000 7f0877fff700 1 --2- 172.21.15.92:0/2937923483 >> [v2:172.21.15.92:6818/3428814547,v1:172.21.15.92:6819/3428814547] conn(0x7f086006ef70 0x7f0860071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:04.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.696+0000 7f08777fe700 1 --2- 172.21.15.92:0/2937923483 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f08600692a0 0x7f086006b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0868004560 tx=0x7f0868009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:04.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.696+0000 7f0877fff700 1 --2- 172.21.15.92:0/2937923483 >> [v2:172.21.15.92:6818/3428814547,v1:172.21.15.92:6819/3428814547] conn(0x7f086006ef70 0x7f0860071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:04.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.697+0000 7f085ffff700 1 -- 172.21.15.92:0/2937923483 <== osd.2 v2:172.21.15.92:6818/3428814547 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f0860071a30 con 0x7f086006ef70 2024-05-06T12:11:04.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.711+0000 7f087deb1700 1 -- 172.21.15.92:0/2937923483 --> [v2:172.21.15.92:6818/3428814547,v1:172.21.15.92:6819/3428814547] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f08780fd430 con 0x7f086006ef70 2024-05-06T12:11:04.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.712+0000 7f085ffff700 1 -- 172.21.15.92:0/2937923483 <== osd.2 v2:172.21.15.92:6818/3428814547 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f08780fd430 con 0x7f086006ef70 2024-05-06T12:11:04.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.712+0000 7f087deb1700 1 -- 172.21.15.92:0/2937923483 >> [v2:172.21.15.92:6818/3428814547,v1:172.21.15.92:6819/3428814547] conn(0x7f086006ef70 msgr2=0x7f0860071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:04.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.712+0000 7f087deb1700 1 --2- 172.21.15.92:0/2937923483 >> [v2:172.21.15.92:6818/3428814547,v1:172.21.15.92:6819/3428814547] conn(0x7f086006ef70 0x7f0860071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:04.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.712+0000 7f087deb1700 1 -- 172.21.15.92:0/2937923483 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f08600692a0 msgr2=0x7f086006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:04.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.712+0000 7f087deb1700 1 --2- 172.21.15.92:0/2937923483 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f08600692a0 0x7f086006b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0868004560 tx=0x7f0868009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:04.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.712+0000 7f087deb1700 1 -- 172.21.15.92:0/2937923483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0878101d90 msgr2=0x7f087810f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:04.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.712+0000 7f087deb1700 1 --2- 172.21.15.92:0/2937923483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0878101d90 0x7f087810f080 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f086c00c830 tx=0x7f086c004470 comp rx=0 tx=0).stop 2024-05-06T12:11:04.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.712+0000 7f087deb1700 1 -- 172.21.15.92:0/2937923483 shutdown_connections 2024-05-06T12:11:04.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.712+0000 7f087deb1700 1 --2- 172.21.15.92:0/2937923483 >> [v2:172.21.15.92:6818/3428814547,v1:172.21.15.92:6819/3428814547] conn(0x7f086006ef70 0x7f0860071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:04.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.712+0000 7f087deb1700 1 --2- 172.21.15.92:0/2937923483 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f08600692a0 0x7f086006b750 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:04.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.712+0000 7f087deb1700 1 --2- 172.21.15.92:0/2937923483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08780fbd40 0x7f087810eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:04.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.712+0000 7f087deb1700 1 --2- 172.21.15.92:0/2937923483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0878101d90 0x7f087810f080 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:04.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.712+0000 7f087deb1700 1 -- 172.21.15.92:0/2937923483 >> 172.21.15.92:0/2937923483 conn(0x7f0878071d00 msgr2=0x7f0878104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:04.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.712+0000 7f087deb1700 1 -- 172.21.15.92:0/2937923483 shutdown_connections 2024-05-06T12:11:04.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:02.712+0000 7f087deb1700 1 -- 172.21.15.92:0/2937923483 wait complete. 2024-05-06T12:11:04.193 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:03 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:11:04.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.276+0000 7efe0afa9700 1 -- 172.21.15.92:0/1331463986 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7efe0410f0c0 con 0x7efe041037c0 2024-05-06T12:11:04.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.277+0000 7efe017fa700 1 -- 172.21.15.92:0/1331463986 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efdf0057d70 con 0x7efe041037c0 2024-05-06T12:11:04.279 INFO:teuthology.orchestra.run.smithi092.stdout:167503724554 2024-05-06T12:11:04.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.278+0000 7efe0afa9700 1 -- 172.21.15.92:0/1331463986 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7efdf40692a0 msgr2=0x7efdf406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:04.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.279+0000 7efe0afa9700 1 --2- 172.21.15.92:0/1331463986 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7efdf40692a0 0x7efdf406b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efe04065e90 tx=0x7efdf8009040 comp rx=0 tx=0).stop 2024-05-06T12:11:04.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.279+0000 7efe0afa9700 1 -- 172.21.15.92:0/1331463986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe041037c0 msgr2=0x7efe04113f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:04.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.279+0000 7efe0afa9700 1 --2- 172.21.15.92:0/1331463986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe041037c0 0x7efe04113f70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efdf0009100 tx=0x7efdf0009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:04.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.279+0000 7efe0afa9700 1 -- 172.21.15.92:0/1331463986 shutdown_connections 2024-05-06T12:11:04.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.279+0000 7efe0afa9700 1 --2- 172.21.15.92:0/1331463986 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7efdf40692a0 0x7efdf406b750 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:04.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.279+0000 7efe0afa9700 1 --2- 172.21.15.92:0/1331463986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe04102e20 0x7efe04113a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:04.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.279+0000 7efe0afa9700 1 --2- 172.21.15.92:0/1331463986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe041037c0 0x7efe04113f70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:04.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.279+0000 7efe0afa9700 1 -- 172.21.15.92:0/1331463986 >> 172.21.15.92:0/1331463986 conn(0x7efe04072380 msgr2=0x7efe040fad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:04.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.279+0000 7efe0afa9700 1 -- 172.21.15.92:0/1331463986 shutdown_connections 2024-05-06T12:11:04.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:04.279+0000 7efe0afa9700 1 -- 172.21.15.92:0/1331463986 wait complete. 2024-05-06T12:11:04.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:03 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:11:04.854 INFO:teuthology.orchestra.run.smithi092.stdout:81604378649 2024-05-06T12:11:04.854 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-06T12:11:05.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:04 smithi092 ceph-mon[26954]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:05.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:04 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/1331463986' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-06T12:11:05.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:04 smithi190 ceph-mon[33798]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:05.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:04 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/1331463986' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-06T12:11:05.421 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:11:05.453 INFO:teuthology.orchestra.run.smithi092.stdout:38654705700 2024-05-06T12:11:05.454 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-06T12:11:07.207 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:06 smithi190 ceph-mon[33798]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:07.217 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:06 smithi092 ceph-mon[26954]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:07.224 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:11:07.224 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:11:07.259 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724552 got 167503724554 for osd.5 2024-05-06T12:11:07.259 DEBUG:teuthology.parallel:result is None 2024-05-06T12:11:09.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:08 smithi190 ceph-mon[33798]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:09.101 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:08 smithi092 ceph-mon[26954]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:09.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.879+0000 7f787a616700 1 -- 172.21.15.92:0/3016311191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7874101d90 msgr2=0x7f7874102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:09.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.879+0000 7f787a616700 1 --2- 172.21.15.92:0/3016311191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7874101d90 0x7f7874102160 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7864004940 tx=0x7f7864009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:09.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.879+0000 7f787a616700 1 -- 172.21.15.92:0/3016311191 shutdown_connections 2024-05-06T12:11:09.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.879+0000 7f787a616700 1 --2- 172.21.15.92:0/3016311191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78740fbd40 0x7f78740fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:09.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.879+0000 7f787a616700 1 --2- 172.21.15.92:0/3016311191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7874101d90 0x7f7874102160 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:09.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.879+0000 7f787a616700 1 -- 172.21.15.92:0/3016311191 >> 172.21.15.92:0/3016311191 conn(0x7f7874071d00 msgr2=0x7f7874072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:09.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.879+0000 7f787a616700 1 -- 172.21.15.92:0/3016311191 shutdown_connections 2024-05-06T12:11:09.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.879+0000 7f787a616700 1 -- 172.21.15.92:0/3016311191 wait complete. 2024-05-06T12:11:09.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.879+0000 7f787a616700 1 Processor -- start 2024-05-06T12:11:09.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.880+0000 7f787a616700 1 -- start start 2024-05-06T12:11:09.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.880+0000 7f787a616700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78740fbd40 0x7f787410eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:09.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.880+0000 7f787a616700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7874101d90 0x7f787410f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:09.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.880+0000 7f787a616700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787410f760 con 0x7f78740fbd40 2024-05-06T12:11:09.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.880+0000 7f787a616700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787410f8a0 con 0x7f7874101d90 2024-05-06T12:11:09.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.880+0000 7f7873fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78740fbd40 0x7f787410eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:09.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.880+0000 7f7873fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78740fbd40 0x7f787410eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52876/0 (socket says 172.21.15.92:52876) 2024-05-06T12:11:09.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.880+0000 7f7873fff700 1 -- 172.21.15.92:0/2205040837 learned_addr learned my addr 172.21.15.92:0/2205040837 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:11:09.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.880+0000 7f78737fe700 1 --2- 172.21.15.92:0/2205040837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7874101d90 0x7f787410f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:09.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.880+0000 7f7873fff700 1 -- 172.21.15.92:0/2205040837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7874101d90 msgr2=0x7f787410f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:09.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.880+0000 7f7873fff700 1 --2- 172.21.15.92:0/2205040837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7874101d90 0x7f787410f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:09.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.880+0000 7f7873fff700 1 -- 172.21.15.92:0/2205040837 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7864005040 con 0x7f78740fbd40 2024-05-06T12:11:09.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.880+0000 7f7873fff700 1 --2- 172.21.15.92:0/2205040837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78740fbd40 0x7f787410eb40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f786400b3b0 tx=0x7f7864009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:09.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.880+0000 7f7870ff9700 1 -- 172.21.15.92:0/2205040837 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f786400e040 con 0x7f78740fbd40 2024-05-06T12:11:09.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.881+0000 7f7870ff9700 1 -- 172.21.15.92:0/2205040837 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f786400bbc0 con 0x7f78740fbd40 2024-05-06T12:11:09.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.881+0000 7f787a616700 1 -- 172.21.15.92:0/2205040837 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7874117bb0 con 0x7f78740fbd40 2024-05-06T12:11:09.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.881+0000 7f7870ff9700 1 -- 172.21.15.92:0/2205040837 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f786401c3f0 con 0x7f78740fbd40 2024-05-06T12:11:09.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.881+0000 7f787a616700 1 -- 172.21.15.92:0/2205040837 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7874118140 con 0x7f78740fbd40 2024-05-06T12:11:09.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.881+0000 7f787a616700 1 -- 172.21.15.92:0/2205040837 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78740fd430 con 0x7f78740fbd40 2024-05-06T12:11:09.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.882+0000 7f7870ff9700 1 -- 172.21.15.92:0/2205040837 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f786400bd30 con 0x7f78740fbd40 2024-05-06T12:11:09.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.883+0000 7f7870ff9700 1 --2- 172.21.15.92:0/2205040837 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f785c0692a0 0x7f785c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:09.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.883+0000 7f78737fe700 1 --2- 172.21.15.92:0/2205040837 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f785c0692a0 0x7f785c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:09.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.883+0000 7f7870ff9700 1 -- 172.21.15.92:0/2205040837 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f78640881a0 con 0x7f78740fbd40 2024-05-06T12:11:09.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.883+0000 7f78737fe700 1 --2- 172.21.15.92:0/2205040837 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f785c0692a0 0x7f785c06b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7868006fd0 tx=0x7f7868008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:09.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:09.887+0000 7f7870ff9700 1 -- 172.21.15.92:0/2205040837 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7864054790 con 0x7f78740fbd40 2024-05-06T12:11:09.901 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:11:10.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:10.083+0000 7f787a616700 1 -- 172.21.15.92:0/2205040837 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f787404bf60 con 0x7f78740fbd40 2024-05-06T12:11:10.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:10.084+0000 7f7870ff9700 1 -- 172.21.15.92:0/2205040837 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7864057db0 con 0x7f78740fbd40 2024-05-06T12:11:10.087 INFO:teuthology.orchestra.run.smithi092.stdout:146028888080 2024-05-06T12:11:10.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:10.086+0000 7f787a616700 1 -- 172.21.15.92:0/2205040837 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f785c0692a0 msgr2=0x7f785c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:10.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:10.086+0000 7f787a616700 1 --2- 172.21.15.92:0/2205040837 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f785c0692a0 0x7f785c06b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7868006fd0 tx=0x7f7868008040 comp rx=0 tx=0).stop 2024-05-06T12:11:10.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:10.086+0000 7f787a616700 1 -- 172.21.15.92:0/2205040837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78740fbd40 msgr2=0x7f787410eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:10.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:10.086+0000 7f787a616700 1 --2- 172.21.15.92:0/2205040837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78740fbd40 0x7f787410eb40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f786400b3b0 tx=0x7f7864009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:10.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:10.086+0000 7f787a616700 1 -- 172.21.15.92:0/2205040837 shutdown_connections 2024-05-06T12:11:10.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:10.086+0000 7f787a616700 1 --2- 172.21.15.92:0/2205040837 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f785c0692a0 0x7f785c06b750 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:10.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:10.086+0000 7f787a616700 1 --2- 172.21.15.92:0/2205040837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78740fbd40 0x7f787410eb40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:10.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:10.086+0000 7f787a616700 1 --2- 172.21.15.92:0/2205040837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7874101d90 0x7f787410f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:10.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:10.086+0000 7f787a616700 1 -- 172.21.15.92:0/2205040837 >> 172.21.15.92:0/2205040837 conn(0x7f7874071d00 msgr2=0x7f78740fa0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:10.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:10.086+0000 7f787a616700 1 -- 172.21.15.92:0/2205040837 shutdown_connections 2024-05-06T12:11:10.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:10.086+0000 7f787a616700 1 -- 172.21.15.92:0/2205040837 wait complete. 2024-05-06T12:11:10.801 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888077 got 146028888080 for osd.4 2024-05-06T12:11:10.801 DEBUG:teuthology.parallel:result is None 2024-05-06T12:11:11.005 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.002+0000 7fd1553a3700 1 -- 172.21.15.92:0/4146998496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1501026a0 msgr2=0x7fd1500fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:11.005 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.002+0000 7fd1553a3700 1 --2- 172.21.15.92:0/4146998496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1501026a0 0x7fd1500fbd10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd144004940 tx=0x7fd144009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:11.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.002+0000 7fd1553a3700 1 -- 172.21.15.92:0/4146998496 shutdown_connections 2024-05-06T12:11:11.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.002+0000 7fd1553a3700 1 --2- 172.21.15.92:0/4146998496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1501026a0 0x7fd1500fbd10 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:11.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.002+0000 7fd1553a3700 1 --2- 172.21.15.92:0/4146998496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd150101d00 0x7fd1501020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:11.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.002+0000 7fd1553a3700 1 -- 172.21.15.92:0/4146998496 >> 172.21.15.92:0/4146998496 conn(0x7fd1500f78d0 msgr2=0x7fd1500f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:11.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.002+0000 7fd1553a3700 1 -- 172.21.15.92:0/4146998496 shutdown_connections 2024-05-06T12:11:11.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.002+0000 7fd1553a3700 1 -- 172.21.15.92:0/4146998496 wait complete. 2024-05-06T12:11:11.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.002+0000 7fd1553a3700 1 Processor -- start 2024-05-06T12:11:11.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.002+0000 7fd1553a3700 1 -- start start 2024-05-06T12:11:11.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.003+0000 7fd1553a3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd150101d00 0x7fd15010eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:11.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.003+0000 7fd1553a3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1501026a0 0x7fd15010f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:11.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.003+0000 7fd1553a3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15010f770 con 0x7fd1501026a0 2024-05-06T12:11:11.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.003+0000 7fd1553a3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15010f8b0 con 0x7fd150101d00 2024-05-06T12:11:11.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.003+0000 7fd14e7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1501026a0 0x7fd15010f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:11.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.003+0000 7fd14e7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1501026a0 0x7fd15010f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52896/0 (socket says 172.21.15.92:52896) 2024-05-06T12:11:11.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.003+0000 7fd14e7fc700 1 -- 172.21.15.92:0/3532899112 learned_addr learned my addr 172.21.15.92:0/3532899112 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:11:11.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.003+0000 7fd14effd700 1 --2- 172.21.15.92:0/3532899112 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd150101d00 0x7fd15010eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:11.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.003+0000 7fd14e7fc700 1 -- 172.21.15.92:0/3532899112 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd150101d00 msgr2=0x7fd15010eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:11.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.003+0000 7fd14e7fc700 1 --2- 172.21.15.92:0/3532899112 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd150101d00 0x7fd15010eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:11.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.003+0000 7fd14e7fc700 1 -- 172.21.15.92:0/3532899112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd144005040 con 0x7fd1501026a0 2024-05-06T12:11:11.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.003+0000 7fd14e7fc700 1 --2- 172.21.15.92:0/3532899112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1501026a0 0x7fd15010f090 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd144009100 tx=0x7fd144009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:11.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.003+0000 7fd1377fe700 1 -- 172.21.15.92:0/3532899112 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd14400e040 con 0x7fd1501026a0 2024-05-06T12:11:11.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.003+0000 7fd1377fe700 1 -- 172.21.15.92:0/3532899112 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd144004290 con 0x7fd1501026a0 2024-05-06T12:11:11.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.003+0000 7fd1553a3700 1 -- 172.21.15.92:0/3532899112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd150117bc0 con 0x7fd1501026a0 2024-05-06T12:11:11.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.003+0000 7fd1377fe700 1 -- 172.21.15.92:0/3532899112 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd14401c3f0 con 0x7fd1501026a0 2024-05-06T12:11:11.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.003+0000 7fd1553a3700 1 -- 172.21.15.92:0/3532899112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd150118150 con 0x7fd1501026a0 2024-05-06T12:11:11.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.004+0000 7fd1553a3700 1 -- 172.21.15.92:0/3532899112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1500fd450 con 0x7fd1501026a0 2024-05-06T12:11:11.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.005+0000 7fd1377fe700 1 -- 172.21.15.92:0/3532899112 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fd144013790 con 0x7fd1501026a0 2024-05-06T12:11:11.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.005+0000 7fd1377fe700 1 --2- 172.21.15.92:0/3532899112 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fd1380692b0 0x7fd13806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:11.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.005+0000 7fd1377fe700 1 -- 172.21.15.92:0/3532899112 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd144088150 con 0x7fd1501026a0 2024-05-06T12:11:11.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.005+0000 7fd14effd700 1 --2- 172.21.15.92:0/3532899112 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fd1380692b0 0x7fd13806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:11.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.005+0000 7fd14effd700 1 --2- 172.21.15.92:0/3532899112 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fd1380692b0 0x7fd13806b760 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd140006fd0 tx=0x7fd140008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:11.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.009+0000 7fd1377fe700 1 -- 172.21.15.92:0/3532899112 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd1440547c0 con 0x7fd1501026a0 2024-05-06T12:11:11.170 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:10 smithi092 ceph-mon[26954]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:11.171 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:10 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/2205040837' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-06T12:11:11.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.203+0000 7fd1553a3700 1 -- 172.21.15.92:0/3532899112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd150108e50 con 0x7fd1501026a0 2024-05-06T12:11:11.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.204+0000 7fd1377fe700 1 -- 172.21.15.92:0/3532899112 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd144013420 con 0x7fd1501026a0 2024-05-06T12:11:11.218 INFO:teuthology.orchestra.run.smithi092.stdout:60129542176 2024-05-06T12:11:11.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.206+0000 7fd1553a3700 1 -- 172.21.15.92:0/3532899112 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fd1380692b0 msgr2=0x7fd13806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:11.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.206+0000 7fd1553a3700 1 --2- 172.21.15.92:0/3532899112 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fd1380692b0 0x7fd13806b760 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd140006fd0 tx=0x7fd140008040 comp rx=0 tx=0).stop 2024-05-06T12:11:11.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.206+0000 7fd1553a3700 1 -- 172.21.15.92:0/3532899112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1501026a0 msgr2=0x7fd15010f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:11.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.206+0000 7fd1553a3700 1 --2- 172.21.15.92:0/3532899112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1501026a0 0x7fd15010f090 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd144009100 tx=0x7fd144009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:11.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.206+0000 7fd1553a3700 1 -- 172.21.15.92:0/3532899112 shutdown_connections 2024-05-06T12:11:11.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.206+0000 7fd1553a3700 1 --2- 172.21.15.92:0/3532899112 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fd1380692b0 0x7fd13806b760 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:11.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.206+0000 7fd1553a3700 1 --2- 172.21.15.92:0/3532899112 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd150101d00 0x7fd15010eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:11.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.206+0000 7fd1553a3700 1 --2- 172.21.15.92:0/3532899112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1501026a0 0x7fd15010f090 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:11.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.206+0000 7fd1553a3700 1 -- 172.21.15.92:0/3532899112 >> 172.21.15.92:0/3532899112 conn(0x7fd1500f78d0 msgr2=0x7fd1500f8870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:11.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.206+0000 7fd1553a3700 1 -- 172.21.15.92:0/3532899112 shutdown_connections 2024-05-06T12:11:11.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.206+0000 7fd1553a3700 1 -- 172.21.15.92:0/3532899112 wait complete. 2024-05-06T12:11:11.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.269+0000 7fe095d37700 1 -- 172.21.15.92:0/2142238807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0900fb5a0 msgr2=0x7fe0900fba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:11.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.269+0000 7fe095d37700 1 --2- 172.21.15.92:0/2142238807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0900fb5a0 0x7fe0900fba10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe084004940 tx=0x7fe084009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:11.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.269+0000 7fe095d37700 1 -- 172.21.15.92:0/2142238807 shutdown_connections 2024-05-06T12:11:11.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.269+0000 7fe095d37700 1 --2- 172.21.15.92:0/2142238807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0900fb5a0 0x7fe0900fba10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:11.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.269+0000 7fe095d37700 1 --2- 172.21.15.92:0/2142238807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0901004d0 0x7fe0901008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:11.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.269+0000 7fe095d37700 1 -- 172.21.15.92:0/2142238807 >> 172.21.15.92:0/2142238807 conn(0x7fe0900714d0 msgr2=0x7fe0900718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:11.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.270+0000 7fe095d37700 1 -- 172.21.15.92:0/2142238807 shutdown_connections 2024-05-06T12:11:11.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.270+0000 7fe095d37700 1 -- 172.21.15.92:0/2142238807 wait complete. 2024-05-06T12:11:11.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.270+0000 7fe095d37700 1 Processor -- start 2024-05-06T12:11:11.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.270+0000 7fe095d37700 1 -- start start 2024-05-06T12:11:11.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.270+0000 7fe095d37700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0900fb5a0 0x7fe0900fd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:11.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.270+0000 7fe095d37700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0901004d0 0x7fe0900fd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:11.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.270+0000 7fe095d37700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0900ff0c0 con 0x7fe0900fb5a0 2024-05-06T12:11:11.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.270+0000 7fe095d37700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0900fdbf0 con 0x7fe0901004d0 2024-05-06T12:11:11.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.270+0000 7fe08f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0900fb5a0 0x7fe0900fd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:11.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.271+0000 7fe08f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0900fb5a0 0x7fe0900fd170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52912/0 (socket says 172.21.15.92:52912) 2024-05-06T12:11:11.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.271+0000 7fe08f7fe700 1 -- 172.21.15.92:0/1487564824 learned_addr learned my addr 172.21.15.92:0/1487564824 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:11:11.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.271+0000 7fe08f7fe700 1 -- 172.21.15.92:0/1487564824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0901004d0 msgr2=0x7fe0900fd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:11.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.271+0000 7fe08f7fe700 1 --2- 172.21.15.92:0/1487564824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0901004d0 0x7fe0900fd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:11.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.271+0000 7fe08f7fe700 1 -- 172.21.15.92:0/1487564824 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe084005040 con 0x7fe0900fb5a0 2024-05-06T12:11:11.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.271+0000 7fe08f7fe700 1 --2- 172.21.15.92:0/1487564824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0900fb5a0 0x7fe0900fd170 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe08000c830 tx=0x7fe080004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:11.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.271+0000 7fe077fff700 1 -- 172.21.15.92:0/1487564824 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe08000a760 con 0x7fe0900fb5a0 2024-05-06T12:11:11.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.271+0000 7fe077fff700 1 -- 172.21.15.92:0/1487564824 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe08000ad90 con 0x7fe0900fb5a0 2024-05-06T12:11:11.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.271+0000 7fe077fff700 1 -- 172.21.15.92:0/1487564824 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0800113a0 con 0x7fe0900fb5a0 2024-05-06T12:11:11.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.271+0000 7fe095d37700 1 -- 172.21.15.92:0/1487564824 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0900fded0 con 0x7fe0900fb5a0 2024-05-06T12:11:11.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.271+0000 7fe095d37700 1 -- 172.21.15.92:0/1487564824 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe09011a0c0 con 0x7fe0900fb5a0 2024-05-06T12:11:11.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.272+0000 7fe095d37700 1 -- 172.21.15.92:0/1487564824 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe090105a70 con 0x7fe0900fb5a0 2024-05-06T12:11:11.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.272+0000 7fe077fff700 1 -- 172.21.15.92:0/1487564824 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fe080012400 con 0x7fe0900fb5a0 2024-05-06T12:11:11.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.273+0000 7fe077fff700 1 --2- 172.21.15.92:0/1487564824 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fe0780692a0 0x7fe07806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:11.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.273+0000 7fe077fff700 1 -- 172.21.15.92:0/1487564824 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fe080086ec0 con 0x7fe0900fb5a0 2024-05-06T12:11:11.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.273+0000 7fe08effd700 1 --2- 172.21.15.92:0/1487564824 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fe0780692a0 0x7fe07806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:11.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.273+0000 7fe08effd700 1 --2- 172.21.15.92:0/1487564824 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fe0780692a0 0x7fe07806b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe084004f90 tx=0x7fe084009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:11.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.277+0000 7fe077fff700 1 -- 172.21.15.92:0/1487564824 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe080053530 con 0x7fe0900fb5a0 2024-05-06T12:11:11.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:10 smithi190 ceph-mon[33798]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:11.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:10 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/2205040837' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-06T12:11:11.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.469+0000 7fe095d37700 1 -- 172.21.15.92:0/1487564824 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe0900fccb0 con 0x7fe0900fb5a0 2024-05-06T12:11:11.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.469+0000 7fe077fff700 1 -- 172.21.15.92:0/1487564824 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe080056b50 con 0x7fe0900fb5a0 2024-05-06T12:11:11.473 INFO:teuthology.orchestra.run.smithi092.stdout:111669149717 2024-05-06T12:11:11.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.471+0000 7fe095d37700 1 -- 172.21.15.92:0/1487564824 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fe0780692a0 msgr2=0x7fe07806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:11.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.471+0000 7fe095d37700 1 --2- 172.21.15.92:0/1487564824 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fe0780692a0 0x7fe07806b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe084004f90 tx=0x7fe084009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:11.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.471+0000 7fe095d37700 1 -- 172.21.15.92:0/1487564824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0900fb5a0 msgr2=0x7fe0900fd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:11.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.471+0000 7fe095d37700 1 --2- 172.21.15.92:0/1487564824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0900fb5a0 0x7fe0900fd170 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe08000c830 tx=0x7fe080004470 comp rx=0 tx=0).stop 2024-05-06T12:11:11.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.471+0000 7fe095d37700 1 -- 172.21.15.92:0/1487564824 shutdown_connections 2024-05-06T12:11:11.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.471+0000 7fe095d37700 1 --2- 172.21.15.92:0/1487564824 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fe0780692a0 0x7fe07806b750 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:11.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.471+0000 7fe095d37700 1 --2- 172.21.15.92:0/1487564824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0900fb5a0 0x7fe0900fd170 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:11.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.471+0000 7fe095d37700 1 --2- 172.21.15.92:0/1487564824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0901004d0 0x7fe0900fd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:11.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.471+0000 7fe095d37700 1 -- 172.21.15.92:0/1487564824 >> 172.21.15.92:0/1487564824 conn(0x7fe0900714d0 msgr2=0x7fe0900f9da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:11.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.471+0000 7fe095d37700 1 -- 172.21.15.92:0/1487564824 shutdown_connections 2024-05-06T12:11:11.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:11.471+0000 7fe095d37700 1 -- 172.21.15.92:0/1487564824 wait complete. 2024-05-06T12:11:11.487 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:11:11.884 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542173 got 60129542176 for osd.1 2024-05-06T12:11:11.884 DEBUG:teuthology.parallel:result is None 2024-05-06T12:11:12.109 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:11 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/3532899112' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-06T12:11:12.110 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:11 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/1487564824' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-06T12:11:12.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:11 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/3532899112' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-06T12:11:12.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:11 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/1487564824' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-06T12:11:13.199 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:12 smithi092 ceph-mon[26954]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:13.229 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149715 got 111669149717 for osd.3 2024-05-06T12:11:13.229 DEBUG:teuthology.parallel:result is None 2024-05-06T12:11:13.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:12 smithi190 ceph-mon[33798]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:15.164 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:14 smithi092 ceph-mon[26954]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:15.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:14 smithi190 ceph-mon[33798]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:15.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.580+0000 7f5068bdc700 1 -- 172.21.15.92:0/2529574787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5064102e50 msgr2=0x7f5064103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:15.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.580+0000 7f5068bdc700 1 --2- 172.21.15.92:0/2529574787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5064102e50 0x7f5064103220 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5058004940 tx=0x7f5058009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:15.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.580+0000 7f5068bdc700 1 -- 172.21.15.92:0/2529574787 shutdown_connections 2024-05-06T12:11:15.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.580+0000 7f5068bdc700 1 --2- 172.21.15.92:0/2529574787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5064065640 0x7f5064065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:15.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.580+0000 7f5068bdc700 1 --2- 172.21.15.92:0/2529574787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5064102e50 0x7f5064103220 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:15.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.580+0000 7f5068bdc700 1 -- 172.21.15.92:0/2529574787 >> 172.21.15.92:0/2529574787 conn(0x7f5064071d00 msgr2=0x7f5064072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:15.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.580+0000 7f5068bdc700 1 -- 172.21.15.92:0/2529574787 shutdown_connections 2024-05-06T12:11:15.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.580+0000 7f5068bdc700 1 -- 172.21.15.92:0/2529574787 wait complete. 2024-05-06T12:11:15.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.580+0000 7f5068bdc700 1 Processor -- start 2024-05-06T12:11:15.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.581+0000 7f5068bdc700 1 -- start start 2024-05-06T12:11:15.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.581+0000 7f5068bdc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5064065640 0x7f5064113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:15.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.581+0000 7f5068bdc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5064102e50 0x7f50641140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:15.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.581+0000 7f5068bdc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50641147a0 con 0x7f5064065640 2024-05-06T12:11:15.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.581+0000 7f5068bdc700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506410db00 con 0x7f5064102e50 2024-05-06T12:11:15.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.581+0000 7f506259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5064065640 0x7f5064113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:15.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.581+0000 7f506259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5064065640 0x7f5064113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60434/0 (socket says 172.21.15.92:60434) 2024-05-06T12:11:15.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.581+0000 7f506259c700 1 -- 172.21.15.92:0/840770622 learned_addr learned my addr 172.21.15.92:0/840770622 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:11:15.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.581+0000 7f5061d9b700 1 --2- 172.21.15.92:0/840770622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5064102e50 0x7f50641140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:15.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.581+0000 7f506259c700 1 -- 172.21.15.92:0/840770622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5064102e50 msgr2=0x7f50641140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:15.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.581+0000 7f506259c700 1 --2- 172.21.15.92:0/840770622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5064102e50 0x7f50641140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:15.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.581+0000 7f506259c700 1 -- 172.21.15.92:0/840770622 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5058005040 con 0x7f5064065640 2024-05-06T12:11:15.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.581+0000 7f5061d9b700 1 --2- 172.21.15.92:0/840770622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5064102e50 0x7f50641140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-06T12:11:15.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.581+0000 7f506259c700 1 --2- 172.21.15.92:0/840770622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5064065640 0x7f5064113b80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f505800b3b0 tx=0x7f5058009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:15.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.581+0000 7f5052ffd700 1 -- 172.21.15.92:0/840770622 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f505800e040 con 0x7f5064065640 2024-05-06T12:11:15.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.582+0000 7f5052ffd700 1 -- 172.21.15.92:0/840770622 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f505800bbc0 con 0x7f5064065640 2024-05-06T12:11:15.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.582+0000 7f5068bdc700 1 -- 172.21.15.92:0/840770622 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f506410dd80 con 0x7f5064065640 2024-05-06T12:11:15.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.582+0000 7f5068bdc700 1 -- 172.21.15.92:0/840770622 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f506410e340 con 0x7f5064065640 2024-05-06T12:11:15.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.582+0000 7f5052ffd700 1 -- 172.21.15.92:0/840770622 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f505801c3f0 con 0x7f5064065640 2024-05-06T12:11:15.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.583+0000 7f5068bdc700 1 -- 172.21.15.92:0/840770622 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50440052b0 con 0x7f5064065640 2024-05-06T12:11:15.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.583+0000 7f5052ffd700 1 -- 172.21.15.92:0/840770622 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f505800bd30 con 0x7f5064065640 2024-05-06T12:11:15.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.584+0000 7f5052ffd700 1 --2- 172.21.15.92:0/840770622 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f504c0692a0 0x7f504c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:15.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.584+0000 7f5052ffd700 1 -- 172.21.15.92:0/840770622 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f5058088030 con 0x7f5064065640 2024-05-06T12:11:15.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.584+0000 7f5061d9b700 1 --2- 172.21.15.92:0/840770622 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f504c0692a0 0x7f504c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:15.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.584+0000 7f5061d9b700 1 --2- 172.21.15.92:0/840770622 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f504c0692a0 0x7f504c06b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f506410ecc0 tx=0x7f5054008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:15.591 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.588+0000 7f5052ffd700 1 -- 172.21.15.92:0/840770622 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f50580580b0 con 0x7f5064065640 2024-05-06T12:11:15.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.769+0000 7f185c62f700 1 -- 172.21.15.92:0/3922723420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1854065640 msgr2=0x7f1854065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:15.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.769+0000 7f185c62f700 1 --2- 172.21.15.92:0/3922723420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1854065640 0x7f1854065ab0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1844004940 tx=0x7f1844009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:15.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.769+0000 7f185c62f700 1 -- 172.21.15.92:0/3922723420 shutdown_connections 2024-05-06T12:11:15.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.769+0000 7f185c62f700 1 --2- 172.21.15.92:0/3922723420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1854065640 0x7f1854065ab0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:15.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.769+0000 7f185c62f700 1 --2- 172.21.15.92:0/3922723420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1854102e50 0x7f1854103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:15.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.769+0000 7f185c62f700 1 -- 172.21.15.92:0/3922723420 >> 172.21.15.92:0/3922723420 conn(0x7f1854071d00 msgr2=0x7f1854072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:15.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.769+0000 7f185c62f700 1 -- 172.21.15.92:0/3922723420 shutdown_connections 2024-05-06T12:11:15.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.769+0000 7f185c62f700 1 -- 172.21.15.92:0/3922723420 wait complete. 2024-05-06T12:11:15.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.769+0000 7f185c62f700 1 Processor -- start 2024-05-06T12:11:15.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.769+0000 7f185c62f700 1 -- start start 2024-05-06T12:11:15.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.770+0000 7f185c62f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1854065640 0x7f18540fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:15.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.770+0000 7f185c62f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1854102e50 0x7f18540fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:15.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.770+0000 7f185c62f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18540fc880 con 0x7f1854102e50 2024-05-06T12:11:15.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.770+0000 7f185c62f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18540fc9c0 con 0x7f1854065640 2024-05-06T12:11:15.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.770+0000 7f1859bca700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1854102e50 0x7f18540fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:15.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.770+0000 7f1859bca700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1854102e50 0x7f18540fc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60466/0 (socket says 172.21.15.92:60466) 2024-05-06T12:11:15.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.770+0000 7f1859bca700 1 -- 172.21.15.92:0/2828284965 learned_addr learned my addr 172.21.15.92:0/2828284965 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:11:15.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.770+0000 7f185a3cb700 1 --2- 172.21.15.92:0/2828284965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1854065640 0x7f18540fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:15.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.770+0000 7f1859bca700 1 -- 172.21.15.92:0/2828284965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1854065640 msgr2=0x7f18540fbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:15.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.770+0000 7f1859bca700 1 --2- 172.21.15.92:0/2828284965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1854065640 0x7f18540fbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:15.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.770+0000 7f1859bca700 1 -- 172.21.15.92:0/2828284965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1844005040 con 0x7f1854102e50 2024-05-06T12:11:15.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.770+0000 7f1859bca700 1 --2- 172.21.15.92:0/2828284965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1854102e50 0x7f18540fc2f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f184400b3b0 tx=0x7f1844009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:15.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.770+0000 7f184affd700 1 -- 172.21.15.92:0/2828284965 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f184400e040 con 0x7f1854102e50 2024-05-06T12:11:15.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.770+0000 7f184affd700 1 -- 172.21.15.92:0/2828284965 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1844013420 con 0x7f1854102e50 2024-05-06T12:11:15.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.770+0000 7f185c62f700 1 -- 172.21.15.92:0/2828284965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18540fcc10 con 0x7f1854102e50 2024-05-06T12:11:15.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.770+0000 7f184affd700 1 -- 172.21.15.92:0/2828284965 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f184401c630 con 0x7f1854102e50 2024-05-06T12:11:15.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.771+0000 7f185c62f700 1 -- 172.21.15.92:0/2828284965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f185411a250 con 0x7f1854102e50 2024-05-06T12:11:15.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.772+0000 7f185c62f700 1 -- 172.21.15.92:0/2828284965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1854100100 con 0x7f1854102e50 2024-05-06T12:11:15.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.772+0000 7f184affd700 1 -- 172.21.15.92:0/2828284965 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f1844013590 con 0x7f1854102e50 2024-05-06T12:11:15.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.772+0000 7f184affd700 1 --2- 172.21.15.92:0/2828284965 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f18400692a0 0x7f184006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:15.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.773+0000 7f184affd700 1 -- 172.21.15.92:0/2828284965 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1844088030 con 0x7f1854102e50 2024-05-06T12:11:15.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.773+0000 7f185a3cb700 1 --2- 172.21.15.92:0/2828284965 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f18400692a0 0x7f184006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:15.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.773+0000 7f185a3cb700 1 --2- 172.21.15.92:0/2828284965 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f18400692a0 0x7f184006b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1850004f40 tx=0x7f185000b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:15.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.777+0000 7f184affd700 1 -- 172.21.15.92:0/2828284965 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f18440546a0 con 0x7f1854102e50 2024-05-06T12:11:15.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.784+0000 7f5068bdc700 1 -- 172.21.15.92:0/840770622 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5044005560 con 0x7f5064065640 2024-05-06T12:11:15.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.784+0000 7f5052ffd700 1 -- 172.21.15.92:0/840770622 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5058057c40 con 0x7f5064065640 2024-05-06T12:11:15.787 INFO:teuthology.orchestra.run.smithi092.stdout:38654705703 2024-05-06T12:11:15.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.786+0000 7f5068bdc700 1 -- 172.21.15.92:0/840770622 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f504c0692a0 msgr2=0x7f504c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:15.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.786+0000 7f5068bdc700 1 --2- 172.21.15.92:0/840770622 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f504c0692a0 0x7f504c06b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f506410ecc0 tx=0x7f5054008040 comp rx=0 tx=0).stop 2024-05-06T12:11:15.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.786+0000 7f5068bdc700 1 -- 172.21.15.92:0/840770622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5064065640 msgr2=0x7f5064113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:15.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.786+0000 7f5068bdc700 1 --2- 172.21.15.92:0/840770622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5064065640 0x7f5064113b80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f505800b3b0 tx=0x7f5058009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:15.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.787+0000 7f5068bdc700 1 -- 172.21.15.92:0/840770622 shutdown_connections 2024-05-06T12:11:15.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.787+0000 7f5068bdc700 1 --2- 172.21.15.92:0/840770622 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f504c0692a0 0x7f504c06b750 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:15.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.787+0000 7f5068bdc700 1 --2- 172.21.15.92:0/840770622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5064065640 0x7f5064113b80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:15.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.787+0000 7f5068bdc700 1 --2- 172.21.15.92:0/840770622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5064102e50 0x7f50641140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:15.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.787+0000 7f5068bdc700 1 -- 172.21.15.92:0/840770622 >> 172.21.15.92:0/840770622 conn(0x7f5064071d00 msgr2=0x7f50640fa0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:15.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.787+0000 7f5068bdc700 1 -- 172.21.15.92:0/840770622 shutdown_connections 2024-05-06T12:11:15.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.787+0000 7f5068bdc700 1 -- 172.21.15.92:0/840770622 wait complete. 2024-05-06T12:11:15.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.987+0000 7f185c62f700 1 -- 172.21.15.92:0/2828284965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f18540fe540 con 0x7f1854102e50 2024-05-06T12:11:15.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.987+0000 7f184affd700 1 -- 172.21.15.92:0/2828284965 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1844057cc0 con 0x7f1854102e50 2024-05-06T12:11:15.990 INFO:teuthology.orchestra.run.smithi092.stdout:81604378652 2024-05-06T12:11:15.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.989+0000 7f185c62f700 1 -- 172.21.15.92:0/2828284965 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f18400692a0 msgr2=0x7f184006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:15.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.989+0000 7f185c62f700 1 --2- 172.21.15.92:0/2828284965 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f18400692a0 0x7f184006b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1850004f40 tx=0x7f185000b040 comp rx=0 tx=0).stop 2024-05-06T12:11:15.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.989+0000 7f185c62f700 1 -- 172.21.15.92:0/2828284965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1854102e50 msgr2=0x7f18540fc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:15.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.989+0000 7f185c62f700 1 --2- 172.21.15.92:0/2828284965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1854102e50 0x7f18540fc2f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f184400b3b0 tx=0x7f1844009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:15.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.989+0000 7f185c62f700 1 -- 172.21.15.92:0/2828284965 shutdown_connections 2024-05-06T12:11:15.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.989+0000 7f185c62f700 1 --2- 172.21.15.92:0/2828284965 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f18400692a0 0x7f184006b750 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:15.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.989+0000 7f185c62f700 1 --2- 172.21.15.92:0/2828284965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1854065640 0x7f18540fbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:15.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.989+0000 7f185c62f700 1 --2- 172.21.15.92:0/2828284965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1854102e50 0x7f18540fc2f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:15.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.989+0000 7f185c62f700 1 -- 172.21.15.92:0/2828284965 >> 172.21.15.92:0/2828284965 conn(0x7f1854071d00 msgr2=0x7f18540fb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:15.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.989+0000 7f185c62f700 1 -- 172.21.15.92:0/2828284965 shutdown_connections 2024-05-06T12:11:15.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:15.989+0000 7f185c62f700 1 -- 172.21.15.92:0/2828284965 wait complete. 2024-05-06T12:11:16.002 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:15 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/840770622' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-06T12:11:16.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:15 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/840770622' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-06T12:11:16.620 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378649 got 81604378652 for osd.2 2024-05-06T12:11:16.620 DEBUG:teuthology.parallel:result is None 2024-05-06T12:11:17.031 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:16 smithi092 ceph-mon[26954]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:17.031 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:16 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/2828284965' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-06T12:11:17.055 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705700 got 38654705703 for osd.0 2024-05-06T12:11:17.055 DEBUG:teuthology.parallel:result is None 2024-05-06T12:11:17.055 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-06T12:11:17.055 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-06T12:11:17.307 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:11:17.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:16 smithi190 ceph-mon[33798]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:17.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:16 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/2828284965' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-06T12:11:18.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.869+0000 7fc01f138700 1 -- 172.21.15.92:0/2907003610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc018103780 msgr2=0x7fc0180fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:18.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.869+0000 7fc01f138700 1 --2- 172.21.15.92:0/2907003610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc018103780 0x7fc0180fcdf0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc004004940 tx=0x7fc004009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:18.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.869+0000 7fc01f138700 1 -- 172.21.15.92:0/2907003610 shutdown_connections 2024-05-06T12:11:18.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.869+0000 7fc01f138700 1 --2- 172.21.15.92:0/2907003610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc018103780 0x7fc0180fcdf0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:18.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.869+0000 7fc01f138700 1 --2- 172.21.15.92:0/2907003610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc018102de0 0x7fc0181031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:18.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.869+0000 7fc01f138700 1 -- 172.21.15.92:0/2907003610 >> 172.21.15.92:0/2907003610 conn(0x7fc0180f8950 msgr2=0x7fc0180fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:18.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.869+0000 7fc01f138700 1 -- 172.21.15.92:0/2907003610 shutdown_connections 2024-05-06T12:11:18.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.869+0000 7fc01f138700 1 -- 172.21.15.92:0/2907003610 wait complete. 2024-05-06T12:11:18.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.869+0000 7fc01f138700 1 Processor -- start 2024-05-06T12:11:18.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.869+0000 7fc01f138700 1 -- start start 2024-05-06T12:11:18.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.870+0000 7fc01f138700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc018102de0 0x7fc018070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:18.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.870+0000 7fc01f138700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc018103780 0x7fc0180707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:18.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.870+0000 7fc01f138700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc018070d30 con 0x7fc018102de0 2024-05-06T12:11:18.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.870+0000 7fc01f138700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc018070e70 con 0x7fc018103780 2024-05-06T12:11:18.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.870+0000 7fc01ced4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc018102de0 0x7fc018070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:18.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.870+0000 7fc01ced4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc018102de0 0x7fc018070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60482/0 (socket says 172.21.15.92:60482) 2024-05-06T12:11:18.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.870+0000 7fc01ced4700 1 -- 172.21.15.92:0/1766397176 learned_addr learned my addr 172.21.15.92:0/1766397176 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:11:18.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.870+0000 7fc017fff700 1 --2- 172.21.15.92:0/1766397176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc018103780 0x7fc0180707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:18.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.870+0000 7fc01ced4700 1 -- 172.21.15.92:0/1766397176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc018103780 msgr2=0x7fc0180707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:18.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.870+0000 7fc01ced4700 1 --2- 172.21.15.92:0/1766397176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc018103780 0x7fc0180707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:18.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.870+0000 7fc01ced4700 1 -- 172.21.15.92:0/1766397176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc004005040 con 0x7fc018102de0 2024-05-06T12:11:18.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.870+0000 7fc01ced4700 1 --2- 172.21.15.92:0/1766397176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc018102de0 0x7fc018070260 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00c830 tx=0x7fc00c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:18.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.871+0000 7fc0157fa700 1 -- 172.21.15.92:0/1766397176 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc00c0106f0 con 0x7fc018102de0 2024-05-06T12:11:18.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.871+0000 7fc01f138700 1 -- 172.21.15.92:0/1766397176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc018119eb0 con 0x7fc018102de0 2024-05-06T12:11:18.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.871+0000 7fc01f138700 1 -- 172.21.15.92:0/1766397176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc01811a2d0 con 0x7fc018102de0 2024-05-06T12:11:18.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.871+0000 7fc0157fa700 1 -- 172.21.15.92:0/1766397176 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc00c010d20 con 0x7fc018102de0 2024-05-06T12:11:18.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.871+0000 7fc0157fa700 1 -- 172.21.15.92:0/1766397176 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc00c0113a0 con 0x7fc018102de0 2024-05-06T12:11:18.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.872+0000 7fc01f138700 1 -- 172.21.15.92:0/1766397176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbffc0052b0 con 0x7fc018102de0 2024-05-06T12:11:18.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.872+0000 7fc0157fa700 1 -- 172.21.15.92:0/1766397176 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fc00c010850 con 0x7fc018102de0 2024-05-06T12:11:18.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.872+0000 7fc0157fa700 1 --2- 172.21.15.92:0/1766397176 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fc0080692a0 0x7fc00806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:18.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.873+0000 7fc0157fa700 1 -- 172.21.15.92:0/1766397176 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc00c086c60 con 0x7fc018102de0 2024-05-06T12:11:18.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.873+0000 7fc017fff700 1 --2- 172.21.15.92:0/1766397176 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fc0080692a0 0x7fc00806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:18.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.873+0000 7fc017fff700 1 --2- 172.21.15.92:0/1766397176 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fc0080692a0 0x7fc00806b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc004004890 tx=0x7fc004009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:18.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:18.877+0000 7fc0157fa700 1 -- 172.21.15.92:0/1766397176 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc00c053370 con 0x7fc018102de0 2024-05-06T12:11:19.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:19.059+0000 7fc01f138700 1 -- 172.21.15.92:0/1766397176 --> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbffc000bc0 con 0x7fc0080692a0 2024-05-06T12:11:19.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:19.060+0000 7fc0157fa700 1 -- 172.21.15.92:0/1766397176 <== mgr.14210 v2:172.21.15.92:6800/3045905943 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27672 (secure 0 0 0) 0x7fbffc000bc0 con 0x7fc0080692a0 2024-05-06T12:11:19.064 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:11:19.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:19.062+0000 7fc01f138700 1 -- 172.21.15.92:0/1766397176 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fc0080692a0 msgr2=0x7fc00806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:19.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:19.062+0000 7fc01f138700 1 --2- 172.21.15.92:0/1766397176 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fc0080692a0 0x7fc00806b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc004004890 tx=0x7fc004009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:19.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:19.062+0000 7fc01f138700 1 -- 172.21.15.92:0/1766397176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc018102de0 msgr2=0x7fc018070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:19.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:19.062+0000 7fc01f138700 1 --2- 172.21.15.92:0/1766397176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc018102de0 0x7fc018070260 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00c830 tx=0x7fc00c004470 comp rx=0 tx=0).stop 2024-05-06T12:11:19.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:19.063+0000 7fc01f138700 1 -- 172.21.15.92:0/1766397176 shutdown_connections 2024-05-06T12:11:19.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:19.063+0000 7fc01f138700 1 --2- 172.21.15.92:0/1766397176 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fc0080692a0 0x7fc00806b750 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:19.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:19.063+0000 7fc01f138700 1 --2- 172.21.15.92:0/1766397176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc018102de0 0x7fc018070260 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:19.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:19.063+0000 7fc01f138700 1 --2- 172.21.15.92:0/1766397176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc018103780 0x7fc0180707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:19.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:19.063+0000 7fc01f138700 1 -- 172.21.15.92:0/1766397176 >> 172.21.15.92:0/1766397176 conn(0x7fc0180f8950 msgr2=0x7fc0180f99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:19.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:19.063+0000 7fc01f138700 1 -- 172.21.15.92:0/1766397176 shutdown_connections 2024-05-06T12:11:19.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:19.063+0000 7fc01f138700 1 -- 172.21.15.92:0/1766397176 wait complete. 2024-05-06T12:11:19.067 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-05-06T12:11:19.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:18 smithi190 ceph-mon[33798]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:19.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:18 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:11:19.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:18 smithi092 ceph-mon[26954]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:19.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:18 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:11:19.957 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2024-05-06T12:11:17.849525+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1788024,"kb_used_data":3876,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682888,"statfs":{"total":575970213888,"available":574139277312,"internally_reserved":0,"allocated":3969024,"data_stored":2606262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001977"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":190,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-06T12:10:26.807054+0000","last_change":"2024-05-06T12:10:00.847340+0000","last_active":"2024-05-06T12:10:26.807054+0000","last_peered":"2024-05-06T12:10:26.807054+0000","last_clean":"2024-05-06T12:10:26.807054+0000","last_became_active":"2024-05-06T12:10:00.847062+0000","last_became_peered":"2024-05-06T12:10:00.847062+0000","last_unstale":"2024-05-06T12:10:26.807054+0000","last_undegraded":"2024-05-06T12:10:26.807054+0000","last_fullsized":"2024-05-06T12:10:26.807054+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-06T12:09:12.780765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-06T12:09:12.780765+0000","last_clean_scrub_stamp":"2024-05-06T12:09:12.780765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-07T23:56:31.341402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]}]},{"osd":4,"up_from":34,"seq":146028888082,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon May 6 12:11:06 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.40400000000000003}]},{"osd":1,"last update":"Mon May 6 12:11:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.53900000000000003}]},{"osd":2,"last update":"Mon May 6 12:11:06 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.52600000000000002}]},{"osd":3,"last update":"Mon May 6 12:11:06 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.48099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]}]},{"osd":3,"up_from":26,"seq":111669149719,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298292,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446860,"statfs":{"total":95995035648,"available":95689584640,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon May 6 12:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.41899999999999998}]},{"osd":1,"last update":"Mon May 6 12:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.51100000000000001}]},{"osd":2,"last update":"Mon May 6 12:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47699999999999998}]},{"osd":4,"last update":"Mon May 6 12:11:06 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.159,"5min":0.159,"15min":0.159},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.308},{"interface":"front","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.34000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297716,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447436,"statfs":{"total":95995035648,"available":95690174464,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon May 6 12:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.35299999999999998,"15min":0.35699999999999998},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.35999999999999999,"15min":0.36599999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.34499999999999997}]},{"osd":1,"last update":"Mon May 6 12:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.372,"15min":0.374},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.38100000000000001,"15min":0.38400000000000001},"min":{"1min":0.25800000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.314}]},{"osd":3,"last update":"Mon May 6 12:10:36 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.70299999999999996}]},{"osd":4,"last update":"Mon May 6 12:11:02 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.56399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]}]},{"osd":0,"up_from":9,"seq":38654705703,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon May 6 12:10:47 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.29599999999999999,"15min":0.28899999999999998},"min":{"1min":0.24199999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.31900000000000001,"15min":0.315},"min":{"1min":0.25700000000000001,"5min":0.193,"15min":0.193},"max":{"1min":0.45100000000000001,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.307}]},{"osd":2,"last update":"Mon May 6 12:10:15 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.35599999999999998}]},{"osd":3,"last update":"Mon May 6 12:10:38 2024","interfaces":[{"interface":"back","average":{"1min":1.619,"5min":1.619,"15min":1.619},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":19.143000000000001,"5min":19.143000000000001,"15min":19.143000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.67200000000000004}]},{"osd":4,"last update":"Mon May 6 12:11:02 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.59499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]}]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon May 6 12:10:47 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"min":{"1min":0.191,"5min":0.19,"15min":0.19},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.442},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.25,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.47299999999999998}]},{"osd":2,"last update":"Mon May 6 12:10:15 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"last":0.41999999999999998}]},{"osd":3,"last update":"Mon May 6 12:10:38 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.624}]},{"osd":4,"last update":"Mon May 6 12:11:05 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.54300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-06T12:11:19.958 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-06T12:11:20.210 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:11:20.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:19 smithi092 ceph-mon[26954]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-06T12:11:20.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:19 smithi190 ceph-mon[33798]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-06T12:11:21.347 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:20 smithi092 ceph-mon[26954]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:21.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:20 smithi190 ceph-mon[33798]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:21.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.860+0000 7feef2b77700 1 -- 172.21.15.92:0/764611470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feeec101e20 msgr2=0x7feeec1021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:21.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.860+0000 7feef2b77700 1 --2- 172.21.15.92:0/764611470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feeec101e20 0x7feeec1021f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feee8009ce0 tx=0x7feee8004d10 comp rx=0 tx=0).stop 2024-05-06T12:11:21.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.860+0000 7feef2b77700 1 -- 172.21.15.92:0/764611470 shutdown_connections 2024-05-06T12:11:21.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.860+0000 7feef2b77700 1 --2- 172.21.15.92:0/764611470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feeec0fc710 0x7feeec0fcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:21.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.860+0000 7feef2b77700 1 --2- 172.21.15.92:0/764611470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feeec101e20 0x7feeec1021f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:21.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.860+0000 7feef2b77700 1 -- 172.21.15.92:0/764611470 >> 172.21.15.92:0/764611470 conn(0x7feeec071460 msgr2=0x7feeec071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:21.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.860+0000 7feef2b77700 1 -- 172.21.15.92:0/764611470 shutdown_connections 2024-05-06T12:11:21.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.860+0000 7feef2b77700 1 -- 172.21.15.92:0/764611470 wait complete. 2024-05-06T12:11:21.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.860+0000 7feef2b77700 1 Processor -- start 2024-05-06T12:11:21.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.861+0000 7feef2b77700 1 -- start start 2024-05-06T12:11:21.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.861+0000 7feef2b77700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feeec0fc710 0x7feeec113d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:21.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.861+0000 7feef2b77700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feeec101e20 0x7feeec1142c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:21.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.861+0000 7feef2b77700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec1149a0 con 0x7feeec101e20 2024-05-06T12:11:21.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.861+0000 7feef2b77700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec10dce0 con 0x7feeec0fc710 2024-05-06T12:11:21.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.861+0000 7feee3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feeec101e20 0x7feeec1142c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:21.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.861+0000 7feee3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feeec101e20 0x7feeec1142c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60502/0 (socket says 172.21.15.92:60502) 2024-05-06T12:11:21.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.861+0000 7feee3fff700 1 -- 172.21.15.92:0/2980197630 learned_addr learned my addr 172.21.15.92:0/2980197630 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:11:21.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.861+0000 7feef0913700 1 --2- 172.21.15.92:0/2980197630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feeec0fc710 0x7feeec113d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:21.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.861+0000 7feee3fff700 1 -- 172.21.15.92:0/2980197630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feeec0fc710 msgr2=0x7feeec113d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:21.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.861+0000 7feee3fff700 1 --2- 172.21.15.92:0/2980197630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feeec0fc710 0x7feeec113d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:21.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.861+0000 7feee3fff700 1 -- 172.21.15.92:0/2980197630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee8009940 con 0x7feeec101e20 2024-05-06T12:11:21.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.862+0000 7feee3fff700 1 --2- 172.21.15.92:0/2980197630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feeec101e20 0x7feeec1142c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7feedc00b790 tx=0x7feedc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:21.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.862+0000 7feee17fa700 1 -- 172.21.15.92:0/2980197630 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feedc00f770 con 0x7feeec101e20 2024-05-06T12:11:21.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.862+0000 7feef2b77700 1 -- 172.21.15.92:0/2980197630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeec10df60 con 0x7feeec101e20 2024-05-06T12:11:21.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.862+0000 7feee17fa700 1 -- 172.21.15.92:0/2980197630 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feedc00fda0 con 0x7feeec101e20 2024-05-06T12:11:21.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.862+0000 7feef2b77700 1 -- 172.21.15.92:0/2980197630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeec10e4a0 con 0x7feeec101e20 2024-05-06T12:11:21.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.862+0000 7feee17fa700 1 -- 172.21.15.92:0/2980197630 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feedc00d310 con 0x7feeec101e20 2024-05-06T12:11:21.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.863+0000 7feef2b77700 1 -- 172.21.15.92:0/2980197630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed80052e0 con 0x7feeec101e20 2024-05-06T12:11:21.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.863+0000 7feee17fa700 1 -- 172.21.15.92:0/2980197630 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7feedc00d5b0 con 0x7feeec101e20 2024-05-06T12:11:21.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.863+0000 7feee17fa700 1 --2- 172.21.15.92:0/2980197630 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7feed40692a0 0x7feed406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:21.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.864+0000 7feee17fa700 1 -- 172.21.15.92:0/2980197630 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7feedc086e00 con 0x7feeec101e20 2024-05-06T12:11:21.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.864+0000 7feef0913700 1 --2- 172.21.15.92:0/2980197630 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7feed40692a0 0x7feed406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:21.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.864+0000 7feef0913700 1 --2- 172.21.15.92:0/2980197630 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7feed40692a0 0x7feed406b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feee8015040 tx=0x7feee8004d10 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:21.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:21.868+0000 7feee17fa700 1 -- 172.21.15.92:0/2980197630 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7feedc056f70 con 0x7feeec101e20 2024-05-06T12:11:22.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:22.048+0000 7feef2b77700 1 -- 172.21.15.92:0/2980197630 --> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feed8000bf0 con 0x7feed40692a0 2024-05-06T12:11:22.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:22.050+0000 7feee17fa700 1 -- 172.21.15.92:0/2980197630 <== mgr.14210 v2:172.21.15.92:6800/3045905943 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27670 (secure 0 0 0) 0x7feed8000bf0 con 0x7feed40692a0 2024-05-06T12:11:22.053 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:11:22.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:22.052+0000 7feef2b77700 1 -- 172.21.15.92:0/2980197630 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7feed40692a0 msgr2=0x7feed406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:22.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:22.052+0000 7feef2b77700 1 --2- 172.21.15.92:0/2980197630 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7feed40692a0 0x7feed406b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feee8015040 tx=0x7feee8004d10 comp rx=0 tx=0).stop 2024-05-06T12:11:22.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:22.052+0000 7feef2b77700 1 -- 172.21.15.92:0/2980197630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feeec101e20 msgr2=0x7feeec1142c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:22.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:22.052+0000 7feef2b77700 1 --2- 172.21.15.92:0/2980197630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feeec101e20 0x7feeec1142c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7feedc00b790 tx=0x7feedc009250 comp rx=0 tx=0).stop 2024-05-06T12:11:22.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:22.052+0000 7feef2b77700 1 -- 172.21.15.92:0/2980197630 shutdown_connections 2024-05-06T12:11:22.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:22.052+0000 7feef2b77700 1 --2- 172.21.15.92:0/2980197630 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7feed40692a0 0x7feed406b750 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:22.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:22.052+0000 7feef2b77700 1 --2- 172.21.15.92:0/2980197630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feeec0fc710 0x7feeec113d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:22.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:22.052+0000 7feef2b77700 1 --2- 172.21.15.92:0/2980197630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feeec101e20 0x7feeec1142c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:22.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:22.052+0000 7feef2b77700 1 -- 172.21.15.92:0/2980197630 >> 172.21.15.92:0/2980197630 conn(0x7feeec071460 msgr2=0x7feeec0ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:22.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:22.052+0000 7feef2b77700 1 -- 172.21.15.92:0/2980197630 shutdown_connections 2024-05-06T12:11:22.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:22.052+0000 7feef2b77700 1 -- 172.21.15.92:0/2980197630 wait complete. 2024-05-06T12:11:22.056 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-05-06T12:11:22.679 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-05-06T12:11:21.850208+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1788024,"kb_used_data":3876,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682888,"statfs":{"total":575970213888,"available":574139277312,"internally_reserved":0,"allocated":3969024,"data_stored":2606262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001920"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":190,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-06T12:10:26.807054+0000","last_change":"2024-05-06T12:10:00.847340+0000","last_active":"2024-05-06T12:10:26.807054+0000","last_peered":"2024-05-06T12:10:26.807054+0000","last_clean":"2024-05-06T12:10:26.807054+0000","last_became_active":"2024-05-06T12:10:00.847062+0000","last_became_peered":"2024-05-06T12:10:00.847062+0000","last_unstale":"2024-05-06T12:10:26.807054+0000","last_undegraded":"2024-05-06T12:10:26.807054+0000","last_fullsized":"2024-05-06T12:10:26.807054+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-06T12:09:12.780765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-06T12:09:12.780765+0000","last_clean_scrub_stamp":"2024-05-06T12:09:12.780765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-07T23:56:31.341402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]}]},{"osd":4,"up_from":34,"seq":146028888083,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon May 6 12:11:06 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.61199999999999999}]},{"osd":1,"last update":"Mon May 6 12:11:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.57999999999999996}]},{"osd":2,"last update":"Mon May 6 12:11:06 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.623}]},{"osd":3,"last update":"Mon May 6 12:11:06 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.41099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]}]},{"osd":3,"up_from":26,"seq":111669149720,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298292,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446860,"statfs":{"total":95995035648,"available":95689584640,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon May 6 12:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.48999999999999999}]},{"osd":1,"last update":"Mon May 6 12:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.45700000000000002}]},{"osd":2,"last update":"Mon May 6 12:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.41099999999999998}]},{"osd":4,"last update":"Mon May 6 12:11:06 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.159,"5min":0.159,"15min":0.159},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.373},{"interface":"front","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.33700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297716,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447436,"statfs":{"total":95995035648,"available":95690174464,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon May 6 12:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.35299999999999998,"15min":0.35699999999999998},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.35999999999999999,"15min":0.36599999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.34499999999999997}]},{"osd":1,"last update":"Mon May 6 12:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.372,"15min":0.374},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.38100000000000001,"15min":0.38400000000000001},"min":{"1min":0.25800000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.314}]},{"osd":3,"last update":"Mon May 6 12:10:36 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.70299999999999996}]},{"osd":4,"last update":"Mon May 6 12:11:02 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.56399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]}]},{"osd":0,"up_from":9,"seq":38654705704,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon May 6 12:10:47 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.29599999999999999,"15min":0.28899999999999998},"min":{"1min":0.24199999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"last":0.25},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.31900000000000001,"15min":0.315},"min":{"1min":0.25700000000000001,"5min":0.193,"15min":0.193},"max":{"1min":0.45100000000000001,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.29099999999999998}]},{"osd":2,"last update":"Mon May 6 12:10:15 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.315}]},{"osd":3,"last update":"Mon May 6 12:10:38 2024","interfaces":[{"interface":"back","average":{"1min":1.619,"5min":1.619,"15min":1.619},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":19.143000000000001,"5min":19.143000000000001,"15min":19.143000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.53600000000000003}]},{"osd":4,"last update":"Mon May 6 12:11:02 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":18.23},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":18.242999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon May 6 12:10:47 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"min":{"1min":0.191,"5min":0.19,"15min":0.19},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.20100000000000001},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.25,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.23499999999999999}]},{"osd":2,"last update":"Mon May 6 12:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.26000000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.248},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.35499999999999998,"15min":0.35799999999999998},"min":{"1min":0.24299999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.26300000000000001}]},{"osd":3,"last update":"Mon May 6 12:10:38 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.60299999999999998}]},{"osd":4,"last update":"Mon May 6 12:11:05 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.50800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-06T12:11:22.680 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-06T12:11:22.680 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-06T12:11:22.680 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-06T12:11:22.680 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph health --format=json 2024-05-06T12:11:22.932 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:11:23.256 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:22 smithi092 ceph-mon[26954]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:23.256 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:22 smithi092 ceph-mon[26954]: from='client.14416 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-06T12:11:23.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:22 smithi190 ceph-mon[33798]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:23.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:22 smithi190 ceph-mon[33798]: from='client.14416 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-06T12:11:24.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.455+0000 7fd413840700 1 -- 172.21.15.92:0/763062187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd40c102e50 msgr2=0x7fd40c103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:24.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.455+0000 7fd413840700 1 --2- 172.21.15.92:0/763062187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd40c102e50 0x7fd40c103220 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd408004940 tx=0x7fd408009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:24.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.455+0000 7fd413840700 1 -- 172.21.15.92:0/763062187 shutdown_connections 2024-05-06T12:11:24.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.455+0000 7fd413840700 1 --2- 172.21.15.92:0/763062187 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd40c065640 0x7fd40c065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:24.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.455+0000 7fd413840700 1 --2- 172.21.15.92:0/763062187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd40c102e50 0x7fd40c103220 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:24.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.455+0000 7fd413840700 1 -- 172.21.15.92:0/763062187 >> 172.21.15.92:0/763062187 conn(0x7fd40c071d00 msgr2=0x7fd40c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:24.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.455+0000 7fd413840700 1 -- 172.21.15.92:0/763062187 shutdown_connections 2024-05-06T12:11:24.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.455+0000 7fd413840700 1 -- 172.21.15.92:0/763062187 wait complete. 2024-05-06T12:11:24.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.456+0000 7fd413840700 1 Processor -- start 2024-05-06T12:11:24.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.456+0000 7fd413840700 1 -- start start 2024-05-06T12:11:24.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.456+0000 7fd413840700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd40c065640 0x7fd40c113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:24.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.456+0000 7fd413840700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd40c102e50 0x7fd40c1140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:24.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.456+0000 7fd413840700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c114710 con 0x7fd40c065640 2024-05-06T12:11:24.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.456+0000 7fd413840700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c10db00 con 0x7fd40c102e50 2024-05-06T12:11:24.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.456+0000 7fd4115dc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd40c065640 0x7fd40c113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:24.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.456+0000 7fd4115dc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd40c065640 0x7fd40c113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49896/0 (socket says 172.21.15.92:49896) 2024-05-06T12:11:24.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.456+0000 7fd4115dc700 1 -- 172.21.15.92:0/4096059580 learned_addr learned my addr 172.21.15.92:0/4096059580 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:11:24.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.456+0000 7fd410ddb700 1 --2- 172.21.15.92:0/4096059580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd40c102e50 0x7fd40c1140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:24.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.456+0000 7fd4115dc700 1 -- 172.21.15.92:0/4096059580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd40c102e50 msgr2=0x7fd40c1140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:24.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.456+0000 7fd4115dc700 1 --2- 172.21.15.92:0/4096059580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd40c102e50 0x7fd40c1140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:24.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.456+0000 7fd4115dc700 1 -- 172.21.15.92:0/4096059580 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd408005040 con 0x7fd40c065640 2024-05-06T12:11:24.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.456+0000 7fd4115dc700 1 --2- 172.21.15.92:0/4096059580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd40c065640 0x7fd40c113b80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd40800b8b0 tx=0x7fd408009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:24.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.457+0000 7fd401ffb700 1 -- 172.21.15.92:0/4096059580 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd40800e040 con 0x7fd40c065640 2024-05-06T12:11:24.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.457+0000 7fd401ffb700 1 -- 172.21.15.92:0/4096059580 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd408004290 con 0x7fd40c065640 2024-05-06T12:11:24.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.457+0000 7fd413840700 1 -- 172.21.15.92:0/4096059580 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd40c10dd80 con 0x7fd40c065640 2024-05-06T12:11:24.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.457+0000 7fd401ffb700 1 -- 172.21.15.92:0/4096059580 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd40801c3f0 con 0x7fd40c065640 2024-05-06T12:11:24.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.457+0000 7fd413840700 1 -- 172.21.15.92:0/4096059580 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd40c10e340 con 0x7fd40c065640 2024-05-06T12:11:24.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.458+0000 7fd413840700 1 -- 172.21.15.92:0/4096059580 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd40c100100 con 0x7fd40c065640 2024-05-06T12:11:24.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.458+0000 7fd401ffb700 1 -- 172.21.15.92:0/4096059580 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fd40801c670 con 0x7fd40c065640 2024-05-06T12:11:24.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.459+0000 7fd401ffb700 1 --2- 172.21.15.92:0/4096059580 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fd3f80692a0 0x7fd3f806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:24.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.459+0000 7fd401ffb700 1 -- 172.21.15.92:0/4096059580 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd4080881c0 con 0x7fd40c065640 2024-05-06T12:11:24.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.459+0000 7fd410ddb700 1 --2- 172.21.15.92:0/4096059580 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fd3f80692a0 0x7fd3f806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:24.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.459+0000 7fd410ddb700 1 --2- 172.21.15.92:0/4096059580 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fd3f80692a0 0x7fd3f806b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc007fd0 tx=0x7fd3fc009040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:24.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.463+0000 7fd401ffb700 1 -- 172.21.15.92:0/4096059580 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd4080582c0 con 0x7fd40c065640 2024-05-06T12:11:24.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.688+0000 7fd413840700 1 -- 172.21.15.92:0/4096059580 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd40c10f1d0 con 0x7fd40c065640 2024-05-06T12:11:24.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.688+0000 7fd401ffb700 1 -- 172.21.15.92:0/4096059580 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fd408057e50 con 0x7fd40c065640 2024-05-06T12:11:24.692 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:11:24.692 INFO:teuthology.orchestra.run.smithi092.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-06T12:11:24.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.690+0000 7fd413840700 1 -- 172.21.15.92:0/4096059580 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fd3f80692a0 msgr2=0x7fd3f806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:24.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.690+0000 7fd413840700 1 --2- 172.21.15.92:0/4096059580 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fd3f80692a0 0x7fd3f806b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc007fd0 tx=0x7fd3fc009040 comp rx=0 tx=0).stop 2024-05-06T12:11:24.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.690+0000 7fd413840700 1 -- 172.21.15.92:0/4096059580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd40c065640 msgr2=0x7fd40c113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:24.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.690+0000 7fd413840700 1 --2- 172.21.15.92:0/4096059580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd40c065640 0x7fd40c113b80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd40800b8b0 tx=0x7fd408009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:24.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.691+0000 7fd413840700 1 -- 172.21.15.92:0/4096059580 shutdown_connections 2024-05-06T12:11:24.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.691+0000 7fd413840700 1 --2- 172.21.15.92:0/4096059580 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fd3f80692a0 0x7fd3f806b750 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:24.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.691+0000 7fd413840700 1 --2- 172.21.15.92:0/4096059580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd40c065640 0x7fd40c113b80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:24.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.691+0000 7fd413840700 1 --2- 172.21.15.92:0/4096059580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd40c102e50 0x7fd40c1140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:24.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.691+0000 7fd413840700 1 -- 172.21.15.92:0/4096059580 >> 172.21.15.92:0/4096059580 conn(0x7fd40c071d00 msgr2=0x7fd40c0fb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:24.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.691+0000 7fd413840700 1 -- 172.21.15.92:0/4096059580 shutdown_connections 2024-05-06T12:11:24.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:24.691+0000 7fd413840700 1 -- 172.21.15.92:0/4096059580 wait complete. 2024-05-06T12:11:25.331 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:24 smithi092 ceph-mon[26954]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:25.331 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:24 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/4096059580' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-06T12:11:25.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:24 smithi190 ceph-mon[33798]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:25.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:24 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/4096059580' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-06T12:11:25.458 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-06T12:11:25.458 INFO:tasks.cephadm:Setup complete, yielding 2024-05-06T12:11:25.458 INFO:teuthology.run_tasks:Running task print... 2024-05-06T12:11:25.468 INFO:teuthology.task.print:**** done end installing quincy cephadm ... 2024-05-06T12:11:25.468 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-06T12:11:25.479 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi092.front.sepia.ceph.com 2024-05-06T12:11:25.479 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-05-06T12:11:25.728 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:11:27.159 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:26 smithi092 ceph-mon[26954]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:27.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:26 smithi190 ceph-mon[33798]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:27.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.377+0000 7f552587b700 1 -- 172.21.15.92:0/1087367993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55200fce80 msgr2=0x7f55200fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:27.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.377+0000 7f552587b700 1 --2- 172.21.15.92:0/1087367993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55200fce80 0x7f55200fd290 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5510004940 tx=0x7f5510009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:27.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.377+0000 7f552587b700 1 -- 172.21.15.92:0/1087367993 shutdown_connections 2024-05-06T12:11:27.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.377+0000 7f552587b700 1 --2- 172.21.15.92:0/1087367993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55200fe080 0x7f55200fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:27.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.377+0000 7f552587b700 1 --2- 172.21.15.92:0/1087367993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55200fce80 0x7f55200fd290 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:27.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.377+0000 7f552587b700 1 -- 172.21.15.92:0/1087367993 >> 172.21.15.92:0/1087367993 conn(0x7f55200f8590 msgr2=0x7f55200fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:27.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.377+0000 7f552587b700 1 -- 172.21.15.92:0/1087367993 shutdown_connections 2024-05-06T12:11:27.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.377+0000 7f552587b700 1 -- 172.21.15.92:0/1087367993 wait complete. 2024-05-06T12:11:27.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.377+0000 7f552587b700 1 Processor -- start 2024-05-06T12:11:27.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.377+0000 7f552587b700 1 -- start start 2024-05-06T12:11:27.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.377+0000 7f552587b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55200fce80 0x7f552010da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:27.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.377+0000 7f552587b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55200fe080 0x7f552010df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:27.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.377+0000 7f552587b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552010e570 con 0x7f55200fe080 2024-05-06T12:11:27.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.377+0000 7f552587b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552010edc0 con 0x7f55200fce80 2024-05-06T12:11:27.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.378+0000 7f5517fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55200fe080 0x7f552010df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:27.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.378+0000 7f5517fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55200fe080 0x7f552010df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49908/0 (socket says 172.21.15.92:49908) 2024-05-06T12:11:27.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.378+0000 7f5517fff700 1 -- 172.21.15.92:0/395396829 learned_addr learned my addr 172.21.15.92:0/395396829 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:11:27.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.378+0000 7f551effd700 1 --2- 172.21.15.92:0/395396829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55200fce80 0x7f552010da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:27.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.378+0000 7f5517fff700 1 -- 172.21.15.92:0/395396829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55200fce80 msgr2=0x7f552010da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:27.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.378+0000 7f5517fff700 1 --2- 172.21.15.92:0/395396829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55200fce80 0x7f552010da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:27.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.378+0000 7f5517fff700 1 -- 172.21.15.92:0/395396829 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5510005040 con 0x7f55200fe080 2024-05-06T12:11:27.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.378+0000 7f5517fff700 1 --2- 172.21.15.92:0/395396829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55200fe080 0x7f552010df70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5508002a50 tx=0x7f55080046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:27.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.378+0000 7f55177fe700 1 -- 172.21.15.92:0/395396829 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5508007cc0 con 0x7f55200fe080 2024-05-06T12:11:27.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.378+0000 7f55177fe700 1 -- 172.21.15.92:0/395396829 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f550800f040 con 0x7f55200fe080 2024-05-06T12:11:27.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.378+0000 7f552587b700 1 -- 172.21.15.92:0/395396829 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f552010f0a0 con 0x7f55200fe080 2024-05-06T12:11:27.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.378+0000 7f55177fe700 1 -- 172.21.15.92:0/395396829 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f550800da30 con 0x7f55200fe080 2024-05-06T12:11:27.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.378+0000 7f552587b700 1 -- 172.21.15.92:0/395396829 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f552010f5e0 con 0x7f55200fe080 2024-05-06T12:11:27.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.379+0000 7f552587b700 1 -- 172.21.15.92:0/395396829 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f552005f5c0 con 0x7f55200fe080 2024-05-06T12:11:27.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.380+0000 7f55177fe700 1 -- 172.21.15.92:0/395396829 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f550800db90 con 0x7f55200fe080 2024-05-06T12:11:27.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.380+0000 7f55177fe700 1 --2- 172.21.15.92:0/395396829 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f550c0692a0 0x7f550c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:27.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.380+0000 7f55177fe700 1 -- 172.21.15.92:0/395396829 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f5508086b30 con 0x7f55200fe080 2024-05-06T12:11:27.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.380+0000 7f551effd700 1 --2- 172.21.15.92:0/395396829 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f550c0692a0 0x7f550c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:27.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.380+0000 7f551effd700 1 --2- 172.21.15.92:0/395396829 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f550c0692a0 0x7f550c06b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5510004890 tx=0x7f5510009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:27.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.384+0000 7f55177fe700 1 -- 172.21.15.92:0/395396829 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5508056c30 con 0x7f55200fe080 2024-05-06T12:11:27.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.570+0000 7f552587b700 1 -- 172.21.15.92:0/395396829 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7f552004bed0 con 0x7f55200fe080 2024-05-06T12:11:27.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.574+0000 7f55177fe700 1 -- 172.21.15.92:0/395396829 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7f55080567c0 con 0x7f55200fe080 2024-05-06T12:11:27.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.576+0000 7f552587b700 1 -- 172.21.15.92:0/395396829 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f550c0692a0 msgr2=0x7f550c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:27.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.576+0000 7f552587b700 1 --2- 172.21.15.92:0/395396829 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f550c0692a0 0x7f550c06b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5510004890 tx=0x7f5510009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:27.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.576+0000 7f552587b700 1 -- 172.21.15.92:0/395396829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55200fe080 msgr2=0x7f552010df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:27.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.576+0000 7f552587b700 1 --2- 172.21.15.92:0/395396829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55200fe080 0x7f552010df70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5508002a50 tx=0x7f55080046b0 comp rx=0 tx=0).stop 2024-05-06T12:11:27.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.576+0000 7f552587b700 1 -- 172.21.15.92:0/395396829 shutdown_connections 2024-05-06T12:11:27.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.576+0000 7f552587b700 1 --2- 172.21.15.92:0/395396829 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f550c0692a0 0x7f550c06b750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:27.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.576+0000 7f552587b700 1 --2- 172.21.15.92:0/395396829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55200fce80 0x7f552010da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:27.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.576+0000 7f552587b700 1 --2- 172.21.15.92:0/395396829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55200fe080 0x7f552010df70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:27.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.576+0000 7f552587b700 1 -- 172.21.15.92:0/395396829 >> 172.21.15.92:0/395396829 conn(0x7f55200f8590 msgr2=0x7f55200f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:27.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.577+0000 7f552587b700 1 -- 172.21.15.92:0/395396829 shutdown_connections 2024-05-06T12:11:27.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:27.577+0000 7f552587b700 1 -- 172.21.15.92:0/395396829 wait complete. 2024-05-06T12:11:28.213 INFO:teuthology.run_tasks:Running task print... 2024-05-06T12:11:28.223 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-05-06T12:11:28.223 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-06T12:11:28.231 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi092.front.sepia.ceph.com 2024-05-06T12:11:28.231 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- bash -c 'ceph orch status' 2024-05-06T12:11:28.518 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:11:28.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:28 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/395396829' entity='client.admin' 2024-05-06T12:11:28.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:28 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:11:28.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:28 smithi190 ceph-mon[33798]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:28.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:28 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:11:28.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:28 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:11:28.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:28 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:11:28.917 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:28 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/395396829' entity='client.admin' 2024-05-06T12:11:28.917 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:28 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:11:28.917 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:28 smithi092 ceph-mon[26954]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:28.918 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:28 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:11:28.918 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:28 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:11:28.918 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:28 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:11:30.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.127+0000 7f6074b93700 1 -- 172.21.15.92:0/2814736002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60700fce60 msgr2=0x7f60700fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:30.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.127+0000 7f6074b93700 1 --2- 172.21.15.92:0/2814736002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60700fce60 0x7f60700fd270 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6058004940 tx=0x7f6058009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:30.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.128+0000 7f6074b93700 1 -- 172.21.15.92:0/2814736002 shutdown_connections 2024-05-06T12:11:30.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.128+0000 7f6074b93700 1 --2- 172.21.15.92:0/2814736002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60700fdfd0 0x7f60700fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:30.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.128+0000 7f6074b93700 1 --2- 172.21.15.92:0/2814736002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60700fce60 0x7f60700fd270 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:30.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.128+0000 7f6074b93700 1 -- 172.21.15.92:0/2814736002 >> 172.21.15.92:0/2814736002 conn(0x7f60700f8540 msgr2=0x7f60700fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:30.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.128+0000 7f6074b93700 1 -- 172.21.15.92:0/2814736002 shutdown_connections 2024-05-06T12:11:30.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.128+0000 7f6074b93700 1 -- 172.21.15.92:0/2814736002 wait complete. 2024-05-06T12:11:30.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.128+0000 7f6074b93700 1 Processor -- start 2024-05-06T12:11:30.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.128+0000 7f6074b93700 1 -- start start 2024-05-06T12:11:30.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.128+0000 7f6074b93700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60700fce60 0x7f607010d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:30.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.128+0000 7f6074b93700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60700fdfd0 0x7f607010de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:30.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.128+0000 7f6074b93700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607010e450 con 0x7f60700fdfd0 2024-05-06T12:11:30.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.128+0000 7f6074b93700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607010e5c0 con 0x7f60700fce60 2024-05-06T12:11:30.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.128+0000 7f606dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60700fdfd0 0x7f607010de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:30.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.128+0000 7f606dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60700fdfd0 0x7f607010de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49924/0 (socket says 172.21.15.92:49924) 2024-05-06T12:11:30.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.128+0000 7f606dd9b700 1 -- 172.21.15.92:0/373574216 learned_addr learned my addr 172.21.15.92:0/373574216 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:11:30.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.129+0000 7f606e59c700 1 --2- 172.21.15.92:0/373574216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60700fce60 0x7f607010d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:30.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.129+0000 7f606dd9b700 1 -- 172.21.15.92:0/373574216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60700fce60 msgr2=0x7f607010d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:30.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.129+0000 7f606dd9b700 1 --2- 172.21.15.92:0/373574216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60700fce60 0x7f607010d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:30.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.129+0000 7f606dd9b700 1 -- 172.21.15.92:0/373574216 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6058005040 con 0x7f60700fdfd0 2024-05-06T12:11:30.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.129+0000 7f606dd9b700 1 --2- 172.21.15.92:0/373574216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60700fdfd0 0x7f607010de50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f606000c830 tx=0x7f6060004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:30.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.129+0000 7f6066ffd700 1 -- 172.21.15.92:0/373574216 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60600106f0 con 0x7f60700fdfd0 2024-05-06T12:11:30.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.129+0000 7f6066ffd700 1 -- 172.21.15.92:0/373574216 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6060010d20 con 0x7f60700fdfd0 2024-05-06T12:11:30.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.129+0000 7f6074b93700 1 -- 172.21.15.92:0/373574216 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6070113020 con 0x7f60700fdfd0 2024-05-06T12:11:30.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.129+0000 7f6066ffd700 1 -- 172.21.15.92:0/373574216 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60600113a0 con 0x7f60700fdfd0 2024-05-06T12:11:30.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.129+0000 7f6074b93700 1 -- 172.21.15.92:0/373574216 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60701135b0 con 0x7f60700fdfd0 2024-05-06T12:11:30.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.130+0000 7f6074b93700 1 -- 172.21.15.92:0/373574216 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f607004c710 con 0x7f60700fdfd0 2024-05-06T12:11:30.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.131+0000 7f6066ffd700 1 -- 172.21.15.92:0/373574216 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f6060012400 con 0x7f60700fdfd0 2024-05-06T12:11:30.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.131+0000 7f6066ffd700 1 --2- 172.21.15.92:0/373574216 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f605c0692a0 0x7f605c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:30.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.131+0000 7f6066ffd700 1 -- 172.21.15.92:0/373574216 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6060086d30 con 0x7f60700fdfd0 2024-05-06T12:11:30.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.131+0000 7f606e59c700 1 --2- 172.21.15.92:0/373574216 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f605c0692a0 0x7f605c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:30.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.131+0000 7f606e59c700 1 --2- 172.21.15.92:0/373574216 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f605c0692a0 0x7f605c06b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f605800b970 tx=0x7f6058009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:30.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.135+0000 7f6066ffd700 1 -- 172.21.15.92:0/373574216 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6060056ed0 con 0x7f60700fdfd0 2024-05-06T12:11:30.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.340+0000 7f6074b93700 1 -- 172.21.15.92:0/373574216 --> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6070102360 con 0x7f605c0692a0 2024-05-06T12:11:30.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.341+0000 7f6066ffd700 1 -- 172.21.15.92:0/373574216 <== mgr.14210 v2:172.21.15.92:6800/3045905943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f6070102360 con 0x7f605c0692a0 2024-05-06T12:11:30.345 INFO:teuthology.orchestra.run.smithi092.stdout:Backend: cephadm 2024-05-06T12:11:30.345 INFO:teuthology.orchestra.run.smithi092.stdout:Available: Yes 2024-05-06T12:11:30.345 INFO:teuthology.orchestra.run.smithi092.stdout:Paused: No 2024-05-06T12:11:30.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.343+0000 7f6074b93700 1 -- 172.21.15.92:0/373574216 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f605c0692a0 msgr2=0x7f605c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:30.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.343+0000 7f6074b93700 1 --2- 172.21.15.92:0/373574216 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f605c0692a0 0x7f605c06b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f605800b970 tx=0x7f6058009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:30.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.343+0000 7f6074b93700 1 -- 172.21.15.92:0/373574216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60700fdfd0 msgr2=0x7f607010de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:30.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.343+0000 7f6074b93700 1 --2- 172.21.15.92:0/373574216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60700fdfd0 0x7f607010de50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f606000c830 tx=0x7f6060004470 comp rx=0 tx=0).stop 2024-05-06T12:11:30.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.343+0000 7f6074b93700 1 -- 172.21.15.92:0/373574216 shutdown_connections 2024-05-06T12:11:30.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.343+0000 7f6074b93700 1 --2- 172.21.15.92:0/373574216 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f605c0692a0 0x7f605c06b750 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:30.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.343+0000 7f6074b93700 1 --2- 172.21.15.92:0/373574216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60700fce60 0x7f607010d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:30.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.344+0000 7f6074b93700 1 --2- 172.21.15.92:0/373574216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60700fdfd0 0x7f607010de50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:30.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.344+0000 7f6074b93700 1 -- 172.21.15.92:0/373574216 >> 172.21.15.92:0/373574216 conn(0x7f60700f8540 msgr2=0x7f60700f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:30.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.344+0000 7f6074b93700 1 -- 172.21.15.92:0/373574216 shutdown_connections 2024-05-06T12:11:30.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:30.344+0000 7f6074b93700 1 -- 172.21.15.92:0/373574216 wait complete. 2024-05-06T12:11:30.945 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-06T12:11:31.204 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:11:31.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:30 smithi092 ceph-mon[26954]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:31.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:30 smithi092 ceph-mon[26954]: from='client.14428 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:11:31.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:30 smithi190 ceph-mon[33798]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:31.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:30 smithi190 ceph-mon[33798]: from='client.14428 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:11:32.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.772+0000 7f0f5bd74700 1 -- 172.21.15.92:0/1136540733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f540fce60 msgr2=0x7f0f540fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:32.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.772+0000 7f0f5bd74700 1 --2- 172.21.15.92:0/1136540733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f540fce60 0x7f0f540fd270 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0f50004940 tx=0x7f0f50009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:32.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.773+0000 7f0f5bd74700 1 -- 172.21.15.92:0/1136540733 shutdown_connections 2024-05-06T12:11:32.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.773+0000 7f0f5bd74700 1 --2- 172.21.15.92:0/1136540733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f540fdfd0 0x7f0f540fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:32.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.773+0000 7f0f5bd74700 1 --2- 172.21.15.92:0/1136540733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f540fce60 0x7f0f540fd270 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:32.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.773+0000 7f0f5bd74700 1 -- 172.21.15.92:0/1136540733 >> 172.21.15.92:0/1136540733 conn(0x7f0f540f8520 msgr2=0x7f0f540fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:32.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.773+0000 7f0f5bd74700 1 -- 172.21.15.92:0/1136540733 shutdown_connections 2024-05-06T12:11:32.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.773+0000 7f0f5bd74700 1 -- 172.21.15.92:0/1136540733 wait complete. 2024-05-06T12:11:32.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.773+0000 7f0f5bd74700 1 Processor -- start 2024-05-06T12:11:32.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.773+0000 7f0f5bd74700 1 -- start start 2024-05-06T12:11:32.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.773+0000 7f0f5bd74700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f540fce60 0x7f0f5410d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:32.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.773+0000 7f0f5bd74700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f540fdfd0 0x7f0f5410dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:32.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.773+0000 7f0f5bd74700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5410e530 con 0x7f0f540fce60 2024-05-06T12:11:32.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.773+0000 7f0f5bd74700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f54112e60 con 0x7f0f540fdfd0 2024-05-06T12:11:32.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.773+0000 7f0f59b10700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f540fce60 0x7f0f5410d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:32.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.773+0000 7f0f59b10700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f540fce60 0x7f0f5410d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59174/0 (socket says 172.21.15.92:59174) 2024-05-06T12:11:32.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.773+0000 7f0f59b10700 1 -- 172.21.15.92:0/2507766389 learned_addr learned my addr 172.21.15.92:0/2507766389 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:11:32.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.774+0000 7f0f5930f700 1 --2- 172.21.15.92:0/2507766389 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f540fdfd0 0x7f0f5410dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:32.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.774+0000 7f0f59b10700 1 -- 172.21.15.92:0/2507766389 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f540fdfd0 msgr2=0x7f0f5410dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:32.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.774+0000 7f0f59b10700 1 --2- 172.21.15.92:0/2507766389 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f540fdfd0 0x7f0f5410dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:32.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.774+0000 7f0f59b10700 1 -- 172.21.15.92:0/2507766389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f50005040 con 0x7f0f540fce60 2024-05-06T12:11:32.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.774+0000 7f0f59b10700 1 --2- 172.21.15.92:0/2507766389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f540fce60 0x7f0f5410d960 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0f5000b3b0 tx=0x7f0f50009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:32.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.774+0000 7f0f4a7fc700 1 -- 172.21.15.92:0/2507766389 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f5000e040 con 0x7f0f540fce60 2024-05-06T12:11:32.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.774+0000 7f0f5bd74700 1 -- 172.21.15.92:0/2507766389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f54113060 con 0x7f0f540fce60 2024-05-06T12:11:32.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.774+0000 7f0f4a7fc700 1 -- 172.21.15.92:0/2507766389 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f5000bcc0 con 0x7f0f540fce60 2024-05-06T12:11:32.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.774+0000 7f0f4a7fc700 1 -- 172.21.15.92:0/2507766389 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f5001c630 con 0x7f0f540fce60 2024-05-06T12:11:32.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.774+0000 7f0f5bd74700 1 -- 172.21.15.92:0/2507766389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f541135d0 con 0x7f0f540fce60 2024-05-06T12:11:32.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.775+0000 7f0f5bd74700 1 -- 172.21.15.92:0/2507766389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f5404c710 con 0x7f0f540fce60 2024-05-06T12:11:32.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.776+0000 7f0f4a7fc700 1 -- 172.21.15.92:0/2507766389 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f0f50013980 con 0x7f0f540fce60 2024-05-06T12:11:32.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.776+0000 7f0f4a7fc700 1 --2- 172.21.15.92:0/2507766389 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f0f400692a0 0x7f0f4006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:32.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.776+0000 7f0f5930f700 1 --2- 172.21.15.92:0/2507766389 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f0f400692a0 0x7f0f4006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:32.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.776+0000 7f0f4a7fc700 1 -- 172.21.15.92:0/2507766389 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0f50087c90 con 0x7f0f540fce60 2024-05-06T12:11:32.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.777+0000 7f0f5930f700 1 --2- 172.21.15.92:0/2507766389 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f0f400692a0 0x7f0f4006b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0f44006fd0 tx=0x7f0f44008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:32.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.781+0000 7f0f4a7fc700 1 -- 172.21.15.92:0/2507766389 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0f50057d10 con 0x7f0f540fce60 2024-05-06T12:11:32.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.976+0000 7f0f5bd74700 1 -- 172.21.15.92:0/2507766389 --> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f541022a0 con 0x7f0f400692a0 2024-05-06T12:11:32.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.984+0000 7f0f4a7fc700 1 -- 172.21.15.92:0/2507766389 <== mgr.14210 v2:172.21.15.92:6800/3045905943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2682 (secure 0 0 0) 0x7f0f541022a0 con 0x7f0f400692a0 2024-05-06T12:11:32.987 INFO:teuthology.orchestra.run.smithi092.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-06T12:11:32.987 INFO:teuthology.orchestra.run.smithi092.stdout:alertmanager.smithi092 smithi092 *:9093,9094 running (3m) 2m ago 5m 26.8M - 0.25.0 c8568f914cd2 dab380a7352e 2024-05-06T12:11:32.988 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi092 smithi092 running (5m) 2m ago 5m 6660k - 17.2.7-925-ge81df420 96d5ae7fd4b4 95dc7e14bd9a 2024-05-06T12:11:32.988 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi190 smithi190 running (4m) 68s ago 4m 6665k - 17.2.7-925-ge81df420 96d5ae7fd4b4 6cf97785406f 2024-05-06T12:11:32.988 INFO:teuthology.orchestra.run.smithi092.stdout:grafana.smithi092 smithi092 *:3000 running (3m) 2m ago 4m 78.2M - 9.4.7 954c08fa6188 841f34687c12 2024-05-06T12:11:32.988 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi092.gyzeqb smithi092 *:9283 running (6m) 2m ago 6m 472M - 17.2.7-925-ge81df420 96d5ae7fd4b4 820ba9ba8a82 2024-05-06T12:11:32.988 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi190.kczxkh smithi190 *:8443,9283 running (4m) 68s ago 4m 431M - 17.2.7-925-ge81df420 96d5ae7fd4b4 fb57adc25b2b 2024-05-06T12:11:32.988 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi092 smithi092 running (6m) 2m ago 6m 42.1M 2048M 17.2.7-925-ge81df420 96d5ae7fd4b4 df5b322e9719 2024-05-06T12:11:32.988 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi190 smithi190 running (4m) 68s ago 4m 38.3M 2048M 17.2.7-925-ge81df420 96d5ae7fd4b4 912ded072601 2024-05-06T12:11:32.988 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi092 smithi092 *:9100 running (5m) 2m ago 5m 16.1M - 1.5.0 0da6a335fe13 86cf73a245ab 2024-05-06T12:11:32.988 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi190 smithi190 *:9100 running (4m) 68s ago 4m 16.0M - 1.5.0 0da6a335fe13 da5c2a030c69 2024-05-06T12:11:32.988 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 smithi092 running (3m) 2m ago 3m 38.5M 4288M 17.2.7-925-ge81df420 96d5ae7fd4b4 ce84e7af9084 2024-05-06T12:11:32.988 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 smithi092 running (2m) 2m ago 2m 41.5M 4288M 17.2.7-925-ge81df420 96d5ae7fd4b4 8ab870f57eb2 2024-05-06T12:11:32.988 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 smithi092 running (2m) 2m ago 2m 40.6M 4288M 17.2.7-925-ge81df420 96d5ae7fd4b4 1e6ed04a0688 2024-05-06T12:11:32.988 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 smithi190 running (2m) 68s ago 2m 38.0M 5312M 17.2.7-925-ge81df420 96d5ae7fd4b4 52f9521286d5 2024-05-06T12:11:32.988 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 smithi190 running (98s) 68s ago 98s 36.7M 5312M 17.2.7-925-ge81df420 96d5ae7fd4b4 3a7da09e4c48 2024-05-06T12:11:32.988 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 smithi190 running (73s) 68s ago 72s 10.5M 5312M 17.2.7-925-ge81df420 96d5ae7fd4b4 e7452c6b5cec 2024-05-06T12:11:32.989 INFO:teuthology.orchestra.run.smithi092.stdout:prometheus.smithi092 smithi092 *:9095 running (3m) 2m ago 4m 38.1M - 2.43.0 a07b618ecd1d ef92f3572328 2024-05-06T12:11:32.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.986+0000 7f0f5bd74700 1 -- 172.21.15.92:0/2507766389 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f0f400692a0 msgr2=0x7f0f4006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:32.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.986+0000 7f0f5bd74700 1 --2- 172.21.15.92:0/2507766389 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f0f400692a0 0x7f0f4006b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0f44006fd0 tx=0x7f0f44008040 comp rx=0 tx=0).stop 2024-05-06T12:11:32.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.986+0000 7f0f5bd74700 1 -- 172.21.15.92:0/2507766389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f540fce60 msgr2=0x7f0f5410d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:32.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.986+0000 7f0f5bd74700 1 --2- 172.21.15.92:0/2507766389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f540fce60 0x7f0f5410d960 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0f5000b3b0 tx=0x7f0f50009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:32.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.986+0000 7f0f5bd74700 1 -- 172.21.15.92:0/2507766389 shutdown_connections 2024-05-06T12:11:32.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.986+0000 7f0f5bd74700 1 --2- 172.21.15.92:0/2507766389 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f0f400692a0 0x7f0f4006b750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:32.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.986+0000 7f0f5bd74700 1 --2- 172.21.15.92:0/2507766389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f540fce60 0x7f0f5410d960 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:32.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.986+0000 7f0f5bd74700 1 --2- 172.21.15.92:0/2507766389 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f540fdfd0 0x7f0f5410dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:32.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.986+0000 7f0f5bd74700 1 -- 172.21.15.92:0/2507766389 >> 172.21.15.92:0/2507766389 conn(0x7f0f540f8520 msgr2=0x7f0f540f9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:32.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.987+0000 7f0f5bd74700 1 -- 172.21.15.92:0/2507766389 shutdown_connections 2024-05-06T12:11:32.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:32.987+0000 7f0f5bd74700 1 -- 172.21.15.92:0/2507766389 wait complete. 2024-05-06T12:11:33.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:32 smithi092 ceph-mon[26954]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:33.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:32 smithi190 ceph-mon[33798]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:33.608 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-06T12:11:33.860 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:11:34.177 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:33 smithi092 ceph-mon[26954]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:11:34.177 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:33 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:11:34.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:33 smithi190 ceph-mon[33798]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:11:34.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:33 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:11:35.056 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:34 smithi092 ceph-mon[26954]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:35.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.245+0000 7f1215795700 1 -- 172.21.15.92:0/2618493370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12100fdfd0 msgr2=0x7f12100fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:35.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.245+0000 7f1215795700 1 --2- 172.21.15.92:0/2618493370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12100fdfd0 0x7f12100fe440 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1204009ce0 tx=0x7f1204004d10 comp rx=0 tx=0).stop 2024-05-06T12:11:35.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.245+0000 7f1215795700 1 -- 172.21.15.92:0/2618493370 shutdown_connections 2024-05-06T12:11:35.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.245+0000 7f1215795700 1 --2- 172.21.15.92:0/2618493370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12100fdfd0 0x7f12100fe440 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:35.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.245+0000 7f1215795700 1 --2- 172.21.15.92:0/2618493370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12100fce60 0x7f12100fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:35.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.245+0000 7f1215795700 1 -- 172.21.15.92:0/2618493370 >> 172.21.15.92:0/2618493370 conn(0x7f12100f8540 msgr2=0x7f12100fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:35.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.245+0000 7f1215795700 1 -- 172.21.15.92:0/2618493370 shutdown_connections 2024-05-06T12:11:35.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.245+0000 7f1215795700 1 -- 172.21.15.92:0/2618493370 wait complete. 2024-05-06T12:11:35.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.246+0000 7f1215795700 1 Processor -- start 2024-05-06T12:11:35.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.246+0000 7f1215795700 1 -- start start 2024-05-06T12:11:35.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.246+0000 7f1215795700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12100fce60 0x7f121010dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:35.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.246+0000 7f1215795700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12100fdfd0 0x7f121010e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:35.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.246+0000 7f1215795700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121010e620 con 0x7f12100fce60 2024-05-06T12:11:35.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.246+0000 7f1215795700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121010e790 con 0x7f12100fdfd0 2024-05-06T12:11:35.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.246+0000 7f120effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12100fce60 0x7f121010dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:35.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.246+0000 7f120effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12100fce60 0x7f121010dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59190/0 (socket says 172.21.15.92:59190) 2024-05-06T12:11:35.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.246+0000 7f120effd700 1 -- 172.21.15.92:0/2609560074 learned_addr learned my addr 172.21.15.92:0/2609560074 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:11:35.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.246+0000 7f120effd700 1 -- 172.21.15.92:0/2609560074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12100fdfd0 msgr2=0x7f121010e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:35.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.246+0000 7f120effd700 1 --2- 172.21.15.92:0/2609560074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12100fdfd0 0x7f121010e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:35.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.246+0000 7f120effd700 1 -- 172.21.15.92:0/2609560074 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1204009940 con 0x7f12100fce60 2024-05-06T12:11:35.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.246+0000 7f120effd700 1 --2- 172.21.15.92:0/2609560074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12100fce60 0x7f121010dae0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f120800c830 tx=0x7f1208004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:35.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.246+0000 7f11f77fe700 1 -- 172.21.15.92:0/2609560074 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f120800a760 con 0x7f12100fce60 2024-05-06T12:11:35.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.247+0000 7f1215795700 1 -- 172.21.15.92:0/2609560074 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12101131f0 con 0x7f12100fce60 2024-05-06T12:11:35.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.247+0000 7f11f77fe700 1 -- 172.21.15.92:0/2609560074 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f120800ad90 con 0x7f12100fce60 2024-05-06T12:11:35.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.247+0000 7f11f77fe700 1 -- 172.21.15.92:0/2609560074 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12080113a0 con 0x7f12100fce60 2024-05-06T12:11:35.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.247+0000 7f1215795700 1 -- 172.21.15.92:0/2609560074 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12101137b0 con 0x7f12100fce60 2024-05-06T12:11:35.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.248+0000 7f1215795700 1 -- 172.21.15.92:0/2609560074 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f121004c710 con 0x7f12100fce60 2024-05-06T12:11:35.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.248+0000 7f11f77fe700 1 -- 172.21.15.92:0/2609560074 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f1208012400 con 0x7f12100fce60 2024-05-06T12:11:35.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.248+0000 7f11f77fe700 1 --2- 172.21.15.92:0/2609560074 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f11f80692a0 0x7f11f806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:35.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.248+0000 7f11f77fe700 1 -- 172.21.15.92:0/2609560074 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1208086ee0 con 0x7f12100fce60 2024-05-06T12:11:35.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.248+0000 7f120e7fc700 1 --2- 172.21.15.92:0/2609560074 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f11f80692a0 0x7f11f806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:35.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.249+0000 7f120e7fc700 1 --2- 172.21.15.92:0/2609560074 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f11f80692a0 0x7f11f806b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1204000f80 tx=0x7f1204004d10 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:35.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.253+0000 7f11f77fe700 1 -- 172.21.15.92:0/2609560074 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1208056fe0 con 0x7f12100fce60 2024-05-06T12:11:35.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:34 smithi190 ceph-mon[33798]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:35.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.446+0000 7f1215795700 1 -- 172.21.15.92:0/2609560074 --> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f12101023b0 con 0x7f11f80692a0 2024-05-06T12:11:35.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.450+0000 7f11f77fe700 1 -- 172.21.15.92:0/2609560074 <== mgr.14210 v2:172.21.15.92:6800/3045905943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1170 (secure 0 0 0) 0x7f12101023b0 con 0x7f11f80692a0 2024-05-06T12:11:35.453 INFO:teuthology.orchestra.run.smithi092.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-06T12:11:35.454 INFO:teuthology.orchestra.run.smithi092.stdout:alertmanager ?:9093,9094 1/1 2m ago 5m count:1 2024-05-06T12:11:35.454 INFO:teuthology.orchestra.run.smithi092.stdout:crash 2/2 2m ago 5m * 2024-05-06T12:11:35.454 INFO:teuthology.orchestra.run.smithi092.stdout:grafana ?:3000 1/1 2m ago 5m count:1 2024-05-06T12:11:35.454 INFO:teuthology.orchestra.run.smithi092.stdout:mgr 2/2 2m ago 5m count:2 2024-05-06T12:11:35.454 INFO:teuthology.orchestra.run.smithi092.stdout:mon 2/2 2m ago 5m smithi092:172.21.15.92=smithi092;smithi190:172.21.15.190=smithi190;count:2 2024-05-06T12:11:35.454 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter ?:9100 2/2 2m ago 5m * 2024-05-06T12:11:35.454 INFO:teuthology.orchestra.run.smithi092.stdout:osd 6 2m ago - 2024-05-06T12:11:35.454 INFO:teuthology.orchestra.run.smithi092.stdout:prometheus ?:9095 1/1 2m ago 5m count:1 2024-05-06T12:11:35.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.452+0000 7f1215795700 1 -- 172.21.15.92:0/2609560074 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f11f80692a0 msgr2=0x7f11f806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:35.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.452+0000 7f1215795700 1 --2- 172.21.15.92:0/2609560074 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f11f80692a0 0x7f11f806b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1204000f80 tx=0x7f1204004d10 comp rx=0 tx=0).stop 2024-05-06T12:11:35.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.452+0000 7f1215795700 1 -- 172.21.15.92:0/2609560074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12100fce60 msgr2=0x7f121010dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:35.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.452+0000 7f1215795700 1 --2- 172.21.15.92:0/2609560074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12100fce60 0x7f121010dae0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f120800c830 tx=0x7f1208004470 comp rx=0 tx=0).stop 2024-05-06T12:11:35.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.452+0000 7f1215795700 1 -- 172.21.15.92:0/2609560074 shutdown_connections 2024-05-06T12:11:35.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.452+0000 7f1215795700 1 --2- 172.21.15.92:0/2609560074 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f11f80692a0 0x7f11f806b750 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:35.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.452+0000 7f1215795700 1 --2- 172.21.15.92:0/2609560074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12100fce60 0x7f121010dae0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:35.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.452+0000 7f1215795700 1 --2- 172.21.15.92:0/2609560074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12100fdfd0 0x7f121010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:35.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.452+0000 7f1215795700 1 -- 172.21.15.92:0/2609560074 >> 172.21.15.92:0/2609560074 conn(0x7f12100f8540 msgr2=0x7f12100f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:35.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.452+0000 7f1215795700 1 -- 172.21.15.92:0/2609560074 shutdown_connections 2024-05-06T12:11:35.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:35.452+0000 7f1215795700 1 -- 172.21.15.92:0/2609560074 wait complete. 2024-05-06T12:11:36.037 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:35 smithi092 ceph-mon[26954]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:11:36.064 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-06T12:11:36.312 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:11:36.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:35 smithi190 ceph-mon[33798]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:11:37.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:36 smithi092 ceph-mon[26954]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:37.249 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:36 smithi190 ceph-mon[33798]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:37.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.902+0000 7efecdc9d700 1 -- 172.21.15.92:0/4010001391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efec80fcf70 msgr2=0x7efec80fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:37.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.902+0000 7efecdc9d700 1 --2- 172.21.15.92:0/4010001391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efec80fcf70 0x7efec80fd3e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efeb0004940 tx=0x7efeb0009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:37.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.902+0000 7efecdc9d700 1 -- 172.21.15.92:0/4010001391 shutdown_connections 2024-05-06T12:11:37.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.902+0000 7efecdc9d700 1 --2- 172.21.15.92:0/4010001391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efec80fcf70 0x7efec80fd3e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:37.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.902+0000 7efecdc9d700 1 --2- 172.21.15.92:0/4010001391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efec80fbcc0 0x7efec80fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:37.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.902+0000 7efecdc9d700 1 -- 172.21.15.92:0/4010001391 >> 172.21.15.92:0/4010001391 conn(0x7efec80f7440 msgr2=0x7efec80f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:37.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.902+0000 7efecdc9d700 1 -- 172.21.15.92:0/4010001391 shutdown_connections 2024-05-06T12:11:37.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.902+0000 7efecdc9d700 1 -- 172.21.15.92:0/4010001391 wait complete. 2024-05-06T12:11:37.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.903+0000 7efecdc9d700 1 Processor -- start 2024-05-06T12:11:37.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.903+0000 7efecdc9d700 1 -- start start 2024-05-06T12:11:37.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.903+0000 7efecdc9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efec80fbcc0 0x7efec810c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:37.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.903+0000 7efecdc9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efec80fcf70 0x7efec810cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:37.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.903+0000 7efecdc9d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec810d3f0 con 0x7efec80fbcc0 2024-05-06T12:11:37.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.903+0000 7efecdc9d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec8111d90 con 0x7efec80fcf70 2024-05-06T12:11:37.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.903+0000 7efec77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efec80fbcc0 0x7efec810c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:37.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.903+0000 7efec77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efec80fbcc0 0x7efec810c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59208/0 (socket says 172.21.15.92:59208) 2024-05-06T12:11:37.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.903+0000 7efec77fe700 1 -- 172.21.15.92:0/237334001 learned_addr learned my addr 172.21.15.92:0/237334001 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:11:37.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.904+0000 7efebeffd700 1 --2- 172.21.15.92:0/237334001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efec80fcf70 0x7efec810cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:37.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.904+0000 7efec77fe700 1 -- 172.21.15.92:0/237334001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efec80fcf70 msgr2=0x7efec810cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:37.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.904+0000 7efec77fe700 1 --2- 172.21.15.92:0/237334001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efec80fcf70 0x7efec810cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:37.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.904+0000 7efec77fe700 1 -- 172.21.15.92:0/237334001 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb0005040 con 0x7efec80fbcc0 2024-05-06T12:11:37.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.904+0000 7efec77fe700 1 --2- 172.21.15.92:0/237334001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efec80fbcc0 0x7efec810c820 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efeb800b790 tx=0x7efeb8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:37.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.904+0000 7efec4ff9700 1 -- 172.21.15.92:0/237334001 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efeb800f770 con 0x7efec80fbcc0 2024-05-06T12:11:37.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.904+0000 7efecdc9d700 1 -- 172.21.15.92:0/237334001 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec8111ff0 con 0x7efec80fbcc0 2024-05-06T12:11:37.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.904+0000 7efec4ff9700 1 -- 172.21.15.92:0/237334001 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efeb800fda0 con 0x7efec80fbcc0 2024-05-06T12:11:37.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.904+0000 7efec4ff9700 1 -- 172.21.15.92:0/237334001 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efeb800d310 con 0x7efec80fbcc0 2024-05-06T12:11:37.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.904+0000 7efecdc9d700 1 -- 172.21.15.92:0/237334001 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec80665a0 con 0x7efec80fbcc0 2024-05-06T12:11:37.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.905+0000 7efecdc9d700 1 -- 172.21.15.92:0/237334001 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efec804c710 con 0x7efec80fbcc0 2024-05-06T12:11:37.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.906+0000 7efec4ff9700 1 -- 172.21.15.92:0/237334001 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7efeb8010420 con 0x7efec80fbcc0 2024-05-06T12:11:37.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.906+0000 7efec4ff9700 1 --2- 172.21.15.92:0/237334001 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7efeb406d6b0 0x7efeb406fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:37.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.906+0000 7efec4ff9700 1 -- 172.21.15.92:0/237334001 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7efeb8086e80 con 0x7efec80fbcc0 2024-05-06T12:11:37.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.906+0000 7efebeffd700 1 --2- 172.21.15.92:0/237334001 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7efeb406d6b0 0x7efeb406fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:37.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.906+0000 7efebeffd700 1 --2- 172.21.15.92:0/237334001 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7efeb406d6b0 0x7efeb406fb60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efeb0000c00 tx=0x7efeb0009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:37.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:37.910+0000 7efec4ff9700 1 -- 172.21.15.92:0/237334001 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7efeb8056f80 con 0x7efec80fbcc0 2024-05-06T12:11:38.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:38.105+0000 7efecdc9d700 1 -- 172.21.15.92:0/237334001 --> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7efec8066dc0 con 0x7efeb406d6b0 2024-05-06T12:11:38.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:38.107+0000 7efec4ff9700 1 -- 172.21.15.92:0/237334001 <== mgr.14210 v2:172.21.15.92:6800/3045905943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7efec8066dc0 con 0x7efeb406d6b0 2024-05-06T12:11:38.108 INFO:teuthology.orchestra.run.smithi092.stdout:HOST ADDR LABELS STATUS 2024-05-06T12:11:38.108 INFO:teuthology.orchestra.run.smithi092.stdout:smithi092 172.21.15.92 2024-05-06T12:11:38.109 INFO:teuthology.orchestra.run.smithi092.stdout:smithi190 172.21.15.190 2024-05-06T12:11:38.109 INFO:teuthology.orchestra.run.smithi092.stdout:2 hosts in cluster 2024-05-06T12:11:38.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:38.109+0000 7efecdc9d700 1 -- 172.21.15.92:0/237334001 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7efeb406d6b0 msgr2=0x7efeb406fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:38.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:38.109+0000 7efecdc9d700 1 --2- 172.21.15.92:0/237334001 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7efeb406d6b0 0x7efeb406fb60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efeb0000c00 tx=0x7efeb0009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:38.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:38.109+0000 7efecdc9d700 1 -- 172.21.15.92:0/237334001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efec80fbcc0 msgr2=0x7efec810c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:38.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:38.109+0000 7efecdc9d700 1 --2- 172.21.15.92:0/237334001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efec80fbcc0 0x7efec810c820 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efeb800b790 tx=0x7efeb8009250 comp rx=0 tx=0).stop 2024-05-06T12:11:38.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:38.109+0000 7efecdc9d700 1 -- 172.21.15.92:0/237334001 shutdown_connections 2024-05-06T12:11:38.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:38.109+0000 7efecdc9d700 1 --2- 172.21.15.92:0/237334001 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7efeb406d6b0 0x7efeb406fb60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:38.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:38.109+0000 7efecdc9d700 1 --2- 172.21.15.92:0/237334001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efec80fbcc0 0x7efec810c820 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:38.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:38.109+0000 7efecdc9d700 1 --2- 172.21.15.92:0/237334001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efec80fcf70 0x7efec810cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:38.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:38.109+0000 7efecdc9d700 1 -- 172.21.15.92:0/237334001 >> 172.21.15.92:0/237334001 conn(0x7efec80f7440 msgr2=0x7efec80f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:38.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:38.109+0000 7efecdc9d700 1 -- 172.21.15.92:0/237334001 shutdown_connections 2024-05-06T12:11:38.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:38.109+0000 7efecdc9d700 1 -- 172.21.15.92:0/237334001 wait complete. 2024-05-06T12:11:38.721 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-06T12:11:38.970 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:11:38.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:38 smithi092 ceph-mon[26954]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:38.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:38 smithi092 ceph-mon[26954]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:11:39.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:38 smithi190 ceph-mon[33798]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:39.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:38 smithi190 ceph-mon[33798]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:11:40.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.492+0000 7fdd520a8700 1 -- 172.21.15.92:0/1251711117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd4c066260 msgr2=0x7fdd4c0ffa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:40.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.492+0000 7fdd520a8700 1 --2- 172.21.15.92:0/1251711117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd4c066260 0x7fdd4c0ffa00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdd40004940 tx=0x7fdd40009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:40.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.492+0000 7fdd520a8700 1 -- 172.21.15.92:0/1251711117 shutdown_connections 2024-05-06T12:11:40.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.492+0000 7fdd520a8700 1 --2- 172.21.15.92:0/1251711117 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd4c0fff40 0x7fdd4c102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:40.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.492+0000 7fdd520a8700 1 --2- 172.21.15.92:0/1251711117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd4c066260 0x7fdd4c0ffa00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:40.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.492+0000 7fdd520a8700 1 -- 172.21.15.92:0/1251711117 >> 172.21.15.92:0/1251711117 conn(0x7fdd4c0f7430 msgr2=0x7fdd4c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:40.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.492+0000 7fdd520a8700 1 -- 172.21.15.92:0/1251711117 shutdown_connections 2024-05-06T12:11:40.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.492+0000 7fdd520a8700 1 -- 172.21.15.92:0/1251711117 wait complete. 2024-05-06T12:11:40.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.492+0000 7fdd520a8700 1 Processor -- start 2024-05-06T12:11:40.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.493+0000 7fdd520a8700 1 -- start start 2024-05-06T12:11:40.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.493+0000 7fdd520a8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd4c066260 0x7fdd4c10daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:40.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.493+0000 7fdd520a8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd4c0fff40 0x7fdd4c10dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:40.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.493+0000 7fdd520a8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c10e670 con 0x7fdd4c066260 2024-05-06T12:11:40.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.493+0000 7fdd520a8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c10ee10 con 0x7fdd4c0fff40 2024-05-06T12:11:40.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.493+0000 7fdd4b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd4c066260 0x7fdd4c10daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:40.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.493+0000 7fdd4b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd4c066260 0x7fdd4c10daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59238/0 (socket says 172.21.15.92:59238) 2024-05-06T12:11:40.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.493+0000 7fdd4b7fe700 1 -- 172.21.15.92:0/2887676790 learned_addr learned my addr 172.21.15.92:0/2887676790 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:11:40.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.493+0000 7fdd4affd700 1 --2- 172.21.15.92:0/2887676790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd4c0fff40 0x7fdd4c10dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:40.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.493+0000 7fdd4b7fe700 1 -- 172.21.15.92:0/2887676790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd4c0fff40 msgr2=0x7fdd4c10dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:40.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.493+0000 7fdd4b7fe700 1 --2- 172.21.15.92:0/2887676790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd4c0fff40 0x7fdd4c10dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:40.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.493+0000 7fdd4b7fe700 1 -- 172.21.15.92:0/2887676790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd40005040 con 0x7fdd4c066260 2024-05-06T12:11:40.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.493+0000 7fdd4affd700 1 --2- 172.21.15.92:0/2887676790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd4c0fff40 0x7fdd4c10dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-06T12:11:40.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.493+0000 7fdd4b7fe700 1 --2- 172.21.15.92:0/2887676790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd4c066260 0x7fdd4c10daa0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdd4000b3b0 tx=0x7fdd40009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:40.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.494+0000 7fdd33fff700 1 -- 172.21.15.92:0/2887676790 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd4000e040 con 0x7fdd4c066260 2024-05-06T12:11:40.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.494+0000 7fdd33fff700 1 -- 172.21.15.92:0/2887676790 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd4000bcc0 con 0x7fdd4c066260 2024-05-06T12:11:40.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.494+0000 7fdd520a8700 1 -- 172.21.15.92:0/2887676790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd4c10f090 con 0x7fdd4c066260 2024-05-06T12:11:40.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.494+0000 7fdd33fff700 1 -- 172.21.15.92:0/2887676790 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd4001c630 con 0x7fdd4c066260 2024-05-06T12:11:40.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.494+0000 7fdd520a8700 1 -- 172.21.15.92:0/2887676790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd4c10f650 con 0x7fdd4c066260 2024-05-06T12:11:40.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.495+0000 7fdd520a8700 1 -- 172.21.15.92:0/2887676790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd4c05f5c0 con 0x7fdd4c066260 2024-05-06T12:11:40.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.495+0000 7fdd33fff700 1 -- 172.21.15.92:0/2887676790 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fdd400139f0 con 0x7fdd4c066260 2024-05-06T12:11:40.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.495+0000 7fdd33fff700 1 --2- 172.21.15.92:0/2887676790 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fdd340692a0 0x7fdd3406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:40.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.495+0000 7fdd33fff700 1 -- 172.21.15.92:0/2887676790 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fdd40087b70 con 0x7fdd4c066260 2024-05-06T12:11:40.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.495+0000 7fdd4affd700 1 --2- 172.21.15.92:0/2887676790 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fdd340692a0 0x7fdd3406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:40.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.496+0000 7fdd4affd700 1 --2- 172.21.15.92:0/2887676790 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fdd340692a0 0x7fdd3406b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c009b10 tx=0x7fdd3c008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:40.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.500+0000 7fdd33fff700 1 -- 172.21.15.92:0/2887676790 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fdd40057d10 con 0x7fdd4c066260 2024-05-06T12:11:40.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.694+0000 7fdd520a8700 1 -- 172.21.15.92:0/2887676790 --> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd4c10f960 con 0x7fdd340692a0 2024-05-06T12:11:40.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.696+0000 7fdd33fff700 1 -- 172.21.15.92:0/2887676790 <== mgr.14210 v2:172.21.15.92:6800/3045905943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7fdd4c10f960 con 0x7fdd340692a0 2024-05-06T12:11:40.698 INFO:teuthology.orchestra.run.smithi092.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-06T12:11:40.698 INFO:teuthology.orchestra.run.smithi092.stdout:smithi092 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400QM400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-05-06T12:11:40.698 INFO:teuthology.orchestra.run.smithi092.stdout:smithi190 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT63210012400BGN 372G No 72s ago Has a FileSystem, LVM detected 2024-05-06T12:11:40.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.698+0000 7fdd520a8700 1 -- 172.21.15.92:0/2887676790 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fdd340692a0 msgr2=0x7fdd3406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:40.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.698+0000 7fdd520a8700 1 --2- 172.21.15.92:0/2887676790 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fdd340692a0 0x7fdd3406b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c009b10 tx=0x7fdd3c008040 comp rx=0 tx=0).stop 2024-05-06T12:11:40.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.698+0000 7fdd520a8700 1 -- 172.21.15.92:0/2887676790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd4c066260 msgr2=0x7fdd4c10daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:40.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.698+0000 7fdd520a8700 1 --2- 172.21.15.92:0/2887676790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd4c066260 0x7fdd4c10daa0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdd4000b3b0 tx=0x7fdd40009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:40.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.698+0000 7fdd520a8700 1 -- 172.21.15.92:0/2887676790 shutdown_connections 2024-05-06T12:11:40.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.698+0000 7fdd520a8700 1 --2- 172.21.15.92:0/2887676790 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fdd340692a0 0x7fdd3406b750 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:40.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.698+0000 7fdd520a8700 1 --2- 172.21.15.92:0/2887676790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd4c066260 0x7fdd4c10daa0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:40.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.698+0000 7fdd520a8700 1 --2- 172.21.15.92:0/2887676790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd4c0fff40 0x7fdd4c10dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:40.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.698+0000 7fdd520a8700 1 -- 172.21.15.92:0/2887676790 >> 172.21.15.92:0/2887676790 conn(0x7fdd4c0f7430 msgr2=0x7fdd4c0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:40.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.698+0000 7fdd520a8700 1 -- 172.21.15.92:0/2887676790 shutdown_connections 2024-05-06T12:11:40.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:40.698+0000 7fdd520a8700 1 -- 172.21.15.92:0/2887676790 wait complete. 2024-05-06T12:11:40.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:40 smithi092 ceph-mon[26954]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:41.346 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-06T12:11:41.356 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi092.front.sepia.ceph.com 2024-05-06T12:11:41.356 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-05-06T12:11:41.357 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:40 smithi190 ceph-mon[33798]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:41.624 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:11:42.026 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:41 smithi092 ceph-mon[26954]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:11:42.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:41 smithi190 ceph-mon[33798]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:11:42.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:42 smithi092 ceph-mon[26954]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:43.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.198+0000 7ffa1eda9700 1 -- 172.21.15.92:0/3402566894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa180fce80 msgr2=0x7ffa180fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:43.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.198+0000 7ffa1eda9700 1 --2- 172.21.15.92:0/3402566894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa180fce80 0x7ffa180fd290 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c004940 tx=0x7ffa0c009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:43.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.198+0000 7ffa1eda9700 1 -- 172.21.15.92:0/3402566894 shutdown_connections 2024-05-06T12:11:43.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.198+0000 7ffa1eda9700 1 --2- 172.21.15.92:0/3402566894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa180fe080 0x7ffa180fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:43.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.198+0000 7ffa1eda9700 1 --2- 172.21.15.92:0/3402566894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa180fce80 0x7ffa180fd290 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:43.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.198+0000 7ffa1eda9700 1 -- 172.21.15.92:0/3402566894 >> 172.21.15.92:0/3402566894 conn(0x7ffa180f8590 msgr2=0x7ffa180fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:43.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.198+0000 7ffa1eda9700 1 -- 172.21.15.92:0/3402566894 shutdown_connections 2024-05-06T12:11:43.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.198+0000 7ffa1eda9700 1 -- 172.21.15.92:0/3402566894 wait complete. 2024-05-06T12:11:43.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.199+0000 7ffa1eda9700 1 Processor -- start 2024-05-06T12:11:43.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.199+0000 7ffa1eda9700 1 -- start start 2024-05-06T12:11:43.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.199+0000 7ffa1eda9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa180fce80 0x7ffa18071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:43.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.199+0000 7ffa1eda9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa180fe080 0x7ffa180702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:43.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.199+0000 7ffa1eda9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa18070860 con 0x7ffa180fce80 2024-05-06T12:11:43.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.199+0000 7ffa1eda9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa180709d0 con 0x7ffa180fe080 2024-05-06T12:11:43.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.199+0000 7ffa1cb45700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa180fce80 0x7ffa18071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:43.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.199+0000 7ffa1cb45700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa180fce80 0x7ffa18071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46152/0 (socket says 172.21.15.92:46152) 2024-05-06T12:11:43.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.199+0000 7ffa1cb45700 1 -- 172.21.15.92:0/447969778 learned_addr learned my addr 172.21.15.92:0/447969778 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:11:43.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.199+0000 7ffa17fff700 1 --2- 172.21.15.92:0/447969778 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa180fe080 0x7ffa180702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:43.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.199+0000 7ffa1cb45700 1 -- 172.21.15.92:0/447969778 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa180fe080 msgr2=0x7ffa180702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:43.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.199+0000 7ffa1cb45700 1 --2- 172.21.15.92:0/447969778 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa180fe080 0x7ffa180702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:43.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.199+0000 7ffa1cb45700 1 -- 172.21.15.92:0/447969778 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa0c005040 con 0x7ffa180fce80 2024-05-06T12:11:43.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.199+0000 7ffa17fff700 1 --2- 172.21.15.92:0/447969778 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa180fe080 0x7ffa180702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-06T12:11:43.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.200+0000 7ffa1cb45700 1 --2- 172.21.15.92:0/447969778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa180fce80 0x7ffa18071de0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00b970 tx=0x7ffa0c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:43.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.200+0000 7ffa157fa700 1 -- 172.21.15.92:0/447969778 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa0c00e040 con 0x7ffa180fce80 2024-05-06T12:11:43.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.200+0000 7ffa1eda9700 1 -- 172.21.15.92:0/447969778 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa18070c50 con 0x7ffa180fce80 2024-05-06T12:11:43.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.200+0000 7ffa157fa700 1 -- 172.21.15.92:0/447969778 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa0c00bbf0 con 0x7ffa180fce80 2024-05-06T12:11:43.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.200+0000 7ffa1eda9700 1 -- 172.21.15.92:0/447969778 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa18115a60 con 0x7ffa180fce80 2024-05-06T12:11:43.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.200+0000 7ffa157fa700 1 -- 172.21.15.92:0/447969778 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa0c01c630 con 0x7ffa180fce80 2024-05-06T12:11:43.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.201+0000 7ffa1eda9700 1 -- 172.21.15.92:0/447969778 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa1805f5c0 con 0x7ffa180fce80 2024-05-06T12:11:43.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.201+0000 7ffa157fa700 1 -- 172.21.15.92:0/447969778 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7ffa0c00bd60 con 0x7ffa180fce80 2024-05-06T12:11:43.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.202+0000 7ffa157fa700 1 --2- 172.21.15.92:0/447969778 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7ffa080692b0 0x7ffa0806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:43.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.202+0000 7ffa157fa700 1 -- 172.21.15.92:0/447969778 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ffa0c088220 con 0x7ffa180fce80 2024-05-06T12:11:43.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.202+0000 7ffa17fff700 1 --2- 172.21.15.92:0/447969778 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7ffa080692b0 0x7ffa0806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:43.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.202+0000 7ffa17fff700 1 --2- 172.21.15.92:0/447969778 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7ffa080692b0 0x7ffa0806b760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffa040095c0 tx=0x7ffa04007f20 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:43.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.206+0000 7ffa157fa700 1 -- 172.21.15.92:0/447969778 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ffa0c054940 con 0x7ffa180fce80 2024-05-06T12:11:43.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:42 smithi190 ceph-mon[33798]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:43.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:43.418+0000 7ffa1eda9700 1 -- 172.21.15.92:0/447969778 --> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa18102440 con 0x7ffa080692b0 2024-05-06T12:11:44.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:43 smithi092 ceph-mon[26954]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:11:44.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:43 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-05-06T12:11:44.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:43 smithi190 ceph-mon[33798]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:11:44.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:43 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-05-06T12:11:44.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:44.970+0000 7ffa157fa700 1 -- 172.21.15.92:0/447969778 <== mgr.14210 v2:172.21.15.92:6800/3045905943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ffa18102440 con 0x7ffa080692b0 2024-05-06T12:11:44.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:44.973+0000 7ffa1eda9700 1 -- 172.21.15.92:0/447969778 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7ffa080692b0 msgr2=0x7ffa0806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:44.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:44.973+0000 7ffa1eda9700 1 --2- 172.21.15.92:0/447969778 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7ffa080692b0 0x7ffa0806b760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffa040095c0 tx=0x7ffa04007f20 comp rx=0 tx=0).stop 2024-05-06T12:11:44.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:44.973+0000 7ffa1eda9700 1 -- 172.21.15.92:0/447969778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa180fce80 msgr2=0x7ffa18071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:44.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:44.973+0000 7ffa1eda9700 1 --2- 172.21.15.92:0/447969778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa180fce80 0x7ffa18071de0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00b970 tx=0x7ffa0c009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:44.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:44.974+0000 7ffa1eda9700 1 -- 172.21.15.92:0/447969778 shutdown_connections 2024-05-06T12:11:44.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:44.974+0000 7ffa1eda9700 1 --2- 172.21.15.92:0/447969778 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7ffa080692b0 0x7ffa0806b760 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:44.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:44.974+0000 7ffa1eda9700 1 --2- 172.21.15.92:0/447969778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa180fce80 0x7ffa18071de0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:44.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:44.974+0000 7ffa1eda9700 1 --2- 172.21.15.92:0/447969778 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa180fe080 0x7ffa180702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:44.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:44.974+0000 7ffa1eda9700 1 -- 172.21.15.92:0/447969778 >> 172.21.15.92:0/447969778 conn(0x7ffa180f8590 msgr2=0x7ffa180f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:44.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:44.974+0000 7ffa1eda9700 1 -- 172.21.15.92:0/447969778 shutdown_connections 2024-05-06T12:11:44.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:44.974+0000 7ffa1eda9700 1 -- 172.21.15.92:0/447969778 wait complete. 2024-05-06T12:11:45.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:44 smithi092 ceph-mon[26954]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:45.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:44 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-05-06T12:11:45.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:44 smithi092 ceph-mon[26954]: osdmap e41: 6 total, 6 up, 6 in 2024-05-06T12:11:45.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:44 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-05-06T12:11:45.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:44 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi092[26950]: 2024-05-06T12:11:44.951+0000 7fae4fb69700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-06T12:11:45.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:44 smithi190 ceph-mon[33798]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:45.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:44 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-05-06T12:11:45.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:44 smithi190 ceph-mon[33798]: osdmap e41: 6 total, 6 up, 6 in 2024-05-06T12:11:45.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:44 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-05-06T12:11:45.669 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-06T12:11:46.214 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:45 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-05-06T12:11:46.214 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:45 smithi092 ceph-mon[26954]: osdmap e42: 6 total, 6 up, 6 in 2024-05-06T12:11:46.214 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:45 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-05-06T12:11:46.215 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:45 smithi092 ceph-mon[26954]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-06T12:11:46.215 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:45 smithi092 ceph-mon[26954]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-06T12:11:46.215 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:45 smithi092 ceph-mon[26954]: osdmap e43: 6 total, 6 up, 6 in 2024-05-06T12:11:46.215 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:45 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-05-06T12:11:46.215 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:45 smithi092 ceph-mon[26954]: fsmap cephfs:0 2024-05-06T12:11:46.215 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:45 smithi092 ceph-mon[26954]: Saving service mds.cephfs spec with placement count:4 2024-05-06T12:11:46.215 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:45 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:11:46.215 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:45 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:11:46.215 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:45 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:11:46.215 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:45 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:11:46.215 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:45 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:11:46.215 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:45 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi092.ijsqgp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-06T12:11:46.215 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:45 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi092.ijsqgp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-06T12:11:46.215 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:45 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:11:46.215 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:45 smithi092 ceph-mon[26954]: Deploying daemon mds.cephfs.smithi092.ijsqgp on smithi092 2024-05-06T12:11:46.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:45 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-05-06T12:11:46.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:45 smithi190 ceph-mon[33798]: osdmap e42: 6 total, 6 up, 6 in 2024-05-06T12:11:46.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:45 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-05-06T12:11:46.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:45 smithi190 ceph-mon[33798]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-06T12:11:46.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:45 smithi190 ceph-mon[33798]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-06T12:11:46.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:45 smithi190 ceph-mon[33798]: osdmap e43: 6 total, 6 up, 6 in 2024-05-06T12:11:46.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:45 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-05-06T12:11:46.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:45 smithi190 ceph-mon[33798]: fsmap cephfs:0 2024-05-06T12:11:46.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:45 smithi190 ceph-mon[33798]: Saving service mds.cephfs spec with placement count:4 2024-05-06T12:11:46.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:45 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:11:46.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:45 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:11:46.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:45 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:11:46.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:45 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:11:46.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:45 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:11:46.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:45 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi092.ijsqgp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-06T12:11:46.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:45 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi092.ijsqgp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-06T12:11:46.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:45 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:11:46.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:45 smithi190 ceph-mon[33798]: Deploying daemon mds.cephfs.smithi092.ijsqgp on smithi092 2024-05-06T12:11:46.972 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:11:47.095 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:46 smithi092 ceph-mon[26954]: pgmap v172: 65 pgs: 7 creating+peering, 1 active+clean, 57 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:47.096 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:46 smithi092 ceph-mon[26954]: osdmap e44: 6 total, 6 up, 6 in 2024-05-06T12:11:47.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:46 smithi190 ceph-mon[33798]: pgmap v172: 65 pgs: 7 creating+peering, 1 active+clean, 57 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:47.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:46 smithi190 ceph-mon[33798]: osdmap e44: 6 total, 6 up, 6 in 2024-05-06T12:11:48.335 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:48 smithi092 ceph-mon[26954]: osdmap e45: 6 total, 6 up, 6 in 2024-05-06T12:11:48.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:48 smithi190 ceph-mon[33798]: osdmap e45: 6 total, 6 up, 6 in 2024-05-06T12:11:48.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.751+0000 7f114f8f7700 1 -- 172.21.15.92:0/2078148235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11480fdfd0 msgr2=0x7f11480fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:48.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.751+0000 7f114f8f7700 1 --2- 172.21.15.92:0/2078148235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11480fdfd0 0x7f11480fe440 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1138004940 tx=0x7f1138009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:48.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.751+0000 7f114f8f7700 1 -- 172.21.15.92:0/2078148235 shutdown_connections 2024-05-06T12:11:48.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.751+0000 7f114f8f7700 1 --2- 172.21.15.92:0/2078148235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11480fdfd0 0x7f11480fe440 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:48.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.751+0000 7f114f8f7700 1 --2- 172.21.15.92:0/2078148235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11480fce60 0x7f11480fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:48.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.751+0000 7f114f8f7700 1 -- 172.21.15.92:0/2078148235 >> 172.21.15.92:0/2078148235 conn(0x7f11480f8540 msgr2=0x7f11480fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:48.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.751+0000 7f114f8f7700 1 -- 172.21.15.92:0/2078148235 shutdown_connections 2024-05-06T12:11:48.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.751+0000 7f114f8f7700 1 -- 172.21.15.92:0/2078148235 wait complete. 2024-05-06T12:11:48.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.751+0000 7f114f8f7700 1 Processor -- start 2024-05-06T12:11:48.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.751+0000 7f114f8f7700 1 -- start start 2024-05-06T12:11:48.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.751+0000 7f114f8f7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11480fce60 0x7f114810d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:48.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.751+0000 7f114f8f7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11480fdfd0 0x7f114810de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:48.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.752+0000 7f114f8f7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114810e4f0 con 0x7f11480fce60 2024-05-06T12:11:48.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.752+0000 7f114f8f7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1148112e90 con 0x7f11480fdfd0 2024-05-06T12:11:48.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.752+0000 7f114d693700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11480fce60 0x7f114810d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:48.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.752+0000 7f114d693700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11480fce60 0x7f114810d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46162/0 (socket says 172.21.15.92:46162) 2024-05-06T12:11:48.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.752+0000 7f114d693700 1 -- 172.21.15.92:0/2882212537 learned_addr learned my addr 172.21.15.92:0/2882212537 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:11:48.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.752+0000 7f114ce92700 1 --2- 172.21.15.92:0/2882212537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11480fdfd0 0x7f114810de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:48.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.752+0000 7f114d693700 1 -- 172.21.15.92:0/2882212537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11480fdfd0 msgr2=0x7f114810de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:48.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.752+0000 7f114d693700 1 --2- 172.21.15.92:0/2882212537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11480fdfd0 0x7f114810de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:48.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.752+0000 7f114d693700 1 -- 172.21.15.92:0/2882212537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1138005040 con 0x7f11480fce60 2024-05-06T12:11:48.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.752+0000 7f114d693700 1 --2- 172.21.15.92:0/2882212537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11480fce60 0x7f114810d920 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f114400b790 tx=0x7f1144009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:48.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.752+0000 7f113dffb700 1 -- 172.21.15.92:0/2882212537 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f114400f6f0 con 0x7f11480fce60 2024-05-06T12:11:48.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.752+0000 7f113dffb700 1 -- 172.21.15.92:0/2882212537 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f114400fd20 con 0x7f11480fce60 2024-05-06T12:11:48.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.752+0000 7f114f8f7700 1 -- 172.21.15.92:0/2882212537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11481130f0 con 0x7f11480fce60 2024-05-06T12:11:48.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.752+0000 7f113dffb700 1 -- 172.21.15.92:0/2882212537 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11440103a0 con 0x7f11480fce60 2024-05-06T12:11:48.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.753+0000 7f114f8f7700 1 -- 172.21.15.92:0/2882212537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1148113660 con 0x7f11480fce60 2024-05-06T12:11:48.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.753+0000 7f114f8f7700 1 -- 172.21.15.92:0/2882212537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f114804c710 con 0x7f11480fce60 2024-05-06T12:11:48.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.754+0000 7f113dffb700 1 -- 172.21.15.92:0/2882212537 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f1144011400 con 0x7f11480fce60 2024-05-06T12:11:48.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.754+0000 7f113dffb700 1 --2- 172.21.15.92:0/2882212537 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f11340692a0 0x7f113406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:48.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.754+0000 7f114ce92700 1 --2- 172.21.15.92:0/2882212537 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f11340692a0 0x7f113406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:48.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.754+0000 7f113dffb700 1 -- 172.21.15.92:0/2882212537 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f1144086ea0 con 0x7f11480fce60 2024-05-06T12:11:48.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.755+0000 7f114ce92700 1 --2- 172.21.15.92:0/2882212537 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f11340692a0 0x7f113406b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1138009fd0 tx=0x7f1138009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:48.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:48.759+0000 7f113dffb700 1 -- 172.21.15.92:0/2882212537 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1144056d00 con 0x7f11480fce60 2024-05-06T12:11:49.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:49.004+0000 7f114f8f7700 1 -- 172.21.15.92:0/2882212537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1148113b10 con 0x7f11480fce60 2024-05-06T12:11:49.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:49.004+0000 7f113dffb700 1 -- 172.21.15.92:0/2882212537 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1076 (secure 0 0 0) 0x7f1144056890 con 0x7f11480fce60 2024-05-06T12:11:49.007 INFO:teuthology.orchestra.run.smithi092.stdout:e2 2024-05-06T12:11:49.008 INFO:teuthology.orchestra.run.smithi092.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-06T12:11:49.008 INFO:teuthology.orchestra.run.smithi092.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:11:49.008 INFO:teuthology.orchestra.run.smithi092.stdout:legacy client fscid: 1 2024-05-06T12:11:49.008 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:11:49.008 INFO:teuthology.orchestra.run.smithi092.stdout:Filesystem 'cephfs' (1) 2024-05-06T12:11:49.008 INFO:teuthology.orchestra.run.smithi092.stdout:fs_name cephfs 2024-05-06T12:11:49.008 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 2 2024-05-06T12:11:49.008 INFO:teuthology.orchestra.run.smithi092.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-06T12:11:49.008 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-05-06T12:11:44.952671+0000 2024-05-06T12:11:49.008 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2024-05-06T12:11:44.952737+0000 2024-05-06T12:11:49.008 INFO:teuthology.orchestra.run.smithi092.stdout:tableserver 0 2024-05-06T12:11:49.008 INFO:teuthology.orchestra.run.smithi092.stdout:root 0 2024-05-06T12:11:49.008 INFO:teuthology.orchestra.run.smithi092.stdout:session_timeout 60 2024-05-06T12:11:49.009 INFO:teuthology.orchestra.run.smithi092.stdout:session_autoclose 300 2024-05-06T12:11:49.009 INFO:teuthology.orchestra.run.smithi092.stdout:max_file_size 1099511627776 2024-05-06T12:11:49.009 INFO:teuthology.orchestra.run.smithi092.stdout:required_client_features {} 2024-05-06T12:11:49.009 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure 0 2024-05-06T12:11:49.009 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure_osd_epoch 0 2024-05-06T12:11:49.009 INFO:teuthology.orchestra.run.smithi092.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:11:49.009 INFO:teuthology.orchestra.run.smithi092.stdout:max_mds 1 2024-05-06T12:11:49.009 INFO:teuthology.orchestra.run.smithi092.stdout:in 2024-05-06T12:11:49.009 INFO:teuthology.orchestra.run.smithi092.stdout:up {} 2024-05-06T12:11:49.009 INFO:teuthology.orchestra.run.smithi092.stdout:failed 2024-05-06T12:11:49.009 INFO:teuthology.orchestra.run.smithi092.stdout:damaged 2024-05-06T12:11:49.009 INFO:teuthology.orchestra.run.smithi092.stdout:stopped 2024-05-06T12:11:49.009 INFO:teuthology.orchestra.run.smithi092.stdout:data_pools [3] 2024-05-06T12:11:49.009 INFO:teuthology.orchestra.run.smithi092.stdout:metadata_pool 2 2024-05-06T12:11:49.009 INFO:teuthology.orchestra.run.smithi092.stdout:inline_data disabled 2024-05-06T12:11:49.010 INFO:teuthology.orchestra.run.smithi092.stdout:balancer 2024-05-06T12:11:49.010 INFO:teuthology.orchestra.run.smithi092.stdout:standby_count_wanted 0 2024-05-06T12:11:49.010 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:11:49.010 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:11:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:49.007+0000 7f114f8f7700 1 -- 172.21.15.92:0/2882212537 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f11340692a0 msgr2=0x7f113406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:49.007+0000 7f114f8f7700 1 --2- 172.21.15.92:0/2882212537 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f11340692a0 0x7f113406b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1138009fd0 tx=0x7f1138009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:49.007+0000 7f114f8f7700 1 -- 172.21.15.92:0/2882212537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11480fce60 msgr2=0x7f114810d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:49.007+0000 7f114f8f7700 1 --2- 172.21.15.92:0/2882212537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11480fce60 0x7f114810d920 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f114400b790 tx=0x7f1144009250 comp rx=0 tx=0).stop 2024-05-06T12:11:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:49.007+0000 7f114f8f7700 1 -- 172.21.15.92:0/2882212537 shutdown_connections 2024-05-06T12:11:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:49.007+0000 7f114f8f7700 1 --2- 172.21.15.92:0/2882212537 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f11340692a0 0x7f113406b750 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:49.007+0000 7f114f8f7700 1 --2- 172.21.15.92:0/2882212537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11480fce60 0x7f114810d920 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:49.007+0000 7f114f8f7700 1 --2- 172.21.15.92:0/2882212537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11480fdfd0 0x7f114810de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:49.007+0000 7f114f8f7700 1 -- 172.21.15.92:0/2882212537 >> 172.21.15.92:0/2882212537 conn(0x7f11480f8540 msgr2=0x7f11480f9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:49.007+0000 7f114f8f7700 1 -- 172.21.15.92:0/2882212537 shutdown_connections 2024-05-06T12:11:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:49.007+0000 7f114f8f7700 1 -- 172.21.15.92:0/2882212537 wait complete. 2024-05-06T12:11:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 2 2024-05-06T12:11:49.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:49 smithi190 ceph-mon[33798]: pgmap v175: 65 pgs: 38 active+clean, 13 creating+peering, 14 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:49.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:49 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:11:49.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:49 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:11:49.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:49 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/2882212537' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:11:49.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:49 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:11:49.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:49 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:11:49.325 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:49 smithi092 ceph-mon[26954]: pgmap v175: 65 pgs: 38 active+clean, 13 creating+peering, 14 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:49.325 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:49 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:11:49.325 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:49 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:11:49.325 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:49 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/2882212537' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:11:49.325 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:49 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:11:49.326 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:49 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:11:49.691 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-06T12:11:49.701 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi092.front.sepia.ceph.com 2024-05-06T12:11:49.701 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- bash -c 'ceph fs set cephfs max_mds 2' 2024-05-06T12:11:49.961 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:11:50.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:50 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:11:50.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:50 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi190.byjvgf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-06T12:11:50.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:50 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi190.byjvgf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-06T12:11:50.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:50 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:11:50.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:50 smithi190 ceph-mon[33798]: Deploying daemon mds.cephfs.smithi190.byjvgf on smithi190 2024-05-06T12:11:50.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:50 smithi190 ceph-mon[33798]: mds.? [v2:172.21.15.92:6826/4255634245,v1:172.21.15.92:6827/4255634245] up:boot 2024-05-06T12:11:50.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:50 smithi190 ceph-mon[33798]: daemon mds.cephfs.smithi092.ijsqgp assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-05-06T12:11:50.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:50 smithi190 ceph-mon[33798]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-06T12:11:50.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:50 smithi190 ceph-mon[33798]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-06T12:11:50.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:50 smithi190 ceph-mon[33798]: Cluster is now healthy 2024-05-06T12:11:50.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:50 smithi190 ceph-mon[33798]: fsmap cephfs:0 1 up:standby 2024-05-06T12:11:50.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:50 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi092.ijsqgp"}]: dispatch 2024-05-06T12:11:50.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:50 smithi190 ceph-mon[33798]: fsmap cephfs:1 {0=cephfs.smithi092.ijsqgp=up:creating} 2024-05-06T12:11:50.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:50 smithi190 ceph-mon[33798]: daemon mds.cephfs.smithi092.ijsqgp is now active in filesystem cephfs as rank 0 2024-05-06T12:11:50.421 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:50 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:11:50.421 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:50 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi190.byjvgf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-06T12:11:50.421 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:50 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi190.byjvgf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-06T12:11:50.421 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:50 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:11:50.421 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:50 smithi092 ceph-mon[26954]: Deploying daemon mds.cephfs.smithi190.byjvgf on smithi190 2024-05-06T12:11:50.421 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:50 smithi092 ceph-mon[26954]: mds.? [v2:172.21.15.92:6826/4255634245,v1:172.21.15.92:6827/4255634245] up:boot 2024-05-06T12:11:50.421 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:50 smithi092 ceph-mon[26954]: daemon mds.cephfs.smithi092.ijsqgp assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-05-06T12:11:50.421 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:50 smithi092 ceph-mon[26954]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-06T12:11:50.421 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:50 smithi092 ceph-mon[26954]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-06T12:11:50.422 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:50 smithi092 ceph-mon[26954]: Cluster is now healthy 2024-05-06T12:11:50.422 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:50 smithi092 ceph-mon[26954]: fsmap cephfs:0 1 up:standby 2024-05-06T12:11:50.422 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:50 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi092.ijsqgp"}]: dispatch 2024-05-06T12:11:50.422 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:50 smithi092 ceph-mon[26954]: fsmap cephfs:1 {0=cephfs.smithi092.ijsqgp=up:creating} 2024-05-06T12:11:50.422 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:50 smithi092 ceph-mon[26954]: daemon mds.cephfs.smithi092.ijsqgp is now active in filesystem cephfs as rank 0 2024-05-06T12:11:51.300 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:51 smithi092 ceph-mon[26954]: pgmap v176: 65 pgs: 59 active+clean, 6 creating+peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:51.300 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:51 smithi092 ceph-mon[26954]: mds.? [v2:172.21.15.92:6826/4255634245,v1:172.21.15.92:6827/4255634245] up:active 2024-05-06T12:11:51.300 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:51 smithi092 ceph-mon[26954]: fsmap cephfs:1 {0=cephfs.smithi092.ijsqgp=up:active} 2024-05-06T12:11:51.321 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:51 smithi190 ceph-mon[33798]: pgmap v176: 65 pgs: 59 active+clean, 6 creating+peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-06T12:11:51.321 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:51 smithi190 ceph-mon[33798]: mds.? [v2:172.21.15.92:6826/4255634245,v1:172.21.15.92:6827/4255634245] up:active 2024-05-06T12:11:51.321 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:51 smithi190 ceph-mon[33798]: fsmap cephfs:1 {0=cephfs.smithi092.ijsqgp=up:active} 2024-05-06T12:11:51.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.518+0000 7f923a3ee700 1 -- 172.21.15.92:0/1780677198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92340fbcc0 msgr2=0x7f92340fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:51.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.518+0000 7f923a3ee700 1 --2- 172.21.15.92:0/1780677198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92340fbcc0 0x7f92340fc0d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9224004940 tx=0x7f9224009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:51.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.518+0000 7f923a3ee700 1 -- 172.21.15.92:0/1780677198 shutdown_connections 2024-05-06T12:11:51.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.518+0000 7f923a3ee700 1 --2- 172.21.15.92:0/1780677198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92340fcf70 0x7f92340fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:51.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.518+0000 7f923a3ee700 1 --2- 172.21.15.92:0/1780677198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92340fbcc0 0x7f92340fc0d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:51.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.518+0000 7f923a3ee700 1 -- 172.21.15.92:0/1780677198 >> 172.21.15.92:0/1780677198 conn(0x7f92340f7440 msgr2=0x7f92340f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:51.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.518+0000 7f923a3ee700 1 -- 172.21.15.92:0/1780677198 shutdown_connections 2024-05-06T12:11:51.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.518+0000 7f923a3ee700 1 -- 172.21.15.92:0/1780677198 wait complete. 2024-05-06T12:11:51.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.518+0000 7f923a3ee700 1 Processor -- start 2024-05-06T12:11:51.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.519+0000 7f923a3ee700 1 -- start start 2024-05-06T12:11:51.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.519+0000 7f923a3ee700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92340fbcc0 0x7f9234066cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:51.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.519+0000 7f923a3ee700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92340fcf70 0x7f9234112740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:51.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.519+0000 7f923a3ee700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9234112c80 con 0x7f92340fbcc0 2024-05-06T12:11:51.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.519+0000 7f923a3ee700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9234112df0 con 0x7f92340fcf70 2024-05-06T12:11:51.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.519+0000 7f9233fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92340fbcc0 0x7f9234066cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:51.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.519+0000 7f9233fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92340fbcc0 0x7f9234066cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46194/0 (socket says 172.21.15.92:46194) 2024-05-06T12:11:51.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.519+0000 7f9233fff700 1 -- 172.21.15.92:0/3550484138 learned_addr learned my addr 172.21.15.92:0/3550484138 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:11:51.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.519+0000 7f9233fff700 1 -- 172.21.15.92:0/3550484138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92340fcf70 msgr2=0x7f9234112740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:51.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.519+0000 7f92337fe700 1 --2- 172.21.15.92:0/3550484138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92340fcf70 0x7f9234112740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:51.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.519+0000 7f9233fff700 1 --2- 172.21.15.92:0/3550484138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92340fcf70 0x7f9234112740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:51.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.519+0000 7f9233fff700 1 -- 172.21.15.92:0/3550484138 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9224005040 con 0x7f92340fbcc0 2024-05-06T12:11:51.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.519+0000 7f92337fe700 1 --2- 172.21.15.92:0/3550484138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92340fcf70 0x7f9234112740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-06T12:11:51.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.519+0000 7f9233fff700 1 --2- 172.21.15.92:0/3550484138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92340fbcc0 0x7f9234066cb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f922400b3b0 tx=0x7f9224009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:51.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.519+0000 7f9230ff9700 1 -- 172.21.15.92:0/3550484138 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f922400e040 con 0x7f92340fbcc0 2024-05-06T12:11:51.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.520+0000 7f9230ff9700 1 -- 172.21.15.92:0/3550484138 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f922400bcc0 con 0x7f92340fbcc0 2024-05-06T12:11:51.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.520+0000 7f923a3ee700 1 -- 172.21.15.92:0/3550484138 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9234113070 con 0x7f92340fbcc0 2024-05-06T12:11:51.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.520+0000 7f9230ff9700 1 -- 172.21.15.92:0/3550484138 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f922401c630 con 0x7f92340fbcc0 2024-05-06T12:11:51.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.520+0000 7f923a3ee700 1 -- 172.21.15.92:0/3550484138 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9234113630 con 0x7f92340fbcc0 2024-05-06T12:11:51.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.521+0000 7f923a3ee700 1 -- 172.21.15.92:0/3550484138 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f923404c710 con 0x7f92340fbcc0 2024-05-06T12:11:51.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.521+0000 7f9230ff9700 1 -- 172.21.15.92:0/3550484138 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f9224026440 con 0x7f92340fbcc0 2024-05-06T12:11:51.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.521+0000 7f9230ff9700 1 --2- 172.21.15.92:0/3550484138 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f921c0692a0 0x7f921c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:51.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.522+0000 7f92337fe700 1 --2- 172.21.15.92:0/3550484138 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f921c0692a0 0x7f921c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:51.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.522+0000 7f9230ff9700 1 -- 172.21.15.92:0/3550484138 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f9224087fb0 con 0x7f92340fbcc0 2024-05-06T12:11:51.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.522+0000 7f92337fe700 1 --2- 172.21.15.92:0/3550484138 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f921c0692a0 0x7f921c06b750 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9228006fd0 tx=0x7f9228008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:51.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.526+0000 7f9230ff9700 1 -- 172.21.15.92:0/3550484138 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9224057d60 con 0x7f92340fbcc0 2024-05-06T12:11:51.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:51.762+0000 7f923a3ee700 1 -- 172.21.15.92:0/3550484138 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7f9234113910 con 0x7f92340fbcc0 2024-05-06T12:11:52.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:52.064+0000 7f9230ff9700 1 -- 172.21.15.92:0/3550484138 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7f92240578f0 con 0x7f92340fbcc0 2024-05-06T12:11:52.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:52.067+0000 7f923a3ee700 1 -- 172.21.15.92:0/3550484138 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f921c0692a0 msgr2=0x7f921c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:52.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:52.067+0000 7f923a3ee700 1 --2- 172.21.15.92:0/3550484138 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f921c0692a0 0x7f921c06b750 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9228006fd0 tx=0x7f9228008040 comp rx=0 tx=0).stop 2024-05-06T12:11:52.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:52.067+0000 7f923a3ee700 1 -- 172.21.15.92:0/3550484138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92340fbcc0 msgr2=0x7f9234066cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:52.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:52.067+0000 7f923a3ee700 1 --2- 172.21.15.92:0/3550484138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92340fbcc0 0x7f9234066cb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f922400b3b0 tx=0x7f9224009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:52.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:52.067+0000 7f923a3ee700 1 -- 172.21.15.92:0/3550484138 shutdown_connections 2024-05-06T12:11:52.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:52.067+0000 7f923a3ee700 1 --2- 172.21.15.92:0/3550484138 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f921c0692a0 0x7f921c06b750 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:52.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:52.067+0000 7f923a3ee700 1 --2- 172.21.15.92:0/3550484138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92340fbcc0 0x7f9234066cb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:52.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:52.067+0000 7f923a3ee700 1 --2- 172.21.15.92:0/3550484138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92340fcf70 0x7f9234112740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:52.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:52.067+0000 7f923a3ee700 1 -- 172.21.15.92:0/3550484138 >> 172.21.15.92:0/3550484138 conn(0x7f92340f7440 msgr2=0x7f92340f8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:52.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:52.067+0000 7f923a3ee700 1 -- 172.21.15.92:0/3550484138 shutdown_connections 2024-05-06T12:11:52.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:52.067+0000 7f923a3ee700 1 -- 172.21.15.92:0/3550484138 wait complete. 2024-05-06T12:11:52.708 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-06T12:11:52.718 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi092.front.sepia.ceph.com 2024-05-06T12:11:52.718 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- bash -c 'ceph fs set cephfs allow_standby_replay true' 2024-05-06T12:11:52.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:52 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:11:52.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:52 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:11:52.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:52 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/3550484138' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-05-06T12:11:52.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:52 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:11:52.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:52 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi092.rosbll", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-06T12:11:52.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:52 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi092.rosbll", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-06T12:11:52.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:52 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:11:52.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:52 smithi092 ceph-mon[26954]: Deploying daemon mds.cephfs.smithi092.rosbll on smithi092 2024-05-06T12:11:52.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:52 smithi092 ceph-mon[26954]: pgmap v177: 65 pgs: 65 active+clean; 578 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 593 B/s wr, 2 op/s 2024-05-06T12:11:52.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:52 smithi092 ceph-mon[26954]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-06T12:11:52.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:52 smithi092 ceph-mon[26954]: mds.? [v2:172.21.15.190:6824/1565151699,v1:172.21.15.190:6825/1565151699] up:boot 2024-05-06T12:11:52.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:52 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/3550484138' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-05-06T12:11:52.974 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:52 smithi092 ceph-mon[26954]: daemon mds.cephfs.smithi190.byjvgf assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-06T12:11:52.974 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:52 smithi092 ceph-mon[26954]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-05-06T12:11:52.974 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:52 smithi092 ceph-mon[26954]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-06T12:11:52.974 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:52 smithi092 ceph-mon[26954]: fsmap cephfs:1 {0=cephfs.smithi092.ijsqgp=up:active} 1 up:standby 2024-05-06T12:11:52.974 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:52 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi190.byjvgf"}]: dispatch 2024-05-06T12:11:52.974 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:52 smithi092 ceph-mon[26954]: fsmap cephfs:2 {0=cephfs.smithi092.ijsqgp=up:active,1=cephfs.smithi190.byjvgf=up:creating} 2024-05-06T12:11:52.974 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:52 smithi092 ceph-mon[26954]: daemon mds.cephfs.smithi190.byjvgf is now active in filesystem cephfs as rank 1 2024-05-06T12:11:53.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:52 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:11:53.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:52 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:11:53.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:52 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/3550484138' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-05-06T12:11:53.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:52 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:11:53.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:52 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi092.rosbll", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-06T12:11:53.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:52 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi092.rosbll", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-06T12:11:53.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:52 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:11:53.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:52 smithi190 ceph-mon[33798]: Deploying daemon mds.cephfs.smithi092.rosbll on smithi092 2024-05-06T12:11:53.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:52 smithi190 ceph-mon[33798]: pgmap v177: 65 pgs: 65 active+clean; 578 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 593 B/s wr, 2 op/s 2024-05-06T12:11:53.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:52 smithi190 ceph-mon[33798]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-06T12:11:53.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:52 smithi190 ceph-mon[33798]: mds.? [v2:172.21.15.190:6824/1565151699,v1:172.21.15.190:6825/1565151699] up:boot 2024-05-06T12:11:53.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:52 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/3550484138' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-05-06T12:11:53.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:52 smithi190 ceph-mon[33798]: daemon mds.cephfs.smithi190.byjvgf assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-06T12:11:53.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:52 smithi190 ceph-mon[33798]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-05-06T12:11:53.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:52 smithi190 ceph-mon[33798]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-06T12:11:53.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:52 smithi190 ceph-mon[33798]: fsmap cephfs:1 {0=cephfs.smithi092.ijsqgp=up:active} 1 up:standby 2024-05-06T12:11:53.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:52 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi190.byjvgf"}]: dispatch 2024-05-06T12:11:53.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:52 smithi190 ceph-mon[33798]: fsmap cephfs:2 {0=cephfs.smithi092.ijsqgp=up:active,1=cephfs.smithi190.byjvgf=up:creating} 2024-05-06T12:11:53.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:52 smithi190 ceph-mon[33798]: daemon mds.cephfs.smithi190.byjvgf is now active in filesystem cephfs as rank 1 2024-05-06T12:11:54.024 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:11:54.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:54 smithi092 ceph-mon[26954]: mds.? [v2:172.21.15.190:6824/1565151699,v1:172.21.15.190:6825/1565151699] up:active 2024-05-06T12:11:54.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:54 smithi092 ceph-mon[26954]: fsmap cephfs:2 {0=cephfs.smithi092.ijsqgp=up:active,1=cephfs.smithi190.byjvgf=up:active} 2024-05-06T12:11:54.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:54 smithi190 ceph-mon[33798]: mds.? [v2:172.21.15.190:6824/1565151699,v1:172.21.15.190:6825/1565151699] up:active 2024-05-06T12:11:54.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:54 smithi190 ceph-mon[33798]: fsmap cephfs:2 {0=cephfs.smithi092.ijsqgp=up:active,1=cephfs.smithi190.byjvgf=up:active} 2024-05-06T12:11:55.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:55 smithi190 ceph-mon[33798]: pgmap v178: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s wr, 6 op/s 2024-05-06T12:11:55.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:55 smithi092 ceph-mon[26954]: pgmap v178: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s wr, 6 op/s 2024-05-06T12:11:55.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.864+0000 7f40c5119700 1 -- 172.21.15.92:0/2302307079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c00fcf60 msgr2=0x7f40c00fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:55.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.864+0000 7f40c5119700 1 --2- 172.21.15.92:0/2302307079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c00fcf60 0x7f40c00fd3d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f40b0004940 tx=0x7f40b0009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:55.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.864+0000 7f40c5119700 1 -- 172.21.15.92:0/2302307079 shutdown_connections 2024-05-06T12:11:55.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.864+0000 7f40c5119700 1 --2- 172.21.15.92:0/2302307079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c00fcf60 0x7f40c00fd3d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:55.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.864+0000 7f40c5119700 1 --2- 172.21.15.92:0/2302307079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40c00fbcb0 0x7f40c00fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:55.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.864+0000 7f40c5119700 1 -- 172.21.15.92:0/2302307079 >> 172.21.15.92:0/2302307079 conn(0x7f40c00f7430 msgr2=0x7f40c00f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:55.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.864+0000 7f40c5119700 1 -- 172.21.15.92:0/2302307079 shutdown_connections 2024-05-06T12:11:55.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.864+0000 7f40c5119700 1 -- 172.21.15.92:0/2302307079 wait complete. 2024-05-06T12:11:55.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.865+0000 7f40c5119700 1 Processor -- start 2024-05-06T12:11:55.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.865+0000 7f40c5119700 1 -- start start 2024-05-06T12:11:55.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.865+0000 7f40c5119700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c00fbcb0 0x7f40c01106a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:55.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.865+0000 7f40c5119700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40c00fcf60 0x7f40c0112bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:55.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.865+0000 7f40c5119700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c0066350 con 0x7f40c00fbcb0 2024-05-06T12:11:55.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.865+0000 7f40c5119700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c0066490 con 0x7f40c00fcf60 2024-05-06T12:11:55.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.865+0000 7f40bed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c00fbcb0 0x7f40c01106a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:55.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.865+0000 7f40bed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c00fbcb0 0x7f40c01106a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52420/0 (socket says 172.21.15.92:52420) 2024-05-06T12:11:55.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.865+0000 7f40bed9d700 1 -- 172.21.15.92:0/4193788290 learned_addr learned my addr 172.21.15.92:0/4193788290 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:11:55.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.865+0000 7f40be59c700 1 --2- 172.21.15.92:0/4193788290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40c00fcf60 0x7f40c0112bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:55.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.865+0000 7f40bed9d700 1 -- 172.21.15.92:0/4193788290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40c00fcf60 msgr2=0x7f40c0112bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:55.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.865+0000 7f40bed9d700 1 --2- 172.21.15.92:0/4193788290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40c00fcf60 0x7f40c0112bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:55.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.866+0000 7f40bed9d700 1 -- 172.21.15.92:0/4193788290 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b0005040 con 0x7f40c00fbcb0 2024-05-06T12:11:55.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.866+0000 7f40bed9d700 1 --2- 172.21.15.92:0/4193788290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c00fbcb0 0x7f40c01106a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f40b4002a80 tx=0x7f40b40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:55.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.866+0000 7f40a77fe700 1 -- 172.21.15.92:0/4193788290 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40b4007cc0 con 0x7f40c00fbcb0 2024-05-06T12:11:55.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.866+0000 7f40c5119700 1 -- 172.21.15.92:0/4193788290 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c01131f0 con 0x7f40c00fbcb0 2024-05-06T12:11:55.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.866+0000 7f40a77fe700 1 -- 172.21.15.92:0/4193788290 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f40b400f040 con 0x7f40c00fbcb0 2024-05-06T12:11:55.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.866+0000 7f40a77fe700 1 -- 172.21.15.92:0/4193788290 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40b400dac0 con 0x7f40c00fbcb0 2024-05-06T12:11:55.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.866+0000 7f40c5119700 1 -- 172.21.15.92:0/4193788290 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c0113780 con 0x7f40c00fbcb0 2024-05-06T12:11:55.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.867+0000 7f40c5119700 1 -- 172.21.15.92:0/4193788290 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c005f5c0 con 0x7f40c00fbcb0 2024-05-06T12:11:55.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.867+0000 7f40a77fe700 1 -- 172.21.15.92:0/4193788290 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f40b400dc20 con 0x7f40c00fbcb0 2024-05-06T12:11:55.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.868+0000 7f40a77fe700 1 --2- 172.21.15.92:0/4193788290 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f40a80692a0 0x7f40a806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:55.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.868+0000 7f40a77fe700 1 -- 172.21.15.92:0/4193788290 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f40b4086e00 con 0x7f40c00fbcb0 2024-05-06T12:11:55.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.868+0000 7f40be59c700 1 --2- 172.21.15.92:0/4193788290 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f40a80692a0 0x7f40a806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:55.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.868+0000 7f40be59c700 1 --2- 172.21.15.92:0/4193788290 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f40a80692a0 0x7f40a806b750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f40b000b970 tx=0x7f40b0009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:55.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:55.872+0000 7f40a77fe700 1 -- 172.21.15.92:0/4193788290 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f40b40530b0 con 0x7f40c00fbcb0 2024-05-06T12:11:56.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:56.119+0000 7f40c5119700 1 -- 172.21.15.92:0/4193788290 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f40c01048d0 con 0x7f40c00fbcb0 2024-05-06T12:11:56.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:56.538+0000 7f40a77fe700 1 -- 172.21.15.92:0/4193788290 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v9) v1 ==== 121+0+0 (secure 0 0 0) 0x7f40b40566d0 con 0x7f40c00fbcb0 2024-05-06T12:11:56.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:56.540+0000 7f40c5119700 1 -- 172.21.15.92:0/4193788290 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f40a80692a0 msgr2=0x7f40a806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:56.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:56.540+0000 7f40c5119700 1 --2- 172.21.15.92:0/4193788290 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f40a80692a0 0x7f40a806b750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f40b000b970 tx=0x7f40b0009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:56.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:56.540+0000 7f40c5119700 1 -- 172.21.15.92:0/4193788290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c00fbcb0 msgr2=0x7f40c01106a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:56.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:56.540+0000 7f40c5119700 1 --2- 172.21.15.92:0/4193788290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c00fbcb0 0x7f40c01106a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f40b4002a80 tx=0x7f40b40046b0 comp rx=0 tx=0).stop 2024-05-06T12:11:56.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:56.540+0000 7f40c5119700 1 -- 172.21.15.92:0/4193788290 shutdown_connections 2024-05-06T12:11:56.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:56.540+0000 7f40c5119700 1 --2- 172.21.15.92:0/4193788290 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f40a80692a0 0x7f40a806b750 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:56.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:56.540+0000 7f40c5119700 1 --2- 172.21.15.92:0/4193788290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c00fbcb0 0x7f40c01106a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:56.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:56.540+0000 7f40c5119700 1 --2- 172.21.15.92:0/4193788290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40c00fcf60 0x7f40c0112bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:56.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:56.540+0000 7f40c5119700 1 -- 172.21.15.92:0/4193788290 >> 172.21.15.92:0/4193788290 conn(0x7f40c00f7430 msgr2=0x7f40c00f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:56.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:56.540+0000 7f40c5119700 1 -- 172.21.15.92:0/4193788290 shutdown_connections 2024-05-06T12:11:56.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:56.540+0000 7f40c5119700 1 -- 172.21.15.92:0/4193788290 wait complete. 2024-05-06T12:11:57.165 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:57 smithi092 ceph-mon[26954]: pgmap v179: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s wr, 6 op/s 2024-05-06T12:11:57.165 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:57 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/4193788290' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-06T12:11:57.165 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:57 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:11:57.165 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:57 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:11:57.165 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:57 smithi092 ceph-mon[26954]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-05-06T12:11:57.165 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:57 smithi092 ceph-mon[26954]: Cluster is now healthy 2024-05-06T12:11:57.165 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:57 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/4193788290' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-06T12:11:57.166 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:57 smithi092 ceph-mon[26954]: mds.? [v2:172.21.15.92:6828/1663253598,v1:172.21.15.92:6829/1663253598] up:boot 2024-05-06T12:11:57.166 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:57 smithi092 ceph-mon[26954]: fsmap cephfs:2 {0=cephfs.smithi092.ijsqgp=up:active,1=cephfs.smithi190.byjvgf=up:active} 1 up:standby-replay 2024-05-06T12:11:57.166 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:57 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi092.rosbll"}]: dispatch 2024-05-06T12:11:57.166 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:57 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:11:57.166 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:57 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi190.idzoff", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-06T12:11:57.166 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:57 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi190.idzoff", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-06T12:11:57.166 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:57 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:11:57.189 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-06T12:11:57.203 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi092.front.sepia.ceph.com 2024-05-06T12:11:57.203 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- bash -c 'ceph fs set cephfs inline_data true --yes-i-really-really-mean-it' 2024-05-06T12:11:57.260 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:57 smithi190 ceph-mon[33798]: pgmap v179: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s wr, 6 op/s 2024-05-06T12:11:57.260 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:57 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/4193788290' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-06T12:11:57.260 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:57 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:11:57.260 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:57 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:11:57.260 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:57 smithi190 ceph-mon[33798]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-05-06T12:11:57.260 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:57 smithi190 ceph-mon[33798]: Cluster is now healthy 2024-05-06T12:11:57.261 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:57 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/4193788290' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-06T12:11:57.261 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:57 smithi190 ceph-mon[33798]: mds.? [v2:172.21.15.92:6828/1663253598,v1:172.21.15.92:6829/1663253598] up:boot 2024-05-06T12:11:57.261 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:57 smithi190 ceph-mon[33798]: fsmap cephfs:2 {0=cephfs.smithi092.ijsqgp=up:active,1=cephfs.smithi190.byjvgf=up:active} 1 up:standby-replay 2024-05-06T12:11:57.261 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:57 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi092.rosbll"}]: dispatch 2024-05-06T12:11:57.261 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:57 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:11:57.261 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:57 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi190.idzoff", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-06T12:11:57.261 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:57 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi190.idzoff", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-06T12:11:57.261 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:57 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:11:57.458 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:11:58.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:58 smithi092 ceph-mon[26954]: Deploying daemon mds.cephfs.smithi190.idzoff on smithi190 2024-05-06T12:11:58.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:58 smithi190 ceph-mon[33798]: Deploying daemon mds.cephfs.smithi190.idzoff on smithi190 2024-05-06T12:11:59.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:58.999+0000 7f36fa58a700 1 -- 172.21.15.92:0/758277799 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36f40fed60 msgr2=0x7f36f40faea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:59.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:58.999+0000 7f36fa58a700 1 --2- 172.21.15.92:0/758277799 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36f40fed60 0x7f36f40faea0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f36e8004940 tx=0x7f36e8009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:59.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:58.999+0000 7f36fa58a700 1 -- 172.21.15.92:0/758277799 shutdown_connections 2024-05-06T12:11:59.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:58.999+0000 7f36fa58a700 1 --2- 172.21.15.92:0/758277799 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36f40fed60 0x7f36f40faea0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:59.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:58.999+0000 7f36fa58a700 1 --2- 172.21.15.92:0/758277799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36f40fe410 0x7f36f40fe820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:59.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:58.999+0000 7f36fa58a700 1 -- 172.21.15.92:0/758277799 >> 172.21.15.92:0/758277799 conn(0x7f36f40f6bd0 msgr2=0x7f36f40f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:59.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:58.999+0000 7f36fa58a700 1 -- 172.21.15.92:0/758277799 shutdown_connections 2024-05-06T12:11:59.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:58.999+0000 7f36fa58a700 1 -- 172.21.15.92:0/758277799 wait complete. 2024-05-06T12:11:59.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.000+0000 7f36fa58a700 1 Processor -- start 2024-05-06T12:11:59.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.000+0000 7f36fa58a700 1 -- start start 2024-05-06T12:11:59.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.000+0000 7f36fa58a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36f40fe410 0x7f36f410d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:59.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.000+0000 7f36fa58a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36f40fed60 0x7f36f410de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:59.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.000+0000 7f36fa58a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f410e420 con 0x7f36f40fed60 2024-05-06T12:11:59.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.000+0000 7f36fa58a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f410e590 con 0x7f36f40fe410 2024-05-06T12:11:59.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.000+0000 7f36f37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36f40fed60 0x7f36f410de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:59.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.000+0000 7f36f37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36f40fed60 0x7f36f410de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52454/0 (socket says 172.21.15.92:52454) 2024-05-06T12:11:59.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.000+0000 7f36f37fe700 1 -- 172.21.15.92:0/2251123644 learned_addr learned my addr 172.21.15.92:0/2251123644 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:11:59.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.000+0000 7f36f3fff700 1 --2- 172.21.15.92:0/2251123644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36f40fe410 0x7f36f410d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:59.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.001+0000 7f36f37fe700 1 -- 172.21.15.92:0/2251123644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36f40fe410 msgr2=0x7f36f410d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:59.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.001+0000 7f36f37fe700 1 --2- 172.21.15.92:0/2251123644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36f40fe410 0x7f36f410d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:59.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.001+0000 7f36f37fe700 1 -- 172.21.15.92:0/2251123644 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e8005040 con 0x7f36f40fed60 2024-05-06T12:11:59.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.001+0000 7f36f3fff700 1 --2- 172.21.15.92:0/2251123644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36f40fe410 0x7f36f410d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-06T12:11:59.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.001+0000 7f36f37fe700 1 --2- 172.21.15.92:0/2251123644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36f40fed60 0x7f36f410de20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f36e8009150 tx=0x7f36e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:59.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.001+0000 7f36f0ff9700 1 -- 172.21.15.92:0/2251123644 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36e800e040 con 0x7f36f40fed60 2024-05-06T12:11:59.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.001+0000 7f36f0ff9700 1 -- 172.21.15.92:0/2251123644 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f36e800bbf0 con 0x7f36f40fed60 2024-05-06T12:11:59.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.001+0000 7f36f0ff9700 1 -- 172.21.15.92:0/2251123644 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36e801c630 con 0x7f36f40fed60 2024-05-06T12:11:59.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.001+0000 7f36fa58a700 1 -- 172.21.15.92:0/2251123644 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36f4112f90 con 0x7f36f40fed60 2024-05-06T12:11:59.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.001+0000 7f36fa58a700 1 -- 172.21.15.92:0/2251123644 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36f4113520 con 0x7f36f40fed60 2024-05-06T12:11:59.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.002+0000 7f36fa58a700 1 -- 172.21.15.92:0/2251123644 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36f404c710 con 0x7f36f40fed60 2024-05-06T12:11:59.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.002+0000 7f36f0ff9700 1 -- 172.21.15.92:0/2251123644 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f36e800bd60 con 0x7f36f40fed60 2024-05-06T12:11:59.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.003+0000 7f36f0ff9700 1 --2- 172.21.15.92:0/2251123644 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f36dc0692a0 0x7f36dc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:11:59.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.003+0000 7f36f3fff700 1 --2- 172.21.15.92:0/2251123644 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f36dc0692a0 0x7f36dc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:11:59.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.003+0000 7f36f0ff9700 1 -- 172.21.15.92:0/2251123644 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f36e80885b0 con 0x7f36f40fed60 2024-05-06T12:11:59.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.003+0000 7f36f3fff700 1 --2- 172.21.15.92:0/2251123644 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f36dc0692a0 0x7f36dc06b750 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f36e4009fd0 tx=0x7f36e4009ee0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:11:59.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.007+0000 7f36f0ff9700 1 -- 172.21.15.92:0/2251123644 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f36e8058370 con 0x7f36f40fed60 2024-05-06T12:11:59.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.257+0000 7f36fa58a700 1 -- 172.21.15.92:0/2251123644 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f36f4113740 con 0x7f36f40fed60 2024-05-06T12:11:59.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:59 smithi190 ceph-mon[33798]: pgmap v180: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-05-06T12:11:59.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:59 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:11:59.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:59 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:11:59.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:11:59 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:11:59.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:59 smithi092 ceph-mon[26954]: pgmap v180: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-05-06T12:11:59.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:59 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:11:59.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:59 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:11:59.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:11:59 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:11:59.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.890+0000 7f36f0ff9700 1 -- 172.21.15.92:0/2251123644 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]=0 inline data enabled v10) v1 ==== 168+0+0 (secure 0 0 0) 0x7f36e8057f00 con 0x7f36f40fed60 2024-05-06T12:11:59.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.893+0000 7f36fa58a700 1 -- 172.21.15.92:0/2251123644 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f36dc0692a0 msgr2=0x7f36dc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:59.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.893+0000 7f36fa58a700 1 --2- 172.21.15.92:0/2251123644 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f36dc0692a0 0x7f36dc06b750 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f36e4009fd0 tx=0x7f36e4009ee0 comp rx=0 tx=0).stop 2024-05-06T12:11:59.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.893+0000 7f36fa58a700 1 -- 172.21.15.92:0/2251123644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36f40fed60 msgr2=0x7f36f410de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:11:59.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.893+0000 7f36fa58a700 1 --2- 172.21.15.92:0/2251123644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36f40fed60 0x7f36f410de20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f36e8009150 tx=0x7f36e8009e30 comp rx=0 tx=0).stop 2024-05-06T12:11:59.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.893+0000 7f36fa58a700 1 -- 172.21.15.92:0/2251123644 shutdown_connections 2024-05-06T12:11:59.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.893+0000 7f36fa58a700 1 --2- 172.21.15.92:0/2251123644 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f36dc0692a0 0x7f36dc06b750 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:59.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.893+0000 7f36fa58a700 1 --2- 172.21.15.92:0/2251123644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36f40fe410 0x7f36f410d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:59.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.893+0000 7f36fa58a700 1 --2- 172.21.15.92:0/2251123644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36f40fed60 0x7f36f410de20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:11:59.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.893+0000 7f36fa58a700 1 -- 172.21.15.92:0/2251123644 >> 172.21.15.92:0/2251123644 conn(0x7f36f40f6bd0 msgr2=0x7f36f40f8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:11:59.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.893+0000 7f36fa58a700 1 -- 172.21.15.92:0/2251123644 shutdown_connections 2024-05-06T12:11:59.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:11:59.893+0000 7f36fa58a700 1 -- 172.21.15.92:0/2251123644 wait complete. 2024-05-06T12:11:59.898 INFO:teuthology.orchestra.run.smithi092.stderr:inline data enabled 2024-05-06T12:12:00.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:00 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:00.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:00 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:00.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:00 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:00.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:00 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:12:00.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:00 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/2251123644' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-06T12:12:00.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:00 smithi190 ceph-mon[33798]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-05-06T12:12:00.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:00 smithi190 ceph-mon[33798]: mds.? [v2:172.21.15.190:6826/2453728380,v1:172.21.15.190:6827/2453728380] up:boot 2024-05-06T12:12:00.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:00 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/2251123644' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-05-06T12:12:00.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:00 smithi190 ceph-mon[33798]: fsmap cephfs:2 {0=cephfs.smithi092.ijsqgp=up:active,1=cephfs.smithi190.byjvgf=up:active} 1 up:standby-replay 1 up:standby 2024-05-06T12:12:00.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:00 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi190.idzoff"}]: dispatch 2024-05-06T12:12:00.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:00 smithi190 ceph-mon[33798]: fsmap cephfs:2 {0=cephfs.smithi092.ijsqgp=up:active,1=cephfs.smithi190.byjvgf=up:active} 2 up:standby-replay 2024-05-06T12:12:00.423 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:00 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:00.423 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:00 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:00.423 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:00 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:00.423 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:00 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:12:00.423 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:00 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/2251123644' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-06T12:12:00.423 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:00 smithi092 ceph-mon[26954]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-05-06T12:12:00.423 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:00 smithi092 ceph-mon[26954]: mds.? [v2:172.21.15.190:6826/2453728380,v1:172.21.15.190:6827/2453728380] up:boot 2024-05-06T12:12:00.423 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:00 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/2251123644' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-05-06T12:12:00.423 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:00 smithi092 ceph-mon[26954]: fsmap cephfs:2 {0=cephfs.smithi092.ijsqgp=up:active,1=cephfs.smithi190.byjvgf=up:active} 1 up:standby-replay 1 up:standby 2024-05-06T12:12:00.423 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:00 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi190.idzoff"}]: dispatch 2024-05-06T12:12:00.423 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:00 smithi092 ceph-mon[26954]: fsmap cephfs:2 {0=cephfs.smithi092.ijsqgp=up:active,1=cephfs.smithi190.byjvgf=up:active} 2 up:standby-replay 2024-05-06T12:12:00.771 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-06T12:12:00.784 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi092.front.sepia.ceph.com 2024-05-06T12:12:00.784 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-06T12:12:01.351 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:01 smithi092 ceph-mon[26954]: pgmap v181: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 255 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-05-06T12:12:01.351 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:01 smithi092 ceph-mon[26954]: mds.? [v2:172.21.15.190:6824/1565151699,v1:172.21.15.190:6825/1565151699] up:active 2024-05-06T12:12:01.351 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:01 smithi092 ceph-mon[26954]: mds.? [v2:172.21.15.92:6828/1663253598,v1:172.21.15.92:6829/1663253598] up:standby-replay 2024-05-06T12:12:01.351 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:01 smithi092 ceph-mon[26954]: Dropping low affinity active daemon mds.cephfs.smithi092.ijsqgp in favor of higher affinity standby. 2024-05-06T12:12:01.351 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:01 smithi092 ceph-mon[26954]: Replacing daemon mds.cephfs.smithi092.ijsqgp as rank 0 with standby daemon mds.cephfs.smithi092.rosbll 2024-05-06T12:12:01.352 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:01 smithi092 ceph-mon[26954]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-06T12:12:01.352 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:01 smithi092 ceph-mon[26954]: fsmap cephfs:2 {0=cephfs.smithi092.ijsqgp=up:active,1=cephfs.smithi190.byjvgf=up:active} 2 up:standby-replay 2024-05-06T12:12:01.352 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:01 smithi092 ceph-mon[26954]: osdmap e46: 6 total, 6 up, 6 in 2024-05-06T12:12:01.352 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:01 smithi092 ceph-mon[26954]: fsmap cephfs:2/2 {0=cephfs.smithi092.rosbll=up:replay,1=cephfs.smithi190.byjvgf=up:active} 1 up:standby-replay 2024-05-06T12:12:01.582 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:12:01.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:01 smithi190 ceph-mon[33798]: pgmap v181: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 255 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-05-06T12:12:01.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:01 smithi190 ceph-mon[33798]: mds.? [v2:172.21.15.190:6824/1565151699,v1:172.21.15.190:6825/1565151699] up:active 2024-05-06T12:12:01.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:01 smithi190 ceph-mon[33798]: mds.? [v2:172.21.15.92:6828/1663253598,v1:172.21.15.92:6829/1663253598] up:standby-replay 2024-05-06T12:12:01.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:01 smithi190 ceph-mon[33798]: Dropping low affinity active daemon mds.cephfs.smithi092.ijsqgp in favor of higher affinity standby. 2024-05-06T12:12:01.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:01 smithi190 ceph-mon[33798]: Replacing daemon mds.cephfs.smithi092.ijsqgp as rank 0 with standby daemon mds.cephfs.smithi092.rosbll 2024-05-06T12:12:01.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:01 smithi190 ceph-mon[33798]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-06T12:12:01.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:01 smithi190 ceph-mon[33798]: fsmap cephfs:2 {0=cephfs.smithi092.ijsqgp=up:active,1=cephfs.smithi190.byjvgf=up:active} 2 up:standby-replay 2024-05-06T12:12:01.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:01 smithi190 ceph-mon[33798]: osdmap e46: 6 total, 6 up, 6 in 2024-05-06T12:12:01.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:01 smithi190 ceph-mon[33798]: fsmap cephfs:2/2 {0=cephfs.smithi092.rosbll=up:replay,1=cephfs.smithi190.byjvgf=up:active} 1 up:standby-replay 2024-05-06T12:12:02.357 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:02 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:02.358 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:02 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:02.358 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:02 smithi092 ceph-mon[26954]: mds.? [v2:172.21.15.92:6826/3613441054,v1:172.21.15.92:6827/3613441054] up:boot 2024-05-06T12:12:02.358 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:02 smithi092 ceph-mon[26954]: mds.? [v2:172.21.15.92:6828/1663253598,v1:172.21.15.92:6829/1663253598] up:resolve 2024-05-06T12:12:02.358 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:02 smithi092 ceph-mon[26954]: fsmap cephfs:2/2 {0=cephfs.smithi092.rosbll=up:resolve,1=cephfs.smithi190.byjvgf=up:active} 1 up:standby-replay 1 up:standby 2024-05-06T12:12:02.358 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:02 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi092.ijsqgp"}]: dispatch 2024-05-06T12:12:02.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:02 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:02.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:02 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:02.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:02 smithi190 ceph-mon[33798]: mds.? [v2:172.21.15.92:6826/3613441054,v1:172.21.15.92:6827/3613441054] up:boot 2024-05-06T12:12:02.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:02 smithi190 ceph-mon[33798]: mds.? [v2:172.21.15.92:6828/1663253598,v1:172.21.15.92:6829/1663253598] up:resolve 2024-05-06T12:12:02.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:02 smithi190 ceph-mon[33798]: fsmap cephfs:2/2 {0=cephfs.smithi092.rosbll=up:resolve,1=cephfs.smithi190.byjvgf=up:active} 1 up:standby-replay 1 up:standby 2024-05-06T12:12:02.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:02 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi092.ijsqgp"}]: dispatch 2024-05-06T12:12:03.295 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:03 smithi092 ceph-mon[26954]: pgmap v183: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-06T12:12:03.295 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:03 smithi092 ceph-mon[26954]: mds.? [v2:172.21.15.92:6828/1663253598,v1:172.21.15.92:6829/1663253598] up:reconnect 2024-05-06T12:12:03.295 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:03 smithi092 ceph-mon[26954]: fsmap cephfs:2/2 {0=cephfs.smithi092.rosbll=up:reconnect,1=cephfs.smithi190.byjvgf=up:active} 1 up:standby-replay 1 up:standby 2024-05-06T12:12:03.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:03 smithi190 ceph-mon[33798]: pgmap v183: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-06T12:12:03.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:03 smithi190 ceph-mon[33798]: mds.? [v2:172.21.15.92:6828/1663253598,v1:172.21.15.92:6829/1663253598] up:reconnect 2024-05-06T12:12:03.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:03 smithi190 ceph-mon[33798]: fsmap cephfs:2/2 {0=cephfs.smithi092.rosbll=up:reconnect,1=cephfs.smithi190.byjvgf=up:active} 1 up:standby-replay 1 up:standby 2024-05-06T12:12:03.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.776+0000 7fabe2112700 1 -- 172.21.15.92:0/583784376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabdc0fdfd0 msgr2=0x7fabdc0fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:03.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.776+0000 7fabe2112700 1 --2- 172.21.15.92:0/583784376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabdc0fdfd0 0x7fabdc0fe440 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fabc4004940 tx=0x7fabc4009e30 comp rx=0 tx=0).stop 2024-05-06T12:12:03.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.776+0000 7fabe2112700 1 -- 172.21.15.92:0/583784376 shutdown_connections 2024-05-06T12:12:03.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.776+0000 7fabe2112700 1 --2- 172.21.15.92:0/583784376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabdc0fdfd0 0x7fabdc0fe440 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:03.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.776+0000 7fabe2112700 1 --2- 172.21.15.92:0/583784376 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabdc0fce60 0x7fabdc0fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:03.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.776+0000 7fabe2112700 1 -- 172.21.15.92:0/583784376 >> 172.21.15.92:0/583784376 conn(0x7fabdc0f8540 msgr2=0x7fabdc0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:03.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.776+0000 7fabe2112700 1 -- 172.21.15.92:0/583784376 shutdown_connections 2024-05-06T12:12:03.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.776+0000 7fabe2112700 1 -- 172.21.15.92:0/583784376 wait complete. 2024-05-06T12:12:03.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.777+0000 7fabe2112700 1 Processor -- start 2024-05-06T12:12:03.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.777+0000 7fabe2112700 1 -- start start 2024-05-06T12:12:03.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.777+0000 7fabe2112700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabdc0fce60 0x7fabdc10d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:03.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.777+0000 7fabe2112700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabdc0fdfd0 0x7fabdc10de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:03.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.777+0000 7fabe2112700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc10e470 con 0x7fabdc0fce60 2024-05-06T12:12:03.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.777+0000 7fabe2112700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc10e5e0 con 0x7fabdc0fdfd0 2024-05-06T12:12:03.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.777+0000 7fabdb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabdc0fce60 0x7fabdc10d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:03.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.777+0000 7fabdb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabdc0fce60 0x7fabdc10d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49014/0 (socket says 172.21.15.92:49014) 2024-05-06T12:12:03.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.777+0000 7fabdb7fe700 1 -- 172.21.15.92:0/2069858989 learned_addr learned my addr 172.21.15.92:0/2069858989 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:12:03.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.777+0000 7fabdaffd700 1 --2- 172.21.15.92:0/2069858989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabdc0fdfd0 0x7fabdc10de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:03.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.778+0000 7fabdb7fe700 1 -- 172.21.15.92:0/2069858989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabdc0fdfd0 msgr2=0x7fabdc10de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:03.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.778+0000 7fabdb7fe700 1 --2- 172.21.15.92:0/2069858989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabdc0fdfd0 0x7fabdc10de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:03.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.778+0000 7fabdb7fe700 1 -- 172.21.15.92:0/2069858989 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc4005040 con 0x7fabdc0fce60 2024-05-06T12:12:03.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.778+0000 7fabdb7fe700 1 --2- 172.21.15.92:0/2069858989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabdc0fce60 0x7fabdc10d930 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fabd4001d00 tx=0x7fabd4009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:03.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.778+0000 7fabd3fff700 1 -- 172.21.15.92:0/2069858989 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabd4001e60 con 0x7fabdc0fce60 2024-05-06T12:12:03.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.778+0000 7fabd3fff700 1 -- 172.21.15.92:0/2069858989 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fabd400f040 con 0x7fabdc0fce60 2024-05-06T12:12:03.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.778+0000 7fabe2112700 1 -- 172.21.15.92:0/2069858989 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabdc112fe0 con 0x7fabdc0fce60 2024-05-06T12:12:03.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.778+0000 7fabd3fff700 1 -- 172.21.15.92:0/2069858989 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabd4009310 con 0x7fabdc0fce60 2024-05-06T12:12:03.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.778+0000 7fabe2112700 1 -- 172.21.15.92:0/2069858989 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabdc113570 con 0x7fabdc0fce60 2024-05-06T12:12:03.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.779+0000 7fabe2112700 1 -- 172.21.15.92:0/2069858989 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabdc04c710 con 0x7fabdc0fce60 2024-05-06T12:12:03.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.779+0000 7fabd3fff700 1 -- 172.21.15.92:0/2069858989 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fabd4009470 con 0x7fabdc0fce60 2024-05-06T12:12:03.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.780+0000 7fabd3fff700 1 --2- 172.21.15.92:0/2069858989 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fabc80692a0 0x7fabc806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:03.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.780+0000 7fabd3fff700 1 -- 172.21.15.92:0/2069858989 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fabd4086b80 con 0x7fabdc0fce60 2024-05-06T12:12:03.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.780+0000 7fabdaffd700 1 --2- 172.21.15.92:0/2069858989 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fabc80692a0 0x7fabc806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:03.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.780+0000 7fabdaffd700 1 --2- 172.21.15.92:0/2069858989 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fabc80692a0 0x7fabc806b750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fabc400b970 tx=0x7fabc4009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:03.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:03.784+0000 7fabd3fff700 1 -- 172.21.15.92:0/2069858989 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fabd404d250 con 0x7fabdc0fce60 2024-05-06T12:12:04.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:04.021+0000 7fabe2112700 1 -- 172.21.15.92:0/2069858989 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fabdc113760 con 0x7fabdc0fce60 2024-05-06T12:12:04.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:04.021+0000 7fabd3fff700 1 -- 172.21.15.92:0/2069858989 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) v1 ==== 76+0+1846 (secure 0 0 0) 0x7fabd4017750 con 0x7fabdc0fce60 2024-05-06T12:12:04.025 INFO:teuthology.orchestra.run.smithi092.stdout:e16 2024-05-06T12:12:04.025 INFO:teuthology.orchestra.run.smithi092.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-06T12:12:04.025 INFO:teuthology.orchestra.run.smithi092.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:12:04.025 INFO:teuthology.orchestra.run.smithi092.stdout:legacy client fscid: 1 2024-05-06T12:12:04.025 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:12:04.025 INFO:teuthology.orchestra.run.smithi092.stdout:Filesystem 'cephfs' (1) 2024-05-06T12:12:04.026 INFO:teuthology.orchestra.run.smithi092.stdout:fs_name cephfs 2024-05-06T12:12:04.026 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 16 2024-05-06T12:12:04.026 INFO:teuthology.orchestra.run.smithi092.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-06T12:12:04.026 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-05-06T12:11:44.952671+0000 2024-05-06T12:12:04.026 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2024-05-06T12:12:03.900287+0000 2024-05-06T12:12:04.027 INFO:teuthology.orchestra.run.smithi092.stdout:tableserver 0 2024-05-06T12:12:04.027 INFO:teuthology.orchestra.run.smithi092.stdout:root 0 2024-05-06T12:12:04.027 INFO:teuthology.orchestra.run.smithi092.stdout:session_timeout 60 2024-05-06T12:12:04.027 INFO:teuthology.orchestra.run.smithi092.stdout:session_autoclose 300 2024-05-06T12:12:04.027 INFO:teuthology.orchestra.run.smithi092.stdout:max_file_size 1099511627776 2024-05-06T12:12:04.027 INFO:teuthology.orchestra.run.smithi092.stdout:required_client_features {} 2024-05-06T12:12:04.027 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure 0 2024-05-06T12:12:04.028 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure_osd_epoch 46 2024-05-06T12:12:04.028 INFO:teuthology.orchestra.run.smithi092.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:12:04.028 INFO:teuthology.orchestra.run.smithi092.stdout:max_mds 2 2024-05-06T12:12:04.028 INFO:teuthology.orchestra.run.smithi092.stdout:in 0,1 2024-05-06T12:12:04.028 INFO:teuthology.orchestra.run.smithi092.stdout:up {0=14472,1=24267} 2024-05-06T12:12:04.028 INFO:teuthology.orchestra.run.smithi092.stdout:failed 2024-05-06T12:12:04.029 INFO:teuthology.orchestra.run.smithi092.stdout:damaged 2024-05-06T12:12:04.029 INFO:teuthology.orchestra.run.smithi092.stdout:stopped 2024-05-06T12:12:04.029 INFO:teuthology.orchestra.run.smithi092.stdout:data_pools [3] 2024-05-06T12:12:04.029 INFO:teuthology.orchestra.run.smithi092.stdout:metadata_pool 2 2024-05-06T12:12:04.029 INFO:teuthology.orchestra.run.smithi092.stdout:inline_data enabled 2024-05-06T12:12:04.029 INFO:teuthology.orchestra.run.smithi092.stdout:balancer 2024-05-06T12:12:04.029 INFO:teuthology.orchestra.run.smithi092.stdout:standby_count_wanted 1 2024-05-06T12:12:04.029 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.rosbll{0:14472} state up:rejoin seq 5 join_fscid=1 addr [v2:172.21.15.92:6828/1663253598,v1:172.21.15.92:6829/1663253598] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:12:04.029 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.byjvgf{1:24267} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.190:6824/1565151699,v1:172.21.15.190:6825/1565151699] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:12:04.030 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.idzoff{1:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.190:6826/2453728380,v1:172.21.15.190:6827/2453728380] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:12:04.030 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:12:04.030 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:12:04.030 INFO:teuthology.orchestra.run.smithi092.stdout:Standby daemons: 2024-05-06T12:12:04.030 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:12:04.030 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.ijsqgp{-1:14484} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.92:6826/3613441054,v1:172.21.15.92:6827/3613441054] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:12:04.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:04.023+0000 7fabe2112700 1 -- 172.21.15.92:0/2069858989 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fabc80692a0 msgr2=0x7fabc806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:04.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:04.023+0000 7fabe2112700 1 --2- 172.21.15.92:0/2069858989 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fabc80692a0 0x7fabc806b750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fabc400b970 tx=0x7fabc4009e30 comp rx=0 tx=0).stop 2024-05-06T12:12:04.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:04.023+0000 7fabe2112700 1 -- 172.21.15.92:0/2069858989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabdc0fce60 msgr2=0x7fabdc10d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:04.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:04.023+0000 7fabe2112700 1 --2- 172.21.15.92:0/2069858989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabdc0fce60 0x7fabdc10d930 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fabd4001d00 tx=0x7fabd4009f30 comp rx=0 tx=0).stop 2024-05-06T12:12:04.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:04.023+0000 7fabe2112700 1 -- 172.21.15.92:0/2069858989 shutdown_connections 2024-05-06T12:12:04.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:04.023+0000 7fabe2112700 1 --2- 172.21.15.92:0/2069858989 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fabc80692a0 0x7fabc806b750 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:04.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:04.023+0000 7fabe2112700 1 --2- 172.21.15.92:0/2069858989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabdc0fce60 0x7fabdc10d930 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:04.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:04.023+0000 7fabe2112700 1 --2- 172.21.15.92:0/2069858989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabdc0fdfd0 0x7fabdc10de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:04.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:04.023+0000 7fabe2112700 1 -- 172.21.15.92:0/2069858989 >> 172.21.15.92:0/2069858989 conn(0x7fabdc0f8540 msgr2=0x7fabdc0f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:04.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:04.023+0000 7fabe2112700 1 -- 172.21.15.92:0/2069858989 shutdown_connections 2024-05-06T12:12:04.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:04.023+0000 7fabe2112700 1 -- 172.21.15.92:0/2069858989 wait complete. 2024-05-06T12:12:04.036 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 16 2024-05-06T12:12:04.881 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-05-06T12:12:05.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:04 smithi092 ceph-mon[26954]: pgmap v184: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 716 B/s wr, 5 op/s 2024-05-06T12:12:05.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:04 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:05.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:04 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:12:05.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:04 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:05.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:04 smithi092 ceph-mon[26954]: mds.? [v2:172.21.15.92:6828/1663253598,v1:172.21.15.92:6829/1663253598] up:rejoin 2024-05-06T12:12:05.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:04 smithi092 ceph-mon[26954]: mds.? [v2:172.21.15.190:6826/2453728380,v1:172.21.15.190:6827/2453728380] up:standby-replay 2024-05-06T12:12:05.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:04 smithi092 ceph-mon[26954]: fsmap cephfs:2/2 {0=cephfs.smithi092.rosbll=up:rejoin,1=cephfs.smithi190.byjvgf=up:active} 1 up:standby-replay 1 up:standby 2024-05-06T12:12:05.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:04 smithi092 ceph-mon[26954]: daemon mds.cephfs.smithi092.rosbll is now active in filesystem cephfs as rank 0 2024-05-06T12:12:05.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:04 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/2069858989' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:12:05.258 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:12:05.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:04 smithi190 ceph-mon[33798]: pgmap v184: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 716 B/s wr, 5 op/s 2024-05-06T12:12:05.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:04 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:05.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:04 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:12:05.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:04 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:05.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:04 smithi190 ceph-mon[33798]: mds.? [v2:172.21.15.92:6828/1663253598,v1:172.21.15.92:6829/1663253598] up:rejoin 2024-05-06T12:12:05.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:04 smithi190 ceph-mon[33798]: mds.? [v2:172.21.15.190:6826/2453728380,v1:172.21.15.190:6827/2453728380] up:standby-replay 2024-05-06T12:12:05.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:04 smithi190 ceph-mon[33798]: fsmap cephfs:2/2 {0=cephfs.smithi092.rosbll=up:rejoin,1=cephfs.smithi190.byjvgf=up:active} 1 up:standby-replay 1 up:standby 2024-05-06T12:12:05.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:04 smithi190 ceph-mon[33798]: daemon mds.cephfs.smithi092.rosbll is now active in filesystem cephfs as rank 0 2024-05-06T12:12:05.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:04 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/2069858989' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:12:06.207 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:05 smithi092 ceph-mon[26954]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-06T12:12:06.208 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:05 smithi092 ceph-mon[26954]: mds.? [v2:172.21.15.92:6828/1663253598,v1:172.21.15.92:6829/1663253598] up:active 2024-05-06T12:12:06.208 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:05 smithi092 ceph-mon[26954]: fsmap cephfs:2 {0=cephfs.smithi092.rosbll=up:active,1=cephfs.smithi190.byjvgf=up:active} 1 up:standby-replay 1 up:standby 2024-05-06T12:12:06.208 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:05 smithi092 ceph-mon[26954]: fsmap cephfs:2 {0=cephfs.smithi092.rosbll=up:active,1=cephfs.smithi190.byjvgf=up:active} 2 up:standby-replay 2024-05-06T12:12:06.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:05 smithi190 ceph-mon[33798]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-06T12:12:06.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:05 smithi190 ceph-mon[33798]: mds.? [v2:172.21.15.92:6828/1663253598,v1:172.21.15.92:6829/1663253598] up:active 2024-05-06T12:12:06.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:05 smithi190 ceph-mon[33798]: fsmap cephfs:2 {0=cephfs.smithi092.rosbll=up:active,1=cephfs.smithi190.byjvgf=up:active} 1 up:standby-replay 1 up:standby 2024-05-06T12:12:06.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:05 smithi190 ceph-mon[33798]: fsmap cephfs:2 {0=cephfs.smithi092.rosbll=up:active,1=cephfs.smithi190.byjvgf=up:active} 2 up:standby-replay 2024-05-06T12:12:07.212 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:06 smithi092 ceph-mon[26954]: pgmap v185: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 102 B/s wr, 5 op/s 2024-05-06T12:12:07.250 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:06 smithi190 ceph-mon[33798]: pgmap v185: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 102 B/s wr, 5 op/s 2024-05-06T12:12:07.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.433+0000 7fa954d36700 1 -- 172.21.15.92:0/2620494686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9500fce60 msgr2=0x7fa9500fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:07.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.433+0000 7fa954d36700 1 --2- 172.21.15.92:0/2620494686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9500fce60 0x7fa9500fd270 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa940004940 tx=0x7fa940009e30 comp rx=0 tx=0).stop 2024-05-06T12:12:07.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.433+0000 7fa954d36700 1 -- 172.21.15.92:0/2620494686 shutdown_connections 2024-05-06T12:12:07.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.433+0000 7fa954d36700 1 --2- 172.21.15.92:0/2620494686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9500fdfd0 0x7fa9500fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:07.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.433+0000 7fa954d36700 1 --2- 172.21.15.92:0/2620494686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9500fce60 0x7fa9500fd270 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:07.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.433+0000 7fa954d36700 1 -- 172.21.15.92:0/2620494686 >> 172.21.15.92:0/2620494686 conn(0x7fa9500f8540 msgr2=0x7fa9500fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:07.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.433+0000 7fa954d36700 1 -- 172.21.15.92:0/2620494686 shutdown_connections 2024-05-06T12:12:07.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.433+0000 7fa954d36700 1 -- 172.21.15.92:0/2620494686 wait complete. 2024-05-06T12:12:07.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.433+0000 7fa954d36700 1 Processor -- start 2024-05-06T12:12:07.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.433+0000 7fa954d36700 1 -- start start 2024-05-06T12:12:07.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.433+0000 7fa954d36700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9500fce60 0x7fa95010d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:07.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.433+0000 7fa954d36700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9500fdfd0 0x7fa95010ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:07.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.433+0000 7fa954d36700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95010e3f0 con 0x7fa9500fce60 2024-05-06T12:12:07.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.433+0000 7fa954d36700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95010e560 con 0x7fa9500fdfd0 2024-05-06T12:12:07.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.433+0000 7fa94e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9500fce60 0x7fa95010d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:07.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.433+0000 7fa94e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9500fce60 0x7fa95010d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49028/0 (socket says 172.21.15.92:49028) 2024-05-06T12:12:07.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.433+0000 7fa94e59c700 1 -- 172.21.15.92:0/4229145230 learned_addr learned my addr 172.21.15.92:0/4229145230 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:12:07.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.434+0000 7fa947fff700 1 --2- 172.21.15.92:0/4229145230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9500fdfd0 0x7fa95010ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:07.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.434+0000 7fa94e59c700 1 -- 172.21.15.92:0/4229145230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9500fdfd0 msgr2=0x7fa95010ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:07.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.434+0000 7fa94e59c700 1 --2- 172.21.15.92:0/4229145230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9500fdfd0 0x7fa95010ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:07.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.434+0000 7fa94e59c700 1 -- 172.21.15.92:0/4229145230 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa940005040 con 0x7fa9500fce60 2024-05-06T12:12:07.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.434+0000 7fa947fff700 1 --2- 172.21.15.92:0/4229145230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9500fdfd0 0x7fa95010ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-06T12:12:07.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.434+0000 7fa94e59c700 1 --2- 172.21.15.92:0/4229145230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9500fce60 0x7fa95010d8b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa94000b3b0 tx=0x7fa940009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:07.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.434+0000 7fa946ffd700 1 -- 172.21.15.92:0/4229145230 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa94000e040 con 0x7fa9500fce60 2024-05-06T12:12:07.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.434+0000 7fa954d36700 1 -- 172.21.15.92:0/4229145230 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa950112ef0 con 0x7fa9500fce60 2024-05-06T12:12:07.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.434+0000 7fa946ffd700 1 -- 172.21.15.92:0/4229145230 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa94000bcc0 con 0x7fa9500fce60 2024-05-06T12:12:07.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.434+0000 7fa946ffd700 1 -- 172.21.15.92:0/4229145230 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa94001c630 con 0x7fa9500fce60 2024-05-06T12:12:07.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.434+0000 7fa954d36700 1 -- 172.21.15.92:0/4229145230 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa950113420 con 0x7fa9500fce60 2024-05-06T12:12:07.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.435+0000 7fa954d36700 1 -- 172.21.15.92:0/4229145230 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa95004c710 con 0x7fa9500fce60 2024-05-06T12:12:07.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.435+0000 7fa946ffd700 1 -- 172.21.15.92:0/4229145230 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fa940013980 con 0x7fa9500fce60 2024-05-06T12:12:07.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.436+0000 7fa946ffd700 1 --2- 172.21.15.92:0/4229145230 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fa93c0692a0 0x7fa93c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:07.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.436+0000 7fa947fff700 1 --2- 172.21.15.92:0/4229145230 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fa93c0692a0 0x7fa93c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:07.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.436+0000 7fa946ffd700 1 -- 172.21.15.92:0/4229145230 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fa940088000 con 0x7fa9500fce60 2024-05-06T12:12:07.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.436+0000 7fa947fff700 1 --2- 172.21.15.92:0/4229145230 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fa93c0692a0 0x7fa93c06b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa938007fd0 tx=0x7fa938009f60 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:07.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.440+0000 7fa946ffd700 1 -- 172.21.15.92:0/4229145230 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa940057cf0 con 0x7fa9500fce60 2024-05-06T12:12:07.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.708+0000 7fa954d36700 1 -- 172.21.15.92:0/4229145230 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9501058c0 con 0x7fa9500fce60 2024-05-06T12:12:07.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.709+0000 7fa946ffd700 1 -- 172.21.15.92:0/4229145230 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4725 (secure 0 0 0) 0x7fa940057880 con 0x7fa9500fce60 2024-05-06T12:12:07.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.711+0000 7fa954d36700 1 -- 172.21.15.92:0/4229145230 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fa93c0692a0 msgr2=0x7fa93c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:07.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.711+0000 7fa954d36700 1 --2- 172.21.15.92:0/4229145230 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fa93c0692a0 0x7fa93c06b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa938007fd0 tx=0x7fa938009f60 comp rx=0 tx=0).stop 2024-05-06T12:12:07.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.711+0000 7fa954d36700 1 -- 172.21.15.92:0/4229145230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9500fce60 msgr2=0x7fa95010d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:07.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.711+0000 7fa954d36700 1 --2- 172.21.15.92:0/4229145230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9500fce60 0x7fa95010d8b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa94000b3b0 tx=0x7fa940009e30 comp rx=0 tx=0).stop 2024-05-06T12:12:07.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.711+0000 7fa954d36700 1 -- 172.21.15.92:0/4229145230 shutdown_connections 2024-05-06T12:12:07.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.711+0000 7fa954d36700 1 --2- 172.21.15.92:0/4229145230 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fa93c0692a0 0x7fa93c06b750 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:07.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.711+0000 7fa954d36700 1 --2- 172.21.15.92:0/4229145230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9500fce60 0x7fa95010d8b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:07.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.711+0000 7fa954d36700 1 --2- 172.21.15.92:0/4229145230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9500fdfd0 0x7fa95010ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:07.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.711+0000 7fa954d36700 1 -- 172.21.15.92:0/4229145230 >> 172.21.15.92:0/4229145230 conn(0x7fa9500f8540 msgr2=0x7fa9500f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:07.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.711+0000 7fa954d36700 1 -- 172.21.15.92:0/4229145230 shutdown_connections 2024-05-06T12:12:07.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:07.711+0000 7fa954d36700 1 -- 172.21.15.92:0/4229145230 wait complete. 2024-05-06T12:12:07.719 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 18 2024-05-06T12:12:07.728 INFO:teuthology.orchestra.run.smithi092.stdout:true 2024-05-06T12:12:08.399 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-05-06T12:12:08.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:08 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:08.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:08 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:08.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:08 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:12:08.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:08 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:12:08.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:08 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:08.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:08 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:12:08.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:08 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/4229145230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-06T12:12:08.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:08 smithi092 ceph-mon[26954]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.5 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-06T12:12:08.963 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:08 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:08.963 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:08 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:08.963 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:08 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:12:08.963 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:08 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:12:08.963 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:08 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:08.964 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:08 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:12:08.964 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:08 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/4229145230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-06T12:12:08.964 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:08 smithi190 ceph-mon[33798]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.5 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-06T12:12:09.217 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:12:10.915 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:10 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:10.915 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:10 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:10.915 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:10 smithi092 ceph-mon[26954]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 7.7 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-06T12:12:11.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.010+0000 7f6774e70700 1 -- 172.21.15.92:0/2148118301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67700fed70 msgr2=0x7f67700faeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:11.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.010+0000 7f6774e70700 1 --2- 172.21.15.92:0/2148118301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67700fed70 0x7f67700faeb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6758004940 tx=0x7f6758009e30 comp rx=0 tx=0).stop 2024-05-06T12:12:11.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.010+0000 7f6774e70700 1 -- 172.21.15.92:0/2148118301 shutdown_connections 2024-05-06T12:12:11.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.010+0000 7f6774e70700 1 --2- 172.21.15.92:0/2148118301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67700fed70 0x7f67700faeb0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:11.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.010+0000 7f6774e70700 1 --2- 172.21.15.92:0/2148118301 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67700fe420 0x7f67700fe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:11.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.010+0000 7f6774e70700 1 -- 172.21.15.92:0/2148118301 >> 172.21.15.92:0/2148118301 conn(0x7f67700f6be0 msgr2=0x7f67700f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:11.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.010+0000 7f6774e70700 1 -- 172.21.15.92:0/2148118301 shutdown_connections 2024-05-06T12:12:11.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.010+0000 7f6774e70700 1 -- 172.21.15.92:0/2148118301 wait complete. 2024-05-06T12:12:11.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.010+0000 7f6774e70700 1 Processor -- start 2024-05-06T12:12:11.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.010+0000 7f6774e70700 1 -- start start 2024-05-06T12:12:11.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.010+0000 7f6774e70700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67700fe420 0x7f677010d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:11.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.010+0000 7f6774e70700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67700fed70 0x7f677010de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:11.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.010+0000 7f6774e70700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677010e430 con 0x7f67700fe420 2024-05-06T12:12:11.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.010+0000 7f6774e70700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677010e5a0 con 0x7f67700fed70 2024-05-06T12:12:11.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.010+0000 7f676e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67700fe420 0x7f677010d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:11.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.010+0000 7f676e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67700fe420 0x7f677010d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49044/0 (socket says 172.21.15.92:49044) 2024-05-06T12:12:11.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.010+0000 7f676e59c700 1 -- 172.21.15.92:0/776697295 learned_addr learned my addr 172.21.15.92:0/776697295 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:12:11.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.011+0000 7f6767fff700 1 --2- 172.21.15.92:0/776697295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67700fed70 0x7f677010de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:11.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.011+0000 7f676e59c700 1 -- 172.21.15.92:0/776697295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67700fed70 msgr2=0x7f677010de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:11.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.011+0000 7f676e59c700 1 --2- 172.21.15.92:0/776697295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67700fed70 0x7f677010de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:11.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.011+0000 7f676e59c700 1 -- 172.21.15.92:0/776697295 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6758005040 con 0x7f67700fe420 2024-05-06T12:12:11.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.011+0000 7f676e59c700 1 --2- 172.21.15.92:0/776697295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67700fe420 0x7f677010d8f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6760002a50 tx=0x7f67600046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:11.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.011+0000 7f6766ffd700 1 -- 172.21.15.92:0/776697295 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6760007cc0 con 0x7f67700fe420 2024-05-06T12:12:11.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.011+0000 7f6774e70700 1 -- 172.21.15.92:0/776697295 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6770112fa0 con 0x7f67700fe420 2024-05-06T12:12:11.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.011+0000 7f6766ffd700 1 -- 172.21.15.92:0/776697295 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f676000f040 con 0x7f67700fe420 2024-05-06T12:12:11.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.011+0000 7f6766ffd700 1 -- 172.21.15.92:0/776697295 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6760009640 con 0x7f67700fe420 2024-05-06T12:12:11.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.011+0000 7f6774e70700 1 -- 172.21.15.92:0/776697295 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6770113530 con 0x7f67700fe420 2024-05-06T12:12:11.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.012+0000 7f6774e70700 1 -- 172.21.15.92:0/776697295 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f677004c710 con 0x7f67700fe420 2024-05-06T12:12:11.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.013+0000 7f6766ffd700 1 -- 172.21.15.92:0/776697295 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f67600097a0 con 0x7f67700fe420 2024-05-06T12:12:11.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.013+0000 7f6766ffd700 1 --2- 172.21.15.92:0/776697295 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f675c0692a0 0x7f675c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:11.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.013+0000 7f6766ffd700 1 -- 172.21.15.92:0/776697295 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f6760087020 con 0x7f67700fe420 2024-05-06T12:12:11.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.013+0000 7f6767fff700 1 --2- 172.21.15.92:0/776697295 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f675c0692a0 0x7f675c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:11.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.014+0000 7f6767fff700 1 --2- 172.21.15.92:0/776697295 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f675c0692a0 0x7f675c06b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f675800b3b0 tx=0x7f6758009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:11.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.019+0000 7f6766ffd700 1 -- 172.21.15.92:0/776697295 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6760056d10 con 0x7f67700fe420 2024-05-06T12:12:11.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:10 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:11.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:10 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:11.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:10 smithi190 ceph-mon[33798]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 7.7 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-06T12:12:11.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.284+0000 7f6774e70700 1 -- 172.21.15.92:0/776697295 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7f6770113720 con 0x7f67700fe420 2024-05-06T12:12:11.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.285+0000 7f6766ffd700 1 -- 172.21.15.92:0/776697295 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v18) v1 ==== 78+0+99 (secure 0 0 0) 0x7f67600568a0 con 0x7f67700fe420 2024-05-06T12:12:11.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.287+0000 7f6774e70700 1 -- 172.21.15.92:0/776697295 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f675c0692a0 msgr2=0x7f675c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:11.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.287+0000 7f6774e70700 1 --2- 172.21.15.92:0/776697295 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f675c0692a0 0x7f675c06b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f675800b3b0 tx=0x7f6758009e30 comp rx=0 tx=0).stop 2024-05-06T12:12:11.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.287+0000 7f6774e70700 1 -- 172.21.15.92:0/776697295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67700fe420 msgr2=0x7f677010d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:11.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.287+0000 7f6774e70700 1 --2- 172.21.15.92:0/776697295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67700fe420 0x7f677010d8f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6760002a50 tx=0x7f67600046b0 comp rx=0 tx=0).stop 2024-05-06T12:12:11.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.287+0000 7f6774e70700 1 -- 172.21.15.92:0/776697295 shutdown_connections 2024-05-06T12:12:11.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.287+0000 7f6774e70700 1 --2- 172.21.15.92:0/776697295 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f675c0692a0 0x7f675c06b750 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:11.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.287+0000 7f6774e70700 1 --2- 172.21.15.92:0/776697295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67700fe420 0x7f677010d8f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:11.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.287+0000 7f6774e70700 1 --2- 172.21.15.92:0/776697295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67700fed70 0x7f677010de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:11.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.287+0000 7f6774e70700 1 -- 172.21.15.92:0/776697295 >> 172.21.15.92:0/776697295 conn(0x7f67700f6be0 msgr2=0x7f67700f8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:11.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.287+0000 7f6774e70700 1 -- 172.21.15.92:0/776697295 shutdown_connections 2024-05-06T12:12:11.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:11.287+0000 7f6774e70700 1 -- 172.21.15.92:0/776697295 wait complete. 2024-05-06T12:12:11.300 INFO:teuthology.orchestra.run.smithi092.stdout:true 2024-05-06T12:12:12.037 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:11 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/776697295' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-05-06T12:12:12.061 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-05-06T12:12:12.083 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-06T12:12:12.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:11 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/776697295' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-05-06T12:12:12.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.200+0000 7f280be6e640 1 -- 172.21.15.92:0/213207811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28040fc330 msgr2=0x7f28040fc7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:12.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.200+0000 7f280be6e640 1 --2- 172.21.15.92:0/213207811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28040fc330 0x7f28040fc7a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f27f40048d0 tx=0x7f27f4009da0 comp rx=0 tx=0).stop 2024-05-06T12:12:12.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.200+0000 7f280be6e640 1 -- 172.21.15.92:0/213207811 shutdown_connections 2024-05-06T12:12:12.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.200+0000 7f280be6e640 1 --2- 172.21.15.92:0/213207811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28040fc330 0x7f28040fc7a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:12.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.200+0000 7f280be6e640 1 --2- 172.21.15.92:0/213207811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28040fb060 0x7f28040fb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:12.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.200+0000 7f280be6e640 1 -- 172.21.15.92:0/213207811 >> 172.21.15.92:0/213207811 conn(0x7f28040f6820 msgr2=0x7f28040f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:12.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.200+0000 7f280be6e640 1 -- 172.21.15.92:0/213207811 shutdown_connections 2024-05-06T12:12:12.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.200+0000 7f280be6e640 1 -- 172.21.15.92:0/213207811 wait complete. 2024-05-06T12:12:12.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.201+0000 7f280be6e640 1 Processor -- start 2024-05-06T12:12:12.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.201+0000 7f280be6e640 1 -- start start 2024-05-06T12:12:12.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.201+0000 7f280be6e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28040fb060 0x7f280410ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:12.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.201+0000 7f280be6e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28040fc330 0x7f280410f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:12.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.201+0000 7f280be6e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280410fa10 con 0x7f28040fb060 2024-05-06T12:12:12.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.201+0000 7f280be6e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2804114410 con 0x7f28040fc330 2024-05-06T12:12:12.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.201+0000 7f2809c0a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28040fb060 0x7f280410ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:12.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.201+0000 7f2809c0a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28040fb060 0x7f280410ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49060/0 (socket says 172.21.15.92:49060) 2024-05-06T12:12:12.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.201+0000 7f2809c0a640 1 -- 172.21.15.92:0/2510622123 learned_addr learned my addr 172.21.15.92:0/2510622123 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:12:12.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.201+0000 7f2809409640 1 --2- 172.21.15.92:0/2510622123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28040fc330 0x7f280410f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:12.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.201+0000 7f2809c0a640 1 -- 172.21.15.92:0/2510622123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28040fc330 msgr2=0x7f280410f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:12.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.201+0000 7f2809c0a640 1 --2- 172.21.15.92:0/2510622123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28040fc330 0x7f280410f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:12.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.201+0000 7f2809c0a640 1 -- 172.21.15.92:0/2510622123 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f4005040 con 0x7f28040fb060 2024-05-06T12:12:12.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.201+0000 7f2809c0a640 1 --2- 172.21.15.92:0/2510622123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28040fb060 0x7f280410ee80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2800002a50 tx=0x7f2800004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:12.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.201+0000 7f27fa7fc640 1 -- 172.21.15.92:0/2510622123 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2800007c30 con 0x7f28040fb060 2024-05-06T12:12:12.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.201+0000 7f280be6e640 1 -- 172.21.15.92:0/2510622123 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28041145b0 con 0x7f28040fb060 2024-05-06T12:12:12.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.202+0000 7f27fa7fc640 1 -- 172.21.15.92:0/2510622123 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f280000f040 con 0x7f28040fb060 2024-05-06T12:12:12.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.202+0000 7f27fa7fc640 1 -- 172.21.15.92:0/2510622123 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f280000db50 con 0x7f28040fb060 2024-05-06T12:12:12.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.202+0000 7f280be6e640 1 -- 172.21.15.92:0/2510622123 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2804114aa0 con 0x7f28040fb060 2024-05-06T12:12:12.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.202+0000 7f280be6e640 1 -- 172.21.15.92:0/2510622123 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f280404bcc0 con 0x7f28040fb060 2024-05-06T12:12:12.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.203+0000 7f27fa7fc640 1 -- 172.21.15.92:0/2510622123 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f280001c400 con 0x7f28040fb060 2024-05-06T12:12:12.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.203+0000 7f27fa7fc640 1 --2- 172.21.15.92:0/2510622123 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f27f0069210 0x7f27f006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:12.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.204+0000 7f2809409640 1 --2- 172.21.15.92:0/2510622123 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f27f0069210 0x7f27f006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:12.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.204+0000 7f27fa7fc640 1 -- 172.21.15.92:0/2510622123 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f2800086ae0 con 0x7f28040fb060 2024-05-06T12:12:12.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.204+0000 7f2809409640 1 --2- 172.21.15.92:0/2510622123 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f27f0069210 0x7f27f006b6c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2804110380 tx=0x7f27f4009da0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:12.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.208+0000 7f27fa7fc640 1 -- 172.21.15.92:0/2510622123 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f280004d250 con 0x7f28040fb060 2024-05-06T12:12:12.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.379+0000 7f280be6e640 1 -- 172.21.15.92:0/2510622123 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2804103bd0 con 0x7f28040fb060 2024-05-06T12:12:12.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.380+0000 7f27fa7fc640 1 -- 172.21.15.92:0/2510622123 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+14395 (secure 0 0 0) 0x7f2800052d40 con 0x7f28040fb060 2024-05-06T12:12:12.382 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:12:12.382 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":46,"fsid":"aee1d912-0ba0-11ef-bc97-c7b262605968","created":"2024-05-06T12:04:51.220909+0000","modified":"2024-05-06T12:12:00.896541+0000","last_up_change":"2024-05-06T12:10:26.803552+0000","last_in_change":"2024-05-06T12:10:07.815689+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-06T12:09:12.129668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-05-06T12:11:43.666342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-05-06T12:11:44.183559+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1b276435-4698-4017-9535-4be41a4e1379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3575291733},{"type":"v1","addr":"172.21.15.92:6803","nonce":3575291733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3575291733},{"type":"v1","addr":"172.21.15.92:6805","nonce":3575291733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3575291733},{"type":"v1","addr":"172.21.15.92:6809","nonce":3575291733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3575291733},{"type":"v1","addr":"172.21.15.92:6807","nonce":3575291733}]},"public_addr":"172.21.15.92:6803/3575291733","cluster_addr":"172.21.15.92:6805/3575291733","heartbeat_back_addr":"172.21.15.92:6809/3575291733","heartbeat_front_addr":"172.21.15.92:6807/3575291733","state":["exists","up"]},{"osd":1,"uuid":"21688119-d15f-4b31-8842-090cc539c76d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":334092652},{"type":"v1","addr":"172.21.15.92:6811","nonce":334092652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":334092652},{"type":"v1","addr":"172.21.15.92:6813","nonce":334092652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":334092652},{"type":"v1","addr":"172.21.15.92:6817","nonce":334092652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":334092652},{"type":"v1","addr":"172.21.15.92:6815","nonce":334092652}]},"public_addr":"172.21.15.92:6811/334092652","cluster_addr":"172.21.15.92:6813/334092652","heartbeat_back_addr":"172.21.15.92:6817/334092652","heartbeat_front_addr":"172.21.15.92:6815/334092652","state":["exists","up"]},{"osd":2,"uuid":"74d16dfb-b118-432d-827a-59706d69c393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3428814547},{"type":"v1","addr":"172.21.15.92:6819","nonce":3428814547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3428814547},{"type":"v1","addr":"172.21.15.92:6821","nonce":3428814547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3428814547},{"type":"v1","addr":"172.21.15.92:6825","nonce":3428814547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3428814547},{"type":"v1","addr":"172.21.15.92:6823","nonce":3428814547}]},"public_addr":"172.21.15.92:6819/3428814547","cluster_addr":"172.21.15.92:6821/3428814547","heartbeat_back_addr":"172.21.15.92:6825/3428814547","heartbeat_front_addr":"172.21.15.92:6823/3428814547","state":["exists","up"]},{"osd":3,"uuid":"5a9aa5ed-2502-4c73-a8ac-cb7d0259500b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2610168113},{"type":"v1","addr":"172.21.15.190:6801","nonce":2610168113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2610168113},{"type":"v1","addr":"172.21.15.190:6803","nonce":2610168113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2610168113},{"type":"v1","addr":"172.21.15.190:6807","nonce":2610168113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2610168113},{"type":"v1","addr":"172.21.15.190:6805","nonce":2610168113}]},"public_addr":"172.21.15.190:6801/2610168113","cluster_addr":"172.21.15.190:6803/2610168113","heartbeat_back_addr":"172.21.15.190:6807/2610168113","heartbeat_front_addr":"172.21.15.190:6805/2610168113","state":["exists","up"]},{"osd":4,"uuid":"d4039e4a-c758-4632-85d6-1b7e725c0293","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1405678389},{"type":"v1","addr":"172.21.15.190:6809","nonce":1405678389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1405678389},{"type":"v1","addr":"172.21.15.190:6811","nonce":1405678389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1405678389},{"type":"v1","addr":"172.21.15.190:6815","nonce":1405678389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1405678389},{"type":"v1","addr":"172.21.15.190:6813","nonce":1405678389}]},"public_addr":"172.21.15.190:6809/1405678389","cluster_addr":"172.21.15.190:6811/1405678389","heartbeat_back_addr":"172.21.15.190:6815/1405678389","heartbeat_front_addr":"172.21.15.190:6813/1405678389","state":["exists","up"]},{"osd":5,"uuid":"ec707eee-5f24-48bb-818a-547f804fc41b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1027118727},{"type":"v1","addr":"172.21.15.190:6817","nonce":1027118727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1027118727},{"type":"v1","addr":"172.21.15.190:6819","nonce":1027118727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1027118727},{"type":"v1","addr":"172.21.15.190:6823","nonce":1027118727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1027118727},{"type":"v1","addr":"172.21.15.190:6821","nonce":1027118727}]},"public_addr":"172.21.15.190:6817/1027118727","cluster_addr":"172.21.15.190:6819/1027118727","heartbeat_back_addr":"172.21.15.190:6823/1027118727","heartbeat_front_addr":"172.21.15.190:6821/1027118727","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:08:15.348582+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:08:42.571577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:09:10.093700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:09:34.257829+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:09:59.795522+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:10:25.131646+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:6827/4255634245":"2024-05-07T12:12:00.896508+0000","172.21.15.92:0/1532977673":"2024-05-07T12:07:03.802107+0000","172.21.15.92:6801/3656822751":"2024-05-07T12:05:25.954232+0000","172.21.15.92:6800/3656822751":"2024-05-07T12:05:25.954232+0000","172.21.15.92:0/3693315250":"2024-05-07T12:05:56.546947+0000","172.21.15.92:0/997820112":"2024-05-07T12:05:25.954232+0000","172.21.15.92:0/2210153116":"2024-05-07T12:05:56.546947+0000","172.21.15.92:0/758782094":"2024-05-07T12:05:25.954232+0000","172.21.15.92:0/895013058":"2024-05-07T12:05:25.954232+0000","172.21.15.92:6801/3731057893":"2024-05-07T12:05:56.546947+0000","172.21.15.92:6800/3731057893":"2024-05-07T12:05:56.546947+0000","172.21.15.92:0/1552275072":"2024-05-07T12:07:03.802107+0000","172.21.15.92:0/2440233745":"2024-05-07T12:07:03.802107+0000","172.21.15.92:6800/543287293":"2024-05-07T12:07:03.802107+0000","172.21.15.92:0/1407601857":"2024-05-07T12:05:56.546947+0000","172.21.15.92:6801/543287293":"2024-05-07T12:07:03.802107+0000","172.21.15.92:6826/4255634245":"2024-05-07T12:12:00.896508+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-06T12:12:12.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.382+0000 7f280be6e640 1 -- 172.21.15.92:0/2510622123 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f27f0069210 msgr2=0x7f27f006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:12.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.382+0000 7f280be6e640 1 --2- 172.21.15.92:0/2510622123 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f27f0069210 0x7f27f006b6c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2804110380 tx=0x7f27f4009da0 comp rx=0 tx=0).stop 2024-05-06T12:12:12.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.382+0000 7f280be6e640 1 -- 172.21.15.92:0/2510622123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28040fb060 msgr2=0x7f280410ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:12.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.382+0000 7f280be6e640 1 --2- 172.21.15.92:0/2510622123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28040fb060 0x7f280410ee80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2800002a50 tx=0x7f2800004600 comp rx=0 tx=0).stop 2024-05-06T12:12:12.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.382+0000 7f280be6e640 1 -- 172.21.15.92:0/2510622123 shutdown_connections 2024-05-06T12:12:12.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.382+0000 7f280be6e640 1 --2- 172.21.15.92:0/2510622123 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f27f0069210 0x7f27f006b6c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:12.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.382+0000 7f280be6e640 1 --2- 172.21.15.92:0/2510622123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28040fc330 0x7f280410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:12.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.382+0000 7f280be6e640 1 --2- 172.21.15.92:0/2510622123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28040fb060 0x7f280410ee80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:12.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.382+0000 7f280be6e640 1 -- 172.21.15.92:0/2510622123 >> 172.21.15.92:0/2510622123 conn(0x7f28040f6820 msgr2=0x7f28040f8320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:12.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.382+0000 7f280be6e640 1 -- 172.21.15.92:0/2510622123 shutdown_connections 2024-05-06T12:12:12.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.382+0000 7f280be6e640 1 -- 172.21.15.92:0/2510622123 wait complete. 2024-05-06T12:12:12.395 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-06T12:09:12.129668+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-05-06T12:11:43.666342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-05-06T12:11:44.183559+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-06T12:12:12.395 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-06T12:12:12.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.492+0000 7f5dd346e640 1 -- 172.21.15.92:0/592648745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5dcc0fa4b0 msgr2=0x7f5dcc0fa8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:12.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.492+0000 7f5dd346e640 1 --2- 172.21.15.92:0/592648745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5dcc0fa4b0 0x7f5dcc0fa8c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5dc80048d0 tx=0x7f5dc8009da0 comp rx=0 tx=0).stop 2024-05-06T12:12:12.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.492+0000 7f5dd346e640 1 -- 172.21.15.92:0/592648745 shutdown_connections 2024-05-06T12:12:12.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.492+0000 7f5dd346e640 1 --2- 172.21.15.92:0/592648745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5dcc0fd560 0x7f5dcc0fd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:12.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.492+0000 7f5dd346e640 1 --2- 172.21.15.92:0/592648745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5dcc0fa4b0 0x7f5dcc0fa8c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:12.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.493+0000 7f5dd346e640 1 -- 172.21.15.92:0/592648745 >> 172.21.15.92:0/592648745 conn(0x7f5dcc0f5f90 msgr2=0x7f5dcc0f83c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:12.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.493+0000 7f5dd346e640 1 -- 172.21.15.92:0/592648745 shutdown_connections 2024-05-06T12:12:12.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.493+0000 7f5dd346e640 1 -- 172.21.15.92:0/592648745 wait complete. 2024-05-06T12:12:12.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.493+0000 7f5dd346e640 1 Processor -- start 2024-05-06T12:12:12.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.493+0000 7f5dd346e640 1 -- start start 2024-05-06T12:12:12.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.493+0000 7f5dd346e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5dcc0fa4b0 0x7f5dcc10cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:12.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.493+0000 7f5dd346e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5dcc0fd560 0x7f5dcc10d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:12.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.493+0000 7f5dd346e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc10d860 con 0x7f5dcc0fd560 2024-05-06T12:12:12.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.493+0000 7f5dd346e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc112220 con 0x7f5dcc0fa4b0 2024-05-06T12:12:12.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.493+0000 7f5dd0a09640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5dcc0fd560 0x7f5dcc10d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:12.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.493+0000 7f5dd0a09640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5dcc0fd560 0x7f5dcc10d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57806/0 (socket says 172.21.15.92:57806) 2024-05-06T12:12:12.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.493+0000 7f5dd0a09640 1 -- 172.21.15.92:0/3358459697 learned_addr learned my addr 172.21.15.92:0/3358459697 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:12:12.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.494+0000 7f5dd120a640 1 --2- 172.21.15.92:0/3358459697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5dcc0fa4b0 0x7f5dcc10cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:12.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.494+0000 7f5dd0a09640 1 -- 172.21.15.92:0/3358459697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5dcc0fa4b0 msgr2=0x7f5dcc10cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:12.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.494+0000 7f5dd0a09640 1 --2- 172.21.15.92:0/3358459697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5dcc0fa4b0 0x7f5dcc10cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:12.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.494+0000 7f5dd0a09640 1 -- 172.21.15.92:0/3358459697 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc8005040 con 0x7f5dcc0fd560 2024-05-06T12:12:12.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.494+0000 7f5dd0a09640 1 --2- 172.21.15.92:0/3358459697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5dcc0fd560 0x7f5dcc10d1e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00c860 tx=0x7f5dbc004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:12.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.494+0000 7f5dc1ffb640 1 -- 172.21.15.92:0/3358459697 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5dbc00aa30 con 0x7f5dcc0fd560 2024-05-06T12:12:12.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.494+0000 7f5dc1ffb640 1 -- 172.21.15.92:0/3358459697 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5dbc00ce50 con 0x7f5dcc0fd560 2024-05-06T12:12:12.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.494+0000 7f5dd346e640 1 -- 172.21.15.92:0/3358459697 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dcc1123c0 con 0x7f5dcc0fd560 2024-05-06T12:12:12.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.494+0000 7f5dc1ffb640 1 -- 172.21.15.92:0/3358459697 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5dbc0115e0 con 0x7f5dcc0fd560 2024-05-06T12:12:12.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.494+0000 7f5dd346e640 1 -- 172.21.15.92:0/3358459697 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dcc1128b0 con 0x7f5dcc0fd560 2024-05-06T12:12:12.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.495+0000 7f5dd346e640 1 -- 172.21.15.92:0/3358459697 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dcc04bcc0 con 0x7f5dcc0fd560 2024-05-06T12:12:12.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.495+0000 7f5dc1ffb640 1 -- 172.21.15.92:0/3358459697 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f5dbc00ab90 con 0x7f5dcc0fd560 2024-05-06T12:12:12.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.496+0000 7f5dc1ffb640 1 --2- 172.21.15.92:0/3358459697 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f5db8069210 0x7f5db806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:12.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.496+0000 7f5dc1ffb640 1 -- 172.21.15.92:0/3358459697 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f5dbc087430 con 0x7f5dcc0fd560 2024-05-06T12:12:12.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.496+0000 7f5dd120a640 1 --2- 172.21.15.92:0/3358459697 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f5db8069210 0x7f5db806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:12.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.496+0000 7f5dd120a640 1 --2- 172.21.15.92:0/3358459697 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f5db8069210 0x7f5db806b6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5dc80317e0 tx=0x7f5dc8009da0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:12.502 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.500+0000 7f5dc1ffb640 1 -- 172.21.15.92:0/3358459697 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5dbc053710 con 0x7f5dcc0fd560 2024-05-06T12:12:12.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.669+0000 7f5dd346e640 1 -- 172.21.15.92:0/3358459697 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5dcc112b30 con 0x7f5dcc0fd560 2024-05-06T12:12:12.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.669+0000 7f5dc1ffb640 1 -- 172.21.15.92:0/3358459697 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5dbc00ae40 con 0x7f5dcc0fd560 2024-05-06T12:12:12.671 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2024-05-06T12:12:12.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.671+0000 7f5dd346e640 1 -- 172.21.15.92:0/3358459697 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f5db8069210 msgr2=0x7f5db806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:12.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.671+0000 7f5dd346e640 1 --2- 172.21.15.92:0/3358459697 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f5db8069210 0x7f5db806b6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5dc80317e0 tx=0x7f5dc8009da0 comp rx=0 tx=0).stop 2024-05-06T12:12:12.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.671+0000 7f5dd346e640 1 -- 172.21.15.92:0/3358459697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5dcc0fd560 msgr2=0x7f5dcc10d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:12.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.671+0000 7f5dd346e640 1 --2- 172.21.15.92:0/3358459697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5dcc0fd560 0x7f5dcc10d1e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00c860 tx=0x7f5dbc004430 comp rx=0 tx=0).stop 2024-05-06T12:12:12.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.671+0000 7f5dd346e640 1 -- 172.21.15.92:0/3358459697 shutdown_connections 2024-05-06T12:12:12.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.671+0000 7f5dd346e640 1 --2- 172.21.15.92:0/3358459697 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f5db8069210 0x7f5db806b6c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:12.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.671+0000 7f5dd346e640 1 --2- 172.21.15.92:0/3358459697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5dcc0fd560 0x7f5dcc10d1e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:12.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.671+0000 7f5dd346e640 1 --2- 172.21.15.92:0/3358459697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5dcc0fa4b0 0x7f5dcc10cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:12.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.671+0000 7f5dd346e640 1 -- 172.21.15.92:0/3358459697 >> 172.21.15.92:0/3358459697 conn(0x7f5dcc0f5f90 msgr2=0x7f5dcc0f7640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:12.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.671+0000 7f5dd346e640 1 -- 172.21.15.92:0/3358459697 shutdown_connections 2024-05-06T12:12:12.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.671+0000 7f5dd346e640 1 -- 172.21.15.92:0/3358459697 wait complete. 2024-05-06T12:12:12.684 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-05-06T12:12:12.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.779+0000 7fb9fb15d640 1 -- 172.21.15.92:0/4281026887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9f40fc340 msgr2=0x7fb9f40fc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:12.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.779+0000 7fb9fb15d640 1 --2- 172.21.15.92:0/4281026887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9f40fc340 0x7fb9f40fc7b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb9e00048d0 tx=0x7fb9e0009da0 comp rx=0 tx=0).stop 2024-05-06T12:12:12.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.779+0000 7fb9fb15d640 1 -- 172.21.15.92:0/4281026887 shutdown_connections 2024-05-06T12:12:12.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.779+0000 7fb9fb15d640 1 --2- 172.21.15.92:0/4281026887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9f40fc340 0x7fb9f40fc7b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:12.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.779+0000 7fb9fb15d640 1 --2- 172.21.15.92:0/4281026887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9f40fb070 0x7fb9f40fb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:12.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.779+0000 7fb9fb15d640 1 -- 172.21.15.92:0/4281026887 >> 172.21.15.92:0/4281026887 conn(0x7fb9f40f6830 msgr2=0x7fb9f40f8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:12.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.779+0000 7fb9fb15d640 1 -- 172.21.15.92:0/4281026887 shutdown_connections 2024-05-06T12:12:12.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.779+0000 7fb9fb15d640 1 -- 172.21.15.92:0/4281026887 wait complete. 2024-05-06T12:12:12.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.779+0000 7fb9fb15d640 1 Processor -- start 2024-05-06T12:12:12.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.779+0000 7fb9fb15d640 1 -- start start 2024-05-06T12:12:12.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.780+0000 7fb9fb15d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9f40fb070 0x7fb9f410bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:12.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.780+0000 7fb9fb15d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9f40fc340 0x7fb9f410c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:12.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.780+0000 7fb9fb15d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f410c7e0 con 0x7fb9f40fb070 2024-05-06T12:12:12.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.780+0000 7fb9fb15d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f4111190 con 0x7fb9f40fc340 2024-05-06T12:12:12.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.780+0000 7fb9f8ef9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9f40fb070 0x7fb9f410bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:12.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.780+0000 7fb9f8ef9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9f40fb070 0x7fb9f410bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57818/0 (socket says 172.21.15.92:57818) 2024-05-06T12:12:12.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.780+0000 7fb9f8ef9640 1 -- 172.21.15.92:0/450909799 learned_addr learned my addr 172.21.15.92:0/450909799 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:12:12.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.780+0000 7fb9f3fff640 1 --2- 172.21.15.92:0/450909799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9f40fc340 0x7fb9f410c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:12.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.780+0000 7fb9f8ef9640 1 -- 172.21.15.92:0/450909799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9f40fc340 msgr2=0x7fb9f410c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:12.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.780+0000 7fb9f8ef9640 1 --2- 172.21.15.92:0/450909799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9f40fc340 0x7fb9f410c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:12.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.780+0000 7fb9f8ef9640 1 -- 172.21.15.92:0/450909799 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e0005040 con 0x7fb9f40fb070 2024-05-06T12:12:12.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.780+0000 7fb9f8ef9640 1 --2- 172.21.15.92:0/450909799 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9f40fb070 0x7fb9f410bc50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb9e800c790 tx=0x7fb9e80043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:12.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.780+0000 7fb9f17fa640 1 -- 172.21.15.92:0/450909799 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9e800a820 con 0x7fb9f40fb070 2024-05-06T12:12:12.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.780+0000 7fb9fb15d640 1 -- 172.21.15.92:0/450909799 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9f4111330 con 0x7fb9f40fb070 2024-05-06T12:12:12.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.780+0000 7fb9f17fa640 1 -- 172.21.15.92:0/450909799 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb9e800ae50 con 0x7fb9f40fb070 2024-05-06T12:12:12.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.781+0000 7fb9f17fa640 1 -- 172.21.15.92:0/450909799 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9e80115e0 con 0x7fb9f40fb070 2024-05-06T12:12:12.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.781+0000 7fb9fb15d640 1 -- 172.21.15.92:0/450909799 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9f4111820 con 0x7fb9f40fb070 2024-05-06T12:12:12.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.781+0000 7fb9fb15d640 1 -- 172.21.15.92:0/450909799 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9f4105570 con 0x7fb9f40fb070 2024-05-06T12:12:12.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.782+0000 7fb9f17fa640 1 -- 172.21.15.92:0/450909799 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fb9e8012400 con 0x7fb9f40fb070 2024-05-06T12:12:12.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.782+0000 7fb9f17fa640 1 --2- 172.21.15.92:0/450909799 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fb9e4069210 0x7fb9e406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:12.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.782+0000 7fb9f17fa640 1 -- 172.21.15.92:0/450909799 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fb9e80873e0 con 0x7fb9f40fb070 2024-05-06T12:12:12.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.783+0000 7fb9f3fff640 1 --2- 172.21.15.92:0/450909799 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fb9e4069210 0x7fb9e406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:12.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.783+0000 7fb9f3fff640 1 --2- 172.21.15.92:0/450909799 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fb9e4069210 0x7fb9e406b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb9f410d100 tx=0x7fb9e0009da0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:12.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.787+0000 7fb9f17fa640 1 -- 172.21.15.92:0/450909799 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb9e80536c0 con 0x7fb9f40fb070 2024-05-06T12:12:12.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.968+0000 7fb9fb15d640 1 -- 172.21.15.92:0/450909799 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fb9f4111b50 con 0x7fb9f40fb070 2024-05-06T12:12:12.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.968+0000 7fb9f17fa640 1 -- 172.21.15.92:0/450909799 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7fb9e8056ce0 con 0x7fb9f40fb070 2024-05-06T12:12:12.970 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2024-05-06T12:12:12.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:12 smithi092 ceph-mon[26954]: pgmap v188: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 7.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-06T12:12:12.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:12 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/2510622123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-06T12:12:12.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:12 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/3358459697' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-06T12:12:12.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.970+0000 7fb9fb15d640 1 -- 172.21.15.92:0/450909799 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fb9e4069210 msgr2=0x7fb9e406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:12.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.970+0000 7fb9fb15d640 1 --2- 172.21.15.92:0/450909799 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fb9e4069210 0x7fb9e406b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb9f410d100 tx=0x7fb9e0009da0 comp rx=0 tx=0).stop 2024-05-06T12:12:12.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.970+0000 7fb9fb15d640 1 -- 172.21.15.92:0/450909799 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9f40fb070 msgr2=0x7fb9f410bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:12.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.970+0000 7fb9fb15d640 1 --2- 172.21.15.92:0/450909799 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9f40fb070 0x7fb9f410bc50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb9e800c790 tx=0x7fb9e80043c0 comp rx=0 tx=0).stop 2024-05-06T12:12:12.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.970+0000 7fb9fb15d640 1 -- 172.21.15.92:0/450909799 shutdown_connections 2024-05-06T12:12:12.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.970+0000 7fb9fb15d640 1 --2- 172.21.15.92:0/450909799 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fb9e4069210 0x7fb9e406b6c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:12.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.970+0000 7fb9fb15d640 1 --2- 172.21.15.92:0/450909799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9f40fc340 0x7fb9f410c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:12.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.970+0000 7fb9fb15d640 1 --2- 172.21.15.92:0/450909799 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9f40fb070 0x7fb9f410bc50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:12.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.970+0000 7fb9fb15d640 1 -- 172.21.15.92:0/450909799 >> 172.21.15.92:0/450909799 conn(0x7fb9f40f6830 msgr2=0x7fb9f40ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:12.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.971+0000 7fb9fb15d640 1 -- 172.21.15.92:0/450909799 shutdown_connections 2024-05-06T12:12:12.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:12.971+0000 7fb9fb15d640 1 -- 172.21.15.92:0/450909799 wait complete. 2024-05-06T12:12:12.984 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-05-06T12:12:13.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.080+0000 7f52330ae640 1 -- 172.21.15.92:0/3940920441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f522c0fc340 msgr2=0x7f522c0fc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:13.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.080+0000 7f52330ae640 1 --2- 172.21.15.92:0/3940920441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f522c0fc340 0x7f522c0fc7b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f521c009c50 tx=0x7f521c004d10 comp rx=0 tx=0).stop 2024-05-06T12:12:13.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.080+0000 7f52330ae640 1 -- 172.21.15.92:0/3940920441 shutdown_connections 2024-05-06T12:12:13.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.080+0000 7f52330ae640 1 --2- 172.21.15.92:0/3940920441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f522c0fc340 0x7f522c0fc7b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:13.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.080+0000 7f52330ae640 1 --2- 172.21.15.92:0/3940920441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f522c0fb070 0x7f522c0fb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:13.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.080+0000 7f52330ae640 1 -- 172.21.15.92:0/3940920441 >> 172.21.15.92:0/3940920441 conn(0x7f522c0f6830 msgr2=0x7f522c0f8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:13.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.080+0000 7f52330ae640 1 -- 172.21.15.92:0/3940920441 shutdown_connections 2024-05-06T12:12:13.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.080+0000 7f52330ae640 1 -- 172.21.15.92:0/3940920441 wait complete. 2024-05-06T12:12:13.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.081+0000 7f52330ae640 1 Processor -- start 2024-05-06T12:12:13.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.081+0000 7f52330ae640 1 -- start start 2024-05-06T12:12:13.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.081+0000 7f52330ae640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f522c0fb070 0x7f522c065ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:13.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.081+0000 7f52330ae640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f522c0fc340 0x7f522c111b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:13.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.081+0000 7f52330ae640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522c1120c0 con 0x7f522c0fc340 2024-05-06T12:12:13.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.081+0000 7f52330ae640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522c112200 con 0x7f522c0fb070 2024-05-06T12:12:13.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.081+0000 7f522bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f522c0fc340 0x7f522c111b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:13.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.081+0000 7f522bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f522c0fc340 0x7f522c111b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57828/0 (socket says 172.21.15.92:57828) 2024-05-06T12:12:13.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.081+0000 7f522bfff640 1 -- 172.21.15.92:0/2647240901 learned_addr learned my addr 172.21.15.92:0/2647240901 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:12:13.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.081+0000 7f5230e4a640 1 --2- 172.21.15.92:0/2647240901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f522c0fb070 0x7f522c065ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:13.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.081+0000 7f522bfff640 1 -- 172.21.15.92:0/2647240901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f522c0fb070 msgr2=0x7f522c065ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:13.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.081+0000 7f522bfff640 1 --2- 172.21.15.92:0/2647240901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f522c0fb070 0x7f522c065ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:13.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.081+0000 7f522bfff640 1 -- 172.21.15.92:0/2647240901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521c0098b0 con 0x7f522c0fc340 2024-05-06T12:12:13.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.081+0000 7f522bfff640 1 --2- 172.21.15.92:0/2647240901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f522c0fc340 0x7f522c111b80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f521c004a30 tx=0x7f521c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:13.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.082+0000 7f52297fa640 1 -- 172.21.15.92:0/2647240901 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f521c03a070 con 0x7f522c0fc340 2024-05-06T12:12:13.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.082+0000 7f52330ae640 1 -- 172.21.15.92:0/2647240901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f522c112400 con 0x7f522c0fc340 2024-05-06T12:12:13.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.082+0000 7f52297fa640 1 -- 172.21.15.92:0/2647240901 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f521c009740 con 0x7f522c0fc340 2024-05-06T12:12:13.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.082+0000 7f52297fa640 1 -- 172.21.15.92:0/2647240901 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f521c03ea90 con 0x7f522c0fc340 2024-05-06T12:12:13.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.082+0000 7f52330ae640 1 -- 172.21.15.92:0/2647240901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f522c112920 con 0x7f522c0fc340 2024-05-06T12:12:13.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.083+0000 7f52330ae640 1 -- 172.21.15.92:0/2647240901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f522c105570 con 0x7f522c0fc340 2024-05-06T12:12:13.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.083+0000 7f52297fa640 1 -- 172.21.15.92:0/2647240901 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f521c032070 con 0x7f522c0fc340 2024-05-06T12:12:13.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.083+0000 7f52297fa640 1 --2- 172.21.15.92:0/2647240901 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f5218069210 0x7f521806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:13.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.084+0000 7f5230e4a640 1 --2- 172.21.15.92:0/2647240901 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f5218069210 0x7f521806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:13.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.084+0000 7f52297fa640 1 -- 172.21.15.92:0/2647240901 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f521c0a96f0 con 0x7f522c0fc340 2024-05-06T12:12:13.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.084+0000 7f5230e4a640 1 --2- 172.21.15.92:0/2647240901 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f5218069210 0x7f521806b6c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f52240049b0 tx=0x7f522400b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:13.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.088+0000 7f52297fa640 1 -- 172.21.15.92:0/2647240901 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f521c075a00 con 0x7f522c0fc340 2024-05-06T12:12:13.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:12 smithi190 ceph-mon[33798]: pgmap v188: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 7.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-06T12:12:13.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:12 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/2510622123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-06T12:12:13.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:12 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/3358459697' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-06T12:12:13.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.267+0000 7f52330ae640 1 -- 172.21.15.92:0/2647240901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f522c103be0 con 0x7f522c0fc340 2024-05-06T12:12:13.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.268+0000 7f52297fa640 1 -- 172.21.15.92:0/2647240901 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7f521c079020 con 0x7f522c0fc340 2024-05-06T12:12:13.271 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2024-05-06T12:12:13.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.270+0000 7f52330ae640 1 -- 172.21.15.92:0/2647240901 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f5218069210 msgr2=0x7f521806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:13.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.270+0000 7f52330ae640 1 --2- 172.21.15.92:0/2647240901 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f5218069210 0x7f521806b6c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f52240049b0 tx=0x7f522400b040 comp rx=0 tx=0).stop 2024-05-06T12:12:13.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.270+0000 7f52330ae640 1 -- 172.21.15.92:0/2647240901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f522c0fc340 msgr2=0x7f522c111b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:13.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.270+0000 7f52330ae640 1 --2- 172.21.15.92:0/2647240901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f522c0fc340 0x7f522c111b80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f521c004a30 tx=0x7f521c004d10 comp rx=0 tx=0).stop 2024-05-06T12:12:13.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.270+0000 7f52330ae640 1 -- 172.21.15.92:0/2647240901 shutdown_connections 2024-05-06T12:12:13.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.270+0000 7f52330ae640 1 --2- 172.21.15.92:0/2647240901 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f5218069210 0x7f521806b6c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:13.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.270+0000 7f52330ae640 1 --2- 172.21.15.92:0/2647240901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f522c0fc340 0x7f522c111b80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:13.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.270+0000 7f52330ae640 1 --2- 172.21.15.92:0/2647240901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f522c0fb070 0x7f522c065ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:13.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.270+0000 7f52330ae640 1 -- 172.21.15.92:0/2647240901 >> 172.21.15.92:0/2647240901 conn(0x7f522c0f6830 msgr2=0x7f522c0ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:13.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.270+0000 7f52330ae640 1 -- 172.21.15.92:0/2647240901 shutdown_connections 2024-05-06T12:12:13.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.270+0000 7f52330ae640 1 -- 172.21.15.92:0/2647240901 wait complete. 2024-05-06T12:12:13.285 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-06T12:12:13.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.378+0000 7f8f2302e640 1 -- 172.21.15.92:0/3249934398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f1c066150 msgr2=0x7f8f1c0fee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:13.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.378+0000 7f8f2302e640 1 --2- 172.21.15.92:0/3249934398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f1c066150 0x7f8f1c0fee00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8f14009c50 tx=0x7f8f14004d10 comp rx=0 tx=0).stop 2024-05-06T12:12:13.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.378+0000 7f8f2302e640 1 -- 172.21.15.92:0/3249934398 shutdown_connections 2024-05-06T12:12:13.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.378+0000 7f8f2302e640 1 --2- 172.21.15.92:0/3249934398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f1c0ff340 0x7f8f1c101720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:13.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.378+0000 7f8f2302e640 1 --2- 172.21.15.92:0/3249934398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f1c066150 0x7f8f1c0fee00 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:13.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.378+0000 7f8f2302e640 1 -- 172.21.15.92:0/3249934398 >> 172.21.15.92:0/3249934398 conn(0x7f8f1c0f6870 msgr2=0x7f8f1c0f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:13.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.378+0000 7f8f2302e640 1 -- 172.21.15.92:0/3249934398 shutdown_connections 2024-05-06T12:12:13.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.378+0000 7f8f2302e640 1 -- 172.21.15.92:0/3249934398 wait complete. 2024-05-06T12:12:13.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.378+0000 7f8f2302e640 1 Processor -- start 2024-05-06T12:12:13.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.378+0000 7f8f2302e640 1 -- start start 2024-05-06T12:12:13.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.379+0000 7f8f2302e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f1c066150 0x7f8f1c10cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:13.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.379+0000 7f8f2302e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f1c0ff340 0x7f8f1c10d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:13.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.379+0000 7f8f2302e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c10d8e0 con 0x7f8f1c0ff340 2024-05-06T12:12:13.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.379+0000 7f8f2302e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c1122a0 con 0x7f8f1c066150 2024-05-06T12:12:13.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.379+0000 7f8f1bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f1c0ff340 0x7f8f1c10d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:13.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.379+0000 7f8f1bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f1c0ff340 0x7f8f1c10d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57858/0 (socket says 172.21.15.92:57858) 2024-05-06T12:12:13.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.379+0000 7f8f1bfff640 1 -- 172.21.15.92:0/1292469071 learned_addr learned my addr 172.21.15.92:0/1292469071 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:12:13.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.379+0000 7f8f1bfff640 1 -- 172.21.15.92:0/1292469071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f1c066150 msgr2=0x7f8f1c10cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:13.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.379+0000 7f8f1bfff640 1 --2- 172.21.15.92:0/1292469071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f1c066150 0x7f8f1c10cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:13.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.379+0000 7f8f1bfff640 1 -- 172.21.15.92:0/1292469071 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f140098b0 con 0x7f8f1c0ff340 2024-05-06T12:12:13.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.379+0000 7f8f1bfff640 1 --2- 172.21.15.92:0/1292469071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f1c0ff340 0x7f8f1c10d290 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8f1000c810 tx=0x7f8f100043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:13.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.379+0000 7f8f197fa640 1 -- 172.21.15.92:0/1292469071 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f1000a820 con 0x7f8f1c0ff340 2024-05-06T12:12:13.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.379+0000 7f8f197fa640 1 -- 172.21.15.92:0/1292469071 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f1000ae50 con 0x7f8f1c0ff340 2024-05-06T12:12:13.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.379+0000 7f8f2302e640 1 -- 172.21.15.92:0/1292469071 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f1c112440 con 0x7f8f1c0ff340 2024-05-06T12:12:13.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.379+0000 7f8f197fa640 1 -- 172.21.15.92:0/1292469071 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f100115c0 con 0x7f8f1c0ff340 2024-05-06T12:12:13.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.379+0000 7f8f2302e640 1 -- 172.21.15.92:0/1292469071 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f1c112990 con 0x7f8f1c0ff340 2024-05-06T12:12:13.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.380+0000 7f8f2302e640 1 -- 172.21.15.92:0/1292469071 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f1c04bcc0 con 0x7f8f1c0ff340 2024-05-06T12:12:13.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.381+0000 7f8f197fa640 1 -- 172.21.15.92:0/1292469071 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f8f1000a980 con 0x7f8f1c0ff340 2024-05-06T12:12:13.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.381+0000 7f8f197fa640 1 --2- 172.21.15.92:0/1292469071 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f8f04069210 0x7f8f0406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:13.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.381+0000 7f8f20dca640 1 --2- 172.21.15.92:0/1292469071 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f8f04069210 0x7f8f0406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:13.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.381+0000 7f8f197fa640 1 -- 172.21.15.92:0/1292469071 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f8f100877b0 con 0x7f8f1c0ff340 2024-05-06T12:12:13.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.382+0000 7f8f20dca640 1 --2- 172.21.15.92:0/1292469071 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f8f04069210 0x7f8f0406b6c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8f14004a30 tx=0x7f8f14004d10 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:13.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.386+0000 7f8f197fa640 1 -- 172.21.15.92:0/1292469071 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8f10053a10 con 0x7f8f1c0ff340 2024-05-06T12:12:13.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.597+0000 7f8f2302e640 1 -- 172.21.15.92:0/1292469071 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f1c104cf0 con 0x7f8f1c0ff340 2024-05-06T12:12:13.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.597+0000 7f8f197fa640 1 -- 172.21.15.92:0/1292469071 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4725 (secure 0 0 0) 0x7f8f10057030 con 0x7f8f1c0ff340 2024-05-06T12:12:13.600 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:12:13.600 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-06T12:11:44.952671+0000","modified":"2024-05-06T12:12:04.907000+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14472,"mds_1":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14472":{"gid":14472,"name":"cephfs.smithi092.rosbll","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6829/1663253598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1663253598},{"type":"v1","addr":"172.21.15.92:6829","nonce":1663253598}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14484":{"gid":14484,"name":"cephfs.smithi092.ijsqgp","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6827/3613441054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3613441054},{"type":"v1","addr":"172.21.15.92:6827","nonce":3613441054}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24267":{"gid":24267,"name":"cephfs.smithi190.byjvgf","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.190:6825/1565151699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1565151699},{"type":"v1","addr":"172.21.15.190:6825","nonce":1565151699}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24277":{"gid":24277,"name":"cephfs.smithi190.idzoff","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.190:6827/2453728380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2453728380},{"type":"v1","addr":"172.21.15.190:6827","nonce":2453728380}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-05-06T12:12:13.600 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 18 2024-05-06T12:12:13.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.599+0000 7f8f2302e640 1 -- 172.21.15.92:0/1292469071 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f8f04069210 msgr2=0x7f8f0406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:13.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.599+0000 7f8f2302e640 1 --2- 172.21.15.92:0/1292469071 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f8f04069210 0x7f8f0406b6c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8f14004a30 tx=0x7f8f14004d10 comp rx=0 tx=0).stop 2024-05-06T12:12:13.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.599+0000 7f8f2302e640 1 -- 172.21.15.92:0/1292469071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f1c0ff340 msgr2=0x7f8f1c10d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:13.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.599+0000 7f8f2302e640 1 --2- 172.21.15.92:0/1292469071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f1c0ff340 0x7f8f1c10d290 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8f1000c810 tx=0x7f8f100043c0 comp rx=0 tx=0).stop 2024-05-06T12:12:13.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.599+0000 7f8f2302e640 1 -- 172.21.15.92:0/1292469071 shutdown_connections 2024-05-06T12:12:13.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.599+0000 7f8f2302e640 1 --2- 172.21.15.92:0/1292469071 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f8f04069210 0x7f8f0406b6c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:13.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.599+0000 7f8f2302e640 1 --2- 172.21.15.92:0/1292469071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f1c0ff340 0x7f8f1c10d290 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:13.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.599+0000 7f8f2302e640 1 --2- 172.21.15.92:0/1292469071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f1c066150 0x7f8f1c10cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:13.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.599+0000 7f8f2302e640 1 -- 172.21.15.92:0/1292469071 >> 172.21.15.92:0/1292469071 conn(0x7f8f1c0f6870 msgr2=0x7f8f1c0f6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:13.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.599+0000 7f8f2302e640 1 -- 172.21.15.92:0/1292469071 shutdown_connections 2024-05-06T12:12:13.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:13.599+0000 7f8f2302e640 1 -- 172.21.15.92:0/1292469071 wait complete. 2024-05-06T12:12:13.613 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 18, 'max_mds': 2, 'flags': 50} 2024-05-06T12:12:13.614 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-06T12:12:13.723 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-06T12:12:13.723 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-05-06T12:12:13.723 INFO:tasks.ceph_fuse:client.0 config is {} 2024-05-06T12:12:13.723 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-06T12:12:13.724 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-06T12:12:13.724 INFO:tasks.ceph_fuse:client.1 config is {} 2024-05-06T12:12:13.724 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-06T12:12:13.724 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-06T12:12:13.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-06T12:12:13.724 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2024-05-06T12:12:13.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-06T12:12:13.744 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link delete ceph-brx 2024-05-06T12:12:13.817 INFO:teuthology.orchestra.run.smithi190.stderr:Cannot find device "ceph-brx" 2024-05-06T12:12:13.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-06T12:12:13.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-06T12:12:13.819 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2024-05-06T12:12:13.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-06T12:12:13.834 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link delete ceph-brx 2024-05-06T12:12:13.895 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:13 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/450909799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-06T12:12:13.896 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:13 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/2647240901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-06T12:12:13.896 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:13 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/1292469071' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-06T12:12:13.901 INFO:teuthology.orchestra.run.smithi092.stderr:Cannot find device "ceph-brx" 2024-05-06T12:12:13.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-06T12:12:13.903 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-06T12:12:13.903 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-06T12:12:14.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:13 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/450909799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-06T12:12:14.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:13 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/2647240901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-06T12:12:14.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:13 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/1292469071' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-06T12:12:14.327 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:12:14.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:14 smithi092 ceph-mon[26954]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.3 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-06T12:12:15.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:14 smithi190 ceph-mon[33798]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.3 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-06T12:12:16.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.013+0000 7f124e0a6700 1 -- 172.21.15.92:0/734268161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12480fb5a0 msgr2=0x7f12480fb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:16.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.013+0000 7f124e0a6700 1 --2- 172.21.15.92:0/734268161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12480fb5a0 0x7f12480fb970 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1238004940 tx=0x7f1238009e30 comp rx=0 tx=0).stop 2024-05-06T12:12:16.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.013+0000 7f124e0a6700 1 -- 172.21.15.92:0/734268161 shutdown_connections 2024-05-06T12:12:16.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.013+0000 7f124e0a6700 1 --2- 172.21.15.92:0/734268161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12480fbf40 0x7f1248101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:16.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.013+0000 7f124e0a6700 1 --2- 172.21.15.92:0/734268161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12480fb5a0 0x7f12480fb970 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:16.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.013+0000 7f124e0a6700 1 -- 172.21.15.92:0/734268161 >> 172.21.15.92:0/734268161 conn(0x7f1248073350 msgr2=0x7f1248073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:16.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.013+0000 7f124e0a6700 1 -- 172.21.15.92:0/734268161 shutdown_connections 2024-05-06T12:12:16.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.013+0000 7f124e0a6700 1 -- 172.21.15.92:0/734268161 wait complete. 2024-05-06T12:12:16.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.013+0000 7f124e0a6700 1 Processor -- start 2024-05-06T12:12:16.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.014+0000 7f124e0a6700 1 -- start start 2024-05-06T12:12:16.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.014+0000 7f124e0a6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12480fb5a0 0x7f12481139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:16.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.014+0000 7f124e0a6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12480fbf40 0x7f1248113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:16.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.014+0000 7f124e0a6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12481145c0 con 0x7f12480fbf40 2024-05-06T12:12:16.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.014+0000 7f124e0a6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124810db50 con 0x7f12480fb5a0 2024-05-06T12:12:16.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.014+0000 7f1246ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12480fbf40 0x7f1248113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:16.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.014+0000 7f1246ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12480fbf40 0x7f1248113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57880/0 (socket says 172.21.15.92:57880) 2024-05-06T12:12:16.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.014+0000 7f1246ffd700 1 -- 172.21.15.92:0/1989242518 learned_addr learned my addr 172.21.15.92:0/1989242518 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:12:16.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.014+0000 7f12477fe700 1 --2- 172.21.15.92:0/1989242518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12480fb5a0 0x7f12481139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:16.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.014+0000 7f1246ffd700 1 -- 172.21.15.92:0/1989242518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12480fb5a0 msgr2=0x7f12481139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:16.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.014+0000 7f1246ffd700 1 --2- 172.21.15.92:0/1989242518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12480fb5a0 0x7f12481139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:16.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.014+0000 7f1246ffd700 1 -- 172.21.15.92:0/1989242518 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1238005040 con 0x7f12480fbf40 2024-05-06T12:12:16.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.014+0000 7f12477fe700 1 --2- 172.21.15.92:0/1989242518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12480fb5a0 0x7f12481139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-06T12:12:16.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.014+0000 7f1246ffd700 1 --2- 172.21.15.92:0/1989242518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12480fbf40 0x7f1248113ee0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f123c00b760 tx=0x7f123c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:16.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.015+0000 7f122ffff700 1 -- 172.21.15.92:0/1989242518 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f123c00f6f0 con 0x7f12480fbf40 2024-05-06T12:12:16.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.015+0000 7f122ffff700 1 -- 172.21.15.92:0/1989242518 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f123c00fd20 con 0x7f12480fbf40 2024-05-06T12:12:16.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.015+0000 7f124e0a6700 1 -- 172.21.15.92:0/1989242518 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f124810de30 con 0x7f12480fbf40 2024-05-06T12:12:16.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.015+0000 7f122ffff700 1 -- 172.21.15.92:0/1989242518 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f123c0103a0 con 0x7f12480fbf40 2024-05-06T12:12:16.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.015+0000 7f124e0a6700 1 -- 172.21.15.92:0/1989242518 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f124810e370 con 0x7f12480fbf40 2024-05-06T12:12:16.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.016+0000 7f122ffff700 1 -- 172.21.15.92:0/1989242518 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f123c00f850 con 0x7f12480fbf40 2024-05-06T12:12:16.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.016+0000 7f124e0a6700 1 -- 172.21.15.92:0/1989242518 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12480fe670 con 0x7f12480fbf40 2024-05-06T12:12:16.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.016+0000 7f122ffff700 1 --2- 172.21.15.92:0/1989242518 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f12300692a0 0x7f123006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:16.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.017+0000 7f122ffff700 1 -- 172.21.15.92:0/1989242518 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f123c086f70 con 0x7f12480fbf40 2024-05-06T12:12:16.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.017+0000 7f12477fe700 1 --2- 172.21.15.92:0/1989242518 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f12300692a0 0x7f123006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:16.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.017+0000 7f12477fe700 1 --2- 172.21.15.92:0/1989242518 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f12300692a0 0x7f123006b750 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1238004f90 tx=0x7f1238009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:16.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.021+0000 7f122ffff700 1 -- 172.21.15.92:0/1989242518 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f123c0532f0 con 0x7f12480fbf40 2024-05-06T12:12:16.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.248+0000 7f124e0a6700 1 -- 172.21.15.92:0/1989242518 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f124804c710 con 0x7f12480fbf40 2024-05-06T12:12:16.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.249+0000 7f122ffff700 1 -- 172.21.15.92:0/1989242518 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+83 (secure 0 0 0) 0x7f123c056910 con 0x7f12480fbf40 2024-05-06T12:12:16.251 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-05-06T12:12:16.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.250+0000 7f124e0a6700 1 -- 172.21.15.92:0/1989242518 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f12300692a0 msgr2=0x7f123006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:16.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.250+0000 7f124e0a6700 1 --2- 172.21.15.92:0/1989242518 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f12300692a0 0x7f123006b750 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1238004f90 tx=0x7f1238009e30 comp rx=0 tx=0).stop 2024-05-06T12:12:16.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.250+0000 7f124e0a6700 1 -- 172.21.15.92:0/1989242518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12480fbf40 msgr2=0x7f1248113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:16.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.250+0000 7f124e0a6700 1 --2- 172.21.15.92:0/1989242518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12480fbf40 0x7f1248113ee0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f123c00b760 tx=0x7f123c009220 comp rx=0 tx=0).stop 2024-05-06T12:12:16.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.250+0000 7f124e0a6700 1 -- 172.21.15.92:0/1989242518 shutdown_connections 2024-05-06T12:12:16.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.250+0000 7f124e0a6700 1 --2- 172.21.15.92:0/1989242518 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f12300692a0 0x7f123006b750 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:16.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.251+0000 7f124e0a6700 1 --2- 172.21.15.92:0/1989242518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12480fb5a0 0x7f12481139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:16.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.251+0000 7f124e0a6700 1 --2- 172.21.15.92:0/1989242518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12480fbf40 0x7f1248113ee0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:16.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.251+0000 7f124e0a6700 1 -- 172.21.15.92:0/1989242518 >> 172.21.15.92:0/1989242518 conn(0x7f1248073350 msgr2=0x7f12480f8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:16.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.251+0000 7f124e0a6700 1 -- 172.21.15.92:0/1989242518 shutdown_connections 2024-05-06T12:12:16.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16.251+0000 7f124e0a6700 1 -- 172.21.15.92:0/1989242518 wait complete. 2024-05-06T12:12:16.804 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-06T12:12:16.805 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-06T12:12:16.805 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2024-05-06T12:12:16.805 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-06T12:12:16.805 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-06T12:12:16.805 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-06T12:12:16.805 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-06T12:12:16.805 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-06T12:12:16.805 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-06T12:12:16.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-06T12:12:16.805 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2024-05-06T12:12:16.822 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-06T12:12:16.822 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-06T12:12:16.822 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2024-05-06T12:12:16.822 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-05-06T12:12:16.822 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2024-05-06T12:12:16.822 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-05-06T12:12:16.822 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-06T12:12:16.822 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2024-05-06T12:12:16.822 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2024-05-06T12:12:16.822 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-06T12:12:16.823 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2024-05-06T12:12:16.823 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2024-05-06T12:12:16.823 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-06T12:12:16.823 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2024-05-06T12:12:16.823 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-06T12:12:16.823 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2024-05-06T12:12:16.823 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-06T12:12:16.823 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42258sec preferred_lft 42258sec 2024-05-06T12:12:16.823 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::519e:3ffc:201e:e942/64 scope link noprefixroute 2024-05-06T12:12:16.823 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-05-06T12:12:16.824 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-06T12:12:16.824 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-06T12:12:16.824 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-05-06T12:12:16.824 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add name ceph-brx type bridge 2024-05-06T12:12:16.824 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip addr flush dev ceph-brx 2024-05-06T12:12:16.824 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set ceph-brx up 2024-05-06T12:12:16.824 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-06T12:12:16.824 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-05-06T12:12:16.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-06T12:12:17.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-06T12:12:17.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-06T12:12:17.012 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-06T12:12:17.035 INFO:teuthology.orchestra.run.smithi092.stdout:1 2024-05-06T12:12:17.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-06T12:12:17.036 DEBUG:teuthology.orchestra.run.smithi092:> route 2024-05-06T12:12:17.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:16 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:17.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:16 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:17.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:16 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:12:17.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:16 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:12:17.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:16 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:17.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:16 smithi190 ceph-mon[33798]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.9 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-06T12:12:17.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:16 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/1989242518' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-06T12:12:17.107 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2024-05-06T12:12:17.107 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-06T12:12:17.108 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-06T12:12:17.108 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-06T12:12:17.108 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-06T12:12:17.108 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-06T12:12:17.108 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-05-06T12:12:17.108 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-06T12:12:17.108 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-06T12:12:17.108 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-06T12:12:17.108 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-05-06T12:12:17.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-06T12:12:17.203 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:16 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:17.203 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:16 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:17.203 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:16 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:12:17.203 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:16 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:12:17.203 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:16 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:17.203 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:16 smithi092 ceph-mon[26954]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.9 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-06T12:12:17.203 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:16 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/1989242518' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-06T12:12:17.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-06T12:12:17.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-06T12:12:17.383 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2024-05-06T12:12:17.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-06T12:12:17.399 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list-id 2024-05-06T12:12:17.454 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-06T12:12:17.454 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-05-06T12:12:17.454 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-06T12:12:17.454 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-06T12:12:17.454 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-05-06T12:12:17.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-06T12:12:17.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-06T12:12:17.568 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-06T12:12:17.569 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-06T12:12:17.569 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-05-06T12:12:17.569 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-06T12:12:17.569 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-06T12:12:17.569 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-06T12:12:17.569 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-06T12:12:17.569 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-06T12:12:17.569 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-05-06T12:12:17.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-06T12:12:17.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-06T12:12:17.697 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-06T12:12:17.698 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-05-06T12:12:17.698 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2024-05-06T12:12:17.698 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set dev brx.0 master ceph-brx 2024-05-06T12:12:17.698 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-05-06T12:12:17.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-06T12:12:17.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-06T12:12:17.772 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-06T12:12:17.772 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-06T12:12:17.773 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-06T12:12:17.787 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-06T12:12:17.787 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-06T12:12:17.787 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-06T12:12:17.842 DEBUG:teuthology.orchestra.run.smithi092:> sudo modprobe fuse 2024-05-06T12:12:17.923 DEBUG:teuthology.orchestra.run.smithi092:> cat /proc/self/mounts | awk '{print $2}' 2024-05-06T12:12:17.979 INFO:teuthology.orchestra.run.smithi092.stdout:/proc 2024-05-06T12:12:17.979 INFO:teuthology.orchestra.run.smithi092.stdout:/sys 2024-05-06T12:12:17.980 INFO:teuthology.orchestra.run.smithi092.stdout:/dev 2024-05-06T12:12:17.980 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/security 2024-05-06T12:12:17.980 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/shm 2024-05-06T12:12:17.980 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/pts 2024-05-06T12:12:17.980 INFO:teuthology.orchestra.run.smithi092.stdout:/run 2024-05-06T12:12:17.980 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/cgroup 2024-05-06T12:12:17.980 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/pstore 2024-05-06T12:12:17.980 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/bpf 2024-05-06T12:12:17.980 INFO:teuthology.orchestra.run.smithi092.stdout:/ 2024-05-06T12:12:17.980 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/selinux 2024-05-06T12:12:17.980 INFO:teuthology.orchestra.run.smithi092.stdout:/proc/sys/fs/binfmt_misc 2024-05-06T12:12:17.980 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/hugepages 2024-05-06T12:12:17.980 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/mqueue 2024-05-06T12:12:17.980 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/debug 2024-05-06T12:12:17.980 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/tracing 2024-05-06T12:12:17.981 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-sysusers.service 2024-05-06T12:12:17.981 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/config 2024-05-06T12:12:17.981 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-06T12:12:17.981 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/fuse/connections 2024-05-06T12:12:17.981 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-sysctl.service 2024-05-06T12:12:17.981 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-06T12:12:17.981 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/nfs/rpc_pipefs 2024-05-06T12:12:17.981 INFO:teuthology.orchestra.run.smithi092.stdout:/run/user/1000 2024-05-06T12:12:17.981 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph 2024-05-06T12:12:17.981 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay 2024-05-06T12:12:17.981 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/6a1a52fce029ad14e30feda9b166a0cd89d4845c5d4bca9f98f3adf414879dfd/merged 2024-05-06T12:12:17.981 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/9b10fac131a0366f7c541b4da0af8da88a23a506219b335b5e9a73e5c5349682/merged 2024-05-06T12:12:17.981 INFO:teuthology.orchestra.run.smithi092.stdout:/run/user/0 2024-05-06T12:12:17.981 INFO:teuthology.orchestra.run.smithi092.stdout:/proc/sys/fs/binfmt_misc 2024-05-06T12:12:17.981 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/78fbbfd106b19b2e18c9a55ca35d6d9ee6072725914d8bf3737e86ab85e1909d/merged 2024-05-06T12:12:17.981 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/7c32af9b1838d435f79e92ada8d95e1e5ca2de88b47f5e8fc25d2dc0694f0532/merged 2024-05-06T12:12:17.982 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/afbeb8c15c34d9e5b5011b6715f382937677dee5649f5b8758e7a47086c2af9a/merged 2024-05-06T12:12:17.982 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/fbf8736868cd5055462767afd990ed711fe6db84697db0477d90c4de3fb3578e/merged 2024-05-06T12:12:17.982 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/b8adb0cf74bd78a8ac8ff11a72b118d9d3882936e5734d5ddf2b3198fdb7220d/merged 2024-05-06T12:12:17.982 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/24040e6088648f8d5d51b1c87007b2d3412e9a209a1908eabea5c063a2e6696b/merged 2024-05-06T12:12:17.982 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/ea459dd6f2a55af508f33b8a2ef47fc7e96f758d20b67d5faf3f3e30148da448/merged 2024-05-06T12:12:17.982 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/57a1ecf394cb72d66056e51f145b7a24322cf1b2c8f9a1e5ecd1ca12d0b0adc5/merged 2024-05-06T12:12:17.982 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/dbf70f0375975eb029499bc767132662b93620945e3175b0cbc60d3f1530f8ba/merged 2024-05-06T12:12:17.982 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/7e99ce7bd482c65597d785e6cf4f777c690580d519f0dec46dbf3a8af1eb47e5/merged 2024-05-06T12:12:17.982 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns 2024-05-06T12:12:17.982 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-06T12:12:17.982 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-06T12:12:17.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-06T12:12:17.983 DEBUG:teuthology.orchestra.run.smithi092:> ls /sys/fs/fuse/connections 2024-05-06T12:12:18.034 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-06T12:12:18.034 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-06T12:12:18.077 DEBUG:teuthology.orchestra.run.smithi092:> sudo modprobe fuse 2024-05-06T12:12:18.099 DEBUG:teuthology.orchestra.run.smithi092:> cat /proc/self/mounts | awk '{print $2}' 2024-05-06T12:12:18.159 INFO:teuthology.orchestra.run.smithi092.stdout:/proc 2024-05-06T12:12:18.159 INFO:teuthology.orchestra.run.smithi092.stdout:/sys 2024-05-06T12:12:18.159 INFO:teuthology.orchestra.run.smithi092.stdout:/dev 2024-05-06T12:12:18.159 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/security 2024-05-06T12:12:18.159 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/shm 2024-05-06T12:12:18.159 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/pts 2024-05-06T12:12:18.159 INFO:teuthology.orchestra.run.smithi092.stdout:/run 2024-05-06T12:12:18.160 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/cgroup 2024-05-06T12:12:18.160 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/pstore 2024-05-06T12:12:18.160 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/bpf 2024-05-06T12:12:18.160 INFO:teuthology.orchestra.run.smithi092.stdout:/ 2024-05-06T12:12:18.160 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/selinux 2024-05-06T12:12:18.160 INFO:teuthology.orchestra.run.smithi092.stdout:/proc/sys/fs/binfmt_misc 2024-05-06T12:12:18.160 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/hugepages 2024-05-06T12:12:18.160 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/mqueue 2024-05-06T12:12:18.160 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/debug 2024-05-06T12:12:18.160 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/tracing 2024-05-06T12:12:18.160 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-sysusers.service 2024-05-06T12:12:18.160 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/config 2024-05-06T12:12:18.160 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-06T12:12:18.161 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/fuse/connections 2024-05-06T12:12:18.161 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-sysctl.service 2024-05-06T12:12:18.161 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-06T12:12:18.161 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/nfs/rpc_pipefs 2024-05-06T12:12:18.161 INFO:teuthology.orchestra.run.smithi092.stdout:/run/user/1000 2024-05-06T12:12:18.161 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph 2024-05-06T12:12:18.161 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay 2024-05-06T12:12:18.161 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/6a1a52fce029ad14e30feda9b166a0cd89d4845c5d4bca9f98f3adf414879dfd/merged 2024-05-06T12:12:18.161 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/9b10fac131a0366f7c541b4da0af8da88a23a506219b335b5e9a73e5c5349682/merged 2024-05-06T12:12:18.161 INFO:teuthology.orchestra.run.smithi092.stdout:/run/user/0 2024-05-06T12:12:18.161 INFO:teuthology.orchestra.run.smithi092.stdout:/proc/sys/fs/binfmt_misc 2024-05-06T12:12:18.161 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/78fbbfd106b19b2e18c9a55ca35d6d9ee6072725914d8bf3737e86ab85e1909d/merged 2024-05-06T12:12:18.161 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/7c32af9b1838d435f79e92ada8d95e1e5ca2de88b47f5e8fc25d2dc0694f0532/merged 2024-05-06T12:12:18.161 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/afbeb8c15c34d9e5b5011b6715f382937677dee5649f5b8758e7a47086c2af9a/merged 2024-05-06T12:12:18.161 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/fbf8736868cd5055462767afd990ed711fe6db84697db0477d90c4de3fb3578e/merged 2024-05-06T12:12:18.162 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/b8adb0cf74bd78a8ac8ff11a72b118d9d3882936e5734d5ddf2b3198fdb7220d/merged 2024-05-06T12:12:18.162 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/24040e6088648f8d5d51b1c87007b2d3412e9a209a1908eabea5c063a2e6696b/merged 2024-05-06T12:12:18.162 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/ea459dd6f2a55af508f33b8a2ef47fc7e96f758d20b67d5faf3f3e30148da448/merged 2024-05-06T12:12:18.162 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/57a1ecf394cb72d66056e51f145b7a24322cf1b2c8f9a1e5ecd1ca12d0b0adc5/merged 2024-05-06T12:12:18.162 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/dbf70f0375975eb029499bc767132662b93620945e3175b0cbc60d3f1530f8ba/merged 2024-05-06T12:12:18.162 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/containers/storage/overlay/7e99ce7bd482c65597d785e6cf4f777c690580d519f0dec46dbf3a8af1eb47e5/merged 2024-05-06T12:12:18.162 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns 2024-05-06T12:12:18.162 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-06T12:12:18.162 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-06T12:12:18.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-06T12:12:18.163 DEBUG:teuthology.orchestra.run.smithi092:> ls /sys/fs/fuse/connections 2024-05-06T12:12:18.163 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi092.stderr:2024-05-06T12:12:18.158+0000 7fb0aa4a4300 -1 init, newargv = 0x55e6351d07e0 newargc=15 2024-05-06T12:12:18.163 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi092.stderr:ceph-fuse[74605]: starting ceph client 2024-05-06T12:12:18.168 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi092.stderr:ceph-fuse[74605]: starting fuse 2024-05-06T12:12:18.178 INFO:teuthology.orchestra.run.smithi092.stdout:134 2024-05-06T12:12:18.178 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [134] 2024-05-06T12:12:18.179 DEBUG:teuthology.orchestra.run.smithi092:> sudo stdin-killer -- python3 -c ' 2024-05-06T12:12:18.179 DEBUG:teuthology.orchestra.run.smithi092:> import glob 2024-05-06T12:12:18.179 DEBUG:teuthology.orchestra.run.smithi092:> import re 2024-05-06T12:12:18.179 DEBUG:teuthology.orchestra.run.smithi092:> import os 2024-05-06T12:12:18.179 DEBUG:teuthology.orchestra.run.smithi092:> import subprocess 2024-05-06T12:12:18.179 DEBUG:teuthology.orchestra.run.smithi092:> 2024-05-06T12:12:18.179 DEBUG:teuthology.orchestra.run.smithi092:> def _find_admin_socket(client_name): 2024-05-06T12:12:18.179 DEBUG:teuthology.orchestra.run.smithi092:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-06T12:12:18.179 DEBUG:teuthology.orchestra.run.smithi092:> files = glob.glob(asok_path) 2024-05-06T12:12:18.179 DEBUG:teuthology.orchestra.run.smithi092:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-06T12:12:18.179 DEBUG:teuthology.orchestra.run.smithi092:> 2024-05-06T12:12:18.179 DEBUG:teuthology.orchestra.run.smithi092:> # Given a non-glob path, it better be there 2024-05-06T12:12:18.179 DEBUG:teuthology.orchestra.run.smithi092:> if "*" not in asok_path: 2024-05-06T12:12:18.179 DEBUG:teuthology.orchestra.run.smithi092:> assert(len(files) == 1) 2024-05-06T12:12:18.180 DEBUG:teuthology.orchestra.run.smithi092:> return files[0] 2024-05-06T12:12:18.180 DEBUG:teuthology.orchestra.run.smithi092:> 2024-05-06T12:12:18.180 DEBUG:teuthology.orchestra.run.smithi092:> for f in files: 2024-05-06T12:12:18.180 DEBUG:teuthology.orchestra.run.smithi092:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-06T12:12:18.180 DEBUG:teuthology.orchestra.run.smithi092:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-06T12:12:18.180 DEBUG:teuthology.orchestra.run.smithi092:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-06T12:12:18.180 DEBUG:teuthology.orchestra.run.smithi092:> contents = proc_f.read() 2024-05-06T12:12:18.180 DEBUG:teuthology.orchestra.run.smithi092:> if mountpoint in contents: 2024-05-06T12:12:18.180 DEBUG:teuthology.orchestra.run.smithi092:> return f 2024-05-06T12:12:18.180 DEBUG:teuthology.orchestra.run.smithi092:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-06T12:12:18.180 DEBUG:teuthology.orchestra.run.smithi092:> 2024-05-06T12:12:18.180 DEBUG:teuthology.orchestra.run.smithi092:> print(_find_admin_socket("client.0")) 2024-05-06T12:12:18.180 DEBUG:teuthology.orchestra.run.smithi092:> ' 2024-05-06T12:12:18.295 INFO:teuthology.orchestra.run.smithi092.stdout:/var/run/ceph/ceph-client.0.74605.asok 2024-05-06T12:12:18.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-06T12:12:18.306 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.74605.asok 2024-05-06T12:12:18.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-06T12:12:18.306 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.74605.asok status 2024-05-06T12:12:18.436 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:12:18.436 INFO:teuthology.orchestra.run.smithi092.stdout: "metadata": { 2024-05-06T12:12:18.436 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_sha1": "e81df4201214a93810b2eaca609e475a6319e220", 2024-05-06T12:12:18.436 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_version": "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)", 2024-05-06T12:12:18.436 INFO:teuthology.orchestra.run.smithi092.stdout: "entity_id": "0", 2024-05-06T12:12:18.436 INFO:teuthology.orchestra.run.smithi092.stdout: "hostname": "smithi092", 2024-05-06T12:12:18.436 INFO:teuthology.orchestra.run.smithi092.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-06T12:12:18.436 INFO:teuthology.orchestra.run.smithi092.stdout: "pid": "74605", 2024-05-06T12:12:18.436 INFO:teuthology.orchestra.run.smithi092.stdout: "root": "/" 2024-05-06T12:12:18.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:12:18.436 INFO:teuthology.orchestra.run.smithi092.stdout: "dentry_count": 0, 2024-05-06T12:12:18.437 INFO:teuthology.orchestra.run.smithi092.stdout: "dentry_pinned_count": 0, 2024-05-06T12:12:18.437 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 14524, 2024-05-06T12:12:18.437 INFO:teuthology.orchestra.run.smithi092.stdout: "inst": { 2024-05-06T12:12:18.437 INFO:teuthology.orchestra.run.smithi092.stdout: "name": { 2024-05-06T12:12:18.437 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "client", 2024-05-06T12:12:18.437 INFO:teuthology.orchestra.run.smithi092.stdout: "num": 14524 2024-05-06T12:12:18.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:12:18.437 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": { 2024-05-06T12:12:18.437 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-05-06T12:12:18.437 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "192.168.0.1:0", 2024-05-06T12:12:18.437 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1455526237 2024-05-06T12:12:18.437 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-06T12:12:18.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:12:18.437 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": { 2024-05-06T12:12:18.437 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-05-06T12:12:18.438 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "192.168.0.1:0", 2024-05-06T12:12:18.438 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1455526237 2024-05-06T12:12:18.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:12:18.438 INFO:teuthology.orchestra.run.smithi092.stdout: "inst_str": "client.14524 192.168.0.1:0/1455526237", 2024-05-06T12:12:18.438 INFO:teuthology.orchestra.run.smithi092.stdout: "addr_str": "192.168.0.1:0/1455526237", 2024-05-06T12:12:18.438 INFO:teuthology.orchestra.run.smithi092.stdout: "inode_count": 1, 2024-05-06T12:12:18.438 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_epoch": 18, 2024-05-06T12:12:18.438 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_epoch": 46, 2024-05-06T12:12:18.438 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_epoch_barrier": 0, 2024-05-06T12:12:18.438 INFO:teuthology.orchestra.run.smithi092.stdout: "blocklisted": false, 2024-05-06T12:12:18.438 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs" 2024-05-06T12:12:18.438 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:12:18.444 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-06T12:12:18.695 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:12:19.143 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:18 smithi092 ceph-mon[26954]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-06T12:12:19.143 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:18 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:19.143 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:18 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:12:19.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:18 smithi190 ceph-mon[33798]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-06T12:12:19.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:18 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:19.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:18 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:12:20.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.277+0000 7fe70de68700 1 -- 172.21.15.92:0/3802373928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7080fe7f0 msgr2=0x7fe708106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:20.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.277+0000 7fe70de68700 1 --2- 172.21.15.92:0/3802373928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7080fe7f0 0x7fe708106e30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc004940 tx=0x7fe6fc009e30 comp rx=0 tx=0).stop 2024-05-06T12:12:20.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.277+0000 7fe70de68700 1 -- 172.21.15.92:0/3802373928 shutdown_connections 2024-05-06T12:12:20.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.277+0000 7fe70de68700 1 --2- 172.21.15.92:0/3802373928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7080fe7f0 0x7fe708106e30 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:20.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.277+0000 7fe70de68700 1 --2- 172.21.15.92:0/3802373928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7080fde50 0x7fe7080fe220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:20.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.277+0000 7fe70de68700 1 -- 172.21.15.92:0/3802373928 >> 172.21.15.92:0/3802373928 conn(0x7fe7080f7860 msgr2=0x7fe7080f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:20.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.277+0000 7fe70de68700 1 -- 172.21.15.92:0/3802373928 shutdown_connections 2024-05-06T12:12:20.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.277+0000 7fe70de68700 1 -- 172.21.15.92:0/3802373928 wait complete. 2024-05-06T12:12:20.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.277+0000 7fe70de68700 1 Processor -- start 2024-05-06T12:12:20.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.277+0000 7fe70de68700 1 -- start start 2024-05-06T12:12:20.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.278+0000 7fe70de68700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7080fde50 0x7fe708113960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:20.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.278+0000 7fe70de68700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7080fe7f0 0x7fe708113ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:20.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.278+0000 7fe70de68700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe708114580 con 0x7fe7080fe7f0 2024-05-06T12:12:20.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.278+0000 7fe70de68700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70810daf0 con 0x7fe7080fde50 2024-05-06T12:12:20.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.278+0000 7fe706ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7080fe7f0 0x7fe708113ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:20.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.278+0000 7fe706ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7080fe7f0 0x7fe708113ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57892/0 (socket says 172.21.15.92:57892) 2024-05-06T12:12:20.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.278+0000 7fe706ffd700 1 -- 172.21.15.92:0/1419004648 learned_addr learned my addr 172.21.15.92:0/1419004648 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:12:20.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.278+0000 7fe7077fe700 1 --2- 172.21.15.92:0/1419004648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7080fde50 0x7fe708113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:20.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.278+0000 7fe706ffd700 1 -- 172.21.15.92:0/1419004648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7080fde50 msgr2=0x7fe708113960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:20.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.278+0000 7fe706ffd700 1 --2- 172.21.15.92:0/1419004648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7080fde50 0x7fe708113960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:20.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.278+0000 7fe706ffd700 1 -- 172.21.15.92:0/1419004648 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6fc005040 con 0x7fe7080fe7f0 2024-05-06T12:12:20.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.278+0000 7fe706ffd700 1 --2- 172.21.15.92:0/1419004648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7080fe7f0 0x7fe708113ea0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc004890 tx=0x7fe6fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:20.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.278+0000 7fe6effff700 1 -- 172.21.15.92:0/1419004648 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6fc00e040 con 0x7fe7080fe7f0 2024-05-06T12:12:20.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.279+0000 7fe70de68700 1 -- 172.21.15.92:0/1419004648 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe70810ddd0 con 0x7fe7080fe7f0 2024-05-06T12:12:20.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.279+0000 7fe6effff700 1 -- 172.21.15.92:0/1419004648 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe6fc009af0 con 0x7fe7080fe7f0 2024-05-06T12:12:20.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.279+0000 7fe6effff700 1 -- 172.21.15.92:0/1419004648 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6fc01c630 con 0x7fe7080fe7f0 2024-05-06T12:12:20.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.279+0000 7fe70de68700 1 -- 172.21.15.92:0/1419004648 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe70810e310 con 0x7fe7080fe7f0 2024-05-06T12:12:20.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.279+0000 7fe70de68700 1 -- 172.21.15.92:0/1419004648 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe708066b50 con 0x7fe7080fe7f0 2024-05-06T12:12:20.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.280+0000 7fe6effff700 1 -- 172.21.15.92:0/1419004648 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fe6fc00bbc0 con 0x7fe7080fe7f0 2024-05-06T12:12:20.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.280+0000 7fe6effff700 1 --2- 172.21.15.92:0/1419004648 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fe6f00692a0 0x7fe6f006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:20.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.280+0000 7fe6effff700 1 -- 172.21.15.92:0/1419004648 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fe6fc0884e0 con 0x7fe7080fe7f0 2024-05-06T12:12:20.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.280+0000 7fe7077fe700 1 --2- 172.21.15.92:0/1419004648 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fe6f00692a0 0x7fe6f006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:20.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.281+0000 7fe7077fe700 1 --2- 172.21.15.92:0/1419004648 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fe6f00692a0 0x7fe6f006b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8006fd0 tx=0x7fe6f8008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:20.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.285+0000 7fe6effff700 1 -- 172.21.15.92:0/1419004648 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe6fc058250 con 0x7fe7080fe7f0 2024-05-06T12:12:20.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.516+0000 7fe70de68700 1 -- 172.21.15.92:0/1419004648 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe70810efb0 con 0x7fe7080fe7f0 2024-05-06T12:12:20.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.516+0000 7fe6effff700 1 -- 172.21.15.92:0/1419004648 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+83 (secure 0 0 0) 0x7fe6fc057de0 con 0x7fe7080fe7f0 2024-05-06T12:12:20.519 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-05-06T12:12:20.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.518+0000 7fe70de68700 1 -- 172.21.15.92:0/1419004648 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fe6f00692a0 msgr2=0x7fe6f006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:20.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.518+0000 7fe70de68700 1 --2- 172.21.15.92:0/1419004648 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fe6f00692a0 0x7fe6f006b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8006fd0 tx=0x7fe6f8008040 comp rx=0 tx=0).stop 2024-05-06T12:12:20.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.518+0000 7fe70de68700 1 -- 172.21.15.92:0/1419004648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7080fe7f0 msgr2=0x7fe708113ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:20.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.518+0000 7fe70de68700 1 --2- 172.21.15.92:0/1419004648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7080fe7f0 0x7fe708113ea0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc004890 tx=0x7fe6fc009e30 comp rx=0 tx=0).stop 2024-05-06T12:12:20.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.518+0000 7fe70de68700 1 -- 172.21.15.92:0/1419004648 shutdown_connections 2024-05-06T12:12:20.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.518+0000 7fe70de68700 1 --2- 172.21.15.92:0/1419004648 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fe6f00692a0 0x7fe6f006b750 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:20.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.518+0000 7fe70de68700 1 --2- 172.21.15.92:0/1419004648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7080fde50 0x7fe708113960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:20.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.518+0000 7fe70de68700 1 --2- 172.21.15.92:0/1419004648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7080fe7f0 0x7fe708113ea0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:20.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.518+0000 7fe70de68700 1 -- 172.21.15.92:0/1419004648 >> 172.21.15.92:0/1419004648 conn(0x7fe7080f7860 msgr2=0x7fe7080f8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:20.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.518+0000 7fe70de68700 1 -- 172.21.15.92:0/1419004648 shutdown_connections 2024-05-06T12:12:20.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:20.518+0000 7fe70de68700 1 -- 172.21.15.92:0/1419004648 wait complete. 2024-05-06T12:12:21.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:20 smithi092 ceph-mon[26954]: pgmap v192: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.0 KiB/s rd, 85 B/s wr, 4 op/s 2024-05-06T12:12:21.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:20 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/1419004648' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-06T12:12:21.287 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-06T12:12:21.287 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-06T12:12:21.287 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi190.front.sepia.ceph.com 2024-05-06T12:12:21.287 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-06T12:12:21.287 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-06T12:12:21.287 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-06T12:12:21.288 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-06T12:12:21.288 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-06T12:12:21.288 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-06T12:12:21.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-06T12:12:21.288 DEBUG:teuthology.orchestra.run.smithi190:> ip addr 2024-05-06T12:12:21.307 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-06T12:12:21.307 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-06T12:12:21.307 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2024-05-06T12:12:21.307 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-05-06T12:12:21.307 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2024-05-06T12:12:21.307 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-05-06T12:12:21.307 INFO:teuthology.orchestra.run.smithi190.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-06T12:12:21.308 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d0 brd ff:ff:ff:ff:ff:ff 2024-05-06T12:12:21.308 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f0 2024-05-06T12:12:21.308 INFO:teuthology.orchestra.run.smithi190.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-06T12:12:21.308 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d1 brd ff:ff:ff:ff:ff:ff 2024-05-06T12:12:21.308 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f1 2024-05-06T12:12:21.308 INFO:teuthology.orchestra.run.smithi190.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-06T12:12:21.308 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:62 brd ff:ff:ff:ff:ff:ff 2024-05-06T12:12:21.308 INFO:teuthology.orchestra.run.smithi190.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-06T12:12:21.308 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:63 brd ff:ff:ff:ff:ff:ff 2024-05-06T12:12:21.308 INFO:teuthology.orchestra.run.smithi190.stdout: inet 172.21.15.190/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-06T12:12:21.308 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft 42245sec preferred_lft 42245sec 2024-05-06T12:12:21.308 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::7122:3a67:be02:dc79/64 scope link noprefixroute 2024-05-06T12:12:21.308 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-05-06T12:12:21.309 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-06T12:12:21.309 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-06T12:12:21.309 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-05-06T12:12:21.309 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link add name ceph-brx type bridge 2024-05-06T12:12:21.309 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip addr flush dev ceph-brx 2024-05-06T12:12:21.309 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set ceph-brx up 2024-05-06T12:12:21.309 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-06T12:12:21.309 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-05-06T12:12:21.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:20 smithi190 ceph-mon[33798]: pgmap v192: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.0 KiB/s rd, 85 B/s wr, 4 op/s 2024-05-06T12:12:21.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:20 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/1419004648' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-06T12:12:21.402 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:12:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-06T12:12:21.489 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:12:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-06T12:12:21.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-06T12:12:21.496 DEBUG:teuthology.orchestra.run.smithi190:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-06T12:12:21.521 INFO:teuthology.orchestra.run.smithi190.stdout:1 2024-05-06T12:12:21.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-06T12:12:21.521 DEBUG:teuthology.orchestra.run.smithi190:> route 2024-05-06T12:12:21.586 INFO:teuthology.orchestra.run.smithi190.stdout:Kernel IP routing table 2024-05-06T12:12:21.586 INFO:teuthology.orchestra.run.smithi190.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-06T12:12:21.586 INFO:teuthology.orchestra.run.smithi190.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-06T12:12:21.586 INFO:teuthology.orchestra.run.smithi190.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-06T12:12:21.586 INFO:teuthology.orchestra.run.smithi190.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-06T12:12:21.587 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-06T12:12:21.587 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-05-06T12:12:21.587 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-06T12:12:21.587 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-06T12:12:21.587 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-06T12:12:21.587 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-05-06T12:12:21.673 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:12:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-06T12:12:21.859 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:12:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-06T12:12:21.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-06T12:12:21.866 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2024-05-06T12:12:21.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-06T12:12:21.881 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list-id 2024-05-06T12:12:21.937 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-06T12:12:21.938 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-05-06T12:12:21.938 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-06T12:12:21.938 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-06T12:12:21.938 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-05-06T12:12:22.026 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:12:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-06T12:12:22.049 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:12:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-06T12:12:22.055 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-06T12:12:22.056 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-06T12:12:22.056 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-05-06T12:12:22.056 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-06T12:12:22.056 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-06T12:12:22.056 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-06T12:12:22.056 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-06T12:12:22.056 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-06T12:12:22.056 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-05-06T12:12:22.104 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:12:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-06T12:12:22.189 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:12:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-06T12:12:22.195 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-06T12:12:22.195 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-05-06T12:12:22.195 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set brx.0 up 2024-05-06T12:12:22.195 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set dev brx.0 master ceph-brx 2024-05-06T12:12:22.195 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-05-06T12:12:22.243 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:12:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-06T12:12:22.264 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:12:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-06T12:12:22.270 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-05-06T12:12:22.270 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-06T12:12:22.270 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-06T12:12:22.284 INFO:teuthology.orchestra.run.smithi190.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-06T12:12:22.285 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-06T12:12:22.285 DEBUG:teuthology.orchestra.run.smithi190:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-06T12:12:22.340 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2024-05-06T12:12:22.405 DEBUG:teuthology.orchestra.run.smithi190:> cat /proc/self/mounts | awk '{print $2}' 2024-05-06T12:12:22.460 INFO:teuthology.orchestra.run.smithi190.stdout:/proc 2024-05-06T12:12:22.461 INFO:teuthology.orchestra.run.smithi190.stdout:/sys 2024-05-06T12:12:22.461 INFO:teuthology.orchestra.run.smithi190.stdout:/dev 2024-05-06T12:12:22.461 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/security 2024-05-06T12:12:22.461 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/shm 2024-05-06T12:12:22.461 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/pts 2024-05-06T12:12:22.461 INFO:teuthology.orchestra.run.smithi190.stdout:/run 2024-05-06T12:12:22.461 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup 2024-05-06T12:12:22.461 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/pstore 2024-05-06T12:12:22.461 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/bpf 2024-05-06T12:12:22.461 INFO:teuthology.orchestra.run.smithi190.stdout:/ 2024-05-06T12:12:22.461 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/selinux 2024-05-06T12:12:22.461 INFO:teuthology.orchestra.run.smithi190.stdout:/proc/sys/fs/binfmt_misc 2024-05-06T12:12:22.462 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/mqueue 2024-05-06T12:12:22.462 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/hugepages 2024-05-06T12:12:22.462 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/debug 2024-05-06T12:12:22.462 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/tracing 2024-05-06T12:12:22.462 INFO:teuthology.orchestra.run.smithi190.stdout:/run/credentials/systemd-sysusers.service 2024-05-06T12:12:22.462 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/config 2024-05-06T12:12:22.462 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/fuse/connections 2024-05-06T12:12:22.462 INFO:teuthology.orchestra.run.smithi190.stdout:/run/credentials/systemd-sysctl.service 2024-05-06T12:12:22.462 INFO:teuthology.orchestra.run.smithi190.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-06T12:12:22.462 INFO:teuthology.orchestra.run.smithi190.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-06T12:12:22.462 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/nfs/rpc_pipefs 2024-05-06T12:12:22.462 INFO:teuthology.orchestra.run.smithi190.stdout:/run/user/1000 2024-05-06T12:12:22.463 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/ceph 2024-05-06T12:12:22.463 INFO:teuthology.orchestra.run.smithi190.stdout:/run/user/0 2024-05-06T12:12:22.463 INFO:teuthology.orchestra.run.smithi190.stdout:/proc/sys/fs/binfmt_misc 2024-05-06T12:12:22.463 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/containers/storage/overlay 2024-05-06T12:12:22.463 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/containers/storage/overlay/a264833a1ba8ebc03b45cb618c0098ab1e3c87a8780b77c39142984165fa4da3/merged 2024-05-06T12:12:22.463 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/containers/storage/overlay/ac4b5b79814888a75c68dd09590a2805381f55c60479dc6d3abec48bac85ab72/merged 2024-05-06T12:12:22.463 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/containers/storage/overlay/72a1b58be90c4a8a917c1f48ddc3274913b42c8754bdb6165e500d1e19f7ccd7/merged 2024-05-06T12:12:22.463 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/containers/storage/overlay/0c212ea13130385ec97b282fd1778f0077da2e3ddf6c01eebfb327d28282bf77/merged 2024-05-06T12:12:22.463 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/containers/storage/overlay/de46f15ce92b583821662d6a9533fdf75b568e241bf649d961228f550f8878d3/merged 2024-05-06T12:12:22.463 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/containers/storage/overlay/04a88556ed4cdf888aa8b4ca4f89926028ae40b73ab1f864fb877d104a37c69c/merged 2024-05-06T12:12:22.463 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/containers/storage/overlay/6174b9c24d87d29db3e80d2b602f3ef36e0494c62af6de3bdadcf43b9d47d5aa/merged 2024-05-06T12:12:22.463 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/containers/storage/overlay/49acec9b92e1443f63933c7d81d95f638cbdf6119295a601dea3e0f2c78477b7/merged 2024-05-06T12:12:22.464 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/containers/storage/overlay/4eb76f020089d822ee4f920d293ebe3ab92be64db05184f627247348ae63b976/merged 2024-05-06T12:12:22.464 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns 2024-05-06T12:12:22.464 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-06T12:12:22.464 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-06T12:12:22.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-06T12:12:22.464 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2024-05-06T12:12:22.516 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-06T12:12:22.516 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-05-06T12:12:22.558 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2024-05-06T12:12:22.581 DEBUG:teuthology.orchestra.run.smithi190:> cat /proc/self/mounts | awk '{print $2}' 2024-05-06T12:12:22.636 INFO:teuthology.orchestra.run.smithi190.stdout:/proc 2024-05-06T12:12:22.636 INFO:teuthology.orchestra.run.smithi190.stdout:/sys 2024-05-06T12:12:22.637 INFO:teuthology.orchestra.run.smithi190.stdout:/dev 2024-05-06T12:12:22.637 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/security 2024-05-06T12:12:22.637 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/shm 2024-05-06T12:12:22.637 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/pts 2024-05-06T12:12:22.637 INFO:teuthology.orchestra.run.smithi190.stdout:/run 2024-05-06T12:12:22.637 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup 2024-05-06T12:12:22.637 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/pstore 2024-05-06T12:12:22.637 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/bpf 2024-05-06T12:12:22.637 INFO:teuthology.orchestra.run.smithi190.stdout:/ 2024-05-06T12:12:22.637 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/selinux 2024-05-06T12:12:22.637 INFO:teuthology.orchestra.run.smithi190.stdout:/proc/sys/fs/binfmt_misc 2024-05-06T12:12:22.637 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/mqueue 2024-05-06T12:12:22.637 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/hugepages 2024-05-06T12:12:22.637 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/debug 2024-05-06T12:12:22.638 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/tracing 2024-05-06T12:12:22.638 INFO:teuthology.orchestra.run.smithi190.stdout:/run/credentials/systemd-sysusers.service 2024-05-06T12:12:22.638 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/config 2024-05-06T12:12:22.638 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/fuse/connections 2024-05-06T12:12:22.638 INFO:teuthology.orchestra.run.smithi190.stdout:/run/credentials/systemd-sysctl.service 2024-05-06T12:12:22.638 INFO:teuthology.orchestra.run.smithi190.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-06T12:12:22.638 INFO:teuthology.orchestra.run.smithi190.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-06T12:12:22.638 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/nfs/rpc_pipefs 2024-05-06T12:12:22.638 INFO:teuthology.orchestra.run.smithi190.stdout:/run/user/1000 2024-05-06T12:12:22.638 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/ceph 2024-05-06T12:12:22.638 INFO:teuthology.orchestra.run.smithi190.stdout:/run/user/0 2024-05-06T12:12:22.638 INFO:teuthology.orchestra.run.smithi190.stdout:/proc/sys/fs/binfmt_misc 2024-05-06T12:12:22.638 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/containers/storage/overlay 2024-05-06T12:12:22.639 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/containers/storage/overlay/a264833a1ba8ebc03b45cb618c0098ab1e3c87a8780b77c39142984165fa4da3/merged 2024-05-06T12:12:22.639 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/containers/storage/overlay/ac4b5b79814888a75c68dd09590a2805381f55c60479dc6d3abec48bac85ab72/merged 2024-05-06T12:12:22.639 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/containers/storage/overlay/72a1b58be90c4a8a917c1f48ddc3274913b42c8754bdb6165e500d1e19f7ccd7/merged 2024-05-06T12:12:22.639 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/containers/storage/overlay/0c212ea13130385ec97b282fd1778f0077da2e3ddf6c01eebfb327d28282bf77/merged 2024-05-06T12:12:22.639 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/containers/storage/overlay/de46f15ce92b583821662d6a9533fdf75b568e241bf649d961228f550f8878d3/merged 2024-05-06T12:12:22.639 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/containers/storage/overlay/04a88556ed4cdf888aa8b4ca4f89926028ae40b73ab1f864fb877d104a37c69c/merged 2024-05-06T12:12:22.639 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/containers/storage/overlay/6174b9c24d87d29db3e80d2b602f3ef36e0494c62af6de3bdadcf43b9d47d5aa/merged 2024-05-06T12:12:22.639 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/containers/storage/overlay/49acec9b92e1443f63933c7d81d95f638cbdf6119295a601dea3e0f2c78477b7/merged 2024-05-06T12:12:22.639 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/containers/storage/overlay/4eb76f020089d822ee4f920d293ebe3ab92be64db05184f627247348ae63b976/merged 2024-05-06T12:12:22.639 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns 2024-05-06T12:12:22.639 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-06T12:12:22.639 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-06T12:12:22.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-06T12:12:22.640 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2024-05-06T12:12:22.657 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:2024-05-06T12:12:22.654+0000 7f44690c5300 -1 init, newargv = 0x559135d4a920 newargc=15 2024-05-06T12:12:22.657 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:ceph-fuse[58948]: starting ceph client 2024-05-06T12:12:22.664 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:ceph-fuse[58948]: starting fuse 2024-05-06T12:12:22.672 INFO:teuthology.orchestra.run.smithi190.stdout:112 2024-05-06T12:12:22.672 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [112] 2024-05-06T12:12:22.673 DEBUG:teuthology.orchestra.run.smithi190:> sudo stdin-killer -- python3 -c ' 2024-05-06T12:12:22.673 DEBUG:teuthology.orchestra.run.smithi190:> import glob 2024-05-06T12:12:22.673 DEBUG:teuthology.orchestra.run.smithi190:> import re 2024-05-06T12:12:22.673 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-05-06T12:12:22.673 DEBUG:teuthology.orchestra.run.smithi190:> import subprocess 2024-05-06T12:12:22.673 DEBUG:teuthology.orchestra.run.smithi190:> 2024-05-06T12:12:22.673 DEBUG:teuthology.orchestra.run.smithi190:> def _find_admin_socket(client_name): 2024-05-06T12:12:22.673 DEBUG:teuthology.orchestra.run.smithi190:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-05-06T12:12:22.673 DEBUG:teuthology.orchestra.run.smithi190:> files = glob.glob(asok_path) 2024-05-06T12:12:22.673 DEBUG:teuthology.orchestra.run.smithi190:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-05-06T12:12:22.673 DEBUG:teuthology.orchestra.run.smithi190:> 2024-05-06T12:12:22.673 DEBUG:teuthology.orchestra.run.smithi190:> # Given a non-glob path, it better be there 2024-05-06T12:12:22.673 DEBUG:teuthology.orchestra.run.smithi190:> if "*" not in asok_path: 2024-05-06T12:12:22.673 DEBUG:teuthology.orchestra.run.smithi190:> assert(len(files) == 1) 2024-05-06T12:12:22.673 DEBUG:teuthology.orchestra.run.smithi190:> return files[0] 2024-05-06T12:12:22.674 DEBUG:teuthology.orchestra.run.smithi190:> 2024-05-06T12:12:22.674 DEBUG:teuthology.orchestra.run.smithi190:> for f in files: 2024-05-06T12:12:22.674 DEBUG:teuthology.orchestra.run.smithi190:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-06T12:12:22.674 DEBUG:teuthology.orchestra.run.smithi190:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-06T12:12:22.674 DEBUG:teuthology.orchestra.run.smithi190:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-06T12:12:22.674 DEBUG:teuthology.orchestra.run.smithi190:> contents = proc_f.read() 2024-05-06T12:12:22.674 DEBUG:teuthology.orchestra.run.smithi190:> if mountpoint in contents: 2024-05-06T12:12:22.674 DEBUG:teuthology.orchestra.run.smithi190:> return f 2024-05-06T12:12:22.674 DEBUG:teuthology.orchestra.run.smithi190:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-06T12:12:22.674 DEBUG:teuthology.orchestra.run.smithi190:> 2024-05-06T12:12:22.674 DEBUG:teuthology.orchestra.run.smithi190:> print(_find_admin_socket("client.1")) 2024-05-06T12:12:22.674 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-05-06T12:12:22.789 INFO:teuthology.orchestra.run.smithi190.stdout:/var/run/ceph/ceph-client.1.58948.asok 2024-05-06T12:12:22.792 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-06T12:12:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-06T12:12:22.800 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.58948.asok 2024-05-06T12:12:22.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-06T12:12:22.800 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.58948.asok status 2024-05-06T12:12:22.930 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-05-06T12:12:22.930 INFO:teuthology.orchestra.run.smithi190.stdout: "metadata": { 2024-05-06T12:12:22.930 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_sha1": "e81df4201214a93810b2eaca609e475a6319e220", 2024-05-06T12:12:22.930 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_version": "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)", 2024-05-06T12:12:22.930 INFO:teuthology.orchestra.run.smithi190.stdout: "entity_id": "1", 2024-05-06T12:12:22.930 INFO:teuthology.orchestra.run.smithi190.stdout: "hostname": "smithi190", 2024-05-06T12:12:22.931 INFO:teuthology.orchestra.run.smithi190.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-05-06T12:12:22.931 INFO:teuthology.orchestra.run.smithi190.stdout: "pid": "58948", 2024-05-06T12:12:22.931 INFO:teuthology.orchestra.run.smithi190.stdout: "root": "/" 2024-05-06T12:12:22.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-06T12:12:22.931 INFO:teuthology.orchestra.run.smithi190.stdout: "dentry_count": 0, 2024-05-06T12:12:22.931 INFO:teuthology.orchestra.run.smithi190.stdout: "dentry_pinned_count": 0, 2024-05-06T12:12:22.931 INFO:teuthology.orchestra.run.smithi190.stdout: "id": 24315, 2024-05-06T12:12:22.931 INFO:teuthology.orchestra.run.smithi190.stdout: "inst": { 2024-05-06T12:12:22.931 INFO:teuthology.orchestra.run.smithi190.stdout: "name": { 2024-05-06T12:12:22.931 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "client", 2024-05-06T12:12:22.931 INFO:teuthology.orchestra.run.smithi190.stdout: "num": 24315 2024-05-06T12:12:22.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-06T12:12:22.931 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": { 2024-05-06T12:12:22.931 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "v1", 2024-05-06T12:12:22.931 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": "192.168.0.1:0", 2024-05-06T12:12:22.932 INFO:teuthology.orchestra.run.smithi190.stdout: "nonce": 3983235695 2024-05-06T12:12:22.932 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-06T12:12:22.932 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-06T12:12:22.932 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": { 2024-05-06T12:12:22.932 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "v1", 2024-05-06T12:12:22.932 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": "192.168.0.1:0", 2024-05-06T12:12:22.932 INFO:teuthology.orchestra.run.smithi190.stdout: "nonce": 3983235695 2024-05-06T12:12:22.932 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-06T12:12:22.932 INFO:teuthology.orchestra.run.smithi190.stdout: "inst_str": "client.24315 192.168.0.1:0/3983235695", 2024-05-06T12:12:22.932 INFO:teuthology.orchestra.run.smithi190.stdout: "addr_str": "192.168.0.1:0/3983235695", 2024-05-06T12:12:22.932 INFO:teuthology.orchestra.run.smithi190.stdout: "inode_count": 1, 2024-05-06T12:12:22.932 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_epoch": 18, 2024-05-06T12:12:22.932 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_epoch": 46, 2024-05-06T12:12:22.932 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_epoch_barrier": 0, 2024-05-06T12:12:22.932 INFO:teuthology.orchestra.run.smithi190.stdout: "blocklisted": false, 2024-05-06T12:12:22.933 INFO:teuthology.orchestra.run.smithi190.stdout: "fs_name": "cephfs" 2024-05-06T12:12:22.933 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-05-06T12:12:22.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-06T12:12:22.939 DEBUG:teuthology.orchestra.run.smithi092:> stat --file-system '--printf=%T 2024-05-06T12:12:22.939 DEBUG:teuthology.orchestra.run.smithi092:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-06T12:12:22.960 INFO:teuthology.orchestra.run.smithi092.stdout:fuseblk 2024-05-06T12:12:22.960 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-06T12:12:22.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-06T12:12:22.960 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-06T12:12:23.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-06T12:12:23.035 DEBUG:teuthology.orchestra.run.smithi190:> stat --file-system '--printf=%T 2024-05-06T12:12:23.035 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1 2024-05-06T12:12:23.051 INFO:teuthology.orchestra.run.smithi190.stdout:fuseblk 2024-05-06T12:12:23.051 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-05-06T12:12:23.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-06T12:12:23.051 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-06T12:12:23.112 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:22 smithi190 ceph-mon[33798]: pgmap v193: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 255 B/s wr, 4 op/s 2024-05-06T12:12:23.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:22 smithi092 ceph-mon[26954]: pgmap v193: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 255 B/s wr, 4 op/s 2024-05-06T12:12:25.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:24 smithi092 ceph-mon[26954]: pgmap v194: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 341 B/s wr, 4 op/s 2024-05-06T12:12:25.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:24 smithi190 ceph-mon[33798]: pgmap v194: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 341 B/s wr, 4 op/s 2024-05-06T12:12:26.400 INFO:teuthology.run_tasks:Running task print... 2024-05-06T12:12:26.413 INFO:teuthology.task.print:**** done client 2024-05-06T12:12:26.413 INFO:teuthology.run_tasks:Running task parallel... 2024-05-06T12:12:26.428 INFO:teuthology.task.parallel:starting parallel... 2024-05-06T12:12:26.429 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-05-06T12:12:26.429 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-05-06T12:12:26.430 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi092.front.sepia.ceph.com 2024-05-06T12:12:26.430 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -e sha1=70ce9539b62a15b9de429c1f0e91a1e012b099aa -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs false || true' 2024-05-06T12:12:26.430 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-05-06T12:12:26.431 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-05-06T12:12:26.434 INFO:tasks.workunit:Pulling workunits from ref 70ce9539b62a15b9de429c1f0e91a1e012b099aa 2024-05-06T12:12:26.435 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-06T12:12:26.435 INFO:tasks.workunit:timeout=3h 2024-05-06T12:12:26.435 INFO:tasks.workunit:cleanup=True 2024-05-06T12:12:26.435 DEBUG:teuthology.orchestra.run.smithi092:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-06T12:12:26.454 INFO:teuthology.orchestra.run.smithi092.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-06T12:12:26.454 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-05-06T12:12:26.455 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 86h/134d Inode: 1 Links: 2 2024-05-06T12:12:26.455 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-06T12:12:26.455 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:fusefs_t:s0 2024-05-06T12:12:26.456 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-05-06T12:12:26.456 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-05-06 12:11:49.865687995 +0000 2024-05-06T12:12:26.456 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-05-06 12:12:23.029023978 +0000 2024-05-06T12:12:26.456 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2024-05-06T12:12:26.456 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-06T12:12:26.456 DEBUG:teuthology.orchestra.run.smithi092:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-06T12:12:26.527 DEBUG:teuthology.orchestra.run.smithi190:> stat -- /home/ubuntu/cephtest/mnt.1 2024-05-06T12:12:26.548 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-05-06T12:12:26.549 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-05-06T12:12:26.549 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 70h/112d Inode: 1 Links: 3 2024-05-06T12:12:26.549 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-06T12:12:26.549 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:fusefs_t:s0 2024-05-06T12:12:26.549 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-05-06T12:12:26.549 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-05-06 12:12:26.518020924 +0000 2024-05-06T12:12:26.550 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-05-06 12:12:26.518020924 +0000 2024-05-06T12:12:26.550 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: - 2024-05-06T12:12:26.550 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-05-06T12:12:26.550 DEBUG:teuthology.orchestra.run.smithi190:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-05-06T12:12:26.622 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 70ce9539b62a15b9de429c1f0e91a1e012b099aa 2024-05-06T12:12:26.622 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 70ce9539b62a15b9de429c1f0e91a1e012b099aa 2024-05-06T12:12:26.638 INFO:tasks.workunit.client.0.smithi092.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-06T12:12:26.679 INFO:tasks.workunit.client.1.smithi190.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-05-06T12:12:26.712 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:12:27.123 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:26 smithi092 ceph-mon[26954]: pgmap v195: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 426 B/s wr, 4 op/s 2024-05-06T12:12:27.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:26 smithi190 ceph-mon[33798]: pgmap v195: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 426 B/s wr, 4 op/s 2024-05-06T12:12:28.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.297+0000 7fe8f2fd6700 1 -- 172.21.15.92:0/1845044444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8ec0fbcb0 msgr2=0x7fe8ec0fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:28.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.297+0000 7fe8f2fd6700 1 --2- 172.21.15.92:0/1845044444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8ec0fbcb0 0x7fe8ec0fc0c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0004940 tx=0x7fe8e0009e30 comp rx=0 tx=0).stop 2024-05-06T12:12:28.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.297+0000 7fe8f2fd6700 1 -- 172.21.15.92:0/1845044444 shutdown_connections 2024-05-06T12:12:28.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.297+0000 7fe8f2fd6700 1 --2- 172.21.15.92:0/1845044444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8ec0fcf60 0x7fe8ec0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:28.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.297+0000 7fe8f2fd6700 1 --2- 172.21.15.92:0/1845044444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8ec0fbcb0 0x7fe8ec0fc0c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:28.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.297+0000 7fe8f2fd6700 1 -- 172.21.15.92:0/1845044444 >> 172.21.15.92:0/1845044444 conn(0x7fe8ec0f7430 msgr2=0x7fe8ec0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:28.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.297+0000 7fe8f2fd6700 1 -- 172.21.15.92:0/1845044444 shutdown_connections 2024-05-06T12:12:28.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.297+0000 7fe8f2fd6700 1 -- 172.21.15.92:0/1845044444 wait complete. 2024-05-06T12:12:28.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.298+0000 7fe8f2fd6700 1 Processor -- start 2024-05-06T12:12:28.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.298+0000 7fe8f2fd6700 1 -- start start 2024-05-06T12:12:28.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.298+0000 7fe8f2fd6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8ec0fbcb0 0x7fe8ec110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:28.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.298+0000 7fe8f2fd6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8ec0fcf60 0x7fe8ec112c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:28.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.298+0000 7fe8f2fd6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec066330 con 0x7fe8ec0fbcb0 2024-05-06T12:12:28.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.298+0000 7fe8f2fd6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec066470 con 0x7fe8ec0fcf60 2024-05-06T12:12:28.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.298+0000 7fe8f0d72700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8ec0fbcb0 0x7fe8ec110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:28.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.298+0000 7fe8f0d72700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8ec0fbcb0 0x7fe8ec110720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44636/0 (socket says 172.21.15.92:44636) 2024-05-06T12:12:28.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.298+0000 7fe8f0d72700 1 -- 172.21.15.92:0/2804527689 learned_addr learned my addr 172.21.15.92:0/2804527689 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:12:28.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.298+0000 7fe8ebfff700 1 --2- 172.21.15.92:0/2804527689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8ec0fcf60 0x7fe8ec112c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:28.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.299+0000 7fe8f0d72700 1 -- 172.21.15.92:0/2804527689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8ec0fcf60 msgr2=0x7fe8ec112c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:28.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.299+0000 7fe8f0d72700 1 --2- 172.21.15.92:0/2804527689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8ec0fcf60 0x7fe8ec112c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:28.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.299+0000 7fe8f0d72700 1 -- 172.21.15.92:0/2804527689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e0005040 con 0x7fe8ec0fbcb0 2024-05-06T12:12:28.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.299+0000 7fe8f0d72700 1 --2- 172.21.15.92:0/2804527689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8ec0fbcb0 0x7fe8ec110720 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009150 tx=0x7fe8e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:28.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.299+0000 7fe8e97fa700 1 -- 172.21.15.92:0/2804527689 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8e000e040 con 0x7fe8ec0fbcb0 2024-05-06T12:12:28.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.299+0000 7fe8e97fa700 1 -- 172.21.15.92:0/2804527689 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe8e000bbf0 con 0x7fe8ec0fbcb0 2024-05-06T12:12:28.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.299+0000 7fe8f2fd6700 1 -- 172.21.15.92:0/2804527689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8ec1131b0 con 0x7fe8ec0fbcb0 2024-05-06T12:12:28.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.299+0000 7fe8e97fa700 1 -- 172.21.15.92:0/2804527689 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8e001c630 con 0x7fe8ec0fbcb0 2024-05-06T12:12:28.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.299+0000 7fe8f2fd6700 1 -- 172.21.15.92:0/2804527689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ec113770 con 0x7fe8ec0fbcb0 2024-05-06T12:12:28.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.300+0000 7fe8f2fd6700 1 -- 172.21.15.92:0/2804527689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8ec05f5c0 con 0x7fe8ec0fbcb0 2024-05-06T12:12:28.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.301+0000 7fe8e97fa700 1 -- 172.21.15.92:0/2804527689 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fe8e000bd60 con 0x7fe8ec0fbcb0 2024-05-06T12:12:28.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.301+0000 7fe8e97fa700 1 --2- 172.21.15.92:0/2804527689 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fe8dc0692b0 0x7fe8dc06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:28.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.301+0000 7fe8e97fa700 1 -- 172.21.15.92:0/2804527689 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fe8e0088550 con 0x7fe8ec0fbcb0 2024-05-06T12:12:28.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.301+0000 7fe8ebfff700 1 --2- 172.21.15.92:0/2804527689 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fe8dc0692b0 0x7fe8dc06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:28.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.301+0000 7fe8ebfff700 1 --2- 172.21.15.92:0/2804527689 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fe8dc0692b0 0x7fe8dc06b760 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8004f40 tx=0x7fe8d800b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:28.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.305+0000 7fe8e97fa700 1 -- 172.21.15.92:0/2804527689 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe8e00548e0 con 0x7fe8ec0fbcb0 2024-05-06T12:12:28.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.494+0000 7fe8f2fd6700 1 -- 172.21.15.92:0/2804527689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7fe8ec04bed0 con 0x7fe8ec0fbcb0 2024-05-06T12:12:28.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:28.495+0000 7fe8e97fa700 1 -- 172.21.15.92:0/2804527689 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=-22 unrecognized config option 'mgr/orchestrator/fail_fs' v21)=-22 unrecognized config option 'mgr/orchestrator/fail_fs' v21) v1 ==== 179+0+0 (secure 0 0 0) 0x7fe8e0013e60 con 0x7fe8ec0fbcb0 2024-05-06T12:12:28.499 INFO:teuthology.orchestra.run.smithi092.stderr:Error EINVAL: unrecognized config option 'mgr/orchestrator/fail_fs' 2024-05-06T12:12:29.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:28 smithi190 ceph-mon[33798]: pgmap v196: 65 pgs: 65 active+clean; 582 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-06T12:12:29.126 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-05-06T12:12:29.126 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi092.front.sepia.ceph.com 2024-05-06T12:12:29.126 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -e sha1=70ce9539b62a15b9de429c1f0e91a1e012b099aa -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-05-06T12:12:29.188 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:28 smithi092 ceph-mon[26954]: pgmap v196: 65 pgs: 65 active+clean; 582 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-06T12:12:29.431 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:12:31.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.013+0000 7fe3ad355700 1 -- 172.21.15.92:0/1972846746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3a80fe420 msgr2=0x7fe3a80fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:31.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.013+0000 7fe3ad355700 1 --2- 172.21.15.92:0/1972846746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3a80fe420 0x7fe3a80fe830 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe398004940 tx=0x7fe398009e30 comp rx=0 tx=0).stop 2024-05-06T12:12:31.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.013+0000 7fe3ad355700 1 -- 172.21.15.92:0/1972846746 shutdown_connections 2024-05-06T12:12:31.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.013+0000 7fe3ad355700 1 --2- 172.21.15.92:0/1972846746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3a80fed70 0x7fe3a80faeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:31.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.013+0000 7fe3ad355700 1 --2- 172.21.15.92:0/1972846746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3a80fe420 0x7fe3a80fe830 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:31.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.013+0000 7fe3ad355700 1 -- 172.21.15.92:0/1972846746 >> 172.21.15.92:0/1972846746 conn(0x7fe3a80f6be0 msgr2=0x7fe3a80f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:31.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.013+0000 7fe3ad355700 1 -- 172.21.15.92:0/1972846746 shutdown_connections 2024-05-06T12:12:31.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.013+0000 7fe3ad355700 1 -- 172.21.15.92:0/1972846746 wait complete. 2024-05-06T12:12:31.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.013+0000 7fe3ad355700 1 Processor -- start 2024-05-06T12:12:31.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.013+0000 7fe3ad355700 1 -- start start 2024-05-06T12:12:31.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.014+0000 7fe3ad355700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3a80fe420 0x7fe3a810d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:31.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.014+0000 7fe3ad355700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3a80fed70 0x7fe3a810de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:31.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.014+0000 7fe3ad355700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a810e4e0 con 0x7fe3a80fe420 2024-05-06T12:12:31.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.014+0000 7fe3ad355700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a8112e10 con 0x7fe3a80fed70 2024-05-06T12:12:31.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.014+0000 7fe3a6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3a80fe420 0x7fe3a810d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:31.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.014+0000 7fe3a6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3a80fe420 0x7fe3a810d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44652/0 (socket says 172.21.15.92:44652) 2024-05-06T12:12:31.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.014+0000 7fe3a6ffd700 1 -- 172.21.15.92:0/1902535036 learned_addr learned my addr 172.21.15.92:0/1902535036 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:12:31.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.014+0000 7fe3a6ffd700 1 -- 172.21.15.92:0/1902535036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3a80fed70 msgr2=0x7fe3a810de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:31.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.014+0000 7fe3a6ffd700 1 --2- 172.21.15.92:0/1902535036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3a80fed70 0x7fe3a810de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:31.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.014+0000 7fe3a6ffd700 1 -- 172.21.15.92:0/1902535036 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe398005040 con 0x7fe3a80fe420 2024-05-06T12:12:31.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.014+0000 7fe3a6ffd700 1 --2- 172.21.15.92:0/1902535036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3a80fe420 0x7fe3a810d910 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe39800b970 tx=0x7fe398009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:31.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.014+0000 7fe39ffff700 1 -- 172.21.15.92:0/1902535036 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe39800e040 con 0x7fe3a80fe420 2024-05-06T12:12:31.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.014+0000 7fe3ad355700 1 -- 172.21.15.92:0/1902535036 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a8113010 con 0x7fe3a80fe420 2024-05-06T12:12:31.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.014+0000 7fe39ffff700 1 -- 172.21.15.92:0/1902535036 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe398004290 con 0x7fe3a80fe420 2024-05-06T12:12:31.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.014+0000 7fe39ffff700 1 -- 172.21.15.92:0/1902535036 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe39801c630 con 0x7fe3a80fe420 2024-05-06T12:12:31.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.014+0000 7fe3ad355700 1 -- 172.21.15.92:0/1902535036 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a8113580 con 0x7fe3a80fe420 2024-05-06T12:12:31.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.015+0000 7fe3ad355700 1 -- 172.21.15.92:0/1902535036 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a804c710 con 0x7fe3a80fe420 2024-05-06T12:12:31.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.016+0000 7fe39ffff700 1 -- 172.21.15.92:0/1902535036 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fe3980139f0 con 0x7fe3a80fe420 2024-05-06T12:12:31.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.016+0000 7fe39ffff700 1 --2- 172.21.15.92:0/1902535036 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fe3940692a0 0x7fe39406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:31.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.016+0000 7fe39ffff700 1 -- 172.21.15.92:0/1902535036 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fe398088490 con 0x7fe3a80fe420 2024-05-06T12:12:31.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.016+0000 7fe39e7fc700 1 --2- 172.21.15.92:0/1902535036 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fe3940692a0 0x7fe39406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:31.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.017+0000 7fe39e7fc700 1 --2- 172.21.15.92:0/1902535036 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fe3940692a0 0x7fe39406b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe390006fd0 tx=0x7fe390006db0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:31.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.021+0000 7fe39ffff700 1 -- 172.21.15.92:0/1902535036 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe3980582a0 con 0x7fe3a80fe420 2024-05-06T12:12:31.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.206+0000 7fe3ad355700 1 -- 172.21.15.92:0/1902535036 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7fe3a81137f0 con 0x7fe3a80fe420 2024-05-06T12:12:31.210 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:30 smithi092 ceph-mon[26954]: pgmap v197: 65 pgs: 65 active+clean; 582 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 597 B/s wr, 4 op/s 2024-05-06T12:12:31.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.207+0000 7fe39ffff700 1 -- 172.21.15.92:0/1902535036 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7fe398057e30 con 0x7fe3a80fe420 2024-05-06T12:12:31.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.209+0000 7fe3ad355700 1 -- 172.21.15.92:0/1902535036 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fe3940692a0 msgr2=0x7fe39406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:31.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.209+0000 7fe3ad355700 1 --2- 172.21.15.92:0/1902535036 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fe3940692a0 0x7fe39406b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe390006fd0 tx=0x7fe390006db0 comp rx=0 tx=0).stop 2024-05-06T12:12:31.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.209+0000 7fe3ad355700 1 -- 172.21.15.92:0/1902535036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3a80fe420 msgr2=0x7fe3a810d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:31.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.209+0000 7fe3ad355700 1 --2- 172.21.15.92:0/1902535036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3a80fe420 0x7fe3a810d910 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe39800b970 tx=0x7fe398009e30 comp rx=0 tx=0).stop 2024-05-06T12:12:31.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.209+0000 7fe3ad355700 1 -- 172.21.15.92:0/1902535036 shutdown_connections 2024-05-06T12:12:31.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.209+0000 7fe3ad355700 1 --2- 172.21.15.92:0/1902535036 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fe3940692a0 0x7fe39406b750 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:31.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.209+0000 7fe3ad355700 1 --2- 172.21.15.92:0/1902535036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3a80fe420 0x7fe3a810d910 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:31.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.209+0000 7fe3ad355700 1 --2- 172.21.15.92:0/1902535036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3a80fed70 0x7fe3a810de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:31.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.209+0000 7fe3ad355700 1 -- 172.21.15.92:0/1902535036 >> 172.21.15.92:0/1902535036 conn(0x7fe3a80f6be0 msgr2=0x7fe3a80f81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:31.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.209+0000 7fe3ad355700 1 -- 172.21.15.92:0/1902535036 shutdown_connections 2024-05-06T12:12:31.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:31.209+0000 7fe3ad355700 1 -- 172.21.15.92:0/1902535036 wait complete. 2024-05-06T12:12:31.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:30 smithi190 ceph-mon[33798]: pgmap v197: 65 pgs: 65 active+clean; 582 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 597 B/s wr, 4 op/s 2024-05-06T12:12:31.908 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -e sha1=70ce9539b62a15b9de429c1f0e91a1e012b099aa -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-05-06T12:12:32.202 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:12:33.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:32 smithi092 ceph-mon[26954]: pgmap v198: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 938 B/s wr, 5 op/s 2024-05-06T12:12:33.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:32 smithi190 ceph-mon[33798]: pgmap v198: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 938 B/s wr, 5 op/s 2024-05-06T12:12:33.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.782+0000 7f965d8ef700 1 -- 172.21.15.92:0/3058402110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96580fffb0 msgr2=0x7f9658102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:33.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.782+0000 7f965d8ef700 1 --2- 172.21.15.92:0/3058402110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96580fffb0 0x7f9658102390 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9648004940 tx=0x7f9648009e30 comp rx=0 tx=0).stop 2024-05-06T12:12:33.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.782+0000 7f965d8ef700 1 -- 172.21.15.92:0/3058402110 shutdown_connections 2024-05-06T12:12:33.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.782+0000 7f965d8ef700 1 --2- 172.21.15.92:0/3058402110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96580fffb0 0x7f9658102390 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:33.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.782+0000 7f965d8ef700 1 --2- 172.21.15.92:0/3058402110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9658066e90 0x7f96580ffa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:33.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.782+0000 7f965d8ef700 1 -- 172.21.15.92:0/3058402110 >> 172.21.15.92:0/3058402110 conn(0x7f96580f7440 msgr2=0x7f96580f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:33.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.782+0000 7f965d8ef700 1 -- 172.21.15.92:0/3058402110 shutdown_connections 2024-05-06T12:12:33.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.782+0000 7f965d8ef700 1 -- 172.21.15.92:0/3058402110 wait complete. 2024-05-06T12:12:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.783+0000 7f965d8ef700 1 Processor -- start 2024-05-06T12:12:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.783+0000 7f965d8ef700 1 -- start start 2024-05-06T12:12:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.783+0000 7f965d8ef700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9658066e90 0x7f965810d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.783+0000 7f965d8ef700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96580fffb0 0x7f965810df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.783+0000 7f965d8ef700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965810e5c0 con 0x7f9658066e90 2024-05-06T12:12:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.783+0000 7f965d8ef700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9658112f60 con 0x7f96580fffb0 2024-05-06T12:12:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.783+0000 7f9656ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9658066e90 0x7f965810d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.783+0000 7f9656ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9658066e90 0x7f965810d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45834/0 (socket says 172.21.15.92:45834) 2024-05-06T12:12:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.783+0000 7f9656ffd700 1 -- 172.21.15.92:0/943351351 learned_addr learned my addr 172.21.15.92:0/943351351 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:12:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.784+0000 7f96567fc700 1 --2- 172.21.15.92:0/943351351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96580fffb0 0x7f965810df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.784+0000 7f9656ffd700 1 -- 172.21.15.92:0/943351351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96580fffb0 msgr2=0x7f965810df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.784+0000 7f9656ffd700 1 --2- 172.21.15.92:0/943351351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96580fffb0 0x7f965810df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.784+0000 7f9656ffd700 1 -- 172.21.15.92:0/943351351 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9648005040 con 0x7f9658066e90 2024-05-06T12:12:33.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.784+0000 7f96567fc700 1 --2- 172.21.15.92:0/943351351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96580fffb0 0x7f965810df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-06T12:12:33.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.784+0000 7f9656ffd700 1 --2- 172.21.15.92:0/943351351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9658066e90 0x7f965810d9f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f964c00b790 tx=0x7f964c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:33.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.784+0000 7f963f7fe700 1 -- 172.21.15.92:0/943351351 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f964c00f6f0 con 0x7f9658066e90 2024-05-06T12:12:33.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.784+0000 7f963f7fe700 1 -- 172.21.15.92:0/943351351 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f964c00fd20 con 0x7f9658066e90 2024-05-06T12:12:33.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.784+0000 7f965d8ef700 1 -- 172.21.15.92:0/943351351 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96581131c0 con 0x7f9658066e90 2024-05-06T12:12:33.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.784+0000 7f963f7fe700 1 -- 172.21.15.92:0/943351351 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f964c0103a0 con 0x7f9658066e90 2024-05-06T12:12:33.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.784+0000 7f965d8ef700 1 -- 172.21.15.92:0/943351351 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9658113730 con 0x7f9658066e90 2024-05-06T12:12:33.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.785+0000 7f965d8ef700 1 -- 172.21.15.92:0/943351351 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f965804c710 con 0x7f9658066e90 2024-05-06T12:12:33.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.785+0000 7f963f7fe700 1 -- 172.21.15.92:0/943351351 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f964c011400 con 0x7f9658066e90 2024-05-06T12:12:33.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.786+0000 7f963f7fe700 1 --2- 172.21.15.92:0/943351351 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f96400692a0 0x7f964006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:33.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.786+0000 7f963f7fe700 1 -- 172.21.15.92:0/943351351 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f964c087090 con 0x7f9658066e90 2024-05-06T12:12:33.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.786+0000 7f96567fc700 1 --2- 172.21.15.92:0/943351351 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f96400692a0 0x7f964006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:33.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.786+0000 7f96567fc700 1 --2- 172.21.15.92:0/943351351 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f96400692a0 0x7f964006b750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f964800b970 tx=0x7f9648009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:33.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.790+0000 7f963f7fe700 1 -- 172.21.15.92:0/943351351 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f964c056d80 con 0x7f9658066e90 2024-05-06T12:12:33.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.982+0000 7f965d8ef700 1 -- 172.21.15.92:0/943351351 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f9658105950 con 0x7f9658066e90 2024-05-06T12:12:33.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.982+0000 7f963f7fe700 1 -- 172.21.15.92:0/943351351 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f964c056910 con 0x7f9658066e90 2024-05-06T12:12:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.984+0000 7f965d8ef700 1 -- 172.21.15.92:0/943351351 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f96400692a0 msgr2=0x7f964006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.984+0000 7f965d8ef700 1 --2- 172.21.15.92:0/943351351 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f96400692a0 0x7f964006b750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f964800b970 tx=0x7f9648009e30 comp rx=0 tx=0).stop 2024-05-06T12:12:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.984+0000 7f965d8ef700 1 -- 172.21.15.92:0/943351351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9658066e90 msgr2=0x7f965810d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.984+0000 7f965d8ef700 1 --2- 172.21.15.92:0/943351351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9658066e90 0x7f965810d9f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f964c00b790 tx=0x7f964c009220 comp rx=0 tx=0).stop 2024-05-06T12:12:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.984+0000 7f965d8ef700 1 -- 172.21.15.92:0/943351351 shutdown_connections 2024-05-06T12:12:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.984+0000 7f965d8ef700 1 --2- 172.21.15.92:0/943351351 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f96400692a0 0x7f964006b750 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.984+0000 7f965d8ef700 1 --2- 172.21.15.92:0/943351351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9658066e90 0x7f965810d9f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.984+0000 7f965d8ef700 1 --2- 172.21.15.92:0/943351351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96580fffb0 0x7f965810df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:33.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.984+0000 7f965d8ef700 1 -- 172.21.15.92:0/943351351 >> 172.21.15.92:0/943351351 conn(0x7f96580f7440 msgr2=0x7f96580f7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:33.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.984+0000 7f965d8ef700 1 -- 172.21.15.92:0/943351351 shutdown_connections 2024-05-06T12:12:33.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:33.984+0000 7f965d8ef700 1 -- 172.21.15.92:0/943351351 wait complete. 2024-05-06T12:12:34.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:33 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:12:34.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:33 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:12:34.772 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -e sha1=70ce9539b62a15b9de429c1f0e91a1e012b099aa -- bash -c 'ceph config set global log_to_journald false --force' 2024-05-06T12:12:35.039 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:12:35.140 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:34 smithi092 ceph-mon[26954]: pgmap v199: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 767 B/s wr, 4 op/s 2024-05-06T12:12:35.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:34 smithi190 ceph-mon[33798]: pgmap v199: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 767 B/s wr, 4 op/s 2024-05-06T12:12:36.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.691+0000 7fd576811700 1 -- 172.21.15.92:0/1875199125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5700fbcc0 msgr2=0x7fd5700fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:36.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.691+0000 7fd576811700 1 --2- 172.21.15.92:0/1875199125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5700fbcc0 0x7fd5700fc0d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd560004940 tx=0x7fd560009e30 comp rx=0 tx=0).stop 2024-05-06T12:12:36.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.692+0000 7fd576811700 1 -- 172.21.15.92:0/1875199125 shutdown_connections 2024-05-06T12:12:36.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.692+0000 7fd576811700 1 --2- 172.21.15.92:0/1875199125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5700fcf70 0x7fd5700fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:36.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.692+0000 7fd576811700 1 --2- 172.21.15.92:0/1875199125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5700fbcc0 0x7fd5700fc0d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:36.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.692+0000 7fd576811700 1 -- 172.21.15.92:0/1875199125 >> 172.21.15.92:0/1875199125 conn(0x7fd5700f7440 msgr2=0x7fd5700f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:36.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.692+0000 7fd576811700 1 -- 172.21.15.92:0/1875199125 shutdown_connections 2024-05-06T12:12:36.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.692+0000 7fd576811700 1 -- 172.21.15.92:0/1875199125 wait complete. 2024-05-06T12:12:36.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.692+0000 7fd576811700 1 Processor -- start 2024-05-06T12:12:36.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.692+0000 7fd576811700 1 -- start start 2024-05-06T12:12:36.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.693+0000 7fd576811700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5700fbcc0 0x7fd570066cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:36.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.693+0000 7fd576811700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5700fcf70 0x7fd5701127c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:36.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.693+0000 7fd576811700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd570112d00 con 0x7fd5700fcf70 2024-05-06T12:12:36.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.693+0000 7fd576811700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd570112e40 con 0x7fd5700fbcc0 2024-05-06T12:12:36.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.693+0000 7fd56f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5700fcf70 0x7fd5701127c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:36.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.693+0000 7fd56f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5700fcf70 0x7fd5701127c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45854/0 (socket says 172.21.15.92:45854) 2024-05-06T12:12:36.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.693+0000 7fd56f7fe700 1 -- 172.21.15.92:0/3155547192 learned_addr learned my addr 172.21.15.92:0/3155547192 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:12:36.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.693+0000 7fd56ffff700 1 --2- 172.21.15.92:0/3155547192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5700fbcc0 0x7fd570066cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:36.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.693+0000 7fd56f7fe700 1 -- 172.21.15.92:0/3155547192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5700fbcc0 msgr2=0x7fd570066cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:36.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.693+0000 7fd56f7fe700 1 --2- 172.21.15.92:0/3155547192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5700fbcc0 0x7fd570066cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:36.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.693+0000 7fd56f7fe700 1 -- 172.21.15.92:0/3155547192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd560005040 con 0x7fd5700fcf70 2024-05-06T12:12:36.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.693+0000 7fd56ffff700 1 --2- 172.21.15.92:0/3155547192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5700fbcc0 0x7fd570066cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-06T12:12:36.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.693+0000 7fd56f7fe700 1 --2- 172.21.15.92:0/3155547192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5700fcf70 0x7fd5701127c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd56400c8c0 tx=0x7fd5640044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:36.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.693+0000 7fd56cff9700 1 -- 172.21.15.92:0/3155547192 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd56400a980 con 0x7fd5700fcf70 2024-05-06T12:12:36.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.693+0000 7fd56cff9700 1 -- 172.21.15.92:0/3155547192 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd56400ce60 con 0x7fd5700fcf70 2024-05-06T12:12:36.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.693+0000 7fd576811700 1 -- 172.21.15.92:0/3155547192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5701130a0 con 0x7fd5700fcf70 2024-05-06T12:12:36.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.693+0000 7fd56cff9700 1 -- 172.21.15.92:0/3155547192 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5640115e0 con 0x7fd5700fcf70 2024-05-06T12:12:36.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.693+0000 7fd576811700 1 -- 172.21.15.92:0/3155547192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd570113660 con 0x7fd5700fcf70 2024-05-06T12:12:36.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.694+0000 7fd576811700 1 -- 172.21.15.92:0/3155547192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd57004c710 con 0x7fd5700fcf70 2024-05-06T12:12:36.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.695+0000 7fd56cff9700 1 -- 172.21.15.92:0/3155547192 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fd564012400 con 0x7fd5700fcf70 2024-05-06T12:12:36.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.695+0000 7fd56cff9700 1 --2- 172.21.15.92:0/3155547192 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fd5580692a0 0x7fd55806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:36.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.695+0000 7fd56ffff700 1 --2- 172.21.15.92:0/3155547192 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fd5580692a0 0x7fd55806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:36.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.695+0000 7fd56cff9700 1 -- 172.21.15.92:0/3155547192 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fd5640872f0 con 0x7fd5700fcf70 2024-05-06T12:12:36.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.696+0000 7fd56ffff700 1 --2- 172.21.15.92:0/3155547192 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fd5580692a0 0x7fd55806b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd560000c00 tx=0x7fd560009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:36.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.700+0000 7fd56cff9700 1 -- 172.21.15.92:0/3155547192 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd564057060 con 0x7fd5700fcf70 2024-05-06T12:12:36.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.885+0000 7fd576811700 1 -- 172.21.15.92:0/3155547192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7fd5701047b0 con 0x7fd5700fcf70 2024-05-06T12:12:36.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.886+0000 7fd56cff9700 1 -- 172.21.15.92:0/3155547192 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7fd5640126b0 con 0x7fd5700fcf70 2024-05-06T12:12:36.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.888+0000 7fd576811700 1 -- 172.21.15.92:0/3155547192 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fd5580692a0 msgr2=0x7fd55806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:36.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.888+0000 7fd576811700 1 --2- 172.21.15.92:0/3155547192 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fd5580692a0 0x7fd55806b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd560000c00 tx=0x7fd560009e30 comp rx=0 tx=0).stop 2024-05-06T12:12:36.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.888+0000 7fd576811700 1 -- 172.21.15.92:0/3155547192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5700fcf70 msgr2=0x7fd5701127c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:36.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.888+0000 7fd576811700 1 --2- 172.21.15.92:0/3155547192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5700fcf70 0x7fd5701127c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd56400c8c0 tx=0x7fd5640044c0 comp rx=0 tx=0).stop 2024-05-06T12:12:36.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.888+0000 7fd576811700 1 -- 172.21.15.92:0/3155547192 shutdown_connections 2024-05-06T12:12:36.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.888+0000 7fd576811700 1 --2- 172.21.15.92:0/3155547192 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fd5580692a0 0x7fd55806b750 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:36.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.888+0000 7fd576811700 1 --2- 172.21.15.92:0/3155547192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5700fbcc0 0x7fd570066cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:36.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.888+0000 7fd576811700 1 --2- 172.21.15.92:0/3155547192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5700fcf70 0x7fd5701127c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:36.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.888+0000 7fd576811700 1 -- 172.21.15.92:0/3155547192 >> 172.21.15.92:0/3155547192 conn(0x7fd5700f7440 msgr2=0x7fd5700f8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:36.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.888+0000 7fd576811700 1 -- 172.21.15.92:0/3155547192 shutdown_connections 2024-05-06T12:12:36.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:36.888+0000 7fd576811700 1 -- 172.21.15.92:0/3155547192 wait complete. 2024-05-06T12:12:37.261 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:36 smithi092 ceph-mon[26954]: pgmap v200: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-05-06T12:12:37.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:36 smithi190 ceph-mon[33798]: pgmap v200: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-05-06T12:12:37.657 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -e sha1=70ce9539b62a15b9de429c1f0e91a1e012b099aa -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-05-06T12:12:37.913 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:12:39.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:38 smithi190 ceph-mon[33798]: pgmap v201: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-06T12:12:39.198 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:38 smithi092 ceph-mon[26954]: pgmap v201: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-06T12:12:39.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.413+0000 7f69d592c700 1 -- 172.21.15.92:0/3232990354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69d00fbcb0 msgr2=0x7f69d00fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:39.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.413+0000 7f69d592c700 1 --2- 172.21.15.92:0/3232990354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69d00fbcb0 0x7f69d00fc0c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f69c0004940 tx=0x7f69c0009e30 comp rx=0 tx=0).stop 2024-05-06T12:12:39.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.413+0000 7f69d592c700 1 -- 172.21.15.92:0/3232990354 shutdown_connections 2024-05-06T12:12:39.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.413+0000 7f69d592c700 1 --2- 172.21.15.92:0/3232990354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69d00fcf60 0x7f69d00fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:39.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.413+0000 7f69d592c700 1 --2- 172.21.15.92:0/3232990354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69d00fbcb0 0x7f69d00fc0c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:39.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.413+0000 7f69d592c700 1 -- 172.21.15.92:0/3232990354 >> 172.21.15.92:0/3232990354 conn(0x7f69d00f7430 msgr2=0x7f69d00f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:39.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.413+0000 7f69d592c700 1 -- 172.21.15.92:0/3232990354 shutdown_connections 2024-05-06T12:12:39.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.413+0000 7f69d592c700 1 -- 172.21.15.92:0/3232990354 wait complete. 2024-05-06T12:12:39.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.414+0000 7f69d592c700 1 Processor -- start 2024-05-06T12:12:39.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.414+0000 7f69d592c700 1 -- start start 2024-05-06T12:12:39.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.414+0000 7f69d592c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69d00fbcb0 0x7f69d01106c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:39.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.414+0000 7f69d592c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69d00fcf60 0x7f69d0112c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:39.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.414+0000 7f69d592c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d0066390 con 0x7f69d00fbcb0 2024-05-06T12:12:39.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.414+0000 7f69d592c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d00664d0 con 0x7f69d00fcf60 2024-05-06T12:12:39.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.414+0000 7f69ceffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69d00fbcb0 0x7f69d01106c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:39.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.414+0000 7f69ceffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69d00fbcb0 0x7f69d01106c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45880/0 (socket says 172.21.15.92:45880) 2024-05-06T12:12:39.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.414+0000 7f69ceffd700 1 -- 172.21.15.92:0/2898463119 learned_addr learned my addr 172.21.15.92:0/2898463119 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:12:39.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.414+0000 7f69ce7fc700 1 --2- 172.21.15.92:0/2898463119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69d00fcf60 0x7f69d0112c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:39.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.414+0000 7f69ceffd700 1 -- 172.21.15.92:0/2898463119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69d00fcf60 msgr2=0x7f69d0112c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:39.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.414+0000 7f69ceffd700 1 --2- 172.21.15.92:0/2898463119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69d00fcf60 0x7f69d0112c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:39.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.414+0000 7f69ceffd700 1 -- 172.21.15.92:0/2898463119 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c0005040 con 0x7f69d00fbcb0 2024-05-06T12:12:39.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.414+0000 7f69ce7fc700 1 --2- 172.21.15.92:0/2898463119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69d00fcf60 0x7f69d0112c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-06T12:12:39.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.415+0000 7f69ceffd700 1 --2- 172.21.15.92:0/2898463119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69d00fbcb0 0x7f69d01106c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f69c000b3b0 tx=0x7f69c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:39.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.415+0000 7f69b77fe700 1 -- 172.21.15.92:0/2898463119 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69c000e040 con 0x7f69d00fbcb0 2024-05-06T12:12:39.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.415+0000 7f69b77fe700 1 -- 172.21.15.92:0/2898463119 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f69c000bcc0 con 0x7f69d00fbcb0 2024-05-06T12:12:39.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.415+0000 7f69d592c700 1 -- 172.21.15.92:0/2898463119 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69d01131b0 con 0x7f69d00fbcb0 2024-05-06T12:12:39.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.415+0000 7f69b77fe700 1 -- 172.21.15.92:0/2898463119 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69c001c630 con 0x7f69d00fbcb0 2024-05-06T12:12:39.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.415+0000 7f69d592c700 1 -- 172.21.15.92:0/2898463119 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69d0113770 con 0x7f69d00fbcb0 2024-05-06T12:12:39.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.416+0000 7f69d592c700 1 -- 172.21.15.92:0/2898463119 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69d005f5c0 con 0x7f69d00fbcb0 2024-05-06T12:12:39.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.416+0000 7f69b77fe700 1 -- 172.21.15.92:0/2898463119 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f69c0013980 con 0x7f69d00fbcb0 2024-05-06T12:12:39.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.417+0000 7f69b77fe700 1 --2- 172.21.15.92:0/2898463119 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f69b80692a0 0x7f69b806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:39.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.417+0000 7f69b77fe700 1 -- 172.21.15.92:0/2898463119 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f69c0088020 con 0x7f69d00fbcb0 2024-05-06T12:12:39.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.417+0000 7f69ce7fc700 1 --2- 172.21.15.92:0/2898463119 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f69b80692a0 0x7f69b806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:39.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.417+0000 7f69ce7fc700 1 --2- 172.21.15.92:0/2898463119 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f69b80692a0 0x7f69b806b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f69c40041d0 tx=0x7f69c4009040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:39.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.421+0000 7f69b77fe700 1 -- 172.21.15.92:0/2898463119 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f69c0054280 con 0x7f69d00fbcb0 2024-05-06T12:12:39.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.629+0000 7f69d592c700 1 -- 172.21.15.92:0/2898463119 --> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:70ce9539b62a15b9de429c1f0e91a1e012b099aa", "target": ["mon-mgr", ""]}) v1 -- 0x7f69d01013c0 con 0x7f69b80692a0 2024-05-06T12:12:39.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.634+0000 7f69b77fe700 1 -- 172.21.15.92:0/2898463119 <== mgr.14210 v2:172.21.15.92:6800/3045905943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f69d01013c0 con 0x7f69b80692a0 2024-05-06T12:12:39.637 INFO:teuthology.orchestra.run.smithi092.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:70ce9539b62a15b9de429c1f0e91a1e012b099aa 2024-05-06T12:12:39.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.636+0000 7f69d592c700 1 -- 172.21.15.92:0/2898463119 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f69b80692a0 msgr2=0x7f69b806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:39.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.636+0000 7f69d592c700 1 --2- 172.21.15.92:0/2898463119 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f69b80692a0 0x7f69b806b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f69c40041d0 tx=0x7f69c4009040 comp rx=0 tx=0).stop 2024-05-06T12:12:39.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.636+0000 7f69d592c700 1 -- 172.21.15.92:0/2898463119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69d00fbcb0 msgr2=0x7f69d01106c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:39.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.636+0000 7f69d592c700 1 --2- 172.21.15.92:0/2898463119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69d00fbcb0 0x7f69d01106c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f69c000b3b0 tx=0x7f69c0009e30 comp rx=0 tx=0).stop 2024-05-06T12:12:39.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.636+0000 7f69d592c700 1 -- 172.21.15.92:0/2898463119 shutdown_connections 2024-05-06T12:12:39.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.636+0000 7f69d592c700 1 --2- 172.21.15.92:0/2898463119 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f69b80692a0 0x7f69b806b750 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:39.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.636+0000 7f69d592c700 1 --2- 172.21.15.92:0/2898463119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69d00fbcb0 0x7f69d01106c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:39.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.636+0000 7f69d592c700 1 --2- 172.21.15.92:0/2898463119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69d00fcf60 0x7f69d0112c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:39.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.636+0000 7f69d592c700 1 -- 172.21.15.92:0/2898463119 >> 172.21.15.92:0/2898463119 conn(0x7f69d00f7430 msgr2=0x7f69d00f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:39.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.636+0000 7f69d592c700 1 -- 172.21.15.92:0/2898463119 shutdown_connections 2024-05-06T12:12:39.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:39.636+0000 7f69d592c700 1 -- 172.21.15.92:0/2898463119 wait complete. 2024-05-06T12:12:40.422 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-05-06T12:12:40.422 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi092.front.sepia.ceph.com 2024-05-06T12:12:40.422 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -e sha1=70ce9539b62a15b9de429c1f0e91a1e012b099aa -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-05-06T12:12:40.683 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:12:40.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:40 smithi092 ceph-mon[26954]: from='client.14548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:70ce9539b62a15b9de429c1f0e91a1e012b099aa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:12:40.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:40 smithi092 ceph-mon[26954]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:70ce9539b62a15b9de429c1f0e91a1e012b099aa 2024-05-06T12:12:40.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:40 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:40.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:40 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:12:40.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:40 smithi092 ceph-mon[26954]: pgmap v202: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 682 B/s wr, 4 op/s 2024-05-06T12:12:40.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:40 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:12:40.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:40 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:12:40.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:40 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:40.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:40 smithi092 ceph-mon[26954]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:70ce9539b62a15b9de429c1f0e91a1e012b099aa 2024-05-06T12:12:41.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:40 smithi190 ceph-mon[33798]: from='client.14548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:70ce9539b62a15b9de429c1f0e91a1e012b099aa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:12:41.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:40 smithi190 ceph-mon[33798]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:70ce9539b62a15b9de429c1f0e91a1e012b099aa 2024-05-06T12:12:41.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:40 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:41.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:40 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:12:41.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:40 smithi190 ceph-mon[33798]: pgmap v202: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 682 B/s wr, 4 op/s 2024-05-06T12:12:41.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:40 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:12:41.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:40 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:12:41.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:40 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:12:41.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:40 smithi190 ceph-mon[33798]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:70ce9539b62a15b9de429c1f0e91a1e012b099aa 2024-05-06T12:12:42.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.548+0000 7f38d5ed3700 1 -- 172.21.15.92:0/1916095950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d00ff050 msgr2=0x7f38d00ff460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:42.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.548+0000 7f38d5ed3700 1 --2- 172.21.15.92:0/1916095950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d00ff050 0x7f38d00ff460 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f38c0004940 tx=0x7f38c0009e30 comp rx=0 tx=0).stop 2024-05-06T12:12:42.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.548+0000 7f38d5ed3700 1 -- 172.21.15.92:0/1916095950 shutdown_connections 2024-05-06T12:12:42.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.548+0000 7f38d5ed3700 1 --2- 172.21.15.92:0/1916095950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38d0100300 0x7f38d0100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:42.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.548+0000 7f38d5ed3700 1 --2- 172.21.15.92:0/1916095950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d00ff050 0x7f38d00ff460 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:42.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.548+0000 7f38d5ed3700 1 -- 172.21.15.92:0/1916095950 >> 172.21.15.92:0/1916095950 conn(0x7f38d00fa8f0 msgr2=0x7f38d00fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:42.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.548+0000 7f38d5ed3700 1 -- 172.21.15.92:0/1916095950 shutdown_connections 2024-05-06T12:12:42.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.548+0000 7f38d5ed3700 1 -- 172.21.15.92:0/1916095950 wait complete. 2024-05-06T12:12:42.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.549+0000 7f38d5ed3700 1 Processor -- start 2024-05-06T12:12:42.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.549+0000 7f38d5ed3700 1 -- start start 2024-05-06T12:12:42.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.549+0000 7f38d5ed3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d00ff050 0x7f38d010fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:42.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.549+0000 7f38d5ed3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38d0100300 0x7f38d0110140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:42.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.549+0000 7f38d5ed3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d0110740 con 0x7f38d00ff050 2024-05-06T12:12:42.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.549+0000 7f38d5ed3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d0110f90 con 0x7f38d0100300 2024-05-06T12:12:42.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.549+0000 7f38cf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d00ff050 0x7f38d010fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:42.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.549+0000 7f38cf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d00ff050 0x7f38d010fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58924/0 (socket says 172.21.15.92:58924) 2024-05-06T12:12:42.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.549+0000 7f38cf7fe700 1 -- 172.21.15.92:0/2314716293 learned_addr learned my addr 172.21.15.92:0/2314716293 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:12:42.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.549+0000 7f38ceffd700 1 --2- 172.21.15.92:0/2314716293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38d0100300 0x7f38d0110140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:42.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.549+0000 7f38cf7fe700 1 -- 172.21.15.92:0/2314716293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38d0100300 msgr2=0x7f38d0110140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:42.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.549+0000 7f38cf7fe700 1 --2- 172.21.15.92:0/2314716293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38d0100300 0x7f38d0110140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:42.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.549+0000 7f38cf7fe700 1 -- 172.21.15.92:0/2314716293 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c0005040 con 0x7f38d00ff050 2024-05-06T12:12:42.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.550+0000 7f38cf7fe700 1 --2- 172.21.15.92:0/2314716293 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d00ff050 0x7f38d010fc00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f38c000b3b0 tx=0x7f38c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:42.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.550+0000 7f38b7fff700 1 -- 172.21.15.92:0/2314716293 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38c000e040 con 0x7f38d00ff050 2024-05-06T12:12:42.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.550+0000 7f38b7fff700 1 -- 172.21.15.92:0/2314716293 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f38c000bbf0 con 0x7f38d00ff050 2024-05-06T12:12:42.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.550+0000 7f38d5ed3700 1 -- 172.21.15.92:0/2314716293 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d0111210 con 0x7f38d00ff050 2024-05-06T12:12:42.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.550+0000 7f38b7fff700 1 -- 172.21.15.92:0/2314716293 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38c001c630 con 0x7f38d00ff050 2024-05-06T12:12:42.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.550+0000 7f38d5ed3700 1 -- 172.21.15.92:0/2314716293 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d0111750 con 0x7f38d00ff050 2024-05-06T12:12:42.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.551+0000 7f38d5ed3700 1 -- 172.21.15.92:0/2314716293 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d005f5c0 con 0x7f38d00ff050 2024-05-06T12:12:42.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.551+0000 7f38b7fff700 1 -- 172.21.15.92:0/2314716293 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f38c000bd60 con 0x7f38d00ff050 2024-05-06T12:12:42.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.552+0000 7f38b7fff700 1 --2- 172.21.15.92:0/2314716293 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f38b80692a0 0x7f38b806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:42.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.552+0000 7f38b7fff700 1 -- 172.21.15.92:0/2314716293 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f38c0087f10 con 0x7f38d00ff050 2024-05-06T12:12:42.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.552+0000 7f38ceffd700 1 --2- 172.21.15.92:0/2314716293 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f38b80692a0 0x7f38b806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:42.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.552+0000 7f38ceffd700 1 --2- 172.21.15.92:0/2314716293 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f38b80692a0 0x7f38b806b750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f38c4007fd0 tx=0x7f38c4009040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:42.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.556+0000 7f38b7fff700 1 -- 172.21.15.92:0/2314716293 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f38c0057c80 con 0x7f38d00ff050 2024-05-06T12:12:42.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.788+0000 7f38d5ed3700 1 -- 172.21.15.92:0/2314716293 --> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f38d0111a30 con 0x7f38b80692a0 2024-05-06T12:12:42.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.789+0000 7f38b7fff700 1 -- 172.21.15.92:0/2314716293 <== mgr.14210 v2:172.21.15.92:6800/3045905943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f38d0111a30 con 0x7f38b80692a0 2024-05-06T12:12:42.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.791+0000 7f38d5ed3700 1 -- 172.21.15.92:0/2314716293 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f38b80692a0 msgr2=0x7f38b806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:42.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.791+0000 7f38d5ed3700 1 --2- 172.21.15.92:0/2314716293 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f38b80692a0 0x7f38b806b750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f38c4007fd0 tx=0x7f38c4009040 comp rx=0 tx=0).stop 2024-05-06T12:12:42.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.791+0000 7f38d5ed3700 1 -- 172.21.15.92:0/2314716293 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d00ff050 msgr2=0x7f38d010fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:42.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.791+0000 7f38d5ed3700 1 --2- 172.21.15.92:0/2314716293 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d00ff050 0x7f38d010fc00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f38c000b3b0 tx=0x7f38c0009e30 comp rx=0 tx=0).stop 2024-05-06T12:12:42.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.791+0000 7f38d5ed3700 1 -- 172.21.15.92:0/2314716293 shutdown_connections 2024-05-06T12:12:42.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.791+0000 7f38d5ed3700 1 --2- 172.21.15.92:0/2314716293 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f38b80692a0 0x7f38b806b750 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:42.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.791+0000 7f38d5ed3700 1 --2- 172.21.15.92:0/2314716293 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d00ff050 0x7f38d010fc00 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:42.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.791+0000 7f38d5ed3700 1 --2- 172.21.15.92:0/2314716293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38d0100300 0x7f38d0110140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:42.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.791+0000 7f38d5ed3700 1 -- 172.21.15.92:0/2314716293 >> 172.21.15.92:0/2314716293 conn(0x7f38d00fa8f0 msgr2=0x7f38d00fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:42.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.791+0000 7f38d5ed3700 1 -- 172.21.15.92:0/2314716293 shutdown_connections 2024-05-06T12:12:42.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.791+0000 7f38d5ed3700 1 -- 172.21.15.92:0/2314716293 wait complete. 2024-05-06T12:12:42.804 INFO:teuthology.orchestra.run.smithi092.stdout:true 2024-05-06T12:12:42.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.922+0000 7febe4511700 1 -- 172.21.15.92:0/3625145060 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febdc066260 msgr2=0x7febdc0ffa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:42.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.922+0000 7febe4511700 1 --2- 172.21.15.92:0/3625145060 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febdc066260 0x7febdc0ffa00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7febd8004940 tx=0x7febd8009e30 comp rx=0 tx=0).stop 2024-05-06T12:12:42.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.922+0000 7febe4511700 1 -- 172.21.15.92:0/3625145060 shutdown_connections 2024-05-06T12:12:42.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.922+0000 7febe4511700 1 --2- 172.21.15.92:0/3625145060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febdc0fff40 0x7febdc102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:42.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.922+0000 7febe4511700 1 --2- 172.21.15.92:0/3625145060 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febdc066260 0x7febdc0ffa00 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:42.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.922+0000 7febe4511700 1 -- 172.21.15.92:0/3625145060 >> 172.21.15.92:0/3625145060 conn(0x7febdc0f7430 msgr2=0x7febdc0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:42.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.923+0000 7febe4511700 1 -- 172.21.15.92:0/3625145060 shutdown_connections 2024-05-06T12:12:42.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.923+0000 7febe4511700 1 -- 172.21.15.92:0/3625145060 wait complete. 2024-05-06T12:12:42.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.923+0000 7febe4511700 1 Processor -- start 2024-05-06T12:12:42.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.923+0000 7febe4511700 1 -- start start 2024-05-06T12:12:42.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.923+0000 7febe4511700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febdc066260 0x7febdc10da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:42.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.923+0000 7febe4511700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febdc0fff40 0x7febdc10df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:42.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.924+0000 7febe4511700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc10e590 con 0x7febdc0fff40 2024-05-06T12:12:42.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.924+0000 7febe4511700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc10ede0 con 0x7febdc066260 2024-05-06T12:12:42.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.924+0000 7febe1aac700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febdc0fff40 0x7febdc10df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:42.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.924+0000 7febe1aac700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febdc0fff40 0x7febdc10df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58952/0 (socket says 172.21.15.92:58952) 2024-05-06T12:12:42.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.924+0000 7febe1aac700 1 -- 172.21.15.92:0/2238991744 learned_addr learned my addr 172.21.15.92:0/2238991744 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:12:42.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.924+0000 7febe22ad700 1 --2- 172.21.15.92:0/2238991744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febdc066260 0x7febdc10da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:42.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.924+0000 7febe1aac700 1 -- 172.21.15.92:0/2238991744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febdc066260 msgr2=0x7febdc10da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:42.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.924+0000 7febe1aac700 1 --2- 172.21.15.92:0/2238991744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febdc066260 0x7febdc10da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:42.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.924+0000 7febe1aac700 1 -- 172.21.15.92:0/2238991744 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd8005040 con 0x7febdc0fff40 2024-05-06T12:12:42.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.924+0000 7febe1aac700 1 --2- 172.21.15.92:0/2238991744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febdc0fff40 0x7febdc10df90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7febcc002a50 tx=0x7febcc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:42.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.924+0000 7febd2ffd700 1 -- 172.21.15.92:0/2238991744 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febcc007cc0 con 0x7febdc0fff40 2024-05-06T12:12:42.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.924+0000 7febe4511700 1 -- 172.21.15.92:0/2238991744 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febdc10f0c0 con 0x7febdc0fff40 2024-05-06T12:12:42.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.924+0000 7febd2ffd700 1 -- 172.21.15.92:0/2238991744 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febcc00f040 con 0x7febdc0fff40 2024-05-06T12:12:42.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.924+0000 7febd2ffd700 1 -- 172.21.15.92:0/2238991744 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febcc00da70 con 0x7febdc0fff40 2024-05-06T12:12:42.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.924+0000 7febe4511700 1 -- 172.21.15.92:0/2238991744 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febdc10f600 con 0x7febdc0fff40 2024-05-06T12:12:42.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.926+0000 7febe4511700 1 -- 172.21.15.92:0/2238991744 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febdc05f5c0 con 0x7febdc0fff40 2024-05-06T12:12:42.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.926+0000 7febd2ffd700 1 -- 172.21.15.92:0/2238991744 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7febcc00dc10 con 0x7febdc0fff40 2024-05-06T12:12:42.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.926+0000 7febd2ffd700 1 --2- 172.21.15.92:0/2238991744 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7febc80692b0 0x7febc806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:42.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.926+0000 7febd2ffd700 1 -- 172.21.15.92:0/2238991744 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7febcc086880 con 0x7febdc0fff40 2024-05-06T12:12:42.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.926+0000 7febe22ad700 1 --2- 172.21.15.92:0/2238991744 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7febc80692b0 0x7febc806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:42.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.927+0000 7febe22ad700 1 --2- 172.21.15.92:0/2238991744 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7febc80692b0 0x7febc806b760 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7febd8000c00 tx=0x7febd8009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:42.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:42.932+0000 7febd2ffd700 1 -- 172.21.15.92:0/2238991744 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7febcc056570 con 0x7febdc0fff40 2024-05-06T12:12:43.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.147+0000 7febe4511700 1 -- 172.21.15.92:0/2238991744 --> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7febdc10f930 con 0x7febc80692b0 2024-05-06T12:12:43.150 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:42 smithi092 ceph-mon[26954]: pgmap v203: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 1023 B/s wr, 5 op/s 2024-05-06T12:12:43.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.148+0000 7febd2ffd700 1 -- 172.21.15.92:0/2238991744 <== mgr.14210 v2:172.21.15.92:6800/3045905943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7febdc10f930 con 0x7febc80692b0 2024-05-06T12:12:43.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.151+0000 7febe4511700 1 -- 172.21.15.92:0/2238991744 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7febc80692b0 msgr2=0x7febc806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:43.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.151+0000 7febe4511700 1 --2- 172.21.15.92:0/2238991744 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7febc80692b0 0x7febc806b760 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7febd8000c00 tx=0x7febd8009e30 comp rx=0 tx=0).stop 2024-05-06T12:12:43.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.151+0000 7febe4511700 1 -- 172.21.15.92:0/2238991744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febdc0fff40 msgr2=0x7febdc10df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:43.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.151+0000 7febe4511700 1 --2- 172.21.15.92:0/2238991744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febdc0fff40 0x7febdc10df90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7febcc002a50 tx=0x7febcc0046b0 comp rx=0 tx=0).stop 2024-05-06T12:12:43.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.151+0000 7febe4511700 1 -- 172.21.15.92:0/2238991744 shutdown_connections 2024-05-06T12:12:43.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.151+0000 7febe4511700 1 --2- 172.21.15.92:0/2238991744 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7febc80692b0 0x7febc806b760 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:43.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.151+0000 7febe4511700 1 --2- 172.21.15.92:0/2238991744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febdc066260 0x7febdc10da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:43.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.151+0000 7febe4511700 1 --2- 172.21.15.92:0/2238991744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febdc0fff40 0x7febdc10df90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:43.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.151+0000 7febe4511700 1 -- 172.21.15.92:0/2238991744 >> 172.21.15.92:0/2238991744 conn(0x7febdc0f7430 msgr2=0x7febdc0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:43.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.151+0000 7febe4511700 1 -- 172.21.15.92:0/2238991744 shutdown_connections 2024-05-06T12:12:43.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.151+0000 7febe4511700 1 -- 172.21.15.92:0/2238991744 wait complete. 2024-05-06T12:12:43.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.255+0000 7f7bc0ae0700 1 -- 172.21.15.92:0/1309070774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bbc066260 msgr2=0x7f7bbc0ffa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:43.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.255+0000 7f7bc0ae0700 1 --2- 172.21.15.92:0/1309070774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bbc066260 0x7f7bbc0ffa00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7bac004940 tx=0x7f7bac009e30 comp rx=0 tx=0).stop 2024-05-06T12:12:43.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.255+0000 7f7bc0ae0700 1 -- 172.21.15.92:0/1309070774 shutdown_connections 2024-05-06T12:12:43.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.255+0000 7f7bc0ae0700 1 --2- 172.21.15.92:0/1309070774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bbc0fff40 0x7f7bbc102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:43.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.255+0000 7f7bc0ae0700 1 --2- 172.21.15.92:0/1309070774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bbc066260 0x7f7bbc0ffa00 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:43.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.255+0000 7f7bc0ae0700 1 -- 172.21.15.92:0/1309070774 >> 172.21.15.92:0/1309070774 conn(0x7f7bbc0f7430 msgr2=0x7f7bbc0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:43.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.255+0000 7f7bc0ae0700 1 -- 172.21.15.92:0/1309070774 shutdown_connections 2024-05-06T12:12:43.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.255+0000 7f7bc0ae0700 1 -- 172.21.15.92:0/1309070774 wait complete. 2024-05-06T12:12:43.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.256+0000 7f7bc0ae0700 1 Processor -- start 2024-05-06T12:12:43.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.256+0000 7f7bc0ae0700 1 -- start start 2024-05-06T12:12:43.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.256+0000 7f7bc0ae0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bbc066260 0x7f7bbc10da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:43.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.256+0000 7f7bc0ae0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bbc0fff40 0x7f7bbc10df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:43.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.256+0000 7f7bc0ae0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc10e570 con 0x7f7bbc066260 2024-05-06T12:12:43.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.256+0000 7f7bc0ae0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc10edc0 con 0x7f7bbc0fff40 2024-05-06T12:12:43.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.256+0000 7f7bba59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bbc066260 0x7f7bbc10da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:43.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.256+0000 7f7bba59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bbc066260 0x7f7bbc10da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58960/0 (socket says 172.21.15.92:58960) 2024-05-06T12:12:43.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.256+0000 7f7bba59c700 1 -- 172.21.15.92:0/352023319 learned_addr learned my addr 172.21.15.92:0/352023319 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:12:43.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.256+0000 7f7bb9d9b700 1 --2- 172.21.15.92:0/352023319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bbc0fff40 0x7f7bbc10df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:43.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.256+0000 7f7bba59c700 1 -- 172.21.15.92:0/352023319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bbc0fff40 msgr2=0x7f7bbc10df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:43.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.256+0000 7f7bba59c700 1 --2- 172.21.15.92:0/352023319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bbc0fff40 0x7f7bbc10df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:43.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.256+0000 7f7bba59c700 1 -- 172.21.15.92:0/352023319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bac005040 con 0x7f7bbc066260 2024-05-06T12:12:43.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.257+0000 7f7bba59c700 1 --2- 172.21.15.92:0/352023319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bbc066260 0x7f7bbc10da30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7bac000c00 tx=0x7f7bac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:43.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.257+0000 7f7baaffd700 1 -- 172.21.15.92:0/352023319 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bac00e040 con 0x7f7bbc066260 2024-05-06T12:12:43.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.257+0000 7f7bc0ae0700 1 -- 172.21.15.92:0/352023319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bbc10f040 con 0x7f7bbc066260 2024-05-06T12:12:43.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.257+0000 7f7baaffd700 1 -- 172.21.15.92:0/352023319 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7bac00bcc0 con 0x7f7bbc066260 2024-05-06T12:12:43.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.257+0000 7f7baaffd700 1 -- 172.21.15.92:0/352023319 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bac01c630 con 0x7f7bbc066260 2024-05-06T12:12:43.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.257+0000 7f7bc0ae0700 1 -- 172.21.15.92:0/352023319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bbc10f580 con 0x7f7bbc066260 2024-05-06T12:12:43.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.258+0000 7f7bc0ae0700 1 -- 172.21.15.92:0/352023319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bbc05f5c0 con 0x7f7bbc066260 2024-05-06T12:12:43.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.259+0000 7f7baaffd700 1 -- 172.21.15.92:0/352023319 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f7bac013980 con 0x7f7bbc066260 2024-05-06T12:12:43.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.259+0000 7f7baaffd700 1 --2- 172.21.15.92:0/352023319 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f7ba40692a0 0x7f7ba406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:43.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.259+0000 7f7baaffd700 1 -- 172.21.15.92:0/352023319 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f7bac088560 con 0x7f7bbc066260 2024-05-06T12:12:43.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.259+0000 7f7bb9d9b700 1 --2- 172.21.15.92:0/352023319 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f7ba40692a0 0x7f7ba406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:43.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.260+0000 7f7bb9d9b700 1 --2- 172.21.15.92:0/352023319 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f7ba40692a0 0x7f7ba406b750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0004990 tx=0x7f7bb0009ee0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:43.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.263+0000 7f7baaffd700 1 -- 172.21.15.92:0/352023319 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7bac058420 con 0x7f7bbc066260 2024-05-06T12:12:43.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:42 smithi190 ceph-mon[33798]: pgmap v203: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 1023 B/s wr, 5 op/s 2024-05-06T12:12:43.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.479+0000 7f7bc0ae0700 1 -- 172.21.15.92:0/352023319 --> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7bbc10f950 con 0x7f7ba40692a0 2024-05-06T12:12:43.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.494+0000 7f7baaffd700 1 -- 172.21.15.92:0/352023319 <== mgr.14210 v2:172.21.15.92:6800/3045905943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7f7bbc10f950 con 0x7f7ba40692a0 2024-05-06T12:12:43.497 INFO:teuthology.orchestra.run.smithi092.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-06T12:12:43.497 INFO:teuthology.orchestra.run.smithi092.stdout:alertmanager.smithi092 smithi092 *:9093,9094 running (4m) 27s ago 6m 25.7M - 0.25.0 c8568f914cd2 dab380a7352e 2024-05-06T12:12:43.497 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi092 smithi092 running (6m) 27s ago 6m 6660k - 17.2.7-925-ge81df420 96d5ae7fd4b4 95dc7e14bd9a 2024-05-06T12:12:43.497 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi190 smithi190 running (5m) 33s ago 5m 6665k - 17.2.7-925-ge81df420 96d5ae7fd4b4 6cf97785406f 2024-05-06T12:12:43.497 INFO:teuthology.orchestra.run.smithi092.stdout:grafana.smithi092 smithi092 *:3000 running (4m) 27s ago 5m 84.6M - 9.4.7 954c08fa6188 841f34687c12 2024-05-06T12:12:43.498 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.ijsqgp smithi092 running (55s) 27s ago 54s 11.4M - 17.2.7-925-ge81df420 96d5ae7fd4b4 b4c2ed6d2c82 2024-05-06T12:12:43.498 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.rosbll smithi092 running (47s) 27s ago 47s 13.7M - 17.2.7-925-ge81df420 96d5ae7fd4b4 5f5426cd1bf5 2024-05-06T12:12:43.498 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.byjvgf smithi190 running (52s) 33s ago 51s 16.7M - 17.2.7-925-ge81df420 96d5ae7fd4b4 24581d2e1c5d 2024-05-06T12:12:43.498 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.idzoff smithi190 running (44s) 33s ago 44s 13.3M - 17.2.7-925-ge81df420 96d5ae7fd4b4 20f2641696ef 2024-05-06T12:12:43.498 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi092.gyzeqb smithi092 *:9283 running (7m) 27s ago 7m 480M - 17.2.7-925-ge81df420 96d5ae7fd4b4 820ba9ba8a82 2024-05-06T12:12:43.498 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi190.kczxkh smithi190 *:8443,9283 running (5m) 33s ago 5m 431M - 17.2.7-925-ge81df420 96d5ae7fd4b4 fb57adc25b2b 2024-05-06T12:12:43.498 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi092 smithi092 running (7m) 27s ago 7m 42.8M 2048M 17.2.7-925-ge81df420 96d5ae7fd4b4 df5b322e9719 2024-05-06T12:12:43.498 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi190 smithi190 running (5m) 33s ago 5m 32.5M 2048M 17.2.7-925-ge81df420 96d5ae7fd4b4 912ded072601 2024-05-06T12:12:43.498 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi092 smithi092 *:9100 running (6m) 27s ago 6m 15.8M - 1.5.0 0da6a335fe13 86cf73a245ab 2024-05-06T12:12:43.498 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi190 smithi190 *:9100 running (5m) 33s ago 5m 16.0M - 1.5.0 0da6a335fe13 da5c2a030c69 2024-05-06T12:12:43.499 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 smithi092 running (4m) 27s ago 4m 47.0M 4288M 17.2.7-925-ge81df420 96d5ae7fd4b4 ce84e7af9084 2024-05-06T12:12:43.499 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 smithi092 running (4m) 27s ago 4m 48.8M 4288M 17.2.7-925-ge81df420 96d5ae7fd4b4 8ab870f57eb2 2024-05-06T12:12:43.499 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 smithi092 running (3m) 27s ago 3m 46.0M 4288M 17.2.7-925-ge81df420 96d5ae7fd4b4 1e6ed04a0688 2024-05-06T12:12:43.499 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 smithi190 running (3m) 33s ago 3m 43.8M 5312M 17.2.7-925-ge81df420 96d5ae7fd4b4 52f9521286d5 2024-05-06T12:12:43.499 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 smithi190 running (2m) 33s ago 2m 42.4M 5312M 17.2.7-925-ge81df420 96d5ae7fd4b4 3a7da09e4c48 2024-05-06T12:12:43.499 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 smithi190 running (2m) 33s ago 2m 41.7M 5312M 17.2.7-925-ge81df420 96d5ae7fd4b4 e7452c6b5cec 2024-05-06T12:12:43.499 INFO:teuthology.orchestra.run.smithi092.stdout:prometheus.smithi092 smithi092 *:9095 running (4m) 27s ago 5m 56.7M - 2.43.0 a07b618ecd1d ef92f3572328 2024-05-06T12:12:43.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.496+0000 7f7bc0ae0700 1 -- 172.21.15.92:0/352023319 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f7ba40692a0 msgr2=0x7f7ba406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:43.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.496+0000 7f7bc0ae0700 1 --2- 172.21.15.92:0/352023319 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f7ba40692a0 0x7f7ba406b750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0004990 tx=0x7f7bb0009ee0 comp rx=0 tx=0).stop 2024-05-06T12:12:43.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.496+0000 7f7bc0ae0700 1 -- 172.21.15.92:0/352023319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bbc066260 msgr2=0x7f7bbc10da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:43.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.496+0000 7f7bc0ae0700 1 --2- 172.21.15.92:0/352023319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bbc066260 0x7f7bbc10da30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7bac000c00 tx=0x7f7bac009e30 comp rx=0 tx=0).stop 2024-05-06T12:12:43.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.496+0000 7f7bc0ae0700 1 -- 172.21.15.92:0/352023319 shutdown_connections 2024-05-06T12:12:43.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.496+0000 7f7bc0ae0700 1 --2- 172.21.15.92:0/352023319 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f7ba40692a0 0x7f7ba406b750 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:43.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.496+0000 7f7bc0ae0700 1 --2- 172.21.15.92:0/352023319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bbc066260 0x7f7bbc10da30 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:43.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.496+0000 7f7bc0ae0700 1 --2- 172.21.15.92:0/352023319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bbc0fff40 0x7f7bbc10df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:43.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.496+0000 7f7bc0ae0700 1 -- 172.21.15.92:0/352023319 >> 172.21.15.92:0/352023319 conn(0x7f7bbc0f7430 msgr2=0x7f7bbc0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:43.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.496+0000 7f7bc0ae0700 1 -- 172.21.15.92:0/352023319 shutdown_connections 2024-05-06T12:12:43.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.496+0000 7f7bc0ae0700 1 -- 172.21.15.92:0/352023319 wait complete. 2024-05-06T12:12:43.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.601+0000 7f3a9b5b6700 1 -- 172.21.15.92:0/390121482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a940fdf70 msgr2=0x7f3a940fe3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:43.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.601+0000 7f3a9b5b6700 1 --2- 172.21.15.92:0/390121482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a940fdf70 0x7f3a940fe3e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3a84004940 tx=0x7f3a84009e30 comp rx=0 tx=0).stop 2024-05-06T12:12:43.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.601+0000 7f3a9b5b6700 1 -- 172.21.15.92:0/390121482 shutdown_connections 2024-05-06T12:12:43.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.601+0000 7f3a9b5b6700 1 --2- 172.21.15.92:0/390121482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a940fdf70 0x7f3a940fe3e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:43.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.601+0000 7f3a9b5b6700 1 --2- 172.21.15.92:0/390121482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a940fce00 0x7f3a940fd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:43.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.601+0000 7f3a9b5b6700 1 -- 172.21.15.92:0/390121482 >> 172.21.15.92:0/390121482 conn(0x7f3a940f84e0 msgr2=0x7f3a940fa8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:43.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.601+0000 7f3a9b5b6700 1 -- 172.21.15.92:0/390121482 shutdown_connections 2024-05-06T12:12:43.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.601+0000 7f3a9b5b6700 1 -- 172.21.15.92:0/390121482 wait complete. 2024-05-06T12:12:43.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.601+0000 7f3a9b5b6700 1 Processor -- start 2024-05-06T12:12:43.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.601+0000 7f3a9b5b6700 1 -- start start 2024-05-06T12:12:43.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.602+0000 7f3a9b5b6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a940fce00 0x7f3a94071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:43.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.602+0000 7f3a9b5b6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a940fdf70 0x7f3a940702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:43.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.602+0000 7f3a9b5b6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a94070860 con 0x7f3a940fdf70 2024-05-06T12:12:43.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.602+0000 7f3a9b5b6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a940709d0 con 0x7f3a940fce00 2024-05-06T12:12:43.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.602+0000 7f3a98b51700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a940fdf70 0x7f3a940702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:43.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.602+0000 7f3a98b51700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a940fdf70 0x7f3a940702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58978/0 (socket says 172.21.15.92:58978) 2024-05-06T12:12:43.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.602+0000 7f3a98b51700 1 -- 172.21.15.92:0/2746512773 learned_addr learned my addr 172.21.15.92:0/2746512773 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:12:43.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.602+0000 7f3a99352700 1 --2- 172.21.15.92:0/2746512773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a940fce00 0x7f3a94071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:43.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.602+0000 7f3a98b51700 1 -- 172.21.15.92:0/2746512773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a940fce00 msgr2=0x7f3a94071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:43.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.602+0000 7f3a98b51700 1 --2- 172.21.15.92:0/2746512773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a940fce00 0x7f3a94071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:43.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.602+0000 7f3a98b51700 1 -- 172.21.15.92:0/2746512773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a84005040 con 0x7f3a940fdf70 2024-05-06T12:12:43.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.602+0000 7f3a98b51700 1 --2- 172.21.15.92:0/2746512773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a940fdf70 0x7f3a940702b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400b3b0 tx=0x7f3a84009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:43.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.602+0000 7f3a89ffb700 1 -- 172.21.15.92:0/2746512773 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a8400e040 con 0x7f3a940fdf70 2024-05-06T12:12:43.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.602+0000 7f3a9b5b6700 1 -- 172.21.15.92:0/2746512773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a94070c50 con 0x7f3a940fdf70 2024-05-06T12:12:43.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.602+0000 7f3a89ffb700 1 -- 172.21.15.92:0/2746512773 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a8400bcc0 con 0x7f3a940fdf70 2024-05-06T12:12:43.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.602+0000 7f3a89ffb700 1 -- 172.21.15.92:0/2746512773 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a8401c630 con 0x7f3a940fdf70 2024-05-06T12:12:43.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.603+0000 7f3a9b5b6700 1 -- 172.21.15.92:0/2746512773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a941159a0 con 0x7f3a940fdf70 2024-05-06T12:12:43.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.604+0000 7f3a9b5b6700 1 -- 172.21.15.92:0/2746512773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a9405f5c0 con 0x7f3a940fdf70 2024-05-06T12:12:43.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.604+0000 7f3a89ffb700 1 -- 172.21.15.92:0/2746512773 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f3a84013980 con 0x7f3a940fdf70 2024-05-06T12:12:43.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.605+0000 7f3a89ffb700 1 --2- 172.21.15.92:0/2746512773 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f3a800692a0 0x7f3a8006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:43.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.605+0000 7f3a89ffb700 1 -- 172.21.15.92:0/2746512773 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f3a84088020 con 0x7f3a940fdf70 2024-05-06T12:12:43.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.605+0000 7f3a99352700 1 --2- 172.21.15.92:0/2746512773 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f3a800692a0 0x7f3a8006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:43.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.605+0000 7f3a99352700 1 --2- 172.21.15.92:0/2746512773 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f3a800692a0 0x7f3a8006b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3a900041d0 tx=0x7f3a90009040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:43.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.610+0000 7f3a89ffb700 1 -- 172.21.15.92:0/2746512773 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3a84054280 con 0x7f3a940fdf70 2024-05-06T12:12:43.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.914+0000 7f3a9b5b6700 1 -- 172.21.15.92:0/2746512773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3a9404bed0 con 0x7f3a940fdf70 2024-05-06T12:12:43.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.915+0000 7f3a89ffb700 1 -- 172.21.15.92:0/2746512773 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f3a840578a0 con 0x7f3a940fdf70 2024-05-06T12:12:43.918 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:12:43.918 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": { 2024-05-06T12:12:43.918 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 2 2024-05-06T12:12:43.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:12:43.919 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": { 2024-05-06T12:12:43.919 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 2 2024-05-06T12:12:43.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:12:43.919 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": { 2024-05-06T12:12:43.919 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 6 2024-05-06T12:12:43.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:12:43.919 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-05-06T12:12:43.919 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 4 2024-05-06T12:12:43.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:12:43.919 INFO:teuthology.orchestra.run.smithi092.stdout: "overall": { 2024-05-06T12:12:43.919 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 14 2024-05-06T12:12:43.919 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-06T12:12:43.919 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:12:43.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.917+0000 7f3a9b5b6700 1 -- 172.21.15.92:0/2746512773 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f3a800692a0 msgr2=0x7f3a8006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:43.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.917+0000 7f3a9b5b6700 1 --2- 172.21.15.92:0/2746512773 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f3a800692a0 0x7f3a8006b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3a900041d0 tx=0x7f3a90009040 comp rx=0 tx=0).stop 2024-05-06T12:12:43.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.917+0000 7f3a9b5b6700 1 -- 172.21.15.92:0/2746512773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a940fdf70 msgr2=0x7f3a940702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:43.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.917+0000 7f3a9b5b6700 1 --2- 172.21.15.92:0/2746512773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a940fdf70 0x7f3a940702b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400b3b0 tx=0x7f3a84009e30 comp rx=0 tx=0).stop 2024-05-06T12:12:43.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.917+0000 7f3a9b5b6700 1 -- 172.21.15.92:0/2746512773 shutdown_connections 2024-05-06T12:12:43.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.917+0000 7f3a9b5b6700 1 --2- 172.21.15.92:0/2746512773 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f3a800692a0 0x7f3a8006b750 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:43.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.917+0000 7f3a9b5b6700 1 --2- 172.21.15.92:0/2746512773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a940fce00 0x7f3a94071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:43.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.917+0000 7f3a9b5b6700 1 --2- 172.21.15.92:0/2746512773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a940fdf70 0x7f3a940702b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:43.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.917+0000 7f3a9b5b6700 1 -- 172.21.15.92:0/2746512773 >> 172.21.15.92:0/2746512773 conn(0x7f3a940f84e0 msgr2=0x7f3a940f9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:43.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.918+0000 7f3a9b5b6700 1 -- 172.21.15.92:0/2746512773 shutdown_connections 2024-05-06T12:12:43.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:43.918+0000 7f3a9b5b6700 1 -- 172.21.15.92:0/2746512773 wait complete. 2024-05-06T12:12:44.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.027+0000 7f9bfa928700 1 -- 172.21.15.92:0/2049162916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bf40ffef0 msgr2=0x7f9bf4102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:44.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.027+0000 7f9bfa928700 1 --2- 172.21.15.92:0/2049162916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bf40ffef0 0x7f9bf4102310 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9be8004940 tx=0x7f9be8009e30 comp rx=0 tx=0).stop 2024-05-06T12:12:44.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.027+0000 7f9bfa928700 1 -- 172.21.15.92:0/2049162916 shutdown_connections 2024-05-06T12:12:44.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.027+0000 7f9bfa928700 1 --2- 172.21.15.92:0/2049162916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bf40ffef0 0x7f9bf4102310 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:44.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.027+0000 7f9bfa928700 1 --2- 172.21.15.92:0/2049162916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bf4066b50 0x7f9bf40ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:44.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.027+0000 7f9bfa928700 1 -- 172.21.15.92:0/2049162916 >> 172.21.15.92:0/2049162916 conn(0x7f9bf40f7420 msgr2=0x7f9bf40f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:44.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.028+0000 7f9bfa928700 1 -- 172.21.15.92:0/2049162916 shutdown_connections 2024-05-06T12:12:44.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.028+0000 7f9bfa928700 1 -- 172.21.15.92:0/2049162916 wait complete. 2024-05-06T12:12:44.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.028+0000 7f9bfa928700 1 Processor -- start 2024-05-06T12:12:44.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.028+0000 7f9bfa928700 1 -- start start 2024-05-06T12:12:44.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.028+0000 7f9bfa928700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bf4066b50 0x7f9bf410da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:44.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.029+0000 7f9bfa928700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bf40ffef0 0x7f9bf410df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:44.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.029+0000 7f9bfa928700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf410e590 con 0x7f9bf4066b50 2024-05-06T12:12:44.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.029+0000 7f9bfa928700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf410ede0 con 0x7f9bf40ffef0 2024-05-06T12:12:44.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.029+0000 7f9bf3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bf4066b50 0x7f9bf410da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:44.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.029+0000 7f9bf3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bf4066b50 0x7f9bf410da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58984/0 (socket says 172.21.15.92:58984) 2024-05-06T12:12:44.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.029+0000 7f9bf3fff700 1 -- 172.21.15.92:0/3952791284 learned_addr learned my addr 172.21.15.92:0/3952791284 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:12:44.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.029+0000 7f9bf37fe700 1 --2- 172.21.15.92:0/3952791284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bf40ffef0 0x7f9bf410df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:44.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.029+0000 7f9bf3fff700 1 -- 172.21.15.92:0/3952791284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bf40ffef0 msgr2=0x7f9bf410df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:44.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.029+0000 7f9bf3fff700 1 --2- 172.21.15.92:0/3952791284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bf40ffef0 0x7f9bf410df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:44.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.029+0000 7f9bf3fff700 1 -- 172.21.15.92:0/3952791284 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be8005040 con 0x7f9bf4066b50 2024-05-06T12:12:44.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.029+0000 7f9bf3fff700 1 --2- 172.21.15.92:0/3952791284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bf4066b50 0x7f9bf410da50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9be4002a50 tx=0x7f9be4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:44.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.029+0000 7f9bf0ff9700 1 -- 172.21.15.92:0/3952791284 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9be4007cc0 con 0x7f9bf4066b50 2024-05-06T12:12:44.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.029+0000 7f9bfa928700 1 -- 172.21.15.92:0/3952791284 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf410f0c0 con 0x7f9bf4066b50 2024-05-06T12:12:44.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.029+0000 7f9bf0ff9700 1 -- 172.21.15.92:0/3952791284 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9be400f040 con 0x7f9bf4066b50 2024-05-06T12:12:44.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.029+0000 7f9bf0ff9700 1 -- 172.21.15.92:0/3952791284 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9be400d9c0 con 0x7f9bf4066b50 2024-05-06T12:12:44.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.030+0000 7f9bfa928700 1 -- 172.21.15.92:0/3952791284 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf410f600 con 0x7f9bf4066b50 2024-05-06T12:12:44.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.031+0000 7f9bfa928700 1 -- 172.21.15.92:0/3952791284 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf405f5c0 con 0x7f9bf4066b50 2024-05-06T12:12:44.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.031+0000 7f9bf0ff9700 1 -- 172.21.15.92:0/3952791284 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f9be4009730 con 0x7f9bf4066b50 2024-05-06T12:12:44.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.032+0000 7f9bf0ff9700 1 --2- 172.21.15.92:0/3952791284 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f9bdc0692a0 0x7f9bdc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:44.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.032+0000 7f9bf0ff9700 1 -- 172.21.15.92:0/3952791284 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f9be4086ec0 con 0x7f9bf4066b50 2024-05-06T12:12:44.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.032+0000 7f9bf37fe700 1 --2- 172.21.15.92:0/3952791284 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f9bdc0692a0 0x7f9bdc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:44.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.033+0000 7f9bf37fe700 1 --2- 172.21.15.92:0/3952791284 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f9bdc0692a0 0x7f9bdc06b750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9be800b970 tx=0x7f9be8009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:44.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.037+0000 7f9bf0ff9700 1 -- 172.21.15.92:0/3952791284 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9be4056c30 con 0x7f9bf4066b50 2024-05-06T12:12:44.179 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:43 smithi092 ceph-mon[26954]: from='client.14552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:12:44.179 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:43 smithi092 ceph-mon[26954]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:12:44.179 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:43 smithi092 ceph-mon[26954]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:12:44.179 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:43 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/2746512773' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:12:44.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.321+0000 7f9bfa928700 1 -- 172.21.15.92:0/3952791284 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9bf410f9d0 con 0x7f9bf4066b50 2024-05-06T12:12:44.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.321+0000 7f9bf0ff9700 1 -- 172.21.15.92:0/3952791284 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f9be40567c0 con 0x7f9bf4066b50 2024-05-06T12:12:44.325 INFO:teuthology.orchestra.run.smithi092.stdout:e18 2024-05-06T12:12:44.325 INFO:teuthology.orchestra.run.smithi092.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-06T12:12:44.325 INFO:teuthology.orchestra.run.smithi092.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:12:44.325 INFO:teuthology.orchestra.run.smithi092.stdout:legacy client fscid: 1 2024-05-06T12:12:44.325 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:12:44.325 INFO:teuthology.orchestra.run.smithi092.stdout:Filesystem 'cephfs' (1) 2024-05-06T12:12:44.325 INFO:teuthology.orchestra.run.smithi092.stdout:fs_name cephfs 2024-05-06T12:12:44.325 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 18 2024-05-06T12:12:44.325 INFO:teuthology.orchestra.run.smithi092.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-06T12:12:44.325 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-05-06T12:11:44.952671+0000 2024-05-06T12:12:44.325 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2024-05-06T12:12:04.907000+0000 2024-05-06T12:12:44.325 INFO:teuthology.orchestra.run.smithi092.stdout:tableserver 0 2024-05-06T12:12:44.325 INFO:teuthology.orchestra.run.smithi092.stdout:root 0 2024-05-06T12:12:44.326 INFO:teuthology.orchestra.run.smithi092.stdout:session_timeout 60 2024-05-06T12:12:44.326 INFO:teuthology.orchestra.run.smithi092.stdout:session_autoclose 300 2024-05-06T12:12:44.326 INFO:teuthology.orchestra.run.smithi092.stdout:max_file_size 1099511627776 2024-05-06T12:12:44.326 INFO:teuthology.orchestra.run.smithi092.stdout:required_client_features {} 2024-05-06T12:12:44.326 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure 0 2024-05-06T12:12:44.326 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure_osd_epoch 46 2024-05-06T12:12:44.326 INFO:teuthology.orchestra.run.smithi092.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:12:44.326 INFO:teuthology.orchestra.run.smithi092.stdout:max_mds 2 2024-05-06T12:12:44.326 INFO:teuthology.orchestra.run.smithi092.stdout:in 0,1 2024-05-06T12:12:44.326 INFO:teuthology.orchestra.run.smithi092.stdout:up {0=14472,1=24267} 2024-05-06T12:12:44.326 INFO:teuthology.orchestra.run.smithi092.stdout:failed 2024-05-06T12:12:44.326 INFO:teuthology.orchestra.run.smithi092.stdout:damaged 2024-05-06T12:12:44.326 INFO:teuthology.orchestra.run.smithi092.stdout:stopped 2024-05-06T12:12:44.326 INFO:teuthology.orchestra.run.smithi092.stdout:data_pools [3] 2024-05-06T12:12:44.326 INFO:teuthology.orchestra.run.smithi092.stdout:metadata_pool 2 2024-05-06T12:12:44.327 INFO:teuthology.orchestra.run.smithi092.stdout:inline_data enabled 2024-05-06T12:12:44.327 INFO:teuthology.orchestra.run.smithi092.stdout:balancer 2024-05-06T12:12:44.327 INFO:teuthology.orchestra.run.smithi092.stdout:standby_count_wanted 1 2024-05-06T12:12:44.327 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.rosbll{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.92:6828/1663253598,v1:172.21.15.92:6829/1663253598] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:12:44.327 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.ijsqgp{0:14484} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.92:6826/3613441054,v1:172.21.15.92:6827/3613441054] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:12:44.327 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.byjvgf{1:24267} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.190:6824/1565151699,v1:172.21.15.190:6825/1565151699] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:12:44.327 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.idzoff{1:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.190:6826/2453728380,v1:172.21.15.190:6827/2453728380] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:12:44.327 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:12:44.327 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:12:44.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.323+0000 7f9bfa928700 1 -- 172.21.15.92:0/3952791284 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f9bdc0692a0 msgr2=0x7f9bdc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:44.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.323+0000 7f9bfa928700 1 --2- 172.21.15.92:0/3952791284 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f9bdc0692a0 0x7f9bdc06b750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9be800b970 tx=0x7f9be8009e30 comp rx=0 tx=0).stop 2024-05-06T12:12:44.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.323+0000 7f9bfa928700 1 -- 172.21.15.92:0/3952791284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bf4066b50 msgr2=0x7f9bf410da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:44.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.323+0000 7f9bfa928700 1 --2- 172.21.15.92:0/3952791284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bf4066b50 0x7f9bf410da50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9be4002a50 tx=0x7f9be4004600 comp rx=0 tx=0).stop 2024-05-06T12:12:44.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.324+0000 7f9bfa928700 1 -- 172.21.15.92:0/3952791284 shutdown_connections 2024-05-06T12:12:44.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.324+0000 7f9bfa928700 1 --2- 172.21.15.92:0/3952791284 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f9bdc0692a0 0x7f9bdc06b750 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:44.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.324+0000 7f9bfa928700 1 --2- 172.21.15.92:0/3952791284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bf4066b50 0x7f9bf410da50 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:44.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.324+0000 7f9bfa928700 1 --2- 172.21.15.92:0/3952791284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bf40ffef0 0x7f9bf410df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:44.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.324+0000 7f9bfa928700 1 -- 172.21.15.92:0/3952791284 >> 172.21.15.92:0/3952791284 conn(0x7f9bf40f7420 msgr2=0x7f9bf40f7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:44.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.324+0000 7f9bfa928700 1 -- 172.21.15.92:0/3952791284 shutdown_connections 2024-05-06T12:12:44.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.324+0000 7f9bfa928700 1 -- 172.21.15.92:0/3952791284 wait complete. 2024-05-06T12:12:44.329 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 18 2024-05-06T12:12:44.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:43 smithi190 ceph-mon[33798]: from='client.14552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:12:44.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:43 smithi190 ceph-mon[33798]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:12:44.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:43 smithi190 ceph-mon[33798]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:12:44.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:43 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/2746512773' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:12:44.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.442+0000 7fce17076700 1 -- 172.21.15.92:0/3583811792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce100fdfd0 msgr2=0x7fce100fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:44.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.442+0000 7fce17076700 1 --2- 172.21.15.92:0/3583811792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce100fdfd0 0x7fce100fe440 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc004940 tx=0x7fcdfc009e30 comp rx=0 tx=0).stop 2024-05-06T12:12:44.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.442+0000 7fce17076700 1 -- 172.21.15.92:0/3583811792 shutdown_connections 2024-05-06T12:12:44.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.442+0000 7fce17076700 1 --2- 172.21.15.92:0/3583811792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce100fdfd0 0x7fce100fe440 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:44.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.442+0000 7fce17076700 1 --2- 172.21.15.92:0/3583811792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce100fce60 0x7fce100fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:44.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.442+0000 7fce17076700 1 -- 172.21.15.92:0/3583811792 >> 172.21.15.92:0/3583811792 conn(0x7fce100f8540 msgr2=0x7fce100fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:44.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.442+0000 7fce17076700 1 -- 172.21.15.92:0/3583811792 shutdown_connections 2024-05-06T12:12:44.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.442+0000 7fce17076700 1 -- 172.21.15.92:0/3583811792 wait complete. 2024-05-06T12:12:44.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.442+0000 7fce17076700 1 Processor -- start 2024-05-06T12:12:44.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.442+0000 7fce17076700 1 -- start start 2024-05-06T12:12:44.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.443+0000 7fce17076700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce100fce60 0x7fce1010da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:44.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.443+0000 7fce17076700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce100fdfd0 0x7fce1010df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:44.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.443+0000 7fce17076700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1010e540 con 0x7fce100fdfd0 2024-05-06T12:12:44.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.443+0000 7fce17076700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1010ed90 con 0x7fce100fce60 2024-05-06T12:12:44.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.443+0000 7fce0ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce100fdfd0 0x7fce1010df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:44.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.443+0000 7fce0ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce100fdfd0 0x7fce1010df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59000/0 (socket says 172.21.15.92:59000) 2024-05-06T12:12:44.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.443+0000 7fce0ffff700 1 -- 172.21.15.92:0/3106205331 learned_addr learned my addr 172.21.15.92:0/3106205331 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:12:44.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.443+0000 7fce14e12700 1 --2- 172.21.15.92:0/3106205331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce100fce60 0x7fce1010da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:44.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.443+0000 7fce0ffff700 1 -- 172.21.15.92:0/3106205331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce100fce60 msgr2=0x7fce1010da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:44.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.443+0000 7fce0ffff700 1 --2- 172.21.15.92:0/3106205331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce100fce60 0x7fce1010da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:44.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.443+0000 7fce0ffff700 1 -- 172.21.15.92:0/3106205331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdfc005040 con 0x7fce100fdfd0 2024-05-06T12:12:44.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.443+0000 7fce14e12700 1 --2- 172.21.15.92:0/3106205331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce100fce60 0x7fce1010da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-06T12:12:44.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.443+0000 7fce0ffff700 1 --2- 172.21.15.92:0/3106205331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce100fdfd0 0x7fce1010df40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00b3b0 tx=0x7fcdfc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:44.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.444+0000 7fce0d7fa700 1 -- 172.21.15.92:0/3106205331 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdfc00e040 con 0x7fce100fdfd0 2024-05-06T12:12:44.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.444+0000 7fce17076700 1 -- 172.21.15.92:0/3106205331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce1010f010 con 0x7fce100fdfd0 2024-05-06T12:12:44.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.444+0000 7fce0d7fa700 1 -- 172.21.15.92:0/3106205331 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcdfc00bcc0 con 0x7fce100fdfd0 2024-05-06T12:12:44.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.444+0000 7fce0d7fa700 1 -- 172.21.15.92:0/3106205331 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdfc01c630 con 0x7fce100fdfd0 2024-05-06T12:12:44.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.444+0000 7fce17076700 1 -- 172.21.15.92:0/3106205331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce1010f550 con 0x7fce100fdfd0 2024-05-06T12:12:44.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.445+0000 7fce17076700 1 -- 172.21.15.92:0/3106205331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce1005f5c0 con 0x7fce100fdfd0 2024-05-06T12:12:44.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.445+0000 7fce0d7fa700 1 -- 172.21.15.92:0/3106205331 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fcdfc013980 con 0x7fce100fdfd0 2024-05-06T12:12:44.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.446+0000 7fce0d7fa700 1 --2- 172.21.15.92:0/3106205331 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fce000692a0 0x7fce0006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:44.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.446+0000 7fce0d7fa700 1 -- 172.21.15.92:0/3106205331 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fcdfc088010 con 0x7fce100fdfd0 2024-05-06T12:12:44.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.446+0000 7fce14e12700 1 --2- 172.21.15.92:0/3106205331 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fce000692a0 0x7fce0006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:44.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.446+0000 7fce14e12700 1 --2- 172.21.15.92:0/3106205331 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fce000692a0 0x7fce0006b750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fce04007fd0 tx=0x7fce04009040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:44.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.452+0000 7fce0d7fa700 1 -- 172.21.15.92:0/3106205331 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fcdfc057d00 con 0x7fce100fdfd0 2024-05-06T12:12:44.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.676+0000 7fce17076700 1 -- 172.21.15.92:0/3106205331 --> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fce1010f830 con 0x7fce000692a0 2024-05-06T12:12:44.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.677+0000 7fce0d7fa700 1 -- 172.21.15.92:0/3106205331 <== mgr.14210 v2:172.21.15.92:6800/3045905943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fce1010f830 con 0x7fce000692a0 2024-05-06T12:12:44.680 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:12:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:70ce9539b62a15b9de429c1f0e91a1e012b099aa", 2024-05-06T12:12:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: "in_progress": true, 2024-05-06T12:12:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-06T12:12:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: "services_complete": [], 2024-05-06T12:12:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: "progress": "", 2024-05-06T12:12:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "Doing first pull of quay.ceph.io/ceph-ci/ceph:70ce9539b62a15b9de429c1f0e91a1e012b099aa image", 2024-05-06T12:12:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: "is_paused": false 2024-05-06T12:12:44.681 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:12:44.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.679+0000 7fce17076700 1 -- 172.21.15.92:0/3106205331 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fce000692a0 msgr2=0x7fce0006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:44.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.679+0000 7fce17076700 1 --2- 172.21.15.92:0/3106205331 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fce000692a0 0x7fce0006b750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fce04007fd0 tx=0x7fce04009040 comp rx=0 tx=0).stop 2024-05-06T12:12:44.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.679+0000 7fce17076700 1 -- 172.21.15.92:0/3106205331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce100fdfd0 msgr2=0x7fce1010df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:44.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.679+0000 7fce17076700 1 --2- 172.21.15.92:0/3106205331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce100fdfd0 0x7fce1010df40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00b3b0 tx=0x7fcdfc009e30 comp rx=0 tx=0).stop 2024-05-06T12:12:44.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.680+0000 7fce17076700 1 -- 172.21.15.92:0/3106205331 shutdown_connections 2024-05-06T12:12:44.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.680+0000 7fce17076700 1 --2- 172.21.15.92:0/3106205331 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fce000692a0 0x7fce0006b750 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:44.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.680+0000 7fce17076700 1 --2- 172.21.15.92:0/3106205331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce100fce60 0x7fce1010da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:44.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.680+0000 7fce17076700 1 --2- 172.21.15.92:0/3106205331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce100fdfd0 0x7fce1010df40 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:44.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.680+0000 7fce17076700 1 -- 172.21.15.92:0/3106205331 >> 172.21.15.92:0/3106205331 conn(0x7fce100f8540 msgr2=0x7fce100f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:44.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.680+0000 7fce17076700 1 -- 172.21.15.92:0/3106205331 shutdown_connections 2024-05-06T12:12:44.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.680+0000 7fce17076700 1 -- 172.21.15.92:0/3106205331 wait complete. 2024-05-06T12:12:44.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.788+0000 7fc8217e9700 1 -- 172.21.15.92:0/1895781718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc81c0fff40 msgr2=0x7fc81c102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:44.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.788+0000 7fc8217e9700 1 --2- 172.21.15.92:0/1895781718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc81c0fff40 0x7fc81c102320 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc810004940 tx=0x7fc810009e30 comp rx=0 tx=0).stop 2024-05-06T12:12:44.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.789+0000 7fc8217e9700 1 -- 172.21.15.92:0/1895781718 shutdown_connections 2024-05-06T12:12:44.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.789+0000 7fc8217e9700 1 --2- 172.21.15.92:0/1895781718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc81c0fff40 0x7fc81c102320 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:44.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.789+0000 7fc8217e9700 1 --2- 172.21.15.92:0/1895781718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc81c066260 0x7fc81c0ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:44.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.789+0000 7fc8217e9700 1 -- 172.21.15.92:0/1895781718 >> 172.21.15.92:0/1895781718 conn(0x7fc81c0f7430 msgr2=0x7fc81c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:44.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.789+0000 7fc8217e9700 1 -- 172.21.15.92:0/1895781718 shutdown_connections 2024-05-06T12:12:44.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.789+0000 7fc8217e9700 1 -- 172.21.15.92:0/1895781718 wait complete. 2024-05-06T12:12:44.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.789+0000 7fc8217e9700 1 Processor -- start 2024-05-06T12:12:44.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.789+0000 7fc8217e9700 1 -- start start 2024-05-06T12:12:44.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.789+0000 7fc8217e9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc81c066260 0x7fc81c10d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:44.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.789+0000 7fc8217e9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc81c0fff40 0x7fc81c10df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:44.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.789+0000 7fc8217e9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c10e520 con 0x7fc81c066260 2024-05-06T12:12:44.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.790+0000 7fc8217e9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c10ed70 con 0x7fc81c0fff40 2024-05-06T12:12:44.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.790+0000 7fc81affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc81c066260 0x7fc81c10d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:44.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.790+0000 7fc81affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc81c066260 0x7fc81c10d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59012/0 (socket says 172.21.15.92:59012) 2024-05-06T12:12:44.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.790+0000 7fc81affd700 1 -- 172.21.15.92:0/3371166153 learned_addr learned my addr 172.21.15.92:0/3371166153 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:12:44.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.790+0000 7fc81a7fc700 1 --2- 172.21.15.92:0/3371166153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc81c0fff40 0x7fc81c10df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:44.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.790+0000 7fc81affd700 1 -- 172.21.15.92:0/3371166153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc81c0fff40 msgr2=0x7fc81c10df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:44.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.790+0000 7fc81affd700 1 --2- 172.21.15.92:0/3371166153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc81c0fff40 0x7fc81c10df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:44.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.790+0000 7fc81affd700 1 -- 172.21.15.92:0/3371166153 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc810005040 con 0x7fc81c066260 2024-05-06T12:12:44.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.790+0000 7fc81affd700 1 --2- 172.21.15.92:0/3371166153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc81c066260 0x7fc81c10d9e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc80c002a80 tx=0x7fc80c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:44.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.790+0000 7fc8037fe700 1 -- 172.21.15.92:0/3371166153 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc80c007cc0 con 0x7fc81c066260 2024-05-06T12:12:44.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.790+0000 7fc8037fe700 1 -- 172.21.15.92:0/3371166153 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc80c00f040 con 0x7fc81c066260 2024-05-06T12:12:44.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.790+0000 7fc8217e9700 1 -- 172.21.15.92:0/3371166153 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc81c10f050 con 0x7fc81c066260 2024-05-06T12:12:44.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.790+0000 7fc8037fe700 1 -- 172.21.15.92:0/3371166153 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc80c00da30 con 0x7fc81c066260 2024-05-06T12:12:44.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.790+0000 7fc8217e9700 1 -- 172.21.15.92:0/3371166153 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc81c10f590 con 0x7fc81c066260 2024-05-06T12:12:44.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.791+0000 7fc8217e9700 1 -- 172.21.15.92:0/3371166153 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc81c05f5c0 con 0x7fc81c066260 2024-05-06T12:12:44.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.792+0000 7fc8037fe700 1 -- 172.21.15.92:0/3371166153 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fc80c00db90 con 0x7fc81c066260 2024-05-06T12:12:44.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.792+0000 7fc8037fe700 1 --2- 172.21.15.92:0/3371166153 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fc8040692a0 0x7fc80406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:12:44.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.792+0000 7fc8037fe700 1 -- 172.21.15.92:0/3371166153 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fc80c086f70 con 0x7fc81c066260 2024-05-06T12:12:44.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.792+0000 7fc81a7fc700 1 --2- 172.21.15.92:0/3371166153 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fc8040692a0 0x7fc80406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:12:44.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.793+0000 7fc81a7fc700 1 --2- 172.21.15.92:0/3371166153 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fc8040692a0 0x7fc80406b750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc81000b970 tx=0x7fc810009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:12:44.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:44.797+0000 7fc8037fe700 1 -- 172.21.15.92:0/3371166153 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc80c056c60 con 0x7fc81c066260 2024-05-06T12:12:45.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:45.093+0000 7fc8217e9700 1 -- 172.21.15.92:0/3371166153 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc81c04bed0 con 0x7fc81c066260 2024-05-06T12:12:45.097 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:44 smithi092 ceph-mon[26954]: pgmap v204: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 4 op/s 2024-05-06T12:12:45.097 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:44 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/3952791284' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:12:45.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:45.094+0000 7fc8037fe700 1 -- 172.21.15.92:0/3371166153 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fc80c0567f0 con 0x7fc81c066260 2024-05-06T12:12:45.097 INFO:teuthology.orchestra.run.smithi092.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-06T12:12:45.098 INFO:teuthology.orchestra.run.smithi092.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-06T12:12:45.098 INFO:teuthology.orchestra.run.smithi092.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-06T12:12:45.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:45.096+0000 7fc8217e9700 1 -- 172.21.15.92:0/3371166153 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fc8040692a0 msgr2=0x7fc80406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:45.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:45.096+0000 7fc8217e9700 1 --2- 172.21.15.92:0/3371166153 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fc8040692a0 0x7fc80406b750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc81000b970 tx=0x7fc810009e30 comp rx=0 tx=0).stop 2024-05-06T12:12:45.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:45.096+0000 7fc8217e9700 1 -- 172.21.15.92:0/3371166153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc81c066260 msgr2=0x7fc81c10d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:12:45.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:45.096+0000 7fc8217e9700 1 --2- 172.21.15.92:0/3371166153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc81c066260 0x7fc81c10d9e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc80c002a80 tx=0x7fc80c004600 comp rx=0 tx=0).stop 2024-05-06T12:12:45.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:45.097+0000 7fc8217e9700 1 -- 172.21.15.92:0/3371166153 shutdown_connections 2024-05-06T12:12:45.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:45.097+0000 7fc8217e9700 1 --2- 172.21.15.92:0/3371166153 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fc8040692a0 0x7fc80406b750 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:45.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:45.097+0000 7fc8217e9700 1 --2- 172.21.15.92:0/3371166153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc81c066260 0x7fc81c10d9e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:45.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:45.097+0000 7fc8217e9700 1 --2- 172.21.15.92:0/3371166153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc81c0fff40 0x7fc81c10df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:12:45.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:45.097+0000 7fc8217e9700 1 -- 172.21.15.92:0/3371166153 >> 172.21.15.92:0/3371166153 conn(0x7fc81c0f7430 msgr2=0x7fc81c0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:12:45.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:45.097+0000 7fc8217e9700 1 -- 172.21.15.92:0/3371166153 shutdown_connections 2024-05-06T12:12:45.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:12:45.097+0000 7fc8217e9700 1 -- 172.21.15.92:0/3371166153 wait complete. 2024-05-06T12:12:45.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:44 smithi190 ceph-mon[33798]: pgmap v204: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 4 op/s 2024-05-06T12:12:45.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:44 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/3952791284' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:12:46.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:45 smithi092 ceph-mon[26954]: from='client.14572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:12:46.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:45 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/3371166153' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:12:46.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:45 smithi190 ceph-mon[33798]: from='client.14572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:12:46.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:45 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/3371166153' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:12:47.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:46 smithi092 ceph-mon[26954]: pgmap v205: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 597 B/s wr, 4 op/s 2024-05-06T12:12:47.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:46 smithi190 ceph-mon[33798]: pgmap v205: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 597 B/s wr, 4 op/s 2024-05-06T12:12:49.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:48 smithi190 ceph-mon[33798]: pgmap v206: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 511 B/s wr, 4 op/s 2024-05-06T12:12:49.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:48 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:12:49.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:48 smithi092 ceph-mon[26954]: pgmap v206: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 511 B/s wr, 4 op/s 2024-05-06T12:12:49.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:48 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:12:51.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:50 smithi092 ceph-mon[26954]: pgmap v207: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 4 op/s 2024-05-06T12:12:51.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:50 smithi190 ceph-mon[33798]: pgmap v207: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 4 op/s 2024-05-06T12:12:52.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:52 smithi190 ceph-mon[33798]: pgmap v208: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 426 B/s wr, 4 op/s 2024-05-06T12:12:52.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:52 smithi092 ceph-mon[26954]: pgmap v208: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 426 B/s wr, 4 op/s 2024-05-06T12:12:55.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:54 smithi092 ceph-mon[26954]: pgmap v209: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:12:55.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:54 smithi190 ceph-mon[33798]: pgmap v209: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:12:57.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:56 smithi092 ceph-mon[26954]: pgmap v210: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:12:57.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:56 smithi190 ceph-mon[33798]: pgmap v210: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:12:59.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:12:58 smithi190 ceph-mon[33798]: pgmap v211: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:12:59.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:12:58 smithi092 ceph-mon[26954]: pgmap v211: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:13:01.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:00 smithi092 ceph-mon[26954]: pgmap v212: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:13:01.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:00 smithi190 ceph-mon[33798]: pgmap v212: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:13:03.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:02 smithi092 ceph-mon[26954]: pgmap v213: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:13:03.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:02 smithi190 ceph-mon[33798]: pgmap v213: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:13:04.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:03 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:13:04.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:03 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:13:05.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:04 smithi092 ceph-mon[26954]: pgmap v214: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:13:05.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:04 smithi190 ceph-mon[33798]: pgmap v214: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:13:07.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:06 smithi092 ceph-mon[26954]: pgmap v215: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:13:07.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:06 smithi190 ceph-mon[33798]: pgmap v215: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:13:09.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:08 smithi190 ceph-mon[33798]: pgmap v216: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:13:09.229 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:08 smithi092 ceph-mon[26954]: pgmap v216: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:13:11.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:10 smithi092 ceph-mon[26954]: pgmap v217: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:13:11.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:10 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:13:11.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:10 smithi092 ceph-mon[26954]: Upgrade: Target is version 19.0.0-2498-g70ce9539 (unknown) 2024-05-06T12:13:11.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:10 smithi092 ceph-mon[26954]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, digests ['quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb'] 2024-05-06T12:13:11.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:10 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:13:11.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:10 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:13:11.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:10 smithi092 ceph-mon[26954]: Upgrade: Need to upgrade myself (mgr.smithi092.gyzeqb) 2024-05-06T12:13:11.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:10 smithi092 ceph-mon[26954]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb on smithi190 2024-05-06T12:13:11.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:10 smithi190 ceph-mon[33798]: pgmap v217: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:13:11.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:10 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:13:11.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:10 smithi190 ceph-mon[33798]: Upgrade: Target is version 19.0.0-2498-g70ce9539 (unknown) 2024-05-06T12:13:11.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:10 smithi190 ceph-mon[33798]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, digests ['quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb'] 2024-05-06T12:13:11.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:10 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:13:11.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:10 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:13:11.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:10 smithi190 ceph-mon[33798]: Upgrade: Need to upgrade myself (mgr.smithi092.gyzeqb) 2024-05-06T12:13:11.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:10 smithi190 ceph-mon[33798]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb on smithi190 2024-05-06T12:13:13.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:12 smithi092 ceph-mon[26954]: pgmap v218: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:13:13.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:12 smithi190 ceph-mon[33798]: pgmap v218: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:13:15.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:14 smithi092 ceph-mon[26954]: pgmap v219: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:13:15.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.228+0000 7f55aa066700 1 -- 172.21.15.92:0/2870041348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55a40fcf60 msgr2=0x7f55a40fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:15.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.228+0000 7f55aa066700 1 --2- 172.21.15.92:0/2870041348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55a40fcf60 0x7f55a40fd3d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f558c004940 tx=0x7f558c009e30 comp rx=0 tx=0).stop 2024-05-06T12:13:15.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.228+0000 7f55aa066700 1 -- 172.21.15.92:0/2870041348 shutdown_connections 2024-05-06T12:13:15.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.228+0000 7f55aa066700 1 --2- 172.21.15.92:0/2870041348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55a40fcf60 0x7f55a40fd3d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:15.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.228+0000 7f55aa066700 1 --2- 172.21.15.92:0/2870041348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a40fbcb0 0x7f55a40fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:15.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.228+0000 7f55aa066700 1 -- 172.21.15.92:0/2870041348 >> 172.21.15.92:0/2870041348 conn(0x7f55a40f7430 msgr2=0x7f55a40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:13:15.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.229+0000 7f55aa066700 1 -- 172.21.15.92:0/2870041348 shutdown_connections 2024-05-06T12:13:15.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.229+0000 7f55aa066700 1 -- 172.21.15.92:0/2870041348 wait complete. 2024-05-06T12:13:15.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.229+0000 7f55aa066700 1 Processor -- start 2024-05-06T12:13:15.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.229+0000 7f55aa066700 1 -- start start 2024-05-06T12:13:15.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.229+0000 7f55aa066700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55a40fbcb0 0x7f55a410c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:15.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.229+0000 7f55aa066700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a40fcf60 0x7f55a410cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:15.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.230+0000 7f55aa066700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a410d3e0 con 0x7f55a40fbcb0 2024-05-06T12:13:15.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.230+0000 7f55aa066700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a410dc30 con 0x7f55a40fcf60 2024-05-06T12:13:15.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.230+0000 7f559bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a40fcf60 0x7f55a410cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:15.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.230+0000 7f559bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a40fcf60 0x7f55a410cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.92:50548/0 (socket says 172.21.15.92:50548) 2024-05-06T12:13:15.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.230+0000 7f559bfff700 1 -- 172.21.15.92:0/385194634 learned_addr learned my addr 172.21.15.92:0/385194634 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:13:15.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.230+0000 7f55a37fe700 1 --2- 172.21.15.92:0/385194634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55a40fbcb0 0x7f55a410c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:15.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.230+0000 7f559bfff700 1 -- 172.21.15.92:0/385194634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55a40fbcb0 msgr2=0x7f55a410c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:15.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.230+0000 7f559bfff700 1 --2- 172.21.15.92:0/385194634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55a40fbcb0 0x7f55a410c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:15.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.230+0000 7f559bfff700 1 -- 172.21.15.92:0/385194634 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f558c005040 con 0x7f55a40fcf60 2024-05-06T12:13:15.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.230+0000 7f559bfff700 1 --2- 172.21.15.92:0/385194634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a40fcf60 0x7f55a410cde0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f558c013870 tx=0x7f558c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:13:15.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.231+0000 7f55a0ff9700 1 -- 172.21.15.92:0/385194634 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f558c00e070 con 0x7f55a40fcf60 2024-05-06T12:13:15.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.231+0000 7f55aa066700 1 -- 172.21.15.92:0/385194634 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a410de30 con 0x7f55a40fcf60 2024-05-06T12:13:15.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.231+0000 7f55a0ff9700 1 -- 172.21.15.92:0/385194634 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f558c01b460 con 0x7f55a40fcf60 2024-05-06T12:13:15.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.231+0000 7f55a0ff9700 1 -- 172.21.15.92:0/385194634 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f558c00b650 con 0x7f55a40fcf60 2024-05-06T12:13:15.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.231+0000 7f55aa066700 1 -- 172.21.15.92:0/385194634 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a410e370 con 0x7f55a40fcf60 2024-05-06T12:13:15.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.232+0000 7f55aa066700 1 -- 172.21.15.92:0/385194634 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a405f5c0 con 0x7f55a40fcf60 2024-05-06T12:13:15.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.233+0000 7f55a0ff9700 1 -- 172.21.15.92:0/385194634 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f558c022030 con 0x7f55a40fcf60 2024-05-06T12:13:15.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.233+0000 7f55a0ff9700 1 --2- 172.21.15.92:0/385194634 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f55840692a0 0x7f558406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:15.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.233+0000 7f55a0ff9700 1 -- 172.21.15.92:0/385194634 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f558c08a680 con 0x7f55a40fcf60 2024-05-06T12:13:15.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.234+0000 7f55a37fe700 1 --2- 172.21.15.92:0/385194634 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f55840692a0 0x7f558406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:15.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.234+0000 7f55a37fe700 1 --2- 172.21.15.92:0/385194634 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f55840692a0 0x7f558406b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5594004600 tx=0x7f559400b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:13:15.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.241+0000 7f55a0ff9700 1 -- 172.21.15.92:0/385194634 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f558c05a350 con 0x7f55a40fcf60 2024-05-06T12:13:15.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:14 smithi190 ceph-mon[33798]: pgmap v219: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:13:15.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.554+0000 7f55aa066700 1 -- 172.21.15.92:0/385194634 --> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f55a410e700 con 0x7f55840692a0 2024-05-06T12:13:15.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.556+0000 7f55a0ff9700 1 -- 172.21.15.92:0/385194634 <== mgr.14210 v2:172.21.15.92:6800/3045905943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f55a410e700 con 0x7f55840692a0 2024-05-06T12:13:15.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.559+0000 7f55aa066700 1 -- 172.21.15.92:0/385194634 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f55840692a0 msgr2=0x7f558406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:15.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.559+0000 7f55aa066700 1 --2- 172.21.15.92:0/385194634 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f55840692a0 0x7f558406b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5594004600 tx=0x7f559400b040 comp rx=0 tx=0).stop 2024-05-06T12:13:15.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.559+0000 7f55aa066700 1 -- 172.21.15.92:0/385194634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a40fcf60 msgr2=0x7f55a410cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:15.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.559+0000 7f55aa066700 1 --2- 172.21.15.92:0/385194634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a40fcf60 0x7f55a410cde0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f558c013870 tx=0x7f558c009e30 comp rx=0 tx=0).stop 2024-05-06T12:13:15.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.559+0000 7f55aa066700 1 -- 172.21.15.92:0/385194634 shutdown_connections 2024-05-06T12:13:15.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.559+0000 7f55aa066700 1 --2- 172.21.15.92:0/385194634 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f55840692a0 0x7f558406b750 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:15.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.559+0000 7f55aa066700 1 --2- 172.21.15.92:0/385194634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55a40fbcb0 0x7f55a410c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:15.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.559+0000 7f55aa066700 1 --2- 172.21.15.92:0/385194634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a40fcf60 0x7f55a410cde0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:15.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.559+0000 7f55aa066700 1 -- 172.21.15.92:0/385194634 >> 172.21.15.92:0/385194634 conn(0x7f55a40f7430 msgr2=0x7f55a40f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:13:15.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.559+0000 7f55aa066700 1 -- 172.21.15.92:0/385194634 shutdown_connections 2024-05-06T12:13:15.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.559+0000 7f55aa066700 1 -- 172.21.15.92:0/385194634 wait complete. 2024-05-06T12:13:15.572 INFO:teuthology.orchestra.run.smithi092.stdout:true 2024-05-06T12:13:15.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.705+0000 7f0b61884700 1 -- 172.21.15.92:0/435207408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b5c0ffee0 msgr2=0x7f0b5c102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:15.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.705+0000 7f0b61884700 1 --2- 172.21.15.92:0/435207408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b5c0ffee0 0x7f0b5c102300 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0b50004940 tx=0x7f0b50009e30 comp rx=0 tx=0).stop 2024-05-06T12:13:15.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.705+0000 7f0b61884700 1 -- 172.21.15.92:0/435207408 shutdown_connections 2024-05-06T12:13:15.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.705+0000 7f0b61884700 1 --2- 172.21.15.92:0/435207408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b5c0ffee0 0x7f0b5c102300 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:15.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.705+0000 7f0b61884700 1 --2- 172.21.15.92:0/435207408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b5c0662c0 0x7f0b5c0ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:15.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.705+0000 7f0b61884700 1 -- 172.21.15.92:0/435207408 >> 172.21.15.92:0/435207408 conn(0x7f0b5c0f7430 msgr2=0x7f0b5c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:13:15.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.706+0000 7f0b61884700 1 -- 172.21.15.92:0/435207408 shutdown_connections 2024-05-06T12:13:15.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.706+0000 7f0b61884700 1 -- 172.21.15.92:0/435207408 wait complete. 2024-05-06T12:13:15.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.706+0000 7f0b61884700 1 Processor -- start 2024-05-06T12:13:15.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.706+0000 7f0b61884700 1 -- start start 2024-05-06T12:13:15.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.706+0000 7f0b61884700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b5c0662c0 0x7f0b5c0fd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:15.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.707+0000 7f0b61884700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b5c0ffee0 0x7f0b5c0fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:15.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.707+0000 7f0b5a7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b5c0ffee0 0x7f0b5c0fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:15.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.707+0000 7f0b5affd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b5c0662c0 0x7f0b5c0fd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:15.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.707+0000 7f0b5a7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b5c0ffee0 0x7f0b5c0fb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41310/0 (socket says 172.21.15.92:41310) 2024-05-06T12:13:15.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.707+0000 7f0b5affd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b5c0662c0 0x7f0b5c0fd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.92:50572/0 (socket says 172.21.15.92:50572) 2024-05-06T12:13:15.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.707+0000 7f0b5a7fc700 1 -- 172.21.15.92:0/498848948 learned_addr learned my addr 172.21.15.92:0/498848948 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:13:15.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.707+0000 7f0b61884700 1 -- 172.21.15.92:0/498848948 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c0fbe50 con 0x7f0b5c0ffee0 2024-05-06T12:13:15.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.707+0000 7f0b61884700 1 -- 172.21.15.92:0/498848948 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c0fbfc0 con 0x7f0b5c0662c0 2024-05-06T12:13:15.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.707+0000 7f0b5a7fc700 1 -- 172.21.15.92:0/498848948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b5c0662c0 msgr2=0x7f0b5c0fd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:15.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.707+0000 7f0b5a7fc700 1 --2- 172.21.15.92:0/498848948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b5c0662c0 0x7f0b5c0fd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:15.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.707+0000 7f0b5a7fc700 1 -- 172.21.15.92:0/498848948 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b50005040 con 0x7f0b5c0ffee0 2024-05-06T12:13:15.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.707+0000 7f0b5a7fc700 1 --2- 172.21.15.92:0/498848948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b5c0ffee0 0x7f0b5c0fb8a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0b50009150 tx=0x7f0b50009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:13:15.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.707+0000 7f0b5affd700 1 --2- 172.21.15.92:0/498848948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b5c0662c0 0x7f0b5c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-06T12:13:15.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.707+0000 7f0b437fe700 1 -- 172.21.15.92:0/498848948 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b5000e040 con 0x7f0b5c0ffee0 2024-05-06T12:13:15.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.707+0000 7f0b61884700 1 -- 172.21.15.92:0/498848948 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b5c0fc240 con 0x7f0b5c0ffee0 2024-05-06T12:13:15.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.707+0000 7f0b437fe700 1 -- 172.21.15.92:0/498848948 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b5000bcc0 con 0x7f0b5c0ffee0 2024-05-06T12:13:15.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.708+0000 7f0b437fe700 1 -- 172.21.15.92:0/498848948 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b5001c630 con 0x7f0b5c0ffee0 2024-05-06T12:13:15.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.708+0000 7f0b61884700 1 -- 172.21.15.92:0/498848948 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b5c1159e0 con 0x7f0b5c0ffee0 2024-05-06T12:13:15.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.709+0000 7f0b61884700 1 -- 172.21.15.92:0/498848948 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b5c05f5c0 con 0x7f0b5c0ffee0 2024-05-06T12:13:15.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.709+0000 7f0b437fe700 1 -- 172.21.15.92:0/498848948 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f0b50013420 con 0x7f0b5c0ffee0 2024-05-06T12:13:15.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.710+0000 7f0b437fe700 1 --2- 172.21.15.92:0/498848948 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f0b440692a0 0x7f0b4406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:15.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.710+0000 7f0b437fe700 1 -- 172.21.15.92:0/498848948 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f0b50087da0 con 0x7f0b5c0ffee0 2024-05-06T12:13:15.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.710+0000 7f0b5affd700 1 --2- 172.21.15.92:0/498848948 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f0b440692a0 0x7f0b4406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:15.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.711+0000 7f0b5affd700 1 --2- 172.21.15.92:0/498848948 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f0b440692a0 0x7f0b4406b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c009b80 tx=0x7f0b4c00b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:13:15.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:15.718+0000 7f0b437fe700 1 -- 172.21.15.92:0/498848948 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0b500541d0 con 0x7f0b5c0ffee0 2024-05-06T12:13:16.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.038+0000 7f0b61884700 1 -- 172.21.15.92:0/498848948 --> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b5c0f8f20 con 0x7f0b440692a0 2024-05-06T12:13:16.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.039+0000 7f0b437fe700 1 -- 172.21.15.92:0/498848948 <== mgr.14210 v2:172.21.15.92:6800/3045905943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f0b5c0f8f20 con 0x7f0b440692a0 2024-05-06T12:13:16.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.042+0000 7f0b61884700 1 -- 172.21.15.92:0/498848948 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f0b440692a0 msgr2=0x7f0b4406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:16.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.042+0000 7f0b61884700 1 --2- 172.21.15.92:0/498848948 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f0b440692a0 0x7f0b4406b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c009b80 tx=0x7f0b4c00b040 comp rx=0 tx=0).stop 2024-05-06T12:13:16.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.042+0000 7f0b61884700 1 -- 172.21.15.92:0/498848948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b5c0ffee0 msgr2=0x7f0b5c0fb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:16.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.042+0000 7f0b61884700 1 --2- 172.21.15.92:0/498848948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b5c0ffee0 0x7f0b5c0fb8a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0b50009150 tx=0x7f0b50009e30 comp rx=0 tx=0).stop 2024-05-06T12:13:16.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.042+0000 7f0b61884700 1 -- 172.21.15.92:0/498848948 shutdown_connections 2024-05-06T12:13:16.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.042+0000 7f0b61884700 1 --2- 172.21.15.92:0/498848948 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f0b440692a0 0x7f0b4406b750 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:16.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.042+0000 7f0b61884700 1 --2- 172.21.15.92:0/498848948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b5c0662c0 0x7f0b5c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:16.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.042+0000 7f0b61884700 1 --2- 172.21.15.92:0/498848948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b5c0ffee0 0x7f0b5c0fb8a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:16.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.042+0000 7f0b61884700 1 -- 172.21.15.92:0/498848948 >> 172.21.15.92:0/498848948 conn(0x7f0b5c0f7430 msgr2=0x7f0b5c0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:13:16.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.043+0000 7f0b61884700 1 -- 172.21.15.92:0/498848948 shutdown_connections 2024-05-06T12:13:16.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.043+0000 7f0b61884700 1 -- 172.21.15.92:0/498848948 wait complete. 2024-05-06T12:13:16.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.165+0000 7f4301016700 1 -- 172.21.15.92:0/4132862480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42fc0fbce0 msgr2=0x7f42fc0fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:16.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.165+0000 7f4301016700 1 --2- 172.21.15.92:0/4132862480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42fc0fbce0 0x7f42fc0fc0f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f42ec004940 tx=0x7f42ec009e30 comp rx=0 tx=0).stop 2024-05-06T12:13:16.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.165+0000 7f4301016700 1 -- 172.21.15.92:0/4132862480 shutdown_connections 2024-05-06T12:13:16.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.165+0000 7f4301016700 1 --2- 172.21.15.92:0/4132862480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42fc0fcf90 0x7f42fc0fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:16.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.165+0000 7f4301016700 1 --2- 172.21.15.92:0/4132862480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42fc0fbce0 0x7f42fc0fc0f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:16.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.165+0000 7f4301016700 1 -- 172.21.15.92:0/4132862480 >> 172.21.15.92:0/4132862480 conn(0x7f42fc0f7420 msgr2=0x7f42fc0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:13:16.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.165+0000 7f4301016700 1 -- 172.21.15.92:0/4132862480 shutdown_connections 2024-05-06T12:13:16.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.165+0000 7f4301016700 1 -- 172.21.15.92:0/4132862480 wait complete. 2024-05-06T12:13:16.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.166+0000 7f4301016700 1 Processor -- start 2024-05-06T12:13:16.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.166+0000 7f4301016700 1 -- start start 2024-05-06T12:13:16.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.166+0000 7f4301016700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42fc0fbce0 0x7f42fc0ffc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:16.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.166+0000 7f4301016700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42fc0fcf90 0x7f42fc0fe2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:16.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.166+0000 7f4301016700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42fc100220 con 0x7f42fc0fcf90 2024-05-06T12:13:16.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.166+0000 7f4301016700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42fc111580 con 0x7f42fc0fbce0 2024-05-06T12:13:16.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.166+0000 7f42fa59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42fc0fcf90 0x7f42fc0fe2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:16.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.166+0000 7f42fa59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42fc0fcf90 0x7f42fc0fe2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41332/0 (socket says 172.21.15.92:41332) 2024-05-06T12:13:16.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.166+0000 7f42fa59c700 1 -- 172.21.15.92:0/3218920832 learned_addr learned my addr 172.21.15.92:0/3218920832 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:13:16.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.166+0000 7f42fad9d700 1 --2- 172.21.15.92:0/3218920832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42fc0fbce0 0x7f42fc0ffc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:16.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.167+0000 7f42fad9d700 1 -- 172.21.15.92:0/3218920832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42fc0fcf90 msgr2=0x7f42fc0fe2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:16.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.167+0000 7f42fad9d700 1 --2- 172.21.15.92:0/3218920832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42fc0fcf90 0x7f42fc0fe2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:16.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.167+0000 7f42fad9d700 1 -- 172.21.15.92:0/3218920832 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ec005040 con 0x7f42fc0fbce0 2024-05-06T12:13:16.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.167+0000 7f42fa59c700 1 --2- 172.21.15.92:0/3218920832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42fc0fcf90 0x7f42fc0fe2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-06T12:13:16.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.167+0000 7f42fad9d700 1 --2- 172.21.15.92:0/3218920832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42fc0fbce0 0x7f42fc0ffc20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f42ec000c00 tx=0x7f42ec009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:13:16.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.167+0000 7f42e37fe700 1 -- 172.21.15.92:0/3218920832 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42ec00e030 con 0x7f42fc0fbce0 2024-05-06T12:13:16.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.167+0000 7f4301016700 1 -- 172.21.15.92:0/3218920832 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42fc1117d0 con 0x7f42fc0fbce0 2024-05-06T12:13:16.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.167+0000 7f4301016700 1 -- 172.21.15.92:0/3218920832 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42fc0febc0 con 0x7f42fc0fbce0 2024-05-06T12:13:16.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.167+0000 7f42e37fe700 1 -- 172.21.15.92:0/3218920832 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f42ec013de0 con 0x7f42fc0fbce0 2024-05-06T12:13:16.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.167+0000 7f42e37fe700 1 -- 172.21.15.92:0/3218920832 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42ec01ce50 con 0x7f42fc0fbce0 2024-05-06T12:13:16.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.168+0000 7f4301016700 1 -- 172.21.15.92:0/3218920832 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42fc107370 con 0x7f42fc0fbce0 2024-05-06T12:13:16.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.169+0000 7f42e37fe700 1 -- 172.21.15.92:0/3218920832 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f42ec01c400 con 0x7f42fc0fbce0 2024-05-06T12:13:16.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.170+0000 7f42e37fe700 1 --2- 172.21.15.92:0/3218920832 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f42e4069240 0x7f42e406b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:16.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.170+0000 7f42e37fe700 1 -- 172.21.15.92:0/3218920832 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f42ec088ad0 con 0x7f42fc0fbce0 2024-05-06T12:13:16.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.170+0000 7f42fa59c700 1 --2- 172.21.15.92:0/3218920832 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f42e4069240 0x7f42e406b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:16.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.170+0000 7f42fa59c700 1 --2- 172.21.15.92:0/3218920832 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f42e4069240 0x7f42e406b6f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f42f00041d0 tx=0x7f42f000b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:13:16.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.177+0000 7f42e37fe700 1 -- 172.21.15.92:0/3218920832 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f42ec0587d0 con 0x7f42fc0fbce0 2024-05-06T12:13:16.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.420+0000 7f4301016700 1 -- 172.21.15.92:0/3218920832 --> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f42fc1117b0 con 0x7f42e4069240 2024-05-06T12:13:16.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.435+0000 7f42e37fe700 1 -- 172.21.15.92:0/3218920832 <== mgr.14210 v2:172.21.15.92:6800/3045905943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7f42fc1117b0 con 0x7f42e4069240 2024-05-06T12:13:16.439 INFO:teuthology.orchestra.run.smithi092.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-06T12:13:16.439 INFO:teuthology.orchestra.run.smithi092.stdout:alertmanager.smithi092 smithi092 *:9093,9094 running (5m) 60s ago 6m 25.7M - 0.25.0 c8568f914cd2 dab380a7352e 2024-05-06T12:13:16.439 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi092 smithi092 running (7m) 60s ago 7m 6660k - 17.2.7-925-ge81df420 96d5ae7fd4b4 95dc7e14bd9a 2024-05-06T12:13:16.439 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi190 smithi190 running (6m) 66s ago 6m 6665k - 17.2.7-925-ge81df420 96d5ae7fd4b4 6cf97785406f 2024-05-06T12:13:16.439 INFO:teuthology.orchestra.run.smithi092.stdout:grafana.smithi092 smithi092 *:3000 running (5m) 60s ago 6m 84.6M - 9.4.7 954c08fa6188 841f34687c12 2024-05-06T12:13:16.439 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.ijsqgp smithi092 running (88s) 60s ago 87s 11.4M - 17.2.7-925-ge81df420 96d5ae7fd4b4 b4c2ed6d2c82 2024-05-06T12:13:16.439 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.rosbll smithi092 running (80s) 60s ago 79s 13.7M - 17.2.7-925-ge81df420 96d5ae7fd4b4 5f5426cd1bf5 2024-05-06T12:13:16.439 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.byjvgf smithi190 running (85s) 66s ago 84s 16.7M - 17.2.7-925-ge81df420 96d5ae7fd4b4 24581d2e1c5d 2024-05-06T12:13:16.439 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.idzoff smithi190 running (77s) 66s ago 77s 13.3M - 17.2.7-925-ge81df420 96d5ae7fd4b4 20f2641696ef 2024-05-06T12:13:16.440 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi092.gyzeqb smithi092 *:9283 running (8m) 60s ago 8m 480M - 17.2.7-925-ge81df420 96d5ae7fd4b4 820ba9ba8a82 2024-05-06T12:13:16.440 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi190.kczxkh smithi190 *:8443,9283 running (5m) 66s ago 5m 431M - 17.2.7-925-ge81df420 96d5ae7fd4b4 fb57adc25b2b 2024-05-06T12:13:16.440 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi092 smithi092 running (8m) 60s ago 8m 42.8M 2048M 17.2.7-925-ge81df420 96d5ae7fd4b4 df5b322e9719 2024-05-06T12:13:16.440 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi190 smithi190 running (5m) 66s ago 5m 32.5M 2048M 17.2.7-925-ge81df420 96d5ae7fd4b4 912ded072601 2024-05-06T12:13:16.440 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi092 smithi092 *:9100 running (6m) 60s ago 6m 15.8M - 1.5.0 0da6a335fe13 86cf73a245ab 2024-05-06T12:13:16.440 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi190 smithi190 *:9100 running (5m) 66s ago 5m 16.0M - 1.5.0 0da6a335fe13 da5c2a030c69 2024-05-06T12:13:16.440 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 smithi092 running (5m) 60s ago 5m 47.0M 4288M 17.2.7-925-ge81df420 96d5ae7fd4b4 ce84e7af9084 2024-05-06T12:13:16.440 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 smithi092 running (4m) 60s ago 4m 48.8M 4288M 17.2.7-925-ge81df420 96d5ae7fd4b4 8ab870f57eb2 2024-05-06T12:13:16.440 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 smithi092 running (4m) 60s ago 4m 46.0M 4288M 17.2.7-925-ge81df420 96d5ae7fd4b4 1e6ed04a0688 2024-05-06T12:13:16.440 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 smithi190 running (3m) 66s ago 3m 43.8M 5312M 17.2.7-925-ge81df420 96d5ae7fd4b4 52f9521286d5 2024-05-06T12:13:16.440 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 smithi190 running (3m) 66s ago 3m 42.4M 5312M 17.2.7-925-ge81df420 96d5ae7fd4b4 3a7da09e4c48 2024-05-06T12:13:16.440 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 smithi190 running (2m) 66s ago 2m 41.7M 5312M 17.2.7-925-ge81df420 96d5ae7fd4b4 e7452c6b5cec 2024-05-06T12:13:16.441 INFO:teuthology.orchestra.run.smithi092.stdout:prometheus.smithi092 smithi092 *:9095 running (5m) 60s ago 6m 56.7M - 2.43.0 a07b618ecd1d ef92f3572328 2024-05-06T12:13:16.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.438+0000 7f4301016700 1 -- 172.21.15.92:0/3218920832 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f42e4069240 msgr2=0x7f42e406b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:16.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.438+0000 7f4301016700 1 --2- 172.21.15.92:0/3218920832 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f42e4069240 0x7f42e406b6f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f42f00041d0 tx=0x7f42f000b040 comp rx=0 tx=0).stop 2024-05-06T12:13:16.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.438+0000 7f4301016700 1 -- 172.21.15.92:0/3218920832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42fc0fbce0 msgr2=0x7f42fc0ffc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:16.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.438+0000 7f4301016700 1 --2- 172.21.15.92:0/3218920832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42fc0fbce0 0x7f42fc0ffc20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f42ec000c00 tx=0x7f42ec009e30 comp rx=0 tx=0).stop 2024-05-06T12:13:16.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.439+0000 7f4301016700 1 -- 172.21.15.92:0/3218920832 shutdown_connections 2024-05-06T12:13:16.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.439+0000 7f4301016700 1 --2- 172.21.15.92:0/3218920832 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f42e4069240 0x7f42e406b6f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:16.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.439+0000 7f4301016700 1 --2- 172.21.15.92:0/3218920832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42fc0fbce0 0x7f42fc0ffc20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:16.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.439+0000 7f4301016700 1 --2- 172.21.15.92:0/3218920832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42fc0fcf90 0x7f42fc0fe2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:16.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.439+0000 7f4301016700 1 -- 172.21.15.92:0/3218920832 >> 172.21.15.92:0/3218920832 conn(0x7f42fc0f7420 msgr2=0x7f42fc0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:13:16.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.439+0000 7f4301016700 1 -- 172.21.15.92:0/3218920832 shutdown_connections 2024-05-06T12:13:16.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.439+0000 7f4301016700 1 -- 172.21.15.92:0/3218920832 wait complete. 2024-05-06T12:13:16.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.550+0000 7f34eb718700 1 -- 172.21.15.92:0/3043752999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34e40fbce0 msgr2=0x7f34e40fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:16.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.550+0000 7f34eb718700 1 --2- 172.21.15.92:0/3043752999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34e40fbce0 0x7f34e40fc0f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f34e0004940 tx=0x7f34e0009e30 comp rx=0 tx=0).stop 2024-05-06T12:13:16.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.550+0000 7f34eb718700 1 -- 172.21.15.92:0/3043752999 shutdown_connections 2024-05-06T12:13:16.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.550+0000 7f34eb718700 1 --2- 172.21.15.92:0/3043752999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34e40fcf90 0x7f34e40fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:16.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.550+0000 7f34eb718700 1 --2- 172.21.15.92:0/3043752999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34e40fbce0 0x7f34e40fc0f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:16.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.550+0000 7f34eb718700 1 -- 172.21.15.92:0/3043752999 >> 172.21.15.92:0/3043752999 conn(0x7f34e40f7420 msgr2=0x7f34e40f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:13:16.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.550+0000 7f34eb718700 1 -- 172.21.15.92:0/3043752999 shutdown_connections 2024-05-06T12:13:16.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.550+0000 7f34eb718700 1 -- 172.21.15.92:0/3043752999 wait complete. 2024-05-06T12:13:16.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.551+0000 7f34eb718700 1 Processor -- start 2024-05-06T12:13:16.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.551+0000 7f34eb718700 1 -- start start 2024-05-06T12:13:16.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.551+0000 7f34eb718700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34e40fbce0 0x7f34e40ffc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:16.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.551+0000 7f34eb718700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34e40fcf90 0x7f34e40fe2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:16.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.551+0000 7f34eb718700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e4100220 con 0x7f34e40fcf90 2024-05-06T12:13:16.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.551+0000 7f34eb718700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e41115b0 con 0x7f34e40fbce0 2024-05-06T12:13:16.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.551+0000 7f34e8cb3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34e40fcf90 0x7f34e40fe2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:16.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.551+0000 7f34e8cb3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34e40fcf90 0x7f34e40fe2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41354/0 (socket says 172.21.15.92:41354) 2024-05-06T12:13:16.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.551+0000 7f34e8cb3700 1 -- 172.21.15.92:0/2008843032 learned_addr learned my addr 172.21.15.92:0/2008843032 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:13:16.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.551+0000 7f34e94b4700 1 --2- 172.21.15.92:0/2008843032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34e40fbce0 0x7f34e40ffc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:16.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.552+0000 7f34e8cb3700 1 -- 172.21.15.92:0/2008843032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34e40fbce0 msgr2=0x7f34e40ffc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:16.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.552+0000 7f34e8cb3700 1 --2- 172.21.15.92:0/2008843032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34e40fbce0 0x7f34e40ffc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:16.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.552+0000 7f34e8cb3700 1 -- 172.21.15.92:0/2008843032 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e0005040 con 0x7f34e40fcf90 2024-05-06T12:13:16.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.552+0000 7f34e8cb3700 1 --2- 172.21.15.92:0/2008843032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34e40fcf90 0x7f34e40fe2d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f34d4002a80 tx=0x7f34d40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:13:16.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.552+0000 7f34d9ffb700 1 -- 172.21.15.92:0/2008843032 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34d4007cc0 con 0x7f34e40fcf90 2024-05-06T12:13:16.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.552+0000 7f34d9ffb700 1 -- 172.21.15.92:0/2008843032 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f34d400f040 con 0x7f34e40fcf90 2024-05-06T12:13:16.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.552+0000 7f34eb718700 1 -- 172.21.15.92:0/2008843032 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34e40fe810 con 0x7f34e40fcf90 2024-05-06T12:13:16.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.552+0000 7f34d9ffb700 1 -- 172.21.15.92:0/2008843032 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34d400dac0 con 0x7f34e40fcf90 2024-05-06T12:13:16.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.553+0000 7f34eb718700 1 -- 172.21.15.92:0/2008843032 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e40fedd0 con 0x7f34e40fcf90 2024-05-06T12:13:16.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.554+0000 7f34eb718700 1 -- 172.21.15.92:0/2008843032 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34e4101b20 con 0x7f34e40fcf90 2024-05-06T12:13:16.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.554+0000 7f34d9ffb700 1 -- 172.21.15.92:0/2008843032 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f34d400dc20 con 0x7f34e40fcf90 2024-05-06T12:13:16.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.555+0000 7f34d9ffb700 1 --2- 172.21.15.92:0/2008843032 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f34d00692a0 0x7f34d006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:16.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.555+0000 7f34d9ffb700 1 -- 172.21.15.92:0/2008843032 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f34d4086d70 con 0x7f34e40fcf90 2024-05-06T12:13:16.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.555+0000 7f34e94b4700 1 --2- 172.21.15.92:0/2008843032 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f34d00692a0 0x7f34d006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:16.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.556+0000 7f34e94b4700 1 --2- 172.21.15.92:0/2008843032 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f34d00692a0 0x7f34d006b750 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f34e000b3b0 tx=0x7f34e0009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:13:16.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.560+0000 7f34d9ffb700 1 -- 172.21.15.92:0/2008843032 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f34d4056ae0 con 0x7f34e40fcf90 2024-05-06T12:13:16.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.861+0000 7f34eb718700 1 -- 172.21.15.92:0/2008843032 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f34e404c710 con 0x7f34e40fcf90 2024-05-06T12:13:16.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.861+0000 7f34d9ffb700 1 -- 172.21.15.92:0/2008843032 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f34d4056670 con 0x7f34e40fcf90 2024-05-06T12:13:16.864 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:13:16.865 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": { 2024-05-06T12:13:16.865 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 2 2024-05-06T12:13:16.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:13:16.865 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": { 2024-05-06T12:13:16.865 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 2 2024-05-06T12:13:16.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:13:16.865 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": { 2024-05-06T12:13:16.865 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 6 2024-05-06T12:13:16.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:13:16.865 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-05-06T12:13:16.865 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 4 2024-05-06T12:13:16.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:13:16.866 INFO:teuthology.orchestra.run.smithi092.stdout: "overall": { 2024-05-06T12:13:16.866 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 14 2024-05-06T12:13:16.866 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-06T12:13:16.866 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:13:16.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.863+0000 7f34eb718700 1 -- 172.21.15.92:0/2008843032 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f34d00692a0 msgr2=0x7f34d006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:16.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.863+0000 7f34eb718700 1 --2- 172.21.15.92:0/2008843032 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f34d00692a0 0x7f34d006b750 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f34e000b3b0 tx=0x7f34e0009e30 comp rx=0 tx=0).stop 2024-05-06T12:13:16.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.863+0000 7f34eb718700 1 -- 172.21.15.92:0/2008843032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34e40fcf90 msgr2=0x7f34e40fe2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:16.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.863+0000 7f34eb718700 1 --2- 172.21.15.92:0/2008843032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34e40fcf90 0x7f34e40fe2d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f34d4002a80 tx=0x7f34d40046b0 comp rx=0 tx=0).stop 2024-05-06T12:13:16.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.864+0000 7f34eb718700 1 -- 172.21.15.92:0/2008843032 shutdown_connections 2024-05-06T12:13:16.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.864+0000 7f34eb718700 1 --2- 172.21.15.92:0/2008843032 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f34d00692a0 0x7f34d006b750 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:16.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.864+0000 7f34eb718700 1 --2- 172.21.15.92:0/2008843032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34e40fbce0 0x7f34e40ffc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:16.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.864+0000 7f34eb718700 1 --2- 172.21.15.92:0/2008843032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34e40fcf90 0x7f34e40fe2d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:16.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.864+0000 7f34eb718700 1 -- 172.21.15.92:0/2008843032 >> 172.21.15.92:0/2008843032 conn(0x7f34e40f7420 msgr2=0x7f34e4066720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:13:16.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.864+0000 7f34eb718700 1 -- 172.21.15.92:0/2008843032 shutdown_connections 2024-05-06T12:13:16.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.864+0000 7f34eb718700 1 -- 172.21.15.92:0/2008843032 wait complete. 2024-05-06T12:13:16.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.969+0000 7fb62b295700 1 -- 172.21.15.92:0/385366734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6240fcff0 msgr2=0x7fb6240fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:16.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.969+0000 7fb62b295700 1 --2- 172.21.15.92:0/385366734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6240fcff0 0x7fb6240fd460 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb614004940 tx=0x7fb614009e30 comp rx=0 tx=0).stop 2024-05-06T12:13:16.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.969+0000 7fb62b295700 1 -- 172.21.15.92:0/385366734 shutdown_connections 2024-05-06T12:13:16.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.969+0000 7fb62b295700 1 --2- 172.21.15.92:0/385366734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6240fcff0 0x7fb6240fd460 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:16.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.969+0000 7fb62b295700 1 --2- 172.21.15.92:0/385366734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6240fbd40 0x7fb6240fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:16.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.969+0000 7fb62b295700 1 -- 172.21.15.92:0/385366734 >> 172.21.15.92:0/385366734 conn(0x7fb6240f74c0 msgr2=0x7fb6240f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:13:16.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.969+0000 7fb62b295700 1 -- 172.21.15.92:0/385366734 shutdown_connections 2024-05-06T12:13:16.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.969+0000 7fb62b295700 1 -- 172.21.15.92:0/385366734 wait complete. 2024-05-06T12:13:16.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.970+0000 7fb62b295700 1 Processor -- start 2024-05-06T12:13:16.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.970+0000 7fb62b295700 1 -- start start 2024-05-06T12:13:16.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.970+0000 7fb62b295700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6240fbd40 0x7fb6241106b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:16.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.970+0000 7fb62b295700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6240fcff0 0x7fb624112c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:16.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.970+0000 7fb62b295700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb624066bc0 con 0x7fb6240fcff0 2024-05-06T12:13:16.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.970+0000 7fb62b295700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb624066d00 con 0x7fb6240fbd40 2024-05-06T12:13:16.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.970+0000 7fb628830700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6240fcff0 0x7fb624112c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:16.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.970+0000 7fb628830700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6240fcff0 0x7fb624112c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41378/0 (socket says 172.21.15.92:41378) 2024-05-06T12:13:16.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.970+0000 7fb628830700 1 -- 172.21.15.92:0/3021229832 learned_addr learned my addr 172.21.15.92:0/3021229832 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:13:16.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.970+0000 7fb629031700 1 --2- 172.21.15.92:0/3021229832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6240fbd40 0x7fb6241106b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:16.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.970+0000 7fb628830700 1 -- 172.21.15.92:0/3021229832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6240fbd40 msgr2=0x7fb6241106b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:16.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.970+0000 7fb628830700 1 --2- 172.21.15.92:0/3021229832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6240fbd40 0x7fb6241106b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:16.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.970+0000 7fb628830700 1 -- 172.21.15.92:0/3021229832 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb614005040 con 0x7fb6240fcff0 2024-05-06T12:13:16.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.971+0000 7fb628830700 1 --2- 172.21.15.92:0/3021229832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6240fcff0 0x7fb624112c00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb614009150 tx=0x7fb614009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:13:16.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.971+0000 7fb619ffb700 1 -- 172.21.15.92:0/3021229832 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb61400e040 con 0x7fb6240fcff0 2024-05-06T12:13:16.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.971+0000 7fb62b295700 1 -- 172.21.15.92:0/3021229832 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb624113140 con 0x7fb6240fcff0 2024-05-06T12:13:16.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.971+0000 7fb619ffb700 1 -- 172.21.15.92:0/3021229832 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb614004290 con 0x7fb6240fcff0 2024-05-06T12:13:16.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.971+0000 7fb619ffb700 1 -- 172.21.15.92:0/3021229832 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb61401c630 con 0x7fb6240fcff0 2024-05-06T12:13:16.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.971+0000 7fb62b295700 1 -- 172.21.15.92:0/3021229832 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb624113700 con 0x7fb6240fcff0 2024-05-06T12:13:16.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.972+0000 7fb62b295700 1 -- 172.21.15.92:0/3021229832 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb62405f5c0 con 0x7fb6240fcff0 2024-05-06T12:13:16.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.973+0000 7fb619ffb700 1 -- 172.21.15.92:0/3021229832 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fb61400bbf0 con 0x7fb6240fcff0 2024-05-06T12:13:16.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.973+0000 7fb619ffb700 1 --2- 172.21.15.92:0/3021229832 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fb6100692a0 0x7fb61006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:16.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.973+0000 7fb619ffb700 1 -- 172.21.15.92:0/3021229832 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fb614087dc0 con 0x7fb6240fcff0 2024-05-06T12:13:16.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.974+0000 7fb629031700 1 --2- 172.21.15.92:0/3021229832 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fb6100692a0 0x7fb61006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:16.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.974+0000 7fb629031700 1 --2- 172.21.15.92:0/3021229832 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fb6100692a0 0x7fb61006b750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb624066840 tx=0x7fb62000b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:13:16.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:16.978+0000 7fb619ffb700 1 -- 172.21.15.92:0/3021229832 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb614054140 con 0x7fb6240fcff0 2024-05-06T12:13:17.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:16 smithi092 ceph-mon[26954]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:13:17.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:16 smithi092 ceph-mon[26954]: pgmap v220: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:13:17.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:16 smithi092 ceph-mon[26954]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:13:17.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:16 smithi092 ceph-mon[26954]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:13:17.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:16 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/2008843032' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:13:17.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.315+0000 7fb62b295700 1 -- 172.21.15.92:0/3021229832 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb62404bed0 con 0x7fb6240fcff0 2024-05-06T12:13:17.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.315+0000 7fb619ffb700 1 -- 172.21.15.92:0/3021229832 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1833 (secure 0 0 0) 0x7fb614057760 con 0x7fb6240fcff0 2024-05-06T12:13:17.319 INFO:teuthology.orchestra.run.smithi092.stdout:e18 2024-05-06T12:13:17.319 INFO:teuthology.orchestra.run.smithi092.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-06T12:13:17.319 INFO:teuthology.orchestra.run.smithi092.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:13:17.319 INFO:teuthology.orchestra.run.smithi092.stdout:legacy client fscid: 1 2024-05-06T12:13:17.319 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:13:17.319 INFO:teuthology.orchestra.run.smithi092.stdout:Filesystem 'cephfs' (1) 2024-05-06T12:13:17.319 INFO:teuthology.orchestra.run.smithi092.stdout:fs_name cephfs 2024-05-06T12:13:17.319 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 18 2024-05-06T12:13:17.319 INFO:teuthology.orchestra.run.smithi092.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-06T12:13:17.319 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-05-06T12:11:44.952671+0000 2024-05-06T12:13:17.319 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2024-05-06T12:12:04.907000+0000 2024-05-06T12:13:17.319 INFO:teuthology.orchestra.run.smithi092.stdout:tableserver 0 2024-05-06T12:13:17.320 INFO:teuthology.orchestra.run.smithi092.stdout:root 0 2024-05-06T12:13:17.320 INFO:teuthology.orchestra.run.smithi092.stdout:session_timeout 60 2024-05-06T12:13:17.320 INFO:teuthology.orchestra.run.smithi092.stdout:session_autoclose 300 2024-05-06T12:13:17.320 INFO:teuthology.orchestra.run.smithi092.stdout:max_file_size 1099511627776 2024-05-06T12:13:17.320 INFO:teuthology.orchestra.run.smithi092.stdout:required_client_features {} 2024-05-06T12:13:17.320 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure 0 2024-05-06T12:13:17.320 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure_osd_epoch 46 2024-05-06T12:13:17.320 INFO:teuthology.orchestra.run.smithi092.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:13:17.320 INFO:teuthology.orchestra.run.smithi092.stdout:max_mds 2 2024-05-06T12:13:17.320 INFO:teuthology.orchestra.run.smithi092.stdout:in 0,1 2024-05-06T12:13:17.320 INFO:teuthology.orchestra.run.smithi092.stdout:up {0=14472,1=24267} 2024-05-06T12:13:17.320 INFO:teuthology.orchestra.run.smithi092.stdout:failed 2024-05-06T12:13:17.320 INFO:teuthology.orchestra.run.smithi092.stdout:damaged 2024-05-06T12:13:17.320 INFO:teuthology.orchestra.run.smithi092.stdout:stopped 2024-05-06T12:13:17.320 INFO:teuthology.orchestra.run.smithi092.stdout:data_pools [3] 2024-05-06T12:13:17.321 INFO:teuthology.orchestra.run.smithi092.stdout:metadata_pool 2 2024-05-06T12:13:17.321 INFO:teuthology.orchestra.run.smithi092.stdout:inline_data enabled 2024-05-06T12:13:17.321 INFO:teuthology.orchestra.run.smithi092.stdout:balancer 2024-05-06T12:13:17.321 INFO:teuthology.orchestra.run.smithi092.stdout:standby_count_wanted 1 2024-05-06T12:13:17.321 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.rosbll{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.92:6828/1663253598,v1:172.21.15.92:6829/1663253598] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:13:17.321 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.ijsqgp{0:14484} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.92:6826/3613441054,v1:172.21.15.92:6827/3613441054] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:13:17.321 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.byjvgf{1:24267} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.190:6824/1565151699,v1:172.21.15.190:6825/1565151699] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:13:17.321 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.idzoff{1:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.190:6826/2453728380,v1:172.21.15.190:6827/2453728380] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:13:17.321 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:13:17.321 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:13:17.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.318+0000 7fb62b295700 1 -- 172.21.15.92:0/3021229832 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fb6100692a0 msgr2=0x7fb61006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:17.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.318+0000 7fb62b295700 1 --2- 172.21.15.92:0/3021229832 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fb6100692a0 0x7fb61006b750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb624066840 tx=0x7fb62000b040 comp rx=0 tx=0).stop 2024-05-06T12:13:17.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.318+0000 7fb62b295700 1 -- 172.21.15.92:0/3021229832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6240fcff0 msgr2=0x7fb624112c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:17.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.318+0000 7fb62b295700 1 --2- 172.21.15.92:0/3021229832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6240fcff0 0x7fb624112c00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb614009150 tx=0x7fb614009e30 comp rx=0 tx=0).stop 2024-05-06T12:13:17.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.318+0000 7fb62b295700 1 -- 172.21.15.92:0/3021229832 shutdown_connections 2024-05-06T12:13:17.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.318+0000 7fb62b295700 1 --2- 172.21.15.92:0/3021229832 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fb6100692a0 0x7fb61006b750 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:17.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.318+0000 7fb62b295700 1 --2- 172.21.15.92:0/3021229832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6240fbd40 0x7fb6241106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:17.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.318+0000 7fb62b295700 1 --2- 172.21.15.92:0/3021229832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6240fcff0 0x7fb624112c00 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:17.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.318+0000 7fb62b295700 1 -- 172.21.15.92:0/3021229832 >> 172.21.15.92:0/3021229832 conn(0x7fb6240f74c0 msgr2=0x7fb6240f8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:13:17.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.318+0000 7fb62b295700 1 -- 172.21.15.92:0/3021229832 shutdown_connections 2024-05-06T12:13:17.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.318+0000 7fb62b295700 1 -- 172.21.15.92:0/3021229832 wait complete. 2024-05-06T12:13:17.323 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 18 2024-05-06T12:13:17.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:16 smithi190 ceph-mon[33798]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:13:17.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:16 smithi190 ceph-mon[33798]: pgmap v220: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:13:17.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:16 smithi190 ceph-mon[33798]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:13:17.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:16 smithi190 ceph-mon[33798]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:13:17.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:16 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/2008843032' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:13:17.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.424+0000 7f9f9e8b5700 1 -- 172.21.15.92:0/4282774736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f98066ed0 msgr2=0x7f9f980ff9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:17.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.424+0000 7f9f9e8b5700 1 --2- 172.21.15.92:0/4282774736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f98066ed0 0x7f9f980ff9d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9f88004940 tx=0x7f9f88009e30 comp rx=0 tx=0).stop 2024-05-06T12:13:17.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.424+0000 7f9f9e8b5700 1 -- 172.21.15.92:0/4282774736 shutdown_connections 2024-05-06T12:13:17.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.424+0000 7f9f9e8b5700 1 --2- 172.21.15.92:0/4282774736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f980fff10 0x7f9f98102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:17.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.424+0000 7f9f9e8b5700 1 --2- 172.21.15.92:0/4282774736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f98066ed0 0x7f9f980ff9d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:17.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.424+0000 7f9f9e8b5700 1 -- 172.21.15.92:0/4282774736 >> 172.21.15.92:0/4282774736 conn(0x7f9f980f7420 msgr2=0x7f9f980f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:13:17.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.424+0000 7f9f9e8b5700 1 -- 172.21.15.92:0/4282774736 shutdown_connections 2024-05-06T12:13:17.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.424+0000 7f9f9e8b5700 1 -- 172.21.15.92:0/4282774736 wait complete. 2024-05-06T12:13:17.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.424+0000 7f9f9e8b5700 1 Processor -- start 2024-05-06T12:13:17.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.425+0000 7f9f9e8b5700 1 -- start start 2024-05-06T12:13:17.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.425+0000 7f9f9e8b5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f98066ed0 0x7f9f9810d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:17.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.425+0000 7f9f9e8b5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f980fff10 0x7f9f9810de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:17.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.425+0000 7f9f9e8b5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9810e440 con 0x7f9f980fff10 2024-05-06T12:13:17.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.425+0000 7f9f9e8b5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9810e5b0 con 0x7f9f98066ed0 2024-05-06T12:13:17.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.425+0000 7f9f977fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f980fff10 0x7f9f9810de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:17.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.425+0000 7f9f97fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f98066ed0 0x7f9f9810d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:17.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.425+0000 7f9f97fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f98066ed0 0x7f9f9810d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.92:50644/0 (socket says 172.21.15.92:50644) 2024-05-06T12:13:17.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.425+0000 7f9f977fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f980fff10 0x7f9f9810de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41390/0 (socket says 172.21.15.92:41390) 2024-05-06T12:13:17.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.425+0000 7f9f97fff700 1 -- 172.21.15.92:0/2564832529 learned_addr learned my addr 172.21.15.92:0/2564832529 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:13:17.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.425+0000 7f9f977fe700 1 -- 172.21.15.92:0/2564832529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f98066ed0 msgr2=0x7f9f9810d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:17.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.426+0000 7f9f977fe700 1 --2- 172.21.15.92:0/2564832529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f98066ed0 0x7f9f9810d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:17.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.426+0000 7f9f977fe700 1 -- 172.21.15.92:0/2564832529 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f88005040 con 0x7f9f980fff10 2024-05-06T12:13:17.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.426+0000 7f9f977fe700 1 --2- 172.21.15.92:0/2564832529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f980fff10 0x7f9f9810de40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c009fd0 tx=0x7f9f8c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:13:17.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.426+0000 7f9f94ff9700 1 -- 172.21.15.92:0/2564832529 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f8c004e90 con 0x7f9f980fff10 2024-05-06T12:13:17.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.426+0000 7f9f9e8b5700 1 -- 172.21.15.92:0/2564832529 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f98112f60 con 0x7f9f980fff10 2024-05-06T12:13:17.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.426+0000 7f9f94ff9700 1 -- 172.21.15.92:0/2564832529 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f8c00f040 con 0x7f9f980fff10 2024-05-06T12:13:17.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.426+0000 7f9f9e8b5700 1 -- 172.21.15.92:0/2564832529 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f981133b0 con 0x7f9f980fff10 2024-05-06T12:13:17.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.426+0000 7f9f94ff9700 1 -- 172.21.15.92:0/2564832529 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f8c00c9b0 con 0x7f9f980fff10 2024-05-06T12:13:17.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.427+0000 7f9f9e8b5700 1 -- 172.21.15.92:0/2564832529 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f9804c710 con 0x7f9f980fff10 2024-05-06T12:13:17.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.428+0000 7f9f94ff9700 1 -- 172.21.15.92:0/2564832529 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f9f8c007cc0 con 0x7f9f980fff10 2024-05-06T12:13:17.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.428+0000 7f9f94ff9700 1 --2- 172.21.15.92:0/2564832529 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f9f800692a0 0x7f9f8006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:17.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.428+0000 7f9f94ff9700 1 -- 172.21.15.92:0/2564832529 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f9f8c086bc0 con 0x7f9f980fff10 2024-05-06T12:13:17.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.429+0000 7f9f97fff700 1 --2- 172.21.15.92:0/2564832529 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f9f800692a0 0x7f9f8006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:17.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.429+0000 7f9f97fff700 1 --2- 172.21.15.92:0/2564832529 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f9f800692a0 0x7f9f8006b750 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9f880136a0 tx=0x7f9f88009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:13:17.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.437+0000 7f9f94ff9700 1 -- 172.21.15.92:0/2564832529 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9f8c04e250 con 0x7f9f980fff10 2024-05-06T12:13:17.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.681+0000 7f9f9e8b5700 1 -- 172.21.15.92:0/2564832529 --> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f980f8f80 con 0x7f9f800692a0 2024-05-06T12:13:17.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.682+0000 7f9f94ff9700 1 -- 172.21.15.92:0/2564832529 <== mgr.14210 v2:172.21.15.92:6800/3045905943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f9f980f8f80 con 0x7f9f800692a0 2024-05-06T12:13:17.685 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:13:17.686 INFO:teuthology.orchestra.run.smithi092.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb", 2024-05-06T12:13:17.686 INFO:teuthology.orchestra.run.smithi092.stdout: "in_progress": true, 2024-05-06T12:13:17.686 INFO:teuthology.orchestra.run.smithi092.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-06T12:13:17.686 INFO:teuthology.orchestra.run.smithi092.stdout: "services_complete": [], 2024-05-06T12:13:17.686 INFO:teuthology.orchestra.run.smithi092.stdout: "progress": "0/21 daemons upgraded", 2024-05-06T12:13:17.686 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb image on host smithi190", 2024-05-06T12:13:17.686 INFO:teuthology.orchestra.run.smithi092.stdout: "is_paused": false 2024-05-06T12:13:17.686 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:13:17.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.685+0000 7f9f9e8b5700 1 -- 172.21.15.92:0/2564832529 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f9f800692a0 msgr2=0x7f9f8006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:17.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.685+0000 7f9f9e8b5700 1 --2- 172.21.15.92:0/2564832529 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f9f800692a0 0x7f9f8006b750 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9f880136a0 tx=0x7f9f88009e30 comp rx=0 tx=0).stop 2024-05-06T12:13:17.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.685+0000 7f9f9e8b5700 1 -- 172.21.15.92:0/2564832529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f980fff10 msgr2=0x7f9f9810de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:17.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.685+0000 7f9f9e8b5700 1 --2- 172.21.15.92:0/2564832529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f980fff10 0x7f9f9810de40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c009fd0 tx=0x7f9f8c00b040 comp rx=0 tx=0).stop 2024-05-06T12:13:17.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.685+0000 7f9f9e8b5700 1 -- 172.21.15.92:0/2564832529 shutdown_connections 2024-05-06T12:13:17.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.685+0000 7f9f9e8b5700 1 --2- 172.21.15.92:0/2564832529 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f9f800692a0 0x7f9f8006b750 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:17.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.685+0000 7f9f9e8b5700 1 --2- 172.21.15.92:0/2564832529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f98066ed0 0x7f9f9810d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:17.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.685+0000 7f9f9e8b5700 1 --2- 172.21.15.92:0/2564832529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f980fff10 0x7f9f9810de40 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:17.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.685+0000 7f9f9e8b5700 1 -- 172.21.15.92:0/2564832529 >> 172.21.15.92:0/2564832529 conn(0x7f9f980f7420 msgr2=0x7f9f980f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:13:17.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.686+0000 7f9f9e8b5700 1 -- 172.21.15.92:0/2564832529 shutdown_connections 2024-05-06T12:13:17.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.686+0000 7f9f9e8b5700 1 -- 172.21.15.92:0/2564832529 wait complete. 2024-05-06T12:13:17.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.790+0000 7efe46388700 1 -- 172.21.15.92:0/4080358120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe40066b10 msgr2=0x7efe400ff9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:17.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.790+0000 7efe46388700 1 --2- 172.21.15.92:0/4080358120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe40066b10 0x7efe400ff9f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7efe30004940 tx=0x7efe30009e30 comp rx=0 tx=0).stop 2024-05-06T12:13:17.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.790+0000 7efe46388700 1 -- 172.21.15.92:0/4080358120 shutdown_connections 2024-05-06T12:13:17.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.790+0000 7efe46388700 1 --2- 172.21.15.92:0/4080358120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe400fff30 0x7efe40102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:17.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.790+0000 7efe46388700 1 --2- 172.21.15.92:0/4080358120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe40066b10 0x7efe400ff9f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:17.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.790+0000 7efe46388700 1 -- 172.21.15.92:0/4080358120 >> 172.21.15.92:0/4080358120 conn(0x7efe400f7420 msgr2=0x7efe400f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:13:17.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.790+0000 7efe46388700 1 -- 172.21.15.92:0/4080358120 shutdown_connections 2024-05-06T12:13:17.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.790+0000 7efe46388700 1 -- 172.21.15.92:0/4080358120 wait complete. 2024-05-06T12:13:17.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.790+0000 7efe46388700 1 Processor -- start 2024-05-06T12:13:17.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.791+0000 7efe46388700 1 -- start start 2024-05-06T12:13:17.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.791+0000 7efe46388700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe40066b10 0x7efe400fd380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:17.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.791+0000 7efe46388700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe400fff30 0x7efe400fb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:17.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.791+0000 7efe46388700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe400fbe30 con 0x7efe400fff30 2024-05-06T12:13:17.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.791+0000 7efe46388700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe400fbf70 con 0x7efe40066b10 2024-05-06T12:13:17.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.791+0000 7efe3ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe40066b10 0x7efe400fd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:17.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.791+0000 7efe3ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe40066b10 0x7efe400fd380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.92:50656/0 (socket says 172.21.15.92:50656) 2024-05-06T12:13:17.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.791+0000 7efe3ffff700 1 -- 172.21.15.92:0/543751259 learned_addr learned my addr 172.21.15.92:0/543751259 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:13:17.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.791+0000 7efe3f7fe700 1 --2- 172.21.15.92:0/543751259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe400fff30 0x7efe400fb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:17.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.791+0000 7efe3ffff700 1 -- 172.21.15.92:0/543751259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe400fff30 msgr2=0x7efe400fb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:17.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.791+0000 7efe3ffff700 1 --2- 172.21.15.92:0/543751259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe400fff30 0x7efe400fb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:17.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.791+0000 7efe3ffff700 1 -- 172.21.15.92:0/543751259 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe30005040 con 0x7efe40066b10 2024-05-06T12:13:17.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.791+0000 7efe3ffff700 1 --2- 172.21.15.92:0/543751259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe40066b10 0x7efe400fd380 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efe30000c00 tx=0x7efe30009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:13:17.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.792+0000 7efe3cff9700 1 -- 172.21.15.92:0/543751259 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe3000e030 con 0x7efe40066b10 2024-05-06T12:13:17.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.792+0000 7efe46388700 1 -- 172.21.15.92:0/543751259 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe400fc1f0 con 0x7efe40066b10 2024-05-06T12:13:17.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.792+0000 7efe3cff9700 1 -- 172.21.15.92:0/543751259 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe30013de0 con 0x7efe40066b10 2024-05-06T12:13:17.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.792+0000 7efe46388700 1 -- 172.21.15.92:0/543751259 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe400fc730 con 0x7efe40066b10 2024-05-06T12:13:17.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.793+0000 7efe3cff9700 1 -- 172.21.15.92:0/543751259 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe3001ce50 con 0x7efe40066b10 2024-05-06T12:13:17.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.793+0000 7efe46388700 1 -- 172.21.15.92:0/543751259 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe4005f5c0 con 0x7efe40066b10 2024-05-06T12:13:17.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.794+0000 7efe3cff9700 1 -- 172.21.15.92:0/543751259 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7efe3000b480 con 0x7efe40066b10 2024-05-06T12:13:17.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.794+0000 7efe3cff9700 1 --2- 172.21.15.92:0/543751259 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7efe280690a0 0x7efe2806b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:17.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.794+0000 7efe3cff9700 1 -- 172.21.15.92:0/543751259 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7efe30088ac0 con 0x7efe40066b10 2024-05-06T12:13:17.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.794+0000 7efe3f7fe700 1 --2- 172.21.15.92:0/543751259 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7efe280690a0 0x7efe2806b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:17.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.795+0000 7efe3f7fe700 1 --2- 172.21.15.92:0/543751259 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7efe280690a0 0x7efe2806b550 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efe34009b80 tx=0x7efe3400b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:13:17.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:17.803+0000 7efe3cff9700 1 -- 172.21.15.92:0/543751259 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7efe30054e10 con 0x7efe40066b10 2024-05-06T12:13:17.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:17 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/3021229832' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:13:18.164 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:18.161+0000 7efe46388700 1 -- 172.21.15.92:0/543751259 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7efe40115b30 con 0x7efe40066b10 2024-05-06T12:13:18.165 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:18.162+0000 7efe3cff9700 1 -- 172.21.15.92:0/543751259 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7efe30058430 con 0x7efe40066b10 2024-05-06T12:13:18.165 INFO:teuthology.orchestra.run.smithi092.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-06T12:13:18.165 INFO:teuthology.orchestra.run.smithi092.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-06T12:13:18.165 INFO:teuthology.orchestra.run.smithi092.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-06T12:13:18.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:18.164+0000 7efe46388700 1 -- 172.21.15.92:0/543751259 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7efe280690a0 msgr2=0x7efe2806b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:18.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:18.164+0000 7efe46388700 1 --2- 172.21.15.92:0/543751259 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7efe280690a0 0x7efe2806b550 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efe34009b80 tx=0x7efe3400b040 comp rx=0 tx=0).stop 2024-05-06T12:13:18.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:18.164+0000 7efe46388700 1 -- 172.21.15.92:0/543751259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe40066b10 msgr2=0x7efe400fd380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:18.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:18.164+0000 7efe46388700 1 --2- 172.21.15.92:0/543751259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe40066b10 0x7efe400fd380 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efe30000c00 tx=0x7efe30009e30 comp rx=0 tx=0).stop 2024-05-06T12:13:18.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:18.164+0000 7efe46388700 1 -- 172.21.15.92:0/543751259 shutdown_connections 2024-05-06T12:13:18.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:18.164+0000 7efe46388700 1 --2- 172.21.15.92:0/543751259 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7efe280690a0 0x7efe2806b550 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:18.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:18.164+0000 7efe46388700 1 --2- 172.21.15.92:0/543751259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe40066b10 0x7efe400fd380 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:18.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:18.164+0000 7efe46388700 1 --2- 172.21.15.92:0/543751259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe400fff30 0x7efe400fb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:18.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:18.164+0000 7efe46388700 1 -- 172.21.15.92:0/543751259 >> 172.21.15.92:0/543751259 conn(0x7efe400f7420 msgr2=0x7efe400f7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:13:18.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:18.164+0000 7efe46388700 1 -- 172.21.15.92:0/543751259 shutdown_connections 2024-05-06T12:13:18.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:18.164+0000 7efe46388700 1 -- 172.21.15.92:0/543751259 wait complete. 2024-05-06T12:13:18.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:17 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/3021229832' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:13:19.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:18 smithi190 ceph-mon[33798]: from='client.14600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:13:19.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:18 smithi190 ceph-mon[33798]: pgmap v221: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:13:19.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:18 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/543751259' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:13:19.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:18 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:13:19.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:18 smithi092 ceph-mon[26954]: from='client.14600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:13:19.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:18 smithi092 ceph-mon[26954]: pgmap v221: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:13:19.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:18 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/543751259' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:13:19.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:18 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:13:21.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:20 smithi092 ceph-mon[26954]: pgmap v222: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:13:21.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:20 smithi190 ceph-mon[33798]: pgmap v222: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:13:23.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:22 smithi092 ceph-mon[26954]: pgmap v223: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:13:23.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:22 smithi190 ceph-mon[33798]: pgmap v223: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:13:25.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:24 smithi092 ceph-mon[26954]: pgmap v224: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:13:25.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:24 smithi190 ceph-mon[33798]: pgmap v224: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:13:27.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:26 smithi092 ceph-mon[26954]: pgmap v225: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:13:27.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:26 smithi190 ceph-mon[33798]: pgmap v225: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:13:29.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:28 smithi190 ceph-mon[33798]: pgmap v226: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:13:29.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:28 smithi092 ceph-mon[26954]: pgmap v226: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:13:31.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:30 smithi092 ceph-mon[26954]: pgmap v227: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:13:31.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:30 smithi190 ceph-mon[33798]: pgmap v227: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:13:33.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:32 smithi092 ceph-mon[26954]: pgmap v228: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:13:33.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:32 smithi190 ceph-mon[33798]: pgmap v228: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:13:34.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:33 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:13:34.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:33 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:13:35.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:34 smithi092 ceph-mon[26954]: pgmap v229: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:13:35.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:34 smithi190 ceph-mon[33798]: pgmap v229: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:13:37.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:36 smithi092 ceph-mon[26954]: pgmap v230: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:13:37.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:36 smithi190 ceph-mon[33798]: pgmap v230: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:13:39.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:38 smithi190 ceph-mon[33798]: pgmap v231: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:13:39.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:38 smithi092 ceph-mon[26954]: pgmap v231: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:13:39.227 INFO:tasks.workunit.client.0.smithi092.stderr:Note: switching to '70ce9539b62a15b9de429c1f0e91a1e012b099aa'. 2024-05-06T12:13:39.227 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-05-06T12:13:39.227 INFO:tasks.workunit.client.0.smithi092.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-06T12:13:39.227 INFO:tasks.workunit.client.0.smithi092.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-06T12:13:39.227 INFO:tasks.workunit.client.0.smithi092.stderr:state without impacting any branches by switching back to a branch. 2024-05-06T12:13:39.227 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-05-06T12:13:39.227 INFO:tasks.workunit.client.0.smithi092.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-06T12:13:39.227 INFO:tasks.workunit.client.0.smithi092.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-06T12:13:39.227 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-05-06T12:13:39.228 INFO:tasks.workunit.client.0.smithi092.stderr: git switch -c 2024-05-06T12:13:39.228 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-05-06T12:13:39.228 INFO:tasks.workunit.client.0.smithi092.stderr:Or undo this operation with: 2024-05-06T12:13:39.228 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-05-06T12:13:39.228 INFO:tasks.workunit.client.0.smithi092.stderr: git switch - 2024-05-06T12:13:39.228 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-05-06T12:13:39.228 INFO:tasks.workunit.client.0.smithi092.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-06T12:13:39.228 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-05-06T12:13:39.228 INFO:tasks.workunit.client.0.smithi092.stderr:HEAD is now at 70ce9539b62 Merge pull request #56964 from adk3798/squid-dashboard-anon-grafana 2024-05-06T12:13:39.232 DEBUG:teuthology.orchestra.run.smithi092:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-06T12:13:39.255 INFO:tasks.workunit.client.0.smithi092.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-06T12:13:39.257 INFO:tasks.workunit.client.0.smithi092.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-06T12:13:39.257 INFO:tasks.workunit.client.0.smithi092.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-06T12:13:39.891 INFO:tasks.workunit.client.0.smithi092.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-06T12:13:39.943 INFO:tasks.workunit.client.0.smithi092.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-06T12:13:39.982 INFO:tasks.workunit.client.0.smithi092.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-06T12:13:39.984 INFO:tasks.workunit.client.0.smithi092.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-06T12:13:39.984 INFO:tasks.workunit.client.0.smithi092.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-06T12:13:40.022 INFO:tasks.workunit.client.0.smithi092.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-06T12:13:40.025 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-06T12:13:40.025 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-06T12:13:40.081 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-05-06T12:13:40.083 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-05-06T12:13:40.083 DEBUG:teuthology.orchestra.run.smithi092:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=70ce9539b62a15b9de429c1f0e91a1e012b099aa TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-05-06T12:13:40.145 INFO:tasks.workunit.client.0.smithi092.stderr:+ mkdir -p fsstress 2024-05-06T12:13:40.147 INFO:tasks.workunit.client.0.smithi092.stderr:+ pushd fsstress 2024-05-06T12:13:40.148 INFO:tasks.workunit.client.0.smithi092.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-05-06T12:13:40.148 INFO:tasks.workunit.client.0.smithi092.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-05-06T12:13:41.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:40 smithi092 ceph-mon[26954]: pgmap v232: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:13:41.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:40 smithi190 ceph-mon[33798]: pgmap v232: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:13:41.365 INFO:tasks.workunit.client.0.smithi092.stderr:+ tar xzf ltp-full.tgz 2024-05-06T12:13:43.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:42 smithi092 ceph-mon[26954]: pgmap v233: 65 pgs: 65 active+clean; 598 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 853 B/s wr, 4 op/s 2024-05-06T12:13:43.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:42 smithi190 ceph-mon[33798]: pgmap v233: 65 pgs: 65 active+clean; 598 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 853 B/s wr, 4 op/s 2024-05-06T12:13:45.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:44 smithi092 ceph-mon[26954]: pgmap v234: 65 pgs: 65 active+clean; 9.5 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 764 KiB/s wr, 18 op/s 2024-05-06T12:13:45.229 INFO:tasks.workunit.client.1.smithi190.stderr:Updating files: 94% (11878/12534) Updating files: 95% (11908/12534) Updating files: 96% (12033/12534) Updating files: 97% (12158/12534) Updating files: 98% (12284/12534) Updating files: 99% (12409/12534) Updating files: 100% (12534/12534) Updating files: 100% (12534/12534), done. 2024-05-06T12:13:45.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:44 smithi190 ceph-mon[33798]: pgmap v234: 65 pgs: 65 active+clean; 9.5 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 764 KiB/s wr, 18 op/s 2024-05-06T12:13:45.740 INFO:tasks.workunit.client.1.smithi190.stderr:Note: switching to '70ce9539b62a15b9de429c1f0e91a1e012b099aa'. 2024-05-06T12:13:45.740 INFO:tasks.workunit.client.1.smithi190.stderr: 2024-05-06T12:13:45.740 INFO:tasks.workunit.client.1.smithi190.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-06T12:13:45.740 INFO:tasks.workunit.client.1.smithi190.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-06T12:13:45.740 INFO:tasks.workunit.client.1.smithi190.stderr:state without impacting any branches by switching back to a branch. 2024-05-06T12:13:45.740 INFO:tasks.workunit.client.1.smithi190.stderr: 2024-05-06T12:13:45.740 INFO:tasks.workunit.client.1.smithi190.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-06T12:13:45.740 INFO:tasks.workunit.client.1.smithi190.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-06T12:13:45.740 INFO:tasks.workunit.client.1.smithi190.stderr: 2024-05-06T12:13:45.741 INFO:tasks.workunit.client.1.smithi190.stderr: git switch -c 2024-05-06T12:13:45.741 INFO:tasks.workunit.client.1.smithi190.stderr: 2024-05-06T12:13:45.741 INFO:tasks.workunit.client.1.smithi190.stderr:Or undo this operation with: 2024-05-06T12:13:45.741 INFO:tasks.workunit.client.1.smithi190.stderr: 2024-05-06T12:13:45.741 INFO:tasks.workunit.client.1.smithi190.stderr: git switch - 2024-05-06T12:13:45.741 INFO:tasks.workunit.client.1.smithi190.stderr: 2024-05-06T12:13:45.741 INFO:tasks.workunit.client.1.smithi190.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-06T12:13:45.741 INFO:tasks.workunit.client.1.smithi190.stderr: 2024-05-06T12:13:45.741 INFO:tasks.workunit.client.1.smithi190.stderr:HEAD is now at 70ce9539b62 Merge pull request #56964 from adk3798/squid-dashboard-anon-grafana 2024-05-06T12:13:45.744 DEBUG:teuthology.orchestra.run.smithi190:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-05-06T12:13:45.915 INFO:tasks.workunit.client.1.smithi190.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-06T12:13:45.917 INFO:tasks.workunit.client.1.smithi190.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-05-06T12:13:45.917 INFO:tasks.workunit.client.1.smithi190.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-06T12:13:47.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:46 smithi092 ceph-mon[26954]: pgmap v235: 65 pgs: 65 active+clean; 10 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 828 KiB/s wr, 28 op/s 2024-05-06T12:13:47.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:46 smithi190 ceph-mon[33798]: pgmap v235: 65 pgs: 65 active+clean; 10 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 828 KiB/s wr, 28 op/s 2024-05-06T12:13:48.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.279+0000 7f86addfc700 1 -- 172.21.15.92:0/1300367064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86a80fe080 msgr2=0x7f86a80fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:48.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.279+0000 7f86addfc700 1 --2- 172.21.15.92:0/1300367064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86a80fe080 0x7f86a80fe4f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f869c004940 tx=0x7f869c009e30 comp rx=0 tx=0).stop 2024-05-06T12:13:48.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.280+0000 7f86addfc700 1 -- 172.21.15.92:0/1300367064 shutdown_connections 2024-05-06T12:13:48.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.280+0000 7f86addfc700 1 --2- 172.21.15.92:0/1300367064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86a80fe080 0x7f86a80fe4f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:48.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.280+0000 7f86addfc700 1 --2- 172.21.15.92:0/1300367064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86a80fce80 0x7f86a80fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:48.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.280+0000 7f86addfc700 1 -- 172.21.15.92:0/1300367064 >> 172.21.15.92:0/1300367064 conn(0x7f86a80f8590 msgr2=0x7f86a80fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:13:48.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.280+0000 7f86addfc700 1 -- 172.21.15.92:0/1300367064 shutdown_connections 2024-05-06T12:13:48.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.280+0000 7f86addfc700 1 -- 172.21.15.92:0/1300367064 wait complete. 2024-05-06T12:13:48.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.280+0000 7f86addfc700 1 Processor -- start 2024-05-06T12:13:48.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.280+0000 7f86addfc700 1 -- start start 2024-05-06T12:13:48.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.280+0000 7f86addfc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86a80fce80 0x7f86a8071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:48.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.280+0000 7f86addfc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86a80fe080 0x7f86a80702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:48.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.280+0000 7f86addfc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a8070860 con 0x7f86a80fce80 2024-05-06T12:13:48.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.280+0000 7f86addfc700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a80709d0 con 0x7f86a80fe080 2024-05-06T12:13:48.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.281+0000 7f86a77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86a80fce80 0x7f86a8071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:48.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.281+0000 7f86a77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86a80fce80 0x7f86a8071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43158/0 (socket says 172.21.15.92:43158) 2024-05-06T12:13:48.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.281+0000 7f86a77fe700 1 -- 172.21.15.92:0/1690187499 learned_addr learned my addr 172.21.15.92:0/1690187499 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:13:48.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.281+0000 7f86a6ffd700 1 --2- 172.21.15.92:0/1690187499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86a80fe080 0x7f86a80702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:48.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.281+0000 7f86a77fe700 1 -- 172.21.15.92:0/1690187499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86a80fe080 msgr2=0x7f86a80702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:48.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.281+0000 7f86a77fe700 1 --2- 172.21.15.92:0/1690187499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86a80fe080 0x7f86a80702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:48.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.281+0000 7f86a77fe700 1 -- 172.21.15.92:0/1690187499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869c005040 con 0x7f86a80fce80 2024-05-06T12:13:48.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.281+0000 7f86a6ffd700 1 --2- 172.21.15.92:0/1690187499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86a80fe080 0x7f86a80702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-06T12:13:48.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.281+0000 7f86a77fe700 1 --2- 172.21.15.92:0/1690187499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86a80fce80 0x7f86a8071de0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f869800eb70 tx=0x7f869800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:13:48.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.281+0000 7f868ffff700 1 -- 172.21.15.92:0/1690187499 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8698007cc0 con 0x7f86a80fce80 2024-05-06T12:13:48.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.281+0000 7f86addfc700 1 -- 172.21.15.92:0/1690187499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86a8070c50 con 0x7f86a80fce80 2024-05-06T12:13:48.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.281+0000 7f868ffff700 1 -- 172.21.15.92:0/1690187499 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f869800f040 con 0x7f86a80fce80 2024-05-06T12:13:48.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.281+0000 7f86addfc700 1 -- 172.21.15.92:0/1690187499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a81159d0 con 0x7f86a80fce80 2024-05-06T12:13:48.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.281+0000 7f868ffff700 1 -- 172.21.15.92:0/1690187499 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f869800cb20 con 0x7f86a80fce80 2024-05-06T12:13:48.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.282+0000 7f86addfc700 1 -- 172.21.15.92:0/1690187499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86a805f5c0 con 0x7f86a80fce80 2024-05-06T12:13:48.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.283+0000 7f868ffff700 1 -- 172.21.15.92:0/1690187499 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f869800cc80 con 0x7f86a80fce80 2024-05-06T12:13:48.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.283+0000 7f868ffff700 1 --2- 172.21.15.92:0/1690187499 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f86900692b0 0x7f869006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:48.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.284+0000 7f868ffff700 1 -- 172.21.15.92:0/1690187499 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f8698086cf0 con 0x7f86a80fce80 2024-05-06T12:13:48.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.284+0000 7f86a6ffd700 1 --2- 172.21.15.92:0/1690187499 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f86900692b0 0x7f869006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:48.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.284+0000 7f86a6ffd700 1 --2- 172.21.15.92:0/1690187499 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f86900692b0 0x7f869006b760 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f869c00b0d0 tx=0x7f869c009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:13:48.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.291+0000 7f868ffff700 1 -- 172.21.15.92:0/1690187499 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f869804e250 con 0x7f86a80fce80 2024-05-06T12:13:48.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.556+0000 7f86addfc700 1 -- 172.21.15.92:0/1690187499 --> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f86a8102360 con 0x7f86900692b0 2024-05-06T12:13:48.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.557+0000 7f868ffff700 1 -- 172.21.15.92:0/1690187499 <== mgr.14210 v2:172.21.15.92:6800/3045905943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f86a8102360 con 0x7f86900692b0 2024-05-06T12:13:48.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.560+0000 7f86addfc700 1 -- 172.21.15.92:0/1690187499 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f86900692b0 msgr2=0x7f869006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:48.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.560+0000 7f86addfc700 1 --2- 172.21.15.92:0/1690187499 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f86900692b0 0x7f869006b760 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f869c00b0d0 tx=0x7f869c009e30 comp rx=0 tx=0).stop 2024-05-06T12:13:48.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.560+0000 7f86addfc700 1 -- 172.21.15.92:0/1690187499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86a80fce80 msgr2=0x7f86a8071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:48.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.560+0000 7f86addfc700 1 --2- 172.21.15.92:0/1690187499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86a80fce80 0x7f86a8071de0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f869800eb70 tx=0x7f869800b040 comp rx=0 tx=0).stop 2024-05-06T12:13:48.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.560+0000 7f86addfc700 1 -- 172.21.15.92:0/1690187499 shutdown_connections 2024-05-06T12:13:48.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.560+0000 7f86addfc700 1 --2- 172.21.15.92:0/1690187499 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f86900692b0 0x7f869006b760 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:48.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.560+0000 7f86addfc700 1 --2- 172.21.15.92:0/1690187499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86a80fce80 0x7f86a8071de0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:48.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.560+0000 7f86addfc700 1 --2- 172.21.15.92:0/1690187499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86a80fe080 0x7f86a80702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:48.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.560+0000 7f86addfc700 1 -- 172.21.15.92:0/1690187499 >> 172.21.15.92:0/1690187499 conn(0x7f86a80f8590 msgr2=0x7f86a80f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:13:48.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.560+0000 7f86addfc700 1 -- 172.21.15.92:0/1690187499 shutdown_connections 2024-05-06T12:13:48.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.560+0000 7f86addfc700 1 -- 172.21.15.92:0/1690187499 wait complete. 2024-05-06T12:13:48.574 INFO:teuthology.orchestra.run.smithi092.stdout:true 2024-05-06T12:13:48.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.690+0000 7feed93d2700 1 -- 172.21.15.92:0/2362257710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feed40fcf10 msgr2=0x7feed40fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:48.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.690+0000 7feed93d2700 1 --2- 172.21.15.92:0/2362257710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feed40fcf10 0x7feed40fd320 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7feec4004940 tx=0x7feec4009e30 comp rx=0 tx=0).stop 2024-05-06T12:13:48.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.691+0000 7feed93d2700 1 -- 172.21.15.92:0/2362257710 shutdown_connections 2024-05-06T12:13:48.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.691+0000 7feed93d2700 1 --2- 172.21.15.92:0/2362257710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feed40fe080 0x7feed40fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:48.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.691+0000 7feed93d2700 1 --2- 172.21.15.92:0/2362257710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feed40fcf10 0x7feed40fd320 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:48.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.691+0000 7feed93d2700 1 -- 172.21.15.92:0/2362257710 >> 172.21.15.92:0/2362257710 conn(0x7feed40f8560 msgr2=0x7feed40fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:13:48.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.691+0000 7feed93d2700 1 -- 172.21.15.92:0/2362257710 shutdown_connections 2024-05-06T12:13:48.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.691+0000 7feed93d2700 1 -- 172.21.15.92:0/2362257710 wait complete. 2024-05-06T12:13:48.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.691+0000 7feed93d2700 1 Processor -- start 2024-05-06T12:13:48.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.691+0000 7feed93d2700 1 -- start start 2024-05-06T12:13:48.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.692+0000 7feed93d2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feed40fcf10 0x7feed410d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:48.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.692+0000 7feed93d2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feed40fe080 0x7feed410df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:48.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.692+0000 7feed93d2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed410e520 con 0x7feed40fe080 2024-05-06T12:13:48.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.692+0000 7feed93d2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed410ed70 con 0x7feed40fcf10 2024-05-06T12:13:48.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.692+0000 7feed27fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feed40fe080 0x7feed410df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:48.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.692+0000 7feed27fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feed40fe080 0x7feed410df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43188/0 (socket says 172.21.15.92:43188) 2024-05-06T12:13:48.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.692+0000 7feed27fc700 1 -- 172.21.15.92:0/3877143000 learned_addr learned my addr 172.21.15.92:0/3877143000 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:13:48.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.692+0000 7feed2ffd700 1 --2- 172.21.15.92:0/3877143000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feed40fcf10 0x7feed410d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:48.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.692+0000 7feed2ffd700 1 -- 172.21.15.92:0/3877143000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feed40fe080 msgr2=0x7feed410df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:48.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.692+0000 7feed2ffd700 1 --2- 172.21.15.92:0/3877143000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feed40fe080 0x7feed410df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:48.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.692+0000 7feed2ffd700 1 -- 172.21.15.92:0/3877143000 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec4005040 con 0x7feed40fcf10 2024-05-06T12:13:48.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.692+0000 7feed27fc700 1 --2- 172.21.15.92:0/3877143000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feed40fe080 0x7feed410df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-06T12:13:48.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.693+0000 7feed2ffd700 1 --2- 172.21.15.92:0/3877143000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feed40fcf10 0x7feed410d9e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7feec400b3b0 tx=0x7feec4009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:13:48.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.693+0000 7feebb7fe700 1 -- 172.21.15.92:0/3877143000 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feec400e040 con 0x7feed40fcf10 2024-05-06T12:13:48.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.693+0000 7feed93d2700 1 -- 172.21.15.92:0/3877143000 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed410eff0 con 0x7feed40fcf10 2024-05-06T12:13:48.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.693+0000 7feebb7fe700 1 -- 172.21.15.92:0/3877143000 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feec400bbf0 con 0x7feed40fcf10 2024-05-06T12:13:48.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.693+0000 7feebb7fe700 1 -- 172.21.15.92:0/3877143000 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feec4013660 con 0x7feed40fcf10 2024-05-06T12:13:48.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.693+0000 7feed93d2700 1 -- 172.21.15.92:0/3877143000 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed410f470 con 0x7feed40fcf10 2024-05-06T12:13:48.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.694+0000 7feed93d2700 1 -- 172.21.15.92:0/3877143000 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed405f5c0 con 0x7feed40fcf10 2024-05-06T12:13:48.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.695+0000 7feebb7fe700 1 -- 172.21.15.92:0/3877143000 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7feec401d440 con 0x7feed40fcf10 2024-05-06T12:13:48.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.695+0000 7feebb7fe700 1 --2- 172.21.15.92:0/3877143000 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7feebc0692a0 0x7feebc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:48.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.695+0000 7feebb7fe700 1 -- 172.21.15.92:0/3877143000 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7feec40885b0 con 0x7feed40fcf10 2024-05-06T12:13:48.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.695+0000 7feed27fc700 1 --2- 172.21.15.92:0/3877143000 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7feebc0692a0 0x7feebc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:48.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.696+0000 7feed27fc700 1 --2- 172.21.15.92:0/3877143000 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7feebc0692a0 0x7feebc06b750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7feec80041d0 tx=0x7feec800b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:13:48.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.700+0000 7feebb7fe700 1 -- 172.21.15.92:0/3877143000 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7feec4058340 con 0x7feed40fcf10 2024-05-06T12:13:48.805 INFO:tasks.workunit.client.1.smithi190.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-06T12:13:48.904 INFO:tasks.workunit.client.1.smithi190.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-06T12:13:48.946 INFO:tasks.workunit.client.1.smithi190.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-05-06T12:13:48.947 INFO:tasks.workunit.client.1.smithi190.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-05-06T12:13:48.947 INFO:tasks.workunit.client.1.smithi190.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-06T12:13:48.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.947+0000 7feed93d2700 1 -- 172.21.15.92:0/3877143000 --> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feed410f8b0 con 0x7feebc0692a0 2024-05-06T12:13:48.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.949+0000 7feebb7fe700 1 -- 172.21.15.92:0/3877143000 <== mgr.14210 v2:172.21.15.92:6800/3045905943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7feed410f8b0 con 0x7feebc0692a0 2024-05-06T12:13:48.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.951+0000 7feed93d2700 1 -- 172.21.15.92:0/3877143000 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7feebc0692a0 msgr2=0x7feebc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:48.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.951+0000 7feed93d2700 1 --2- 172.21.15.92:0/3877143000 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7feebc0692a0 0x7feebc06b750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7feec80041d0 tx=0x7feec800b040 comp rx=0 tx=0).stop 2024-05-06T12:13:48.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.951+0000 7feed93d2700 1 -- 172.21.15.92:0/3877143000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feed40fcf10 msgr2=0x7feed410d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:48.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.951+0000 7feed93d2700 1 --2- 172.21.15.92:0/3877143000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feed40fcf10 0x7feed410d9e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7feec400b3b0 tx=0x7feec4009e30 comp rx=0 tx=0).stop 2024-05-06T12:13:48.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.951+0000 7feed93d2700 1 -- 172.21.15.92:0/3877143000 shutdown_connections 2024-05-06T12:13:48.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.951+0000 7feed93d2700 1 --2- 172.21.15.92:0/3877143000 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7feebc0692a0 0x7feebc06b750 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:48.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.951+0000 7feed93d2700 1 --2- 172.21.15.92:0/3877143000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feed40fcf10 0x7feed410d9e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:48.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.951+0000 7feed93d2700 1 --2- 172.21.15.92:0/3877143000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feed40fe080 0x7feed410df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:48.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.951+0000 7feed93d2700 1 -- 172.21.15.92:0/3877143000 >> 172.21.15.92:0/3877143000 conn(0x7feed40f8560 msgr2=0x7feed40f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:13:48.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.951+0000 7feed93d2700 1 -- 172.21.15.92:0/3877143000 shutdown_connections 2024-05-06T12:13:48.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:48.951+0000 7feed93d2700 1 -- 172.21.15.92:0/3877143000 wait complete. 2024-05-06T12:13:48.986 INFO:tasks.workunit.client.1.smithi190.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-05-06T12:13:48.990 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-06T12:13:48.990 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-05-06T12:13:49.008 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-05-06T12:13:49.010 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-05-06T12:13:49.010 DEBUG:teuthology.orchestra.run.smithi190:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=70ce9539b62a15b9de429c1f0e91a1e012b099aa TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-05-06T12:13:49.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.064+0000 7f9f63464700 1 -- 172.21.15.92:0/2144966445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f5c0fbcb0 msgr2=0x7f9f5c0fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:49.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.064+0000 7f9f63464700 1 --2- 172.21.15.92:0/2144966445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f5c0fbcb0 0x7f9f5c0fc0c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9f58004940 tx=0x7f9f58009e30 comp rx=0 tx=0).stop 2024-05-06T12:13:49.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.064+0000 7f9f63464700 1 -- 172.21.15.92:0/2144966445 shutdown_connections 2024-05-06T12:13:49.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.064+0000 7f9f63464700 1 --2- 172.21.15.92:0/2144966445 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f5c0fcf60 0x7f9f5c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:49.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.064+0000 7f9f63464700 1 --2- 172.21.15.92:0/2144966445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f5c0fbcb0 0x7f9f5c0fc0c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:49.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.064+0000 7f9f63464700 1 -- 172.21.15.92:0/2144966445 >> 172.21.15.92:0/2144966445 conn(0x7f9f5c0f7430 msgr2=0x7f9f5c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:13:49.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.065+0000 7f9f63464700 1 -- 172.21.15.92:0/2144966445 shutdown_connections 2024-05-06T12:13:49.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.065+0000 7f9f63464700 1 -- 172.21.15.92:0/2144966445 wait complete. 2024-05-06T12:13:49.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.065+0000 7f9f63464700 1 Processor -- start 2024-05-06T12:13:49.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.065+0000 7f9f63464700 1 -- start start 2024-05-06T12:13:49.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.065+0000 7f9f63464700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f5c0fbcb0 0x7f9f5c10fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:49.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.065+0000 7f9f63464700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f5c0fcf60 0x7f9f5c110130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:49.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.065+0000 7f9f63464700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c110730 con 0x7f9f5c0fcf60 2024-05-06T12:13:49.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.065+0000 7f9f63464700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c110f30 con 0x7f9f5c0fbcb0 2024-05-06T12:13:49.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.066+0000 7f9f609ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f5c0fcf60 0x7f9f5c110130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:49.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.066+0000 7f9f609ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f5c0fcf60 0x7f9f5c110130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43214/0 (socket says 172.21.15.92:43214) 2024-05-06T12:13:49.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.066+0000 7f9f609ff700 1 -- 172.21.15.92:0/415077467 learned_addr learned my addr 172.21.15.92:0/415077467 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:13:49.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.066+0000 7f9f61200700 1 --2- 172.21.15.92:0/415077467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f5c0fbcb0 0x7f9f5c10fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:49.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.066+0000 7f9f609ff700 1 -- 172.21.15.92:0/415077467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f5c0fbcb0 msgr2=0x7f9f5c10fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:49.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.066+0000 7f9f609ff700 1 --2- 172.21.15.92:0/415077467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f5c0fbcb0 0x7f9f5c10fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:49.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.066+0000 7f9f609ff700 1 -- 172.21.15.92:0/415077467 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f58005040 con 0x7f9f5c0fcf60 2024-05-06T12:13:49.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.066+0000 7f9f609ff700 1 --2- 172.21.15.92:0/415077467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f5c0fcf60 0x7f9f5c110130 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00eb40 tx=0x7f9f4c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:13:49.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.066+0000 7f9f51ffb700 1 -- 172.21.15.92:0/415077467 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f4c007cc0 con 0x7f9f5c0fcf60 2024-05-06T12:13:49.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.066+0000 7f9f63464700 1 -- 172.21.15.92:0/415077467 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f5c1111b0 con 0x7f9f5c0fcf60 2024-05-06T12:13:49.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.066+0000 7f9f51ffb700 1 -- 172.21.15.92:0/415077467 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f4c00f040 con 0x7f9f5c0fcf60 2024-05-06T12:13:49.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.066+0000 7f9f51ffb700 1 -- 172.21.15.92:0/415077467 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f4c00ca10 con 0x7f9f5c0fcf60 2024-05-06T12:13:49.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.067+0000 7f9f63464700 1 -- 172.21.15.92:0/415077467 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f5c1116f0 con 0x7f9f5c0fcf60 2024-05-06T12:13:49.072 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.068+0000 7f9f63464700 1 -- 172.21.15.92:0/415077467 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f5c1018f0 con 0x7f9f5c0fcf60 2024-05-06T12:13:49.072 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.068+0000 7f9f51ffb700 1 -- 172.21.15.92:0/415077467 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f9f4c00cc80 con 0x7f9f5c0fcf60 2024-05-06T12:13:49.072 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.068+0000 7f9f51ffb700 1 --2- 172.21.15.92:0/415077467 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f9f480692b0 0x7f9f4806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:49.072 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.069+0000 7f9f51ffb700 1 -- 172.21.15.92:0/415077467 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f9f4c086de0 con 0x7f9f5c0fcf60 2024-05-06T12:13:49.072 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.069+0000 7f9f61200700 1 --2- 172.21.15.92:0/415077467 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f9f480692b0 0x7f9f4806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:49.072 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.069+0000 7f9f61200700 1 --2- 172.21.15.92:0/415077467 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f9f480692b0 0x7f9f4806b760 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9f5800b0d0 tx=0x7f9f58009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:13:49.073 INFO:tasks.workunit.client.1.smithi190.stderr:+ mkdir -p fsstress 2024-05-06T12:13:49.074 INFO:tasks.workunit.client.1.smithi190.stderr:+ pushd fsstress 2024-05-06T12:13:49.076 INFO:tasks.workunit.client.1.smithi190.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-05-06T12:13:49.076 INFO:tasks.workunit.client.1.smithi190.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-05-06T12:13:49.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.074+0000 7f9f51ffb700 1 -- 172.21.15.92:0/415077467 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9f4c04e250 con 0x7f9f5c0fcf60 2024-05-06T12:13:49.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:48 smithi190 ceph-mon[33798]: pgmap v236: 65 pgs: 65 active+clean; 18 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 250 KiB/s rd, 1.5 MiB/s wr, 64 op/s 2024-05-06T12:13:49.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:48 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:13:49.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:48 smithi092 ceph-mon[26954]: pgmap v236: 65 pgs: 65 active+clean; 18 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 250 KiB/s rd, 1.5 MiB/s wr, 64 op/s 2024-05-06T12:13:49.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:48 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:13:49.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.311+0000 7f9f63464700 1 -- 172.21.15.92:0/415077467 --> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f5c104910 con 0x7f9f480692b0 2024-05-06T12:13:49.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.325+0000 7f9f51ffb700 1 -- 172.21.15.92:0/415077467 <== mgr.14210 v2:172.21.15.92:6800/3045905943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7f9f5c104910 con 0x7f9f480692b0 2024-05-06T12:13:49.328 INFO:teuthology.orchestra.run.smithi092.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-06T12:13:49.329 INFO:teuthology.orchestra.run.smithi092.stdout:alertmanager.smithi092 smithi092 *:9093,9094 running (6m) 93s ago 7m 25.7M - 0.25.0 c8568f914cd2 dab380a7352e 2024-05-06T12:13:49.329 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi092 smithi092 running (7m) 93s ago 7m 6660k - 17.2.7-925-ge81df420 96d5ae7fd4b4 95dc7e14bd9a 2024-05-06T12:13:49.329 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi190 smithi190 running (6m) 99s ago 6m 6665k - 17.2.7-925-ge81df420 96d5ae7fd4b4 6cf97785406f 2024-05-06T12:13:49.329 INFO:teuthology.orchestra.run.smithi092.stdout:grafana.smithi092 smithi092 *:3000 running (5m) 93s ago 7m 84.6M - 9.4.7 954c08fa6188 841f34687c12 2024-05-06T12:13:49.329 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.ijsqgp smithi092 running (2m) 93s ago 2m 11.4M - 17.2.7-925-ge81df420 96d5ae7fd4b4 b4c2ed6d2c82 2024-05-06T12:13:49.329 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.rosbll smithi092 running (113s) 93s ago 112s 13.7M - 17.2.7-925-ge81df420 96d5ae7fd4b4 5f5426cd1bf5 2024-05-06T12:13:49.329 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.byjvgf smithi190 running (118s) 99s ago 117s 16.7M - 17.2.7-925-ge81df420 96d5ae7fd4b4 24581d2e1c5d 2024-05-06T12:13:49.329 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.idzoff smithi190 running (110s) 99s ago 110s 13.3M - 17.2.7-925-ge81df420 96d5ae7fd4b4 20f2641696ef 2024-05-06T12:13:49.329 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi092.gyzeqb smithi092 *:9283 running (8m) 93s ago 8m 480M - 17.2.7-925-ge81df420 96d5ae7fd4b4 820ba9ba8a82 2024-05-06T12:13:49.329 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi190.kczxkh smithi190 *:8443,9283 running (6m) 99s ago 6m 431M - 17.2.7-925-ge81df420 96d5ae7fd4b4 fb57adc25b2b 2024-05-06T12:13:49.329 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi092 smithi092 running (8m) 93s ago 8m 42.8M 2048M 17.2.7-925-ge81df420 96d5ae7fd4b4 df5b322e9719 2024-05-06T12:13:49.329 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi190 smithi190 running (6m) 99s ago 6m 32.5M 2048M 17.2.7-925-ge81df420 96d5ae7fd4b4 912ded072601 2024-05-06T12:13:49.329 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi092 smithi092 *:9100 running (7m) 93s ago 7m 15.8M - 1.5.0 0da6a335fe13 86cf73a245ab 2024-05-06T12:13:49.329 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi190 smithi190 *:9100 running (6m) 99s ago 6m 16.0M - 1.5.0 0da6a335fe13 da5c2a030c69 2024-05-06T12:13:49.330 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 smithi092 running (5m) 93s ago 5m 47.0M 4288M 17.2.7-925-ge81df420 96d5ae7fd4b4 ce84e7af9084 2024-05-06T12:13:49.330 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 smithi092 running (5m) 93s ago 5m 48.8M 4288M 17.2.7-925-ge81df420 96d5ae7fd4b4 8ab870f57eb2 2024-05-06T12:13:49.330 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 smithi092 running (4m) 93s ago 4m 46.0M 4288M 17.2.7-925-ge81df420 96d5ae7fd4b4 1e6ed04a0688 2024-05-06T12:13:49.330 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 smithi190 running (4m) 99s ago 4m 43.8M 5312M 17.2.7-925-ge81df420 96d5ae7fd4b4 52f9521286d5 2024-05-06T12:13:49.330 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 smithi190 running (3m) 99s ago 3m 42.4M 5312M 17.2.7-925-ge81df420 96d5ae7fd4b4 3a7da09e4c48 2024-05-06T12:13:49.330 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 smithi190 running (3m) 99s ago 3m 41.7M 5312M 17.2.7-925-ge81df420 96d5ae7fd4b4 e7452c6b5cec 2024-05-06T12:13:49.330 INFO:teuthology.orchestra.run.smithi092.stdout:prometheus.smithi092 smithi092 *:9095 running (5m) 93s ago 6m 56.7M - 2.43.0 a07b618ecd1d ef92f3572328 2024-05-06T12:13:49.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.327+0000 7f9f63464700 1 -- 172.21.15.92:0/415077467 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f9f480692b0 msgr2=0x7f9f4806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:49.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.327+0000 7f9f63464700 1 --2- 172.21.15.92:0/415077467 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f9f480692b0 0x7f9f4806b760 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9f5800b0d0 tx=0x7f9f58009e30 comp rx=0 tx=0).stop 2024-05-06T12:13:49.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.327+0000 7f9f63464700 1 -- 172.21.15.92:0/415077467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f5c0fcf60 msgr2=0x7f9f5c110130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:49.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.327+0000 7f9f63464700 1 --2- 172.21.15.92:0/415077467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f5c0fcf60 0x7f9f5c110130 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00eb40 tx=0x7f9f4c00b040 comp rx=0 tx=0).stop 2024-05-06T12:13:49.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.328+0000 7f9f63464700 1 -- 172.21.15.92:0/415077467 shutdown_connections 2024-05-06T12:13:49.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.328+0000 7f9f63464700 1 --2- 172.21.15.92:0/415077467 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f9f480692b0 0x7f9f4806b760 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:49.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.328+0000 7f9f63464700 1 --2- 172.21.15.92:0/415077467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f5c0fbcb0 0x7f9f5c10fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:49.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.328+0000 7f9f63464700 1 --2- 172.21.15.92:0/415077467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f5c0fcf60 0x7f9f5c110130 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:49.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.328+0000 7f9f63464700 1 -- 172.21.15.92:0/415077467 >> 172.21.15.92:0/415077467 conn(0x7f9f5c0f7430 msgr2=0x7f9f5c065b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:13:49.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.328+0000 7f9f63464700 1 -- 172.21.15.92:0/415077467 shutdown_connections 2024-05-06T12:13:49.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.328+0000 7f9f63464700 1 -- 172.21.15.92:0/415077467 wait complete. 2024-05-06T12:13:49.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.443+0000 7fe39c4fd700 1 -- 172.21.15.92:0/3293443246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3940fcf60 msgr2=0x7fe3940fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:49.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.443+0000 7fe39c4fd700 1 --2- 172.21.15.92:0/3293443246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3940fcf60 0x7fe3940fd3d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe384004940 tx=0x7fe384009e30 comp rx=0 tx=0).stop 2024-05-06T12:13:49.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.443+0000 7fe39c4fd700 1 -- 172.21.15.92:0/3293443246 shutdown_connections 2024-05-06T12:13:49.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.443+0000 7fe39c4fd700 1 --2- 172.21.15.92:0/3293443246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3940fcf60 0x7fe3940fd3d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:49.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.443+0000 7fe39c4fd700 1 --2- 172.21.15.92:0/3293443246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3940fbcb0 0x7fe3940fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:49.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.443+0000 7fe39c4fd700 1 -- 172.21.15.92:0/3293443246 >> 172.21.15.92:0/3293443246 conn(0x7fe3940f7430 msgr2=0x7fe3940f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:13:49.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.443+0000 7fe39c4fd700 1 -- 172.21.15.92:0/3293443246 shutdown_connections 2024-05-06T12:13:49.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.443+0000 7fe39c4fd700 1 -- 172.21.15.92:0/3293443246 wait complete. 2024-05-06T12:13:49.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.444+0000 7fe39c4fd700 1 Processor -- start 2024-05-06T12:13:49.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.444+0000 7fe39c4fd700 1 -- start start 2024-05-06T12:13:49.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.444+0000 7fe39c4fd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3940fbcb0 0x7fe39410da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:49.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.444+0000 7fe39c4fd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3940fcf60 0x7fe39410dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:49.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.444+0000 7fe39c4fd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39410e5c0 con 0x7fe3940fcf60 2024-05-06T12:13:49.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.444+0000 7fe39c4fd700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39410ed70 con 0x7fe3940fbcb0 2024-05-06T12:13:49.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.444+0000 7fe399a98700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3940fcf60 0x7fe39410dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:49.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.444+0000 7fe399a98700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3940fcf60 0x7fe39410dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43228/0 (socket says 172.21.15.92:43228) 2024-05-06T12:13:49.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.444+0000 7fe399a98700 1 -- 172.21.15.92:0/1459543196 learned_addr learned my addr 172.21.15.92:0/1459543196 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:13:49.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.444+0000 7fe39a299700 1 --2- 172.21.15.92:0/1459543196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3940fbcb0 0x7fe39410da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:49.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.444+0000 7fe399a98700 1 -- 172.21.15.92:0/1459543196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3940fbcb0 msgr2=0x7fe39410da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:49.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.444+0000 7fe399a98700 1 --2- 172.21.15.92:0/1459543196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3940fbcb0 0x7fe39410da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:49.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.444+0000 7fe399a98700 1 -- 172.21.15.92:0/1459543196 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe384005040 con 0x7fe3940fcf60 2024-05-06T12:13:49.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.444+0000 7fe399a98700 1 --2- 172.21.15.92:0/1459543196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3940fcf60 0x7fe39410dfc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe384013870 tx=0x7fe384009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:13:49.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.445+0000 7fe38affd700 1 -- 172.21.15.92:0/1459543196 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe38400e070 con 0x7fe3940fcf60 2024-05-06T12:13:49.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.445+0000 7fe39c4fd700 1 -- 172.21.15.92:0/1459543196 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe39410eff0 con 0x7fe3940fcf60 2024-05-06T12:13:49.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.445+0000 7fe38affd700 1 -- 172.21.15.92:0/1459543196 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe384013de0 con 0x7fe3940fcf60 2024-05-06T12:13:49.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.445+0000 7fe39c4fd700 1 -- 172.21.15.92:0/1459543196 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe39410f4d0 con 0x7fe3940fcf60 2024-05-06T12:13:49.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.445+0000 7fe38affd700 1 -- 172.21.15.92:0/1459543196 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe38401cea0 con 0x7fe3940fcf60 2024-05-06T12:13:49.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.446+0000 7fe39c4fd700 1 -- 172.21.15.92:0/1459543196 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3780052b0 con 0x7fe3940fcf60 2024-05-06T12:13:49.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.447+0000 7fe38affd700 1 -- 172.21.15.92:0/1459543196 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fe384013910 con 0x7fe3940fcf60 2024-05-06T12:13:49.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.447+0000 7fe38affd700 1 --2- 172.21.15.92:0/1459543196 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fe3800692b0 0x7fe38006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:49.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.447+0000 7fe38affd700 1 -- 172.21.15.92:0/1459543196 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fe384088780 con 0x7fe3940fcf60 2024-05-06T12:13:49.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.447+0000 7fe39a299700 1 --2- 172.21.15.92:0/1459543196 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fe3800692b0 0x7fe38006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:49.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.448+0000 7fe39a299700 1 --2- 172.21.15.92:0/1459543196 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fe3800692b0 0x7fe38006b760 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe390009b80 tx=0x7fe39000b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:13:49.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.455+0000 7fe38affd700 1 -- 172.21.15.92:0/1459543196 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe384058590 con 0x7fe3940fcf60 2024-05-06T12:13:49.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.821+0000 7fe39c4fd700 1 -- 172.21.15.92:0/1459543196 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe378005d70 con 0x7fe3940fcf60 2024-05-06T12:13:49.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.822+0000 7fe38affd700 1 -- 172.21.15.92:0/1459543196 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7fe384004490 con 0x7fe3940fcf60 2024-05-06T12:13:49.825 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:13:49.825 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": { 2024-05-06T12:13:49.825 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 2 2024-05-06T12:13:49.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:13:49.826 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": { 2024-05-06T12:13:49.826 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 2 2024-05-06T12:13:49.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:13:49.826 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": { 2024-05-06T12:13:49.826 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 6 2024-05-06T12:13:49.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:13:49.826 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-05-06T12:13:49.826 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 4 2024-05-06T12:13:49.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:13:49.826 INFO:teuthology.orchestra.run.smithi092.stdout: "overall": { 2024-05-06T12:13:49.826 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 14 2024-05-06T12:13:49.826 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-06T12:13:49.826 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:13:49.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.825+0000 7fe39c4fd700 1 -- 172.21.15.92:0/1459543196 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fe3800692b0 msgr2=0x7fe38006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:49.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.825+0000 7fe39c4fd700 1 --2- 172.21.15.92:0/1459543196 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fe3800692b0 0x7fe38006b760 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe390009b80 tx=0x7fe39000b040 comp rx=0 tx=0).stop 2024-05-06T12:13:49.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.825+0000 7fe39c4fd700 1 -- 172.21.15.92:0/1459543196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3940fcf60 msgr2=0x7fe39410dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:49.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.825+0000 7fe39c4fd700 1 --2- 172.21.15.92:0/1459543196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3940fcf60 0x7fe39410dfc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe384013870 tx=0x7fe384009e30 comp rx=0 tx=0).stop 2024-05-06T12:13:49.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.825+0000 7fe39c4fd700 1 -- 172.21.15.92:0/1459543196 shutdown_connections 2024-05-06T12:13:49.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.825+0000 7fe39c4fd700 1 --2- 172.21.15.92:0/1459543196 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fe3800692b0 0x7fe38006b760 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:49.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.825+0000 7fe39c4fd700 1 --2- 172.21.15.92:0/1459543196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3940fbcb0 0x7fe39410da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:49.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.825+0000 7fe39c4fd700 1 --2- 172.21.15.92:0/1459543196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3940fcf60 0x7fe39410dfc0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:49.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.825+0000 7fe39c4fd700 1 -- 172.21.15.92:0/1459543196 >> 172.21.15.92:0/1459543196 conn(0x7fe3940f7430 msgr2=0x7fe394065c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:13:49.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.825+0000 7fe39c4fd700 1 -- 172.21.15.92:0/1459543196 shutdown_connections 2024-05-06T12:13:49.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.825+0000 7fe39c4fd700 1 -- 172.21.15.92:0/1459543196 wait complete. 2024-05-06T12:13:49.901 INFO:tasks.workunit.client.1.smithi190.stderr:+ tar xzf ltp-full.tgz 2024-05-06T12:13:49.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.952+0000 7f2553173700 1 -- 172.21.15.92:0/3264848867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f254c0fc290 msgr2=0x7f254c0fc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:49.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.952+0000 7f2553173700 1 --2- 172.21.15.92:0/3264848867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f254c0fc290 0x7f254c0fc6a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2540004940 tx=0x7f2540009e30 comp rx=0 tx=0).stop 2024-05-06T12:13:49.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.952+0000 7f2553173700 1 -- 172.21.15.92:0/3264848867 shutdown_connections 2024-05-06T12:13:49.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.952+0000 7f2553173700 1 --2- 172.21.15.92:0/3264848867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f254c0fcbe0 0x7f254c0fd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:49.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.952+0000 7f2553173700 1 --2- 172.21.15.92:0/3264848867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f254c0fc290 0x7f254c0fc6a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:49.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.952+0000 7f2553173700 1 -- 172.21.15.92:0/3264848867 >> 172.21.15.92:0/3264848867 conn(0x7f254c0f6b80 msgr2=0x7f254c0f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:13:49.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.952+0000 7f2553173700 1 -- 172.21.15.92:0/3264848867 shutdown_connections 2024-05-06T12:13:49.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.952+0000 7f2553173700 1 -- 172.21.15.92:0/3264848867 wait complete. 2024-05-06T12:13:49.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.953+0000 7f2553173700 1 Processor -- start 2024-05-06T12:13:49.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.953+0000 7f2553173700 1 -- start start 2024-05-06T12:13:49.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.953+0000 7f2553173700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f254c0fc290 0x7f254c113ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:49.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.953+0000 7f2553173700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f254c0fcbe0 0x7f254c114530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:49.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.953+0000 7f2553173700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c114b30 con 0x7f254c0fc290 2024-05-06T12:13:49.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.953+0000 7f2553173700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c115380 con 0x7f254c0fcbe0 2024-05-06T12:13:49.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.954+0000 7f2550f0f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f254c0fc290 0x7f254c113ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:49.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.954+0000 7f2550f0f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f254c0fc290 0x7f254c113ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43248/0 (socket says 172.21.15.92:43248) 2024-05-06T12:13:49.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.954+0000 7f2550f0f700 1 -- 172.21.15.92:0/4113644291 learned_addr learned my addr 172.21.15.92:0/4113644291 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:13:49.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.954+0000 7f2550f0f700 1 -- 172.21.15.92:0/4113644291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f254c0fcbe0 msgr2=0x7f254c114530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:49.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.954+0000 7f2550f0f700 1 --2- 172.21.15.92:0/4113644291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f254c0fcbe0 0x7f254c114530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:49.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.954+0000 7f2550f0f700 1 -- 172.21.15.92:0/4113644291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2540005040 con 0x7f254c0fc290 2024-05-06T12:13:49.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.954+0000 7f2550f0f700 1 --2- 172.21.15.92:0/4113644291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f254c0fc290 0x7f254c113ff0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2540000c00 tx=0x7f2540009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:13:49.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.955+0000 7f25497fa700 1 -- 172.21.15.92:0/4113644291 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f254000e070 con 0x7f254c0fc290 2024-05-06T12:13:49.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.955+0000 7f25497fa700 1 -- 172.21.15.92:0/4113644291 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f254001c400 con 0x7f254c0fc290 2024-05-06T12:13:49.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.955+0000 7f2553173700 1 -- 172.21.15.92:0/4113644291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f254c115600 con 0x7f254c0fc290 2024-05-06T12:13:49.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.955+0000 7f25497fa700 1 -- 172.21.15.92:0/4113644291 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f254000b650 con 0x7f254c0fc290 2024-05-06T12:13:49.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.956+0000 7f2553173700 1 -- 172.21.15.92:0/4113644291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f254c115ae0 con 0x7f254c0fc290 2024-05-06T12:13:49.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.957+0000 7f2553173700 1 -- 172.21.15.92:0/4113644291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f254c1019e0 con 0x7f254c0fc290 2024-05-06T12:13:49.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.957+0000 7f25497fa700 1 -- 172.21.15.92:0/4113644291 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f254000b8c0 con 0x7f254c0fc290 2024-05-06T12:13:49.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.957+0000 7f25497fa700 1 --2- 172.21.15.92:0/4113644291 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f253c0692a0 0x7f253c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:49.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.958+0000 7f25497fa700 1 -- 172.21.15.92:0/4113644291 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f2540088790 con 0x7f254c0fc290 2024-05-06T12:13:49.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.958+0000 7f254bfff700 1 --2- 172.21.15.92:0/4113644291 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f253c0692a0 0x7f253c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:49.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.958+0000 7f254bfff700 1 --2- 172.21.15.92:0/4113644291 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f253c0692a0 0x7f253c06b750 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2538009fd0 tx=0x7f253800b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:13:49.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:49.963+0000 7f25497fa700 1 -- 172.21.15.92:0/4113644291 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2540058610 con 0x7f254c0fc290 2024-05-06T12:13:50.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.304+0000 7f2553173700 1 -- 172.21.15.92:0/4113644291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f254c101400 con 0x7f254c0fc290 2024-05-06T12:13:50.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.305+0000 7f25497fa700 1 -- 172.21.15.92:0/4113644291 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f25400581a0 con 0x7f254c0fc290 2024-05-06T12:13:50.308 INFO:teuthology.orchestra.run.smithi092.stdout:e18 2024-05-06T12:13:50.308 INFO:teuthology.orchestra.run.smithi092.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-06T12:13:50.308 INFO:teuthology.orchestra.run.smithi092.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:13:50.308 INFO:teuthology.orchestra.run.smithi092.stdout:legacy client fscid: 1 2024-05-06T12:13:50.308 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:13:50.308 INFO:teuthology.orchestra.run.smithi092.stdout:Filesystem 'cephfs' (1) 2024-05-06T12:13:50.308 INFO:teuthology.orchestra.run.smithi092.stdout:fs_name cephfs 2024-05-06T12:13:50.309 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 18 2024-05-06T12:13:50.309 INFO:teuthology.orchestra.run.smithi092.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-06T12:13:50.309 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-05-06T12:11:44.952671+0000 2024-05-06T12:13:50.309 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2024-05-06T12:12:04.907000+0000 2024-05-06T12:13:50.309 INFO:teuthology.orchestra.run.smithi092.stdout:tableserver 0 2024-05-06T12:13:50.309 INFO:teuthology.orchestra.run.smithi092.stdout:root 0 2024-05-06T12:13:50.309 INFO:teuthology.orchestra.run.smithi092.stdout:session_timeout 60 2024-05-06T12:13:50.309 INFO:teuthology.orchestra.run.smithi092.stdout:session_autoclose 300 2024-05-06T12:13:50.309 INFO:teuthology.orchestra.run.smithi092.stdout:max_file_size 1099511627776 2024-05-06T12:13:50.309 INFO:teuthology.orchestra.run.smithi092.stdout:required_client_features {} 2024-05-06T12:13:50.309 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure 0 2024-05-06T12:13:50.309 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure_osd_epoch 46 2024-05-06T12:13:50.309 INFO:teuthology.orchestra.run.smithi092.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:13:50.309 INFO:teuthology.orchestra.run.smithi092.stdout:max_mds 2 2024-05-06T12:13:50.309 INFO:teuthology.orchestra.run.smithi092.stdout:in 0,1 2024-05-06T12:13:50.310 INFO:teuthology.orchestra.run.smithi092.stdout:up {0=14472,1=24267} 2024-05-06T12:13:50.310 INFO:teuthology.orchestra.run.smithi092.stdout:failed 2024-05-06T12:13:50.310 INFO:teuthology.orchestra.run.smithi092.stdout:damaged 2024-05-06T12:13:50.310 INFO:teuthology.orchestra.run.smithi092.stdout:stopped 2024-05-06T12:13:50.310 INFO:teuthology.orchestra.run.smithi092.stdout:data_pools [3] 2024-05-06T12:13:50.310 INFO:teuthology.orchestra.run.smithi092.stdout:metadata_pool 2 2024-05-06T12:13:50.310 INFO:teuthology.orchestra.run.smithi092.stdout:inline_data enabled 2024-05-06T12:13:50.310 INFO:teuthology.orchestra.run.smithi092.stdout:balancer 2024-05-06T12:13:50.310 INFO:teuthology.orchestra.run.smithi092.stdout:standby_count_wanted 1 2024-05-06T12:13:50.310 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.rosbll{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.92:6828/1663253598,v1:172.21.15.92:6829/1663253598] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:13:50.310 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.ijsqgp{0:14484} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.92:6826/3613441054,v1:172.21.15.92:6827/3613441054] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:13:50.310 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.byjvgf{1:24267} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.190:6824/1565151699,v1:172.21.15.190:6825/1565151699] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:13:50.310 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.idzoff{1:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.190:6826/2453728380,v1:172.21.15.190:6827/2453728380] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:13:50.310 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:13:50.311 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:13:50.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.307+0000 7f2553173700 1 -- 172.21.15.92:0/4113644291 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f253c0692a0 msgr2=0x7f253c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:50.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.307+0000 7f2553173700 1 --2- 172.21.15.92:0/4113644291 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f253c0692a0 0x7f253c06b750 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2538009fd0 tx=0x7f253800b040 comp rx=0 tx=0).stop 2024-05-06T12:13:50.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.307+0000 7f2553173700 1 -- 172.21.15.92:0/4113644291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f254c0fc290 msgr2=0x7f254c113ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:50.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.307+0000 7f2553173700 1 --2- 172.21.15.92:0/4113644291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f254c0fc290 0x7f254c113ff0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2540000c00 tx=0x7f2540009e30 comp rx=0 tx=0).stop 2024-05-06T12:13:50.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.307+0000 7f2553173700 1 -- 172.21.15.92:0/4113644291 shutdown_connections 2024-05-06T12:13:50.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.307+0000 7f2553173700 1 --2- 172.21.15.92:0/4113644291 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7f253c0692a0 0x7f253c06b750 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:50.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.307+0000 7f2553173700 1 --2- 172.21.15.92:0/4113644291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f254c0fc290 0x7f254c113ff0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:50.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.307+0000 7f2553173700 1 --2- 172.21.15.92:0/4113644291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f254c0fcbe0 0x7f254c114530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:50.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.307+0000 7f2553173700 1 -- 172.21.15.92:0/4113644291 >> 172.21.15.92:0/4113644291 conn(0x7f254c0f6b80 msgr2=0x7f254c0f8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:13:50.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.307+0000 7f2553173700 1 -- 172.21.15.92:0/4113644291 shutdown_connections 2024-05-06T12:13:50.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.307+0000 7f2553173700 1 -- 172.21.15.92:0/4113644291 wait complete. 2024-05-06T12:13:50.312 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 18 2024-05-06T12:13:50.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.433+0000 7ff882f67700 1 -- 172.21.15.92:0/3786978637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff87c0fed30 msgr2=0x7ff87c0fae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:50.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.433+0000 7ff882f67700 1 --2- 172.21.15.92:0/3786978637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff87c0fed30 0x7ff87c0fae70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff868004940 tx=0x7ff868009e30 comp rx=0 tx=0).stop 2024-05-06T12:13:50.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.433+0000 7ff882f67700 1 -- 172.21.15.92:0/3786978637 shutdown_connections 2024-05-06T12:13:50.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.433+0000 7ff882f67700 1 --2- 172.21.15.92:0/3786978637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff87c0fed30 0x7ff87c0fae70 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:50.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.433+0000 7ff882f67700 1 --2- 172.21.15.92:0/3786978637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff87c0fe3e0 0x7ff87c0fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:50.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.433+0000 7ff882f67700 1 -- 172.21.15.92:0/3786978637 >> 172.21.15.92:0/3786978637 conn(0x7ff87c0f6b80 msgr2=0x7ff87c0f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:13:50.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.433+0000 7ff882f67700 1 -- 172.21.15.92:0/3786978637 shutdown_connections 2024-05-06T12:13:50.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.433+0000 7ff882f67700 1 -- 172.21.15.92:0/3786978637 wait complete. 2024-05-06T12:13:50.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.433+0000 7ff882f67700 1 Processor -- start 2024-05-06T12:13:50.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.434+0000 7ff882f67700 1 -- start start 2024-05-06T12:13:50.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.434+0000 7ff882f67700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff87c0fe3e0 0x7ff87c0fdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:50.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.434+0000 7ff882f67700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff87c0fed30 0x7ff87c0fc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:50.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.434+0000 7ff882f67700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c1115f0 con 0x7ff87c0fe3e0 2024-05-06T12:13:50.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.434+0000 7ff882f67700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c111760 con 0x7ff87c0fed30 2024-05-06T12:13:50.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.434+0000 7ff880d03700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff87c0fe3e0 0x7ff87c0fdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:50.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.434+0000 7ff880d03700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff87c0fe3e0 0x7ff87c0fdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43262/0 (socket says 172.21.15.92:43262) 2024-05-06T12:13:50.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.434+0000 7ff880d03700 1 -- 172.21.15.92:0/3960892372 learned_addr learned my addr 172.21.15.92:0/3960892372 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:13:50.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.434+0000 7ff87bfff700 1 --2- 172.21.15.92:0/3960892372 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff87c0fed30 0x7ff87c0fc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:50.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.434+0000 7ff880d03700 1 -- 172.21.15.92:0/3960892372 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff87c0fed30 msgr2=0x7ff87c0fc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:50.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.434+0000 7ff880d03700 1 --2- 172.21.15.92:0/3960892372 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff87c0fed30 0x7ff87c0fc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:50.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.435+0000 7ff880d03700 1 -- 172.21.15.92:0/3960892372 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff868005040 con 0x7ff87c0fe3e0 2024-05-06T12:13:50.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.435+0000 7ff87bfff700 1 --2- 172.21.15.92:0/3960892372 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff87c0fed30 0x7ff87c0fc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-06T12:13:50.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.435+0000 7ff880d03700 1 --2- 172.21.15.92:0/3960892372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff87c0fe3e0 0x7ff87c0fdbd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff87000eb40 tx=0x7ff87000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:13:50.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.435+0000 7ff8797fa700 1 -- 172.21.15.92:0/3960892372 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff870007cc0 con 0x7ff87c0fe3e0 2024-05-06T12:13:50.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.435+0000 7ff882f67700 1 -- 172.21.15.92:0/3960892372 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff87c0fc6a0 con 0x7ff87c0fe3e0 2024-05-06T12:13:50.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.435+0000 7ff8797fa700 1 -- 172.21.15.92:0/3960892372 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff87000f040 con 0x7ff87c0fe3e0 2024-05-06T12:13:50.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.435+0000 7ff8797fa700 1 -- 172.21.15.92:0/3960892372 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff87000ca10 con 0x7ff87c0fe3e0 2024-05-06T12:13:50.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.435+0000 7ff882f67700 1 -- 172.21.15.92:0/3960892372 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff87c0fcb90 con 0x7ff87c0fe3e0 2024-05-06T12:13:50.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.436+0000 7ff882f67700 1 -- 172.21.15.92:0/3960892372 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8600052e0 con 0x7ff87c0fe3e0 2024-05-06T12:13:50.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.436+0000 7ff8797fa700 1 -- 172.21.15.92:0/3960892372 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7ff87000cb70 con 0x7ff87c0fe3e0 2024-05-06T12:13:50.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.437+0000 7ff8797fa700 1 --2- 172.21.15.92:0/3960892372 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7ff86c0692a0 0x7ff86c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:50.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.437+0000 7ff8797fa700 1 -- 172.21.15.92:0/3960892372 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7ff870086ca0 con 0x7ff87c0fe3e0 2024-05-06T12:13:50.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.437+0000 7ff87bfff700 1 --2- 172.21.15.92:0/3960892372 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7ff86c0692a0 0x7ff86c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:50.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.438+0000 7ff87bfff700 1 --2- 172.21.15.92:0/3960892372 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7ff86c0692a0 0x7ff86c06b750 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff868009b30 tx=0x7ff868009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:13:50.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.442+0000 7ff8797fa700 1 -- 172.21.15.92:0/3960892372 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff87004e250 con 0x7ff87c0fe3e0 2024-05-06T12:13:50.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:50 smithi190 ceph-mon[33798]: from='client.14608 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:13:50.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:50 smithi190 ceph-mon[33798]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:13:50.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:50 smithi190 ceph-mon[33798]: Upgrade: Updating mgr.smithi190.kczxkh 2024-05-06T12:13:50.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:50 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:13:50.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:50 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi190.kczxkh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-06T12:13:50.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:50 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-06T12:13:50.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:50 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:13:50.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:50 smithi190 ceph-mon[33798]: Deploying daemon mgr.smithi190.kczxkh on smithi190 2024-05-06T12:13:50.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:50 smithi190 ceph-mon[33798]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:13:50.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:50 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/1459543196' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:13:50.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:50 smithi092 ceph-mon[26954]: from='client.14608 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:13:50.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:50 smithi092 ceph-mon[26954]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:13:50.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:50 smithi092 ceph-mon[26954]: Upgrade: Updating mgr.smithi190.kczxkh 2024-05-06T12:13:50.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:50 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:13:50.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:50 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi190.kczxkh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-06T12:13:50.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:50 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-06T12:13:50.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:50 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:13:50.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:50 smithi092 ceph-mon[26954]: Deploying daemon mgr.smithi190.kczxkh on smithi190 2024-05-06T12:13:50.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:50 smithi092 ceph-mon[26954]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:13:50.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:50 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/1459543196' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:13:50.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.779+0000 7ff882f67700 1 -- 172.21.15.92:0/3960892372 --> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff860000bf0 con 0x7ff86c0692a0 2024-05-06T12:13:50.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.780+0000 7ff8797fa700 1 -- 172.21.15.92:0/3960892372 <== mgr.14210 v2:172.21.15.92:6800/3045905943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7ff860000bf0 con 0x7ff86c0692a0 2024-05-06T12:13:50.783 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:13:50.783 INFO:teuthology.orchestra.run.smithi092.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb", 2024-05-06T12:13:50.783 INFO:teuthology.orchestra.run.smithi092.stdout: "in_progress": true, 2024-05-06T12:13:50.783 INFO:teuthology.orchestra.run.smithi092.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-06T12:13:50.783 INFO:teuthology.orchestra.run.smithi092.stdout: "services_complete": [], 2024-05-06T12:13:50.784 INFO:teuthology.orchestra.run.smithi092.stdout: "progress": "0/21 daemons upgraded", 2024-05-06T12:13:50.784 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "Currently upgrading mgr daemons", 2024-05-06T12:13:50.784 INFO:teuthology.orchestra.run.smithi092.stdout: "is_paused": false 2024-05-06T12:13:50.784 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:13:50.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.783+0000 7ff882f67700 1 -- 172.21.15.92:0/3960892372 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7ff86c0692a0 msgr2=0x7ff86c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:50.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.783+0000 7ff882f67700 1 --2- 172.21.15.92:0/3960892372 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7ff86c0692a0 0x7ff86c06b750 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff868009b30 tx=0x7ff868009e30 comp rx=0 tx=0).stop 2024-05-06T12:13:50.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.783+0000 7ff882f67700 1 -- 172.21.15.92:0/3960892372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff87c0fe3e0 msgr2=0x7ff87c0fdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:50.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.783+0000 7ff882f67700 1 --2- 172.21.15.92:0/3960892372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff87c0fe3e0 0x7ff87c0fdbd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff87000eb40 tx=0x7ff87000b040 comp rx=0 tx=0).stop 2024-05-06T12:13:50.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.783+0000 7ff882f67700 1 -- 172.21.15.92:0/3960892372 shutdown_connections 2024-05-06T12:13:50.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.783+0000 7ff882f67700 1 --2- 172.21.15.92:0/3960892372 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7ff86c0692a0 0x7ff86c06b750 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:50.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.783+0000 7ff882f67700 1 --2- 172.21.15.92:0/3960892372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff87c0fe3e0 0x7ff87c0fdbd0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:50.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.783+0000 7ff882f67700 1 --2- 172.21.15.92:0/3960892372 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff87c0fed30 0x7ff87c0fc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:50.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.783+0000 7ff882f67700 1 -- 172.21.15.92:0/3960892372 >> 172.21.15.92:0/3960892372 conn(0x7ff87c0f6b80 msgr2=0x7ff87c0f8190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:13:50.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.783+0000 7ff882f67700 1 -- 172.21.15.92:0/3960892372 shutdown_connections 2024-05-06T12:13:50.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.783+0000 7ff882f67700 1 -- 172.21.15.92:0/3960892372 wait complete. 2024-05-06T12:13:50.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.925+0000 7fd4f8d71700 1 -- 172.21.15.92:0/1655556563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4f40ffee0 msgr2=0x7fd4f4102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:50.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.925+0000 7fd4f8d71700 1 --2- 172.21.15.92:0/1655556563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4f40ffee0 0x7fd4f4102300 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8004940 tx=0x7fd4e8009e30 comp rx=0 tx=0).stop 2024-05-06T12:13:50.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.925+0000 7fd4f8d71700 1 -- 172.21.15.92:0/1655556563 shutdown_connections 2024-05-06T12:13:50.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.925+0000 7fd4f8d71700 1 --2- 172.21.15.92:0/1655556563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4f40ffee0 0x7fd4f4102300 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:50.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.925+0000 7fd4f8d71700 1 --2- 172.21.15.92:0/1655556563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4f40662c0 0x7fd4f40ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:50.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.925+0000 7fd4f8d71700 1 -- 172.21.15.92:0/1655556563 >> 172.21.15.92:0/1655556563 conn(0x7fd4f40f7430 msgr2=0x7fd4f40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:13:50.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.925+0000 7fd4f8d71700 1 -- 172.21.15.92:0/1655556563 shutdown_connections 2024-05-06T12:13:50.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.925+0000 7fd4f8d71700 1 -- 172.21.15.92:0/1655556563 wait complete. 2024-05-06T12:13:50.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.926+0000 7fd4f8d71700 1 Processor -- start 2024-05-06T12:13:50.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.926+0000 7fd4f8d71700 1 -- start start 2024-05-06T12:13:50.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.926+0000 7fd4f8d71700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4f40662c0 0x7fd4f40fd380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:50.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.926+0000 7fd4f8d71700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4f40ffee0 0x7fd4f40fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:50.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.926+0000 7fd4f8d71700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f4111640 con 0x7fd4f40662c0 2024-05-06T12:13:50.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.926+0000 7fd4f8d71700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f4111780 con 0x7fd4f40ffee0 2024-05-06T12:13:50.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.926+0000 7fd4f259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4f40662c0 0x7fd4f40fd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:50.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.926+0000 7fd4f259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4f40662c0 0x7fd4f40fd380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43290/0 (socket says 172.21.15.92:43290) 2024-05-06T12:13:50.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.926+0000 7fd4f259c700 1 -- 172.21.15.92:0/1377962104 learned_addr learned my addr 172.21.15.92:0/1377962104 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:13:50.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.926+0000 7fd4f1d9b700 1 --2- 172.21.15.92:0/1377962104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4f40ffee0 0x7fd4f40fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:50.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.927+0000 7fd4f259c700 1 -- 172.21.15.92:0/1377962104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4f40ffee0 msgr2=0x7fd4f40fb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:50.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.927+0000 7fd4f259c700 1 --2- 172.21.15.92:0/1377962104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4f40ffee0 0x7fd4f40fb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:50.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.927+0000 7fd4f259c700 1 -- 172.21.15.92:0/1377962104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e8005040 con 0x7fd4f40662c0 2024-05-06T12:13:50.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.927+0000 7fd4f259c700 1 --2- 172.21.15.92:0/1377962104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4f40662c0 0x7fd4f40fd380 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400eb40 tx=0x7fd4e400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:13:50.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.927+0000 7fd4e2ffd700 1 -- 172.21.15.92:0/1377962104 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4e4007cc0 con 0x7fd4f40662c0 2024-05-06T12:13:50.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.927+0000 7fd4f8d71700 1 -- 172.21.15.92:0/1377962104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f40fbe40 con 0x7fd4f40662c0 2024-05-06T12:13:50.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.927+0000 7fd4e2ffd700 1 -- 172.21.15.92:0/1377962104 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd4e400f040 con 0x7fd4f40662c0 2024-05-06T12:13:50.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.927+0000 7fd4e2ffd700 1 -- 172.21.15.92:0/1377962104 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4e400c9d0 con 0x7fd4f40662c0 2024-05-06T12:13:50.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.927+0000 7fd4f8d71700 1 -- 172.21.15.92:0/1377962104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f40fc400 con 0x7fd4f40662c0 2024-05-06T12:13:50.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.928+0000 7fd4f8d71700 1 -- 172.21.15.92:0/1377962104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4f405f5c0 con 0x7fd4f40662c0 2024-05-06T12:13:50.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.929+0000 7fd4e2ffd700 1 -- 172.21.15.92:0/1377962104 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fd4e400cb30 con 0x7fd4f40662c0 2024-05-06T12:13:50.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.930+0000 7fd4e2ffd700 1 --2- 172.21.15.92:0/1377962104 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fd4dc0692a0 0x7fd4dc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:13:50.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.930+0000 7fd4e2ffd700 1 -- 172.21.15.92:0/1377962104 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fd4e4086d60 con 0x7fd4f40662c0 2024-05-06T12:13:50.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.930+0000 7fd4f1d9b700 1 --2- 172.21.15.92:0/1377962104 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fd4dc0692a0 0x7fd4dc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:13:50.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.930+0000 7fd4f1d9b700 1 --2- 172.21.15.92:0/1377962104 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fd4dc0692a0 0x7fd4dc06b750 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40fd960 tx=0x7fd4e8009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:13:50.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:50.936+0000 7fd4e2ffd700 1 -- 172.21.15.92:0/1377962104 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd4e4053190 con 0x7fd4f40662c0 2024-05-06T12:13:51.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:51.333+0000 7fd4f8d71700 1 -- 172.21.15.92:0/1377962104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd4f404bed0 con 0x7fd4f40662c0 2024-05-06T12:13:51.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:51.333+0000 7fd4e2ffd700 1 -- 172.21.15.92:0/1377962104 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fd4e40567b0 con 0x7fd4f40662c0 2024-05-06T12:13:51.337 INFO:teuthology.orchestra.run.smithi092.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-06T12:13:51.337 INFO:teuthology.orchestra.run.smithi092.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-06T12:13:51.337 INFO:teuthology.orchestra.run.smithi092.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-06T12:13:51.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:51.336+0000 7fd4f8d71700 1 -- 172.21.15.92:0/1377962104 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fd4dc0692a0 msgr2=0x7fd4dc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:51.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:51.336+0000 7fd4f8d71700 1 --2- 172.21.15.92:0/1377962104 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fd4dc0692a0 0x7fd4dc06b750 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40fd960 tx=0x7fd4e8009e30 comp rx=0 tx=0).stop 2024-05-06T12:13:51.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:51.336+0000 7fd4f8d71700 1 -- 172.21.15.92:0/1377962104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4f40662c0 msgr2=0x7fd4f40fd380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:13:51.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:51.336+0000 7fd4f8d71700 1 --2- 172.21.15.92:0/1377962104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4f40662c0 0x7fd4f40fd380 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400eb40 tx=0x7fd4e400b040 comp rx=0 tx=0).stop 2024-05-06T12:13:51.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:51.336+0000 7fd4f8d71700 1 -- 172.21.15.92:0/1377962104 shutdown_connections 2024-05-06T12:13:51.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:51.336+0000 7fd4f8d71700 1 --2- 172.21.15.92:0/1377962104 >> [v2:172.21.15.92:6800/3045905943,v1:172.21.15.92:6801/3045905943] conn(0x7fd4dc0692a0 0x7fd4dc06b750 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:51.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:51.336+0000 7fd4f8d71700 1 --2- 172.21.15.92:0/1377962104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4f40662c0 0x7fd4f40fd380 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:51.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:51.336+0000 7fd4f8d71700 1 --2- 172.21.15.92:0/1377962104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4f40ffee0 0x7fd4f40fb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:13:51.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:51.336+0000 7fd4f8d71700 1 -- 172.21.15.92:0/1377962104 >> 172.21.15.92:0/1377962104 conn(0x7fd4f40f7430 msgr2=0x7fd4f40f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:13:51.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:51.336+0000 7fd4f8d71700 1 -- 172.21.15.92:0/1377962104 shutdown_connections 2024-05-06T12:13:51.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:13:51.336+0000 7fd4f8d71700 1 -- 172.21.15.92:0/1377962104 wait complete. 2024-05-06T12:13:51.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:51 smithi190 ceph-mon[33798]: pgmap v237: 65 pgs: 65 active+clean; 23 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 591 KiB/s rd, 1.9 MiB/s wr, 80 op/s 2024-05-06T12:13:51.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:51 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/4113644291' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:13:51.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:51 smithi092 ceph-mon[26954]: pgmap v237: 65 pgs: 65 active+clean; 23 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 591 KiB/s rd, 1.9 MiB/s wr, 80 op/s 2024-05-06T12:13:51.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:51 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/4113644291' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:13:52.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:52 smithi190 ceph-mon[33798]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:13:52.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:52 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/1377962104' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:13:52.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:52 smithi092 ceph-mon[26954]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:13:52.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:52 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/1377962104' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:13:53.432 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:53 smithi190 ceph-mon[33798]: pgmap v238: 65 pgs: 65 active+clean; 30 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 848 KiB/s rd, 2.4 MiB/s wr, 93 op/s 2024-05-06T12:13:53.432 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:53 smithi190 ceph-mon[33798]: fsmap cephfs:2 {0=cephfs.smithi092.rosbll=up:active,1=cephfs.smithi190.byjvgf=up:active} 2 up:standby-replay 2024-05-06T12:13:53.432 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:53 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:13:53.432 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:53 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:13:53.432 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:53 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:13:53.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:53 smithi092 ceph-mon[26954]: pgmap v238: 65 pgs: 65 active+clean; 30 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 848 KiB/s rd, 2.4 MiB/s wr, 93 op/s 2024-05-06T12:13:53.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:53 smithi092 ceph-mon[26954]: fsmap cephfs:2 {0=cephfs.smithi092.rosbll=up:active,1=cephfs.smithi190.byjvgf=up:active} 2 up:standby-replay 2024-05-06T12:13:53.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:53 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:13:53.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:53 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:13:53.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:53 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:13:55.525 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:55 smithi190 ceph-mon[33798]: pgmap v239: 65 pgs: 65 active+clean; 44 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 138 op/s 2024-05-06T12:13:55.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:55 smithi092 ceph-mon[26954]: pgmap v239: 65 pgs: 65 active+clean; 44 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 138 op/s 2024-05-06T12:13:57.675 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:57 smithi190 ceph-mon[33798]: pgmap v240: 65 pgs: 65 active+clean; 55 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 151 op/s 2024-05-06T12:13:57.675 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:57 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:13:57.675 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:57 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:13:57.675 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:57 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:13:57.675 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:57 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:13:57.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:57 smithi092 ceph-mon[26954]: pgmap v240: 65 pgs: 65 active+clean; 55 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 151 op/s 2024-05-06T12:13:57.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:57 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:13:57.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:57 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:13:57.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:57 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:13:57.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:57 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:13:59.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:13:59 smithi190 ceph-mon[33798]: pgmap v241: 65 pgs: 65 active+clean; 66 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 4.7 MiB/s wr, 178 op/s 2024-05-06T12:13:59.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:13:59 smithi092 ceph-mon[26954]: pgmap v241: 65 pgs: 65 active+clean; 66 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 4.7 MiB/s wr, 178 op/s 2024-05-06T12:14:01.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:01 smithi092 ceph-mon[26954]: pgmap v242: 65 pgs: 65 active+clean; 73 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 161 op/s 2024-05-06T12:14:01.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:01 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:14:01.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:01 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:14:01.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:01 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:14:01.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:01 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:14:01.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:01 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:14:01.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:01 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:14:01.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:01 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mgr fail", "who": "smithi092.gyzeqb"}]: dispatch 2024-05-06T12:14:01.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:01 smithi092 ceph-mon[26954]: Activating manager daemon smithi190.kczxkh 2024-05-06T12:14:01.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:01 smithi092 ceph-mon[26954]: osdmap e47: 6 total, 6 up, 6 in 2024-05-06T12:14:01.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:01 smithi092 ceph-mon[26954]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "mgr fail", "who": "smithi092.gyzeqb"}]': finished 2024-05-06T12:14:01.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:01 smithi092 ceph-mon[26954]: mgrmap e20: smithi190.kczxkh(active, starting, since 0.00676831s) 2024-05-06T12:14:01.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:01 smithi190 ceph-mon[33798]: pgmap v242: 65 pgs: 65 active+clean; 73 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 161 op/s 2024-05-06T12:14:01.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:01 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:14:01.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:01 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:14:01.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:01 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:14:01.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:01 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:14:01.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:01 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:14:01.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:01 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:14:01.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:01 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mgr fail", "who": "smithi092.gyzeqb"}]: dispatch 2024-05-06T12:14:01.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:01 smithi190 ceph-mon[33798]: Activating manager daemon smithi190.kczxkh 2024-05-06T12:14:01.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:01 smithi190 ceph-mon[33798]: osdmap e47: 6 total, 6 up, 6 in 2024-05-06T12:14:01.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:01 smithi190 ceph-mon[33798]: from='mgr.14210 172.21.15.92:0/2483794148' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "mgr fail", "who": "smithi092.gyzeqb"}]': finished 2024-05-06T12:14:01.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:01 smithi190 ceph-mon[33798]: mgrmap e20: smithi190.kczxkh(active, starting, since 0.00676831s) 2024-05-06T12:14:02.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:02 smithi092 ceph-mon[26954]: Active manager daemon smithi190.kczxkh restarted 2024-05-06T12:14:02.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:02 smithi092 ceph-mon[26954]: Activating manager daemon smithi190.kczxkh 2024-05-06T12:14:02.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:02 smithi092 ceph-mon[26954]: osdmap e48: 6 total, 6 up, 6 in 2024-05-06T12:14:02.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:02 smithi092 ceph-mon[26954]: mgrmap e21: smithi190.kczxkh(active, starting, since 0.00741343s) 2024-05-06T12:14:02.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:02 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi190.kczxkh/crt"}]: dispatch 2024-05-06T12:14:02.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:02 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "mon metadata", "id": "smithi092"}]: dispatch 2024-05-06T12:14:02.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:02 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "mon metadata", "id": "smithi190"}]: dispatch 2024-05-06T12:14:02.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:02 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi092.rosbll"}]: dispatch 2024-05-06T12:14:02.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:02 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi092.ijsqgp"}]: dispatch 2024-05-06T12:14:02.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:02 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi190.byjvgf"}]: dispatch 2024-05-06T12:14:02.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:02 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi190.idzoff"}]: dispatch 2024-05-06T12:14:02.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:02 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "mgr metadata", "who": "smithi190.kczxkh", "id": "smithi190.kczxkh"}]: dispatch 2024-05-06T12:14:02.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:02 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-06T12:14:02.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:02 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-06T12:14:02.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:02 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-06T12:14:02.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:02 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-06T12:14:02.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:02 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-06T12:14:02.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:02 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-06T12:14:02.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:02 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-06T12:14:02.475 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:02 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-06T12:14:02.475 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:02 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-06T12:14:02.475 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:02 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-06T12:14:02.475 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:02 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi190.kczxkh/key"}]: dispatch 2024-05-06T12:14:02.475 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:02 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-06T12:14:02.475 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:02 smithi092 ceph-mon[26954]: Manager daemon smithi190.kczxkh is now available 2024-05-06T12:14:02.475 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:02 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:14:02.475 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:02 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:14:02.475 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:02 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi190.kczxkh/mirror_snapshot_schedule"}]: dispatch 2024-05-06T12:14:02.475 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:02 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi190.kczxkh/mirror_snapshot_schedule"}]: dispatch 2024-05-06T12:14:02.475 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:02 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi190.kczxkh/trash_purge_schedule"}]: dispatch 2024-05-06T12:14:02.475 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:02 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi190.kczxkh/trash_purge_schedule"}]: dispatch 2024-05-06T12:14:02.475 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:02 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:02.475 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:02 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:02.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:02 smithi190 ceph-mon[33798]: Active manager daemon smithi190.kczxkh restarted 2024-05-06T12:14:02.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:02 smithi190 ceph-mon[33798]: Activating manager daemon smithi190.kczxkh 2024-05-06T12:14:02.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:02 smithi190 ceph-mon[33798]: osdmap e48: 6 total, 6 up, 6 in 2024-05-06T12:14:02.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:02 smithi190 ceph-mon[33798]: mgrmap e21: smithi190.kczxkh(active, starting, since 0.00741343s) 2024-05-06T12:14:02.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:02 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi190.kczxkh/crt"}]: dispatch 2024-05-06T12:14:02.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:02 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "mon metadata", "id": "smithi092"}]: dispatch 2024-05-06T12:14:02.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:02 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "mon metadata", "id": "smithi190"}]: dispatch 2024-05-06T12:14:02.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:02 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi092.rosbll"}]: dispatch 2024-05-06T12:14:02.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:02 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi092.ijsqgp"}]: dispatch 2024-05-06T12:14:02.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:02 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi190.byjvgf"}]: dispatch 2024-05-06T12:14:02.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:02 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi190.idzoff"}]: dispatch 2024-05-06T12:14:02.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:02 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "mgr metadata", "who": "smithi190.kczxkh", "id": "smithi190.kczxkh"}]: dispatch 2024-05-06T12:14:02.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:02 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-06T12:14:02.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:02 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-06T12:14:02.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:02 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-06T12:14:02.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:02 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-06T12:14:02.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:02 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-06T12:14:02.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:02 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-06T12:14:02.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:02 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-06T12:14:02.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:02 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-06T12:14:02.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:02 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-06T12:14:02.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:02 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-06T12:14:02.851 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:02 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi190.kczxkh/key"}]: dispatch 2024-05-06T12:14:02.851 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:02 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-06T12:14:02.851 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:02 smithi190 ceph-mon[33798]: Manager daemon smithi190.kczxkh is now available 2024-05-06T12:14:02.851 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:02 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:14:02.851 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:02 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:14:02.851 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:02 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi190.kczxkh/mirror_snapshot_schedule"}]: dispatch 2024-05-06T12:14:02.851 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:02 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi190.kczxkh/mirror_snapshot_schedule"}]: dispatch 2024-05-06T12:14:02.851 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:02 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi190.kczxkh/trash_purge_schedule"}]: dispatch 2024-05-06T12:14:02.851 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:02 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi190.kczxkh/trash_purge_schedule"}]: dispatch 2024-05-06T12:14:02.851 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:02 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:02.851 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:02 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:03.876 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:03 smithi092 ceph-mon[26954]: Deploying cephadm binary to smithi092 2024-05-06T12:14:03.876 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:03 smithi092 ceph-mon[26954]: Deploying cephadm binary to smithi190 2024-05-06T12:14:03.876 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:03 smithi092 ceph-mon[26954]: mgrmap e22: smithi190.kczxkh(active, since 1.00945s) 2024-05-06T12:14:03.876 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:03 smithi092 ceph-mon[26954]: pgmap v3: 65 pgs: 65 active+clean; 95 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-05-06T12:14:04.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:03 smithi190 ceph-mon[33798]: Deploying cephadm binary to smithi092 2024-05-06T12:14:04.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:03 smithi190 ceph-mon[33798]: Deploying cephadm binary to smithi190 2024-05-06T12:14:04.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:03 smithi190 ceph-mon[33798]: mgrmap e22: smithi190.kczxkh(active, since 1.00945s) 2024-05-06T12:14:04.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:03 smithi190 ceph-mon[33798]: pgmap v3: 65 pgs: 65 active+clean; 95 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-05-06T12:14:04.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:04 smithi092 ceph-mon[26954]: pgmap v4: 65 pgs: 65 active+clean; 95 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-05-06T12:14:04.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:04 smithi092 ceph-mon[26954]: mgrmap e23: smithi190.kczxkh(active, since 2s) 2024-05-06T12:14:05.033 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:04 smithi190 ceph-mon[33798]: pgmap v4: 65 pgs: 65 active+clean; 95 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-05-06T12:14:05.033 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:04 smithi190 ceph-mon[33798]: mgrmap e23: smithi190.kczxkh(active, since 2s) 2024-05-06T12:14:05.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:05 smithi092 ceph-mon[26954]: [06/May/2024:12:14:04] ENGINE Bus STARTING 2024-05-06T12:14:05.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:05 smithi092 ceph-mon[26954]: [06/May/2024:12:14:04] ENGINE Serving on https://172.21.15.190:7150 2024-05-06T12:14:05.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:05 smithi092 ceph-mon[26954]: [06/May/2024:12:14:04] ENGINE Serving on http://172.21.15.190:8765 2024-05-06T12:14:05.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:05 smithi092 ceph-mon[26954]: [06/May/2024:12:14:04] ENGINE Bus STARTED 2024-05-06T12:14:06.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:05 smithi190 ceph-mon[33798]: [06/May/2024:12:14:04] ENGINE Bus STARTING 2024-05-06T12:14:06.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:05 smithi190 ceph-mon[33798]: [06/May/2024:12:14:04] ENGINE Serving on https://172.21.15.190:7150 2024-05-06T12:14:06.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:05 smithi190 ceph-mon[33798]: [06/May/2024:12:14:04] ENGINE Serving on http://172.21.15.190:8765 2024-05-06T12:14:06.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:05 smithi190 ceph-mon[33798]: [06/May/2024:12:14:04] ENGINE Bus STARTED 2024-05-06T12:14:06.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:06 smithi092 ceph-mon[26954]: pgmap v5: 65 pgs: 65 active+clean; 95 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-05-06T12:14:06.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:06 smithi092 ceph-mon[26954]: mgrmap e24: smithi190.kczxkh(active, since 4s) 2024-05-06T12:14:07.006 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:06 smithi190 ceph-mon[33798]: pgmap v5: 65 pgs: 65 active+clean; 95 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-05-06T12:14:07.006 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:06 smithi190 ceph-mon[33798]: mgrmap e24: smithi190.kczxkh(active, since 4s) 2024-05-06T12:14:08.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:08 smithi092 ceph-mon[26954]: pgmap v6: 65 pgs: 65 active+clean; 116 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 195 op/s 2024-05-06T12:14:09.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:08 smithi190 ceph-mon[33798]: pgmap v6: 65 pgs: 65 active+clean; 116 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 195 op/s 2024-05-06T12:14:09.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:09 smithi092 ceph-mon[26954]: Standby manager daemon smithi092.gyzeqb started 2024-05-06T12:14:09.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:09 smithi092 ceph-mon[26954]: from='mgr.? 172.21.15.92:0/248688558' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi092.gyzeqb/crt"}]: dispatch 2024-05-06T12:14:09.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:09 smithi092 ceph-mon[26954]: from='mgr.? 172.21.15.92:0/248688558' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-06T12:14:09.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:09 smithi092 ceph-mon[26954]: from='mgr.? 172.21.15.92:0/248688558' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi092.gyzeqb/key"}]: dispatch 2024-05-06T12:14:09.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:09 smithi092 ceph-mon[26954]: from='mgr.? 172.21.15.92:0/248688558' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-06T12:14:10.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:09 smithi190 ceph-mon[33798]: Standby manager daemon smithi092.gyzeqb started 2024-05-06T12:14:10.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:09 smithi190 ceph-mon[33798]: from='mgr.? 172.21.15.92:0/248688558' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi092.gyzeqb/crt"}]: dispatch 2024-05-06T12:14:10.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:09 smithi190 ceph-mon[33798]: from='mgr.? 172.21.15.92:0/248688558' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-06T12:14:10.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:09 smithi190 ceph-mon[33798]: from='mgr.? 172.21.15.92:0/248688558' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi092.gyzeqb/key"}]: dispatch 2024-05-06T12:14:10.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:09 smithi190 ceph-mon[33798]: from='mgr.? 172.21.15.92:0/248688558' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-06T12:14:10.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:10 smithi092 ceph-mon[26954]: pgmap v7: 65 pgs: 65 active+clean; 124 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 254 op/s 2024-05-06T12:14:10.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:10 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "mgr metadata", "who": "smithi092.gyzeqb", "id": "smithi092.gyzeqb"}]: dispatch 2024-05-06T12:14:10.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:10 smithi092 ceph-mon[26954]: mgrmap e25: smithi190.kczxkh(active, since 8s), standbys: smithi092.gyzeqb 2024-05-06T12:14:11.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:10 smithi190 ceph-mon[33798]: pgmap v7: 65 pgs: 65 active+clean; 124 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 254 op/s 2024-05-06T12:14:11.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:10 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "mgr metadata", "who": "smithi092.gyzeqb", "id": "smithi092.gyzeqb"}]: dispatch 2024-05-06T12:14:11.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:10 smithi190 ceph-mon[33798]: mgrmap e25: smithi190.kczxkh(active, since 8s), standbys: smithi092.gyzeqb 2024-05-06T12:14:12.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:12 smithi190 ceph-mon[33798]: pgmap v8: 65 pgs: 65 active+clean; 124 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 197 op/s 2024-05-06T12:14:12.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:12 smithi092 ceph-mon[26954]: pgmap v8: 65 pgs: 65 active+clean; 124 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 197 op/s 2024-05-06T12:14:14.888 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:14 smithi092 ceph-mon[26954]: pgmap v9: 65 pgs: 65 active+clean; 144 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s rd, 4.6 MiB/s wr, 297 op/s 2024-05-06T12:14:15.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:14 smithi190 ceph-mon[33798]: pgmap v9: 65 pgs: 65 active+clean; 144 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s rd, 4.6 MiB/s wr, 297 op/s 2024-05-06T12:14:16.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:16 smithi092 ceph-mon[26954]: pgmap v10: 65 pgs: 65 active+clean; 144 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 4.2 MiB/s wr, 272 op/s 2024-05-06T12:14:17.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:16 smithi190 ceph-mon[33798]: pgmap v10: 65 pgs: 65 active+clean; 144 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 4.2 MiB/s wr, 272 op/s 2024-05-06T12:14:17.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:17 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:14:18.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:17 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:14:18.894 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:18 smithi190 ceph-mon[33798]: pgmap v11: 65 pgs: 65 active+clean; 155 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 362 op/s 2024-05-06T12:14:18.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:18 smithi092 ceph-mon[26954]: pgmap v11: 65 pgs: 65 active+clean; 155 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 362 op/s 2024-05-06T12:14:20.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:20 smithi092 ceph-mon[26954]: pgmap v12: 65 pgs: 65 active+clean; 167 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s rd, 4.4 MiB/s wr, 357 op/s 2024-05-06T12:14:20.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:20 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:20.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:20 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:21.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:20 smithi190 ceph-mon[33798]: pgmap v12: 65 pgs: 65 active+clean; 167 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s rd, 4.4 MiB/s wr, 357 op/s 2024-05-06T12:14:21.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:20 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:21.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:20 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:21.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.483+0000 7f260a7f2700 1 -- 172.21.15.92:0/1075198389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26040fcf70 msgr2=0x7f26040fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:21.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.483+0000 7f260a7f2700 1 --2- 172.21.15.92:0/1075198389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26040fcf70 0x7f26040fd3e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f25f8004880 tx=0x7f25f8009e30 comp rx=0 tx=0).stop 2024-05-06T12:14:21.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.483+0000 7f260a7f2700 1 -- 172.21.15.92:0/1075198389 shutdown_connections 2024-05-06T12:14:21.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.483+0000 7f260a7f2700 1 --2- 172.21.15.92:0/1075198389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26040fcf70 0x7f26040fd3e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:21.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.483+0000 7f260a7f2700 1 --2- 172.21.15.92:0/1075198389 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26040fbcc0 0x7f26040fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:21.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.483+0000 7f260a7f2700 1 -- 172.21.15.92:0/1075198389 >> 172.21.15.92:0/1075198389 conn(0x7f26040f7440 msgr2=0x7f26040f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:14:21.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.483+0000 7f260a7f2700 1 -- 172.21.15.92:0/1075198389 shutdown_connections 2024-05-06T12:14:21.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.483+0000 7f260a7f2700 1 -- 172.21.15.92:0/1075198389 wait complete. 2024-05-06T12:14:21.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.484+0000 7f260a7f2700 1 Processor -- start 2024-05-06T12:14:21.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.484+0000 7f260a7f2700 1 -- start start 2024-05-06T12:14:21.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.485+0000 7f260a7f2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26040fbcc0 0x7f260410b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:21.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.485+0000 7f260a7f2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26040fcf70 0x7f260410bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:21.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.485+0000 7f260a7f2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260410c280 con 0x7f26040fcf70 2024-05-06T12:14:21.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.485+0000 7f260a7f2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260410c3f0 con 0x7f26040fbcc0 2024-05-06T12:14:21.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.485+0000 7f2603fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26040fbcc0 0x7f260410b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:21.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.485+0000 7f2603fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26040fbcc0 0x7f260410b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.92:56400/0 (socket says 172.21.15.92:56400) 2024-05-06T12:14:21.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.485+0000 7f2603fff700 1 -- 172.21.15.92:0/1532677747 learned_addr learned my addr 172.21.15.92:0/1532677747 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:14:21.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.485+0000 7f26037fe700 1 --2- 172.21.15.92:0/1532677747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26040fcf70 0x7f260410bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:21.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.486+0000 7f2603fff700 1 -- 172.21.15.92:0/1532677747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26040fcf70 msgr2=0x7f260410bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:21.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.486+0000 7f2603fff700 1 --2- 172.21.15.92:0/1532677747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26040fcf70 0x7f260410bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:21.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.486+0000 7f2603fff700 1 -- 172.21.15.92:0/1532677747 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f8005040 con 0x7f26040fbcc0 2024-05-06T12:14:21.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.486+0000 7f26037fe700 1 --2- 172.21.15.92:0/1532677747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26040fcf70 0x7f260410bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-06T12:14:21.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.486+0000 7f2603fff700 1 --2- 172.21.15.92:0/1532677747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26040fbcc0 0x7f260410b740 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f25f400e190 tx=0x7f25f400d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:14:21.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.486+0000 7f2600ff9700 1 -- 172.21.15.92:0/1532677747 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25f400ee30 con 0x7f26040fbcc0 2024-05-06T12:14:21.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.486+0000 7f2600ff9700 1 -- 172.21.15.92:0/1532677747 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f25f4011040 con 0x7f26040fbcc0 2024-05-06T12:14:21.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.486+0000 7f2600ff9700 1 -- 172.21.15.92:0/1532677747 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25f400b600 con 0x7f26040fbcc0 2024-05-06T12:14:21.491 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.486+0000 7f260a7f2700 1 -- 172.21.15.92:0/1532677747 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2604110df0 con 0x7f26040fbcc0 2024-05-06T12:14:21.491 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.486+0000 7f260a7f2700 1 -- 172.21.15.92:0/1532677747 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2604111220 con 0x7f26040fbcc0 2024-05-06T12:14:21.491 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.488+0000 7f260a7f2700 1 -- 172.21.15.92:0/1532677747 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f260404c710 con 0x7f26040fbcc0 2024-05-06T12:14:21.491 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.488+0000 7f2600ff9700 1 -- 172.21.15.92:0/1532677747 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7f25f4016480 con 0x7f26040fbcc0 2024-05-06T12:14:21.492 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.489+0000 7f2600ff9700 1 --2- 172.21.15.92:0/1532677747 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f25ec06b7a0 0x7f25ec06dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:21.492 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.489+0000 7f2600ff9700 1 -- 172.21.15.92:0/1532677747 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f25f408ca80 con 0x7f26040fbcc0 2024-05-06T12:14:21.492 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.489+0000 7f26037fe700 1 --2- 172.21.15.92:0/1532677747 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f25ec06b7a0 0x7f25ec06dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:21.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.489+0000 7f26037fe700 1 --2- 172.21.15.92:0/1532677747 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f25ec06b7a0 0x7f25ec06dc50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f25f8009fd0 tx=0x7f25f8009e30 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:14:21.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.495+0000 7f2600ff9700 1 -- 172.21.15.92:0/1532677747 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f25f405a0f0 con 0x7f26040fbcc0 2024-05-06T12:14:21.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.788+0000 7f260a7f2700 1 -- 172.21.15.92:0/1532677747 --> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2604101330 con 0x7f25ec06b7a0 2024-05-06T12:14:21.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.789+0000 7f2600ff9700 1 -- 172.21.15.92:0/1532677747 <== mgr.14636 v2:172.21.15.190:6828/3484477933 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f2604101330 con 0x7f25ec06b7a0 2024-05-06T12:14:21.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.792+0000 7f260a7f2700 1 -- 172.21.15.92:0/1532677747 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f25ec06b7a0 msgr2=0x7f25ec06dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:21.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.792+0000 7f260a7f2700 1 --2- 172.21.15.92:0/1532677747 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f25ec06b7a0 0x7f25ec06dc50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f25f8009fd0 tx=0x7f25f8009e30 comp rx=0 tx=0).stop 2024-05-06T12:14:21.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.792+0000 7f260a7f2700 1 -- 172.21.15.92:0/1532677747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26040fbcc0 msgr2=0x7f260410b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:21.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.792+0000 7f260a7f2700 1 --2- 172.21.15.92:0/1532677747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26040fbcc0 0x7f260410b740 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f25f400e190 tx=0x7f25f400d040 comp rx=0 tx=0).stop 2024-05-06T12:14:21.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.792+0000 7f260a7f2700 1 -- 172.21.15.92:0/1532677747 shutdown_connections 2024-05-06T12:14:21.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.792+0000 7f260a7f2700 1 --2- 172.21.15.92:0/1532677747 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f25ec06b7a0 0x7f25ec06dc50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:21.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.792+0000 7f260a7f2700 1 --2- 172.21.15.92:0/1532677747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26040fbcc0 0x7f260410b740 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:21.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.792+0000 7f260a7f2700 1 --2- 172.21.15.92:0/1532677747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26040fcf70 0x7f260410bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:21.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.792+0000 7f260a7f2700 1 -- 172.21.15.92:0/1532677747 >> 172.21.15.92:0/1532677747 conn(0x7f26040f7440 msgr2=0x7f26040f8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:14:21.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.793+0000 7f260a7f2700 1 -- 172.21.15.92:0/1532677747 shutdown_connections 2024-05-06T12:14:21.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.793+0000 7f260a7f2700 1 -- 172.21.15.92:0/1532677747 wait complete. 2024-05-06T12:14:21.809 INFO:teuthology.orchestra.run.smithi092.stdout:true 2024-05-06T12:14:21.859 INFO:tasks.workunit.client.0.smithi092.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-05-06T12:14:21.867 INFO:tasks.workunit.client.0.smithi092.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-05-06T12:14:21.867 INFO:tasks.workunit.client.0.smithi092.stderr:+ make 2024-05-06T12:14:21.904 INFO:tasks.workunit.client.0.smithi092.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-05-06T12:14:21.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.934+0000 7fe7d01fa700 1 -- 172.21.15.92:0/4001758495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7c80fbcb0 msgr2=0x7fe7c80fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:21.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.934+0000 7fe7d01fa700 1 --2- 172.21.15.92:0/4001758495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7c80fbcb0 0x7fe7c80fc0c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4004940 tx=0x7fe7c4009e30 comp rx=0 tx=0).stop 2024-05-06T12:14:21.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.934+0000 7fe7d01fa700 1 -- 172.21.15.92:0/4001758495 shutdown_connections 2024-05-06T12:14:21.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.934+0000 7fe7d01fa700 1 --2- 172.21.15.92:0/4001758495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7c80fcf60 0x7fe7c80fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:21.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.934+0000 7fe7d01fa700 1 --2- 172.21.15.92:0/4001758495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7c80fbcb0 0x7fe7c80fc0c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:21.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.934+0000 7fe7d01fa700 1 -- 172.21.15.92:0/4001758495 >> 172.21.15.92:0/4001758495 conn(0x7fe7c80f7430 msgr2=0x7fe7c80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:14:21.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.934+0000 7fe7d01fa700 1 -- 172.21.15.92:0/4001758495 shutdown_connections 2024-05-06T12:14:21.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.935+0000 7fe7d01fa700 1 -- 172.21.15.92:0/4001758495 wait complete. 2024-05-06T12:14:21.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.935+0000 7fe7d01fa700 1 Processor -- start 2024-05-06T12:14:21.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.935+0000 7fe7d01fa700 1 -- start start 2024-05-06T12:14:21.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.936+0000 7fe7d01fa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7c80fbcb0 0x7fe7c8112ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:21.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.936+0000 7fe7d01fa700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7c80fcf60 0x7fe7c8115030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:21.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.936+0000 7fe7d01fa700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c80661d0 con 0x7fe7c80fbcb0 2024-05-06T12:14:21.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.936+0000 7fe7cdf96700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7c80fbcb0 0x7fe7c8112ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:21.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.936+0000 7fe7cdf96700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7c80fbcb0 0x7fe7c8112ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46094/0 (socket says 172.21.15.92:46094) 2024-05-06T12:14:21.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.936+0000 7fe7cdf96700 1 -- 172.21.15.92:0/3542333818 learned_addr learned my addr 172.21.15.92:0/3542333818 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:14:21.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.936+0000 7fe7d01fa700 1 -- 172.21.15.92:0/3542333818 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c8066310 con 0x7fe7c80fcf60 2024-05-06T12:14:21.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.936+0000 7fe7cd795700 1 --2- 172.21.15.92:0/3542333818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7c80fcf60 0x7fe7c8115030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:21.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.936+0000 7fe7cd795700 1 -- 172.21.15.92:0/3542333818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7c80fbcb0 msgr2=0x7fe7c8112ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:21.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.936+0000 7fe7cd795700 1 --2- 172.21.15.92:0/3542333818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7c80fbcb0 0x7fe7c8112ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:21.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.936+0000 7fe7cd795700 1 -- 172.21.15.92:0/3542333818 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c4005040 con 0x7fe7c80fcf60 2024-05-06T12:14:21.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.937+0000 7fe7cdf96700 1 --2- 172.21.15.92:0/3542333818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7c80fbcb0 0x7fe7c8112ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-06T12:14:21.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.937+0000 7fe7cd795700 1 --2- 172.21.15.92:0/3542333818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7c80fcf60 0x7fe7c8115030 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009fd0 tx=0x7fe7b800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:14:21.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.937+0000 7fe7be7fc700 1 -- 172.21.15.92:0/3542333818 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7b8004e90 con 0x7fe7c80fcf60 2024-05-06T12:14:21.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.937+0000 7fe7be7fc700 1 -- 172.21.15.92:0/3542333818 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe7b800f040 con 0x7fe7c80fcf60 2024-05-06T12:14:21.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.937+0000 7fe7d01fa700 1 -- 172.21.15.92:0/3542333818 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7c8115570 con 0x7fe7c80fcf60 2024-05-06T12:14:21.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.937+0000 7fe7d01fa700 1 -- 172.21.15.92:0/3542333818 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c81158c0 con 0x7fe7c80fcf60 2024-05-06T12:14:21.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.938+0000 7fe7be7fc700 1 -- 172.21.15.92:0/3542333818 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7b800ca50 con 0x7fe7c80fcf60 2024-05-06T12:14:21.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.939+0000 7fe7d01fa700 1 -- 172.21.15.92:0/3542333818 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7c8104fe0 con 0x7fe7c80fcf60 2024-05-06T12:14:21.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.940+0000 7fe7be7fc700 1 -- 172.21.15.92:0/3542333818 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7fe7b800cbb0 con 0x7fe7c80fcf60 2024-05-06T12:14:21.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.940+0000 7fe7be7fc700 1 --2- 172.21.15.92:0/3542333818 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fe7b406b860 0x7fe7b406dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:21.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.941+0000 7fe7cdf96700 1 --2- 172.21.15.92:0/3542333818 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fe7b406b860 0x7fe7b406dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:21.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.941+0000 7fe7be7fc700 1 -- 172.21.15.92:0/3542333818 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fe7b8089e10 con 0x7fe7c80fcf60 2024-05-06T12:14:21.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.941+0000 7fe7cdf96700 1 --2- 172.21.15.92:0/3542333818 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fe7b406b860 0x7fe7b406dd10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4013740 tx=0x7fe7c4009e30 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:14:21.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:21.949+0000 7fe7be7fc700 1 -- 172.21.15.92:0/3542333818 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fe7b8050100 con 0x7fe7c80fcf60 2024-05-06T12:14:22.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.196+0000 7fe7d01fa700 1 -- 172.21.15.92:0/3542333818 --> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7c8101250 con 0x7fe7b406b860 2024-05-06T12:14:22.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.198+0000 7fe7be7fc700 1 -- 172.21.15.92:0/3542333818 <== mgr.14636 v2:172.21.15.190:6828/3484477933 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fe7c8101250 con 0x7fe7b406b860 2024-05-06T12:14:22.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.200+0000 7fe7d01fa700 1 -- 172.21.15.92:0/3542333818 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fe7b406b860 msgr2=0x7fe7b406dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:22.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.200+0000 7fe7d01fa700 1 --2- 172.21.15.92:0/3542333818 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fe7b406b860 0x7fe7b406dd10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4013740 tx=0x7fe7c4009e30 comp rx=0 tx=0).stop 2024-05-06T12:14:22.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.200+0000 7fe7d01fa700 1 -- 172.21.15.92:0/3542333818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7c80fcf60 msgr2=0x7fe7c8115030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:22.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.200+0000 7fe7d01fa700 1 --2- 172.21.15.92:0/3542333818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7c80fcf60 0x7fe7c8115030 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009fd0 tx=0x7fe7b800b040 comp rx=0 tx=0).stop 2024-05-06T12:14:22.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.200+0000 7fe7d01fa700 1 -- 172.21.15.92:0/3542333818 shutdown_connections 2024-05-06T12:14:22.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.200+0000 7fe7d01fa700 1 --2- 172.21.15.92:0/3542333818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7c80fbcb0 0x7fe7c8112ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:22.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.200+0000 7fe7d01fa700 1 --2- 172.21.15.92:0/3542333818 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fe7b406b860 0x7fe7b406dd10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:22.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.200+0000 7fe7d01fa700 1 --2- 172.21.15.92:0/3542333818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7c80fcf60 0x7fe7c8115030 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:22.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.200+0000 7fe7d01fa700 1 -- 172.21.15.92:0/3542333818 >> 172.21.15.92:0/3542333818 conn(0x7fe7c80f7430 msgr2=0x7fe7c80f8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:14:22.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.200+0000 7fe7d01fa700 1 -- 172.21.15.92:0/3542333818 shutdown_connections 2024-05-06T12:14:22.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.200+0000 7fe7d01fa700 1 -- 172.21.15.92:0/3542333818 wait complete. 2024-05-06T12:14:22.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.301+0000 7fa04d7d7700 1 -- 172.21.15.92:0/4163752433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0480fbcb0 msgr2=0x7fa0480fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:22.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.301+0000 7fa04d7d7700 1 --2- 172.21.15.92:0/4163752433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0480fbcb0 0x7fa0480fc0c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa038004940 tx=0x7fa038009e30 comp rx=0 tx=0).stop 2024-05-06T12:14:22.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.301+0000 7fa04d7d7700 1 -- 172.21.15.92:0/4163752433 shutdown_connections 2024-05-06T12:14:22.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.301+0000 7fa04d7d7700 1 --2- 172.21.15.92:0/4163752433 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0480fcf60 0x7fa0480fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:22.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.301+0000 7fa04d7d7700 1 --2- 172.21.15.92:0/4163752433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0480fbcb0 0x7fa0480fc0c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:22.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.301+0000 7fa04d7d7700 1 -- 172.21.15.92:0/4163752433 >> 172.21.15.92:0/4163752433 conn(0x7fa0480f7430 msgr2=0x7fa0480f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:14:22.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.301+0000 7fa04d7d7700 1 -- 172.21.15.92:0/4163752433 shutdown_connections 2024-05-06T12:14:22.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.301+0000 7fa04d7d7700 1 -- 172.21.15.92:0/4163752433 wait complete. 2024-05-06T12:14:22.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.301+0000 7fa04d7d7700 1 Processor -- start 2024-05-06T12:14:22.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.301+0000 7fa04d7d7700 1 -- start start 2024-05-06T12:14:22.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.301+0000 7fa04d7d7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0480fbcb0 0x7fa04810da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:22.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.301+0000 7fa04d7d7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0480fcf60 0x7fa04810dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:22.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.301+0000 7fa04d7d7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04810e5a0 con 0x7fa0480fcf60 2024-05-06T12:14:22.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.301+0000 7fa04d7d7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04810ed50 con 0x7fa0480fbcb0 2024-05-06T12:14:22.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.302+0000 7fa0467fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0480fcf60 0x7fa04810dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:22.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.302+0000 7fa0467fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0480fcf60 0x7fa04810dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46118/0 (socket says 172.21.15.92:46118) 2024-05-06T12:14:22.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.302+0000 7fa0467fc700 1 -- 172.21.15.92:0/881547492 learned_addr learned my addr 172.21.15.92:0/881547492 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:14:22.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.302+0000 7fa046ffd700 1 --2- 172.21.15.92:0/881547492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0480fbcb0 0x7fa04810da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:22.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.302+0000 7fa0467fc700 1 -- 172.21.15.92:0/881547492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0480fbcb0 msgr2=0x7fa04810da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:22.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.302+0000 7fa0467fc700 1 --2- 172.21.15.92:0/881547492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0480fbcb0 0x7fa04810da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:22.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.302+0000 7fa0467fc700 1 -- 172.21.15.92:0/881547492 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa038005040 con 0x7fa0480fcf60 2024-05-06T12:14:22.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.302+0000 7fa0467fc700 1 --2- 172.21.15.92:0/881547492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0480fcf60 0x7fa04810dfa0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa03c00eb40 tx=0x7fa03c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:14:22.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.302+0000 7fa02f7fe700 1 -- 172.21.15.92:0/881547492 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa03c007cc0 con 0x7fa0480fcf60 2024-05-06T12:14:22.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.302+0000 7fa02f7fe700 1 -- 172.21.15.92:0/881547492 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa03c00f040 con 0x7fa0480fcf60 2024-05-06T12:14:22.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.302+0000 7fa02f7fe700 1 -- 172.21.15.92:0/881547492 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa03c00ca10 con 0x7fa0480fcf60 2024-05-06T12:14:22.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.302+0000 7fa04d7d7700 1 -- 172.21.15.92:0/881547492 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa04810efd0 con 0x7fa0480fcf60 2024-05-06T12:14:22.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.302+0000 7fa04d7d7700 1 -- 172.21.15.92:0/881547492 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa04810f510 con 0x7fa0480fcf60 2024-05-06T12:14:22.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.303+0000 7fa04d7d7700 1 -- 172.21.15.92:0/881547492 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa048101af0 con 0x7fa0480fcf60 2024-05-06T12:14:22.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.304+0000 7fa02f7fe700 1 -- 172.21.15.92:0/881547492 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7fa03c00cb70 con 0x7fa0480fcf60 2024-05-06T12:14:22.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.304+0000 7fa02f7fe700 1 --2- 172.21.15.92:0/881547492 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fa03006b790 0x7fa03006dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:22.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.304+0000 7fa02f7fe700 1 -- 172.21.15.92:0/881547492 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fa03c089f90 con 0x7fa0480fcf60 2024-05-06T12:14:22.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.305+0000 7fa046ffd700 1 --2- 172.21.15.92:0/881547492 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fa03006b790 0x7fa03006dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:22.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.305+0000 7fa046ffd700 1 --2- 172.21.15.92:0/881547492 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fa03006b790 0x7fa03006dc40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa038009b30 tx=0x7fa038009e30 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:14:22.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.309+0000 7fa02f7fe700 1 -- 172.21.15.92:0/881547492 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fa03c050100 con 0x7fa0480fcf60 2024-05-06T12:14:22.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.524+0000 7fa04d7d7700 1 -- 172.21.15.92:0/881547492 --> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0480008d0 con 0x7fa03006b790 2024-05-06T12:14:22.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.538+0000 7fa02f7fe700 1 -- 172.21.15.92:0/881547492 <== mgr.14636 v2:172.21.15.190:6828/3484477933 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7fa0480008d0 con 0x7fa03006b790 2024-05-06T12:14:22.541 INFO:teuthology.orchestra.run.smithi092.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-06T12:14:22.541 INFO:teuthology.orchestra.run.smithi092.stdout:alertmanager.smithi092 smithi092 *:9093,9094 running (6m) 1s ago 7m 25.6M - 0.25.0 c8568f914cd2 dab380a7352e 2024-05-06T12:14:22.541 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi092 smithi092 running (8m) 1s ago 8m 6660k - 17.2.7-925-ge81df420 96d5ae7fd4b4 95dc7e14bd9a 2024-05-06T12:14:22.541 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi190 smithi190 running (7m) 26s ago 7m 6665k - 17.2.7-925-ge81df420 96d5ae7fd4b4 6cf97785406f 2024-05-06T12:14:22.541 INFO:teuthology.orchestra.run.smithi092.stdout:grafana.smithi092 smithi092 *:3000 running (6m) 1s ago 7m 85.5M - 9.4.7 954c08fa6188 841f34687c12 2024-05-06T12:14:22.541 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.ijsqgp smithi092 running (2m) 1s ago 2m 79.3M - 17.2.7-925-ge81df420 96d5ae7fd4b4 b4c2ed6d2c82 2024-05-06T12:14:22.541 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.rosbll smithi092 running (2m) 1s ago 2m 101M - 17.2.7-925-ge81df420 96d5ae7fd4b4 5f5426cd1bf5 2024-05-06T12:14:22.541 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.byjvgf smithi190 running (2m) 26s ago 2m 38.6M - 17.2.7-925-ge81df420 96d5ae7fd4b4 24581d2e1c5d 2024-05-06T12:14:22.541 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.idzoff smithi190 running (2m) 26s ago 2m 28.8M - 17.2.7-925-ge81df420 96d5ae7fd4b4 20f2641696ef 2024-05-06T12:14:22.541 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi092.gyzeqb smithi092 *:9283 running (9m) 1s ago 9m 419M - 17.2.7-925-ge81df420 96d5ae7fd4b4 820ba9ba8a82 2024-05-06T12:14:22.542 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi190.kczxkh smithi190 *:8443,9283 running (29s) 26s ago 7m 44.7M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 efed230a6048 2024-05-06T12:14:22.542 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi092 smithi092 running (9m) 1s ago 9m 46.6M 2048M 17.2.7-925-ge81df420 96d5ae7fd4b4 df5b322e9719 2024-05-06T12:14:22.542 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi190 smithi190 running (6m) 26s ago 6m 34.1M 2048M 17.2.7-925-ge81df420 96d5ae7fd4b4 912ded072601 2024-05-06T12:14:22.542 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi092 smithi092 *:9100 running (8m) 1s ago 8m 16.2M - 1.5.0 0da6a335fe13 86cf73a245ab 2024-05-06T12:14:22.542 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi190 smithi190 *:9100 running (7m) 26s ago 7m 16.0M - 1.5.0 0da6a335fe13 da5c2a030c69 2024-05-06T12:14:22.542 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 smithi092 running (6m) 1s ago 6m 77.5M 1557M 17.2.7-925-ge81df420 96d5ae7fd4b4 ce84e7af9084 2024-05-06T12:14:22.542 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 smithi092 running (5m) 1s ago 5m 81.5M 1557M 17.2.7-925-ge81df420 96d5ae7fd4b4 8ab870f57eb2 2024-05-06T12:14:22.542 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 smithi092 running (5m) 1s ago 5m 67.4M 1557M 17.2.7-925-ge81df420 96d5ae7fd4b4 1e6ed04a0688 2024-05-06T12:14:22.542 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 smithi190 running (4m) 26s ago 4m 61.5M 5312M 17.2.7-925-ge81df420 96d5ae7fd4b4 52f9521286d5 2024-05-06T12:14:22.542 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 smithi190 running (4m) 26s ago 4m 59.5M 5312M 17.2.7-925-ge81df420 96d5ae7fd4b4 3a7da09e4c48 2024-05-06T12:14:22.542 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 smithi190 running (4m) 26s ago 4m 58.8M 5312M 17.2.7-925-ge81df420 96d5ae7fd4b4 e7452c6b5cec 2024-05-06T12:14:22.542 INFO:teuthology.orchestra.run.smithi092.stdout:prometheus.smithi092 smithi092 *:9095 running (6m) 1s ago 7m 58.8M - 2.43.0 a07b618ecd1d ef92f3572328 2024-05-06T12:14:22.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.540+0000 7fa04d7d7700 1 -- 172.21.15.92:0/881547492 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fa03006b790 msgr2=0x7fa03006dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:22.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.540+0000 7fa04d7d7700 1 --2- 172.21.15.92:0/881547492 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fa03006b790 0x7fa03006dc40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa038009b30 tx=0x7fa038009e30 comp rx=0 tx=0).stop 2024-05-06T12:14:22.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.540+0000 7fa04d7d7700 1 -- 172.21.15.92:0/881547492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0480fcf60 msgr2=0x7fa04810dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:22.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.540+0000 7fa04d7d7700 1 --2- 172.21.15.92:0/881547492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0480fcf60 0x7fa04810dfa0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa03c00eb40 tx=0x7fa03c00b040 comp rx=0 tx=0).stop 2024-05-06T12:14:22.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.540+0000 7fa04d7d7700 1 -- 172.21.15.92:0/881547492 shutdown_connections 2024-05-06T12:14:22.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.540+0000 7fa04d7d7700 1 --2- 172.21.15.92:0/881547492 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fa03006b790 0x7fa03006dc40 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:22.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.540+0000 7fa04d7d7700 1 --2- 172.21.15.92:0/881547492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0480fbcb0 0x7fa04810da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:22.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.540+0000 7fa04d7d7700 1 --2- 172.21.15.92:0/881547492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0480fcf60 0x7fa04810dfa0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:22.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.540+0000 7fa04d7d7700 1 -- 172.21.15.92:0/881547492 >> 172.21.15.92:0/881547492 conn(0x7fa0480f7430 msgr2=0x7fa048065b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:14:22.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.540+0000 7fa04d7d7700 1 -- 172.21.15.92:0/881547492 shutdown_connections 2024-05-06T12:14:22.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.540+0000 7fa04d7d7700 1 -- 172.21.15.92:0/881547492 wait complete. 2024-05-06T12:14:22.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.649+0000 7fa4b0b91700 1 -- 172.21.15.92:0/1867421471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4ac0fbcb0 msgr2=0x7fa4ac0fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:22.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.649+0000 7fa4b0b91700 1 --2- 172.21.15.92:0/1867421471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4ac0fbcb0 0x7fa4ac0fc0c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa49c004940 tx=0x7fa49c009e30 comp rx=0 tx=0).stop 2024-05-06T12:14:22.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.649+0000 7fa4b0b91700 1 -- 172.21.15.92:0/1867421471 shutdown_connections 2024-05-06T12:14:22.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.649+0000 7fa4b0b91700 1 --2- 172.21.15.92:0/1867421471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4ac0fcf60 0x7fa4ac0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:22.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.649+0000 7fa4b0b91700 1 --2- 172.21.15.92:0/1867421471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4ac0fbcb0 0x7fa4ac0fc0c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:22.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.649+0000 7fa4b0b91700 1 -- 172.21.15.92:0/1867421471 >> 172.21.15.92:0/1867421471 conn(0x7fa4ac0f7430 msgr2=0x7fa4ac0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:14:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.649+0000 7fa4b0b91700 1 -- 172.21.15.92:0/1867421471 shutdown_connections 2024-05-06T12:14:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.649+0000 7fa4b0b91700 1 -- 172.21.15.92:0/1867421471 wait complete. 2024-05-06T12:14:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.649+0000 7fa4b0b91700 1 Processor -- start 2024-05-06T12:14:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.649+0000 7fa4b0b91700 1 -- start start 2024-05-06T12:14:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.650+0000 7fa4b0b91700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4ac0fbcb0 0x7fa4ac10dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.650+0000 7fa4b0b91700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4ac0fcf60 0x7fa4ac10e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.650+0000 7fa4b0b91700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac10e610 con 0x7fa4ac0fbcb0 2024-05-06T12:14:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.650+0000 7fa4b0b91700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac10edc0 con 0x7fa4ac0fcf60 2024-05-06T12:14:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.650+0000 7fa4aa59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4ac0fbcb0 0x7fa4ac10dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.650+0000 7fa4aa59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4ac0fbcb0 0x7fa4ac10dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50940/0 (socket says 172.21.15.92:50940) 2024-05-06T12:14:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.650+0000 7fa4aa59c700 1 -- 172.21.15.92:0/4203096567 learned_addr learned my addr 172.21.15.92:0/4203096567 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:14:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.650+0000 7fa4a9d9b700 1 --2- 172.21.15.92:0/4203096567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4ac0fcf60 0x7fa4ac10e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:22.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.650+0000 7fa4aa59c700 1 -- 172.21.15.92:0/4203096567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4ac0fcf60 msgr2=0x7fa4ac10e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:22.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.650+0000 7fa4aa59c700 1 --2- 172.21.15.92:0/4203096567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4ac0fcf60 0x7fa4ac10e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:22.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.650+0000 7fa4aa59c700 1 -- 172.21.15.92:0/4203096567 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49c005040 con 0x7fa4ac0fbcb0 2024-05-06T12:14:22.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.650+0000 7fa4aa59c700 1 --2- 172.21.15.92:0/4203096567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4ac0fbcb0 0x7fa4ac10dad0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00b3b0 tx=0x7fa49c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:14:22.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.650+0000 7fa49affd700 1 -- 172.21.15.92:0/4203096567 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa49c00e040 con 0x7fa4ac0fbcb0 2024-05-06T12:14:22.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.650+0000 7fa49affd700 1 -- 172.21.15.92:0/4203096567 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa49c00bbf0 con 0x7fa4ac0fbcb0 2024-05-06T12:14:22.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.650+0000 7fa4b0b91700 1 -- 172.21.15.92:0/4203096567 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4ac10f040 con 0x7fa4ac0fbcb0 2024-05-06T12:14:22.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.651+0000 7fa49affd700 1 -- 172.21.15.92:0/4203096567 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa49c01c630 con 0x7fa4ac0fbcb0 2024-05-06T12:14:22.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.651+0000 7fa4b0b91700 1 -- 172.21.15.92:0/4203096567 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4ac10f580 con 0x7fa4ac0fbcb0 2024-05-06T12:14:22.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.652+0000 7fa4b0b91700 1 -- 172.21.15.92:0/4203096567 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4ac05f5c0 con 0x7fa4ac0fbcb0 2024-05-06T12:14:22.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.652+0000 7fa49affd700 1 -- 172.21.15.92:0/4203096567 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7fa49c00bd60 con 0x7fa4ac0fbcb0 2024-05-06T12:14:22.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.652+0000 7fa49affd700 1 --2- 172.21.15.92:0/4203096567 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fa49406b790 0x7fa49406dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:22.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.653+0000 7fa49affd700 1 -- 172.21.15.92:0/4203096567 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fa49c08aef0 con 0x7fa4ac0fbcb0 2024-05-06T12:14:22.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.653+0000 7fa4a9d9b700 1 --2- 172.21.15.92:0/4203096567 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fa49406b790 0x7fa49406dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:22.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.653+0000 7fa4a9d9b700 1 --2- 172.21.15.92:0/4203096567 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fa49406b790 0x7fa49406dc40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0004f60 tx=0x7fa4a000b040 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:14:22.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:22.657+0000 7fa49affd700 1 -- 172.21.15.92:0/4203096567 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fa49c058740 con 0x7fa4ac0fbcb0 2024-05-06T12:14:22.854 INFO:tasks.workunit.client.0.smithi092.stderr:++ readlink -f fsstress 2024-05-06T12:14:22.856 INFO:tasks.workunit.client.0.smithi092.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-05-06T12:14:22.856 INFO:tasks.workunit.client.0.smithi092.stderr:+ popd 2024-05-06T12:14:22.857 INFO:tasks.workunit.client.0.smithi092.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-05-06T12:14:22.857 INFO:tasks.workunit.client.0.smithi092.stderr:+ popd 2024-05-06T12:14:22.857 INFO:tasks.workunit.client.0.smithi092.stdout:~/cephtest/mnt.0/client.0/tmp 2024-05-06T12:14:22.857 INFO:tasks.workunit.client.0.smithi092.stderr:++ mktemp -d -p . 2024-05-06T12:14:22.860 INFO:tasks.workunit.client.0.smithi092.stderr:+ T=./tmp.yrc4lArKdJ 2024-05-06T12:14:22.861 INFO:tasks.workunit.client.0.smithi092.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.yrc4lArKdJ -l 1 -n 1000 -p 10 -v 2024-05-06T12:14:22.864 INFO:tasks.workunit.client.0.smithi092.stdout:seed = 1714808253 2024-05-06T12:14:22.869 INFO:tasks.workunit.client.0.smithi092.stdout:1/0: dwrite - no filename 2024-05-06T12:14:22.869 INFO:tasks.workunit.client.0.smithi092.stdout:1/1: truncate - no filename 2024-05-06T12:14:22.869 INFO:tasks.workunit.client.0.smithi092.stdout:1/2: truncate - no filename 2024-05-06T12:14:22.869 INFO:tasks.workunit.client.0.smithi092.stdout:1/3: stat - no entries 2024-05-06T12:14:22.869 INFO:tasks.workunit.client.0.smithi092.stdout:1/4: readlink - no filename 2024-05-06T12:14:22.870 INFO:tasks.workunit.client.0.smithi092.stdout:0/0: symlink l0 0 2024-05-06T12:14:22.870 INFO:tasks.workunit.client.0.smithi092.stdout:0/1: chown l0 4360377 1 2024-05-06T12:14:22.871 INFO:tasks.workunit.client.0.smithi092.stdout:2/0: rmdir - no directory 2024-05-06T12:14:22.871 INFO:tasks.workunit.client.0.smithi092.stdout:2/1: rename - no filename 2024-05-06T12:14:22.871 INFO:tasks.workunit.client.0.smithi092.stdout:2/2: rename - no filename 2024-05-06T12:14:22.873 INFO:tasks.workunit.client.0.smithi092.stdout:0/2: creat f1 x:0 0 0 2024-05-06T12:14:22.873 INFO:tasks.workunit.client.0.smithi092.stdout:1/5: creat f0 x:0 0 0 2024-05-06T12:14:22.874 INFO:tasks.workunit.client.0.smithi092.stdout:3/0: fdatasync - no filename 2024-05-06T12:14:22.875 INFO:tasks.workunit.client.0.smithi092.stdout:0/3: creat f2 x:0 0 0 2024-05-06T12:14:22.877 INFO:tasks.workunit.client.0.smithi092.stdout:1/6: symlink l1 0 2024-05-06T12:14:22.877 INFO:tasks.workunit.client.0.smithi092.stdout:2/3: mknod c0 0 2024-05-06T12:14:22.877 INFO:tasks.workunit.client.0.smithi092.stdout:2/4: stat c0 0 2024-05-06T12:14:22.878 INFO:tasks.workunit.client.0.smithi092.stdout:2/5: stat c0 0 2024-05-06T12:14:22.878 INFO:tasks.workunit.client.0.smithi092.stdout:2/6: fsync - no filename 2024-05-06T12:14:22.878 INFO:tasks.workunit.client.0.smithi092.stdout:2/7: stat c0 0 2024-05-06T12:14:22.878 INFO:tasks.workunit.client.0.smithi092.stdout:2/8: rmdir - no directory 2024-05-06T12:14:22.878 INFO:tasks.workunit.client.0.smithi092.stdout:2/9: dread - no filename 2024-05-06T12:14:22.878 INFO:tasks.workunit.client.0.smithi092.stdout:2/10: dread - no filename 2024-05-06T12:14:22.879 INFO:tasks.workunit.client.0.smithi092.stdout:4/0: write - no filename 2024-05-06T12:14:22.879 INFO:tasks.workunit.client.0.smithi092.stdout:4/1: dwrite - no filename 2024-05-06T12:14:22.879 INFO:tasks.workunit.client.0.smithi092.stdout:0/4: unlink f2 0 2024-05-06T12:14:22.880 INFO:tasks.workunit.client.0.smithi092.stdout:3/1: creat f0 x:0 0 0 2024-05-06T12:14:22.881 INFO:tasks.workunit.client.0.smithi092.stdout:2/11: creat f1 x:0 0 0 2024-05-06T12:14:22.882 INFO:tasks.workunit.client.0.smithi092.stdout:0/5: chown l0 2212 1 2024-05-06T12:14:22.882 INFO:tasks.workunit.client.0.smithi092.stdout:2/12: dread - f1 zero size 2024-05-06T12:14:22.884 INFO:tasks.workunit.client.0.smithi092.stdout:3/2: chown f0 641 1 2024-05-06T12:14:22.884 INFO:tasks.workunit.client.0.smithi092.stdout:4/2: symlink l0 0 2024-05-06T12:14:22.884 INFO:tasks.workunit.client.0.smithi092.stdout:5/0: truncate - no filename 2024-05-06T12:14:22.886 INFO:tasks.workunit.client.0.smithi092.stdout:2/13: write f1 [787461,16113] 0 2024-05-06T12:14:22.887 INFO:tasks.workunit.client.0.smithi092.stdout:0/6: symlink l3 0 2024-05-06T12:14:22.887 INFO:tasks.workunit.client.0.smithi092.stdout:0/7: rmdir - no directory 2024-05-06T12:14:22.888 INFO:tasks.workunit.client.0.smithi092.stdout:3/3: truncate f0 1002776 0 2024-05-06T12:14:22.889 INFO:tasks.workunit.client.0.smithi092.stdout:0/8: dread - f1 zero size 2024-05-06T12:14:22.890 INFO:tasks.workunit.client.0.smithi092.stdout:4/3: rename l0 to l1 0 2024-05-06T12:14:22.890 INFO:tasks.workunit.client.0.smithi092.stdout:6/0: dwrite - no filename 2024-05-06T12:14:22.890 INFO:tasks.workunit.client.0.smithi092.stdout:6/1: unlink - no file 2024-05-06T12:14:22.890 INFO:tasks.workunit.client.0.smithi092.stdout:1/7: dwrite f0 [0,4194304] 0 2024-05-06T12:14:22.892 INFO:tasks.workunit.client.0.smithi092.stdout:7/0: write - no filename 2024-05-06T12:14:22.892 INFO:tasks.workunit.client.0.smithi092.stdout:7/1: dread - no filename 2024-05-06T12:14:22.893 INFO:tasks.workunit.client.0.smithi092.stdout:5/1: creat f0 x:0 0 0 2024-05-06T12:14:22.896 INFO:tasks.workunit.client.0.smithi092.stdout:2/14: mknod c2 0 2024-05-06T12:14:22.902 INFO:tasks.workunit.client.0.smithi092.stdout:8/0: unlink - no file 2024-05-06T12:14:22.902 INFO:tasks.workunit.client.0.smithi092.stdout:8/1: rmdir - no directory 2024-05-06T12:14:22.902 INFO:tasks.workunit.client.0.smithi092.stdout:8/2: dwrite - no filename 2024-05-06T12:14:22.904 INFO:tasks.workunit.client.0.smithi092.stdout:5/2: mknod c1 0 2024-05-06T12:14:22.904 INFO:tasks.workunit.client.0.smithi092.stdout:2/15: dread f1 [0,4194304] 0 2024-05-06T12:14:22.905 INFO:tasks.workunit.client.0.smithi092.stdout:3/4: link f0 f1 0 2024-05-06T12:14:22.906 INFO:tasks.workunit.client.0.smithi092.stdout:1/8: fdatasync f0 0 2024-05-06T12:14:22.907 INFO:tasks.workunit.client.0.smithi092.stdout:6/2: creat f0 x:0 0 0 2024-05-06T12:14:22.907 INFO:tasks.workunit.client.0.smithi092.stdout:6/3: dread - f0 zero size 2024-05-06T12:14:22.908 INFO:tasks.workunit.client.0.smithi092.stdout:6/4: chown f0 1 1 2024-05-06T12:14:22.909 INFO:tasks.workunit.client.0.smithi092.stdout:6/5: dread - f0 zero size 2024-05-06T12:14:22.910 INFO:tasks.workunit.client.0.smithi092.stdout:6/6: truncate f0 143098 0 2024-05-06T12:14:22.910 INFO:tasks.workunit.client.0.smithi092.stdout:6/7: stat f0 0 2024-05-06T12:14:22.910 INFO:tasks.workunit.client.0.smithi092.stdout:6/8: fdatasync f0 0 2024-05-06T12:14:22.911 INFO:tasks.workunit.client.0.smithi092.stdout:7/2: creat f0 x:0 0 0 2024-05-06T12:14:22.912 INFO:tasks.workunit.client.0.smithi092.stdout:7/3: chown f0 80112 1 2024-05-06T12:14:22.913 INFO:tasks.workunit.client.0.smithi092.stdout:7/4: write f0 [346437,96210] 0 2024-05-06T12:14:22.918 INFO:tasks.workunit.client.0.smithi092.stdout:2/16: creat f3 x:0 0 0 2024-05-06T12:14:22.919 INFO:tasks.workunit.client.0.smithi092.stdout:3/5: write f0 [529696,110079] 0 2024-05-06T12:14:22.922 INFO:tasks.workunit.client.0.smithi092.stdout:1/9: creat f2 x:0 0 0 2024-05-06T12:14:22.923 INFO:tasks.workunit.client.0.smithi092.stdout:7/5: dwrite f0 [0,4194304] 0 2024-05-06T12:14:22.923 INFO:tasks.workunit.client.0.smithi092.stdout:7/6: chown f0 279214 1 2024-05-06T12:14:22.924 INFO:tasks.workunit.client.0.smithi092.stdout:8/3: creat f0 x:0 0 0 2024-05-06T12:14:22.925 INFO:tasks.workunit.client.0.smithi092.stdout:8/4: truncate f0 937859 0 2024-05-06T12:14:22.925 INFO:tasks.workunit.client.0.smithi092.stdout:8/5: chown f0 335775 1 2024-05-06T12:14:22.926 INFO:tasks.workunit.client.0.smithi092.stdout:8/6: chown f0 15 1 2024-05-06T12:14:22.926 INFO:tasks.workunit.client.0.smithi092.stdout:6/9: rename f0 to f1 0 2024-05-06T12:14:22.941 INFO:tasks.workunit.client.0.smithi092.stdout:2/17: rename f3 to f4 0 2024-05-06T12:14:22.942 INFO:tasks.workunit.client.0.smithi092.stdout:9/0: creat f0 x:0 0 0 2024-05-06T12:14:22.942 INFO:tasks.workunit.client.0.smithi092.stdout:1/10: mkdir d3 0 2024-05-06T12:14:22.943 INFO:tasks.workunit.client.0.smithi092.stdout:6/10: mknod c2 0 2024-05-06T12:14:22.944 INFO:tasks.workunit.client.0.smithi092.stdout:2/18: chown c2 5455 1 2024-05-06T12:14:22.944 INFO:tasks.workunit.client.0.smithi092.stdout:9/1: truncate f0 20382 0 2024-05-06T12:14:22.945 INFO:tasks.workunit.client.0.smithi092.stdout:2/19: read - f4 zero size 2024-05-06T12:14:22.948 INFO:tasks.workunit.client.0.smithi092.stdout:9/2: write f0 [973752,57943] 0 2024-05-06T12:14:22.948 INFO:tasks.workunit.client.0.smithi092.stdout:9/3: readlink - no filename 2024-05-06T12:14:22.950 INFO:tasks.workunit.client.0.smithi092.stdout:8/7: dwrite f0 [0,4194304] 0 2024-05-06T12:14:22.958 INFO:tasks.workunit.client.0.smithi092.stdout:8/8: dread f0 [0,4194304] 0 2024-05-06T12:14:22.966 INFO:tasks.workunit.client.0.smithi092.stdout:8/9: dwrite f0 [0,4194304] 0 2024-05-06T12:14:22.974 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:22 smithi092 ceph-mon[26954]: pgmap v13: 65 pgs: 65 active+clean; 167 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 290 op/s 2024-05-06T12:14:22.974 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:22 smithi092 ceph-mon[26954]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:14:22.974 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:22 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:22.974 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:22 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:22.974 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:22 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:14:22.974 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:22 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:14:22.974 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:22 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:14:22.974 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:22 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:14:22.974 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:22 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:14:22.975 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:22 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:14:22.975 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:22 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:23.036 INFO:tasks.workunit.client.0.smithi092.stdout:0/9: fsync f1 0 2024-05-06T12:14:23.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.045+0000 7fa4b0b91700 1 -- 172.21.15.92:0/4203096567 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa4ac02b2a0 con 0x7fa4ac0fbcb0 2024-05-06T12:14:23.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.046+0000 7fa49affd700 1 -- 172.21.15.92:0/4203096567 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7fa49c0582d0 con 0x7fa4ac0fbcb0 2024-05-06T12:14:23.049 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:14:23.049 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": { 2024-05-06T12:14:23.049 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 2 2024-05-06T12:14:23.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:14:23.049 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": { 2024-05-06T12:14:23.049 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 1, 2024-05-06T12:14:23.050 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 1 2024-05-06T12:14:23.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:14:23.050 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": { 2024-05-06T12:14:23.050 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 6 2024-05-06T12:14:23.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:14:23.050 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-05-06T12:14:23.050 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 4 2024-05-06T12:14:23.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:14:23.050 INFO:teuthology.orchestra.run.smithi092.stdout: "overall": { 2024-05-06T12:14:23.050 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 13, 2024-05-06T12:14:23.050 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 1 2024-05-06T12:14:23.050 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-06T12:14:23.050 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:14:23.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.049+0000 7fa4b0b91700 1 -- 172.21.15.92:0/4203096567 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fa49406b790 msgr2=0x7fa49406dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:23.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.049+0000 7fa4b0b91700 1 --2- 172.21.15.92:0/4203096567 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fa49406b790 0x7fa49406dc40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0004f60 tx=0x7fa4a000b040 comp rx=0 tx=0).stop 2024-05-06T12:14:23.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.049+0000 7fa4b0b91700 1 -- 172.21.15.92:0/4203096567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4ac0fbcb0 msgr2=0x7fa4ac10dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:23.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.049+0000 7fa4b0b91700 1 --2- 172.21.15.92:0/4203096567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4ac0fbcb0 0x7fa4ac10dad0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00b3b0 tx=0x7fa49c009e30 comp rx=0 tx=0).stop 2024-05-06T12:14:23.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.049+0000 7fa4b0b91700 1 -- 172.21.15.92:0/4203096567 shutdown_connections 2024-05-06T12:14:23.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.049+0000 7fa4b0b91700 1 --2- 172.21.15.92:0/4203096567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4ac0fbcb0 0x7fa4ac10dad0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:23.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.049+0000 7fa4b0b91700 1 --2- 172.21.15.92:0/4203096567 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fa49406b790 0x7fa49406dc40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:23.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.049+0000 7fa4b0b91700 1 --2- 172.21.15.92:0/4203096567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4ac0fcf60 0x7fa4ac10e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:23.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.049+0000 7fa4b0b91700 1 -- 172.21.15.92:0/4203096567 >> 172.21.15.92:0/4203096567 conn(0x7fa4ac0f7430 msgr2=0x7fa4ac065b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:14:23.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.050+0000 7fa4b0b91700 1 -- 172.21.15.92:0/4203096567 shutdown_connections 2024-05-06T12:14:23.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.050+0000 7fa4b0b91700 1 -- 172.21.15.92:0/4203096567 wait complete. 2024-05-06T12:14:23.055 INFO:tasks.workunit.client.0.smithi092.stdout:5/3: fsync f0 0 2024-05-06T12:14:23.056 INFO:tasks.workunit.client.0.smithi092.stdout:5/4: dread - f0 zero size 2024-05-06T12:14:23.056 INFO:tasks.workunit.client.0.smithi092.stdout:5/5: readlink - no filename 2024-05-06T12:14:23.060 INFO:tasks.workunit.client.0.smithi092.stdout:3/6: fsync f0 0 2024-05-06T12:14:23.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:22 smithi190 ceph-mon[33798]: pgmap v13: 65 pgs: 65 active+clean; 167 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 290 op/s 2024-05-06T12:14:23.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:22 smithi190 ceph-mon[33798]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:14:23.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:22 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:23.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:22 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:23.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:22 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:14:23.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:22 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:14:23.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:22 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:14:23.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:22 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:14:23.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:22 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:14:23.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:22 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:14:23.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:22 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:23.119 INFO:tasks.workunit.client.0.smithi092.stdout:6/11: symlink l3 0 2024-05-06T12:14:23.122 INFO:tasks.workunit.client.0.smithi092.stdout:2/20: dread f1 [0,4194304] 0 2024-05-06T12:14:23.123 INFO:tasks.workunit.client.0.smithi092.stdout:8/10: mknod c1 0 2024-05-06T12:14:23.124 INFO:tasks.workunit.client.0.smithi092.stdout:0/10: symlink l4 0 2024-05-06T12:14:23.126 INFO:tasks.workunit.client.0.smithi092.stdout:4/4: rename l1 to l2 0 2024-05-06T12:14:23.126 INFO:tasks.workunit.client.0.smithi092.stdout:4/5: dwrite - no filename 2024-05-06T12:14:23.127 INFO:tasks.workunit.client.0.smithi092.stdout:5/6: creat f2 x:0 0 0 2024-05-06T12:14:23.129 INFO:tasks.workunit.client.0.smithi092.stdout:5/7: dread - f0 zero size 2024-05-06T12:14:23.131 INFO:tasks.workunit.client.0.smithi092.stdout:1/11: creat d3/f4 x:0 0 0 2024-05-06T12:14:23.133 INFO:tasks.workunit.client.0.smithi092.stdout:0/11: dwrite f1 [0,4194304] 0 2024-05-06T12:14:23.133 INFO:tasks.workunit.client.0.smithi092.stdout:6/12: dread f1 [0,4194304] 0 2024-05-06T12:14:23.134 INFO:tasks.workunit.client.0.smithi092.stdout:0/12: write f1 [615020,45794] 0 2024-05-06T12:14:23.134 INFO:tasks.workunit.client.0.smithi092.stdout:6/13: dread f1 [0,4194304] 0 2024-05-06T12:14:23.136 INFO:tasks.workunit.client.0.smithi092.stdout:2/21: truncate f1 848782 0 2024-05-06T12:14:23.138 INFO:tasks.workunit.client.0.smithi092.stdout:0/13: dread f1 [0,4194304] 0 2024-05-06T12:14:23.139 INFO:tasks.workunit.client.0.smithi092.stdout:8/11: creat f2 x:0 0 0 2024-05-06T12:14:23.140 INFO:tasks.workunit.client.0.smithi092.stdout:4/6: mknod c3 0 2024-05-06T12:14:23.140 INFO:tasks.workunit.client.0.smithi092.stdout:4/7: read - no filename 2024-05-06T12:14:23.140 INFO:tasks.workunit.client.0.smithi092.stdout:4/8: write - no filename 2024-05-06T12:14:23.140 INFO:tasks.workunit.client.0.smithi092.stdout:4/9: dwrite - no filename 2024-05-06T12:14:23.141 INFO:tasks.workunit.client.0.smithi092.stdout:0/14: dread f1 [0,4194304] 0 2024-05-06T12:14:23.142 INFO:tasks.workunit.client.0.smithi092.stdout:3/7: unlink f0 0 2024-05-06T12:14:23.145 INFO:tasks.workunit.client.0.smithi092.stdout:5/8: symlink l3 0 2024-05-06T12:14:23.146 INFO:tasks.workunit.client.0.smithi092.stdout:0/15: dread f1 [0,4194304] 0 2024-05-06T12:14:23.146 INFO:tasks.workunit.client.0.smithi092.stdout:0/16: rmdir - no directory 2024-05-06T12:14:23.146 INFO:tasks.workunit.client.0.smithi092.stdout:1/12: creat d3/f5 x:0 0 0 2024-05-06T12:14:23.146 INFO:tasks.workunit.client.0.smithi092.stdout:1/13: read - d3/f4 zero size 2024-05-06T12:14:23.154 INFO:tasks.workunit.client.0.smithi092.stdout:6/14: write f1 [717908,127834] 0 2024-05-06T12:14:23.157 INFO:tasks.workunit.client.0.smithi092.stdout:1/14: dwrite f0 [4194304,4194304] 0 2024-05-06T12:14:23.157 INFO:tasks.workunit.client.0.smithi092.stdout:8/12: creat f3 x:0 0 0 2024-05-06T12:14:23.158 INFO:tasks.workunit.client.0.smithi092.stdout:8/13: dread - f2 zero size 2024-05-06T12:14:23.158 INFO:tasks.workunit.client.0.smithi092.stdout:8/14: readlink - no filename 2024-05-06T12:14:23.158 INFO:tasks.workunit.client.0.smithi092.stdout:4/10: mknod c4 0 2024-05-06T12:14:23.158 INFO:tasks.workunit.client.0.smithi092.stdout:4/11: dread - no filename 2024-05-06T12:14:23.161 INFO:tasks.workunit.client.0.smithi092.stdout:5/9: unlink f2 0 2024-05-06T12:14:23.164 INFO:tasks.workunit.client.0.smithi092.stdout:0/17: mknod c5 0 2024-05-06T12:14:23.165 INFO:tasks.workunit.client.0.smithi092.stdout:9/4: getdents . 0 2024-05-06T12:14:23.170 INFO:tasks.workunit.client.0.smithi092.stdout:1/15: mknod d3/c6 0 2024-05-06T12:14:23.172 INFO:tasks.workunit.client.0.smithi092.stdout:5/10: dwrite f0 [0,4194304] 0 2024-05-06T12:14:23.172 INFO:tasks.workunit.client.0.smithi092.stdout:8/15: mknod c4 0 2024-05-06T12:14:23.172 INFO:tasks.workunit.client.0.smithi092.stdout:8/16: read - f2 zero size 2024-05-06T12:14:23.174 INFO:tasks.workunit.client.0.smithi092.stdout:4/12: creat f5 x:0 0 0 2024-05-06T12:14:23.177 INFO:tasks.workunit.client.0.smithi092.stdout:0/18: creat f6 x:0 0 0 2024-05-06T12:14:23.181 INFO:tasks.workunit.client.0.smithi092.stdout:8/17: dwrite f3 [0,4194304] 0 2024-05-06T12:14:23.182 INFO:tasks.workunit.client.0.smithi092.stdout:1/16: mknod d3/c7 0 2024-05-06T12:14:23.183 INFO:tasks.workunit.client.0.smithi092.stdout:1/17: chown d3/c7 379 1 2024-05-06T12:14:23.184 INFO:tasks.workunit.client.0.smithi092.stdout:1/18: dread - d3/f4 zero size 2024-05-06T12:14:23.185 INFO:tasks.workunit.client.0.smithi092.stdout:1/19: write d3/f4 [515934,36540] 0 2024-05-06T12:14:23.186 INFO:tasks.workunit.client.0.smithi092.stdout:1/20: dread - d3/f5 zero size 2024-05-06T12:14:23.186 INFO:tasks.workunit.client.0.smithi092.stdout:4/13: mknod c6 0 2024-05-06T12:14:23.186 INFO:tasks.workunit.client.0.smithi092.stdout:1/21: readlink l1 0 2024-05-06T12:14:23.187 INFO:tasks.workunit.client.0.smithi092.stdout:4/14: dread - f5 zero size 2024-05-06T12:14:23.189 INFO:tasks.workunit.client.0.smithi092.stdout:0/19: mkdir d7 0 2024-05-06T12:14:23.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.186+0000 7fe1d3e22700 1 -- 172.21.15.92:0/3463129690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1cc0fb470 msgr2=0x7fe1cc0fb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:23.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.186+0000 7fe1d3e22700 1 --2- 172.21.15.92:0/3463129690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1cc0fb470 0x7fe1cc0fb880 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4004e20 tx=0x7fe1c400acf0 comp rx=0 tx=0).stop 2024-05-06T12:14:23.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.187+0000 7fe1d3e22700 1 -- 172.21.15.92:0/3463129690 shutdown_connections 2024-05-06T12:14:23.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.187+0000 7fe1d3e22700 1 --2- 172.21.15.92:0/3463129690 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1cc0fc720 0x7fe1cc0fcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:23.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.187+0000 7fe1d3e22700 1 --2- 172.21.15.92:0/3463129690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1cc0fb470 0x7fe1cc0fb880 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:23.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.187+0000 7fe1d3e22700 1 -- 172.21.15.92:0/3463129690 >> 172.21.15.92:0/3463129690 conn(0x7fe1cc0f6bb0 msgr2=0x7fe1cc0f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:14:23.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.187+0000 7fe1d3e22700 1 -- 172.21.15.92:0/3463129690 shutdown_connections 2024-05-06T12:14:23.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.187+0000 7fe1d3e22700 1 -- 172.21.15.92:0/3463129690 wait complete. 2024-05-06T12:14:23.191 INFO:tasks.workunit.client.0.smithi092.stdout:4/15: write f5 [124721,101671] 0 2024-05-06T12:14:23.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.188+0000 7fe1d3e22700 1 Processor -- start 2024-05-06T12:14:23.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.188+0000 7fe1d3e22700 1 -- start start 2024-05-06T12:14:23.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.188+0000 7fe1d3e22700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1cc0fb470 0x7fe1cc113ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:23.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.188+0000 7fe1d3e22700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1cc0fc720 0x7fe1cc1143e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:23.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.188+0000 7fe1d3e22700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc1149e0 con 0x7fe1cc0fb470 2024-05-06T12:14:23.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.188+0000 7fe1d3e22700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc114b50 con 0x7fe1cc0fc720 2024-05-06T12:14:23.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.189+0000 7fe1d1bbe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1cc0fb470 0x7fe1cc113ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:23.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.189+0000 7fe1d1bbe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1cc0fb470 0x7fe1cc113ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50950/0 (socket says 172.21.15.92:50950) 2024-05-06T12:14:23.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.189+0000 7fe1d1bbe700 1 -- 172.21.15.92:0/1512740957 learned_addr learned my addr 172.21.15.92:0/1512740957 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:14:23.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.189+0000 7fe1d13bd700 1 --2- 172.21.15.92:0/1512740957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1cc0fc720 0x7fe1cc1143e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:23.193 INFO:tasks.workunit.client.0.smithi092.stdout:8/18: unlink c4 0 2024-05-06T12:14:23.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.189+0000 7fe1d1bbe700 1 -- 172.21.15.92:0/1512740957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1cc0fc720 msgr2=0x7fe1cc1143e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:23.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.189+0000 7fe1d1bbe700 1 --2- 172.21.15.92:0/1512740957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1cc0fc720 0x7fe1cc1143e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:23.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.189+0000 7fe1d1bbe700 1 -- 172.21.15.92:0/1512740957 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c40049b0 con 0x7fe1cc0fb470 2024-05-06T12:14:23.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.189+0000 7fe1d1bbe700 1 --2- 172.21.15.92:0/1512740957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1cc0fb470 0x7fe1cc113ea0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe1c40199f0 tx=0x7fe1c400c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:14:23.194 INFO:tasks.workunit.client.0.smithi092.stdout:4/16: truncate f5 885827 0 2024-05-06T12:14:23.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.190+0000 7fe1c27fc700 1 -- 172.21.15.92:0/1512740957 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1c400a0d0 con 0x7fe1cc0fb470 2024-05-06T12:14:23.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.190+0000 7fe1d3e22700 1 -- 172.21.15.92:0/1512740957 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1cc119550 con 0x7fe1cc0fb470 2024-05-06T12:14:23.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.190+0000 7fe1c27fc700 1 -- 172.21.15.92:0/1512740957 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe1c400a700 con 0x7fe1cc0fb470 2024-05-06T12:14:23.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.190+0000 7fe1c27fc700 1 -- 172.21.15.92:0/1512740957 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1c4020c70 con 0x7fe1cc0fb470 2024-05-06T12:14:23.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.190+0000 7fe1d3e22700 1 -- 172.21.15.92:0/1512740957 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1cc119b10 con 0x7fe1cc0fb470 2024-05-06T12:14:23.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.192+0000 7fe1d3e22700 1 -- 172.21.15.92:0/1512740957 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1cc04c710 con 0x7fe1cc0fb470 2024-05-06T12:14:23.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.194+0000 7fe1c27fc700 1 -- 172.21.15.92:0/1512740957 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7fe1c400a230 con 0x7fe1cc0fb470 2024-05-06T12:14:23.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.194+0000 7fe1c27fc700 1 --2- 172.21.15.92:0/1512740957 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fe1b806b8a0 0x7fe1b806dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:23.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.195+0000 7fe1c27fc700 1 -- 172.21.15.92:0/1512740957 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fe1c408c680 con 0x7fe1cc0fb470 2024-05-06T12:14:23.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.195+0000 7fe1d13bd700 1 --2- 172.21.15.92:0/1512740957 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fe1b806b8a0 0x7fe1b806dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:23.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.196+0000 7fe1d13bd700 1 --2- 172.21.15.92:0/1512740957 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fe1b806b8a0 0x7fe1b806dd50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc004de0 tx=0x7fe1bc00d040 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:14:23.199 INFO:tasks.workunit.client.0.smithi092.stdout:0/20: write f1 [3216786,83541] 0 2024-05-06T12:14:23.199 INFO:tasks.workunit.client.0.smithi092.stdout:1/22: dwrite d3/f4 [0,4194304] 0 2024-05-06T12:14:23.201 INFO:tasks.workunit.client.0.smithi092.stdout:8/19: rename f3 to f5 0 2024-05-06T12:14:23.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.198+0000 7fe1c27fc700 1 -- 172.21.15.92:0/1512740957 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fe1c4059cf0 con 0x7fe1cc0fb470 2024-05-06T12:14:23.203 INFO:tasks.workunit.client.0.smithi092.stdout:4/17: rename c4 to c7 0 2024-05-06T12:14:23.209 INFO:tasks.workunit.client.0.smithi092.stdout:1/23: rename d3/f4 to d3/f8 0 2024-05-06T12:14:23.213 INFO:tasks.workunit.client.0.smithi092.stdout:8/20: creat f6 x:0 0 0 2024-05-06T12:14:23.213 INFO:tasks.workunit.client.0.smithi092.stdout:4/18: dwrite f5 [0,4194304] 0 2024-05-06T12:14:23.218 INFO:tasks.workunit.client.0.smithi092.stdout:8/21: write f0 [2827351,129875] 0 2024-05-06T12:14:23.218 INFO:tasks.workunit.client.0.smithi092.stdout:8/22: chown f0 2815 1 2024-05-06T12:14:23.219 INFO:tasks.workunit.client.0.smithi092.stdout:8/23: chown c1 183 1 2024-05-06T12:14:23.219 INFO:tasks.workunit.client.0.smithi092.stdout:8/24: dread - f2 zero size 2024-05-06T12:14:23.228 INFO:tasks.workunit.client.0.smithi092.stdout:8/25: dwrite f2 [0,4194304] 0 2024-05-06T12:14:23.274 INFO:tasks.workunit.client.0.smithi092.stdout:7/7: truncate f0 3216737 0 2024-05-06T12:14:23.279 INFO:tasks.workunit.client.0.smithi092.stdout:2/22: dread f1 [0,4194304] 0 2024-05-06T12:14:23.280 INFO:tasks.workunit.client.0.smithi092.stdout:2/23: chown f4 37 1 2024-05-06T12:14:23.281 INFO:tasks.workunit.client.0.smithi092.stdout:2/24: dread f1 [0,4194304] 0 2024-05-06T12:14:23.282 INFO:tasks.workunit.client.0.smithi092.stdout:2/25: truncate f4 1002023 0 2024-05-06T12:14:23.288 INFO:tasks.workunit.client.0.smithi092.stdout:0/21: dwrite f1 [4194304,4194304] 0 2024-05-06T12:14:23.288 INFO:tasks.workunit.client.0.smithi092.stdout:2/26: creat f5 x:0 0 0 2024-05-06T12:14:23.294 INFO:tasks.workunit.client.0.smithi092.stdout:0/22: dread f1 [4194304,4194304] 0 2024-05-06T12:14:23.295 INFO:tasks.workunit.client.0.smithi092.stdout:0/23: truncate f6 244829 0 2024-05-06T12:14:23.296 INFO:tasks.workunit.client.0.smithi092.stdout:4/19: fsync f5 0 2024-05-06T12:14:23.298 INFO:tasks.workunit.client.0.smithi092.stdout:0/24: dread f6 [0,4194304] 0 2024-05-06T12:14:23.299 INFO:tasks.workunit.client.0.smithi092.stdout:6/15: truncate f1 364931 0 2024-05-06T12:14:23.305 INFO:tasks.workunit.client.0.smithi092.stdout:5/11: truncate f0 2557723 0 2024-05-06T12:14:23.305 INFO:tasks.workunit.client.0.smithi092.stdout:2/27: dwrite f4 [0,4194304] 0 2024-05-06T12:14:23.306 INFO:tasks.workunit.client.0.smithi092.stdout:6/16: mkdir d4 0 2024-05-06T12:14:23.308 INFO:tasks.workunit.client.0.smithi092.stdout:4/20: dwrite f5 [0,4194304] 0 2024-05-06T12:14:23.309 INFO:tasks.workunit.client.0.smithi092.stdout:4/21: write f5 [2546841,21355] 0 2024-05-06T12:14:23.310 INFO:tasks.workunit.client.0.smithi092.stdout:8/26: rename f5 to f7 0 2024-05-06T12:14:23.310 INFO:tasks.workunit.client.0.smithi092.stdout:8/27: fdatasync f6 0 2024-05-06T12:14:23.311 INFO:tasks.workunit.client.0.smithi092.stdout:8/28: dread - f6 zero size 2024-05-06T12:14:23.311 INFO:tasks.workunit.client.0.smithi092.stdout:5/12: mkdir d4 0 2024-05-06T12:14:23.312 INFO:tasks.workunit.client.0.smithi092.stdout:8/29: read f2 [1574897,124828] 0 2024-05-06T12:14:23.313 INFO:tasks.workunit.client.0.smithi092.stdout:2/28: creat f6 x:0 0 0 2024-05-06T12:14:23.317 INFO:tasks.workunit.client.0.smithi092.stdout:4/22: mknod c8 0 2024-05-06T12:14:23.317 INFO:tasks.workunit.client.0.smithi092.stdout:4/23: chown c3 62130500 1 2024-05-06T12:14:23.321 INFO:tasks.workunit.client.0.smithi092.stdout:1/24: truncate d3/f8 2334629 0 2024-05-06T12:14:23.322 INFO:tasks.workunit.client.0.smithi092.stdout:1/25: fdatasync f0 0 2024-05-06T12:14:23.323 INFO:tasks.workunit.client.0.smithi092.stdout:2/29: creat f7 x:0 0 0 2024-05-06T12:14:23.325 INFO:tasks.workunit.client.0.smithi092.stdout:4/24: mkdir d9 0 2024-05-06T12:14:23.328 INFO:tasks.workunit.client.0.smithi092.stdout:5/13: mkdir d4/d5 0 2024-05-06T12:14:23.332 INFO:tasks.workunit.client.0.smithi092.stdout:7/8: dwrite f0 [0,4194304] 0 2024-05-06T12:14:23.333 INFO:tasks.workunit.client.0.smithi092.stdout:7/9: write f0 [2696207,68497] 0 2024-05-06T12:14:23.336 INFO:tasks.workunit.client.0.smithi092.stdout:3/8: unlink f1 0 2024-05-06T12:14:23.336 INFO:tasks.workunit.client.0.smithi092.stdout:3/9: fsync - no filename 2024-05-06T12:14:23.337 INFO:tasks.workunit.client.0.smithi092.stdout:8/30: rename f2 to f8 0 2024-05-06T12:14:23.339 INFO:tasks.workunit.client.0.smithi092.stdout:4/25: fdatasync f5 0 2024-05-06T12:14:23.340 INFO:tasks.workunit.client.0.smithi092.stdout:2/30: creat f8 x:0 0 0 2024-05-06T12:14:23.342 INFO:tasks.workunit.client.0.smithi092.stdout:0/25: truncate f1 6094288 0 2024-05-06T12:14:23.346 INFO:tasks.workunit.client.0.smithi092.stdout:6/17: dwrite f1 [0,4194304] 0 2024-05-06T12:14:23.349 INFO:tasks.workunit.client.0.smithi092.stdout:3/10: creat f2 x:0 0 0 2024-05-06T12:14:23.349 INFO:tasks.workunit.client.0.smithi092.stdout:3/11: rmdir - no directory 2024-05-06T12:14:23.350 INFO:tasks.workunit.client.0.smithi092.stdout:8/31: mkdir d9 0 2024-05-06T12:14:23.352 INFO:tasks.workunit.client.0.smithi092.stdout:5/14: dwrite f0 [0,4194304] 0 2024-05-06T12:14:23.355 INFO:tasks.workunit.client.0.smithi092.stdout:2/31: rename f8 to f9 0 2024-05-06T12:14:23.360 INFO:tasks.workunit.client.0.smithi092.stdout:0/26: rename l0 to d7/l8 0 2024-05-06T12:14:23.363 INFO:tasks.workunit.client.0.smithi092.stdout:7/10: link f0 f1 0 2024-05-06T12:14:23.367 INFO:tasks.workunit.client.0.smithi092.stdout:1/26: dwrite d3/f8 [0,4194304] 0 2024-05-06T12:14:23.367 INFO:tasks.workunit.client.0.smithi092.stdout:3/12: creat f3 x:0 0 0 2024-05-06T12:14:23.368 INFO:tasks.workunit.client.0.smithi092.stdout:3/13: dread - f2 zero size 2024-05-06T12:14:23.368 INFO:tasks.workunit.client.0.smithi092.stdout:1/27: dread - d3/f5 zero size 2024-05-06T12:14:23.368 INFO:tasks.workunit.client.0.smithi092.stdout:4/26: creat d9/fa x:0 0 0 2024-05-06T12:14:23.369 INFO:tasks.workunit.client.0.smithi092.stdout:3/14: write f3 [518671,104098] 0 2024-05-06T12:14:23.370 INFO:tasks.workunit.client.0.smithi092.stdout:4/27: truncate d9/fa 358248 0 2024-05-06T12:14:23.371 INFO:tasks.workunit.client.0.smithi092.stdout:4/28: chown c8 2852024 1 2024-05-06T12:14:23.372 INFO:tasks.workunit.client.0.smithi092.stdout:4/29: write f5 [1957077,30105] 0 2024-05-06T12:14:23.379 INFO:tasks.workunit.client.0.smithi092.stdout:5/15: rename l3 to d4/d5/l6 0 2024-05-06T12:14:23.380 INFO:tasks.workunit.client.0.smithi092.stdout:5/16: write f0 [227547,111768] 0 2024-05-06T12:14:23.381 INFO:tasks.workunit.client.0.smithi092.stdout:6/18: fsync f1 0 2024-05-06T12:14:23.381 INFO:tasks.workunit.client.0.smithi092.stdout:7/11: fdatasync f1 0 2024-05-06T12:14:23.383 INFO:tasks.workunit.client.0.smithi092.stdout:8/32: symlink d9/la 0 2024-05-06T12:14:23.384 INFO:tasks.workunit.client.0.smithi092.stdout:1/28: creat d3/f9 x:0 0 0 2024-05-06T12:14:23.384 INFO:tasks.workunit.client.0.smithi092.stdout:1/29: write f2 [773676,86326] 0 2024-05-06T12:14:23.387 INFO:tasks.workunit.client.0.smithi092.stdout:2/32: link f4 fa 0 2024-05-06T12:14:23.389 INFO:tasks.workunit.client.0.smithi092.stdout:4/30: symlink d9/lb 0 2024-05-06T12:14:23.389 INFO:tasks.workunit.client.0.smithi092.stdout:4/31: chown d9/lb 84 1 2024-05-06T12:14:23.393 INFO:tasks.workunit.client.0.smithi092.stdout:5/17: mknod d4/d5/c7 0 2024-05-06T12:14:23.395 INFO:tasks.workunit.client.0.smithi092.stdout:7/12: symlink l2 0 2024-05-06T12:14:23.397 INFO:tasks.workunit.client.0.smithi092.stdout:3/15: link f3 f4 0 2024-05-06T12:14:23.398 INFO:tasks.workunit.client.0.smithi092.stdout:1/30: creat d3/fa x:0 0 0 2024-05-06T12:14:23.399 INFO:tasks.workunit.client.0.smithi092.stdout:2/33: mknod cb 0 2024-05-06T12:14:23.403 INFO:tasks.workunit.client.0.smithi092.stdout:5/18: symlink d4/d5/l8 0 2024-05-06T12:14:23.404 INFO:tasks.workunit.client.0.smithi092.stdout:6/19: creat d4/f5 x:0 0 0 2024-05-06T12:14:23.405 INFO:tasks.workunit.client.0.smithi092.stdout:6/20: stat l3 0 2024-05-06T12:14:23.405 INFO:tasks.workunit.client.0.smithi092.stdout:7/13: creat f3 x:0 0 0 2024-05-06T12:14:23.406 INFO:tasks.workunit.client.0.smithi092.stdout:6/21: stat c2 0 2024-05-06T12:14:23.408 INFO:tasks.workunit.client.0.smithi092.stdout:6/22: dread - d4/f5 zero size 2024-05-06T12:14:23.408 INFO:tasks.workunit.client.0.smithi092.stdout:0/27: dwrite f1 [4194304,4194304] 0 2024-05-06T12:14:23.409 INFO:tasks.workunit.client.0.smithi092.stdout:1/31: symlink d3/lb 0 2024-05-06T12:14:23.410 INFO:tasks.workunit.client.0.smithi092.stdout:6/23: write f1 [497183,118366] 0 2024-05-06T12:14:23.413 INFO:tasks.workunit.client.0.smithi092.stdout:6/24: write d4/f5 [762020,90643] 0 2024-05-06T12:14:23.413 INFO:tasks.workunit.client.0.smithi092.stdout:2/34: unlink cb 0 2024-05-06T12:14:23.415 INFO:tasks.workunit.client.0.smithi092.stdout:2/35: chown f5 25023221 1 2024-05-06T12:14:23.417 INFO:tasks.workunit.client.0.smithi092.stdout:4/32: rename d9/fa to d9/fc 0 2024-05-06T12:14:23.419 INFO:tasks.workunit.client.0.smithi092.stdout:7/14: dwrite f0 [0,4194304] 0 2024-05-06T12:14:23.419 INFO:tasks.workunit.client.0.smithi092.stdout:5/19: creat d4/f9 x:0 0 0 2024-05-06T12:14:23.419 INFO:tasks.workunit.client.0.smithi092.stdout:8/33: link d9/la d9/lb 0 2024-05-06T12:14:23.420 INFO:tasks.workunit.client.0.smithi092.stdout:8/34: write f6 [795014,80329] 0 2024-05-06T12:14:23.421 INFO:tasks.workunit.client.0.smithi092.stdout:3/16: link f2 f5 0 2024-05-06T12:14:23.422 INFO:tasks.workunit.client.0.smithi092.stdout:0/28: creat d7/f9 x:0 0 0 2024-05-06T12:14:23.423 INFO:tasks.workunit.client.0.smithi092.stdout:0/29: stat f6 0 2024-05-06T12:14:23.423 INFO:tasks.workunit.client.0.smithi092.stdout:0/30: chown c5 138832276 1 2024-05-06T12:14:23.424 INFO:tasks.workunit.client.0.smithi092.stdout:6/25: creat d4/f6 x:0 0 0 2024-05-06T12:14:23.426 INFO:tasks.workunit.client.0.smithi092.stdout:2/36: rename f9 to fc 0 2024-05-06T12:14:23.429 INFO:tasks.workunit.client.0.smithi092.stdout:7/15: creat f4 x:0 0 0 2024-05-06T12:14:23.430 INFO:tasks.workunit.client.0.smithi092.stdout:5/20: mknod d4/ca 0 2024-05-06T12:14:23.433 INFO:tasks.workunit.client.0.smithi092.stdout:3/17: symlink l6 0 2024-05-06T12:14:23.436 INFO:tasks.workunit.client.0.smithi092.stdout:1/32: truncate f0 7696758 0 2024-05-06T12:14:23.439 INFO:tasks.workunit.client.0.smithi092.stdout:2/37: creat fd x:0 0 0 2024-05-06T12:14:23.440 INFO:tasks.workunit.client.0.smithi092.stdout:0/31: dread f6 [0,4194304] 0 2024-05-06T12:14:23.441 INFO:tasks.workunit.client.0.smithi092.stdout:3/18: fdatasync f2 0 2024-05-06T12:14:23.442 INFO:tasks.workunit.client.0.smithi092.stdout:4/33: rmdir d9 39 2024-05-06T12:14:23.444 INFO:tasks.workunit.client.0.smithi092.stdout:5/21: dwrite d4/f9 [0,4194304] 0 2024-05-06T12:14:23.444 INFO:tasks.workunit.client.0.smithi092.stdout:8/35: dwrite f8 [0,4194304] 0 2024-05-06T12:14:23.445 INFO:tasks.workunit.client.0.smithi092.stdout:5/22: stat d4/d5/l8 0 2024-05-06T12:14:23.445 INFO:tasks.workunit.client.0.smithi092.stdout:1/33: rename d3/f9 to d3/fc 0 2024-05-06T12:14:23.450 INFO:tasks.workunit.client.0.smithi092.stdout:0/32: unlink l4 0 2024-05-06T12:14:23.452 INFO:tasks.workunit.client.0.smithi092.stdout:3/19: mknod c7 0 2024-05-06T12:14:23.454 INFO:tasks.workunit.client.0.smithi092.stdout:6/26: link c2 d4/c7 0 2024-05-06T12:14:23.455 INFO:tasks.workunit.client.0.smithi092.stdout:6/27: write f1 [845326,3869] 0 2024-05-06T12:14:23.456 INFO:tasks.workunit.client.0.smithi092.stdout:6/28: write f1 [585661,113254] 0 2024-05-06T12:14:23.456 INFO:tasks.workunit.client.0.smithi092.stdout:8/36: mkdir d9/dc 0 2024-05-06T12:14:23.458 INFO:tasks.workunit.client.0.smithi092.stdout:8/37: write f8 [4466559,113249] 0 2024-05-06T12:14:23.461 INFO:tasks.workunit.client.0.smithi092.stdout:0/33: mkdir d7/da 0 2024-05-06T12:14:23.463 INFO:tasks.workunit.client.0.smithi092.stdout:3/20: mkdir d8 0 2024-05-06T12:14:23.464 INFO:tasks.workunit.client.0.smithi092.stdout:4/34: mkdir d9/dd 0 2024-05-06T12:14:23.465 INFO:tasks.workunit.client.0.smithi092.stdout:4/35: read f5 [2575298,87555] 0 2024-05-06T12:14:23.466 INFO:tasks.workunit.client.0.smithi092.stdout:2/38: write fa [2662715,55108] 0 2024-05-06T12:14:23.467 INFO:tasks.workunit.client.0.smithi092.stdout:2/39: truncate fc 549860 0 2024-05-06T12:14:23.469 INFO:tasks.workunit.client.0.smithi092.stdout:7/16: getdents . 0 2024-05-06T12:14:23.469 INFO:tasks.workunit.client.0.smithi092.stdout:7/17: stat f0 0 2024-05-06T12:14:23.470 INFO:tasks.workunit.client.0.smithi092.stdout:6/29: creat d4/f8 x:0 0 0 2024-05-06T12:14:23.470 INFO:tasks.workunit.client.0.smithi092.stdout:6/30: chown d4 118369 1 2024-05-06T12:14:23.471 INFO:tasks.workunit.client.0.smithi092.stdout:4/36: dread f5 [0,4194304] 0 2024-05-06T12:14:23.472 INFO:tasks.workunit.client.0.smithi092.stdout:8/38: mkdir d9/dd 0 2024-05-06T12:14:23.473 INFO:tasks.workunit.client.0.smithi092.stdout:2/40: dread fa [0,4194304] 0 2024-05-06T12:14:23.474 INFO:tasks.workunit.client.0.smithi092.stdout:0/34: mknod d7/cb 0 2024-05-06T12:14:23.476 INFO:tasks.workunit.client.0.smithi092.stdout:3/21: fsync f3 0 2024-05-06T12:14:23.476 INFO:tasks.workunit.client.0.smithi092.stdout:3/22: chown f3 1 1 2024-05-06T12:14:23.479 INFO:tasks.workunit.client.0.smithi092.stdout:7/18: rename f4 to f5 0 2024-05-06T12:14:23.481 INFO:tasks.workunit.client.0.smithi092.stdout:6/31: mknod d4/c9 0 2024-05-06T12:14:23.484 INFO:tasks.workunit.client.0.smithi092.stdout:0/35: mkdir d7/dc 0 2024-05-06T12:14:23.485 INFO:tasks.workunit.client.0.smithi092.stdout:0/36: read f6 [139999,90914] 0 2024-05-06T12:14:23.486 INFO:tasks.workunit.client.0.smithi092.stdout:0/37: write d7/f9 [168035,69774] 0 2024-05-06T12:14:23.490 INFO:tasks.workunit.client.0.smithi092.stdout:5/23: truncate d4/f9 2441716 0 2024-05-06T12:14:23.491 INFO:tasks.workunit.client.0.smithi092.stdout:7/19: rename l2 to l6 0 2024-05-06T12:14:23.492 INFO:tasks.workunit.client.0.smithi092.stdout:6/32: stat c2 0 2024-05-06T12:14:23.493 INFO:tasks.workunit.client.0.smithi092.stdout:4/37: symlink d9/dd/le 0 2024-05-06T12:14:23.495 INFO:tasks.workunit.client.0.smithi092.stdout:2/41: unlink f4 0 2024-05-06T12:14:23.495 INFO:tasks.workunit.client.0.smithi092.stdout:3/23: mkdir d8/d9 0 2024-05-06T12:14:23.496 INFO:tasks.workunit.client.0.smithi092.stdout:3/24: dread - f5 zero size 2024-05-06T12:14:23.497 INFO:tasks.workunit.client.0.smithi092.stdout:5/24: mknod d4/d5/cb 0 2024-05-06T12:14:23.499 INFO:tasks.workunit.client.0.smithi092.stdout:7/20: creat f7 x:0 0 0 2024-05-06T12:14:23.500 INFO:tasks.workunit.client.0.smithi092.stdout:4/38: mkdir d9/df 0 2024-05-06T12:14:23.500 INFO:tasks.workunit.client.0.smithi092.stdout:0/38: mknod d7/dc/cd 0 2024-05-06T12:14:23.502 INFO:tasks.workunit.client.0.smithi092.stdout:6/33: dwrite d4/f5 [0,4194304] 0 2024-05-06T12:14:23.502 INFO:tasks.workunit.client.0.smithi092.stdout:3/25: creat d8/fa x:0 0 0 2024-05-06T12:14:23.505 INFO:tasks.workunit.client.0.smithi092.stdout:4/39: mknod d9/c10 0 2024-05-06T12:14:23.509 INFO:tasks.workunit.client.0.smithi092.stdout:2/42: dwrite f5 [0,4194304] 0 2024-05-06T12:14:23.510 INFO:tasks.workunit.client.0.smithi092.stdout:4/40: symlink d9/l11 0 2024-05-06T12:14:23.514 INFO:tasks.workunit.client.0.smithi092.stdout:5/25: write d4/f9 [346960,40674] 0 2024-05-06T12:14:23.519 INFO:tasks.workunit.client.0.smithi092.stdout:4/41: mknod d9/dd/c12 0 2024-05-06T12:14:23.520 INFO:tasks.workunit.client.0.smithi092.stdout:6/34: link l3 d4/la 0 2024-05-06T12:14:23.522 INFO:tasks.workunit.client.0.smithi092.stdout:5/26: unlink d4/d5/l8 0 2024-05-06T12:14:23.528 INFO:tasks.workunit.client.0.smithi092.stdout:6/35: rename d4/f8 to d4/fb 0 2024-05-06T12:14:23.530 INFO:tasks.workunit.client.0.smithi092.stdout:7/21: link l6 l8 0 2024-05-06T12:14:23.533 INFO:tasks.workunit.client.0.smithi092.stdout:3/26: getdents d8 0 2024-05-06T12:14:23.535 INFO:tasks.workunit.client.0.smithi092.stdout:6/36: dwrite d4/f5 [0,4194304] 0 2024-05-06T12:14:23.538 INFO:tasks.workunit.client.0.smithi092.stdout:0/39: write f6 [840483,78597] 0 2024-05-06T12:14:23.539 INFO:tasks.workunit.client.0.smithi092.stdout:0/40: fdatasync d7/f9 0 2024-05-06T12:14:23.543 INFO:tasks.workunit.client.0.smithi092.stdout:7/22: mkdir d9 0 2024-05-06T12:14:23.544 INFO:tasks.workunit.client.0.smithi092.stdout:3/27: mkdir d8/db 0 2024-05-06T12:14:23.544 INFO:tasks.workunit.client.0.smithi092.stdout:3/28: dread - f5 zero size 2024-05-06T12:14:23.547 INFO:tasks.workunit.client.0.smithi092.stdout:4/42: write f5 [944524,75690] 0 2024-05-06T12:14:23.548 INFO:tasks.workunit.client.0.smithi092.stdout:0/41: readlink d7/l8 0 2024-05-06T12:14:23.549 INFO:tasks.workunit.client.0.smithi092.stdout:5/27: link d4/d5/c7 d4/d5/cc 0 2024-05-06T12:14:23.554 INFO:tasks.workunit.client.0.smithi092.stdout:6/37: unlink l3 0 2024-05-06T12:14:23.555 INFO:tasks.workunit.client.0.smithi092.stdout:6/38: read - d4/f6 zero size 2024-05-06T12:14:23.556 INFO:tasks.workunit.client.0.smithi092.stdout:6/39: chown d4/fb 0 1 2024-05-06T12:14:23.556 INFO:tasks.workunit.client.0.smithi092.stdout:6/40: chown d4/c9 2373751 1 2024-05-06T12:14:23.557 INFO:tasks.workunit.client.0.smithi092.stdout:4/43: symlink d9/l13 0 2024-05-06T12:14:23.557 INFO:tasks.workunit.client.0.smithi092.stdout:4/44: chown c8 5233264 1 2024-05-06T12:14:23.559 INFO:tasks.workunit.client.0.smithi092.stdout:0/42: creat d7/dc/fe x:0 0 0 2024-05-06T12:14:23.561 INFO:tasks.workunit.client.0.smithi092.stdout:7/23: symlink d9/la 0 2024-05-06T12:14:23.561 INFO:tasks.workunit.client.0.smithi092.stdout:3/29: getdents d8/db 0 2024-05-06T12:14:23.562 INFO:tasks.workunit.client.0.smithi092.stdout:7/24: readlink d9/la 0 2024-05-06T12:14:23.562 INFO:tasks.workunit.client.0.smithi092.stdout:6/41: mknod d4/cc 0 2024-05-06T12:14:23.562 INFO:tasks.workunit.client.0.smithi092.stdout:7/25: dread - f5 zero size 2024-05-06T12:14:23.566 INFO:tasks.workunit.client.0.smithi092.stdout:0/43: dwrite f1 [0,4194304] 0 2024-05-06T12:14:23.567 INFO:tasks.workunit.client.0.smithi092.stdout:0/44: truncate d7/f9 578058 0 2024-05-06T12:14:23.568 INFO:tasks.workunit.client.0.smithi092.stdout:4/45: creat d9/df/f14 x:0 0 0 2024-05-06T12:14:23.569 INFO:tasks.workunit.client.0.smithi092.stdout:6/42: mknod d4/cd 0 2024-05-06T12:14:23.569 INFO:tasks.workunit.client.0.smithi092.stdout:4/46: readlink d9/lb 0 2024-05-06T12:14:23.569 INFO:tasks.workunit.client.0.smithi092.stdout:0/45: stat d7/cb 0 2024-05-06T12:14:23.571 INFO:tasks.workunit.client.0.smithi092.stdout:4/47: dread d9/fc [0,4194304] 0 2024-05-06T12:14:23.572 INFO:tasks.workunit.client.0.smithi092.stdout:7/26: rename f5 to d9/fb 0 2024-05-06T12:14:23.574 INFO:tasks.workunit.client.0.smithi092.stdout:4/48: read d9/fc [182224,105480] 0 2024-05-06T12:14:23.578 INFO:tasks.workunit.client.0.smithi092.stdout:0/46: mknod d7/dc/cf 0 2024-05-06T12:14:23.579 INFO:tasks.workunit.client.0.smithi092.stdout:6/43: dwrite d4/f5 [0,4194304] 0 2024-05-06T12:14:23.579 INFO:tasks.workunit.client.0.smithi092.stdout:5/28: truncate f0 1459664 0 2024-05-06T12:14:23.580 INFO:tasks.workunit.client.0.smithi092.stdout:6/44: chown f1 57 1 2024-05-06T12:14:23.582 INFO:tasks.workunit.client.0.smithi092.stdout:3/30: write f3 [1013854,4397] 0 2024-05-06T12:14:23.588 INFO:tasks.workunit.client.0.smithi092.stdout:4/49: dwrite d9/df/f14 [0,4194304] 0 2024-05-06T12:14:23.588 INFO:tasks.workunit.client.0.smithi092.stdout:0/47: symlink d7/l10 0 2024-05-06T12:14:23.589 INFO:tasks.workunit.client.0.smithi092.stdout:0/48: fdatasync d7/dc/fe 0 2024-05-06T12:14:23.590 INFO:tasks.workunit.client.0.smithi092.stdout:5/29: stat d4/d5/cc 0 2024-05-06T12:14:23.592 INFO:tasks.workunit.client.0.smithi092.stdout:0/49: write d7/dc/fe [920201,97951] 0 2024-05-06T12:14:23.596 INFO:tasks.workunit.client.0.smithi092.stdout:7/27: rename l8 to d9/lc 0 2024-05-06T12:14:23.598 INFO:tasks.workunit.client.0.smithi092.stdout:3/31: mknod d8/cc 0 2024-05-06T12:14:23.599 INFO:tasks.workunit.client.0.smithi092.stdout:5/30: dwrite d4/f9 [0,4194304] 0 2024-05-06T12:14:23.600 INFO:tasks.workunit.client.0.smithi092.stdout:3/32: truncate d8/fa 869001 0 2024-05-06T12:14:23.601 INFO:tasks.workunit.client.0.smithi092.stdout:4/50: creat d9/df/f15 x:0 0 0 2024-05-06T12:14:23.602 INFO:tasks.workunit.client.0.smithi092.stdout:3/33: write f4 [1084622,37587] 0 2024-05-06T12:14:23.604 INFO:tasks.workunit.client.0.smithi092.stdout:3/34: write f5 [297080,33725] 0 2024-05-06T12:14:23.607 INFO:tasks.workunit.client.0.smithi092.stdout:0/50: unlink l3 0 2024-05-06T12:14:23.607 INFO:tasks.workunit.client.0.smithi092.stdout:0/51: fsync f6 0 2024-05-06T12:14:23.609 INFO:tasks.workunit.client.0.smithi092.stdout:7/28: unlink f7 0 2024-05-06T12:14:23.610 INFO:tasks.workunit.client.0.smithi092.stdout:3/35: dread d8/fa [0,4194304] 0 2024-05-06T12:14:23.610 INFO:tasks.workunit.client.0.smithi092.stdout:7/29: dread - d9/fb zero size 2024-05-06T12:14:23.611 INFO:tasks.workunit.client.0.smithi092.stdout:5/31: mkdir d4/dd 0 2024-05-06T12:14:23.615 INFO:tasks.workunit.client.0.smithi092.stdout:6/45: readlink d4/la 0 2024-05-06T12:14:23.616 INFO:tasks.workunit.client.0.smithi092.stdout:6/46: truncate f1 4490574 0 2024-05-06T12:14:23.616 INFO:tasks.workunit.client.0.smithi092.stdout:6/47: fdatasync f1 0 2024-05-06T12:14:23.622 INFO:tasks.workunit.client.0.smithi092.stdout:5/32: symlink d4/le 0 2024-05-06T12:14:23.629 INFO:tasks.workunit.client.0.smithi092.stdout:0/52: creat d7/da/f11 x:0 0 0 2024-05-06T12:14:23.632 INFO:tasks.workunit.client.0.smithi092.stdout:7/30: unlink d9/lc 0 2024-05-06T12:14:23.634 INFO:tasks.workunit.client.0.smithi092.stdout:5/33: rename d4/le to d4/d5/lf 0 2024-05-06T12:14:23.637 INFO:tasks.workunit.client.0.smithi092.stdout:3/36: truncate f4 650683 0 2024-05-06T12:14:23.638 INFO:tasks.workunit.client.0.smithi092.stdout:5/34: dread d4/f9 [0,4194304] 0 2024-05-06T12:14:23.639 INFO:tasks.workunit.client.0.smithi092.stdout:7/31: mkdir d9/dd 0 2024-05-06T12:14:23.639 INFO:tasks.workunit.client.0.smithi092.stdout:7/32: dread - f3 zero size 2024-05-06T12:14:23.640 INFO:tasks.workunit.client.0.smithi092.stdout:4/51: getdents d9/dd 0 2024-05-06T12:14:23.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.639+0000 7fe1d3e22700 1 -- 172.21.15.92:0/1512740957 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe1cc11a020 con 0x7fe1cc0fb470 2024-05-06T12:14:23.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.639+0000 7fe1c27fc700 1 -- 172.21.15.92:0/1512740957 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1850 (secure 0 0 0) 0x7fe1c4025750 con 0x7fe1cc0fb470 2024-05-06T12:14:23.642 INFO:teuthology.orchestra.run.smithi092.stdout:e19 2024-05-06T12:14:23.642 INFO:teuthology.orchestra.run.smithi092.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-06T12:14:23.642 INFO:teuthology.orchestra.run.smithi092.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:14:23.643 INFO:teuthology.orchestra.run.smithi092.stdout:legacy client fscid: 1 2024-05-06T12:14:23.643 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:14:23.643 INFO:teuthology.orchestra.run.smithi092.stdout:Filesystem 'cephfs' (1) 2024-05-06T12:14:23.643 INFO:teuthology.orchestra.run.smithi092.stdout:fs_name cephfs 2024-05-06T12:14:23.643 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 19 2024-05-06T12:14:23.643 INFO:teuthology.orchestra.run.smithi092.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-06T12:14:23.643 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-05-06T12:11:44.952671+0000 2024-05-06T12:14:23.643 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2024-05-06T12:13:52.244842+0000 2024-05-06T12:14:23.643 INFO:teuthology.orchestra.run.smithi092.stdout:tableserver 0 2024-05-06T12:14:23.643 INFO:teuthology.orchestra.run.smithi092.stdout:root 0 2024-05-06T12:14:23.643 INFO:teuthology.orchestra.run.smithi092.stdout:session_timeout 60 2024-05-06T12:14:23.643 INFO:teuthology.orchestra.run.smithi092.stdout:session_autoclose 300 2024-05-06T12:14:23.643 INFO:teuthology.orchestra.run.smithi092.stdout:max_file_size 1099511627776 2024-05-06T12:14:23.643 INFO:teuthology.orchestra.run.smithi092.stdout:required_client_features {} 2024-05-06T12:14:23.643 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure 0 2024-05-06T12:14:23.644 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure_osd_epoch 46 2024-05-06T12:14:23.644 INFO:teuthology.orchestra.run.smithi092.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:14:23.644 INFO:teuthology.orchestra.run.smithi092.stdout:max_mds 2 2024-05-06T12:14:23.644 INFO:teuthology.orchestra.run.smithi092.stdout:in 0,1 2024-05-06T12:14:23.644 INFO:teuthology.orchestra.run.smithi092.stdout:up {0=14472,1=24267} 2024-05-06T12:14:23.644 INFO:teuthology.orchestra.run.smithi092.stdout:failed 2024-05-06T12:14:23.644 INFO:teuthology.orchestra.run.smithi092.stdout:damaged 2024-05-06T12:14:23.644 INFO:teuthology.orchestra.run.smithi092.stdout:stopped 2024-05-06T12:14:23.644 INFO:teuthology.orchestra.run.smithi092.stdout:data_pools [3] 2024-05-06T12:14:23.644 INFO:teuthology.orchestra.run.smithi092.stdout:metadata_pool 2 2024-05-06T12:14:23.644 INFO:teuthology.orchestra.run.smithi092.stdout:inline_data enabled 2024-05-06T12:14:23.644 INFO:teuthology.orchestra.run.smithi092.stdout:balancer 2024-05-06T12:14:23.644 INFO:teuthology.orchestra.run.smithi092.stdout:standby_count_wanted 1 2024-05-06T12:14:23.644 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.rosbll{0:14472} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.92:6828/1663253598,v1:172.21.15.92:6829/1663253598] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:14:23.645 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.ijsqgp{0:14484} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.92:6826/3613441054,v1:172.21.15.92:6827/3613441054] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:14:23.645 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.byjvgf{1:24267} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.190:6824/1565151699,v1:172.21.15.190:6825/1565151699] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:14:23.645 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.idzoff{1:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.190:6826/2453728380,v1:172.21.15.190:6827/2453728380] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:14:23.645 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:14:23.645 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:14:23.645 INFO:tasks.workunit.client.0.smithi092.stdout:0/53: truncate f6 627543 0 2024-05-06T12:14:23.645 INFO:tasks.workunit.client.0.smithi092.stdout:0/54: chown d7 24 1 2024-05-06T12:14:23.646 INFO:tasks.workunit.client.0.smithi092.stdout:5/35: mkdir d4/d5/d10 0 2024-05-06T12:14:23.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.643+0000 7fe1d3e22700 1 -- 172.21.15.92:0/1512740957 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fe1b806b8a0 msgr2=0x7fe1b806dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:23.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.643+0000 7fe1d3e22700 1 --2- 172.21.15.92:0/1512740957 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fe1b806b8a0 0x7fe1b806dd50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc004de0 tx=0x7fe1bc00d040 comp rx=0 tx=0).stop 2024-05-06T12:14:23.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.643+0000 7fe1d3e22700 1 -- 172.21.15.92:0/1512740957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1cc0fb470 msgr2=0x7fe1cc113ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:23.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.643+0000 7fe1d3e22700 1 --2- 172.21.15.92:0/1512740957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1cc0fb470 0x7fe1cc113ea0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe1c40199f0 tx=0x7fe1c400c040 comp rx=0 tx=0).stop 2024-05-06T12:14:23.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.643+0000 7fe1d3e22700 1 -- 172.21.15.92:0/1512740957 shutdown_connections 2024-05-06T12:14:23.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.643+0000 7fe1d3e22700 1 --2- 172.21.15.92:0/1512740957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1cc0fb470 0x7fe1cc113ea0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:23.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.643+0000 7fe1d3e22700 1 --2- 172.21.15.92:0/1512740957 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fe1b806b8a0 0x7fe1b806dd50 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:23.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.643+0000 7fe1d3e22700 1 --2- 172.21.15.92:0/1512740957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1cc0fc720 0x7fe1cc1143e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:23.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.643+0000 7fe1d3e22700 1 -- 172.21.15.92:0/1512740957 >> 172.21.15.92:0/1512740957 conn(0x7fe1cc0f6bb0 msgr2=0x7fe1cc0f81d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:14:23.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.644+0000 7fe1d3e22700 1 -- 172.21.15.92:0/1512740957 shutdown_connections 2024-05-06T12:14:23.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.644+0000 7fe1d3e22700 1 -- 172.21.15.92:0/1512740957 wait complete. 2024-05-06T12:14:23.648 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 19 2024-05-06T12:14:23.649 INFO:tasks.workunit.client.0.smithi092.stdout:0/55: truncate d7/f9 1459218 0 2024-05-06T12:14:23.649 INFO:tasks.workunit.client.0.smithi092.stdout:6/48: dwrite d4/fb [0,4194304] 0 2024-05-06T12:14:23.650 INFO:tasks.workunit.client.0.smithi092.stdout:0/56: chown c5 2219 1 2024-05-06T12:14:23.650 INFO:tasks.workunit.client.0.smithi092.stdout:0/57: fsync f1 0 2024-05-06T12:14:23.650 INFO:tasks.workunit.client.0.smithi092.stdout:7/33: mkdir d9/de 0 2024-05-06T12:14:23.652 INFO:tasks.workunit.client.0.smithi092.stdout:0/58: write d7/f9 [1549842,24264] 0 2024-05-06T12:14:23.660 INFO:tasks.workunit.client.0.smithi092.stdout:3/37: dread f4 [0,4194304] 0 2024-05-06T12:14:23.662 INFO:tasks.workunit.client.0.smithi092.stdout:7/34: dwrite f3 [0,4194304] 0 2024-05-06T12:14:23.665 INFO:tasks.workunit.client.0.smithi092.stdout:0/59: fdatasync f6 0 2024-05-06T12:14:23.667 INFO:tasks.workunit.client.0.smithi092.stdout:5/36: rename d4/d5/l6 to d4/dd/l11 0 2024-05-06T12:14:23.671 INFO:tasks.workunit.client.0.smithi092.stdout:7/35: dwrite f0 [0,4194304] 0 2024-05-06T12:14:23.676 INFO:tasks.workunit.client.0.smithi092.stdout:0/60: mkdir d7/da/d12 0 2024-05-06T12:14:23.678 INFO:tasks.workunit.client.0.smithi092.stdout:3/38: mknod d8/d9/cd 0 2024-05-06T12:14:23.681 INFO:tasks.workunit.client.0.smithi092.stdout:5/37: mkdir d4/d5/d10/d12 0 2024-05-06T12:14:23.682 INFO:tasks.workunit.client.0.smithi092.stdout:6/49: getdents d4 0 2024-05-06T12:14:23.683 INFO:tasks.workunit.client.0.smithi092.stdout:3/39: mknod d8/ce 0 2024-05-06T12:14:23.683 INFO:tasks.workunit.client.0.smithi092.stdout:6/50: chown d4/c9 3542813 1 2024-05-06T12:14:23.684 INFO:tasks.workunit.client.0.smithi092.stdout:6/51: write d4/f6 [651117,34056] 0 2024-05-06T12:14:23.685 INFO:tasks.workunit.client.0.smithi092.stdout:7/36: rename l6 to d9/lf 0 2024-05-06T12:14:23.685 INFO:tasks.workunit.client.0.smithi092.stdout:7/37: chown d9 147 1 2024-05-06T12:14:23.686 INFO:tasks.workunit.client.0.smithi092.stdout:5/38: truncate d4/f9 24573 0 2024-05-06T12:14:23.687 INFO:tasks.workunit.client.0.smithi092.stdout:3/40: dread d8/fa [0,4194304] 0 2024-05-06T12:14:23.688 INFO:tasks.workunit.client.0.smithi092.stdout:0/61: rename d7/dc/fe to d7/da/f13 0 2024-05-06T12:14:23.689 INFO:tasks.workunit.client.0.smithi092.stdout:6/52: rename d4 to d4/de 22 2024-05-06T12:14:23.689 INFO:tasks.workunit.client.0.smithi092.stdout:3/41: read f3 [538129,116120] 0 2024-05-06T12:14:23.689 INFO:tasks.workunit.client.0.smithi092.stdout:3/42: stat d8 0 2024-05-06T12:14:23.689 INFO:tasks.workunit.client.0.smithi092.stdout:6/53: stat d4/cd 0 2024-05-06T12:14:23.690 INFO:tasks.workunit.client.0.smithi092.stdout:0/62: dread - d7/da/f11 zero size 2024-05-06T12:14:23.691 INFO:tasks.workunit.client.0.smithi092.stdout:7/38: mkdir d9/de/d10 0 2024-05-06T12:14:23.692 INFO:tasks.workunit.client.0.smithi092.stdout:7/39: read f0 [2613839,46872] 0 2024-05-06T12:14:23.693 INFO:tasks.workunit.client.0.smithi092.stdout:7/40: fdatasync f3 0 2024-05-06T12:14:23.693 INFO:tasks.workunit.client.0.smithi092.stdout:7/41: fdatasync f0 0 2024-05-06T12:14:23.693 INFO:tasks.workunit.client.0.smithi092.stdout:6/54: symlink d4/lf 0 2024-05-06T12:14:23.694 INFO:tasks.workunit.client.0.smithi092.stdout:7/42: dread - d9/fb zero size 2024-05-06T12:14:23.696 INFO:tasks.workunit.client.0.smithi092.stdout:3/43: rename d8/cc to d8/d9/cf 0 2024-05-06T12:14:23.701 INFO:tasks.workunit.client.0.smithi092.stdout:0/63: creat d7/da/d12/f14 x:0 0 0 2024-05-06T12:14:23.701 INFO:tasks.workunit.client.0.smithi092.stdout:7/43: rmdir d9 39 2024-05-06T12:14:23.702 INFO:tasks.workunit.client.0.smithi092.stdout:0/64: readlink d7/l8 0 2024-05-06T12:14:23.703 INFO:tasks.workunit.client.0.smithi092.stdout:5/39: read d4/f9 [22365,13026] 0 2024-05-06T12:14:23.704 INFO:tasks.workunit.client.0.smithi092.stdout:5/40: dread d4/f9 [0,4194304] 0 2024-05-06T12:14:23.705 INFO:tasks.workunit.client.0.smithi092.stdout:0/65: dread f6 [0,4194304] 0 2024-05-06T12:14:23.706 INFO:tasks.workunit.client.0.smithi092.stdout:0/66: read - d7/da/d12/f14 zero size 2024-05-06T12:14:23.710 INFO:tasks.workunit.client.0.smithi092.stdout:5/41: mknod d4/dd/c13 0 2024-05-06T12:14:23.711 INFO:tasks.workunit.client.0.smithi092.stdout:5/42: dread d4/f9 [0,4194304] 0 2024-05-06T12:14:23.712 INFO:tasks.workunit.client.0.smithi092.stdout:0/67: write d7/da/f13 [1751063,38670] 0 2024-05-06T12:14:23.713 INFO:tasks.workunit.client.0.smithi092.stdout:5/43: dread d4/f9 [0,4194304] 0 2024-05-06T12:14:23.717 INFO:tasks.workunit.client.0.smithi092.stdout:7/44: mknod d9/dd/c11 0 2024-05-06T12:14:23.718 INFO:tasks.workunit.client.0.smithi092.stdout:0/68: dread d7/f9 [0,4194304] 0 2024-05-06T12:14:23.719 INFO:tasks.workunit.client.0.smithi092.stdout:5/44: symlink d4/dd/l14 0 2024-05-06T12:14:23.720 INFO:tasks.workunit.client.0.smithi092.stdout:6/55: symlink d4/l10 0 2024-05-06T12:14:23.721 INFO:tasks.workunit.client.0.smithi092.stdout:3/44: link d8/d9/cf d8/d9/c10 0 2024-05-06T12:14:23.721 INFO:tasks.workunit.client.0.smithi092.stdout:3/45: chown l6 3701 1 2024-05-06T12:14:23.722 INFO:tasks.workunit.client.0.smithi092.stdout:7/45: write d9/fb [218387,81857] 0 2024-05-06T12:14:23.723 INFO:tasks.workunit.client.0.smithi092.stdout:0/69: mknod d7/c15 0 2024-05-06T12:14:23.726 INFO:tasks.workunit.client.0.smithi092.stdout:6/56: write d4/fb [1938528,20936] 0 2024-05-06T12:14:23.727 INFO:tasks.workunit.client.0.smithi092.stdout:0/70: creat d7/f16 x:0 0 0 2024-05-06T12:14:23.728 INFO:tasks.workunit.client.0.smithi092.stdout:0/71: chown d7/da/f13 59 1 2024-05-06T12:14:23.731 INFO:tasks.workunit.client.0.smithi092.stdout:3/46: creat d8/db/f11 x:0 0 0 2024-05-06T12:14:23.732 INFO:tasks.workunit.client.0.smithi092.stdout:7/46: creat d9/de/d10/f12 x:0 0 0 2024-05-06T12:14:23.737 INFO:tasks.workunit.client.0.smithi092.stdout:6/57: creat d4/f11 x:0 0 0 2024-05-06T12:14:23.738 INFO:tasks.workunit.client.0.smithi092.stdout:6/58: truncate d4/f11 387028 0 2024-05-06T12:14:23.741 INFO:tasks.workunit.client.0.smithi092.stdout:7/47: getdents d9/de 0 2024-05-06T12:14:23.741 INFO:tasks.workunit.client.0.smithi092.stdout:7/48: readlink d9/la 0 2024-05-06T12:14:23.742 INFO:tasks.workunit.client.0.smithi092.stdout:7/49: read - d9/de/d10/f12 zero size 2024-05-06T12:14:23.743 INFO:tasks.workunit.client.0.smithi092.stdout:6/59: mknod d4/c12 0 2024-05-06T12:14:23.743 INFO:tasks.workunit.client.0.smithi092.stdout:7/50: write f3 [4275070,82559] 0 2024-05-06T12:14:23.744 INFO:tasks.workunit.client.0.smithi092.stdout:7/51: fdatasync d9/de/d10/f12 0 2024-05-06T12:14:23.745 INFO:tasks.workunit.client.0.smithi092.stdout:7/52: readlink d9/la 0 2024-05-06T12:14:23.747 INFO:tasks.workunit.client.0.smithi092.stdout:7/53: chown d9/de/d10/f12 5 1 2024-05-06T12:14:23.749 INFO:tasks.workunit.client.0.smithi092.stdout:3/47: write f3 [298584,98146] 0 2024-05-06T12:14:23.750 INFO:tasks.workunit.client.0.smithi092.stdout:6/60: dread d4/f6 [0,4194304] 0 2024-05-06T12:14:23.751 INFO:tasks.workunit.client.0.smithi092.stdout:3/48: chown d8/ce 7123 1 2024-05-06T12:14:23.753 INFO:tasks.workunit.client.0.smithi092.stdout:5/45: dwrite f0 [0,4194304] 0 2024-05-06T12:14:23.758 INFO:tasks.workunit.client.0.smithi092.stdout:6/61: symlink d4/l13 0 2024-05-06T12:14:23.761 INFO:tasks.workunit.client.0.smithi092.stdout:5/46: link d4/ca d4/d5/c15 0 2024-05-06T12:14:23.762 INFO:tasks.workunit.client.0.smithi092.stdout:3/49: getdents d8/db 0 2024-05-06T12:14:23.762 INFO:tasks.workunit.client.0.smithi092.stdout:7/54: dwrite f1 [0,4194304] 0 2024-05-06T12:14:23.763 INFO:tasks.workunit.client.0.smithi092.stdout:3/50: dread - d8/db/f11 zero size 2024-05-06T12:14:23.764 INFO:tasks.workunit.client.0.smithi092.stdout:7/55: dread - d9/de/d10/f12 zero size 2024-05-06T12:14:23.766 INFO:tasks.workunit.client.0.smithi092.stdout:6/62: creat d4/f14 x:0 0 0 2024-05-06T12:14:23.767 INFO:tasks.workunit.client.0.smithi092.stdout:3/51: creat d8/f12 x:0 0 0 2024-05-06T12:14:23.768 INFO:tasks.workunit.client.0.smithi092.stdout:5/47: dread f0 [0,4194304] 0 2024-05-06T12:14:23.768 INFO:tasks.workunit.client.0.smithi092.stdout:7/56: mkdir d9/dd/d13 0 2024-05-06T12:14:23.773 INFO:tasks.workunit.client.0.smithi092.stdout:7/57: creat d9/dd/d13/f14 x:0 0 0 2024-05-06T12:14:23.779 INFO:tasks.workunit.client.0.smithi092.stdout:6/63: getdents d4 0 2024-05-06T12:14:23.780 INFO:tasks.workunit.client.0.smithi092.stdout:3/52: write d8/fa [1565999,111547] 0 2024-05-06T12:14:23.781 INFO:tasks.workunit.client.0.smithi092.stdout:5/48: write d4/f9 [222558,98732] 0 2024-05-06T12:14:23.781 INFO:tasks.workunit.client.0.smithi092.stdout:6/64: symlink d4/l15 0 2024-05-06T12:14:23.782 INFO:tasks.workunit.client.0.smithi092.stdout:3/53: read - d8/db/f11 zero size 2024-05-06T12:14:23.782 INFO:tasks.workunit.client.0.smithi092.stdout:3/54: readlink l6 0 2024-05-06T12:14:23.783 INFO:tasks.workunit.client.0.smithi092.stdout:3/55: chown d8/d9/cd 1 1 2024-05-06T12:14:23.783 INFO:tasks.workunit.client.0.smithi092.stdout:7/58: rename d9/dd/d13 to d9/dd/d15 0 2024-05-06T12:14:23.785 INFO:tasks.workunit.client.0.smithi092.stdout:5/49: chown d4/ca 61718838 1 2024-05-06T12:14:23.785 INFO:tasks.workunit.client.0.smithi092.stdout:5/50: chown d4 297 1 2024-05-06T12:14:23.787 INFO:tasks.workunit.client.0.smithi092.stdout:6/65: mkdir d4/d16 0 2024-05-06T12:14:23.788 INFO:tasks.workunit.client.0.smithi092.stdout:5/51: write d4/f9 [697097,98960] 0 2024-05-06T12:14:23.792 INFO:tasks.workunit.client.0.smithi092.stdout:5/52: write d4/f9 [1325842,44770] 0 2024-05-06T12:14:23.796 INFO:tasks.workunit.client.0.smithi092.stdout:7/59: dwrite f0 [0,4194304] 0 2024-05-06T12:14:23.796 INFO:tasks.workunit.client.0.smithi092.stdout:3/56: dwrite d8/db/f11 [0,4194304] 0 2024-05-06T12:14:23.800 INFO:tasks.workunit.client.0.smithi092.stdout:6/66: rename d4/f14 to d4/f17 0 2024-05-06T12:14:23.804 INFO:tasks.workunit.client.0.smithi092.stdout:3/57: mkdir d8/d13 0 2024-05-06T12:14:23.807 INFO:tasks.workunit.client.0.smithi092.stdout:7/60: dwrite d9/dd/d15/f14 [0,4194304] 0 2024-05-06T12:14:23.809 INFO:tasks.workunit.client.0.smithi092.stdout:6/67: symlink d4/l18 0 2024-05-06T12:14:23.811 INFO:tasks.workunit.client.0.smithi092.stdout:7/61: mknod d9/de/c16 0 2024-05-06T12:14:23.812 INFO:tasks.workunit.client.0.smithi092.stdout:3/58: dread f5 [0,4194304] 0 2024-05-06T12:14:23.813 INFO:tasks.workunit.client.0.smithi092.stdout:7/62: write f3 [1604303,65789] 0 2024-05-06T12:14:23.815 INFO:tasks.workunit.client.0.smithi092.stdout:6/68: unlink d4/la 0 2024-05-06T12:14:23.816 INFO:tasks.workunit.client.0.smithi092.stdout:3/59: mkdir d8/db/d14 0 2024-05-06T12:14:23.817 INFO:tasks.workunit.client.0.smithi092.stdout:7/63: mkdir d9/de/d10/d17 0 2024-05-06T12:14:23.818 INFO:tasks.workunit.client.0.smithi092.stdout:7/64: dread - d9/de/d10/f12 zero size 2024-05-06T12:14:23.821 INFO:tasks.workunit.client.0.smithi092.stdout:3/60: fsync d8/db/f11 0 2024-05-06T12:14:23.823 INFO:tasks.workunit.client.0.smithi092.stdout:7/65: creat d9/de/f18 x:0 0 0 2024-05-06T12:14:23.824 INFO:tasks.workunit.client.0.smithi092.stdout:5/53: truncate f0 641005 0 2024-05-06T12:14:23.825 INFO:tasks.workunit.client.0.smithi092.stdout:6/69: creat d4/f19 x:0 0 0 2024-05-06T12:14:23.826 INFO:tasks.workunit.client.0.smithi092.stdout:3/61: fsync f5 0 2024-05-06T12:14:23.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.827+0000 7fe47759e700 1 -- 172.21.15.92:0/3939842660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe470096200 msgr2=0x7fe470096610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:23.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.827+0000 7fe47759e700 1 --2- 172.21.15.92:0/3939842660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe470096200 0x7fe470096610 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe468004940 tx=0x7fe468009e30 comp rx=0 tx=0).stop 2024-05-06T12:14:23.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.827+0000 7fe47759e700 1 -- 172.21.15.92:0/3939842660 shutdown_connections 2024-05-06T12:14:23.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.827+0000 7fe47759e700 1 --2- 172.21.15.92:0/3939842660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe470096b50 0x7fe470092c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:23.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.827+0000 7fe47759e700 1 --2- 172.21.15.92:0/3939842660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe470096200 0x7fe470096610 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:23.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.827+0000 7fe47759e700 1 -- 172.21.15.92:0/3939842660 >> 172.21.15.92:0/3939842660 conn(0x7fe47008e980 msgr2=0x7fe470090dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:14:23.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.827+0000 7fe47759e700 1 -- 172.21.15.92:0/3939842660 shutdown_connections 2024-05-06T12:14:23.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.827+0000 7fe47759e700 1 -- 172.21.15.92:0/3939842660 wait complete. 2024-05-06T12:14:23.831 INFO:tasks.workunit.client.0.smithi092.stdout:3/62: dread d8/db/f11 [0,4194304] 0 2024-05-06T12:14:23.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.828+0000 7fe47759e700 1 Processor -- start 2024-05-06T12:14:23.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.828+0000 7fe47759e700 1 -- start start 2024-05-06T12:14:23.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.828+0000 7fe47759e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe470096200 0x7fe470095810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:23.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.828+0000 7fe47759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe470096b50 0x7fe470093ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:23.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.828+0000 7fe47759e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe470095e10 con 0x7fe470096b50 2024-05-06T12:14:23.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.828+0000 7fe47759e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4700a9380 con 0x7fe470096200 2024-05-06T12:14:23.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.828+0000 7fe47659c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe470096200 0x7fe470095810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:23.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.828+0000 7fe47659c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe470096200 0x7fe470095810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.92:41460/0 (socket says 172.21.15.92:41460) 2024-05-06T12:14:23.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.828+0000 7fe47659c700 1 -- 172.21.15.92:0/515870211 learned_addr learned my addr 172.21.15.92:0/515870211 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:14:23.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.829+0000 7fe475d9b700 1 --2- 172.21.15.92:0/515870211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe470096b50 0x7fe470093ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:23.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.829+0000 7fe47659c700 1 -- 172.21.15.92:0/515870211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe470096b50 msgr2=0x7fe470093ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:23.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.829+0000 7fe47659c700 1 --2- 172.21.15.92:0/515870211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe470096b50 0x7fe470093ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:23.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.829+0000 7fe47659c700 1 -- 172.21.15.92:0/515870211 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46c005040 con 0x7fe470096200 2024-05-06T12:14:23.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.829+0000 7fe47659c700 1 --2- 172.21.15.92:0/515870211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe470096200 0x7fe470095810 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe468009fd0 tx=0x7fe468009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:14:23.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.829+0000 7fe466ffd700 1 -- 172.21.15.92:0/515870211 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe46800e030 con 0x7fe470096200 2024-05-06T12:14:23.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.829+0000 7fe466ffd700 1 -- 172.21.15.92:0/515870211 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe46801c400 con 0x7fe470096200 2024-05-06T12:14:23.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.829+0000 7fe466ffd700 1 -- 172.21.15.92:0/515870211 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe46800b650 con 0x7fe470096200 2024-05-06T12:14:23.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.829+0000 7fe47759e700 1 -- 172.21.15.92:0/515870211 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe468005040 con 0x7fe470096200 2024-05-06T12:14:23.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.829+0000 7fe47759e700 1 -- 172.21.15.92:0/515870211 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe470094690 con 0x7fe470096200 2024-05-06T12:14:23.834 INFO:tasks.workunit.client.0.smithi092.stdout:6/70: creat d4/f1a x:0 0 0 2024-05-06T12:14:23.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.831+0000 7fe47759e700 1 -- 172.21.15.92:0/515870211 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe470005550 con 0x7fe470096200 2024-05-06T12:14:23.835 INFO:tasks.workunit.client.0.smithi092.stdout:7/66: getdents d9/dd 0 2024-05-06T12:14:23.835 INFO:tasks.workunit.client.0.smithi092.stdout:6/71: write d4/f19 [113875,110355] 0 2024-05-06T12:14:23.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.831+0000 7fe466ffd700 1 -- 172.21.15.92:0/515870211 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7fe46800b8c0 con 0x7fe470096200 2024-05-06T12:14:23.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.832+0000 7fe466ffd700 1 --2- 172.21.15.92:0/515870211 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fe46006b790 0x7fe46006dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:23.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.832+0000 7fe466ffd700 1 -- 172.21.15.92:0/515870211 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fe46808bd10 con 0x7fe470096200 2024-05-06T12:14:23.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.832+0000 7fe475d9b700 1 --2- 172.21.15.92:0/515870211 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fe46006b790 0x7fe46006dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:23.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.833+0000 7fe475d9b700 1 --2- 172.21.15.92:0/515870211 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fe46006b790 0x7fe46006dc40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe46c009fd0 tx=0x7fe46c00b040 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:14:23.836 INFO:tasks.workunit.client.0.smithi092.stdout:7/67: read f3 [1422663,46444] 0 2024-05-06T12:14:23.838 INFO:tasks.workunit.client.0.smithi092.stdout:7/68: truncate d9/de/d10/f12 365278 0 2024-05-06T12:14:23.839 INFO:tasks.workunit.client.0.smithi092.stdout:3/63: dwrite f3 [0,4194304] 0 2024-05-06T12:14:23.840 INFO:tasks.workunit.client.0.smithi092.stdout:7/69: chown d9/fb 3444162 1 2024-05-06T12:14:23.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:23.838+0000 7fe466ffd700 1 -- 172.21.15.92:0/515870211 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fe468059660 con 0x7fe470096200 2024-05-06T12:14:23.844 INFO:tasks.workunit.client.0.smithi092.stdout:7/70: symlink d9/l19 0 2024-05-06T12:14:23.849 INFO:tasks.workunit.client.0.smithi092.stdout:6/72: getdents d4 0 2024-05-06T12:14:23.853 INFO:tasks.workunit.client.0.smithi092.stdout:3/64: dwrite d8/fa [0,4194304] 0 2024-05-06T12:14:23.855 INFO:tasks.workunit.client.0.smithi092.stdout:3/65: chown d8/d9/cd 31 1 2024-05-06T12:14:23.856 INFO:tasks.workunit.client.0.smithi092.stdout:6/73: creat d4/d16/f1b x:0 0 0 2024-05-06T12:14:23.857 INFO:tasks.workunit.client.0.smithi092.stdout:7/71: dwrite d9/fb [0,4194304] 0 2024-05-06T12:14:23.859 INFO:tasks.workunit.client.0.smithi092.stdout:7/72: fdatasync d9/de/d10/f12 0 2024-05-06T12:14:23.859 INFO:tasks.workunit.client.0.smithi092.stdout:6/74: dread f1 [4194304,4194304] 0 2024-05-06T12:14:23.860 INFO:tasks.workunit.client.0.smithi092.stdout:6/75: write d4/f1a [748891,18423] 0 2024-05-06T12:14:23.861 INFO:tasks.workunit.client.0.smithi092.stdout:7/73: symlink d9/dd/l1a 0 2024-05-06T12:14:23.863 INFO:tasks.workunit.client.0.smithi092.stdout:6/76: creat d4/d16/f1c x:0 0 0 2024-05-06T12:14:23.864 INFO:tasks.workunit.client.0.smithi092.stdout:7/74: unlink f0 0 2024-05-06T12:14:23.869 INFO:tasks.workunit.client.0.smithi092.stdout:6/77: getdents d4 0 2024-05-06T12:14:23.870 INFO:tasks.workunit.client.0.smithi092.stdout:7/75: link d9/la d9/l1b 0 2024-05-06T12:14:23.871 INFO:tasks.workunit.client.0.smithi092.stdout:6/78: dread d4/f6 [0,4194304] 0 2024-05-06T12:14:23.872 INFO:tasks.workunit.client.0.smithi092.stdout:6/79: read - d4/d16/f1b zero size 2024-05-06T12:14:23.872 INFO:tasks.workunit.client.0.smithi092.stdout:7/76: symlink d9/de/l1c 0 2024-05-06T12:14:23.875 INFO:tasks.workunit.client.0.smithi092.stdout:7/77: symlink d9/dd/l1d 0 2024-05-06T12:14:23.877 INFO:tasks.workunit.client.0.smithi092.stdout:6/80: truncate f1 1646473 0 2024-05-06T12:14:23.878 INFO:tasks.workunit.client.0.smithi092.stdout:6/81: write d4/d16/f1c [448259,103810] 0 2024-05-06T12:14:23.879 INFO:tasks.workunit.client.0.smithi092.stdout:7/78: creat d9/dd/f1e x:0 0 0 2024-05-06T12:14:23.879 INFO:tasks.workunit.client.0.smithi092.stdout:7/79: readlink d9/l19 0 2024-05-06T12:14:23.882 INFO:tasks.workunit.client.0.smithi092.stdout:7/80: mkdir d9/de/d1f 0 2024-05-06T12:14:23.888 INFO:tasks.workunit.client.0.smithi092.stdout:6/82: dwrite d4/f6 [0,4194304] 0 2024-05-06T12:14:23.894 INFO:tasks.workunit.client.0.smithi092.stdout:7/81: dwrite d9/dd/f1e [0,4194304] 0 2024-05-06T12:14:23.895 INFO:tasks.workunit.client.0.smithi092.stdout:6/83: mknod d4/d16/c1d 0 2024-05-06T12:14:23.895 INFO:tasks.workunit.client.0.smithi092.stdout:6/84: fsync d4/f11 0 2024-05-06T12:14:23.902 INFO:tasks.workunit.client.0.smithi092.stdout:6/85: creat d4/d16/f1e x:0 0 0 2024-05-06T12:14:23.904 INFO:tasks.workunit.client.0.smithi092.stdout:7/82: chown d9/la 9260952 1 2024-05-06T12:14:23.910 INFO:tasks.workunit.client.0.smithi092.stdout:7/83: dwrite d9/fb [0,4194304] 0 2024-05-06T12:14:23.911 INFO:tasks.workunit.client.0.smithi092.stdout:7/84: write d9/de/f18 [506471,40005] 0 2024-05-06T12:14:23.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:23 smithi092 ceph-mon[26954]: Adjusting osd_memory_target on smithi092 to 1557M 2024-05-06T12:14:23.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:23 smithi092 ceph-mon[26954]: from='client.24431 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:14:23.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:23 smithi092 ceph-mon[26954]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:14:23.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:23 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/4203096567' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:14:23.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:23 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/1512740957' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:14:24.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:23 smithi190 ceph-mon[33798]: Adjusting osd_memory_target on smithi092 to 1557M 2024-05-06T12:14:24.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:23 smithi190 ceph-mon[33798]: from='client.24431 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:14:24.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:23 smithi190 ceph-mon[33798]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:14:24.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:23 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/4203096567' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:14:24.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:23 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/1512740957' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:14:24.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.115+0000 7fe47759e700 1 -- 172.21.15.92:0/515870211 --> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe47009a1f0 con 0x7fe46006b790 2024-05-06T12:14:24.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.116+0000 7fe466ffd700 1 -- 172.21.15.92:0/515870211 <== mgr.14636 v2:172.21.15.190:6828/3484477933 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fe47009a1f0 con 0x7fe46006b790 2024-05-06T12:14:24.119 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:14:24.119 INFO:teuthology.orchestra.run.smithi092.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb", 2024-05-06T12:14:24.119 INFO:teuthology.orchestra.run.smithi092.stdout: "in_progress": true, 2024-05-06T12:14:24.119 INFO:teuthology.orchestra.run.smithi092.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-06T12:14:24.119 INFO:teuthology.orchestra.run.smithi092.stdout: "services_complete": [], 2024-05-06T12:14:24.119 INFO:teuthology.orchestra.run.smithi092.stdout: "progress": "1/21 daemons upgraded", 2024-05-06T12:14:24.120 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "", 2024-05-06T12:14:24.120 INFO:teuthology.orchestra.run.smithi092.stdout: "is_paused": false 2024-05-06T12:14:24.120 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:14:24.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.118+0000 7fe47759e700 1 -- 172.21.15.92:0/515870211 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fe46006b790 msgr2=0x7fe46006dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:24.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.118+0000 7fe47759e700 1 --2- 172.21.15.92:0/515870211 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fe46006b790 0x7fe46006dc40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe46c009fd0 tx=0x7fe46c00b040 comp rx=0 tx=0).stop 2024-05-06T12:14:24.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.118+0000 7fe47759e700 1 -- 172.21.15.92:0/515870211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe470096200 msgr2=0x7fe470095810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:24.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.118+0000 7fe47759e700 1 --2- 172.21.15.92:0/515870211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe470096200 0x7fe470095810 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe468009fd0 tx=0x7fe468009e30 comp rx=0 tx=0).stop 2024-05-06T12:14:24.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.118+0000 7fe47759e700 1 -- 172.21.15.92:0/515870211 shutdown_connections 2024-05-06T12:14:24.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.118+0000 7fe47759e700 1 --2- 172.21.15.92:0/515870211 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fe46006b790 0x7fe46006dc40 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:24.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.118+0000 7fe47759e700 1 --2- 172.21.15.92:0/515870211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe470096200 0x7fe470095810 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:24.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.118+0000 7fe47759e700 1 --2- 172.21.15.92:0/515870211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe470096b50 0x7fe470093ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:24.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.118+0000 7fe47759e700 1 -- 172.21.15.92:0/515870211 >> 172.21.15.92:0/515870211 conn(0x7fe47008e980 msgr2=0x7fe470090350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:14:24.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.118+0000 7fe47759e700 1 -- 172.21.15.92:0/515870211 shutdown_connections 2024-05-06T12:14:24.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.118+0000 7fe47759e700 1 -- 172.21.15.92:0/515870211 wait complete. 2024-05-06T12:14:24.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.226+0000 7f27d7cca700 1 -- 172.21.15.92:0/1770096224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27d00ff050 msgr2=0x7f27d00ff460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:24.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.226+0000 7f27d7cca700 1 --2- 172.21.15.92:0/1770096224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27d00ff050 0x7f27d00ff460 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f27cc004940 tx=0x7f27cc009e30 comp rx=0 tx=0).stop 2024-05-06T12:14:24.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.226+0000 7f27d7cca700 1 -- 172.21.15.92:0/1770096224 shutdown_connections 2024-05-06T12:14:24.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.226+0000 7f27d7cca700 1 --2- 172.21.15.92:0/1770096224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27d0100300 0x7f27d0100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:24.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.226+0000 7f27d7cca700 1 --2- 172.21.15.92:0/1770096224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27d00ff050 0x7f27d00ff460 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:24.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.226+0000 7f27d7cca700 1 -- 172.21.15.92:0/1770096224 >> 172.21.15.92:0/1770096224 conn(0x7f27d00fa8f0 msgr2=0x7f27d00fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:14:24.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.226+0000 7f27d7cca700 1 -- 172.21.15.92:0/1770096224 shutdown_connections 2024-05-06T12:14:24.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.226+0000 7f27d7cca700 1 -- 172.21.15.92:0/1770096224 wait complete. 2024-05-06T12:14:24.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.226+0000 7f27d7cca700 1 Processor -- start 2024-05-06T12:14:24.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.226+0000 7f27d7cca700 1 -- start start 2024-05-06T12:14:24.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.226+0000 7f27d7cca700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27d00ff050 0x7f27d010fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:24.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.226+0000 7f27d7cca700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27d0100300 0x7f27d0110170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:24.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.226+0000 7f27d7cca700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d0110770 con 0x7f27d00ff050 2024-05-06T12:14:24.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.226+0000 7f27d7cca700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d0110fc0 con 0x7f27d0100300 2024-05-06T12:14:24.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.227+0000 7f27d5a66700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27d00ff050 0x7f27d010fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:24.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.227+0000 7f27d5a66700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27d00ff050 0x7f27d010fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50988/0 (socket says 172.21.15.92:50988) 2024-05-06T12:14:24.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.227+0000 7f27d5a66700 1 -- 172.21.15.92:0/2304802280 learned_addr learned my addr 172.21.15.92:0/2304802280 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:14:24.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.227+0000 7f27d5265700 1 --2- 172.21.15.92:0/2304802280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27d0100300 0x7f27d0110170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:24.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.227+0000 7f27d5a66700 1 -- 172.21.15.92:0/2304802280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27d0100300 msgr2=0x7f27d0110170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:24.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.227+0000 7f27d5a66700 1 --2- 172.21.15.92:0/2304802280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27d0100300 0x7f27d0110170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:24.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.227+0000 7f27d5a66700 1 -- 172.21.15.92:0/2304802280 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27cc005040 con 0x7f27d00ff050 2024-05-06T12:14:24.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.227+0000 7f27d5a66700 1 --2- 172.21.15.92:0/2304802280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27d00ff050 0x7f27d010fc30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f27cc00b3b0 tx=0x7f27cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:14:24.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.227+0000 7f27c67fc700 1 -- 172.21.15.92:0/2304802280 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27cc00e040 con 0x7f27d00ff050 2024-05-06T12:14:24.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.227+0000 7f27d7cca700 1 -- 172.21.15.92:0/2304802280 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27d0111240 con 0x7f27d00ff050 2024-05-06T12:14:24.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.227+0000 7f27c67fc700 1 -- 172.21.15.92:0/2304802280 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f27cc00bbf0 con 0x7f27d00ff050 2024-05-06T12:14:24.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.227+0000 7f27c67fc700 1 -- 172.21.15.92:0/2304802280 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27cc01c630 con 0x7f27d00ff050 2024-05-06T12:14:24.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.227+0000 7f27d7cca700 1 -- 172.21.15.92:0/2304802280 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d0111780 con 0x7f27d00ff050 2024-05-06T12:14:24.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.228+0000 7f27d7cca700 1 -- 172.21.15.92:0/2304802280 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27d005f5c0 con 0x7f27d00ff050 2024-05-06T12:14:24.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.229+0000 7f27c67fc700 1 -- 172.21.15.92:0/2304802280 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7f27cc00bd60 con 0x7f27d00ff050 2024-05-06T12:14:24.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.229+0000 7f27c67fc700 1 --2- 172.21.15.92:0/2304802280 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f27bc06b790 0x7f27bc06dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:24.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.229+0000 7f27c67fc700 1 -- 172.21.15.92:0/2304802280 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f27cc08afe0 con 0x7f27d00ff050 2024-05-06T12:14:24.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.229+0000 7f27d5265700 1 --2- 172.21.15.92:0/2304802280 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f27bc06b790 0x7f27bc06dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:24.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.230+0000 7f27d5265700 1 --2- 172.21.15.92:0/2304802280 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f27bc06b790 0x7f27bc06dc40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f27c0004f40 tx=0x7f27c000b040 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:14:24.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.234+0000 7f27c67fc700 1 -- 172.21.15.92:0/2304802280 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f27cc058770 con 0x7f27d00ff050 2024-05-06T12:14:24.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.537+0000 7f27d7cca700 1 -- 172.21.15.92:0/2304802280 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f27d004bed0 con 0x7f27d00ff050 2024-05-06T12:14:24.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.537+0000 7f27c67fc700 1 -- 172.21.15.92:0/2304802280 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f27cc058300 con 0x7f27d00ff050 2024-05-06T12:14:24.541 INFO:teuthology.orchestra.run.smithi092.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-06T12:14:24.541 INFO:teuthology.orchestra.run.smithi092.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-06T12:14:24.541 INFO:teuthology.orchestra.run.smithi092.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-06T12:14:24.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.540+0000 7f27d7cca700 1 -- 172.21.15.92:0/2304802280 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f27bc06b790 msgr2=0x7f27bc06dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:24.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.540+0000 7f27d7cca700 1 --2- 172.21.15.92:0/2304802280 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f27bc06b790 0x7f27bc06dc40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f27c0004f40 tx=0x7f27c000b040 comp rx=0 tx=0).stop 2024-05-06T12:14:24.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.540+0000 7f27d7cca700 1 -- 172.21.15.92:0/2304802280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27d00ff050 msgr2=0x7f27d010fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:24.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.540+0000 7f27d7cca700 1 --2- 172.21.15.92:0/2304802280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27d00ff050 0x7f27d010fc30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f27cc00b3b0 tx=0x7f27cc009e30 comp rx=0 tx=0).stop 2024-05-06T12:14:24.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.540+0000 7f27d7cca700 1 -- 172.21.15.92:0/2304802280 shutdown_connections 2024-05-06T12:14:24.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.540+0000 7f27d7cca700 1 --2- 172.21.15.92:0/2304802280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27d00ff050 0x7f27d010fc30 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:24.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.540+0000 7f27d7cca700 1 --2- 172.21.15.92:0/2304802280 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f27bc06b790 0x7f27bc06dc40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:24.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.540+0000 7f27d7cca700 1 --2- 172.21.15.92:0/2304802280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27d0100300 0x7f27d0110170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:24.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.540+0000 7f27d7cca700 1 -- 172.21.15.92:0/2304802280 >> 172.21.15.92:0/2304802280 conn(0x7f27d00fa8f0 msgr2=0x7f27d00fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:14:24.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.540+0000 7f27d7cca700 1 -- 172.21.15.92:0/2304802280 shutdown_connections 2024-05-06T12:14:24.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:24.540+0000 7f27d7cca700 1 -- 172.21.15.92:0/2304802280 wait complete. 2024-05-06T12:14:24.968 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:24 smithi092 ceph-mon[26954]: pgmap v14: 65 pgs: 65 active+clean; 195 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 3.4 MiB/s rd, 6.1 MiB/s wr, 433 op/s 2024-05-06T12:14:24.969 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:24 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/2304802280' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:14:25.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:24 smithi190 ceph-mon[33798]: pgmap v14: 65 pgs: 65 active+clean; 195 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 3.4 MiB/s rd, 6.1 MiB/s wr, 433 op/s 2024-05-06T12:14:25.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:24 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/2304802280' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:14:25.857 INFO:tasks.workunit.client.0.smithi092.stdout:9/5: sync 2024-05-06T12:14:25.857 INFO:tasks.workunit.client.0.smithi092.stdout:9/6: truncate f0 1170697 0 2024-05-06T12:14:25.858 INFO:tasks.workunit.client.0.smithi092.stdout:9/7: readlink - no filename 2024-05-06T12:14:25.859 INFO:tasks.workunit.client.0.smithi092.stdout:9/8: getdents . 0 2024-05-06T12:14:25.859 INFO:tasks.workunit.client.0.smithi092.stdout:9/9: truncate f0 1418596 0 2024-05-06T12:14:25.861 INFO:tasks.workunit.client.0.smithi092.stdout:9/10: rename f0 to f1 0 2024-05-06T12:14:25.910 INFO:tasks.workunit.client.0.smithi092.stdout:8/39: sync 2024-05-06T12:14:25.910 INFO:tasks.workunit.client.0.smithi092.stdout:5/54: sync 2024-05-06T12:14:25.910 INFO:tasks.workunit.client.0.smithi092.stdout:4/52: sync 2024-05-06T12:14:25.911 INFO:tasks.workunit.client.0.smithi092.stdout:0/72: sync 2024-05-06T12:14:25.911 INFO:tasks.workunit.client.0.smithi092.stdout:6/86: sync 2024-05-06T12:14:25.911 INFO:tasks.workunit.client.0.smithi092.stdout:3/66: sync 2024-05-06T12:14:25.911 INFO:tasks.workunit.client.0.smithi092.stdout:2/43: sync 2024-05-06T12:14:25.911 INFO:tasks.workunit.client.0.smithi092.stdout:7/85: sync 2024-05-06T12:14:25.911 INFO:tasks.workunit.client.0.smithi092.stdout:1/34: sync 2024-05-06T12:14:25.912 INFO:tasks.workunit.client.0.smithi092.stdout:6/87: chown d4/cc 29592750 1 2024-05-06T12:14:25.912 INFO:tasks.workunit.client.0.smithi092.stdout:3/67: truncate f5 1164404 0 2024-05-06T12:14:25.913 INFO:tasks.workunit.client.0.smithi092.stdout:8/40: fsync f7 0 2024-05-06T12:14:25.914 INFO:tasks.workunit.client.0.smithi092.stdout:8/41: fdatasync f6 0 2024-05-06T12:14:25.914 INFO:tasks.workunit.client.0.smithi092.stdout:3/68: chown d8/db/f11 191990 1 2024-05-06T12:14:25.917 INFO:tasks.workunit.client.0.smithi092.stdout:3/69: chown d8/f12 1 1 2024-05-06T12:14:25.917 INFO:tasks.workunit.client.0.smithi092.stdout:0/73: truncate d7/da/d12/f14 600723 0 2024-05-06T12:14:25.919 INFO:tasks.workunit.client.0.smithi092.stdout:6/88: mknod d4/d16/c1f 0 2024-05-06T12:14:25.919 INFO:tasks.workunit.client.0.smithi092.stdout:5/55: dread d4/f9 [0,4194304] 0 2024-05-06T12:14:25.919 INFO:tasks.workunit.client.0.smithi092.stdout:7/86: dread f1 [0,4194304] 0 2024-05-06T12:14:25.921 INFO:tasks.workunit.client.0.smithi092.stdout:7/87: truncate d9/de/f18 1326930 0 2024-05-06T12:14:25.922 INFO:tasks.workunit.client.0.smithi092.stdout:8/42: creat d9/fe x:0 0 0 2024-05-06T12:14:25.922 INFO:tasks.workunit.client.0.smithi092.stdout:8/43: chown f0 447678482 1 2024-05-06T12:14:25.924 INFO:tasks.workunit.client.0.smithi092.stdout:0/74: mknod d7/dc/c17 0 2024-05-06T12:14:25.925 INFO:tasks.workunit.client.0.smithi092.stdout:8/44: dread f8 [0,4194304] 0 2024-05-06T12:14:25.926 INFO:tasks.workunit.client.0.smithi092.stdout:5/56: read f0 [72553,48163] 0 2024-05-06T12:14:25.928 INFO:tasks.workunit.client.0.smithi092.stdout:7/88: mknod d9/dd/d15/c20 0 2024-05-06T12:14:25.931 INFO:tasks.workunit.client.0.smithi092.stdout:3/70: mkdir d8/db/d14/d15 0 2024-05-06T12:14:25.933 INFO:tasks.workunit.client.0.smithi092.stdout:3/71: write f2 [1997169,125025] 0 2024-05-06T12:14:25.933 INFO:tasks.workunit.client.0.smithi092.stdout:0/75: rename d7/cb to d7/c18 0 2024-05-06T12:14:25.934 INFO:tasks.workunit.client.0.smithi092.stdout:3/72: stat d8/ce 0 2024-05-06T12:14:25.935 INFO:tasks.workunit.client.0.smithi092.stdout:8/45: unlink c1 0 2024-05-06T12:14:25.936 INFO:tasks.workunit.client.0.smithi092.stdout:0/76: write d7/da/f13 [2784727,4674] 0 2024-05-06T12:14:25.937 INFO:tasks.workunit.client.0.smithi092.stdout:7/89: dwrite d9/fb [0,4194304] 0 2024-05-06T12:14:25.937 INFO:tasks.workunit.client.0.smithi092.stdout:7/90: chown d9/dd 1762850 1 2024-05-06T12:14:25.938 INFO:tasks.workunit.client.0.smithi092.stdout:5/57: fsync f0 0 2024-05-06T12:14:25.945 INFO:tasks.workunit.client.0.smithi092.stdout:0/77: creat d7/da/f19 x:0 0 0 2024-05-06T12:14:25.948 INFO:tasks.workunit.client.0.smithi092.stdout:3/73: creat d8/d13/f16 x:0 0 0 2024-05-06T12:14:25.949 INFO:tasks.workunit.client.0.smithi092.stdout:3/74: chown f2 2 1 2024-05-06T12:14:25.951 INFO:tasks.workunit.client.0.smithi092.stdout:7/91: mkdir d9/de/d10/d17/d21 0 2024-05-06T12:14:25.952 INFO:tasks.workunit.client.0.smithi092.stdout:0/78: mkdir d7/da/d12/d1a 0 2024-05-06T12:14:25.953 INFO:tasks.workunit.client.0.smithi092.stdout:5/58: symlink d4/d5/d10/d12/l16 0 2024-05-06T12:14:25.955 INFO:tasks.workunit.client.0.smithi092.stdout:3/75: unlink c7 0 2024-05-06T12:14:25.956 INFO:tasks.workunit.client.0.smithi092.stdout:8/46: rmdir d9/dd 0 2024-05-06T12:14:25.958 INFO:tasks.workunit.client.0.smithi092.stdout:3/76: truncate d8/f12 890834 0 2024-05-06T12:14:25.959 INFO:tasks.workunit.client.0.smithi092.stdout:3/77: chown f2 7 1 2024-05-06T12:14:25.959 INFO:tasks.workunit.client.0.smithi092.stdout:0/79: dwrite f1 [0,4194304] 0 2024-05-06T12:14:25.960 INFO:tasks.workunit.client.0.smithi092.stdout:5/59: link d4/f9 d4/d5/d10/f17 0 2024-05-06T12:14:25.962 INFO:tasks.workunit.client.0.smithi092.stdout:3/78: creat d8/d9/f17 x:0 0 0 2024-05-06T12:14:25.963 INFO:tasks.workunit.client.0.smithi092.stdout:3/79: fdatasync f3 0 2024-05-06T12:14:25.965 INFO:tasks.workunit.client.0.smithi092.stdout:8/47: creat d9/dc/ff x:0 0 0 2024-05-06T12:14:25.966 INFO:tasks.workunit.client.0.smithi092.stdout:5/60: creat d4/d5/d10/d12/f18 x:0 0 0 2024-05-06T12:14:25.967 INFO:tasks.workunit.client.0.smithi092.stdout:0/80: dwrite d7/da/f13 [0,4194304] 0 2024-05-06T12:14:25.967 INFO:tasks.workunit.client.0.smithi092.stdout:3/80: creat d8/db/d14/f18 x:0 0 0 2024-05-06T12:14:25.969 INFO:tasks.workunit.client.0.smithi092.stdout:8/48: symlink d9/dc/l10 0 2024-05-06T12:14:25.969 INFO:tasks.workunit.client.0.smithi092.stdout:5/61: write d4/d5/d10/d12/f18 [228458,65681] 0 2024-05-06T12:14:25.969 INFO:tasks.workunit.client.0.smithi092.stdout:3/81: chown d8/d13/f16 27 1 2024-05-06T12:14:25.970 INFO:tasks.workunit.client.0.smithi092.stdout:8/49: fsync f0 0 2024-05-06T12:14:25.970 INFO:tasks.workunit.client.0.smithi092.stdout:0/81: chown d7/dc/cd 3 1 2024-05-06T12:14:25.971 INFO:tasks.workunit.client.0.smithi092.stdout:0/82: chown f1 65940982 1 2024-05-06T12:14:25.971 INFO:tasks.workunit.client.0.smithi092.stdout:0/83: dread - d7/da/f11 zero size 2024-05-06T12:14:25.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:25 smithi092 ceph-mon[26954]: from='client.24447 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:14:25.972 INFO:tasks.workunit.client.0.smithi092.stdout:5/62: creat d4/f19 x:0 0 0 2024-05-06T12:14:25.973 INFO:tasks.workunit.client.0.smithi092.stdout:0/84: read f1 [2767492,123579] 0 2024-05-06T12:14:25.974 INFO:tasks.workunit.client.0.smithi092.stdout:3/82: symlink d8/l19 0 2024-05-06T12:14:25.980 INFO:tasks.workunit.client.0.smithi092.stdout:0/85: creat d7/da/d12/f1b x:0 0 0 2024-05-06T12:14:25.996 INFO:tasks.workunit.client.0.smithi092.stdout:3/83: dread f4 [0,4194304] 0 2024-05-06T12:14:25.996 INFO:tasks.workunit.client.0.smithi092.stdout:8/50: rename d9/la to d9/l11 0 2024-05-06T12:14:26.094 INFO:tasks.workunit.client.0.smithi092.stdout:2/44: sync 2024-05-06T12:14:26.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:25 smithi190 ceph-mon[33798]: from='client.24447 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:14:26.129 INFO:tasks.workunit.client.0.smithi092.stdout:7/92: sync 2024-05-06T12:14:26.129 INFO:tasks.workunit.client.0.smithi092.stdout:1/35: sync 2024-05-06T12:14:26.129 INFO:tasks.workunit.client.0.smithi092.stdout:7/93: fsync d9/de/d10/f12 0 2024-05-06T12:14:26.137 INFO:tasks.workunit.client.0.smithi092.stdout:7/94: dwrite d9/dd/d15/f14 [0,4194304] 0 2024-05-06T12:14:26.209 INFO:tasks.workunit.client.0.smithi092.stdout:3/84: mknod d8/d9/c1a 0 2024-05-06T12:14:26.210 INFO:tasks.workunit.client.0.smithi092.stdout:2/45: chown fc 646308 1 2024-05-06T12:14:26.211 INFO:tasks.workunit.client.0.smithi092.stdout:8/51: dread f8 [0,4194304] 0 2024-05-06T12:14:26.213 INFO:tasks.workunit.client.0.smithi092.stdout:0/86: creat d7/da/d12/d1a/f1c x:0 0 0 2024-05-06T12:14:26.214 INFO:tasks.workunit.client.0.smithi092.stdout:3/85: mkdir d8/d1b 0 2024-05-06T12:14:26.217 INFO:tasks.workunit.client.0.smithi092.stdout:2/46: dread - fd zero size 2024-05-06T12:14:26.218 INFO:tasks.workunit.client.0.smithi092.stdout:8/52: mknod d9/c12 0 2024-05-06T12:14:26.219 INFO:tasks.workunit.client.0.smithi092.stdout:7/95: symlink d9/de/d1f/l22 0 2024-05-06T12:14:26.221 INFO:tasks.workunit.client.0.smithi092.stdout:0/87: mknod d7/c1d 0 2024-05-06T12:14:26.223 INFO:tasks.workunit.client.0.smithi092.stdout:8/53: mknod d9/c13 0 2024-05-06T12:14:26.223 INFO:tasks.workunit.client.0.smithi092.stdout:8/54: chown d9/dc 76 1 2024-05-06T12:14:26.224 INFO:tasks.workunit.client.0.smithi092.stdout:7/96: mkdir d9/dd/d23 0 2024-05-06T12:14:26.226 INFO:tasks.workunit.client.0.smithi092.stdout:0/88: mkdir d7/da/d1e 0 2024-05-06T12:14:26.229 INFO:tasks.workunit.client.0.smithi092.stdout:8/55: mknod d9/c14 0 2024-05-06T12:14:26.229 INFO:tasks.workunit.client.0.smithi092.stdout:0/89: dread d7/f9 [0,4194304] 0 2024-05-06T12:14:26.231 INFO:tasks.workunit.client.0.smithi092.stdout:3/86: rmdir d8/db/d14/d15 0 2024-05-06T12:14:26.232 INFO:tasks.workunit.client.0.smithi092.stdout:7/97: dwrite d9/dd/f1e [0,4194304] 0 2024-05-06T12:14:26.232 INFO:tasks.workunit.client.0.smithi092.stdout:3/87: readlink l6 0 2024-05-06T12:14:26.234 INFO:tasks.workunit.client.0.smithi092.stdout:2/47: creat fe x:0 0 0 2024-05-06T12:14:26.236 INFO:tasks.workunit.client.0.smithi092.stdout:7/98: write d9/fb [5152079,105674] 0 2024-05-06T12:14:26.236 INFO:tasks.workunit.client.0.smithi092.stdout:0/90: mkdir d7/da/d12/d1a/d1f 0 2024-05-06T12:14:26.236 INFO:tasks.workunit.client.0.smithi092.stdout:8/56: dread f7 [0,4194304] 0 2024-05-06T12:14:26.236 INFO:tasks.workunit.client.0.smithi092.stdout:7/99: read d9/de/d10/f12 [285049,57443] 0 2024-05-06T12:14:26.238 INFO:tasks.workunit.client.0.smithi092.stdout:3/88: symlink d8/d13/l1c 0 2024-05-06T12:14:26.239 INFO:tasks.workunit.client.0.smithi092.stdout:0/91: write d7/da/d12/f14 [893268,3879] 0 2024-05-06T12:14:26.240 INFO:tasks.workunit.client.0.smithi092.stdout:8/57: write d9/fe [954721,130054] 0 2024-05-06T12:14:26.240 INFO:tasks.workunit.client.0.smithi092.stdout:7/100: chown d9/dd/d15/c20 106916070 1 2024-05-06T12:14:26.241 INFO:tasks.workunit.client.0.smithi092.stdout:3/89: write f5 [2095156,58939] 0 2024-05-06T12:14:26.245 INFO:tasks.workunit.client.0.smithi092.stdout:7/101: symlink d9/dd/d15/l24 0 2024-05-06T12:14:26.247 INFO:tasks.workunit.client.0.smithi092.stdout:7/102: write d9/de/f18 [1395381,68027] 0 2024-05-06T12:14:26.249 INFO:tasks.workunit.client.0.smithi092.stdout:8/58: rename d9/lb to d9/l15 0 2024-05-06T12:14:26.250 INFO:tasks.workunit.client.0.smithi092.stdout:0/92: dwrite d7/da/f19 [0,4194304] 0 2024-05-06T12:14:26.251 INFO:tasks.workunit.client.0.smithi092.stdout:7/103: fdatasync d9/de/f18 0 2024-05-06T12:14:26.258 INFO:tasks.workunit.client.0.smithi092.stdout:7/104: unlink d9/dd/c11 0 2024-05-06T12:14:26.260 INFO:tasks.workunit.client.0.smithi092.stdout:7/105: chown d9/dd/l1d 3 1 2024-05-06T12:14:26.261 INFO:tasks.workunit.client.0.smithi092.stdout:7/106: chown d9/de 235798331 1 2024-05-06T12:14:26.263 INFO:tasks.workunit.client.0.smithi092.stdout:0/93: dwrite d7/f9 [0,4194304] 0 2024-05-06T12:14:26.294 INFO:tasks.workunit.client.0.smithi092.stdout:9/11: getdents . 0 2024-05-06T12:14:26.295 INFO:tasks.workunit.client.0.smithi092.stdout:9/12: rename f1 to f2 0 2024-05-06T12:14:26.301 INFO:tasks.workunit.client.0.smithi092.stdout:4/53: dwrite d9/fc [0,4194304] 0 2024-05-06T12:14:26.304 INFO:tasks.workunit.client.0.smithi092.stdout:6/89: truncate d4/f6 759468 0 2024-05-06T12:14:26.309 INFO:tasks.workunit.client.0.smithi092.stdout:4/54: dwrite d9/df/f14 [0,4194304] 0 2024-05-06T12:14:26.311 INFO:tasks.workunit.client.0.smithi092.stdout:6/90: getdents d4/d16 0 2024-05-06T12:14:26.312 INFO:tasks.workunit.client.0.smithi092.stdout:4/55: creat d9/dd/f16 x:0 0 0 2024-05-06T12:14:26.313 INFO:tasks.workunit.client.0.smithi092.stdout:4/56: write f5 [1239754,48097] 0 2024-05-06T12:14:26.315 INFO:tasks.workunit.client.0.smithi092.stdout:6/91: rename d4/d16 to d4/d20 0 2024-05-06T12:14:26.320 INFO:tasks.workunit.client.0.smithi092.stdout:4/57: link d9/df/f14 d9/dd/f17 0 2024-05-06T12:14:26.324 INFO:tasks.workunit.client.0.smithi092.stdout:5/63: fsync d4/f19 0 2024-05-06T12:14:26.326 INFO:tasks.workunit.client.0.smithi092.stdout:5/64: mknod d4/dd/c1a 0 2024-05-06T12:14:26.326 INFO:tasks.workunit.client.0.smithi092.stdout:0/94: getdents d7/da/d12 0 2024-05-06T12:14:26.328 INFO:tasks.workunit.client.0.smithi092.stdout:0/95: read d7/da/f13 [206879,130801] 0 2024-05-06T12:14:26.328 INFO:tasks.workunit.client.0.smithi092.stdout:5/65: dread d4/d5/d10/f17 [0,4194304] 0 2024-05-06T12:14:26.332 INFO:tasks.workunit.client.0.smithi092.stdout:5/66: dread d4/d5/d10/f17 [0,4194304] 0 2024-05-06T12:14:26.335 INFO:tasks.workunit.client.0.smithi092.stdout:0/96: dwrite f6 [0,4194304] 0 2024-05-06T12:14:26.339 INFO:tasks.workunit.client.0.smithi092.stdout:0/97: rename d7/dc/cf to d7/c20 0 2024-05-06T12:14:26.347 INFO:tasks.workunit.client.0.smithi092.stdout:0/98: dwrite d7/da/d12/f14 [0,4194304] 0 2024-05-06T12:14:26.425 INFO:tasks.workunit.client.0.smithi092.stdout:1/36: truncate d3/f8 1831035 0 2024-05-06T12:14:26.431 INFO:tasks.workunit.client.0.smithi092.stdout:1/37: link d3/c6 d3/cd 0 2024-05-06T12:14:26.433 INFO:tasks.workunit.client.0.smithi092.stdout:3/90: rmdir d8/db/d14 39 2024-05-06T12:14:26.436 INFO:tasks.workunit.client.0.smithi092.stdout:3/91: link f5 d8/f1d 0 2024-05-06T12:14:26.437 INFO:tasks.workunit.client.0.smithi092.stdout:3/92: dread - d8/db/d14/f18 zero size 2024-05-06T12:14:26.439 INFO:tasks.workunit.client.0.smithi092.stdout:3/93: mknod d8/d9/c1e 0 2024-05-06T12:14:26.440 INFO:tasks.workunit.client.0.smithi092.stdout:8/59: chown d9/l15 3338 1 2024-05-06T12:14:26.440 INFO:tasks.workunit.client.0.smithi092.stdout:8/60: chown d9 89145 1 2024-05-06T12:14:26.442 INFO:tasks.workunit.client.0.smithi092.stdout:8/61: creat d9/dc/f16 x:0 0 0 2024-05-06T12:14:26.443 INFO:tasks.workunit.client.0.smithi092.stdout:7/107: truncate d9/dd/d15/f14 3364222 0 2024-05-06T12:14:26.444 INFO:tasks.workunit.client.0.smithi092.stdout:3/94: dread d8/db/f11 [0,4194304] 0 2024-05-06T12:14:26.444 INFO:tasks.workunit.client.0.smithi092.stdout:8/62: mknod d9/c17 0 2024-05-06T12:14:26.445 INFO:tasks.workunit.client.0.smithi092.stdout:8/63: write d9/fe [1528288,61114] 0 2024-05-06T12:14:26.446 INFO:tasks.workunit.client.0.smithi092.stdout:9/13: rename f2 to f3 0 2024-05-06T12:14:26.447 INFO:tasks.workunit.client.0.smithi092.stdout:7/108: creat d9/de/d10/f25 x:0 0 0 2024-05-06T12:14:26.447 INFO:tasks.workunit.client.0.smithi092.stdout:7/109: readlink d9/l1b 0 2024-05-06T12:14:26.451 INFO:tasks.workunit.client.0.smithi092.stdout:3/95: unlink d8/d13/l1c 0 2024-05-06T12:14:26.452 INFO:tasks.workunit.client.0.smithi092.stdout:3/96: chown d8/ce 88255 1 2024-05-06T12:14:26.452 INFO:tasks.workunit.client.0.smithi092.stdout:8/64: fdatasync f8 0 2024-05-06T12:14:26.455 INFO:tasks.workunit.client.0.smithi092.stdout:7/110: dwrite d9/dd/f1e [0,4194304] 0 2024-05-06T12:14:26.455 INFO:tasks.workunit.client.0.smithi092.stdout:8/65: dread - d9/dc/ff zero size 2024-05-06T12:14:26.456 INFO:tasks.workunit.client.0.smithi092.stdout:8/66: chown d9 451514587 1 2024-05-06T12:14:26.456 INFO:tasks.workunit.client.0.smithi092.stdout:7/111: chown d9/de 1667 1 2024-05-06T12:14:26.457 INFO:tasks.workunit.client.0.smithi092.stdout:4/58: truncate d9/df/f14 3486214 0 2024-05-06T12:14:26.460 INFO:tasks.workunit.client.0.smithi092.stdout:6/92: chown d4/d20/f1b 14274441 1 2024-05-06T12:14:26.461 INFO:tasks.workunit.client.0.smithi092.stdout:8/67: creat d9/f18 x:0 0 0 2024-05-06T12:14:26.463 INFO:tasks.workunit.client.0.smithi092.stdout:9/14: dwrite f3 [0,4194304] 0 2024-05-06T12:14:26.464 INFO:tasks.workunit.client.0.smithi092.stdout:8/68: chown d9/dc/l10 101753 1 2024-05-06T12:14:26.469 INFO:tasks.workunit.client.0.smithi092.stdout:6/93: symlink d4/d20/l21 0 2024-05-06T12:14:26.471 INFO:tasks.workunit.client.0.smithi092.stdout:5/67: dwrite d4/f9 [0,4194304] 0 2024-05-06T12:14:26.472 INFO:tasks.workunit.client.0.smithi092.stdout:0/99: truncate f1 2772998 0 2024-05-06T12:14:26.473 INFO:tasks.workunit.client.0.smithi092.stdout:0/100: write d7/f9 [4421369,71504] 0 2024-05-06T12:14:26.474 INFO:tasks.workunit.client.0.smithi092.stdout:9/15: mknod c4 0 2024-05-06T12:14:26.474 INFO:tasks.workunit.client.0.smithi092.stdout:9/16: readlink - no filename 2024-05-06T12:14:26.474 INFO:tasks.workunit.client.0.smithi092.stdout:9/17: chown c4 4312 1 2024-05-06T12:14:26.476 INFO:tasks.workunit.client.0.smithi092.stdout:8/69: mknod d9/c19 0 2024-05-06T12:14:26.478 INFO:tasks.workunit.client.0.smithi092.stdout:6/94: write d4/f1a [103430,125239] 0 2024-05-06T12:14:26.479 INFO:tasks.workunit.client.0.smithi092.stdout:3/97: getdents d8/db/d14 0 2024-05-06T12:14:26.479 INFO:tasks.workunit.client.0.smithi092.stdout:9/18: dread f3 [0,4194304] 0 2024-05-06T12:14:26.481 INFO:tasks.workunit.client.0.smithi092.stdout:5/68: creat d4/dd/f1b x:0 0 0 2024-05-06T12:14:26.483 INFO:tasks.workunit.client.0.smithi092.stdout:0/101: symlink d7/da/d12/l21 0 2024-05-06T12:14:26.484 INFO:tasks.workunit.client.0.smithi092.stdout:8/70: dwrite d9/fe [0,4194304] 0 2024-05-06T12:14:26.487 INFO:tasks.workunit.client.0.smithi092.stdout:6/95: mkdir d4/d20/d22 0 2024-05-06T12:14:26.488 INFO:tasks.workunit.client.0.smithi092.stdout:9/19: creat f5 x:0 0 0 2024-05-06T12:14:26.491 INFO:tasks.workunit.client.0.smithi092.stdout:5/69: truncate f0 240156 0 2024-05-06T12:14:26.495 INFO:tasks.workunit.client.0.smithi092.stdout:3/98: creat d8/d1b/f1f x:0 0 0 2024-05-06T12:14:26.497 INFO:tasks.workunit.client.0.smithi092.stdout:5/70: mkdir d4/dd/d1c 0 2024-05-06T12:14:26.500 INFO:tasks.workunit.client.0.smithi092.stdout:3/99: dread f5 [0,4194304] 0 2024-05-06T12:14:26.501 INFO:tasks.workunit.client.0.smithi092.stdout:3/100: write d8/f12 [396394,79149] 0 2024-05-06T12:14:26.503 INFO:tasks.workunit.client.0.smithi092.stdout:0/102: getdents d7 0 2024-05-06T12:14:26.505 INFO:tasks.workunit.client.0.smithi092.stdout:6/96: dwrite d4/f17 [0,4194304] 0 2024-05-06T12:14:26.508 INFO:tasks.workunit.client.0.smithi092.stdout:6/97: stat d4/f19 0 2024-05-06T12:14:26.509 INFO:tasks.workunit.client.0.smithi092.stdout:3/101: symlink d8/db/l20 0 2024-05-06T12:14:26.510 INFO:tasks.workunit.client.0.smithi092.stdout:0/103: creat d7/da/d1e/f22 x:0 0 0 2024-05-06T12:14:26.511 INFO:tasks.workunit.client.0.smithi092.stdout:6/98: mkdir d4/d20/d23 0 2024-05-06T12:14:26.513 INFO:tasks.workunit.client.0.smithi092.stdout:3/102: mknod d8/d13/c21 0 2024-05-06T12:14:26.516 INFO:tasks.workunit.client.0.smithi092.stdout:0/104: mknod d7/da/d12/d1a/d1f/c23 0 2024-05-06T12:14:26.516 INFO:tasks.workunit.client.0.smithi092.stdout:0/105: fsync d7/da/f11 0 2024-05-06T12:14:26.517 INFO:tasks.workunit.client.0.smithi092.stdout:6/99: unlink f1 0 2024-05-06T12:14:26.518 INFO:tasks.workunit.client.0.smithi092.stdout:0/106: write d7/f16 [574251,60016] 0 2024-05-06T12:14:26.519 INFO:tasks.workunit.client.0.smithi092.stdout:6/100: write d4/f17 [3712289,60209] 0 2024-05-06T12:14:26.519 INFO:tasks.workunit.client.0.smithi092.stdout:0/107: chown d7/da/d1e/f22 252683226 1 2024-05-06T12:14:26.521 INFO:tasks.workunit.client.0.smithi092.stdout:0/108: creat d7/da/d12/d1a/f24 x:0 0 0 2024-05-06T12:14:26.521 INFO:tasks.workunit.client.0.smithi092.stdout:8/71: rmdir d9/dc 39 2024-05-06T12:14:26.525 INFO:tasks.workunit.client.0.smithi092.stdout:0/109: mknod d7/da/d1e/c25 0 2024-05-06T12:14:26.526 INFO:tasks.workunit.client.0.smithi092.stdout:8/72: mknod d9/c1a 0 2024-05-06T12:14:26.527 INFO:tasks.workunit.client.0.smithi092.stdout:0/110: mkdir d7/da/d1e/d26 0 2024-05-06T12:14:26.530 INFO:tasks.workunit.client.0.smithi092.stdout:8/73: write f8 [3235084,96548] 0 2024-05-06T12:14:26.532 INFO:tasks.workunit.client.0.smithi092.stdout:4/59: fsync d9/dd/f17 0 2024-05-06T12:14:26.535 INFO:tasks.workunit.client.0.smithi092.stdout:4/60: rename d9/dd to d9/dd/d18 22 2024-05-06T12:14:26.537 INFO:tasks.workunit.client.0.smithi092.stdout:0/111: dwrite d7/f16 [0,4194304] 0 2024-05-06T12:14:26.538 INFO:tasks.workunit.client.0.smithi092.stdout:8/74: dread f8 [0,4194304] 0 2024-05-06T12:14:26.548 INFO:tasks.workunit.client.0.smithi092.stdout:4/61: link d9/df/f14 d9/dd/f19 0 2024-05-06T12:14:26.552 INFO:tasks.workunit.client.0.smithi092.stdout:8/75: creat d9/f1b x:0 0 0 2024-05-06T12:14:26.553 INFO:tasks.workunit.client.0.smithi092.stdout:4/62: symlink d9/df/l1a 0 2024-05-06T12:14:26.554 INFO:tasks.workunit.client.0.smithi092.stdout:4/63: dread - d9/dd/f16 zero size 2024-05-06T12:14:26.554 INFO:tasks.workunit.client.0.smithi092.stdout:4/64: stat c6 0 2024-05-06T12:14:26.554 INFO:tasks.workunit.client.0.smithi092.stdout:0/112: getdents d7 0 2024-05-06T12:14:26.557 INFO:tasks.workunit.client.0.smithi092.stdout:8/76: write d9/dc/ff [985983,29882] 0 2024-05-06T12:14:26.559 INFO:tasks.workunit.client.0.smithi092.stdout:4/65: dread d9/fc [0,4194304] 0 2024-05-06T12:14:26.561 INFO:tasks.workunit.client.0.smithi092.stdout:0/113: dread d7/da/f13 [0,4194304] 0 2024-05-06T12:14:26.562 INFO:tasks.workunit.client.0.smithi092.stdout:8/77: mknod d9/c1c 0 2024-05-06T12:14:26.562 INFO:tasks.workunit.client.0.smithi092.stdout:8/78: fsync d9/f18 0 2024-05-06T12:14:26.563 INFO:tasks.workunit.client.0.smithi092.stdout:4/66: symlink d9/df/l1b 0 2024-05-06T12:14:26.563 INFO:tasks.workunit.client.0.smithi092.stdout:0/114: write d7/da/d12/d1a/f1c [240364,119820] 0 2024-05-06T12:14:26.565 INFO:tasks.workunit.client.0.smithi092.stdout:0/115: write d7/da/f11 [224009,33058] 0 2024-05-06T12:14:26.565 INFO:tasks.workunit.client.0.smithi092.stdout:8/79: symlink d9/dc/l1d 0 2024-05-06T12:14:26.567 INFO:tasks.workunit.client.0.smithi092.stdout:9/20: fsync f3 0 2024-05-06T12:14:26.567 INFO:tasks.workunit.client.0.smithi092.stdout:0/116: rename d7/da/d12/d1a to d7/da/d12/d1a/d1f/d27 22 2024-05-06T12:14:26.569 INFO:tasks.workunit.client.0.smithi092.stdout:4/67: dread d9/dd/f17 [0,4194304] 0 2024-05-06T12:14:26.574 INFO:tasks.workunit.client.0.smithi092.stdout:8/80: mkdir d9/dc/d1e 0 2024-05-06T12:14:26.574 INFO:tasks.workunit.client.0.smithi092.stdout:0/117: mkdir d7/dc/d28 0 2024-05-06T12:14:26.575 INFO:tasks.workunit.client.0.smithi092.stdout:4/68: creat d9/f1c x:0 0 0 2024-05-06T12:14:26.575 INFO:tasks.workunit.client.0.smithi092.stdout:4/69: read - d9/df/f15 zero size 2024-05-06T12:14:26.576 INFO:tasks.workunit.client.0.smithi092.stdout:9/21: truncate f3 3469419 0 2024-05-06T12:14:26.578 INFO:tasks.workunit.client.0.smithi092.stdout:0/118: creat d7/dc/f29 x:0 0 0 2024-05-06T12:14:26.579 INFO:tasks.workunit.client.0.smithi092.stdout:0/119: chown d7/dc/f29 86090 1 2024-05-06T12:14:26.583 INFO:tasks.workunit.client.0.smithi092.stdout:4/70: rename l2 to d9/l1d 0 2024-05-06T12:14:26.584 INFO:tasks.workunit.client.0.smithi092.stdout:8/81: rmdir d9/dc/d1e 0 2024-05-06T12:14:26.586 INFO:tasks.workunit.client.0.smithi092.stdout:8/82: truncate d9/dc/f16 121786 0 2024-05-06T12:14:26.590 INFO:tasks.workunit.client.0.smithi092.stdout:7/112: write d9/dd/d15/f14 [94799,26325] 0 2024-05-06T12:14:26.590 INFO:tasks.workunit.client.0.smithi092.stdout:8/83: dread f6 [0,4194304] 0 2024-05-06T12:14:26.593 INFO:tasks.workunit.client.0.smithi092.stdout:7/113: symlink d9/de/d1f/l26 0 2024-05-06T12:14:26.596 INFO:tasks.workunit.client.0.smithi092.stdout:8/84: creat d9/f1f x:0 0 0 2024-05-06T12:14:26.599 INFO:tasks.workunit.client.0.smithi092.stdout:8/85: mknod d9/c20 0 2024-05-06T12:14:26.601 INFO:tasks.workunit.client.0.smithi092.stdout:0/120: write f1 [442792,88378] 0 2024-05-06T12:14:26.602 INFO:tasks.workunit.client.0.smithi092.stdout:0/121: truncate d7/f9 5451082 0 2024-05-06T12:14:26.604 INFO:tasks.workunit.client.0.smithi092.stdout:5/71: write f0 [1066885,98786] 0 2024-05-06T12:14:26.606 INFO:tasks.workunit.client.0.smithi092.stdout:0/122: rename d7/da/d12/l21 to d7/dc/l2a 0 2024-05-06T12:14:26.607 INFO:tasks.workunit.client.0.smithi092.stdout:0/123: write f6 [333719,38635] 0 2024-05-06T12:14:26.607 INFO:tasks.workunit.client.0.smithi092.stdout:5/72: creat d4/d5/f1d x:0 0 0 2024-05-06T12:14:26.613 INFO:tasks.workunit.client.0.smithi092.stdout:4/71: getdents d9 0 2024-05-06T12:14:26.615 INFO:tasks.workunit.client.0.smithi092.stdout:0/124: rename d7/da/d12/d1a/f24 to d7/da/d1e/d26/f2b 0 2024-05-06T12:14:26.616 INFO:tasks.workunit.client.0.smithi092.stdout:0/125: chown d7/da/d12/d1a 1 1 2024-05-06T12:14:26.617 INFO:tasks.workunit.client.0.smithi092.stdout:0/126: read d7/da/f19 [3276813,42336] 0 2024-05-06T12:14:26.619 INFO:tasks.workunit.client.0.smithi092.stdout:6/101: write d4/f6 [942510,588] 0 2024-05-06T12:14:26.622 INFO:tasks.workunit.client.0.smithi092.stdout:7/114: truncate d9/fb 1524156 0 2024-05-06T12:14:26.625 INFO:tasks.workunit.client.0.smithi092.stdout:4/72: unlink c3 0 2024-05-06T12:14:26.626 INFO:tasks.workunit.client.0.smithi092.stdout:4/73: write d9/f1c [499329,16561] 0 2024-05-06T12:14:26.628 INFO:tasks.workunit.client.0.smithi092.stdout:0/127: readlink d7/dc/l2a 0 2024-05-06T12:14:26.628 INFO:tasks.workunit.client.0.smithi092.stdout:9/22: link f3 f6 0 2024-05-06T12:14:26.629 INFO:tasks.workunit.client.0.smithi092.stdout:5/73: link c1 d4/d5/d10/d12/c1e 0 2024-05-06T12:14:26.629 INFO:tasks.workunit.client.0.smithi092.stdout:5/74: chown d4 95385 1 2024-05-06T12:14:26.629 INFO:tasks.workunit.client.0.smithi092.stdout:7/115: rmdir d9/de/d1f 39 2024-05-06T12:14:26.632 INFO:tasks.workunit.client.0.smithi092.stdout:9/23: dread f6 [0,4194304] 0 2024-05-06T12:14:26.632 INFO:tasks.workunit.client.0.smithi092.stdout:4/74: symlink d9/l1e 0 2024-05-06T12:14:26.678 INFO:tasks.workunit.client.0.smithi092.stdout:2/48: sync 2024-05-06T12:14:26.680 INFO:tasks.workunit.client.0.smithi092.stdout:0/128: creat d7/da/d12/d1a/f2c x:0 0 0 2024-05-06T12:14:26.684 INFO:tasks.workunit.client.0.smithi092.stdout:5/75: chown d4/dd/l11 133434255 1 2024-05-06T12:14:26.685 INFO:tasks.workunit.client.0.smithi092.stdout:5/76: chown d4/f9 54 1 2024-05-06T12:14:26.688 INFO:tasks.workunit.client.0.smithi092.stdout:9/24: fdatasync f6 0 2024-05-06T12:14:26.690 INFO:tasks.workunit.client.0.smithi092.stdout:4/75: rename d9/dd/c12 to d9/df/c1f 0 2024-05-06T12:14:26.695 INFO:tasks.workunit.client.0.smithi092.stdout:9/25: creat f7 x:0 0 0 2024-05-06T12:14:26.698 INFO:tasks.workunit.client.0.smithi092.stdout:0/129: dwrite d7/da/d1e/d26/f2b [0,4194304] 0 2024-05-06T12:14:26.698 INFO:tasks.workunit.client.0.smithi092.stdout:5/77: rename d4/dd/d1c to d4/d5/d10/d12/d1f 0 2024-05-06T12:14:26.700 INFO:tasks.workunit.client.0.smithi092.stdout:5/78: dread - d4/dd/f1b zero size 2024-05-06T12:14:26.701 INFO:tasks.workunit.client.0.smithi092.stdout:2/49: mkdir df 0 2024-05-06T12:14:26.704 INFO:tasks.workunit.client.0.smithi092.stdout:9/26: mknod c8 0 2024-05-06T12:14:26.706 INFO:tasks.workunit.client.0.smithi092.stdout:0/130: creat d7/da/d12/d1a/d1f/f2d x:0 0 0 2024-05-06T12:14:26.707 INFO:tasks.workunit.client.0.smithi092.stdout:7/116: getdents d9/dd 0 2024-05-06T12:14:26.708 INFO:tasks.workunit.client.0.smithi092.stdout:7/117: write d9/dd/d15/f14 [3609700,66663] 0 2024-05-06T12:14:26.708 INFO:tasks.workunit.client.0.smithi092.stdout:5/79: mkdir d4/d5/d10/d12/d1f/d20 0 2024-05-06T12:14:26.709 INFO:tasks.workunit.client.0.smithi092.stdout:7/118: read f1 [2062502,125956] 0 2024-05-06T12:14:26.710 INFO:tasks.workunit.client.0.smithi092.stdout:9/27: fsync f6 0 2024-05-06T12:14:26.713 INFO:tasks.workunit.client.0.smithi092.stdout:0/131: rename d7/f16 to d7/dc/d28/f2e 0 2024-05-06T12:14:26.715 INFO:tasks.workunit.client.0.smithi092.stdout:4/76: write d9/dd/f17 [1018706,100880] 0 2024-05-06T12:14:26.717 INFO:tasks.workunit.client.0.smithi092.stdout:2/50: symlink df/l10 0 2024-05-06T12:14:26.720 INFO:tasks.workunit.client.0.smithi092.stdout:5/80: dwrite d4/f19 [0,4194304] 0 2024-05-06T12:14:26.724 INFO:tasks.workunit.client.0.smithi092.stdout:5/81: chown d4/dd/c1a 0 1 2024-05-06T12:14:26.726 INFO:tasks.workunit.client.0.smithi092.stdout:5/82: dread - d4/dd/f1b zero size 2024-05-06T12:14:26.726 INFO:tasks.workunit.client.0.smithi092.stdout:9/28: write f6 [2096042,107848] 0 2024-05-06T12:14:26.726 INFO:tasks.workunit.client.0.smithi092.stdout:0/132: dwrite d7/dc/f29 [0,4194304] 0 2024-05-06T12:14:26.728 INFO:tasks.workunit.client.0.smithi092.stdout:4/77: dread d9/dd/f17 [0,4194304] 0 2024-05-06T12:14:26.732 INFO:tasks.workunit.client.0.smithi092.stdout:7/119: unlink d9/de/d1f/l22 0 2024-05-06T12:14:26.733 INFO:tasks.workunit.client.0.smithi092.stdout:7/120: readlink d9/de/l1c 0 2024-05-06T12:14:26.734 INFO:tasks.workunit.client.0.smithi092.stdout:9/29: dread f6 [0,4194304] 0 2024-05-06T12:14:26.735 INFO:tasks.workunit.client.0.smithi092.stdout:9/30: rmdir - no directory 2024-05-06T12:14:26.735 INFO:tasks.workunit.client.0.smithi092.stdout:9/31: chown c8 15901 1 2024-05-06T12:14:26.735 INFO:tasks.workunit.client.0.smithi092.stdout:0/133: dread d7/dc/d28/f2e [0,4194304] 0 2024-05-06T12:14:26.736 INFO:tasks.workunit.client.0.smithi092.stdout:4/78: dread d9/f1c [0,4194304] 0 2024-05-06T12:14:26.737 INFO:tasks.workunit.client.0.smithi092.stdout:0/134: write d7/da/d1e/d26/f2b [995961,40152] 0 2024-05-06T12:14:26.737 INFO:tasks.workunit.client.0.smithi092.stdout:2/51: rename fa to df/f11 0 2024-05-06T12:14:26.738 INFO:tasks.workunit.client.0.smithi092.stdout:2/52: dread - fe zero size 2024-05-06T12:14:26.739 INFO:tasks.workunit.client.0.smithi092.stdout:7/121: creat d9/de/d10/f27 x:0 0 0 2024-05-06T12:14:26.741 INFO:tasks.workunit.client.0.smithi092.stdout:9/32: creat f9 x:0 0 0 2024-05-06T12:14:26.741 INFO:tasks.workunit.client.0.smithi092.stdout:9/33: fsync f5 0 2024-05-06T12:14:26.743 INFO:tasks.workunit.client.0.smithi092.stdout:4/79: rename d9/l11 to d9/df/l20 0 2024-05-06T12:14:26.745 INFO:tasks.workunit.client.0.smithi092.stdout:0/135: creat d7/da/d12/d1a/d1f/f2f x:0 0 0 2024-05-06T12:14:26.746 INFO:tasks.workunit.client.0.smithi092.stdout:7/122: symlink d9/de/d10/l28 0 2024-05-06T12:14:26.749 INFO:tasks.workunit.client.0.smithi092.stdout:0/136: creat d7/dc/f30 x:0 0 0 2024-05-06T12:14:26.751 INFO:tasks.workunit.client.0.smithi092.stdout:7/123: rename d9/dd/l1d to d9/l29 0 2024-05-06T12:14:26.753 INFO:tasks.workunit.client.0.smithi092.stdout:2/53: link f7 df/f12 0 2024-05-06T12:14:26.754 INFO:tasks.workunit.client.0.smithi092.stdout:4/80: dwrite d9/dd/f19 [0,4194304] 0 2024-05-06T12:14:26.754 INFO:tasks.workunit.client.0.smithi092.stdout:4/81: readlink d9/lb 0 2024-05-06T12:14:26.754 INFO:tasks.workunit.client.0.smithi092.stdout:4/82: chown d9 2172 1 2024-05-06T12:14:26.756 INFO:tasks.workunit.client.0.smithi092.stdout:0/137: truncate d7/da/f13 696048 0 2024-05-06T12:14:26.757 INFO:tasks.workunit.client.0.smithi092.stdout:7/124: rmdir d9 39 2024-05-06T12:14:26.759 INFO:tasks.workunit.client.0.smithi092.stdout:5/83: truncate f0 567676 0 2024-05-06T12:14:26.760 INFO:tasks.workunit.client.0.smithi092.stdout:5/84: chown d4/d5/d10/d12/d1f/d20 19436 1 2024-05-06T12:14:26.761 INFO:tasks.workunit.client.0.smithi092.stdout:9/34: getdents . 0 2024-05-06T12:14:26.762 INFO:tasks.workunit.client.0.smithi092.stdout:9/35: dread - f7 zero size 2024-05-06T12:14:26.762 INFO:tasks.workunit.client.0.smithi092.stdout:0/138: mkdir d7/dc/d31 0 2024-05-06T12:14:26.766 INFO:tasks.workunit.client.0.smithi092.stdout:2/54: truncate f1 1121333 0 2024-05-06T12:14:26.768 INFO:tasks.workunit.client.0.smithi092.stdout:4/83: dwrite d9/fc [0,4194304] 0 2024-05-06T12:14:26.770 INFO:tasks.workunit.client.0.smithi092.stdout:0/139: unlink d7/da/d12/d1a/d1f/f2d 0 2024-05-06T12:14:26.772 INFO:tasks.workunit.client.0.smithi092.stdout:2/55: symlink df/l13 0 2024-05-06T12:14:26.772 INFO:tasks.workunit.client.0.smithi092.stdout:5/85: chown f0 3 1 2024-05-06T12:14:26.773 INFO:tasks.workunit.client.0.smithi092.stdout:9/36: dwrite f7 [0,4194304] 0 2024-05-06T12:14:26.778 INFO:tasks.workunit.client.0.smithi092.stdout:0/140: symlink d7/da/d12/d1a/l32 0 2024-05-06T12:14:26.778 INFO:tasks.workunit.client.0.smithi092.stdout:0/141: stat d7/c18 0 2024-05-06T12:14:26.780 INFO:tasks.workunit.client.0.smithi092.stdout:2/56: mknod df/c14 0 2024-05-06T12:14:26.780 INFO:tasks.workunit.client.0.smithi092.stdout:0/142: fdatasync d7/dc/f30 0 2024-05-06T12:14:26.781 INFO:tasks.workunit.client.0.smithi092.stdout:0/143: readlink d7/da/d12/d1a/l32 0 2024-05-06T12:14:26.786 INFO:tasks.workunit.client.0.smithi092.stdout:2/57: mkdir df/d15 0 2024-05-06T12:14:26.787 INFO:tasks.workunit.client.0.smithi092.stdout:5/86: mkdir d4/d5/d10/d12/d1f/d20/d21 0 2024-05-06T12:14:26.788 INFO:tasks.workunit.client.0.smithi092.stdout:0/144: dread d7/da/d12/f14 [0,4194304] 0 2024-05-06T12:14:26.789 INFO:tasks.workunit.client.0.smithi092.stdout:2/58: symlink df/d15/l16 0 2024-05-06T12:14:26.790 INFO:tasks.workunit.client.0.smithi092.stdout:9/37: dwrite f3 [0,4194304] 0 2024-05-06T12:14:26.791 INFO:tasks.workunit.client.0.smithi092.stdout:9/38: stat f3 0 2024-05-06T12:14:26.793 INFO:tasks.workunit.client.0.smithi092.stdout:2/59: unlink df/d15/l16 0 2024-05-06T12:14:26.796 INFO:tasks.workunit.client.0.smithi092.stdout:2/60: mkdir df/d17 0 2024-05-06T12:14:26.798 INFO:tasks.workunit.client.0.smithi092.stdout:0/145: link d7/l10 d7/da/d12/l33 0 2024-05-06T12:14:26.801 INFO:tasks.workunit.client.0.smithi092.stdout:0/146: write d7/f9 [85672,55155] 0 2024-05-06T12:14:26.801 INFO:tasks.workunit.client.0.smithi092.stdout:5/87: link d4/d5/cc d4/d5/d10/d12/d1f/d20/d21/c22 0 2024-05-06T12:14:26.804 INFO:tasks.workunit.client.0.smithi092.stdout:5/88: dread - d4/dd/f1b zero size 2024-05-06T12:14:26.804 INFO:tasks.workunit.client.0.smithi092.stdout:2/61: creat df/d17/f18 x:0 0 0 2024-05-06T12:14:26.806 INFO:tasks.workunit.client.0.smithi092.stdout:0/147: symlink d7/da/d12/d1a/l34 0 2024-05-06T12:14:26.806 INFO:tasks.workunit.client.0.smithi092.stdout:7/125: dwrite f1 [4194304,4194304] 0 2024-05-06T12:14:26.811 INFO:tasks.workunit.client.0.smithi092.stdout:2/62: unlink df/l13 0 2024-05-06T12:14:26.815 INFO:tasks.workunit.client.0.smithi092.stdout:4/84: truncate d9/fc 3719398 0 2024-05-06T12:14:26.817 INFO:tasks.workunit.client.0.smithi092.stdout:0/148: dwrite d7/da/d12/d1a/f1c [0,4194304] 0 2024-05-06T12:14:26.818 INFO:tasks.workunit.client.0.smithi092.stdout:4/85: read - d9/dd/f16 zero size 2024-05-06T12:14:26.820 INFO:tasks.workunit.client.0.smithi092.stdout:0/149: stat d7/c20 0 2024-05-06T12:14:26.825 INFO:tasks.workunit.client.0.smithi092.stdout:2/63: dwrite fd [0,4194304] 0 2024-05-06T12:14:26.825 INFO:tasks.workunit.client.0.smithi092.stdout:4/86: creat d9/f21 x:0 0 0 2024-05-06T12:14:26.831 INFO:tasks.workunit.client.0.smithi092.stdout:0/150: mknod d7/dc/d31/c35 0 2024-05-06T12:14:26.833 INFO:tasks.workunit.client.0.smithi092.stdout:7/126: rename d9/de/f18 to d9/de/f2a 0 2024-05-06T12:14:26.835 INFO:tasks.workunit.client.0.smithi092.stdout:7/127: write d9/dd/d15/f14 [3805149,105126] 0 2024-05-06T12:14:26.836 INFO:tasks.workunit.client.0.smithi092.stdout:0/151: creat d7/da/d1e/d26/f36 x:0 0 0 2024-05-06T12:14:26.839 INFO:tasks.workunit.client.0.smithi092.stdout:7/128: stat d9/fb 0 2024-05-06T12:14:26.842 INFO:tasks.workunit.client.0.smithi092.stdout:1/38: sync 2024-05-06T12:14:26.844 INFO:tasks.workunit.client.0.smithi092.stdout:2/64: write f7 [1017544,95145] 0 2024-05-06T12:14:26.845 INFO:tasks.workunit.client.0.smithi092.stdout:1/39: write f2 [1349384,28862] 0 2024-05-06T12:14:26.848 INFO:tasks.workunit.client.0.smithi092.stdout:2/65: read f7 [515114,102130] 0 2024-05-06T12:14:26.849 INFO:tasks.workunit.client.0.smithi092.stdout:0/152: dwrite d7/da/d12/d1a/f2c [0,4194304] 0 2024-05-06T12:14:26.894 INFO:tasks.workunit.client.0.smithi092.stdout:1/40: write d3/f5 [611056,97409] 0 2024-05-06T12:14:26.894 INFO:tasks.workunit.client.0.smithi092.stdout:5/89: write f0 [895334,104623] 0 2024-05-06T12:14:26.898 INFO:tasks.workunit.client.0.smithi092.stdout:0/153: mknod d7/da/c37 0 2024-05-06T12:14:26.899 INFO:tasks.workunit.client.0.smithi092.stdout:1/41: creat d3/fe x:0 0 0 2024-05-06T12:14:26.899 INFO:tasks.workunit.client.0.smithi092.stdout:7/129: dwrite d9/de/d10/f12 [0,4194304] 0 2024-05-06T12:14:26.900 INFO:tasks.workunit.client.0.smithi092.stdout:0/154: readlink d7/l10 0 2024-05-06T12:14:26.901 INFO:tasks.workunit.client.0.smithi092.stdout:7/130: truncate d9/de/f2a 2097056 0 2024-05-06T12:14:26.903 INFO:tasks.workunit.client.0.smithi092.stdout:5/90: rename d4/d5/d10/d12/d1f/d20/d21 to d4/d5/d10/d12/d1f/d20/d23 0 2024-05-06T12:14:26.904 INFO:tasks.workunit.client.0.smithi092.stdout:7/131: creat d9/de/d10/f2b x:0 0 0 2024-05-06T12:14:26.905 INFO:tasks.workunit.client.0.smithi092.stdout:7/132: dread - d9/de/d10/f25 zero size 2024-05-06T12:14:26.906 INFO:tasks.workunit.client.0.smithi092.stdout:7/133: write f1 [7468948,57321] 0 2024-05-06T12:14:26.908 INFO:tasks.workunit.client.0.smithi092.stdout:7/134: fdatasync d9/de/d10/f25 0 2024-05-06T12:14:26.908 INFO:tasks.workunit.client.0.smithi092.stdout:5/91: rename d4/dd to d4/d5/d10/d12/d1f/d20/d23/d24 0 2024-05-06T12:14:26.908 INFO:tasks.workunit.client.0.smithi092.stdout:1/42: read d3/f5 [385776,61915] 0 2024-05-06T12:14:26.909 INFO:tasks.workunit.client.0.smithi092.stdout:5/92: fsync d4/d5/d10/d12/f18 0 2024-05-06T12:14:26.910 INFO:tasks.workunit.client.0.smithi092.stdout:5/93: fdatasync d4/f9 0 2024-05-06T12:14:26.912 INFO:tasks.workunit.client.0.smithi092.stdout:1/43: mknod d3/cf 0 2024-05-06T12:14:26.913 INFO:tasks.workunit.client.0.smithi092.stdout:7/135: dread d9/dd/f1e [0,4194304] 0 2024-05-06T12:14:26.914 INFO:tasks.workunit.client.0.smithi092.stdout:5/94: creat d4/d5/d10/d12/d1f/d20/d23/f25 x:0 0 0 2024-05-06T12:14:26.915 INFO:tasks.workunit.client.0.smithi092.stdout:1/44: mkdir d3/d10 0 2024-05-06T12:14:26.915 INFO:tasks.workunit.client.0.smithi092.stdout:5/95: stat d4/d5/d10/d12/d1f/d20/d23/d24/c1a 0 2024-05-06T12:14:26.917 INFO:tasks.workunit.client.0.smithi092.stdout:5/96: mknod d4/d5/d10/d12/c26 0 2024-05-06T12:14:26.918 INFO:tasks.workunit.client.0.smithi092.stdout:5/97: chown d4/d5/d10/d12/d1f/d20/d23/d24/l11 2 1 2024-05-06T12:14:26.925 INFO:tasks.workunit.client.0.smithi092.stdout:5/98: dwrite d4/d5/d10/d12/d1f/d20/d23/d24/f1b [0,4194304] 0 2024-05-06T12:14:26.927 INFO:tasks.workunit.client.0.smithi092.stdout:5/99: link d4/f9 d4/d5/d10/f27 0 2024-05-06T12:14:26.929 INFO:tasks.workunit.client.0.smithi092.stdout:5/100: write d4/f9 [1291336,68165] 0 2024-05-06T12:14:26.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:26 smithi092 ceph-mon[26954]: pgmap v15: 65 pgs: 65 active+clean; 195 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 309 op/s 2024-05-06T12:14:26.972 INFO:tasks.workunit.client.0.smithi092.stdout:7/136: fdatasync f1 0 2024-05-06T12:14:26.975 INFO:tasks.workunit.client.0.smithi092.stdout:7/137: mknod d9/de/d10/c2c 0 2024-05-06T12:14:26.979 INFO:tasks.workunit.client.0.smithi092.stdout:7/138: mkdir d9/de/d2d 0 2024-05-06T12:14:26.980 INFO:tasks.workunit.client.0.smithi092.stdout:7/139: truncate d9/de/d10/f25 733427 0 2024-05-06T12:14:26.985 INFO:tasks.workunit.client.0.smithi092.stdout:1/45: rmdir d3 39 2024-05-06T12:14:26.990 INFO:tasks.workunit.client.0.smithi092.stdout:7/140: dwrite d9/de/d10/f2b [0,4194304] 0 2024-05-06T12:14:26.990 INFO:tasks.workunit.client.0.smithi092.stdout:5/101: fsync d4/f9 0 2024-05-06T12:14:26.992 INFO:tasks.workunit.client.0.smithi092.stdout:1/46: creat d3/d10/f11 x:0 0 0 2024-05-06T12:14:26.994 INFO:tasks.workunit.client.0.smithi092.stdout:5/102: mkdir d4/d5/d10/d12/d1f/d20/d23/d24/d28 0 2024-05-06T12:14:27.008 INFO:tasks.workunit.client.0.smithi092.stdout:5/103: mkdir d4/d5/d10/d12/d1f/d20/d23/d24/d29 0 2024-05-06T12:14:27.008 INFO:tasks.workunit.client.0.smithi092.stdout:1/47: rmdir d3/d10 39 2024-05-06T12:14:27.008 INFO:tasks.workunit.client.0.smithi092.stdout:7/141: dwrite f1 [4194304,4194304] 0 2024-05-06T12:14:27.009 INFO:tasks.workunit.client.0.smithi092.stdout:7/142: readlink d9/la 0 2024-05-06T12:14:27.014 INFO:tasks.workunit.client.0.smithi092.stdout:1/48: dread f2 [0,4194304] 0 2024-05-06T12:14:27.026 INFO:tasks.workunit.client.0.smithi092.stdout:7/143: fdatasync f1 0 2024-05-06T12:14:27.046 INFO:tasks.workunit.client.0.smithi092.stdout:3/103: sync 2024-05-06T12:14:27.047 INFO:tasks.workunit.client.0.smithi092.stdout:3/104: chown d8/f12 39733 1 2024-05-06T12:14:27.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:26 smithi190 ceph-mon[33798]: pgmap v15: 65 pgs: 65 active+clean; 195 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 309 op/s 2024-05-06T12:14:27.103 INFO:tasks.workunit.client.0.smithi092.stdout:5/104: creat d4/d5/f2a x:0 0 0 2024-05-06T12:14:27.105 INFO:tasks.workunit.client.0.smithi092.stdout:3/105: fdatasync f5 0 2024-05-06T12:14:27.108 INFO:tasks.workunit.client.0.smithi092.stdout:7/144: mkdir d9/de/d10/d17/d21/d2e 0 2024-05-06T12:14:27.109 INFO:tasks.workunit.client.0.smithi092.stdout:7/145: chown d9/de/d10/c2c 83 1 2024-05-06T12:14:27.109 INFO:tasks.workunit.client.0.smithi092.stdout:3/106: mknod d8/d13/c22 0 2024-05-06T12:14:27.110 INFO:tasks.workunit.client.0.smithi092.stdout:7/146: chown d9/de/d10/d17/d21/d2e 923158830 1 2024-05-06T12:14:27.110 INFO:tasks.workunit.client.0.smithi092.stdout:5/105: mkdir d4/d5/d10/d12/d1f/d20/d23/d24/d28/d2b 0 2024-05-06T12:14:27.113 INFO:tasks.workunit.client.0.smithi092.stdout:7/147: mknod d9/dd/d15/c2f 0 2024-05-06T12:14:27.114 INFO:tasks.workunit.client.0.smithi092.stdout:7/148: read d9/fb [443675,57919] 0 2024-05-06T12:14:27.114 INFO:tasks.workunit.client.0.smithi092.stdout:5/106: symlink d4/d5/d10/l2c 0 2024-05-06T12:14:27.115 INFO:tasks.workunit.client.0.smithi092.stdout:7/149: readlink d9/l19 0 2024-05-06T12:14:27.115 INFO:tasks.workunit.client.0.smithi092.stdout:5/107: dread - d4/d5/f2a zero size 2024-05-06T12:14:27.116 INFO:tasks.workunit.client.0.smithi092.stdout:1/49: rename f0 to d3/f12 0 2024-05-06T12:14:27.117 INFO:tasks.workunit.client.0.smithi092.stdout:1/50: write d3/fe [571462,20887] 0 2024-05-06T12:14:27.119 INFO:tasks.workunit.client.0.smithi092.stdout:5/108: symlink d4/d5/d10/d12/d1f/d20/l2d 0 2024-05-06T12:14:27.121 INFO:tasks.workunit.client.0.smithi092.stdout:7/150: truncate f3 2544302 0 2024-05-06T12:14:27.122 INFO:tasks.workunit.client.0.smithi092.stdout:1/51: mkdir d3/d10/d13 0 2024-05-06T12:14:27.124 INFO:tasks.workunit.client.0.smithi092.stdout:5/109: creat d4/d5/d10/d12/d1f/d20/d23/d24/f2e x:0 0 0 2024-05-06T12:14:27.125 INFO:tasks.workunit.client.0.smithi092.stdout:7/151: fdatasync d9/fb 0 2024-05-06T12:14:27.127 INFO:tasks.workunit.client.0.smithi092.stdout:1/52: creat d3/d10/d13/f14 x:0 0 0 2024-05-06T12:14:27.129 INFO:tasks.workunit.client.0.smithi092.stdout:7/152: creat d9/dd/d15/f30 x:0 0 0 2024-05-06T12:14:27.130 INFO:tasks.workunit.client.0.smithi092.stdout:5/110: symlink d4/d5/d10/d12/d1f/d20/d23/d24/d28/d2b/l2f 0 2024-05-06T12:14:27.131 INFO:tasks.workunit.client.0.smithi092.stdout:1/53: dread d3/f5 [0,4194304] 0 2024-05-06T12:14:27.132 INFO:tasks.workunit.client.0.smithi092.stdout:5/111: write d4/d5/d10/d12/f18 [549311,104217] 0 2024-05-06T12:14:27.132 INFO:tasks.workunit.client.0.smithi092.stdout:7/153: creat d9/de/d1f/f31 x:0 0 0 2024-05-06T12:14:27.134 INFO:tasks.workunit.client.0.smithi092.stdout:7/154: write d9/dd/d15/f30 [31902,111272] 0 2024-05-06T12:14:27.134 INFO:tasks.workunit.client.0.smithi092.stdout:7/155: fsync f1 0 2024-05-06T12:14:27.136 INFO:tasks.workunit.client.0.smithi092.stdout:1/54: rename f2 to d3/d10/f15 0 2024-05-06T12:14:27.137 INFO:tasks.workunit.client.0.smithi092.stdout:5/112: mkdir d4/d5/d30 0 2024-05-06T12:14:27.140 INFO:tasks.workunit.client.0.smithi092.stdout:5/113: unlink d4/d5/cb 0 2024-05-06T12:14:27.143 INFO:tasks.workunit.client.0.smithi092.stdout:7/156: dwrite f1 [4194304,4194304] 0 2024-05-06T12:14:27.145 INFO:tasks.workunit.client.0.smithi092.stdout:8/86: sync 2024-05-06T12:14:27.146 INFO:tasks.workunit.client.0.smithi092.stdout:6/102: sync 2024-05-06T12:14:27.147 INFO:tasks.workunit.client.0.smithi092.stdout:1/55: dwrite d3/d10/f11 [0,4194304] 0 2024-05-06T12:14:27.147 INFO:tasks.workunit.client.0.smithi092.stdout:5/114: write d4/d5/d10/d12/d1f/d20/d23/d24/f1b [3119489,7659] 0 2024-05-06T12:14:27.150 INFO:tasks.workunit.client.0.smithi092.stdout:8/87: mkdir d9/d21 0 2024-05-06T12:14:27.151 INFO:tasks.workunit.client.0.smithi092.stdout:5/115: dread - d4/d5/d10/d12/d1f/d20/d23/f25 zero size 2024-05-06T12:14:27.152 INFO:tasks.workunit.client.0.smithi092.stdout:8/88: readlink d9/dc/l10 0 2024-05-06T12:14:27.153 INFO:tasks.workunit.client.0.smithi092.stdout:5/116: write f0 [112417,51328] 0 2024-05-06T12:14:27.155 INFO:tasks.workunit.client.0.smithi092.stdout:6/103: dread d4/f17 [0,4194304] 0 2024-05-06T12:14:27.155 INFO:tasks.workunit.client.0.smithi092.stdout:1/56: symlink d3/l16 0 2024-05-06T12:14:27.156 INFO:tasks.workunit.client.0.smithi092.stdout:5/117: write d4/f9 [2604147,25230] 0 2024-05-06T12:14:27.156 INFO:tasks.workunit.client.0.smithi092.stdout:1/57: readlink d3/lb 0 2024-05-06T12:14:27.158 INFO:tasks.workunit.client.0.smithi092.stdout:7/157: creat d9/de/d10/d17/d21/d2e/f32 x:0 0 0 2024-05-06T12:14:27.160 INFO:tasks.workunit.client.0.smithi092.stdout:8/89: rename d9/dc/l1d to d9/d21/l22 0 2024-05-06T12:14:27.163 INFO:tasks.workunit.client.0.smithi092.stdout:5/118: mkdir d4/d5/d10/d31 0 2024-05-06T12:14:27.166 INFO:tasks.workunit.client.0.smithi092.stdout:6/104: mkdir d4/d24 0 2024-05-06T12:14:27.167 INFO:tasks.workunit.client.0.smithi092.stdout:8/90: symlink d9/dc/l23 0 2024-05-06T12:14:27.168 INFO:tasks.workunit.client.0.smithi092.stdout:1/58: dwrite d3/d10/d13/f14 [0,4194304] 0 2024-05-06T12:14:27.172 INFO:tasks.workunit.client.0.smithi092.stdout:5/119: mknod d4/d5/d30/c32 0 2024-05-06T12:14:27.173 INFO:tasks.workunit.client.0.smithi092.stdout:7/158: dwrite d9/dd/d15/f30 [0,4194304] 0 2024-05-06T12:14:27.174 INFO:tasks.workunit.client.0.smithi092.stdout:1/59: rmdir d3 39 2024-05-06T12:14:27.177 INFO:tasks.workunit.client.0.smithi092.stdout:5/120: stat d4/d5/f1d 0 2024-05-06T12:14:27.179 INFO:tasks.workunit.client.0.smithi092.stdout:7/159: write d9/de/d10/f2b [3893427,94956] 0 2024-05-06T12:14:27.180 INFO:tasks.workunit.client.0.smithi092.stdout:8/91: dread d9/dc/ff [0,4194304] 0 2024-05-06T12:14:27.180 INFO:tasks.workunit.client.0.smithi092.stdout:0/155: sync 2024-05-06T12:14:27.180 INFO:tasks.workunit.client.0.smithi092.stdout:4/87: sync 2024-05-06T12:14:27.180 INFO:tasks.workunit.client.0.smithi092.stdout:2/66: sync 2024-05-06T12:14:27.180 INFO:tasks.workunit.client.0.smithi092.stdout:9/39: sync 2024-05-06T12:14:27.181 INFO:tasks.workunit.client.0.smithi092.stdout:7/160: stat d9/dd 0 2024-05-06T12:14:27.183 INFO:tasks.workunit.client.0.smithi092.stdout:0/156: readlink d7/l8 0 2024-05-06T12:14:27.183 INFO:tasks.workunit.client.0.smithi092.stdout:9/40: truncate f9 463379 0 2024-05-06T12:14:27.183 INFO:tasks.workunit.client.0.smithi092.stdout:9/41: rmdir - no directory 2024-05-06T12:14:27.186 INFO:tasks.workunit.client.0.smithi092.stdout:0/157: fsync d7/f9 0 2024-05-06T12:14:27.186 INFO:tasks.workunit.client.0.smithi092.stdout:6/105: dread d4/f1a [0,4194304] 0 2024-05-06T12:14:27.187 INFO:tasks.workunit.client.0.smithi092.stdout:2/67: dread - f6 zero size 2024-05-06T12:14:27.187 INFO:tasks.workunit.client.0.smithi092.stdout:5/121: rename d4/f19 to d4/d5/d10/f33 0 2024-05-06T12:14:27.188 INFO:tasks.workunit.client.0.smithi092.stdout:2/68: chown fc 61495074 1 2024-05-06T12:14:27.189 INFO:tasks.workunit.client.0.smithi092.stdout:9/42: creat fa x:0 0 0 2024-05-06T12:14:27.192 INFO:tasks.workunit.client.0.smithi092.stdout:4/88: creat d9/f22 x:0 0 0 2024-05-06T12:14:27.192 INFO:tasks.workunit.client.0.smithi092.stdout:9/43: read f3 [4169751,83683] 0 2024-05-06T12:14:27.195 INFO:tasks.workunit.client.0.smithi092.stdout:8/92: dwrite f6 [0,4194304] 0 2024-05-06T12:14:27.198 INFO:tasks.workunit.client.0.smithi092.stdout:2/69: unlink f7 0 2024-05-06T12:14:27.200 INFO:tasks.workunit.client.0.smithi092.stdout:4/89: truncate d9/f22 933892 0 2024-05-06T12:14:27.200 INFO:tasks.workunit.client.0.smithi092.stdout:0/158: dread d7/dc/d28/f2e [0,4194304] 0 2024-05-06T12:14:27.200 INFO:tasks.workunit.client.0.smithi092.stdout:9/44: write f7 [2063761,55921] 0 2024-05-06T12:14:27.200 INFO:tasks.workunit.client.0.smithi092.stdout:7/161: getdents d9/de/d2d 0 2024-05-06T12:14:27.202 INFO:tasks.workunit.client.0.smithi092.stdout:8/93: chown d9/dc/l23 1712307038 1 2024-05-06T12:14:27.206 INFO:tasks.workunit.client.0.smithi092.stdout:1/60: rename d3/fc to d3/f17 0 2024-05-06T12:14:27.208 INFO:tasks.workunit.client.0.smithi092.stdout:1/61: chown d3/d10/d13 19462 1 2024-05-06T12:14:27.211 INFO:tasks.workunit.client.0.smithi092.stdout:1/62: write d3/fe [407990,7910] 0 2024-05-06T12:14:27.212 INFO:tasks.workunit.client.0.smithi092.stdout:6/106: dwrite d4/f6 [0,4194304] 0 2024-05-06T12:14:27.214 INFO:tasks.workunit.client.0.smithi092.stdout:4/90: creat d9/dd/f23 x:0 0 0 2024-05-06T12:14:27.215 INFO:tasks.workunit.client.0.smithi092.stdout:9/45: dread f7 [0,4194304] 0 2024-05-06T12:14:27.228 INFO:tasks.workunit.client.0.smithi092.stdout:9/46: creat fb x:0 0 0 2024-05-06T12:14:27.229 INFO:tasks.workunit.client.0.smithi092.stdout:7/162: symlink d9/dd/d23/l33 0 2024-05-06T12:14:27.237 INFO:tasks.workunit.client.0.smithi092.stdout:1/63: mknod d3/c18 0 2024-05-06T12:14:27.240 INFO:tasks.workunit.client.0.smithi092.stdout:9/47: creat fc x:0 0 0 2024-05-06T12:14:27.241 INFO:tasks.workunit.client.0.smithi092.stdout:7/163: fdatasync d9/fb 0 2024-05-06T12:14:27.243 INFO:tasks.workunit.client.0.smithi092.stdout:0/159: link d7/dc/d31/c35 d7/da/d12/d1a/c38 0 2024-05-06T12:14:27.244 INFO:tasks.workunit.client.0.smithi092.stdout:0/160: chown d7/da/d12/d1a/d1f 2 1 2024-05-06T12:14:27.246 INFO:tasks.workunit.client.0.smithi092.stdout:6/107: mknod d4/d20/d23/c25 0 2024-05-06T12:14:27.249 INFO:tasks.workunit.client.0.smithi092.stdout:9/48: unlink f5 0 2024-05-06T12:14:27.256 INFO:tasks.workunit.client.0.smithi092.stdout:1/64: unlink d3/cd 0 2024-05-06T12:14:27.259 INFO:tasks.workunit.client.0.smithi092.stdout:6/108: write d4/f19 [740396,6225] 0 2024-05-06T12:14:27.263 INFO:tasks.workunit.client.0.smithi092.stdout:6/109: readlink d4/l10 0 2024-05-06T12:14:27.264 INFO:tasks.workunit.client.0.smithi092.stdout:9/49: dread f3 [0,4194304] 0 2024-05-06T12:14:27.265 INFO:tasks.workunit.client.0.smithi092.stdout:7/164: creat d9/de/f34 x:0 0 0 2024-05-06T12:14:27.267 INFO:tasks.workunit.client.0.smithi092.stdout:9/50: symlink ld 0 2024-05-06T12:14:27.268 INFO:tasks.workunit.client.0.smithi092.stdout:6/110: mkdir d4/d24/d26 0 2024-05-06T12:14:27.268 INFO:tasks.workunit.client.0.smithi092.stdout:9/51: truncate fc 721394 0 2024-05-06T12:14:27.272 INFO:tasks.workunit.client.0.smithi092.stdout:9/52: symlink le 0 2024-05-06T12:14:27.273 INFO:tasks.workunit.client.0.smithi092.stdout:9/53: stat fb 0 2024-05-06T12:14:27.273 INFO:tasks.workunit.client.0.smithi092.stdout:7/165: dwrite d9/de/f34 [0,4194304] 0 2024-05-06T12:14:27.274 INFO:tasks.workunit.client.0.smithi092.stdout:9/54: read fc [617670,108007] 0 2024-05-06T12:14:27.275 INFO:tasks.workunit.client.0.smithi092.stdout:9/55: readlink le 0 2024-05-06T12:14:27.276 INFO:tasks.workunit.client.0.smithi092.stdout:7/166: fdatasync d9/de/d10/d17/d21/d2e/f32 0 2024-05-06T12:14:27.277 INFO:tasks.workunit.client.0.smithi092.stdout:6/111: dread d4/fb [0,4194304] 0 2024-05-06T12:14:27.278 INFO:tasks.workunit.client.0.smithi092.stdout:9/56: rename c8 to cf 0 2024-05-06T12:14:27.280 INFO:tasks.workunit.client.0.smithi092.stdout:7/167: creat d9/de/d1f/f35 x:0 0 0 2024-05-06T12:14:27.280 INFO:tasks.workunit.client.0.smithi092.stdout:9/57: mknod c10 0 2024-05-06T12:14:27.286 INFO:tasks.workunit.client.0.smithi092.stdout:6/112: link d4/d20/l21 d4/d24/d26/l27 0 2024-05-06T12:14:27.288 INFO:tasks.workunit.client.0.smithi092.stdout:7/168: dread d9/dd/f1e [0,4194304] 0 2024-05-06T12:14:27.290 INFO:tasks.workunit.client.0.smithi092.stdout:9/58: dwrite fa [0,4194304] 0 2024-05-06T12:14:27.293 INFO:tasks.workunit.client.0.smithi092.stdout:7/169: truncate d9/dd/d15/f30 4882541 0 2024-05-06T12:14:27.296 INFO:tasks.workunit.client.0.smithi092.stdout:7/170: unlink d9/l1b 0 2024-05-06T12:14:27.300 INFO:tasks.workunit.client.0.smithi092.stdout:9/59: dwrite fb [0,4194304] 0 2024-05-06T12:14:27.301 INFO:tasks.workunit.client.0.smithi092.stdout:7/171: getdents d9/dd 0 2024-05-06T12:14:27.304 INFO:tasks.workunit.client.0.smithi092.stdout:9/60: rename fa to f11 0 2024-05-06T12:14:27.307 INFO:tasks.workunit.client.0.smithi092.stdout:7/172: symlink d9/de/d10/l36 0 2024-05-06T12:14:27.307 INFO:tasks.workunit.client.0.smithi092.stdout:7/173: chown d9/de/l1c 84314111 1 2024-05-06T12:14:27.308 INFO:tasks.workunit.client.0.smithi092.stdout:9/61: creat f12 x:0 0 0 2024-05-06T12:14:27.308 INFO:tasks.workunit.client.0.smithi092.stdout:9/62: readlink ld 0 2024-05-06T12:14:27.309 INFO:tasks.workunit.client.0.smithi092.stdout:7/174: mkdir d9/de/d1f/d37 0 2024-05-06T12:14:27.311 INFO:tasks.workunit.client.0.smithi092.stdout:9/63: write f7 [3509210,57934] 0 2024-05-06T12:14:27.311 INFO:tasks.workunit.client.0.smithi092.stdout:9/64: stat c4 0 2024-05-06T12:14:27.312 INFO:tasks.workunit.client.0.smithi092.stdout:7/175: mkdir d9/de/d10/d17/d21/d38 0 2024-05-06T12:14:27.313 INFO:tasks.workunit.client.0.smithi092.stdout:9/65: mknod c13 0 2024-05-06T12:14:27.316 INFO:tasks.workunit.client.0.smithi092.stdout:9/66: rename ld to l14 0 2024-05-06T12:14:27.318 INFO:tasks.workunit.client.0.smithi092.stdout:3/107: write d8/f1d [1610069,103416] 0 2024-05-06T12:14:27.320 INFO:tasks.workunit.client.0.smithi092.stdout:9/67: dread f3 [0,4194304] 0 2024-05-06T12:14:27.321 INFO:tasks.workunit.client.0.smithi092.stdout:9/68: fdatasync f9 0 2024-05-06T12:14:27.321 INFO:tasks.workunit.client.0.smithi092.stdout:3/108: mknod d8/db/c23 0 2024-05-06T12:14:27.323 INFO:tasks.workunit.client.0.smithi092.stdout:9/69: unlink f12 0 2024-05-06T12:14:27.325 INFO:tasks.workunit.client.0.smithi092.stdout:3/109: rename d8/d13 to d8/d24 0 2024-05-06T12:14:27.326 INFO:tasks.workunit.client.0.smithi092.stdout:3/110: dread - d8/d9/f17 zero size 2024-05-06T12:14:27.326 INFO:tasks.workunit.client.0.smithi092.stdout:3/111: chown d8/d9/c1a 2692082 1 2024-05-06T12:14:27.327 INFO:tasks.workunit.client.0.smithi092.stdout:3/112: chown f2 18780425 1 2024-05-06T12:14:27.330 INFO:tasks.workunit.client.0.smithi092.stdout:3/113: mknod d8/d9/c25 0 2024-05-06T12:14:27.332 INFO:tasks.workunit.client.0.smithi092.stdout:3/114: truncate d8/d1b/f1f 599583 0 2024-05-06T12:14:27.334 INFO:tasks.workunit.client.0.smithi092.stdout:9/70: dwrite f3 [0,4194304] 0 2024-05-06T12:14:27.335 INFO:tasks.workunit.client.0.smithi092.stdout:5/122: rename d4/d5/d10/d12/d1f/d20/d23/d24 to d4/d34 0 2024-05-06T12:14:27.337 INFO:tasks.workunit.client.0.smithi092.stdout:6/113: rename d4/d20/f1e to d4/d20/d23/f28 0 2024-05-06T12:14:27.339 INFO:tasks.workunit.client.0.smithi092.stdout:5/123: mknod d4/d5/d30/c35 0 2024-05-06T12:14:27.341 INFO:tasks.workunit.client.0.smithi092.stdout:6/114: dread d4/f19 [0,4194304] 0 2024-05-06T12:14:27.345 INFO:tasks.workunit.client.0.smithi092.stdout:5/124: rename d4/d5/c7 to d4/d5/c36 0 2024-05-06T12:14:27.350 INFO:tasks.workunit.client.0.smithi092.stdout:6/115: dwrite d4/d20/f1b [0,4194304] 0 2024-05-06T12:14:27.353 INFO:tasks.workunit.client.0.smithi092.stdout:6/116: readlink d4/l18 0 2024-05-06T12:14:27.358 INFO:tasks.workunit.client.0.smithi092.stdout:6/117: dread d4/d20/f1c [0,4194304] 0 2024-05-06T12:14:27.360 INFO:tasks.workunit.client.0.smithi092.stdout:6/118: fsync d4/f19 0 2024-05-06T12:14:27.363 INFO:tasks.workunit.client.0.smithi092.stdout:2/70: write df/f11 [5115020,9310] 0 2024-05-06T12:14:27.365 INFO:tasks.workunit.client.0.smithi092.stdout:4/91: write d9/f1c [732190,94499] 0 2024-05-06T12:14:27.367 INFO:tasks.workunit.client.0.smithi092.stdout:8/94: truncate d9/fe 2223164 0 2024-05-06T12:14:27.368 INFO:tasks.workunit.client.0.smithi092.stdout:4/92: creat d9/f24 x:0 0 0 2024-05-06T12:14:27.369 INFO:tasks.workunit.client.0.smithi092.stdout:0/161: rmdir d7/da 39 2024-05-06T12:14:27.370 INFO:tasks.workunit.client.0.smithi092.stdout:8/95: chown d9/d21/l22 871091118 1 2024-05-06T12:14:27.373 INFO:tasks.workunit.client.0.smithi092.stdout:4/93: dread d9/fc [0,4194304] 0 2024-05-06T12:14:27.376 INFO:tasks.workunit.client.0.smithi092.stdout:0/162: dread d7/da/f19 [0,4194304] 0 2024-05-06T12:14:27.377 INFO:tasks.workunit.client.0.smithi092.stdout:4/94: rename c8 to d9/df/c25 0 2024-05-06T12:14:27.379 INFO:tasks.workunit.client.0.smithi092.stdout:0/163: dread f1 [0,4194304] 0 2024-05-06T12:14:27.380 INFO:tasks.workunit.client.0.smithi092.stdout:0/164: fdatasync d7/da/d1e/d26/f2b 0 2024-05-06T12:14:27.381 INFO:tasks.workunit.client.0.smithi092.stdout:8/96: link d9/d21/l22 d9/d21/l24 0 2024-05-06T12:14:27.384 INFO:tasks.workunit.client.0.smithi092.stdout:0/165: write d7/da/d12/f14 [3502012,91050] 0 2024-05-06T12:14:27.388 INFO:tasks.workunit.client.0.smithi092.stdout:0/166: mkdir d7/da/d1e/d39 0 2024-05-06T12:14:27.388 INFO:tasks.workunit.client.0.smithi092.stdout:4/95: link d9/df/c1f d9/df/c26 0 2024-05-06T12:14:27.389 INFO:tasks.workunit.client.0.smithi092.stdout:4/96: readlink d9/l1e 0 2024-05-06T12:14:27.390 INFO:tasks.workunit.client.0.smithi092.stdout:8/97: creat d9/f25 x:0 0 0 2024-05-06T12:14:27.391 INFO:tasks.workunit.client.0.smithi092.stdout:0/167: rmdir d7/da/d1e 39 2024-05-06T12:14:27.395 INFO:tasks.workunit.client.0.smithi092.stdout:4/97: dwrite d9/dd/f17 [0,4194304] 0 2024-05-06T12:14:27.395 INFO:tasks.workunit.client.0.smithi092.stdout:0/168: unlink d7/da/d12/d1a/d1f/c23 0 2024-05-06T12:14:27.397 INFO:tasks.workunit.client.0.smithi092.stdout:0/169: truncate d7/da/f11 567349 0 2024-05-06T12:14:27.400 INFO:tasks.workunit.client.0.smithi092.stdout:4/98: dread f5 [0,4194304] 0 2024-05-06T12:14:27.400 INFO:tasks.workunit.client.0.smithi092.stdout:4/99: readlink d9/l1e 0 2024-05-06T12:14:27.401 INFO:tasks.workunit.client.0.smithi092.stdout:4/100: dread - d9/df/f15 zero size 2024-05-06T12:14:27.404 INFO:tasks.workunit.client.0.smithi092.stdout:0/170: mkdir d7/da/d1e/d26/d3a 0 2024-05-06T12:14:27.405 INFO:tasks.workunit.client.0.smithi092.stdout:0/171: fsync d7/da/d1e/d26/f36 0 2024-05-06T12:14:27.405 INFO:tasks.workunit.client.0.smithi092.stdout:4/101: readlink d9/df/l20 0 2024-05-06T12:14:27.410 INFO:tasks.workunit.client.0.smithi092.stdout:9/71: unlink l14 0 2024-05-06T12:14:27.412 INFO:tasks.workunit.client.0.smithi092.stdout:4/102: dwrite d9/f1c [0,4194304] 0 2024-05-06T12:14:27.413 INFO:tasks.workunit.client.0.smithi092.stdout:9/72: symlink l15 0 2024-05-06T12:14:27.413 INFO:tasks.workunit.client.0.smithi092.stdout:4/103: read d9/f1c [3552561,46432] 0 2024-05-06T12:14:27.414 INFO:tasks.workunit.client.0.smithi092.stdout:0/172: creat d7/da/f3b x:0 0 0 2024-05-06T12:14:27.416 INFO:tasks.workunit.client.0.smithi092.stdout:7/176: write f3 [829703,67117] 0 2024-05-06T12:14:27.417 INFO:tasks.workunit.client.0.smithi092.stdout:9/73: fdatasync f11 0 2024-05-06T12:14:27.418 INFO:tasks.workunit.client.0.smithi092.stdout:9/74: fsync f9 0 2024-05-06T12:14:27.419 INFO:tasks.workunit.client.0.smithi092.stdout:9/75: chown fb 1 1 2024-05-06T12:14:27.426 INFO:tasks.workunit.client.0.smithi092.stdout:0/173: rename d7/da/f13 to d7/da/d12/d1a/d1f/f3c 0 2024-05-06T12:14:27.430 INFO:tasks.workunit.client.0.smithi092.stdout:7/177: mkdir d9/de/d1f/d37/d39 0 2024-05-06T12:14:27.431 INFO:tasks.workunit.client.0.smithi092.stdout:9/76: truncate f11 1294332 0 2024-05-06T12:14:27.432 INFO:tasks.workunit.client.0.smithi092.stdout:6/119: truncate d4/f17 1968109 0 2024-05-06T12:14:27.439 INFO:tasks.workunit.client.0.smithi092.stdout:4/104: rmdir d9/df 39 2024-05-06T12:14:27.440 INFO:tasks.workunit.client.0.smithi092.stdout:4/105: readlink d9/l13 0 2024-05-06T12:14:27.440 INFO:tasks.workunit.client.0.smithi092.stdout:7/178: dwrite d9/dd/f1e [0,4194304] 0 2024-05-06T12:14:27.441 INFO:tasks.workunit.client.0.smithi092.stdout:2/71: creat df/d17/f19 x:0 0 0 2024-05-06T12:14:27.442 INFO:tasks.workunit.client.0.smithi092.stdout:6/120: mkdir d4/d29 0 2024-05-06T12:14:27.444 INFO:tasks.workunit.client.0.smithi092.stdout:8/98: getdents d9 0 2024-05-06T12:14:27.446 INFO:tasks.workunit.client.0.smithi092.stdout:8/99: write d9/f18 [606490,69674] 0 2024-05-06T12:14:27.447 INFO:tasks.workunit.client.0.smithi092.stdout:4/106: fdatasync f5 0 2024-05-06T12:14:27.449 INFO:tasks.workunit.client.0.smithi092.stdout:7/179: dwrite f3 [0,4194304] 0 2024-05-06T12:14:27.453 INFO:tasks.workunit.client.0.smithi092.stdout:4/107: dread f5 [0,4194304] 0 2024-05-06T12:14:27.453 INFO:tasks.workunit.client.0.smithi092.stdout:0/174: rename d7/da/d12/d1a/c38 to d7/c3d 0 2024-05-06T12:14:27.455 INFO:tasks.workunit.client.0.smithi092.stdout:8/100: unlink d9/c12 0 2024-05-06T12:14:27.459 INFO:tasks.workunit.client.0.smithi092.stdout:6/121: mkdir d4/d2a 0 2024-05-06T12:14:27.460 INFO:tasks.workunit.client.0.smithi092.stdout:0/175: write d7/dc/d28/f2e [2216068,80878] 0 2024-05-06T12:14:27.461 INFO:tasks.workunit.client.0.smithi092.stdout:0/176: fdatasync d7/da/d12/d1a/f1c 0 2024-05-06T12:14:27.462 INFO:tasks.workunit.client.0.smithi092.stdout:0/177: read f1 [455143,115381] 0 2024-05-06T12:14:27.463 INFO:tasks.workunit.client.0.smithi092.stdout:7/180: unlink d9/l29 0 2024-05-06T12:14:27.465 INFO:tasks.workunit.client.0.smithi092.stdout:2/72: mknod df/d15/c1a 0 2024-05-06T12:14:27.467 INFO:tasks.workunit.client.0.smithi092.stdout:6/122: stat d4/l15 0 2024-05-06T12:14:27.471 INFO:tasks.workunit.client.0.smithi092.stdout:0/178: unlink d7/c18 0 2024-05-06T12:14:27.473 INFO:tasks.workunit.client.0.smithi092.stdout:2/73: chown df/l10 187443 1 2024-05-06T12:14:27.476 INFO:tasks.workunit.client.0.smithi092.stdout:0/179: rmdir d7/da/d12/d1a 39 2024-05-06T12:14:27.478 INFO:tasks.workunit.client.0.smithi092.stdout:7/181: dwrite d9/de/f2a [0,4194304] 0 2024-05-06T12:14:27.478 INFO:tasks.workunit.client.0.smithi092.stdout:6/123: symlink d4/d2a/l2b 0 2024-05-06T12:14:27.480 INFO:tasks.workunit.client.0.smithi092.stdout:9/77: truncate f11 1979287 0 2024-05-06T12:14:27.481 INFO:tasks.workunit.client.0.smithi092.stdout:7/182: read - d9/de/d10/d17/d21/d2e/f32 zero size 2024-05-06T12:14:27.486 INFO:tasks.workunit.client.0.smithi092.stdout:0/180: creat d7/da/d1e/f3e x:0 0 0 2024-05-06T12:14:27.487 INFO:tasks.workunit.client.0.smithi092.stdout:2/74: dwrite f5 [0,4194304] 0 2024-05-06T12:14:27.487 INFO:tasks.workunit.client.0.smithi092.stdout:6/124: rmdir d4/d20 39 2024-05-06T12:14:27.488 INFO:tasks.workunit.client.0.smithi092.stdout:7/183: truncate d9/de/d10/d17/d21/d2e/f32 141720 0 2024-05-06T12:14:27.489 INFO:tasks.workunit.client.0.smithi092.stdout:9/78: dread f7 [0,4194304] 0 2024-05-06T12:14:27.489 INFO:tasks.workunit.client.0.smithi092.stdout:8/101: link d9/c1c d9/d21/c26 0 2024-05-06T12:14:27.495 INFO:tasks.workunit.client.0.smithi092.stdout:9/79: truncate fc 320199 0 2024-05-06T12:14:27.497 INFO:tasks.workunit.client.0.smithi092.stdout:6/125: dwrite d4/f5 [0,4194304] 0 2024-05-06T12:14:27.501 INFO:tasks.workunit.client.0.smithi092.stdout:8/102: rename f8 to d9/dc/f27 0 2024-05-06T12:14:27.502 INFO:tasks.workunit.client.0.smithi092.stdout:9/80: mkdir d16 0 2024-05-06T12:14:27.504 INFO:tasks.workunit.client.0.smithi092.stdout:6/126: dwrite d4/f6 [0,4194304] 0 2024-05-06T12:14:27.507 INFO:tasks.workunit.client.0.smithi092.stdout:8/103: truncate d9/f1f 398164 0 2024-05-06T12:14:27.510 INFO:tasks.workunit.client.0.smithi092.stdout:7/184: truncate f1 7148192 0 2024-05-06T12:14:27.511 INFO:tasks.workunit.client.0.smithi092.stdout:0/181: getdents d7/dc/d31 0 2024-05-06T12:14:27.513 INFO:tasks.workunit.client.0.smithi092.stdout:6/127: stat d4/d20/d23/f28 0 2024-05-06T12:14:27.513 INFO:tasks.workunit.client.0.smithi092.stdout:8/104: dread f0 [0,4194304] 0 2024-05-06T12:14:27.513 INFO:tasks.workunit.client.0.smithi092.stdout:6/128: readlink d4/l18 0 2024-05-06T12:14:27.515 INFO:tasks.workunit.client.0.smithi092.stdout:6/129: write d4/d20/f1b [473413,105987] 0 2024-05-06T12:14:27.516 INFO:tasks.workunit.client.0.smithi092.stdout:6/130: dread d4/f1a [0,4194304] 0 2024-05-06T12:14:27.517 INFO:tasks.workunit.client.0.smithi092.stdout:0/182: dread d7/da/d12/f14 [0,4194304] 0 2024-05-06T12:14:27.519 INFO:tasks.workunit.client.0.smithi092.stdout:7/185: creat d9/de/d2d/f3a x:0 0 0 2024-05-06T12:14:27.522 INFO:tasks.workunit.client.0.smithi092.stdout:6/131: dread d4/f11 [0,4194304] 0 2024-05-06T12:14:27.523 INFO:tasks.workunit.client.0.smithi092.stdout:8/105: dread d9/fe [0,4194304] 0 2024-05-06T12:14:27.524 INFO:tasks.workunit.client.0.smithi092.stdout:8/106: chown d9/fe 359 1 2024-05-06T12:14:27.525 INFO:tasks.workunit.client.0.smithi092.stdout:7/186: dread d9/de/d10/f12 [0,4194304] 0 2024-05-06T12:14:27.528 INFO:tasks.workunit.client.0.smithi092.stdout:0/183: rename d7/da/d12/d1a/d1f/f2f to d7/da/d12/d1a/f3f 0 2024-05-06T12:14:27.529 INFO:tasks.workunit.client.0.smithi092.stdout:6/132: dread d4/f17 [0,4194304] 0 2024-05-06T12:14:27.530 INFO:tasks.workunit.client.0.smithi092.stdout:8/107: dread d9/dc/f27 [0,4194304] 0 2024-05-06T12:14:27.532 INFO:tasks.workunit.client.0.smithi092.stdout:7/187: symlink d9/dd/d23/l3b 0 2024-05-06T12:14:27.534 INFO:tasks.workunit.client.0.smithi092.stdout:7/188: fsync d9/dd/f1e 0 2024-05-06T12:14:27.535 INFO:tasks.workunit.client.0.smithi092.stdout:9/81: fdatasync fc 0 2024-05-06T12:14:27.536 INFO:tasks.workunit.client.0.smithi092.stdout:7/189: chown d9/de/f34 137147513 1 2024-05-06T12:14:27.536 INFO:tasks.workunit.client.0.smithi092.stdout:0/184: dwrite d7/da/f11 [0,4194304] 0 2024-05-06T12:14:27.537 INFO:tasks.workunit.client.0.smithi092.stdout:7/190: chown f3 1611622 1 2024-05-06T12:14:27.539 INFO:tasks.workunit.client.0.smithi092.stdout:8/108: creat d9/dc/f28 x:0 0 0 2024-05-06T12:14:27.543 INFO:tasks.workunit.client.0.smithi092.stdout:0/185: creat d7/da/d12/d1a/f40 x:0 0 0 2024-05-06T12:14:27.545 INFO:tasks.workunit.client.0.smithi092.stdout:7/191: dread d9/de/f2a [0,4194304] 0 2024-05-06T12:14:27.546 INFO:tasks.workunit.client.0.smithi092.stdout:6/133: dwrite d4/d20/d23/f28 [0,4194304] 0 2024-05-06T12:14:27.551 INFO:tasks.workunit.client.0.smithi092.stdout:9/82: rename f11 to d16/f17 0 2024-05-06T12:14:27.554 INFO:tasks.workunit.client.0.smithi092.stdout:7/192: symlink d9/dd/d15/l3c 0 2024-05-06T12:14:27.556 INFO:tasks.workunit.client.0.smithi092.stdout:8/109: dwrite d9/dc/ff [0,4194304] 0 2024-05-06T12:14:27.558 INFO:tasks.workunit.client.0.smithi092.stdout:9/83: symlink d16/l18 0 2024-05-06T12:14:27.560 INFO:tasks.workunit.client.0.smithi092.stdout:0/186: link d7/da/d1e/f3e d7/da/d1e/f41 0 2024-05-06T12:14:27.561 INFO:tasks.workunit.client.0.smithi092.stdout:0/187: chown d7/da/d12/d1a/f40 78977633 1 2024-05-06T12:14:27.562 INFO:tasks.workunit.client.0.smithi092.stdout:0/188: stat d7/da/d1e/f3e 0 2024-05-06T12:14:27.563 INFO:tasks.workunit.client.0.smithi092.stdout:0/189: rename d7/da/d1e/d26/d3a to d7/da/d1e/d26/d3a/d42 22 2024-05-06T12:14:27.563 INFO:tasks.workunit.client.0.smithi092.stdout:6/134: rmdir d4/d29 0 2024-05-06T12:14:27.565 INFO:tasks.workunit.client.0.smithi092.stdout:6/135: truncate d4/d20/f1c 850844 0 2024-05-06T12:14:27.565 INFO:tasks.workunit.client.0.smithi092.stdout:9/84: mknod d16/c19 0 2024-05-06T12:14:27.565 INFO:tasks.workunit.client.0.smithi092.stdout:9/85: readlink d16/l18 0 2024-05-06T12:14:27.569 INFO:tasks.workunit.client.0.smithi092.stdout:6/136: dread d4/fb [0,4194304] 0 2024-05-06T12:14:27.570 INFO:tasks.workunit.client.0.smithi092.stdout:8/110: creat d9/f29 x:0 0 0 2024-05-06T12:14:27.575 INFO:tasks.workunit.client.0.smithi092.stdout:0/190: getdents d7/da/d12/d1a 0 2024-05-06T12:14:27.578 INFO:tasks.workunit.client.0.smithi092.stdout:9/86: getdents d16 0 2024-05-06T12:14:27.579 INFO:tasks.workunit.client.0.smithi092.stdout:6/137: dwrite d4/f6 [0,4194304] 0 2024-05-06T12:14:27.580 INFO:tasks.workunit.client.0.smithi092.stdout:7/193: truncate d9/dd/d15/f14 558185 0 2024-05-06T12:14:27.581 INFO:tasks.workunit.client.0.smithi092.stdout:8/111: dwrite d9/dc/f16 [0,4194304] 0 2024-05-06T12:14:27.583 INFO:tasks.workunit.client.0.smithi092.stdout:0/191: write d7/da/d12/f14 [3151551,9331] 0 2024-05-06T12:14:27.587 INFO:tasks.workunit.client.0.smithi092.stdout:9/87: rmdir d16 39 2024-05-06T12:14:27.588 INFO:tasks.workunit.client.0.smithi092.stdout:7/194: mknod d9/de/c3d 0 2024-05-06T12:14:27.593 INFO:tasks.workunit.client.0.smithi092.stdout:6/138: dwrite d4/f11 [0,4194304] 0 2024-05-06T12:14:27.593 INFO:tasks.workunit.client.0.smithi092.stdout:8/112: creat d9/f2a x:0 0 0 2024-05-06T12:14:27.594 INFO:tasks.workunit.client.0.smithi092.stdout:0/192: rename d7/da/d12/d1a to d7/d43 0 2024-05-06T12:14:27.596 INFO:tasks.workunit.client.0.smithi092.stdout:8/113: symlink d9/d21/l2b 0 2024-05-06T12:14:27.601 INFO:tasks.workunit.client.0.smithi092.stdout:6/139: rename d4/cc to d4/d20/d22/c2c 0 2024-05-06T12:14:27.605 INFO:tasks.workunit.client.0.smithi092.stdout:8/114: rename d9/dc/l10 to d9/dc/l2c 0 2024-05-06T12:14:27.607 INFO:tasks.workunit.client.0.smithi092.stdout:6/140: mknod d4/d20/d23/c2d 0 2024-05-06T12:14:27.609 INFO:tasks.workunit.client.0.smithi092.stdout:7/195: dwrite d9/de/d10/f12 [4194304,4194304] 0 2024-05-06T12:14:27.609 INFO:tasks.workunit.client.0.smithi092.stdout:0/193: dwrite d7/dc/f30 [0,4194304] 0 2024-05-06T12:14:27.614 INFO:tasks.workunit.client.0.smithi092.stdout:8/115: rename d9/f1f to d9/dc/f2d 0 2024-05-06T12:14:27.616 INFO:tasks.workunit.client.0.smithi092.stdout:7/196: creat d9/de/d10/d17/d21/f3e x:0 0 0 2024-05-06T12:14:27.617 INFO:tasks.workunit.client.0.smithi092.stdout:6/141: symlink d4/d24/d26/l2e 0 2024-05-06T12:14:27.617 INFO:tasks.workunit.client.0.smithi092.stdout:0/194: dread d7/dc/f29 [0,4194304] 0 2024-05-06T12:14:27.619 INFO:tasks.workunit.client.0.smithi092.stdout:0/195: dread - d7/da/d12/f1b zero size 2024-05-06T12:14:27.624 INFO:tasks.workunit.client.0.smithi092.stdout:6/142: symlink d4/d20/l2f 0 2024-05-06T12:14:27.626 INFO:tasks.workunit.client.0.smithi092.stdout:6/143: write d4/d20/f1b [423077,116244] 0 2024-05-06T12:14:27.626 INFO:tasks.workunit.client.0.smithi092.stdout:6/144: fsync d4/f11 0 2024-05-06T12:14:27.631 INFO:tasks.workunit.client.0.smithi092.stdout:7/197: mknod d9/de/d1f/d37/d39/c3f 0 2024-05-06T12:14:27.633 INFO:tasks.workunit.client.0.smithi092.stdout:6/145: rename d4/d2a to d4/d20/d23/d30 0 2024-05-06T12:14:27.633 INFO:tasks.workunit.client.0.smithi092.stdout:0/196: getdents d7/da/d1e/d26/d3a 0 2024-05-06T12:14:27.635 INFO:tasks.workunit.client.0.smithi092.stdout:6/146: write d4/f1a [1026997,86244] 0 2024-05-06T12:14:27.635 INFO:tasks.workunit.client.0.smithi092.stdout:0/197: fsync f1 0 2024-05-06T12:14:27.636 INFO:tasks.workunit.client.0.smithi092.stdout:0/198: readlink d7/d43/l34 0 2024-05-06T12:14:27.642 INFO:tasks.workunit.client.0.smithi092.stdout:8/116: write d9/fe [2102682,78363] 0 2024-05-06T12:14:27.644 INFO:tasks.workunit.client.0.smithi092.stdout:8/117: dread - d9/dc/f28 zero size 2024-05-06T12:14:27.644 INFO:tasks.workunit.client.0.smithi092.stdout:6/147: dwrite d4/d20/d23/f28 [0,4194304] 0 2024-05-06T12:14:27.645 INFO:tasks.workunit.client.0.smithi092.stdout:8/118: fsync d9/dc/ff 0 2024-05-06T12:14:27.646 INFO:tasks.workunit.client.0.smithi092.stdout:7/198: write f1 [6093446,58641] 0 2024-05-06T12:14:27.649 INFO:tasks.workunit.client.0.smithi092.stdout:6/148: creat d4/d20/f31 x:0 0 0 2024-05-06T12:14:27.650 INFO:tasks.workunit.client.0.smithi092.stdout:8/119: mknod d9/dc/c2e 0 2024-05-06T12:14:27.651 INFO:tasks.workunit.client.0.smithi092.stdout:7/199: creat d9/f40 x:0 0 0 2024-05-06T12:14:27.655 INFO:tasks.workunit.client.0.smithi092.stdout:0/199: dwrite d7/da/d1e/f41 [0,4194304] 0 2024-05-06T12:14:27.656 INFO:tasks.workunit.client.0.smithi092.stdout:0/200: chown d7/da/d1e/d39 135021 1 2024-05-06T12:14:27.656 INFO:tasks.workunit.client.0.smithi092.stdout:7/200: symlink d9/de/d10/d17/d21/d2e/l41 0 2024-05-06T12:14:27.658 INFO:tasks.workunit.client.0.smithi092.stdout:7/201: creat d9/dd/d15/f42 x:0 0 0 2024-05-06T12:14:27.659 INFO:tasks.workunit.client.0.smithi092.stdout:0/201: symlink d7/da/l44 0 2024-05-06T12:14:27.660 INFO:tasks.workunit.client.0.smithi092.stdout:7/202: dread - d9/dd/d15/f42 zero size 2024-05-06T12:14:27.661 INFO:tasks.workunit.client.0.smithi092.stdout:8/120: rmdir d9 39 2024-05-06T12:14:27.661 INFO:tasks.workunit.client.0.smithi092.stdout:0/202: write d7/da/d1e/f41 [1737595,93955] 0 2024-05-06T12:14:27.663 INFO:tasks.workunit.client.0.smithi092.stdout:7/203: creat d9/dd/f43 x:0 0 0 2024-05-06T12:14:27.663 INFO:tasks.workunit.client.0.smithi092.stdout:0/203: write d7/da/d1e/d26/f2b [1972839,10553] 0 2024-05-06T12:14:27.666 INFO:tasks.workunit.client.0.smithi092.stdout:6/149: truncate d4/f6 3653912 0 2024-05-06T12:14:27.667 INFO:tasks.workunit.client.0.smithi092.stdout:0/204: mknod d7/da/d1e/c45 0 2024-05-06T12:14:27.670 INFO:tasks.workunit.client.0.smithi092.stdout:0/205: write d7/da/f11 [852388,115193] 0 2024-05-06T12:14:27.673 INFO:tasks.workunit.client.0.smithi092.stdout:8/121: rename d9/d21/c26 to d9/dc/c2f 0 2024-05-06T12:14:27.674 INFO:tasks.workunit.client.0.smithi092.stdout:7/204: getdents d9/de/d10/d17/d21/d2e 0 2024-05-06T12:14:27.675 INFO:tasks.workunit.client.0.smithi092.stdout:6/150: dwrite d4/d20/f31 [0,4194304] 0 2024-05-06T12:14:27.675 INFO:tasks.workunit.client.0.smithi092.stdout:7/205: chown d9/de/d10 1 1 2024-05-06T12:14:27.676 INFO:tasks.workunit.client.0.smithi092.stdout:0/206: mkdir d7/da/d1e/d46 0 2024-05-06T12:14:27.676 INFO:tasks.workunit.client.0.smithi092.stdout:7/206: stat d9/dd/f1e 0 2024-05-06T12:14:27.686 INFO:tasks.workunit.client.0.smithi092.stdout:7/207: mknod d9/de/c44 0 2024-05-06T12:14:27.686 INFO:tasks.workunit.client.0.smithi092.stdout:7/208: fdatasync f3 0 2024-05-06T12:14:27.687 INFO:tasks.workunit.client.0.smithi092.stdout:7/209: truncate d9/dd/d15/f42 416641 0 2024-05-06T12:14:27.689 INFO:tasks.workunit.client.0.smithi092.stdout:7/210: read d9/dd/d15/f30 [4149119,102886] 0 2024-05-06T12:14:27.689 INFO:tasks.workunit.client.0.smithi092.stdout:7/211: read d9/de/d10/f12 [991944,104836] 0 2024-05-06T12:14:27.690 INFO:tasks.workunit.client.0.smithi092.stdout:0/207: creat d7/da/d1e/d39/f47 x:0 0 0 2024-05-06T12:14:27.691 INFO:tasks.workunit.client.0.smithi092.stdout:0/208: write d7/d43/f40 [611617,105881] 0 2024-05-06T12:14:27.694 INFO:tasks.workunit.client.0.smithi092.stdout:7/212: symlink d9/de/d10/l45 0 2024-05-06T12:14:27.695 INFO:tasks.workunit.client.0.smithi092.stdout:7/213: mknod d9/de/d10/c46 0 2024-05-06T12:14:27.696 INFO:tasks.workunit.client.0.smithi092.stdout:7/214: write d9/de/d1f/f35 [1044664,59060] 0 2024-05-06T12:14:27.699 INFO:tasks.workunit.client.0.smithi092.stdout:7/215: link d9/dd/d15/f30 d9/de/d10/f47 0 2024-05-06T12:14:27.700 INFO:tasks.workunit.client.0.smithi092.stdout:7/216: chown d9/lf 24 1 2024-05-06T12:14:27.700 INFO:tasks.workunit.client.0.smithi092.stdout:7/217: chown d9/de/d1f/f35 222702 1 2024-05-06T12:14:27.702 INFO:tasks.workunit.client.0.smithi092.stdout:7/218: mkdir d9/de/d1f/d37/d48 0 2024-05-06T12:14:27.703 INFO:tasks.workunit.client.0.smithi092.stdout:7/219: truncate d9/de/d1f/f35 1711049 0 2024-05-06T12:14:27.704 INFO:tasks.workunit.client.0.smithi092.stdout:7/220: readlink d9/dd/d23/l3b 0 2024-05-06T12:14:27.706 INFO:tasks.workunit.client.0.smithi092.stdout:7/221: rename d9/de/d10/f47 to d9/de/d10/d17/d21/f49 0 2024-05-06T12:14:27.707 INFO:tasks.workunit.client.0.smithi092.stdout:6/151: truncate d4/f5 1886102 0 2024-05-06T12:14:27.712 INFO:tasks.workunit.client.0.smithi092.stdout:7/222: dwrite d9/de/d10/f25 [0,4194304] 0 2024-05-06T12:14:27.718 INFO:tasks.workunit.client.0.smithi092.stdout:1/65: sync 2024-05-06T12:14:27.722 INFO:tasks.workunit.client.0.smithi092.stdout:8/122: truncate d9/f18 480783 0 2024-05-06T12:14:27.725 INFO:tasks.workunit.client.0.smithi092.stdout:8/123: stat d9/c1c 0 2024-05-06T12:14:27.725 INFO:tasks.workunit.client.0.smithi092.stdout:7/223: link d9/l19 d9/de/d10/l4a 0 2024-05-06T12:14:27.726 INFO:tasks.workunit.client.0.smithi092.stdout:7/224: chown d9/dd 348 1 2024-05-06T12:14:27.726 INFO:tasks.workunit.client.0.smithi092.stdout:7/225: stat d9/de/d1f/f31 0 2024-05-06T12:14:27.727 INFO:tasks.workunit.client.0.smithi092.stdout:8/124: creat d9/dc/f30 x:0 0 0 2024-05-06T12:14:27.728 INFO:tasks.workunit.client.0.smithi092.stdout:1/66: getdents d3/d10 0 2024-05-06T12:14:27.731 INFO:tasks.workunit.client.0.smithi092.stdout:1/67: mkdir d3/d10/d13/d19 0 2024-05-06T12:14:27.733 INFO:tasks.workunit.client.0.smithi092.stdout:7/226: creat d9/de/d1f/d37/d48/f4b x:0 0 0 2024-05-06T12:14:27.735 INFO:tasks.workunit.client.0.smithi092.stdout:1/68: dread d3/d10/f15 [0,4194304] 0 2024-05-06T12:14:27.737 INFO:tasks.workunit.client.0.smithi092.stdout:8/125: dwrite d9/f25 [0,4194304] 0 2024-05-06T12:14:27.739 INFO:tasks.workunit.client.0.smithi092.stdout:7/227: creat d9/de/d1f/f4c x:0 0 0 2024-05-06T12:14:27.740 INFO:tasks.workunit.client.0.smithi092.stdout:7/228: chown d9/de/d10/d17/d21/f49 0 1 2024-05-06T12:14:27.742 INFO:tasks.workunit.client.0.smithi092.stdout:1/69: mknod d3/d10/d13/c1a 0 2024-05-06T12:14:27.747 INFO:tasks.workunit.client.0.smithi092.stdout:7/229: dread f1 [4194304,4194304] 0 2024-05-06T12:14:27.752 INFO:tasks.workunit.client.0.smithi092.stdout:1/70: link d3/cf d3/d10/c1b 0 2024-05-06T12:14:27.753 INFO:tasks.workunit.client.0.smithi092.stdout:6/152: dwrite d4/f17 [0,4194304] 0 2024-05-06T12:14:27.754 INFO:tasks.workunit.client.0.smithi092.stdout:6/153: read d4/f11 [1315105,20157] 0 2024-05-06T12:14:27.755 INFO:tasks.workunit.client.0.smithi092.stdout:1/71: chown d3/d10/f11 429953660 1 2024-05-06T12:14:27.755 INFO:tasks.workunit.client.0.smithi092.stdout:8/126: dwrite d9/dc/f2d [0,4194304] 0 2024-05-06T12:14:27.755 INFO:tasks.workunit.client.0.smithi092.stdout:7/230: creat d9/de/d10/d17/d21/d2e/f4d x:0 0 0 2024-05-06T12:14:27.757 INFO:tasks.workunit.client.0.smithi092.stdout:1/72: chown d3/d10/d13/c1a 0 1 2024-05-06T12:14:27.758 INFO:tasks.workunit.client.0.smithi092.stdout:7/231: chown d9/de/d10/l28 125 1 2024-05-06T12:14:27.759 INFO:tasks.workunit.client.0.smithi092.stdout:8/127: chown d9/d21/l24 13 1 2024-05-06T12:14:27.761 INFO:tasks.workunit.client.0.smithi092.stdout:6/154: creat d4/d24/d26/f32 x:0 0 0 2024-05-06T12:14:27.762 INFO:tasks.workunit.client.0.smithi092.stdout:6/155: truncate d4/f19 1367715 0 2024-05-06T12:14:27.763 INFO:tasks.workunit.client.0.smithi092.stdout:8/128: fsync d9/dc/f27 0 2024-05-06T12:14:27.765 INFO:tasks.workunit.client.0.smithi092.stdout:7/232: rename d9/de/d10/c46 to d9/de/d2d/c4e 0 2024-05-06T12:14:27.770 INFO:tasks.workunit.client.0.smithi092.stdout:7/233: truncate d9/de/d10/d17/d21/f49 351830 0 2024-05-06T12:14:27.770 INFO:tasks.workunit.client.0.smithi092.stdout:1/73: dread d3/fe [0,4194304] 0 2024-05-06T12:14:27.770 INFO:tasks.workunit.client.0.smithi092.stdout:8/129: mknod d9/c31 0 2024-05-06T12:14:27.772 INFO:tasks.workunit.client.0.smithi092.stdout:7/234: read d9/de/d10/d17/d21/d2e/f32 [13514,5011] 0 2024-05-06T12:14:27.773 INFO:tasks.workunit.client.0.smithi092.stdout:8/130: truncate d9/dc/f28 875625 0 2024-05-06T12:14:27.777 INFO:tasks.workunit.client.0.smithi092.stdout:7/235: dread d9/de/d10/f12 [4194304,4194304] 0 2024-05-06T12:14:27.778 INFO:tasks.workunit.client.0.smithi092.stdout:7/236: write d9/de/d10/f2b [453447,100144] 0 2024-05-06T12:14:27.779 INFO:tasks.workunit.client.0.smithi092.stdout:7/237: write d9/de/d2d/f3a [898216,36926] 0 2024-05-06T12:14:27.780 INFO:tasks.workunit.client.0.smithi092.stdout:7/238: chown d9/dd 2590650 1 2024-05-06T12:14:27.780 INFO:tasks.workunit.client.0.smithi092.stdout:7/239: chown d9/dd 350352 1 2024-05-06T12:14:27.781 INFO:tasks.workunit.client.0.smithi092.stdout:7/240: truncate d9/de/d10/f27 79346 0 2024-05-06T12:14:27.783 INFO:tasks.workunit.client.0.smithi092.stdout:7/241: creat d9/de/d10/d17/d21/d2e/f4f x:0 0 0 2024-05-06T12:14:27.785 INFO:tasks.workunit.client.0.smithi092.stdout:7/242: dread - d9/de/d1f/d37/d48/f4b zero size 2024-05-06T12:14:27.786 INFO:tasks.workunit.client.0.smithi092.stdout:6/156: rename d4/f5 to d4/d20/d23/d30/f33 0 2024-05-06T12:14:27.789 INFO:tasks.workunit.client.0.smithi092.stdout:6/157: chown d4/l13 1638719 1 2024-05-06T12:14:27.791 INFO:tasks.workunit.client.0.smithi092.stdout:8/131: rename d9/dc/c2e to d9/d21/c32 0 2024-05-06T12:14:27.792 INFO:tasks.workunit.client.0.smithi092.stdout:1/74: truncate d3/d10/d13/f14 1762269 0 2024-05-06T12:14:27.794 INFO:tasks.workunit.client.0.smithi092.stdout:7/243: creat d9/de/f50 x:0 0 0 2024-05-06T12:14:27.798 INFO:tasks.workunit.client.0.smithi092.stdout:6/158: dwrite d4/d24/d26/f32 [0,4194304] 0 2024-05-06T12:14:27.800 INFO:tasks.workunit.client.0.smithi092.stdout:7/244: mknod d9/de/d1f/d37/d39/c51 0 2024-05-06T12:14:27.804 INFO:tasks.workunit.client.0.smithi092.stdout:7/245: creat d9/de/d1f/f52 x:0 0 0 2024-05-06T12:14:27.811 INFO:tasks.workunit.client.0.smithi092.stdout:7/246: dwrite d9/de/d10/f27 [0,4194304] 0 2024-05-06T12:14:27.814 INFO:tasks.workunit.client.0.smithi092.stdout:6/159: write d4/d20/d23/d30/f33 [1385131,23251] 0 2024-05-06T12:14:27.815 INFO:tasks.workunit.client.0.smithi092.stdout:6/160: stat d4/l10 0 2024-05-06T12:14:27.815 INFO:tasks.workunit.client.0.smithi092.stdout:7/247: symlink d9/de/d10/d17/d21/d2e/l53 0 2024-05-06T12:14:27.817 INFO:tasks.workunit.client.0.smithi092.stdout:6/161: mknod d4/d24/c34 0 2024-05-06T12:14:27.819 INFO:tasks.workunit.client.0.smithi092.stdout:7/248: creat d9/dd/d15/f54 x:0 0 0 2024-05-06T12:14:27.819 INFO:tasks.workunit.client.0.smithi092.stdout:7/249: chown d9/de/d1f 222702 1 2024-05-06T12:14:27.821 INFO:tasks.workunit.client.0.smithi092.stdout:6/162: unlink d4/d20/d22/c2c 0 2024-05-06T12:14:27.822 INFO:tasks.workunit.client.0.smithi092.stdout:7/250: mknod d9/de/d10/d17/d21/d38/c55 0 2024-05-06T12:14:27.825 INFO:tasks.workunit.client.0.smithi092.stdout:7/251: mknod d9/de/d10/d17/d21/c56 0 2024-05-06T12:14:27.828 INFO:tasks.workunit.client.0.smithi092.stdout:7/252: mknod d9/de/d10/d17/d21/d38/c57 0 2024-05-06T12:14:27.829 INFO:tasks.workunit.client.0.smithi092.stdout:6/163: dwrite d4/fb [0,4194304] 0 2024-05-06T12:14:27.833 INFO:tasks.workunit.client.0.smithi092.stdout:6/164: symlink d4/d20/d23/d30/l35 0 2024-05-06T12:14:27.834 INFO:tasks.workunit.client.0.smithi092.stdout:1/75: dread d3/f12 [4194304,4194304] 0 2024-05-06T12:14:27.835 INFO:tasks.workunit.client.0.smithi092.stdout:7/253: dread d9/de/d2d/f3a [0,4194304] 0 2024-05-06T12:14:27.836 INFO:tasks.workunit.client.0.smithi092.stdout:6/165: creat d4/d20/d23/d30/f36 x:0 0 0 2024-05-06T12:14:27.837 INFO:tasks.workunit.client.0.smithi092.stdout:1/76: creat d3/d10/d13/d19/f1c x:0 0 0 2024-05-06T12:14:27.838 INFO:tasks.workunit.client.0.smithi092.stdout:6/166: write d4/f1a [839433,87829] 0 2024-05-06T12:14:27.839 INFO:tasks.workunit.client.0.smithi092.stdout:7/254: symlink d9/de/d2d/l58 0 2024-05-06T12:14:27.839 INFO:tasks.workunit.client.0.smithi092.stdout:1/77: stat d3/d10/d13/d19 0 2024-05-06T12:14:27.840 INFO:tasks.workunit.client.0.smithi092.stdout:7/255: dread - d9/de/f50 zero size 2024-05-06T12:14:27.843 INFO:tasks.workunit.client.0.smithi092.stdout:7/256: creat d9/de/d1f/d37/d39/f59 x:0 0 0 2024-05-06T12:14:27.845 INFO:tasks.workunit.client.0.smithi092.stdout:6/167: fdatasync d4/f1a 0 2024-05-06T12:14:27.847 INFO:tasks.workunit.client.0.smithi092.stdout:6/168: chown d4/d20/d23/d30/l2b 438816394 1 2024-05-06T12:14:27.849 INFO:tasks.workunit.client.0.smithi092.stdout:7/257: dwrite f1 [0,4194304] 0 2024-05-06T12:14:27.851 INFO:tasks.workunit.client.0.smithi092.stdout:6/169: creat d4/d20/d23/f37 x:0 0 0 2024-05-06T12:14:27.852 INFO:tasks.workunit.client.0.smithi092.stdout:7/258: creat d9/f5a x:0 0 0 2024-05-06T12:14:27.856 INFO:tasks.workunit.client.0.smithi092.stdout:7/259: unlink d9/de/d10/d17/d21/d38/c57 0 2024-05-06T12:14:27.856 INFO:tasks.workunit.client.0.smithi092.stdout:1/78: rmdir d3/d10/d13/d19 39 2024-05-06T12:14:27.858 INFO:tasks.workunit.client.0.smithi092.stdout:6/170: dwrite d4/fb [0,4194304] 0 2024-05-06T12:14:27.860 INFO:tasks.workunit.client.0.smithi092.stdout:7/260: mknod d9/de/d10/d17/c5b 0 2024-05-06T12:14:27.860 INFO:tasks.workunit.client.0.smithi092.stdout:1/79: chown d3/d10/d13/f14 2014832 1 2024-05-06T12:14:27.862 INFO:tasks.workunit.client.0.smithi092.stdout:6/171: mkdir d4/d24/d38 0 2024-05-06T12:14:27.862 INFO:tasks.workunit.client.0.smithi092.stdout:1/80: truncate d3/f17 424981 0 2024-05-06T12:14:27.864 INFO:tasks.workunit.client.0.smithi092.stdout:6/172: creat d4/d24/f39 x:0 0 0 2024-05-06T12:14:27.865 INFO:tasks.workunit.client.0.smithi092.stdout:7/261: link d9/de/d10/l36 d9/dd/d15/l5c 0 2024-05-06T12:14:27.866 INFO:tasks.workunit.client.0.smithi092.stdout:1/81: symlink d3/l1d 0 2024-05-06T12:14:27.866 INFO:tasks.workunit.client.0.smithi092.stdout:7/262: write f3 [2557336,4342] 0 2024-05-06T12:14:27.867 INFO:tasks.workunit.client.0.smithi092.stdout:1/82: chown d3/c6 6588 1 2024-05-06T12:14:27.873 INFO:tasks.workunit.client.0.smithi092.stdout:6/173: dwrite d4/fb [0,4194304] 0 2024-05-06T12:14:27.873 INFO:tasks.workunit.client.0.smithi092.stdout:1/83: creat d3/d10/d13/d19/f1e x:0 0 0 2024-05-06T12:14:27.875 INFO:tasks.workunit.client.0.smithi092.stdout:1/84: read - d3/d10/d13/d19/f1e zero size 2024-05-06T12:14:27.876 INFO:tasks.workunit.client.0.smithi092.stdout:7/263: dwrite d9/de/d10/d17/d21/d2e/f4d [0,4194304] 0 2024-05-06T12:14:27.881 INFO:tasks.workunit.client.0.smithi092.stdout:6/174: rename d4/d20/d23/f37 to d4/d24/d26/f3a 0 2024-05-06T12:14:27.883 INFO:tasks.workunit.client.0.smithi092.stdout:7/264: unlink d9/dd/f1e 0 2024-05-06T12:14:27.884 INFO:tasks.workunit.client.0.smithi092.stdout:1/85: fsync d3/f8 0 2024-05-06T12:14:27.886 INFO:tasks.workunit.client.0.smithi092.stdout:1/86: dread d3/fe [0,4194304] 0 2024-05-06T12:14:27.886 INFO:tasks.workunit.client.0.smithi092.stdout:1/87: stat d3/f8 0 2024-05-06T12:14:27.888 INFO:tasks.workunit.client.0.smithi092.stdout:6/175: creat d4/d20/f3b x:0 0 0 2024-05-06T12:14:27.891 INFO:tasks.workunit.client.0.smithi092.stdout:1/88: fsync d3/fe 0 2024-05-06T12:14:27.892 INFO:tasks.workunit.client.0.smithi092.stdout:1/89: dread - d3/d10/d13/d19/f1c zero size 2024-05-06T12:14:27.893 INFO:tasks.workunit.client.0.smithi092.stdout:7/265: truncate d9/dd/d15/f30 474063 0 2024-05-06T12:14:27.901 INFO:tasks.workunit.client.0.smithi092.stdout:7/266: dread d9/dd/d15/f30 [0,4194304] 0 2024-05-06T12:14:27.902 INFO:tasks.workunit.client.0.smithi092.stdout:6/176: truncate d4/d20/d23/f28 829142 0 2024-05-06T12:14:27.905 INFO:tasks.workunit.client.0.smithi092.stdout:7/267: creat d9/de/d10/d17/d21/f5d x:0 0 0 2024-05-06T12:14:27.905 INFO:tasks.workunit.client.0.smithi092.stdout:6/177: dread d4/d20/f1c [0,4194304] 0 2024-05-06T12:14:27.907 INFO:tasks.workunit.client.0.smithi092.stdout:1/90: dwrite d3/d10/f15 [0,4194304] 0 2024-05-06T12:14:27.907 INFO:tasks.workunit.client.0.smithi092.stdout:6/178: creat d4/d24/f3c x:0 0 0 2024-05-06T12:14:27.908 INFO:tasks.workunit.client.0.smithi092.stdout:6/179: fsync d4/d20/f31 0 2024-05-06T12:14:27.912 INFO:tasks.workunit.client.0.smithi092.stdout:7/268: rename d9/de/d10/d17/d21/d2e/l53 to d9/dd/d15/l5e 0 2024-05-06T12:14:27.913 INFO:tasks.workunit.client.0.smithi092.stdout:7/269: dread - d9/f5a zero size 2024-05-06T12:14:27.915 INFO:tasks.workunit.client.0.smithi092.stdout:7/270: rename d9/de/d10/d17/d21/d2e to d9/de/d10/d17/d21/d2e/d5f 22 2024-05-06T12:14:27.915 INFO:tasks.workunit.client.0.smithi092.stdout:1/91: creat d3/d10/d13/f1f x:0 0 0 2024-05-06T12:14:27.916 INFO:tasks.workunit.client.0.smithi092.stdout:1/92: read d3/fe [454227,110785] 0 2024-05-06T12:14:27.916 INFO:tasks.workunit.client.0.smithi092.stdout:6/180: creat d4/f3d x:0 0 0 2024-05-06T12:14:27.918 INFO:tasks.workunit.client.0.smithi092.stdout:1/93: write d3/d10/d13/f1f [358791,107417] 0 2024-05-06T12:14:27.948 INFO:tasks.workunit.client.0.smithi092.stdout:6/181: rmdir d4/d20/d23/d30 39 2024-05-06T12:14:27.950 INFO:tasks.workunit.client.0.smithi092.stdout:6/182: creat d4/d24/f3e x:0 0 0 2024-05-06T12:14:27.955 INFO:tasks.workunit.client.0.smithi092.stdout:6/183: write d4/f6 [553985,53206] 0 2024-05-06T12:14:27.957 INFO:tasks.workunit.client.0.smithi092.stdout:6/184: chown d4/d20/c1f 12611 1 2024-05-06T12:14:27.958 INFO:tasks.workunit.client.0.smithi092.stdout:7/271: dwrite d9/de/d10/d17/d21/d2e/f32 [0,4194304] 0 2024-05-06T12:14:27.959 INFO:tasks.workunit.client.0.smithi092.stdout:7/272: fdatasync d9/dd/d15/f42 0 2024-05-06T12:14:27.960 INFO:tasks.workunit.client.0.smithi092.stdout:1/94: dread d3/f8 [0,4194304] 0 2024-05-06T12:14:27.961 INFO:tasks.workunit.client.0.smithi092.stdout:6/185: symlink d4/l3f 0 2024-05-06T12:14:27.962 INFO:tasks.workunit.client.0.smithi092.stdout:1/95: chown d3/c18 211 1 2024-05-06T12:14:27.966 INFO:tasks.workunit.client.0.smithi092.stdout:7/273: rmdir d9 39 2024-05-06T12:14:27.967 INFO:tasks.workunit.client.0.smithi092.stdout:6/186: mknod d4/d20/c40 0 2024-05-06T12:14:27.970 INFO:tasks.workunit.client.0.smithi092.stdout:7/274: creat d9/de/d1f/d37/f60 x:0 0 0 2024-05-06T12:14:27.971 INFO:tasks.workunit.client.0.smithi092.stdout:1/96: creat d3/f20 x:0 0 0 2024-05-06T12:14:27.974 INFO:tasks.workunit.client.0.smithi092.stdout:6/187: dwrite d4/f17 [0,4194304] 0 2024-05-06T12:14:27.975 INFO:tasks.workunit.client.0.smithi092.stdout:6/188: chown d4/f17 553237751 1 2024-05-06T12:14:27.976 INFO:tasks.workunit.client.0.smithi092.stdout:1/97: unlink d3/fe 0 2024-05-06T12:14:27.977 INFO:tasks.workunit.client.0.smithi092.stdout:6/189: rmdir d4/d24 39 2024-05-06T12:14:27.979 INFO:tasks.workunit.client.0.smithi092.stdout:6/190: fdatasync d4/d20/d23/d30/f36 0 2024-05-06T12:14:27.984 INFO:tasks.workunit.client.0.smithi092.stdout:6/191: dread d4/d20/d23/f28 [0,4194304] 0 2024-05-06T12:14:27.985 INFO:tasks.workunit.client.0.smithi092.stdout:7/275: dwrite d9/fb [0,4194304] 0 2024-05-06T12:14:27.987 INFO:tasks.workunit.client.0.smithi092.stdout:7/276: fdatasync d9/de/f2a 0 2024-05-06T12:14:27.988 INFO:tasks.workunit.client.0.smithi092.stdout:7/277: stat d9/dd/d23/l33 0 2024-05-06T12:14:27.988 INFO:tasks.workunit.client.0.smithi092.stdout:6/192: symlink d4/d24/l41 0 2024-05-06T12:14:27.990 INFO:tasks.workunit.client.0.smithi092.stdout:6/193: mknod d4/d20/d22/c42 0 2024-05-06T12:14:27.991 INFO:tasks.workunit.client.0.smithi092.stdout:6/194: write d4/f3d [91945,109247] 0 2024-05-06T12:14:27.998 INFO:tasks.workunit.client.0.smithi092.stdout:6/195: dwrite d4/f1a [0,4194304] 0 2024-05-06T12:14:27.999 INFO:tasks.workunit.client.0.smithi092.stdout:6/196: truncate d4/d24/d26/f3a 229482 0 2024-05-06T12:14:28.005 INFO:tasks.workunit.client.0.smithi092.stdout:7/278: fsync d9/fb 0 2024-05-06T12:14:28.006 INFO:tasks.workunit.client.0.smithi092.stdout:6/197: link d4/d20/c40 d4/d20/c43 0 2024-05-06T12:14:28.011 INFO:tasks.workunit.client.0.smithi092.stdout:6/198: unlink d4/d24/d26/f3a 0 2024-05-06T12:14:28.013 INFO:tasks.workunit.client.0.smithi092.stdout:7/279: link d9/dd/d15/f14 d9/de/d1f/f61 0 2024-05-06T12:14:28.015 INFO:tasks.workunit.client.0.smithi092.stdout:6/199: dread d4/d20/f31 [0,4194304] 0 2024-05-06T12:14:28.019 INFO:tasks.workunit.client.0.smithi092.stdout:6/200: creat d4/d24/d38/f44 x:0 0 0 2024-05-06T12:14:28.019 INFO:tasks.workunit.client.0.smithi092.stdout:7/280: dwrite f3 [0,4194304] 0 2024-05-06T12:14:28.021 INFO:tasks.workunit.client.0.smithi092.stdout:6/201: creat d4/d20/d22/f45 x:0 0 0 2024-05-06T12:14:28.023 INFO:tasks.workunit.client.0.smithi092.stdout:6/202: dread - d4/d20/d23/d30/f36 zero size 2024-05-06T12:14:28.024 INFO:tasks.workunit.client.0.smithi092.stdout:6/203: chown d4/f11 2109498902 1 2024-05-06T12:14:28.028 INFO:tasks.workunit.client.0.smithi092.stdout:7/281: dwrite d9/de/d10/d17/d21/d2e/f4d [4194304,4194304] 0 2024-05-06T12:14:28.031 INFO:tasks.workunit.client.0.smithi092.stdout:6/204: rename d4/d24/d38/f44 to d4/d20/d23/f46 0 2024-05-06T12:14:28.033 INFO:tasks.workunit.client.0.smithi092.stdout:7/282: mkdir d9/d62 0 2024-05-06T12:14:28.035 INFO:tasks.workunit.client.0.smithi092.stdout:6/205: rename d4/d20/d23/d30/l2b to d4/d20/l47 0 2024-05-06T12:14:28.036 INFO:tasks.workunit.client.0.smithi092.stdout:7/283: symlink d9/dd/d15/l63 0 2024-05-06T12:14:28.039 INFO:tasks.workunit.client.0.smithi092.stdout:7/284: mknod d9/dd/d23/c64 0 2024-05-06T12:14:28.041 INFO:tasks.workunit.client.0.smithi092.stdout:7/285: mkdir d9/de/d1f/d37/d48/d65 0 2024-05-06T12:14:28.042 INFO:tasks.workunit.client.0.smithi092.stdout:6/206: dwrite d4/d24/f3e [0,4194304] 0 2024-05-06T12:14:28.046 INFO:tasks.workunit.client.0.smithi092.stdout:6/207: mkdir d4/d20/d23/d30/d48 0 2024-05-06T12:14:28.048 INFO:tasks.workunit.client.0.smithi092.stdout:6/208: chown d4/d20/l21 121037 1 2024-05-06T12:14:28.049 INFO:tasks.workunit.client.0.smithi092.stdout:7/286: dwrite d9/de/d10/f25 [0,4194304] 0 2024-05-06T12:14:28.051 INFO:tasks.workunit.client.0.smithi092.stdout:6/209: mknod d4/d24/c49 0 2024-05-06T12:14:28.053 INFO:tasks.workunit.client.0.smithi092.stdout:7/287: write d9/de/d2d/f3a [530675,76782] 0 2024-05-06T12:14:28.057 INFO:tasks.workunit.client.0.smithi092.stdout:6/210: link d4/f17 d4/d20/d23/d30/d48/f4a 0 2024-05-06T12:14:28.057 INFO:tasks.workunit.client.0.smithi092.stdout:7/288: rmdir d9/de/d10/d17/d21/d2e 39 2024-05-06T12:14:28.058 INFO:tasks.workunit.client.0.smithi092.stdout:7/289: fsync d9/dd/d15/f42 0 2024-05-06T12:14:28.058 INFO:tasks.workunit.client.0.smithi092.stdout:6/211: read d4/d20/d23/d30/f33 [596570,5959] 0 2024-05-06T12:14:28.059 INFO:tasks.workunit.client.0.smithi092.stdout:7/290: readlink d9/dd/l1a 0 2024-05-06T12:14:28.059 INFO:tasks.workunit.client.0.smithi092.stdout:7/291: fdatasync f3 0 2024-05-06T12:14:28.060 INFO:tasks.workunit.client.0.smithi092.stdout:6/212: chown d4/d24/d26/l2e 32092 1 2024-05-06T12:14:28.063 INFO:tasks.workunit.client.0.smithi092.stdout:6/213: unlink d4/d24/f3c 0 2024-05-06T12:14:28.064 INFO:tasks.workunit.client.0.smithi092.stdout:7/292: dread d9/de/d10/f27 [0,4194304] 0 2024-05-06T12:14:28.065 INFO:tasks.workunit.client.0.smithi092.stdout:6/214: fdatasync d4/f17 0 2024-05-06T12:14:28.068 INFO:tasks.workunit.client.0.smithi092.stdout:6/215: creat d4/d24/d26/f4b x:0 0 0 2024-05-06T12:14:28.069 INFO:tasks.workunit.client.0.smithi092.stdout:7/293: creat d9/de/d1f/d37/d48/d65/f66 x:0 0 0 2024-05-06T12:14:28.070 INFO:tasks.workunit.client.0.smithi092.stdout:6/216: read d4/d20/d23/d30/f33 [1495023,14404] 0 2024-05-06T12:14:28.074 INFO:tasks.workunit.client.0.smithi092.stdout:7/294: dread d9/de/d10/f27 [0,4194304] 0 2024-05-06T12:14:28.077 INFO:tasks.workunit.client.0.smithi092.stdout:7/295: mknod d9/de/c67 0 2024-05-06T12:14:28.077 INFO:tasks.workunit.client.0.smithi092.stdout:6/217: dwrite d4/d20/f1b [0,4194304] 0 2024-05-06T12:14:28.078 INFO:tasks.workunit.client.0.smithi092.stdout:6/218: write d4/d20/f1b [1315021,103656] 0 2024-05-06T12:14:28.081 INFO:tasks.workunit.client.0.smithi092.stdout:6/219: write d4/f19 [473081,38330] 0 2024-05-06T12:14:28.082 INFO:tasks.workunit.client.0.smithi092.stdout:7/296: rename d9/dd/d23/c64 to d9/de/c68 0 2024-05-06T12:14:28.083 INFO:tasks.workunit.client.0.smithi092.stdout:7/297: fsync d9/de/f2a 0 2024-05-06T12:14:28.084 INFO:tasks.workunit.client.0.smithi092.stdout:6/220: readlink d4/d20/l47 0 2024-05-06T12:14:28.086 INFO:tasks.workunit.client.0.smithi092.stdout:6/221: mkdir d4/d4c 0 2024-05-06T12:14:28.088 INFO:tasks.workunit.client.0.smithi092.stdout:6/222: dread d4/f19 [0,4194304] 0 2024-05-06T12:14:28.089 INFO:tasks.workunit.client.0.smithi092.stdout:6/223: stat d4/lf 0 2024-05-06T12:14:28.092 INFO:tasks.workunit.client.0.smithi092.stdout:6/224: dread d4/d20/f1b [0,4194304] 0 2024-05-06T12:14:28.093 INFO:tasks.workunit.client.0.smithi092.stdout:6/225: chown d4/d20/d23/c2d 13719 1 2024-05-06T12:14:28.095 INFO:tasks.workunit.client.0.smithi092.stdout:6/226: unlink d4/d20/d23/d30/l35 0 2024-05-06T12:14:28.098 INFO:tasks.workunit.client.0.smithi092.stdout:6/227: getdents d4/d24/d38 0 2024-05-06T12:14:28.101 INFO:tasks.workunit.client.0.smithi092.stdout:6/228: symlink d4/d20/d23/d30/d48/l4d 0 2024-05-06T12:14:28.101 INFO:tasks.workunit.client.0.smithi092.stdout:6/229: dread - d4/d24/f39 zero size 2024-05-06T12:14:28.102 INFO:tasks.workunit.client.0.smithi092.stdout:6/230: dread - d4/d24/f39 zero size 2024-05-06T12:14:28.103 INFO:tasks.workunit.client.0.smithi092.stdout:6/231: write d4/fb [2798293,68371] 0 2024-05-06T12:14:28.112 INFO:tasks.workunit.client.0.smithi092.stdout:3/115: sync 2024-05-06T12:14:28.112 INFO:tasks.workunit.client.0.smithi092.stdout:5/125: sync 2024-05-06T12:14:28.113 INFO:tasks.workunit.client.0.smithi092.stdout:5/126: write d4/d34/f1b [1507663,24121] 0 2024-05-06T12:14:28.114 INFO:tasks.workunit.client.0.smithi092.stdout:3/116: mknod d8/d24/c26 0 2024-05-06T12:14:28.114 INFO:tasks.workunit.client.0.smithi092.stdout:5/127: truncate d4/d34/f2e 919536 0 2024-05-06T12:14:28.116 INFO:tasks.workunit.client.0.smithi092.stdout:3/117: truncate d8/db/f11 4862810 0 2024-05-06T12:14:28.116 INFO:tasks.workunit.client.0.smithi092.stdout:3/118: stat l6 0 2024-05-06T12:14:28.116 INFO:tasks.workunit.client.0.smithi092.stdout:5/128: mkdir d4/d5/d10/d12/d37 0 2024-05-06T12:14:28.117 INFO:tasks.workunit.client.0.smithi092.stdout:3/119: chown d8/db/d14/f18 96204 1 2024-05-06T12:14:28.119 INFO:tasks.workunit.client.0.smithi092.stdout:3/120: mkdir d8/d24/d27 0 2024-05-06T12:14:28.119 INFO:tasks.workunit.client.0.smithi092.stdout:3/121: write d8/d24/f16 [141000,20763] 0 2024-05-06T12:14:28.131 INFO:tasks.workunit.client.0.smithi092.stdout:3/122: dread f3 [0,4194304] 0 2024-05-06T12:14:28.131 INFO:tasks.workunit.client.0.smithi092.stdout:5/129: dread d4/f9 [0,4194304] 0 2024-05-06T12:14:28.132 INFO:tasks.workunit.client.0.smithi092.stdout:3/123: write d8/f1d [2012096,110178] 0 2024-05-06T12:14:28.134 INFO:tasks.workunit.client.0.smithi092.stdout:3/124: write d8/d24/f16 [52693,119204] 0 2024-05-06T12:14:28.224 INFO:tasks.workunit.client.0.smithi092.stdout:5/130: getdents d4/d5/d30 0 2024-05-06T12:14:28.226 INFO:tasks.workunit.client.0.smithi092.stdout:3/125: getdents d8/d9 0 2024-05-06T12:14:28.227 INFO:tasks.workunit.client.0.smithi092.stdout:5/131: rename c1 to d4/d5/c38 0 2024-05-06T12:14:28.228 INFO:tasks.workunit.client.0.smithi092.stdout:3/126: read d8/fa [3070596,72969] 0 2024-05-06T12:14:28.229 INFO:tasks.workunit.client.0.smithi092.stdout:3/127: chown d8/d9/c1e 819417952 1 2024-05-06T12:14:28.229 INFO:tasks.workunit.client.0.smithi092.stdout:3/128: chown d8/db/d14 555757 1 2024-05-06T12:14:28.231 INFO:tasks.workunit.client.0.smithi092.stdout:3/129: write d8/d1b/f1f [748759,56324] 0 2024-05-06T12:14:28.236 INFO:tasks.workunit.client.0.smithi092.stdout:5/132: dwrite d4/d5/f2a [0,4194304] 0 2024-05-06T12:14:28.239 INFO:tasks.workunit.client.0.smithi092.stdout:3/130: dwrite f2 [0,4194304] 0 2024-05-06T12:14:28.240 INFO:tasks.workunit.client.0.smithi092.stdout:5/133: creat d4/d34/d28/f39 x:0 0 0 2024-05-06T12:14:28.242 INFO:tasks.workunit.client.0.smithi092.stdout:5/134: creat d4/d34/d28/f3a x:0 0 0 2024-05-06T12:14:28.243 INFO:tasks.workunit.client.0.smithi092.stdout:3/131: getdents d8/d9 0 2024-05-06T12:14:28.245 INFO:tasks.workunit.client.0.smithi092.stdout:5/135: creat d4/d5/f3b x:0 0 0 2024-05-06T12:14:28.246 INFO:tasks.workunit.client.0.smithi092.stdout:5/136: read d4/d5/d10/f17 [1202069,128985] 0 2024-05-06T12:14:28.246 INFO:tasks.workunit.client.0.smithi092.stdout:3/132: rmdir d8/d1b 39 2024-05-06T12:14:28.247 INFO:tasks.workunit.client.0.smithi092.stdout:4/108: sync 2024-05-06T12:14:28.248 INFO:tasks.workunit.client.0.smithi092.stdout:5/137: write d4/d5/f2a [2647601,101586] 0 2024-05-06T12:14:28.251 INFO:tasks.workunit.client.0.smithi092.stdout:5/138: creat d4/d5/d30/f3c x:0 0 0 2024-05-06T12:14:28.251 INFO:tasks.workunit.client.0.smithi092.stdout:3/133: getdents d8/d24/d27 0 2024-05-06T12:14:28.253 INFO:tasks.workunit.client.0.smithi092.stdout:4/109: getdents d9/df 0 2024-05-06T12:14:28.254 INFO:tasks.workunit.client.0.smithi092.stdout:4/110: chown d9/f1c 215 1 2024-05-06T12:14:28.256 INFO:tasks.workunit.client.0.smithi092.stdout:3/134: truncate f4 328652 0 2024-05-06T12:14:28.257 INFO:tasks.workunit.client.0.smithi092.stdout:3/135: fdatasync f4 0 2024-05-06T12:14:28.258 INFO:tasks.workunit.client.0.smithi092.stdout:5/139: dread f0 [0,4194304] 0 2024-05-06T12:14:28.259 INFO:tasks.workunit.client.0.smithi092.stdout:4/111: getdents d9 0 2024-05-06T12:14:28.259 INFO:tasks.workunit.client.0.smithi092.stdout:5/140: read - d4/d34/d28/f3a zero size 2024-05-06T12:14:28.260 INFO:tasks.workunit.client.0.smithi092.stdout:3/136: mknod d8/db/c28 0 2024-05-06T12:14:28.261 INFO:tasks.workunit.client.0.smithi092.stdout:5/141: fdatasync d4/d5/d10/f33 0 2024-05-06T12:14:28.265 INFO:tasks.workunit.client.0.smithi092.stdout:4/112: dread d9/f1c [0,4194304] 0 2024-05-06T12:14:28.271 INFO:tasks.workunit.client.0.smithi092.stdout:3/137: dwrite f2 [0,4194304] 0 2024-05-06T12:14:28.273 INFO:tasks.workunit.client.0.smithi092.stdout:5/142: dwrite d4/d34/d28/f39 [0,4194304] 0 2024-05-06T12:14:28.277 INFO:tasks.workunit.client.0.smithi092.stdout:4/113: dwrite d9/dd/f16 [0,4194304] 0 2024-05-06T12:14:28.279 INFO:tasks.workunit.client.0.smithi092.stdout:3/138: link d8/d24/c21 d8/d24/d27/c29 0 2024-05-06T12:14:28.280 INFO:tasks.workunit.client.0.smithi092.stdout:4/114: mkdir d9/d27 0 2024-05-06T12:14:28.281 INFO:tasks.workunit.client.0.smithi092.stdout:3/139: truncate d8/d9/f17 181791 0 2024-05-06T12:14:28.287 INFO:tasks.workunit.client.0.smithi092.stdout:3/140: dwrite d8/f1d [4194304,4194304] 0 2024-05-06T12:14:28.290 INFO:tasks.workunit.client.0.smithi092.stdout:3/141: creat d8/d9/f2a x:0 0 0 2024-05-06T12:14:28.291 INFO:tasks.workunit.client.0.smithi092.stdout:3/142: chown f5 700 1 2024-05-06T12:14:28.298 INFO:tasks.workunit.client.0.smithi092.stdout:3/143: dwrite d8/d1b/f1f [0,4194304] 0 2024-05-06T12:14:28.301 INFO:tasks.workunit.client.0.smithi092.stdout:3/144: rename d8/d9/f2a to d8/d1b/f2b 0 2024-05-06T12:14:28.302 INFO:tasks.workunit.client.0.smithi092.stdout:3/145: symlink d8/d24/l2c 0 2024-05-06T12:14:28.303 INFO:tasks.workunit.client.0.smithi092.stdout:3/146: write d8/f1d [3811092,122925] 0 2024-05-06T12:14:28.309 INFO:tasks.workunit.client.0.smithi092.stdout:5/143: dread d4/d5/d10/f33 [0,4194304] 0 2024-05-06T12:14:28.371 INFO:tasks.workunit.client.0.smithi092.stdout:3/147: truncate f2 6231677 0 2024-05-06T12:14:28.372 INFO:tasks.workunit.client.0.smithi092.stdout:5/144: truncate d4/d5/d10/d12/f18 82789 0 2024-05-06T12:14:28.372 INFO:tasks.workunit.client.0.smithi092.stdout:3/148: rmdir d8/d24/d27 39 2024-05-06T12:14:28.373 INFO:tasks.workunit.client.0.smithi092.stdout:5/145: write d4/f9 [3413189,18096] 0 2024-05-06T12:14:28.374 INFO:tasks.workunit.client.0.smithi092.stdout:3/149: symlink d8/db/l2d 0 2024-05-06T12:14:28.375 INFO:tasks.workunit.client.0.smithi092.stdout:5/146: symlink d4/d5/d10/d31/l3d 0 2024-05-06T12:14:28.377 INFO:tasks.workunit.client.0.smithi092.stdout:5/147: rmdir d4/d5/d10/d12/d1f/d20/d23 39 2024-05-06T12:14:28.379 INFO:tasks.workunit.client.0.smithi092.stdout:5/148: fdatasync d4/d5/d10/d12/d1f/d20/d23/f25 0 2024-05-06T12:14:28.383 INFO:tasks.workunit.client.0.smithi092.stdout:5/149: dread d4/d34/d28/f39 [0,4194304] 0 2024-05-06T12:14:28.385 INFO:tasks.workunit.client.0.smithi092.stdout:5/150: unlink d4/d34/f2e 0 2024-05-06T12:14:28.385 INFO:tasks.workunit.client.0.smithi092.stdout:2/75: sync 2024-05-06T12:14:28.386 INFO:tasks.workunit.client.0.smithi092.stdout:2/76: chown df/d17 101 1 2024-05-06T12:14:28.386 INFO:tasks.workunit.client.0.smithi092.stdout:5/151: read d4/f9 [759202,98314] 0 2024-05-06T12:14:28.386 INFO:tasks.workunit.client.0.smithi092.stdout:2/77: readlink df/l10 0 2024-05-06T12:14:28.390 INFO:tasks.workunit.client.0.smithi092.stdout:3/150: truncate f5 1674832 0 2024-05-06T12:14:28.394 INFO:tasks.workunit.client.0.smithi092.stdout:2/78: dwrite fe [0,4194304] 0 2024-05-06T12:14:28.394 INFO:tasks.workunit.client.0.smithi092.stdout:3/151: creat d8/d24/d27/f2e x:0 0 0 2024-05-06T12:14:28.398 INFO:tasks.workunit.client.0.smithi092.stdout:5/152: dwrite d4/d34/d28/f39 [4194304,4194304] 0 2024-05-06T12:14:28.405 INFO:tasks.workunit.client.0.smithi092.stdout:2/79: dread f1 [0,4194304] 0 2024-05-06T12:14:28.406 INFO:tasks.workunit.client.0.smithi092.stdout:3/152: rename d8/f1d to d8/d24/d27/f2f 0 2024-05-06T12:14:28.406 INFO:tasks.workunit.client.0.smithi092.stdout:3/153: fsync d8/db/f11 0 2024-05-06T12:14:28.408 INFO:tasks.workunit.client.0.smithi092.stdout:2/80: mkdir df/d1b 0 2024-05-06T12:14:28.409 INFO:tasks.workunit.client.0.smithi092.stdout:3/154: unlink d8/d24/l2c 0 2024-05-06T12:14:28.410 INFO:tasks.workunit.client.0.smithi092.stdout:3/155: readlink d8/db/l20 0 2024-05-06T12:14:28.411 INFO:tasks.workunit.client.0.smithi092.stdout:2/81: creat df/d1b/f1c x:0 0 0 2024-05-06T12:14:28.413 INFO:tasks.workunit.client.0.smithi092.stdout:2/82: write fe [2208379,45402] 0 2024-05-06T12:14:28.413 INFO:tasks.workunit.client.0.smithi092.stdout:3/156: rename d8/ce to d8/db/d14/c30 0 2024-05-06T12:14:28.414 INFO:tasks.workunit.client.0.smithi092.stdout:2/83: stat c2 0 2024-05-06T12:14:28.416 INFO:tasks.workunit.client.0.smithi092.stdout:3/157: dread d8/d24/f16 [0,4194304] 0 2024-05-06T12:14:28.416 INFO:tasks.workunit.client.0.smithi092.stdout:2/84: mkdir df/d17/d1d 0 2024-05-06T12:14:28.417 INFO:tasks.workunit.client.0.smithi092.stdout:2/85: write df/d1b/f1c [780836,62385] 0 2024-05-06T12:14:28.417 INFO:tasks.workunit.client.0.smithi092.stdout:5/153: dread d4/d34/f1b [0,4194304] 0 2024-05-06T12:14:28.419 INFO:tasks.workunit.client.0.smithi092.stdout:3/158: creat d8/d24/f31 x:0 0 0 2024-05-06T12:14:28.421 INFO:tasks.workunit.client.0.smithi092.stdout:5/154: readlink d4/d5/lf 0 2024-05-06T12:14:28.422 INFO:tasks.workunit.client.0.smithi092.stdout:2/86: creat df/f1e x:0 0 0 2024-05-06T12:14:28.424 INFO:tasks.workunit.client.0.smithi092.stdout:5/155: unlink d4/d5/d10/d12/c26 0 2024-05-06T12:14:28.428 INFO:tasks.workunit.client.0.smithi092.stdout:3/159: creat d8/d24/f32 x:0 0 0 2024-05-06T12:14:28.429 INFO:tasks.workunit.client.0.smithi092.stdout:3/160: write d8/db/d14/f18 [429173,48968] 0 2024-05-06T12:14:28.430 INFO:tasks.workunit.client.0.smithi092.stdout:5/156: creat d4/d5/d10/d12/d1f/d20/f3e x:0 0 0 2024-05-06T12:14:28.431 INFO:tasks.workunit.client.0.smithi092.stdout:2/87: getdents df/d1b 0 2024-05-06T12:14:28.431 INFO:tasks.workunit.client.0.smithi092.stdout:5/157: dread - d4/d5/f3b zero size 2024-05-06T12:14:28.434 INFO:tasks.workunit.client.0.smithi092.stdout:3/161: mkdir d8/d24/d33 0 2024-05-06T12:14:28.437 INFO:tasks.workunit.client.0.smithi092.stdout:2/88: dwrite df/d17/f18 [0,4194304] 0 2024-05-06T12:14:28.438 INFO:tasks.workunit.client.0.smithi092.stdout:2/89: chown df/d17/f19 6615442 1 2024-05-06T12:14:28.438 INFO:tasks.workunit.client.0.smithi092.stdout:3/162: mknod d8/d24/d33/c34 0 2024-05-06T12:14:28.439 INFO:tasks.workunit.client.0.smithi092.stdout:5/158: fsync d4/d5/d10/d12/d1f/d20/f3e 0 2024-05-06T12:14:28.439 INFO:tasks.workunit.client.0.smithi092.stdout:2/90: chown fd 0 1 2024-05-06T12:14:28.440 INFO:tasks.workunit.client.0.smithi092.stdout:5/159: dread - d4/d5/f1d zero size 2024-05-06T12:14:28.440 INFO:tasks.workunit.client.0.smithi092.stdout:7/298: sync 2024-05-06T12:14:28.440 INFO:tasks.workunit.client.0.smithi092.stdout:9/88: sync 2024-05-06T12:14:28.441 INFO:tasks.workunit.client.0.smithi092.stdout:3/163: truncate d8/db/d14/f18 1448286 0 2024-05-06T12:14:28.441 INFO:tasks.workunit.client.0.smithi092.stdout:2/91: write df/d17/f19 [1029064,92531] 0 2024-05-06T12:14:28.445 INFO:tasks.workunit.client.0.smithi092.stdout:5/160: mknod d4/d5/d10/c3f 0 2024-05-06T12:14:28.449 INFO:tasks.workunit.client.0.smithi092.stdout:9/89: getdents d16 0 2024-05-06T12:14:28.450 INFO:tasks.workunit.client.0.smithi092.stdout:2/92: rename fc to df/f1f 0 2024-05-06T12:14:28.454 INFO:tasks.workunit.client.0.smithi092.stdout:2/93: symlink df/d17/l20 0 2024-05-06T12:14:28.456 INFO:tasks.workunit.client.0.smithi092.stdout:3/164: dwrite d8/d24/f16 [0,4194304] 0 2024-05-06T12:14:28.456 INFO:tasks.workunit.client.0.smithi092.stdout:7/299: dwrite d9/de/d10/d17/d21/d2e/f32 [0,4194304] 0 2024-05-06T12:14:28.457 INFO:tasks.workunit.client.0.smithi092.stdout:7/300: dread - d9/de/d1f/f52 zero size 2024-05-06T12:14:28.460 INFO:tasks.workunit.client.0.smithi092.stdout:5/161: truncate d4/d34/d28/f39 7483254 0 2024-05-06T12:14:28.460 INFO:tasks.workunit.client.0.smithi092.stdout:2/94: read df/f11 [444758,74323] 0 2024-05-06T12:14:28.461 INFO:tasks.workunit.client.0.smithi092.stdout:2/95: fdatasync df/f1e 0 2024-05-06T12:14:28.461 INFO:tasks.workunit.client.0.smithi092.stdout:3/165: symlink d8/d9/l35 0 2024-05-06T12:14:28.462 INFO:tasks.workunit.client.0.smithi092.stdout:5/162: read - d4/d5/d10/d12/d1f/d20/d23/f25 zero size 2024-05-06T12:14:28.465 INFO:tasks.workunit.client.0.smithi092.stdout:7/301: creat d9/de/d1f/f69 x:0 0 0 2024-05-06T12:14:28.467 INFO:tasks.workunit.client.0.smithi092.stdout:5/163: write d4/d34/f1b [207785,113890] 0 2024-05-06T12:14:28.467 INFO:tasks.workunit.client.0.smithi092.stdout:3/166: symlink d8/d9/l36 0 2024-05-06T12:14:28.469 INFO:tasks.workunit.client.0.smithi092.stdout:2/96: creat df/d17/d1d/f21 x:0 0 0 2024-05-06T12:14:28.470 INFO:tasks.workunit.client.0.smithi092.stdout:7/302: mkdir d9/de/d10/d17/d6a 0 2024-05-06T12:14:28.472 INFO:tasks.workunit.client.0.smithi092.stdout:7/303: readlink d9/dd/d23/l33 0 2024-05-06T12:14:28.473 INFO:tasks.workunit.client.0.smithi092.stdout:2/97: dread f1 [0,4194304] 0 2024-05-06T12:14:28.474 INFO:tasks.workunit.client.0.smithi092.stdout:7/304: dread - d9/de/f50 zero size 2024-05-06T12:14:28.475 INFO:tasks.workunit.client.0.smithi092.stdout:5/164: dwrite d4/d5/d10/f33 [0,4194304] 0 2024-05-06T12:14:28.477 INFO:tasks.workunit.client.0.smithi092.stdout:3/167: link d8/d9/c10 d8/db/c37 0 2024-05-06T12:14:28.477 INFO:tasks.workunit.client.0.smithi092.stdout:2/98: write df/f1f [1545524,127548] 0 2024-05-06T12:14:28.477 INFO:tasks.workunit.client.0.smithi092.stdout:7/305: creat d9/de/d1f/d37/d48/d65/f6b x:0 0 0 2024-05-06T12:14:28.482 INFO:tasks.workunit.client.0.smithi092.stdout:4/115: sync 2024-05-06T12:14:28.482 INFO:tasks.workunit.client.0.smithi092.stdout:7/306: symlink d9/de/d10/d17/l6c 0 2024-05-06T12:14:28.482 INFO:tasks.workunit.client.0.smithi092.stdout:0/209: sync 2024-05-06T12:14:28.482 INFO:tasks.workunit.client.0.smithi092.stdout:1/98: sync 2024-05-06T12:14:28.482 INFO:tasks.workunit.client.0.smithi092.stdout:6/232: sync 2024-05-06T12:14:28.482 INFO:tasks.workunit.client.0.smithi092.stdout:8/132: sync 2024-05-06T12:14:28.487 INFO:tasks.workunit.client.0.smithi092.stdout:1/99: mkdir d3/d10/d13/d19/d21 0 2024-05-06T12:14:28.487 INFO:tasks.workunit.client.0.smithi092.stdout:9/90: dread fb [0,4194304] 0 2024-05-06T12:14:28.487 INFO:tasks.workunit.client.0.smithi092.stdout:6/233: truncate d4/d24/f39 202698 0 2024-05-06T12:14:28.489 INFO:tasks.workunit.client.0.smithi092.stdout:3/168: dwrite d8/f12 [0,4194304] 0 2024-05-06T12:14:28.491 INFO:tasks.workunit.client.0.smithi092.stdout:0/210: read d7/d43/f1c [325172,98388] 0 2024-05-06T12:14:28.492 INFO:tasks.workunit.client.0.smithi092.stdout:7/307: creat d9/de/d10/d17/d6a/f6d x:0 0 0 2024-05-06T12:14:28.493 INFO:tasks.workunit.client.0.smithi092.stdout:3/169: write d8/d24/f32 [381698,30748] 0 2024-05-06T12:14:28.494 INFO:tasks.workunit.client.0.smithi092.stdout:4/116: mknod d9/d27/c28 0 2024-05-06T12:14:28.495 INFO:tasks.workunit.client.0.smithi092.stdout:7/308: fdatasync d9/de/f34 0 2024-05-06T12:14:28.497 INFO:tasks.workunit.client.0.smithi092.stdout:9/91: fsync d16/f17 0 2024-05-06T12:14:28.497 INFO:tasks.workunit.client.0.smithi092.stdout:1/100: creat d3/d10/d13/d19/d21/f22 x:0 0 0 2024-05-06T12:14:28.498 INFO:tasks.workunit.client.0.smithi092.stdout:2/99: rmdir df/d17/d1d 39 2024-05-06T12:14:28.499 INFO:tasks.workunit.client.0.smithi092.stdout:9/92: chown fb 10298861 1 2024-05-06T12:14:28.502 INFO:tasks.workunit.client.0.smithi092.stdout:9/93: chown d16/l18 41 1 2024-05-06T12:14:28.502 INFO:tasks.workunit.client.0.smithi092.stdout:1/101: truncate d3/d10/d13/d19/f1c 315204 0 2024-05-06T12:14:28.504 INFO:tasks.workunit.client.0.smithi092.stdout:9/94: fsync f9 0 2024-05-06T12:14:28.504 INFO:tasks.workunit.client.0.smithi092.stdout:6/234: dwrite d4/d20/d23/d30/d48/f4a [0,4194304] 0 2024-05-06T12:14:28.504 INFO:tasks.workunit.client.0.smithi092.stdout:9/95: fdatasync fc 0 2024-05-06T12:14:28.505 INFO:tasks.workunit.client.0.smithi092.stdout:5/165: truncate f0 727663 0 2024-05-06T12:14:28.506 INFO:tasks.workunit.client.0.smithi092.stdout:9/96: dread f9 [0,4194304] 0 2024-05-06T12:14:28.507 INFO:tasks.workunit.client.0.smithi092.stdout:2/100: chown df/d17/d1d 1 1 2024-05-06T12:14:28.508 INFO:tasks.workunit.client.0.smithi092.stdout:9/97: fsync f3 0 2024-05-06T12:14:28.509 INFO:tasks.workunit.client.0.smithi092.stdout:7/309: mknod d9/de/d10/d17/d6a/c6e 0 2024-05-06T12:14:28.509 INFO:tasks.workunit.client.0.smithi092.stdout:7/310: read - d9/f5a zero size 2024-05-06T12:14:28.510 INFO:tasks.workunit.client.0.smithi092.stdout:9/98: read f3 [4067054,127562] 0 2024-05-06T12:14:28.515 INFO:tasks.workunit.client.0.smithi092.stdout:1/102: symlink d3/d10/d13/d19/l23 0 2024-05-06T12:14:28.515 INFO:tasks.workunit.client.0.smithi092.stdout:2/101: dwrite df/d17/f19 [0,4194304] 0 2024-05-06T12:14:28.516 INFO:tasks.workunit.client.0.smithi092.stdout:1/103: stat d3/d10/d13/d19/l23 0 2024-05-06T12:14:28.516 INFO:tasks.workunit.client.0.smithi092.stdout:6/235: mkdir d4/d24/d4e 0 2024-05-06T12:14:28.524 INFO:tasks.workunit.client.0.smithi092.stdout:9/99: creat d16/f1a x:0 0 0 2024-05-06T12:14:28.524 INFO:tasks.workunit.client.0.smithi092.stdout:0/211: link d7/dc/cd d7/da/d1e/c48 0 2024-05-06T12:14:28.530 INFO:tasks.workunit.client.0.smithi092.stdout:3/170: dwrite d8/d24/d27/f2f [0,4194304] 0 2024-05-06T12:14:28.531 INFO:tasks.workunit.client.0.smithi092.stdout:5/166: link d4/d5/f1d d4/d5/d30/f40 0 2024-05-06T12:14:28.531 INFO:tasks.workunit.client.0.smithi092.stdout:1/104: rmdir d3/d10/d13/d19 39 2024-05-06T12:14:28.537 INFO:tasks.workunit.client.0.smithi092.stdout:0/212: fdatasync d7/da/f19 0 2024-05-06T12:14:28.539 INFO:tasks.workunit.client.0.smithi092.stdout:0/213: write d7/da/d1e/f3e [1366266,95092] 0 2024-05-06T12:14:28.540 INFO:tasks.workunit.client.0.smithi092.stdout:4/117: link d9/df/l1a d9/l29 0 2024-05-06T12:14:28.541 INFO:tasks.workunit.client.0.smithi092.stdout:3/171: creat d8/db/d14/f38 x:0 0 0 2024-05-06T12:14:28.543 INFO:tasks.workunit.client.0.smithi092.stdout:5/167: rmdir d4/d5/d10/d12 39 2024-05-06T12:14:28.548 INFO:tasks.workunit.client.0.smithi092.stdout:3/172: dwrite d8/d24/d27/f2f [0,4194304] 0 2024-05-06T12:14:28.550 INFO:tasks.workunit.client.0.smithi092.stdout:0/214: creat d7/d43/f49 x:0 0 0 2024-05-06T12:14:28.553 INFO:tasks.workunit.client.0.smithi092.stdout:4/118: fdatasync d9/dd/f17 0 2024-05-06T12:14:28.556 INFO:tasks.workunit.client.0.smithi092.stdout:6/236: truncate d4/f6 1427897 0 2024-05-06T12:14:28.557 INFO:tasks.workunit.client.0.smithi092.stdout:7/311: getdents d9/de/d10/d17/d21 0 2024-05-06T12:14:28.559 INFO:tasks.workunit.client.0.smithi092.stdout:3/173: mkdir d8/d24/d33/d39 0 2024-05-06T12:14:28.560 INFO:tasks.workunit.client.0.smithi092.stdout:0/215: rmdir d7/da/d1e/d26 39 2024-05-06T12:14:28.561 INFO:tasks.workunit.client.0.smithi092.stdout:9/100: dwrite d16/f17 [0,4194304] 0 2024-05-06T12:14:28.567 INFO:tasks.workunit.client.0.smithi092.stdout:1/105: dwrite d3/d10/d13/f14 [0,4194304] 0 2024-05-06T12:14:28.568 INFO:tasks.workunit.client.0.smithi092.stdout:4/119: rename d9/df/l20 to d9/df/l2a 0 2024-05-06T12:14:28.570 INFO:tasks.workunit.client.0.smithi092.stdout:7/312: stat d9/de/d10/l4a 0 2024-05-06T12:14:28.574 INFO:tasks.workunit.client.0.smithi092.stdout:3/174: chown d8/d9/c10 5408 1 2024-05-06T12:14:28.574 INFO:tasks.workunit.client.0.smithi092.stdout:9/101: creat d16/f1b x:0 0 0 2024-05-06T12:14:28.574 INFO:tasks.workunit.client.0.smithi092.stdout:7/313: write d9/de/d1f/d37/d48/d65/f66 [477536,33111] 0 2024-05-06T12:14:28.576 INFO:tasks.workunit.client.0.smithi092.stdout:5/168: link d4/d34/l14 d4/d34/d29/l41 0 2024-05-06T12:14:28.576 INFO:tasks.workunit.client.0.smithi092.stdout:1/106: dwrite d3/d10/f15 [0,4194304] 0 2024-05-06T12:14:28.578 INFO:tasks.workunit.client.0.smithi092.stdout:0/216: rename d7/da/f19 to d7/d43/d1f/f4a 0 2024-05-06T12:14:28.579 INFO:tasks.workunit.client.0.smithi092.stdout:6/237: creat d4/d4c/f4f x:0 0 0 2024-05-06T12:14:28.581 INFO:tasks.workunit.client.0.smithi092.stdout:9/102: chown cf 34 1 2024-05-06T12:14:28.581 INFO:tasks.workunit.client.0.smithi092.stdout:3/175: mkdir d8/d24/d27/d3a 0 2024-05-06T12:14:28.582 INFO:tasks.workunit.client.0.smithi092.stdout:5/169: fdatasync d4/f9 0 2024-05-06T12:14:28.585 INFO:tasks.workunit.client.0.smithi092.stdout:4/120: mkdir d9/d2b 0 2024-05-06T12:14:28.589 INFO:tasks.workunit.client.0.smithi092.stdout:6/238: write d4/d20/d23/f46 [841612,60277] 0 2024-05-06T12:14:28.590 INFO:tasks.workunit.client.0.smithi092.stdout:9/103: mkdir d16/d1c 0 2024-05-06T12:14:28.591 INFO:tasks.workunit.client.0.smithi092.stdout:6/239: readlink d4/lf 0 2024-05-06T12:14:28.592 INFO:tasks.workunit.client.0.smithi092.stdout:3/176: dwrite d8/db/d14/f18 [0,4194304] 0 2024-05-06T12:14:28.596 INFO:tasks.workunit.client.0.smithi092.stdout:6/240: chown d4/d20/d23/f46 14300 1 2024-05-06T12:14:28.596 INFO:tasks.workunit.client.0.smithi092.stdout:1/107: dwrite d3/d10/d13/d19/f1c [0,4194304] 0 2024-05-06T12:14:28.598 INFO:tasks.workunit.client.0.smithi092.stdout:1/108: chown d3/d10/d13/f14 12107463 1 2024-05-06T12:14:28.598 INFO:tasks.workunit.client.0.smithi092.stdout:7/314: truncate d9/dd/d15/f42 365505 0 2024-05-06T12:14:28.599 INFO:tasks.workunit.client.0.smithi092.stdout:7/315: fsync d9/de/d1f/d37/d39/f59 0 2024-05-06T12:14:28.600 INFO:tasks.workunit.client.0.smithi092.stdout:1/109: write d3/d10/d13/f14 [2808070,91937] 0 2024-05-06T12:14:28.602 INFO:tasks.workunit.client.0.smithi092.stdout:1/110: write d3/d10/f15 [2100482,27789] 0 2024-05-06T12:14:28.603 INFO:tasks.workunit.client.0.smithi092.stdout:1/111: dread - d3/d10/d13/d19/f1e zero size 2024-05-06T12:14:28.604 INFO:tasks.workunit.client.0.smithi092.stdout:1/112: stat d3 0 2024-05-06T12:14:28.606 INFO:tasks.workunit.client.0.smithi092.stdout:5/170: fsync f0 0 2024-05-06T12:14:28.608 INFO:tasks.workunit.client.0.smithi092.stdout:1/113: dread d3/f12 [4194304,4194304] 0 2024-05-06T12:14:28.608 INFO:tasks.workunit.client.0.smithi092.stdout:3/177: symlink d8/d9/l3b 0 2024-05-06T12:14:28.609 INFO:tasks.workunit.client.0.smithi092.stdout:1/114: chown d3/f17 1651648812 1 2024-05-06T12:14:28.610 INFO:tasks.workunit.client.0.smithi092.stdout:3/178: write d8/db/d14/f38 [62477,125382] 0 2024-05-06T12:14:28.610 INFO:tasks.workunit.client.0.smithi092.stdout:1/115: truncate d3/f5 1126283 0 2024-05-06T12:14:28.611 INFO:tasks.workunit.client.0.smithi092.stdout:6/241: symlink d4/d20/d23/d30/l50 0 2024-05-06T12:14:28.612 INFO:tasks.workunit.client.0.smithi092.stdout:1/116: fsync d3/d10/d13/d19/d21/f22 0 2024-05-06T12:14:28.614 INFO:tasks.workunit.client.0.smithi092.stdout:7/316: creat d9/f6f x:0 0 0 2024-05-06T12:14:28.618 INFO:tasks.workunit.client.0.smithi092.stdout:4/121: getdents d9/d2b 0 2024-05-06T12:14:28.618 INFO:tasks.workunit.client.0.smithi092.stdout:9/104: getdents d16/d1c 0 2024-05-06T12:14:28.619 INFO:tasks.workunit.client.0.smithi092.stdout:4/122: write d9/dd/f19 [1126516,6282] 0 2024-05-06T12:14:28.620 INFO:tasks.workunit.client.0.smithi092.stdout:9/105: write d16/f17 [2936081,15971] 0 2024-05-06T12:14:28.624 INFO:tasks.workunit.client.0.smithi092.stdout:1/117: creat d3/d10/d13/d19/d21/f24 x:0 0 0 2024-05-06T12:14:28.625 INFO:tasks.workunit.client.0.smithi092.stdout:7/317: mknod d9/c70 0 2024-05-06T12:14:28.626 INFO:tasks.workunit.client.0.smithi092.stdout:0/217: link d7/da/l44 d7/da/d1e/d26/l4b 0 2024-05-06T12:14:28.627 INFO:tasks.workunit.client.0.smithi092.stdout:1/118: read d3/f17 [197826,101919] 0 2024-05-06T12:14:28.631 INFO:tasks.workunit.client.0.smithi092.stdout:7/318: mknod d9/dd/d15/c71 0 2024-05-06T12:14:28.631 INFO:tasks.workunit.client.0.smithi092.stdout:9/106: write f7 [2438506,101989] 0 2024-05-06T12:14:28.632 INFO:tasks.workunit.client.0.smithi092.stdout:6/242: dwrite d4/d20/f1b [4194304,4194304] 0 2024-05-06T12:14:28.632 INFO:tasks.workunit.client.0.smithi092.stdout:1/119: dread d3/f17 [0,4194304] 0 2024-05-06T12:14:28.634 INFO:tasks.workunit.client.0.smithi092.stdout:5/171: rename d4/d34/c1a to d4/d5/d10/c42 0 2024-05-06T12:14:28.634 INFO:tasks.workunit.client.0.smithi092.stdout:7/319: chown d9/de/d1f/d37/d48/d65/f66 5736 1 2024-05-06T12:14:28.637 INFO:tasks.workunit.client.0.smithi092.stdout:5/172: dread - d4/d5/d30/f40 zero size 2024-05-06T12:14:28.638 INFO:tasks.workunit.client.0.smithi092.stdout:0/218: symlink d7/da/d1e/d46/l4c 0 2024-05-06T12:14:28.639 INFO:tasks.workunit.client.0.smithi092.stdout:3/179: dread d8/db/f11 [0,4194304] 0 2024-05-06T12:14:28.641 INFO:tasks.workunit.client.0.smithi092.stdout:1/120: mknod d3/d10/d13/d19/c25 0 2024-05-06T12:14:28.642 INFO:tasks.workunit.client.0.smithi092.stdout:3/180: chown d8/db/d14/f18 149 1 2024-05-06T12:14:28.645 INFO:tasks.workunit.client.0.smithi092.stdout:7/320: symlink d9/de/d1f/d37/d39/l72 0 2024-05-06T12:14:28.648 INFO:tasks.workunit.client.0.smithi092.stdout:9/107: symlink d16/d1c/l1d 0 2024-05-06T12:14:28.649 INFO:tasks.workunit.client.0.smithi092.stdout:4/123: link d9/df/l1a d9/df/l2c 0 2024-05-06T12:14:28.650 INFO:tasks.workunit.client.0.smithi092.stdout:6/243: dwrite d4/f17 [4194304,4194304] 0 2024-05-06T12:14:28.651 INFO:tasks.workunit.client.0.smithi092.stdout:0/219: dwrite d7/da/d1e/d26/f2b [0,4194304] 0 2024-05-06T12:14:28.652 INFO:tasks.workunit.client.0.smithi092.stdout:5/173: rename d4/d5/d10/c3f to d4/d5/d10/d12/c43 0 2024-05-06T12:14:28.659 INFO:tasks.workunit.client.0.smithi092.stdout:9/108: mkdir d16/d1e 0 2024-05-06T12:14:28.659 INFO:tasks.workunit.client.0.smithi092.stdout:9/109: write f6 [138687,57185] 0 2024-05-06T12:14:28.660 INFO:tasks.workunit.client.0.smithi092.stdout:4/124: mknod d9/df/c2d 0 2024-05-06T12:14:28.662 INFO:tasks.workunit.client.0.smithi092.stdout:6/244: mknod d4/d20/d23/d30/c51 0 2024-05-06T12:14:28.663 INFO:tasks.workunit.client.0.smithi092.stdout:7/321: rename d9/de/d1f/d37/d48/f4b to d9/de/d1f/d37/d48/d65/f73 0 2024-05-06T12:14:28.665 INFO:tasks.workunit.client.0.smithi092.stdout:0/220: creat d7/dc/d28/f4d x:0 0 0 2024-05-06T12:14:28.665 INFO:tasks.workunit.client.0.smithi092.stdout:3/181: link d8/db/c23 d8/d1b/c3c 0 2024-05-06T12:14:28.668 INFO:tasks.workunit.client.0.smithi092.stdout:9/110: rename fb to d16/f1f 0 2024-05-06T12:14:28.669 INFO:tasks.workunit.client.0.smithi092.stdout:7/322: creat d9/de/d1f/d37/f74 x:0 0 0 2024-05-06T12:14:28.670 INFO:tasks.workunit.client.0.smithi092.stdout:5/174: creat d4/d5/d10/d12/d37/f44 x:0 0 0 2024-05-06T12:14:28.670 INFO:tasks.workunit.client.0.smithi092.stdout:3/182: dread d8/d24/d27/f2f [0,4194304] 0 2024-05-06T12:14:28.671 INFO:tasks.workunit.client.0.smithi092.stdout:0/221: chown d7/da/l44 29 1 2024-05-06T12:14:28.671 INFO:tasks.workunit.client.0.smithi092.stdout:3/183: fdatasync f3 0 2024-05-06T12:14:28.671 INFO:tasks.workunit.client.0.smithi092.stdout:4/125: mknod d9/c2e 0 2024-05-06T12:14:28.672 INFO:tasks.workunit.client.0.smithi092.stdout:5/175: chown d4/d5/d10/d12/d1f/d20/d23 12542 1 2024-05-06T12:14:28.672 INFO:tasks.workunit.client.0.smithi092.stdout:0/222: fdatasync d7/da/f3b 0 2024-05-06T12:14:28.674 INFO:tasks.workunit.client.0.smithi092.stdout:6/245: rename d4/d20/d23/d30/d48 to d4/d4c/d52 0 2024-05-06T12:14:28.674 INFO:tasks.workunit.client.0.smithi092.stdout:9/111: rename d16/d1c to d16/d1c/d20 22 2024-05-06T12:14:28.675 INFO:tasks.workunit.client.0.smithi092.stdout:7/323: write d9/de/d10/f27 [1968700,125382] 0 2024-05-06T12:14:28.677 INFO:tasks.workunit.client.0.smithi092.stdout:5/176: symlink d4/d34/d28/d2b/l45 0 2024-05-06T12:14:28.678 INFO:tasks.workunit.client.0.smithi092.stdout:0/223: mkdir d7/da/d1e/d26/d4e 0 2024-05-06T12:14:28.679 INFO:tasks.workunit.client.0.smithi092.stdout:6/246: mknod d4/d20/d23/d30/c53 0 2024-05-06T12:14:28.682 INFO:tasks.workunit.client.0.smithi092.stdout:0/224: truncate d7/da/d1e/d26/f2b 4321951 0 2024-05-06T12:14:28.682 INFO:tasks.workunit.client.0.smithi092.stdout:9/112: rename d16/f1a to d16/f21 0 2024-05-06T12:14:28.683 INFO:tasks.workunit.client.0.smithi092.stdout:3/184: mkdir d8/d24/d27/d3a/d3d 0 2024-05-06T12:14:28.685 INFO:tasks.workunit.client.0.smithi092.stdout:9/113: truncate d16/f1b 335453 0 2024-05-06T12:14:28.685 INFO:tasks.workunit.client.0.smithi092.stdout:6/247: mkdir d4/d24/d54 0 2024-05-06T12:14:28.685 INFO:tasks.workunit.client.0.smithi092.stdout:0/225: fsync d7/dc/f29 0 2024-05-06T12:14:28.686 INFO:tasks.workunit.client.0.smithi092.stdout:6/248: read d4/fb [2203952,10288] 0 2024-05-06T12:14:28.688 INFO:tasks.workunit.client.0.smithi092.stdout:4/126: dwrite d9/fc [0,4194304] 0 2024-05-06T12:14:28.688 INFO:tasks.workunit.client.0.smithi092.stdout:7/324: mkdir d9/de/d75 0 2024-05-06T12:14:28.692 INFO:tasks.workunit.client.0.smithi092.stdout:3/185: creat d8/d1b/f3e x:0 0 0 2024-05-06T12:14:28.693 INFO:tasks.workunit.client.0.smithi092.stdout:3/186: write d8/d1b/f1f [2162812,113492] 0 2024-05-06T12:14:28.698 INFO:tasks.workunit.client.0.smithi092.stdout:3/187: dread d8/d1b/f1f [0,4194304] 0 2024-05-06T12:14:28.700 INFO:tasks.workunit.client.0.smithi092.stdout:3/188: read d8/d9/f17 [3390,88542] 0 2024-05-06T12:14:28.704 INFO:tasks.workunit.client.0.smithi092.stdout:6/249: dwrite d4/d20/f1c [0,4194304] 0 2024-05-06T12:14:28.713 INFO:tasks.workunit.client.0.smithi092.stdout:4/127: mknod d9/d2b/c2f 0 2024-05-06T12:14:28.713 INFO:tasks.workunit.client.0.smithi092.stdout:7/325: dread d9/dd/d15/f14 [0,4194304] 0 2024-05-06T12:14:28.714 INFO:tasks.workunit.client.0.smithi092.stdout:7/326: stat d9/de/d10/l45 0 2024-05-06T12:14:28.715 INFO:tasks.workunit.client.0.smithi092.stdout:7/327: chown d9/dd/d15/f30 2542 1 2024-05-06T12:14:28.716 INFO:tasks.workunit.client.0.smithi092.stdout:0/226: truncate d7/d43/d1f/f4a 2528042 0 2024-05-06T12:14:28.718 INFO:tasks.workunit.client.0.smithi092.stdout:0/227: write d7/dc/f30 [1535973,102101] 0 2024-05-06T12:14:28.718 INFO:tasks.workunit.client.0.smithi092.stdout:5/177: getdents d4/d5/d10 0 2024-05-06T12:14:28.723 INFO:tasks.workunit.client.0.smithi092.stdout:4/128: mknod d9/d2b/c30 0 2024-05-06T12:14:28.724 INFO:tasks.workunit.client.0.smithi092.stdout:7/328: dwrite d9/de/d1f/f4c [0,4194304] 0 2024-05-06T12:14:28.724 INFO:tasks.workunit.client.0.smithi092.stdout:3/189: mkdir d8/db/d3f 0 2024-05-06T12:14:28.728 INFO:tasks.workunit.client.0.smithi092.stdout:5/178: creat d4/d5/d30/f46 x:0 0 0 2024-05-06T12:14:28.732 INFO:tasks.workunit.client.0.smithi092.stdout:3/190: rmdir d8/db 39 2024-05-06T12:14:28.734 INFO:tasks.workunit.client.0.smithi092.stdout:5/179: symlink d4/d5/d10/d12/d1f/d20/d23/l47 0 2024-05-06T12:14:28.737 INFO:tasks.workunit.client.0.smithi092.stdout:0/228: creat d7/f4f x:0 0 0 2024-05-06T12:14:28.739 INFO:tasks.workunit.client.0.smithi092.stdout:9/114: truncate f6 1700016 0 2024-05-06T12:14:28.739 INFO:tasks.workunit.client.0.smithi092.stdout:9/115: readlink l15 0 2024-05-06T12:14:28.740 INFO:tasks.workunit.client.0.smithi092.stdout:9/116: chown d16/f17 5973725 1 2024-05-06T12:14:28.740 INFO:tasks.workunit.client.0.smithi092.stdout:9/117: chown c4 48 1 2024-05-06T12:14:28.741 INFO:tasks.workunit.client.0.smithi092.stdout:7/329: getdents d9/de/d1f/d37/d39 0 2024-05-06T12:14:28.743 INFO:tasks.workunit.client.0.smithi092.stdout:7/330: dread - d9/de/d10/d17/d6a/f6d zero size 2024-05-06T12:14:28.743 INFO:tasks.workunit.client.0.smithi092.stdout:0/229: mknod d7/da/d1e/c50 0 2024-05-06T12:14:28.745 INFO:tasks.workunit.client.0.smithi092.stdout:3/191: getdents d8/d24/d27/d3a 0 2024-05-06T12:14:28.746 INFO:tasks.workunit.client.0.smithi092.stdout:7/331: write d9/de/d10/f25 [2499180,106862] 0 2024-05-06T12:14:28.746 INFO:tasks.workunit.client.0.smithi092.stdout:4/129: dread d9/dd/f19 [0,4194304] 0 2024-05-06T12:14:28.747 INFO:tasks.workunit.client.0.smithi092.stdout:7/332: readlink d9/de/l1c 0 2024-05-06T12:14:28.748 INFO:tasks.workunit.client.0.smithi092.stdout:7/333: truncate d9/de/d1f/f31 800571 0 2024-05-06T12:14:28.751 INFO:tasks.workunit.client.0.smithi092.stdout:3/192: unlink f5 0 2024-05-06T12:14:28.752 INFO:tasks.workunit.client.0.smithi092.stdout:9/118: link d16/d1c/l1d d16/d1e/l22 0 2024-05-06T12:14:28.752 INFO:tasks.workunit.client.0.smithi092.stdout:7/334: dread f3 [0,4194304] 0 2024-05-06T12:14:28.753 INFO:tasks.workunit.client.0.smithi092.stdout:7/335: write f1 [6802110,56881] 0 2024-05-06T12:14:28.753 INFO:tasks.workunit.client.0.smithi092.stdout:0/230: symlink d7/da/l51 0 2024-05-06T12:14:28.754 INFO:tasks.workunit.client.0.smithi092.stdout:4/130: fdatasync d9/dd/f19 0 2024-05-06T12:14:28.755 INFO:tasks.workunit.client.0.smithi092.stdout:3/193: rmdir d8/d24/d27/d3a 39 2024-05-06T12:14:28.760 INFO:tasks.workunit.client.0.smithi092.stdout:0/231: rename d7/d43/f3f to d7/da/d1e/d26/d3a/f52 0 2024-05-06T12:14:28.762 INFO:tasks.workunit.client.0.smithi092.stdout:5/180: write d4/d5/f1d [925612,56595] 0 2024-05-06T12:14:28.765 INFO:tasks.workunit.client.0.smithi092.stdout:7/336: mkdir d9/de/d75/d76 0 2024-05-06T12:14:28.768 INFO:tasks.workunit.client.0.smithi092.stdout:0/232: dwrite d7/da/d1e/d39/f47 [0,4194304] 0 2024-05-06T12:14:28.769 INFO:tasks.workunit.client.0.smithi092.stdout:0/233: fsync d7/d43/f49 0 2024-05-06T12:14:28.770 INFO:tasks.workunit.client.0.smithi092.stdout:9/119: link cf d16/d1c/c23 0 2024-05-06T12:14:28.772 INFO:tasks.workunit.client.0.smithi092.stdout:3/194: dread f2 [0,4194304] 0 2024-05-06T12:14:28.773 INFO:tasks.workunit.client.0.smithi092.stdout:5/181: creat d4/d5/d10/d12/d1f/d20/d23/f48 x:0 0 0 2024-05-06T12:14:28.774 INFO:tasks.workunit.client.0.smithi092.stdout:7/337: mkdir d9/de/d10/d17/d21/d38/d77 0 2024-05-06T12:14:28.779 INFO:tasks.workunit.client.0.smithi092.stdout:4/131: getdents d9/d27 0 2024-05-06T12:14:28.781 INFO:tasks.workunit.client.0.smithi092.stdout:7/338: unlink d9/de/c16 0 2024-05-06T12:14:28.782 INFO:tasks.workunit.client.0.smithi092.stdout:7/339: fdatasync d9/de/d1f/d37/d39/f59 0 2024-05-06T12:14:28.783 INFO:tasks.workunit.client.0.smithi092.stdout:9/120: unlink d16/d1e/l22 0 2024-05-06T12:14:28.783 INFO:tasks.workunit.client.0.smithi092.stdout:9/121: chown le 212848373 1 2024-05-06T12:14:28.785 INFO:tasks.workunit.client.0.smithi092.stdout:4/132: chown d9/f21 19 1 2024-05-06T12:14:28.785 INFO:tasks.workunit.client.0.smithi092.stdout:5/182: mkdir d4/d34/d29/d49 0 2024-05-06T12:14:28.787 INFO:tasks.workunit.client.0.smithi092.stdout:5/183: chown d4/d5/d10/d12/d37/f44 387 1 2024-05-06T12:14:28.787 INFO:tasks.workunit.client.0.smithi092.stdout:9/122: mknod d16/d1c/c24 0 2024-05-06T12:14:28.788 INFO:tasks.workunit.client.0.smithi092.stdout:4/133: creat d9/d27/f31 x:0 0 0 2024-05-06T12:14:28.790 INFO:tasks.workunit.client.0.smithi092.stdout:4/134: write d9/df/f15 [358311,69840] 0 2024-05-06T12:14:28.791 INFO:tasks.workunit.client.0.smithi092.stdout:7/340: link d9/de/d10/d17/d21/d2e/f4d d9/de/d10/d17/d6a/f78 0 2024-05-06T12:14:28.792 INFO:tasks.workunit.client.0.smithi092.stdout:5/184: creat d4/d5/d10/d31/f4a x:0 0 0 2024-05-06T12:14:28.793 INFO:tasks.workunit.client.0.smithi092.stdout:7/341: readlink d9/de/d10/d17/l6c 0 2024-05-06T12:14:28.794 INFO:tasks.workunit.client.0.smithi092.stdout:4/135: creat d9/d2b/f32 x:0 0 0 2024-05-06T12:14:28.795 INFO:tasks.workunit.client.0.smithi092.stdout:3/195: write d8/d1b/f1f [1873271,49794] 0 2024-05-06T12:14:28.795 INFO:tasks.workunit.client.0.smithi092.stdout:9/123: dwrite f9 [0,4194304] 0 2024-05-06T12:14:28.797 INFO:tasks.workunit.client.0.smithi092.stdout:4/136: chown d9/l1e 0 1 2024-05-06T12:14:28.797 INFO:tasks.workunit.client.0.smithi092.stdout:7/342: mknod d9/de/d1f/c79 0 2024-05-06T12:14:28.799 INFO:tasks.workunit.client.0.smithi092.stdout:5/185: mkdir d4/d34/d29/d49/d4b 0 2024-05-06T12:14:28.801 INFO:tasks.workunit.client.0.smithi092.stdout:9/124: fdatasync d16/f21 0 2024-05-06T12:14:28.803 INFO:tasks.workunit.client.0.smithi092.stdout:7/343: creat d9/de/d10/d17/d6a/f7a x:0 0 0 2024-05-06T12:14:28.804 INFO:tasks.workunit.client.0.smithi092.stdout:5/186: truncate f0 1294818 0 2024-05-06T12:14:28.805 INFO:tasks.workunit.client.0.smithi092.stdout:4/137: rmdir d9/d27 39 2024-05-06T12:14:28.806 INFO:tasks.workunit.client.0.smithi092.stdout:5/187: chown d4/d5/d10/f33 113936837 1 2024-05-06T12:14:28.807 INFO:tasks.workunit.client.0.smithi092.stdout:3/196: mkdir d8/d24/d27/d3a/d40 0 2024-05-06T12:14:28.807 INFO:tasks.workunit.client.0.smithi092.stdout:5/188: fdatasync d4/d5/d10/d12/d1f/d20/d23/f25 0 2024-05-06T12:14:28.810 INFO:tasks.workunit.client.0.smithi092.stdout:4/138: dread d9/df/f14 [0,4194304] 0 2024-05-06T12:14:28.810 INFO:tasks.workunit.client.0.smithi092.stdout:5/189: dread - d4/d5/d10/d12/d1f/d20/d23/f25 zero size 2024-05-06T12:14:28.811 INFO:tasks.workunit.client.0.smithi092.stdout:9/125: truncate f3 1387895 0 2024-05-06T12:14:28.812 INFO:tasks.workunit.client.0.smithi092.stdout:9/126: dread fc [0,4194304] 0 2024-05-06T12:14:28.812 INFO:tasks.workunit.client.0.smithi092.stdout:3/197: rename f2 to d8/d9/f41 0 2024-05-06T12:14:28.817 INFO:tasks.workunit.client.0.smithi092.stdout:9/127: creat d16/d1e/f25 x:0 0 0 2024-05-06T12:14:28.818 INFO:tasks.workunit.client.0.smithi092.stdout:4/139: rename d9/df/c25 to d9/c33 0 2024-05-06T12:14:28.822 INFO:tasks.workunit.client.0.smithi092.stdout:4/140: chown d9/lb 1428609 1 2024-05-06T12:14:28.822 INFO:tasks.workunit.client.0.smithi092.stdout:3/198: dwrite d8/db/d14/f38 [0,4194304] 0 2024-05-06T12:14:28.822 INFO:tasks.workunit.client.0.smithi092.stdout:4/141: write d9/dd/f19 [510054,31724] 0 2024-05-06T12:14:28.824 INFO:tasks.workunit.client.0.smithi092.stdout:4/142: creat d9/f34 x:0 0 0 2024-05-06T12:14:28.830 INFO:tasks.workunit.client.0.smithi092.stdout:4/143: dwrite d9/f24 [0,4194304] 0 2024-05-06T12:14:28.833 INFO:tasks.workunit.client.0.smithi092.stdout:4/144: symlink d9/d27/l35 0 2024-05-06T12:14:28.833 INFO:tasks.workunit.client.0.smithi092.stdout:8/133: sync 2024-05-06T12:14:28.836 INFO:tasks.workunit.client.0.smithi092.stdout:4/145: symlink d9/df/l36 0 2024-05-06T12:14:28.841 INFO:tasks.workunit.client.0.smithi092.stdout:8/134: rename d9/dc/f27 to d9/f33 0 2024-05-06T12:14:28.843 INFO:tasks.workunit.client.0.smithi092.stdout:4/146: dwrite d9/fc [0,4194304] 0 2024-05-06T12:14:28.844 INFO:tasks.workunit.client.0.smithi092.stdout:8/135: creat d9/f34 x:0 0 0 2024-05-06T12:14:28.844 INFO:tasks.workunit.client.0.smithi092.stdout:8/136: fsync d9/f2a 0 2024-05-06T12:14:28.846 INFO:tasks.workunit.client.0.smithi092.stdout:8/137: creat d9/f35 x:0 0 0 2024-05-06T12:14:28.849 INFO:tasks.workunit.client.0.smithi092.stdout:8/138: creat d9/f36 x:0 0 0 2024-05-06T12:14:28.851 INFO:tasks.workunit.client.0.smithi092.stdout:8/139: mkdir d9/d21/d37 0 2024-05-06T12:14:28.853 INFO:tasks.workunit.client.0.smithi092.stdout:8/140: creat d9/d21/f38 x:0 0 0 2024-05-06T12:14:28.855 INFO:tasks.workunit.client.0.smithi092.stdout:8/141: unlink d9/dc/l2c 0 2024-05-06T12:14:28.857 INFO:tasks.workunit.client.0.smithi092.stdout:8/142: symlink d9/d21/d37/l39 0 2024-05-06T12:14:28.873 INFO:tasks.workunit.client.0.smithi092.stdout:4/147: fsync d9/dd/f19 0 2024-05-06T12:14:28.874 INFO:tasks.workunit.client.0.smithi092.stdout:8/143: read f6 [1209006,52459] 0 2024-05-06T12:14:28.877 INFO:tasks.workunit.client.0.smithi092.stdout:8/144: write d9/fe [1044720,39621] 0 2024-05-06T12:14:28.878 INFO:tasks.workunit.client.0.smithi092.stdout:4/148: mkdir d9/d2b/d37 0 2024-05-06T12:14:28.878 INFO:tasks.workunit.client.0.smithi092.stdout:7/344: dread d9/de/d10/f12 [0,4194304] 0 2024-05-06T12:14:28.881 INFO:tasks.workunit.client.0.smithi092.stdout:4/149: write d9/df/f14 [1919230,115227] 0 2024-05-06T12:14:28.881 INFO:tasks.workunit.client.0.smithi092.stdout:8/145: read d9/f25 [3256857,7139] 0 2024-05-06T12:14:28.883 INFO:tasks.workunit.client.0.smithi092.stdout:5/190: dwrite d4/d34/d28/f39 [0,4194304] 0 2024-05-06T12:14:28.887 INFO:tasks.workunit.client.0.smithi092.stdout:4/150: dread d9/fc [0,4194304] 0 2024-05-06T12:14:28.888 INFO:tasks.workunit.client.0.smithi092.stdout:7/345: creat d9/de/d10/d17/d6a/f7b x:0 0 0 2024-05-06T12:14:28.888 INFO:tasks.workunit.client.0.smithi092.stdout:8/146: rmdir d9/d21/d37 39 2024-05-06T12:14:28.889 INFO:tasks.workunit.client.0.smithi092.stdout:4/151: write d9/f21 [998253,123786] 0 2024-05-06T12:14:28.890 INFO:tasks.workunit.client.0.smithi092.stdout:7/346: chown d9/dd/d15/c20 20 1 2024-05-06T12:14:28.890 INFO:tasks.workunit.client.0.smithi092.stdout:4/152: fdatasync d9/dd/f23 0 2024-05-06T12:14:28.891 INFO:tasks.workunit.client.0.smithi092.stdout:5/191: creat d4/f4c x:0 0 0 2024-05-06T12:14:28.891 INFO:tasks.workunit.client.0.smithi092.stdout:7/347: dread - d9/f40 zero size 2024-05-06T12:14:28.896 INFO:tasks.workunit.client.0.smithi092.stdout:4/153: fdatasync d9/f24 0 2024-05-06T12:14:28.898 INFO:tasks.workunit.client.0.smithi092.stdout:7/348: unlink d9/de/d1f/d37/d39/l72 0 2024-05-06T12:14:28.899 INFO:tasks.workunit.client.0.smithi092.stdout:7/349: chown d9/de/d10/d17/d6a/f7a 10 1 2024-05-06T12:14:28.899 INFO:tasks.workunit.client.0.smithi092.stdout:7/350: readlink d9/de/d10/l45 0 2024-05-06T12:14:28.901 INFO:tasks.workunit.client.0.smithi092.stdout:7/351: dread - d9/de/d1f/d37/d48/d65/f6b zero size 2024-05-06T12:14:28.902 INFO:tasks.workunit.client.0.smithi092.stdout:4/154: rmdir d9 39 2024-05-06T12:14:28.903 INFO:tasks.workunit.client.0.smithi092.stdout:7/352: write d9/de/d1f/f52 [219653,23680] 0 2024-05-06T12:14:28.907 INFO:tasks.workunit.client.0.smithi092.stdout:5/192: write d4/d5/d10/f27 [3705852,46844] 0 2024-05-06T12:14:28.909 INFO:tasks.workunit.client.0.smithi092.stdout:9/128: dwrite f6 [0,4194304] 0 2024-05-06T12:14:28.911 INFO:tasks.workunit.client.0.smithi092.stdout:8/147: dread d9/dc/f16 [0,4194304] 0 2024-05-06T12:14:28.911 INFO:tasks.workunit.client.0.smithi092.stdout:7/353: creat d9/de/d1f/d37/d48/d65/f7c x:0 0 0 2024-05-06T12:14:28.913 INFO:tasks.workunit.client.0.smithi092.stdout:4/155: link d9/fc d9/f38 0 2024-05-06T12:14:28.914 INFO:tasks.workunit.client.0.smithi092.stdout:9/129: creat d16/d1e/f26 x:0 0 0 2024-05-06T12:14:28.915 INFO:tasks.workunit.client.0.smithi092.stdout:7/354: creat d9/dd/d23/f7d x:0 0 0 2024-05-06T12:14:28.916 INFO:tasks.workunit.client.0.smithi092.stdout:5/193: creat d4/d34/d29/d49/d4b/f4d x:0 0 0 2024-05-06T12:14:28.918 INFO:tasks.workunit.client.0.smithi092.stdout:8/148: symlink d9/d21/l3a 0 2024-05-06T12:14:28.920 INFO:tasks.workunit.client.0.smithi092.stdout:4/156: write d9/f1c [3216687,68404] 0 2024-05-06T12:14:28.922 INFO:tasks.workunit.client.0.smithi092.stdout:7/355: unlink d9/de/d10/d17/d21/d2e/f4d 0 2024-05-06T12:14:28.924 INFO:tasks.workunit.client.0.smithi092.stdout:4/157: creat d9/d2b/f39 x:0 0 0 2024-05-06T12:14:28.925 INFO:tasks.workunit.client.0.smithi092.stdout:5/194: dwrite d4/d5/f3b [0,4194304] 0 2024-05-06T12:14:28.926 INFO:tasks.workunit.client.0.smithi092.stdout:8/149: creat d9/d21/d37/f3b x:0 0 0 2024-05-06T12:14:28.933 INFO:tasks.workunit.client.0.smithi092.stdout:8/150: unlink d9/dc/f28 0 2024-05-06T12:14:28.934 INFO:tasks.workunit.client.0.smithi092.stdout:8/151: write d9/f36 [43895,96240] 0 2024-05-06T12:14:28.935 INFO:tasks.workunit.client.0.smithi092.stdout:7/356: rmdir d9/de/d75/d76 0 2024-05-06T12:14:28.937 INFO:tasks.workunit.client.0.smithi092.stdout:9/130: truncate f9 4094793 0 2024-05-06T12:14:28.938 INFO:tasks.workunit.client.0.smithi092.stdout:9/131: truncate d16/f1b 1037737 0 2024-05-06T12:14:28.939 INFO:tasks.workunit.client.0.smithi092.stdout:8/152: symlink d9/dc/l3c 0 2024-05-06T12:14:28.939 INFO:tasks.workunit.client.0.smithi092.stdout:8/153: write f7 [885237,19721] 0 2024-05-06T12:14:28.941 INFO:tasks.workunit.client.0.smithi092.stdout:4/158: rmdir d9/d2b 39 2024-05-06T12:14:28.945 INFO:tasks.workunit.client.0.smithi092.stdout:8/154: symlink d9/d21/d37/l3d 0 2024-05-06T12:14:28.946 INFO:tasks.workunit.client.0.smithi092.stdout:8/155: stat d9/c31 0 2024-05-06T12:14:28.947 INFO:tasks.workunit.client.0.smithi092.stdout:9/132: dwrite f3 [0,4194304] 0 2024-05-06T12:14:28.951 INFO:tasks.workunit.client.0.smithi092.stdout:5/195: dwrite d4/d5/f3b [4194304,4194304] 0 2024-05-06T12:14:28.953 INFO:tasks.workunit.client.0.smithi092.stdout:4/159: unlink d9/d27/c28 0 2024-05-06T12:14:28.959 INFO:tasks.workunit.client.0.smithi092.stdout:7/357: getdents d9/de/d10 0 2024-05-06T12:14:28.960 INFO:tasks.workunit.client.0.smithi092.stdout:7/358: chown d9/de/d1f/d37/f74 309948591 1 2024-05-06T12:14:28.961 INFO:tasks.workunit.client.0.smithi092.stdout:7/359: readlink d9/de/d10/l4a 0 2024-05-06T12:14:28.962 INFO:tasks.workunit.client.0.smithi092.stdout:7/360: chown d9/dd/d23/l3b 61131544 1 2024-05-06T12:14:28.962 INFO:tasks.workunit.client.0.smithi092.stdout:5/196: rename d4/d34/d29/d49 to d4/d34/d28/d4e 0 2024-05-06T12:14:28.968 INFO:tasks.workunit.client.0.smithi092.stdout:5/197: chown d4/d34/d29/l41 687 1 2024-05-06T12:14:28.971 INFO:tasks.workunit.client.0.smithi092.stdout:7/361: creat d9/de/d1f/d37/d48/f7e x:0 0 0 2024-05-06T12:14:28.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:28 smithi092 ceph-mon[26954]: pgmap v16: 65 pgs: 65 active+clean; 333 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 3.2 MiB/s rd, 25 MiB/s wr, 401 op/s 2024-05-06T12:14:28.972 INFO:tasks.workunit.client.0.smithi092.stdout:7/362: chown d9/de/c67 35743675 1 2024-05-06T12:14:28.975 INFO:tasks.workunit.client.0.smithi092.stdout:5/198: dwrite d4/d5/d10/f27 [0,4194304] 0 2024-05-06T12:14:28.976 INFO:tasks.workunit.client.0.smithi092.stdout:8/156: getdents d9/d21/d37 0 2024-05-06T12:14:28.978 INFO:tasks.workunit.client.0.smithi092.stdout:9/133: unlink f9 0 2024-05-06T12:14:28.979 INFO:tasks.workunit.client.0.smithi092.stdout:8/157: write d9/dc/f2d [1243830,66219] 0 2024-05-06T12:14:28.979 INFO:tasks.workunit.client.0.smithi092.stdout:9/134: chown c13 6375305 1 2024-05-06T12:14:28.981 INFO:tasks.workunit.client.0.smithi092.stdout:5/199: creat d4/d34/d28/d4e/f4f x:0 0 0 2024-05-06T12:14:28.981 INFO:tasks.workunit.client.0.smithi092.stdout:9/135: dread - d16/f21 zero size 2024-05-06T12:14:28.981 INFO:tasks.workunit.client.0.smithi092.stdout:8/158: write f7 [2907854,112704] 0 2024-05-06T12:14:28.983 INFO:tasks.workunit.client.0.smithi092.stdout:7/363: dwrite d9/dd/d15/f54 [0,4194304] 0 2024-05-06T12:14:28.986 INFO:tasks.workunit.client.0.smithi092.stdout:2/102: sync 2024-05-06T12:14:28.986 INFO:tasks.workunit.client.0.smithi092.stdout:1/121: sync 2024-05-06T12:14:28.988 INFO:tasks.workunit.client.0.smithi092.stdout:9/136: write d16/f1f [341240,101730] 0 2024-05-06T12:14:28.989 INFO:tasks.workunit.client.0.smithi092.stdout:2/103: chown df/d17 520719 1 2024-05-06T12:14:28.990 INFO:tasks.workunit.client.0.smithi092.stdout:5/200: symlink d4/d5/d10/d12/d37/l50 0 2024-05-06T12:14:28.992 INFO:tasks.workunit.client.0.smithi092.stdout:5/201: readlink d4/d34/l11 0 2024-05-06T12:14:28.994 INFO:tasks.workunit.client.0.smithi092.stdout:7/364: fsync d9/dd/d15/f42 0 2024-05-06T12:14:28.995 INFO:tasks.workunit.client.0.smithi092.stdout:1/122: creat d3/f26 x:0 0 0 2024-05-06T12:14:28.996 INFO:tasks.workunit.client.0.smithi092.stdout:8/159: dwrite d9/dc/f2d [0,4194304] 0 2024-05-06T12:14:28.997 INFO:tasks.workunit.client.0.smithi092.stdout:2/104: creat df/d17/f22 x:0 0 0 2024-05-06T12:14:28.998 INFO:tasks.workunit.client.0.smithi092.stdout:7/365: stat d9/de/d1f/l26 0 2024-05-06T12:14:28.999 INFO:tasks.workunit.client.0.smithi092.stdout:5/202: creat d4/d5/d10/d12/d37/f51 x:0 0 0 2024-05-06T12:14:29.001 INFO:tasks.workunit.client.0.smithi092.stdout:7/366: fsync d9/de/d2d/f3a 0 2024-05-06T12:14:29.001 INFO:tasks.workunit.client.0.smithi092.stdout:9/137: dwrite fc [0,4194304] 0 2024-05-06T12:14:29.001 INFO:tasks.workunit.client.0.smithi092.stdout:2/105: mknod df/d15/c23 0 2024-05-06T12:14:29.003 INFO:tasks.workunit.client.0.smithi092.stdout:8/160: rename d9/c1c to d9/dc/c3e 0 2024-05-06T12:14:29.003 INFO:tasks.workunit.client.0.smithi092.stdout:7/367: readlink d9/dd/d23/l33 0 2024-05-06T12:14:29.006 INFO:tasks.workunit.client.0.smithi092.stdout:9/138: creat d16/d1c/f27 x:0 0 0 2024-05-06T12:14:29.007 INFO:tasks.workunit.client.0.smithi092.stdout:5/203: dread d4/d5/d10/d12/f18 [0,4194304] 0 2024-05-06T12:14:29.041 INFO:tasks.workunit.client.0.smithi092.stdout:1/123: getdents d3/d10/d13/d19/d21 0 2024-05-06T12:14:29.041 INFO:tasks.workunit.client.0.smithi092.stdout:8/161: creat d9/d21/f3f x:0 0 0 2024-05-06T12:14:29.041 INFO:tasks.workunit.client.0.smithi092.stdout:7/368: link d9/de/d10/l4a d9/dd/d15/l7f 0 2024-05-06T12:14:29.043 INFO:tasks.workunit.client.0.smithi092.stdout:5/204: dwrite d4/d5/d10/f27 [0,4194304] 0 2024-05-06T12:14:29.045 INFO:tasks.workunit.client.0.smithi092.stdout:7/369: unlink d9/dd/d15/f42 0 2024-05-06T12:14:29.046 INFO:tasks.workunit.client.0.smithi092.stdout:5/205: symlink d4/d5/d10/l52 0 2024-05-06T12:14:29.048 INFO:tasks.workunit.client.0.smithi092.stdout:8/162: dread d9/dc/f16 [0,4194304] 0 2024-05-06T12:14:29.051 INFO:tasks.workunit.client.0.smithi092.stdout:1/124: dwrite d3/d10/d13/d19/d21/f22 [0,4194304] 0 2024-05-06T12:14:29.051 INFO:tasks.workunit.client.0.smithi092.stdout:8/163: chown d9/c17 15995 1 2024-05-06T12:14:29.053 INFO:tasks.workunit.client.0.smithi092.stdout:8/164: read - d9/d21/f38 zero size 2024-05-06T12:14:29.055 INFO:tasks.workunit.client.0.smithi092.stdout:7/370: dwrite d9/dd/f43 [0,4194304] 0 2024-05-06T12:14:29.058 INFO:tasks.workunit.client.0.smithi092.stdout:7/371: creat d9/f80 x:0 0 0 2024-05-06T12:14:29.058 INFO:tasks.workunit.client.0.smithi092.stdout:1/125: dread d3/d10/d13/d19/f1c [0,4194304] 0 2024-05-06T12:14:29.060 INFO:tasks.workunit.client.0.smithi092.stdout:1/126: mknod d3/d10/d13/d19/c27 0 2024-05-06T12:14:29.062 INFO:tasks.workunit.client.0.smithi092.stdout:7/372: creat d9/de/d10/d17/d6a/f81 x:0 0 0 2024-05-06T12:14:29.065 INFO:tasks.workunit.client.0.smithi092.stdout:7/373: mknod d9/de/d10/d17/d21/c82 0 2024-05-06T12:14:29.067 INFO:tasks.workunit.client.0.smithi092.stdout:1/127: dwrite d3/f17 [0,4194304] 0 2024-05-06T12:14:29.067 INFO:tasks.workunit.client.0.smithi092.stdout:1/128: fsync d3/f26 0 2024-05-06T12:14:29.069 INFO:tasks.workunit.client.0.smithi092.stdout:7/374: creat d9/dd/d23/f83 x:0 0 0 2024-05-06T12:14:29.070 INFO:tasks.workunit.client.0.smithi092.stdout:1/129: symlink d3/d10/d13/d19/d21/l28 0 2024-05-06T12:14:29.071 INFO:tasks.workunit.client.0.smithi092.stdout:7/375: symlink d9/de/d1f/d37/d48/d65/l84 0 2024-05-06T12:14:29.071 INFO:tasks.workunit.client.0.smithi092.stdout:1/130: truncate d3/d10/f15 5131630 0 2024-05-06T12:14:29.073 INFO:tasks.workunit.client.0.smithi092.stdout:1/131: dread d3/d10/f15 [4194304,4194304] 0 2024-05-06T12:14:29.075 INFO:tasks.workunit.client.0.smithi092.stdout:1/132: symlink d3/d10/l29 0 2024-05-06T12:14:29.076 INFO:tasks.workunit.client.0.smithi092.stdout:7/376: dwrite d9/de/d10/f25 [0,4194304] 0 2024-05-06T12:14:29.082 INFO:tasks.workunit.client.0.smithi092.stdout:1/133: dwrite d3/d10/d13/d19/f1e [0,4194304] 0 2024-05-06T12:14:29.082 INFO:tasks.workunit.client.0.smithi092.stdout:1/134: stat d3/d10/f15 0 2024-05-06T12:14:29.084 INFO:tasks.workunit.client.0.smithi092.stdout:1/135: write d3/d10/f15 [5133374,101224] 0 2024-05-06T12:14:29.084 INFO:tasks.workunit.client.0.smithi092.stdout:1/136: chown d3/f20 0 1 2024-05-06T12:14:29.085 INFO:tasks.workunit.client.0.smithi092.stdout:1/137: symlink d3/d10/d13/d19/l2a 0 2024-05-06T12:14:29.090 INFO:tasks.workunit.client.0.smithi092.stdout:1/138: dwrite d3/d10/d13/f14 [0,4194304] 0 2024-05-06T12:14:29.094 INFO:tasks.workunit.client.0.smithi092.stdout:1/139: link d3/d10/d13/d19/d21/l28 d3/l2b 0 2024-05-06T12:14:29.094 INFO:tasks.workunit.client.0.smithi092.stdout:1/140: dread - d3/f26 zero size 2024-05-06T12:14:29.096 INFO:tasks.workunit.client.0.smithi092.stdout:1/141: creat d3/f2c x:0 0 0 2024-05-06T12:14:29.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:28 smithi190 ceph-mon[33798]: pgmap v16: 65 pgs: 65 active+clean; 333 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 3.2 MiB/s rd, 25 MiB/s wr, 401 op/s 2024-05-06T12:14:29.099 INFO:tasks.workunit.client.0.smithi092.stdout:1/142: dread d3/d10/d13/f1f [0,4194304] 0 2024-05-06T12:14:29.101 INFO:tasks.workunit.client.0.smithi092.stdout:1/143: write d3/f5 [854250,95074] 0 2024-05-06T12:14:29.103 INFO:tasks.workunit.client.0.smithi092.stdout:1/144: rename d3/d10/d13/d19/c27 to d3/d10/d13/d19/d21/c2d 0 2024-05-06T12:14:29.103 INFO:tasks.workunit.client.0.smithi092.stdout:1/145: fdatasync d3/f2c 0 2024-05-06T12:14:29.113 INFO:tasks.workunit.client.0.smithi092.stdout:1/146: dwrite d3/d10/d13/d19/f1c [0,4194304] 0 2024-05-06T12:14:29.122 INFO:tasks.workunit.client.0.smithi092.stdout:1/147: creat d3/d10/d13/d19/f2e x:0 0 0 2024-05-06T12:14:29.127 INFO:tasks.workunit.client.0.smithi092.stdout:1/148: creat d3/d10/d13/d19/d21/f2f x:0 0 0 2024-05-06T12:14:29.129 INFO:tasks.workunit.client.0.smithi092.stdout:8/165: rmdir d9/dc 39 2024-05-06T12:14:29.129 INFO:tasks.workunit.client.0.smithi092.stdout:8/166: stat d9/f29 0 2024-05-06T12:14:29.131 INFO:tasks.workunit.client.0.smithi092.stdout:1/149: symlink d3/d10/l30 0 2024-05-06T12:14:29.131 INFO:tasks.workunit.client.0.smithi092.stdout:1/150: stat d3/d10/d13/d19/f1c 0 2024-05-06T12:14:29.134 INFO:tasks.workunit.client.0.smithi092.stdout:8/167: unlink d9/dc/f30 0 2024-05-06T12:14:29.135 INFO:tasks.workunit.client.0.smithi092.stdout:8/168: write f0 [4839434,53300] 0 2024-05-06T12:14:29.135 INFO:tasks.workunit.client.0.smithi092.stdout:1/151: creat d3/d10/d13/d19/f31 x:0 0 0 2024-05-06T12:14:29.136 INFO:tasks.workunit.client.0.smithi092.stdout:9/139: truncate fc 113018 0 2024-05-06T12:14:29.138 INFO:tasks.workunit.client.0.smithi092.stdout:1/152: mknod d3/d10/c32 0 2024-05-06T12:14:29.138 INFO:tasks.workunit.client.0.smithi092.stdout:1/153: readlink d3/lb 0 2024-05-06T12:14:29.139 INFO:tasks.workunit.client.0.smithi092.stdout:9/140: creat d16/d1e/f28 x:0 0 0 2024-05-06T12:14:29.142 INFO:tasks.workunit.client.0.smithi092.stdout:9/141: creat d16/f29 x:0 0 0 2024-05-06T12:14:29.143 INFO:tasks.workunit.client.0.smithi092.stdout:9/142: dread - d16/f21 zero size 2024-05-06T12:14:29.144 INFO:tasks.workunit.client.0.smithi092.stdout:5/206: mkdir d4/d5/d10/d31/d53 0 2024-05-06T12:14:29.145 INFO:tasks.workunit.client.0.smithi092.stdout:5/207: fsync d4/d5/d30/f3c 0 2024-05-06T12:14:29.145 INFO:tasks.workunit.client.0.smithi092.stdout:9/143: creat d16/f2a x:0 0 0 2024-05-06T12:14:29.147 INFO:tasks.workunit.client.0.smithi092.stdout:5/208: mkdir d4/d34/d29/d54 0 2024-05-06T12:14:29.153 INFO:tasks.workunit.client.0.smithi092.stdout:8/169: dread f6 [0,4194304] 0 2024-05-06T12:14:29.155 INFO:tasks.workunit.client.0.smithi092.stdout:8/170: read d9/f25 [2374094,75620] 0 2024-05-06T12:14:29.156 INFO:tasks.workunit.client.0.smithi092.stdout:8/171: fdatasync d9/f34 0 2024-05-06T12:14:29.157 INFO:tasks.workunit.client.0.smithi092.stdout:8/172: fdatasync d9/f18 0 2024-05-06T12:14:29.159 INFO:tasks.workunit.client.0.smithi092.stdout:5/209: fsync d4/d5/d10/f27 0 2024-05-06T12:14:29.160 INFO:tasks.workunit.client.0.smithi092.stdout:8/173: mknod d9/dc/c40 0 2024-05-06T12:14:29.161 INFO:tasks.workunit.client.0.smithi092.stdout:8/174: write d9/d21/f38 [350611,53413] 0 2024-05-06T12:14:29.164 INFO:tasks.workunit.client.0.smithi092.stdout:5/210: mknod d4/d5/d10/d12/d37/c55 0 2024-05-06T12:14:29.166 INFO:tasks.workunit.client.0.smithi092.stdout:8/175: mknod d9/d21/c41 0 2024-05-06T12:14:29.168 INFO:tasks.workunit.client.0.smithi092.stdout:5/211: getdents d4/d5/d10/d31/d53 0 2024-05-06T12:14:29.169 INFO:tasks.workunit.client.0.smithi092.stdout:5/212: dread - d4/d5/d10/d12/d1f/d20/d23/f48 zero size 2024-05-06T12:14:29.171 INFO:tasks.workunit.client.0.smithi092.stdout:8/176: rename f7 to d9/f42 0 2024-05-06T12:14:29.174 INFO:tasks.workunit.client.0.smithi092.stdout:8/177: read d9/fe [259209,84675] 0 2024-05-06T12:14:29.174 INFO:tasks.workunit.client.0.smithi092.stdout:7/377: truncate d9/de/d10/f25 543526 0 2024-05-06T12:14:29.175 INFO:tasks.workunit.client.0.smithi092.stdout:7/378: chown d9/de/d1f/d37/d48/d65/f73 763455 1 2024-05-06T12:14:29.176 INFO:tasks.workunit.client.0.smithi092.stdout:5/213: rename d4/d5/d10/f27 to d4/d34/f56 0 2024-05-06T12:14:29.178 INFO:tasks.workunit.client.0.smithi092.stdout:8/178: dread - d9/f1b zero size 2024-05-06T12:14:29.180 INFO:tasks.workunit.client.0.smithi092.stdout:8/179: truncate d9/dc/f2d 4920368 0 2024-05-06T12:14:29.180 INFO:tasks.workunit.client.0.smithi092.stdout:7/379: readlink d9/la 0 2024-05-06T12:14:29.181 INFO:tasks.workunit.client.0.smithi092.stdout:7/380: write d9/dd/d15/f54 [4583265,45481] 0 2024-05-06T12:14:29.183 INFO:tasks.workunit.client.0.smithi092.stdout:7/381: write d9/de/d10/f2b [4780664,43472] 0 2024-05-06T12:14:29.187 INFO:tasks.workunit.client.0.smithi092.stdout:6/250: sync 2024-05-06T12:14:29.215 INFO:tasks.workunit.client.0.smithi092.stdout:8/180: rename d9/dc/l23 to d9/d21/d37/l43 0 2024-05-06T12:14:29.220 INFO:tasks.workunit.client.0.smithi092.stdout:7/382: creat d9/dd/f85 x:0 0 0 2024-05-06T12:14:29.226 INFO:tasks.workunit.client.0.smithi092.stdout:8/181: symlink d9/l44 0 2024-05-06T12:14:29.226 INFO:tasks.workunit.client.0.smithi092.stdout:9/144: getdents d16/d1e 0 2024-05-06T12:14:29.230 INFO:tasks.workunit.client.0.smithi092.stdout:1/154: link d3/d10/d13/d19/d21/c2d d3/c33 0 2024-05-06T12:14:29.230 INFO:tasks.workunit.client.0.smithi092.stdout:9/145: write d16/f21 [135808,104293] 0 2024-05-06T12:14:29.230 INFO:tasks.workunit.client.0.smithi092.stdout:9/146: write f6 [1218398,34632] 0 2024-05-06T12:14:29.230 INFO:tasks.workunit.client.0.smithi092.stdout:9/147: truncate fc 681878 0 2024-05-06T12:14:29.230 INFO:tasks.workunit.client.0.smithi092.stdout:6/251: dread d4/f6 [0,4194304] 0 2024-05-06T12:14:29.233 INFO:tasks.workunit.client.0.smithi092.stdout:8/182: unlink d9/c14 0 2024-05-06T12:14:29.234 INFO:tasks.workunit.client.0.smithi092.stdout:8/183: fdatasync d9/d21/f3f 0 2024-05-06T12:14:29.234 INFO:tasks.workunit.client.0.smithi092.stdout:9/148: creat d16/d1c/f2b x:0 0 0 2024-05-06T12:14:29.236 INFO:tasks.workunit.client.0.smithi092.stdout:9/149: mkdir d16/d2c 0 2024-05-06T12:14:29.237 INFO:tasks.workunit.client.0.smithi092.stdout:1/155: dwrite d3/d10/d13/f1f [0,4194304] 0 2024-05-06T12:14:29.333 INFO:tasks.workunit.client.0.smithi092.stdout:8/184: mkdir d9/dc/d45 0 2024-05-06T12:14:29.334 INFO:tasks.workunit.client.0.smithi092.stdout:7/383: dread d9/de/d10/f25 [0,4194304] 0 2024-05-06T12:14:29.335 INFO:tasks.workunit.client.0.smithi092.stdout:1/156: mkdir d3/d10/d13/d19/d34 0 2024-05-06T12:14:29.335 INFO:tasks.workunit.client.0.smithi092.stdout:9/150: symlink d16/d2c/l2d 0 2024-05-06T12:14:29.337 INFO:tasks.workunit.client.0.smithi092.stdout:7/384: write d9/de/d10/f2b [242897,7706] 0 2024-05-06T12:14:29.337 INFO:tasks.workunit.client.0.smithi092.stdout:8/185: rmdir d9/d21 39 2024-05-06T12:14:29.338 INFO:tasks.workunit.client.0.smithi092.stdout:9/151: write d16/d1e/f26 [698337,117504] 0 2024-05-06T12:14:29.338 INFO:tasks.workunit.client.0.smithi092.stdout:5/214: dread - d4/d5/d10/d12/d37/f44 zero size 2024-05-06T12:14:29.339 INFO:tasks.workunit.client.0.smithi092.stdout:9/152: fsync d16/d1e/f28 0 2024-05-06T12:14:29.341 INFO:tasks.workunit.client.0.smithi092.stdout:1/157: rename d3/d10/c32 to d3/d10/c35 0 2024-05-06T12:14:29.343 INFO:tasks.workunit.client.0.smithi092.stdout:5/215: mknod d4/d5/d10/d12/d1f/c57 0 2024-05-06T12:14:29.347 INFO:tasks.workunit.client.0.smithi092.stdout:1/158: creat d3/d10/d13/d19/d34/f36 x:0 0 0 2024-05-06T12:14:29.348 INFO:tasks.workunit.client.0.smithi092.stdout:5/216: creat d4/d34/d28/d4e/f58 x:0 0 0 2024-05-06T12:14:29.349 INFO:tasks.workunit.client.0.smithi092.stdout:8/186: dwrite d9/d21/d37/f3b [0,4194304] 0 2024-05-06T12:14:29.351 INFO:tasks.workunit.client.0.smithi092.stdout:9/153: dwrite d16/d1e/f26 [0,4194304] 0 2024-05-06T12:14:29.352 INFO:tasks.workunit.client.0.smithi092.stdout:7/385: getdents d9/de/d10/d17/d21/d38/d77 0 2024-05-06T12:14:29.352 INFO:tasks.workunit.client.0.smithi092.stdout:8/187: truncate d9/f29 413878 0 2024-05-06T12:14:29.353 INFO:tasks.workunit.client.0.smithi092.stdout:1/159: creat d3/d10/f37 x:0 0 0 2024-05-06T12:14:29.355 INFO:tasks.workunit.client.0.smithi092.stdout:1/160: rename d3 to d3/d10/d13/d19/d34/d38 22 2024-05-06T12:14:29.355 INFO:tasks.workunit.client.0.smithi092.stdout:5/217: mkdir d4/d59 0 2024-05-06T12:14:29.357 INFO:tasks.workunit.client.0.smithi092.stdout:9/154: creat d16/d1e/f2e x:0 0 0 2024-05-06T12:14:29.358 INFO:tasks.workunit.client.0.smithi092.stdout:9/155: fsync d16/f1b 0 2024-05-06T12:14:29.359 INFO:tasks.workunit.client.0.smithi092.stdout:8/188: fsync d9/f25 0 2024-05-06T12:14:29.360 INFO:tasks.workunit.client.0.smithi092.stdout:8/189: chown d9/l44 12945 1 2024-05-06T12:14:29.361 INFO:tasks.workunit.client.0.smithi092.stdout:5/218: dread - d4/d5/d10/d31/f4a zero size 2024-05-06T12:14:29.363 INFO:tasks.workunit.client.0.smithi092.stdout:5/219: write d4/d5/d10/d12/d1f/d20/d23/f48 [29362,84158] 0 2024-05-06T12:14:29.365 INFO:tasks.workunit.client.0.smithi092.stdout:8/190: mkdir d9/d21/d37/d46 0 2024-05-06T12:14:29.366 INFO:tasks.workunit.client.0.smithi092.stdout:7/386: mknod d9/de/d10/d17/d6a/c86 0 2024-05-06T12:14:29.367 INFO:tasks.workunit.client.0.smithi092.stdout:1/161: creat d3/f39 x:0 0 0 2024-05-06T12:14:29.368 INFO:tasks.workunit.client.0.smithi092.stdout:5/220: creat d4/d5/d10/d12/d1f/d20/d23/f5a x:0 0 0 2024-05-06T12:14:29.369 INFO:tasks.workunit.client.0.smithi092.stdout:1/162: write d3/d10/d13/d19/d21/f24 [935724,118044] 0 2024-05-06T12:14:29.370 INFO:tasks.workunit.client.0.smithi092.stdout:8/191: mknod d9/d21/d37/d46/c47 0 2024-05-06T12:14:29.373 INFO:tasks.workunit.client.0.smithi092.stdout:7/387: mkdir d9/de/d10/d17/d21/d2e/d87 0 2024-05-06T12:14:29.376 INFO:tasks.workunit.client.0.smithi092.stdout:5/221: dwrite d4/d5/f2a [0,4194304] 0 2024-05-06T12:14:29.376 INFO:tasks.workunit.client.0.smithi092.stdout:8/192: unlink d9/d21/l2b 0 2024-05-06T12:14:29.380 INFO:tasks.workunit.client.0.smithi092.stdout:1/163: link d3/f17 d3/d10/d13/d19/f3a 0 2024-05-06T12:14:29.381 INFO:tasks.workunit.client.0.smithi092.stdout:1/164: chown d3/d10/d13/d19/d21/f2f 152748307 1 2024-05-06T12:14:29.383 INFO:tasks.workunit.client.0.smithi092.stdout:0/234: sync 2024-05-06T12:14:29.383 INFO:tasks.workunit.client.0.smithi092.stdout:7/388: truncate f3 2875680 0 2024-05-06T12:14:29.384 INFO:tasks.workunit.client.0.smithi092.stdout:5/222: rmdir d4/d5/d10/d12/d1f/d20 39 2024-05-06T12:14:29.387 INFO:tasks.workunit.client.0.smithi092.stdout:8/193: rmdir d9/dc/d45 0 2024-05-06T12:14:29.389 INFO:tasks.workunit.client.0.smithi092.stdout:7/389: fsync d9/de/d10/d17/d21/f3e 0 2024-05-06T12:14:29.390 INFO:tasks.workunit.client.0.smithi092.stdout:1/165: dwrite d3/d10/d13/d19/f1e [0,4194304] 0 2024-05-06T12:14:29.392 INFO:tasks.workunit.client.0.smithi092.stdout:1/166: chown d3/d10/d13 1 1 2024-05-06T12:14:29.393 INFO:tasks.workunit.client.0.smithi092.stdout:5/223: rename d4/d5/cc to d4/d5/d10/d12/c5b 0 2024-05-06T12:14:29.393 INFO:tasks.workunit.client.0.smithi092.stdout:8/194: rename d9/d21 to d9/d21/d37/d46/d48 22 2024-05-06T12:14:29.394 INFO:tasks.workunit.client.0.smithi092.stdout:7/390: rename d9 to d9/de/d1f/d37/d88 22 2024-05-06T12:14:29.394 INFO:tasks.workunit.client.0.smithi092.stdout:5/224: chown d4/d34/d28/d2b 532779174 1 2024-05-06T12:14:29.400 INFO:tasks.workunit.client.0.smithi092.stdout:1/167: truncate d3/f17 3829061 0 2024-05-06T12:14:29.402 INFO:tasks.workunit.client.0.smithi092.stdout:8/195: symlink d9/d21/d37/d46/l49 0 2024-05-06T12:14:29.403 INFO:tasks.workunit.client.0.smithi092.stdout:5/225: dwrite d4/d34/d28/f3a [0,4194304] 0 2024-05-06T12:14:29.404 INFO:tasks.workunit.client.0.smithi092.stdout:7/391: creat d9/de/d10/d17/d21/d38/f89 x:0 0 0 2024-05-06T12:14:29.406 INFO:tasks.workunit.client.0.smithi092.stdout:7/392: chown d9/de/d10/d17/d21/d2e/l41 894033052 1 2024-05-06T12:14:29.409 INFO:tasks.workunit.client.0.smithi092.stdout:8/196: mkdir d9/dc/d4a 0 2024-05-06T12:14:29.411 INFO:tasks.workunit.client.0.smithi092.stdout:0/235: dread d7/dc/f29 [0,4194304] 0 2024-05-06T12:14:29.413 INFO:tasks.workunit.client.0.smithi092.stdout:7/393: creat d9/de/d2d/f8a x:0 0 0 2024-05-06T12:14:29.416 INFO:tasks.workunit.client.0.smithi092.stdout:5/226: rename d4/d5/d10/d12/c43 to d4/d5/d10/d12/c5c 0 2024-05-06T12:14:29.417 INFO:tasks.workunit.client.0.smithi092.stdout:5/227: readlink d4/d34/l11 0 2024-05-06T12:14:29.418 INFO:tasks.workunit.client.0.smithi092.stdout:1/168: creat d3/d10/d13/f3b x:0 0 0 2024-05-06T12:14:29.419 INFO:tasks.workunit.client.0.smithi092.stdout:1/169: read - d3/d10/d13/d19/f31 zero size 2024-05-06T12:14:29.420 INFO:tasks.workunit.client.0.smithi092.stdout:1/170: truncate d3/f26 433624 0 2024-05-06T12:14:29.421 INFO:tasks.workunit.client.0.smithi092.stdout:7/394: rmdir d9/de/d1f 39 2024-05-06T12:14:29.423 INFO:tasks.workunit.client.0.smithi092.stdout:8/197: mknod d9/c4b 0 2024-05-06T12:14:29.426 INFO:tasks.workunit.client.0.smithi092.stdout:5/228: rename d4/d34/d28/d2b to d4/d34/d28/d4e/d5d 0 2024-05-06T12:14:29.428 INFO:tasks.workunit.client.0.smithi092.stdout:0/236: truncate f1 783693 0 2024-05-06T12:14:29.430 INFO:tasks.workunit.client.0.smithi092.stdout:8/198: symlink d9/d21/d37/d46/l4c 0 2024-05-06T12:14:29.433 INFO:tasks.workunit.client.0.smithi092.stdout:7/395: rename d9/dd/d15/c71 to d9/de/d10/d17/d21/d2e/d87/c8b 0 2024-05-06T12:14:29.434 INFO:tasks.workunit.client.0.smithi092.stdout:0/237: creat d7/da/d1e/d26/f53 x:0 0 0 2024-05-06T12:14:29.435 INFO:tasks.workunit.client.0.smithi092.stdout:5/229: dread f0 [0,4194304] 0 2024-05-06T12:14:29.436 INFO:tasks.workunit.client.0.smithi092.stdout:0/238: fsync d7/da/d1e/d39/f47 0 2024-05-06T12:14:29.438 INFO:tasks.workunit.client.0.smithi092.stdout:1/171: rename d3/f39 to d3/f3c 0 2024-05-06T12:14:29.445 INFO:tasks.workunit.client.0.smithi092.stdout:7/396: creat d9/de/d1f/d37/f8c x:0 0 0 2024-05-06T12:14:29.447 INFO:tasks.workunit.client.0.smithi092.stdout:1/172: unlink d3/d10/f15 0 2024-05-06T12:14:29.448 INFO:tasks.workunit.client.0.smithi092.stdout:7/397: dread - d9/de/d1f/d37/d48/d65/f6b zero size 2024-05-06T12:14:29.448 INFO:tasks.workunit.client.0.smithi092.stdout:8/199: dwrite d9/f18 [0,4194304] 0 2024-05-06T12:14:29.449 INFO:tasks.workunit.client.0.smithi092.stdout:5/230: creat d4/d5/d10/d12/d1f/d20/d23/f5e x:0 0 0 2024-05-06T12:14:29.451 INFO:tasks.workunit.client.0.smithi092.stdout:8/200: truncate d9/f36 147212 0 2024-05-06T12:14:29.454 INFO:tasks.workunit.client.0.smithi092.stdout:7/398: read d9/de/f34 [2486256,113002] 0 2024-05-06T12:14:29.455 INFO:tasks.workunit.client.0.smithi092.stdout:5/231: mknod d4/d5/d10/d31/c5f 0 2024-05-06T12:14:29.456 INFO:tasks.workunit.client.0.smithi092.stdout:0/239: dwrite d7/dc/f29 [0,4194304] 0 2024-05-06T12:14:29.460 INFO:tasks.workunit.client.0.smithi092.stdout:0/240: fsync d7/d43/f49 0 2024-05-06T12:14:29.464 INFO:tasks.workunit.client.0.smithi092.stdout:1/173: dwrite d3/d10/d13/f14 [0,4194304] 0 2024-05-06T12:14:29.466 INFO:tasks.workunit.client.0.smithi092.stdout:7/399: dwrite d9/de/d10/f27 [0,4194304] 0 2024-05-06T12:14:29.470 INFO:tasks.workunit.client.0.smithi092.stdout:3/199: sync 2024-05-06T12:14:29.470 INFO:tasks.workunit.client.0.smithi092.stdout:2/106: sync 2024-05-06T12:14:29.470 INFO:tasks.workunit.client.0.smithi092.stdout:4/160: sync 2024-05-06T12:14:29.470 INFO:tasks.workunit.client.0.smithi092.stdout:0/241: rename d7/da to d7/dc/d28/d54 0 2024-05-06T12:14:29.471 INFO:tasks.workunit.client.0.smithi092.stdout:3/200: chown d8/d24/c22 1910834 1 2024-05-06T12:14:29.474 INFO:tasks.workunit.client.0.smithi092.stdout:4/161: write d9/f21 [1797,30283] 0 2024-05-06T12:14:29.474 INFO:tasks.workunit.client.0.smithi092.stdout:5/232: symlink d4/d5/d10/d12/d1f/d20/l60 0 2024-05-06T12:14:29.474 INFO:tasks.workunit.client.0.smithi092.stdout:3/201: truncate d8/d1b/f3e 472263 0 2024-05-06T12:14:29.477 INFO:tasks.workunit.client.0.smithi092.stdout:8/201: truncate f0 4880613 0 2024-05-06T12:14:29.478 INFO:tasks.workunit.client.0.smithi092.stdout:2/107: truncate f1 1632969 0 2024-05-06T12:14:29.478 INFO:tasks.workunit.client.0.smithi092.stdout:5/233: dread d4/d5/d10/d12/f18 [0,4194304] 0 2024-05-06T12:14:29.480 INFO:tasks.workunit.client.0.smithi092.stdout:2/108: fsync df/d17/d1d/f21 0 2024-05-06T12:14:29.481 INFO:tasks.workunit.client.0.smithi092.stdout:7/400: dwrite d9/dd/f85 [0,4194304] 0 2024-05-06T12:14:29.483 INFO:tasks.workunit.client.0.smithi092.stdout:8/202: rmdir d9/dc 39 2024-05-06T12:14:29.483 INFO:tasks.workunit.client.0.smithi092.stdout:2/109: chown df/d17/f22 203 1 2024-05-06T12:14:29.485 INFO:tasks.workunit.client.0.smithi092.stdout:1/174: rename d3/f3c to d3/f3d 0 2024-05-06T12:14:29.486 INFO:tasks.workunit.client.0.smithi092.stdout:5/234: creat d4/d5/d30/f61 x:0 0 0 2024-05-06T12:14:29.488 INFO:tasks.workunit.client.0.smithi092.stdout:4/162: dwrite d9/d2b/f32 [0,4194304] 0 2024-05-06T12:14:29.490 INFO:tasks.workunit.client.0.smithi092.stdout:1/175: fdatasync d3/d10/d13/f14 0 2024-05-06T12:14:29.491 INFO:tasks.workunit.client.0.smithi092.stdout:2/110: mknod df/d15/c24 0 2024-05-06T12:14:29.492 INFO:tasks.workunit.client.0.smithi092.stdout:3/202: creat d8/d24/d27/f42 x:0 0 0 2024-05-06T12:14:29.496 INFO:tasks.workunit.client.0.smithi092.stdout:3/203: chown d8/d24/d27 2093532 1 2024-05-06T12:14:29.496 INFO:tasks.workunit.client.0.smithi092.stdout:2/111: stat df/d17/l20 0 2024-05-06T12:14:29.496 INFO:tasks.workunit.client.0.smithi092.stdout:2/112: chown df/d1b 7950 1 2024-05-06T12:14:29.496 INFO:tasks.workunit.client.0.smithi092.stdout:2/113: stat df/d15/c1a 0 2024-05-06T12:14:29.498 INFO:tasks.workunit.client.0.smithi092.stdout:2/114: write fe [1713766,98843] 0 2024-05-06T12:14:29.499 INFO:tasks.workunit.client.0.smithi092.stdout:8/203: creat d9/dc/f4d x:0 0 0 2024-05-06T12:14:29.502 INFO:tasks.workunit.client.0.smithi092.stdout:8/204: write d9/dc/ff [1151865,26673] 0 2024-05-06T12:14:29.502 INFO:tasks.workunit.client.0.smithi092.stdout:4/163: dwrite d9/f21 [0,4194304] 0 2024-05-06T12:14:29.503 INFO:tasks.workunit.client.0.smithi092.stdout:8/205: readlink d9/d21/l24 0 2024-05-06T12:14:29.504 INFO:tasks.workunit.client.0.smithi092.stdout:3/204: dread d8/db/d14/f18 [0,4194304] 0 2024-05-06T12:14:29.505 INFO:tasks.workunit.client.0.smithi092.stdout:0/242: link d7/dc/d28/d54/d1e/c48 d7/c55 0 2024-05-06T12:14:29.505 INFO:tasks.workunit.client.0.smithi092.stdout:9/156: sync 2024-05-06T12:14:29.505 INFO:tasks.workunit.client.0.smithi092.stdout:6/252: sync 2024-05-06T12:14:29.506 INFO:tasks.workunit.client.0.smithi092.stdout:6/253: write d4/d20/d23/f46 [1185954,72048] 0 2024-05-06T12:14:29.507 INFO:tasks.workunit.client.0.smithi092.stdout:7/401: rename d9/de/d1f/f31 to d9/de/d10/d17/d21/f8d 0 2024-05-06T12:14:29.507 INFO:tasks.workunit.client.0.smithi092.stdout:2/115: rmdir df/d17/d1d 39 2024-05-06T12:14:29.509 INFO:tasks.workunit.client.0.smithi092.stdout:6/254: write d4/d20/d23/d30/f36 [687088,81964] 0 2024-05-06T12:14:29.510 INFO:tasks.workunit.client.0.smithi092.stdout:5/235: link d4/d5/d30/f3c d4/d34/d29/d54/f62 0 2024-05-06T12:14:29.511 INFO:tasks.workunit.client.0.smithi092.stdout:4/164: mknod d9/dd/c3a 0 2024-05-06T12:14:29.512 INFO:tasks.workunit.client.0.smithi092.stdout:4/165: chown d9/lb 451108 1 2024-05-06T12:14:29.515 INFO:tasks.workunit.client.0.smithi092.stdout:4/166: read d9/f1c [2474341,73621] 0 2024-05-06T12:14:29.515 INFO:tasks.workunit.client.0.smithi092.stdout:9/157: mkdir d16/d2c/d2f 0 2024-05-06T12:14:29.519 INFO:tasks.workunit.client.0.smithi092.stdout:4/167: write d9/dd/f17 [4697592,57751] 0 2024-05-06T12:14:29.521 INFO:tasks.workunit.client.0.smithi092.stdout:5/236: dwrite d4/d5/d30/f46 [0,4194304] 0 2024-05-06T12:14:29.521 INFO:tasks.workunit.client.0.smithi092.stdout:2/116: stat df/c14 0 2024-05-06T12:14:29.523 INFO:tasks.workunit.client.0.smithi092.stdout:1/176: creat d3/f3e x:0 0 0 2024-05-06T12:14:29.525 INFO:tasks.workunit.client.0.smithi092.stdout:6/255: creat d4/d24/f55 x:0 0 0 2024-05-06T12:14:29.528 INFO:tasks.workunit.client.0.smithi092.stdout:3/205: mkdir d8/d24/d33/d39/d43 0 2024-05-06T12:14:29.531 INFO:tasks.workunit.client.0.smithi092.stdout:4/168: dwrite d9/dd/f19 [0,4194304] 0 2024-05-06T12:14:29.532 INFO:tasks.workunit.client.0.smithi092.stdout:3/206: truncate d8/d24/d27/f42 453779 0 2024-05-06T12:14:29.535 INFO:tasks.workunit.client.0.smithi092.stdout:8/206: symlink d9/d21/l4e 0 2024-05-06T12:14:29.536 INFO:tasks.workunit.client.0.smithi092.stdout:2/117: truncate df/f11 4492326 0 2024-05-06T12:14:29.539 INFO:tasks.workunit.client.0.smithi092.stdout:6/256: creat d4/d4c/d52/f56 x:0 0 0 2024-05-06T12:14:29.540 INFO:tasks.workunit.client.0.smithi092.stdout:4/169: creat d9/dd/f3b x:0 0 0 2024-05-06T12:14:29.542 INFO:tasks.workunit.client.0.smithi092.stdout:7/402: getdents d9/de/d10 0 2024-05-06T12:14:29.544 INFO:tasks.workunit.client.0.smithi092.stdout:8/207: write d9/dc/f16 [3545569,17437] 0 2024-05-06T12:14:29.546 INFO:tasks.workunit.client.0.smithi092.stdout:8/208: readlink d9/dc/l3c 0 2024-05-06T12:14:29.547 INFO:tasks.workunit.client.0.smithi092.stdout:4/170: dread d9/df/f14 [0,4194304] 0 2024-05-06T12:14:29.548 INFO:tasks.workunit.client.0.smithi092.stdout:8/209: chown d9/d21/c41 6609813 1 2024-05-06T12:14:29.551 INFO:tasks.workunit.client.0.smithi092.stdout:2/118: mknod df/d17/c25 0 2024-05-06T12:14:29.551 INFO:tasks.workunit.client.0.smithi092.stdout:2/119: chown f6 525632 1 2024-05-06T12:14:29.553 INFO:tasks.workunit.client.0.smithi092.stdout:1/177: rename d3/d10/f11 to d3/d10/d13/f3f 0 2024-05-06T12:14:29.556 INFO:tasks.workunit.client.0.smithi092.stdout:2/120: dread df/d17/f19 [0,4194304] 0 2024-05-06T12:14:29.556 INFO:tasks.workunit.client.0.smithi092.stdout:6/257: truncate d4/d20/f31 4028898 0 2024-05-06T12:14:29.557 INFO:tasks.workunit.client.0.smithi092.stdout:4/171: stat d9/df/c26 0 2024-05-06T12:14:29.558 INFO:tasks.workunit.client.0.smithi092.stdout:2/121: read f5 [657363,40023] 0 2024-05-06T12:14:29.558 INFO:tasks.workunit.client.0.smithi092.stdout:2/122: fsync df/d17/f18 0 2024-05-06T12:14:29.559 INFO:tasks.workunit.client.0.smithi092.stdout:8/210: symlink d9/d21/d37/l4f 0 2024-05-06T12:14:29.560 INFO:tasks.workunit.client.0.smithi092.stdout:8/211: stat d9/dc 0 2024-05-06T12:14:29.561 INFO:tasks.workunit.client.0.smithi092.stdout:9/158: truncate d16/d1e/f26 2845944 0 2024-05-06T12:14:29.568 INFO:tasks.workunit.client.0.smithi092.stdout:3/207: dwrite d8/d9/f41 [0,4194304] 0 2024-05-06T12:14:29.570 INFO:tasks.workunit.client.0.smithi092.stdout:7/403: write d9/de/d10/d17/d21/f49 [906756,117431] 0 2024-05-06T12:14:29.572 INFO:tasks.workunit.client.0.smithi092.stdout:8/212: unlink d9/d21/d37/d46/c47 0 2024-05-06T12:14:29.583 INFO:tasks.workunit.client.0.smithi092.stdout:8/213: dwrite d9/f29 [0,4194304] 0 2024-05-06T12:14:29.584 INFO:tasks.workunit.client.0.smithi092.stdout:9/159: dwrite fc [0,4194304] 0 2024-05-06T12:14:29.587 INFO:tasks.workunit.client.0.smithi092.stdout:4/172: rename d9/df/l2c to d9/df/l3c 0 2024-05-06T12:14:29.588 INFO:tasks.workunit.client.0.smithi092.stdout:1/178: truncate d3/f5 85301 0 2024-05-06T12:14:29.588 INFO:tasks.workunit.client.0.smithi092.stdout:2/123: fdatasync df/f12 0 2024-05-06T12:14:29.590 INFO:tasks.workunit.client.0.smithi092.stdout:1/179: truncate d3/f20 870374 0 2024-05-06T12:14:29.590 INFO:tasks.workunit.client.0.smithi092.stdout:3/208: rename d8/db/d14/f38 to d8/d24/d33/d39/f44 0 2024-05-06T12:14:29.593 INFO:tasks.workunit.client.0.smithi092.stdout:7/404: chown d9/de/d10/l4a 6384 1 2024-05-06T12:14:29.593 INFO:tasks.workunit.client.0.smithi092.stdout:9/160: dwrite d16/d1c/f2b [0,4194304] 0 2024-05-06T12:14:29.593 INFO:tasks.workunit.client.0.smithi092.stdout:6/258: truncate d4/f17 4819116 0 2024-05-06T12:14:29.595 INFO:tasks.workunit.client.0.smithi092.stdout:9/161: dread - d16/d1c/f27 zero size 2024-05-06T12:14:29.600 INFO:tasks.workunit.client.0.smithi092.stdout:2/124: read f5 [1188110,122417] 0 2024-05-06T12:14:29.601 INFO:tasks.workunit.client.0.smithi092.stdout:1/180: creat d3/d10/d13/d19/f40 x:0 0 0 2024-05-06T12:14:29.602 INFO:tasks.workunit.client.0.smithi092.stdout:3/209: dread d8/d9/f17 [0,4194304] 0 2024-05-06T12:14:29.602 INFO:tasks.workunit.client.0.smithi092.stdout:9/162: creat d16/d1c/f30 x:0 0 0 2024-05-06T12:14:29.604 INFO:tasks.workunit.client.0.smithi092.stdout:9/163: chown d16/f17 187156222 1 2024-05-06T12:14:29.604 INFO:tasks.workunit.client.0.smithi092.stdout:2/125: mknod df/d15/c26 0 2024-05-06T12:14:29.604 INFO:tasks.workunit.client.0.smithi092.stdout:9/164: chown f6 167095 1 2024-05-06T12:14:29.605 INFO:tasks.workunit.client.0.smithi092.stdout:1/181: symlink d3/d10/d13/d19/l41 0 2024-05-06T12:14:29.606 INFO:tasks.workunit.client.0.smithi092.stdout:1/182: dread - d3/d10/d13/d19/f40 zero size 2024-05-06T12:14:29.607 INFO:tasks.workunit.client.0.smithi092.stdout:8/214: rename d9/d21/d37/d46/l4c to d9/l50 0 2024-05-06T12:14:29.608 INFO:tasks.workunit.client.0.smithi092.stdout:3/210: fsync d8/d9/f17 0 2024-05-06T12:14:29.609 INFO:tasks.workunit.client.0.smithi092.stdout:8/215: chown d9/l44 10477638 1 2024-05-06T12:14:29.609 INFO:tasks.workunit.client.0.smithi092.stdout:8/216: stat d9/d21/l24 0 2024-05-06T12:14:29.610 INFO:tasks.workunit.client.0.smithi092.stdout:2/126: write df/d17/d1d/f21 [1007485,108371] 0 2024-05-06T12:14:29.612 INFO:tasks.workunit.client.0.smithi092.stdout:4/173: rename d9/l1d to d9/l3d 0 2024-05-06T12:14:29.614 INFO:tasks.workunit.client.0.smithi092.stdout:6/259: rename d4/l13 to d4/d24/d4e/l57 0 2024-05-06T12:14:29.616 INFO:tasks.workunit.client.0.smithi092.stdout:7/405: getdents d9/de/d2d 0 2024-05-06T12:14:29.617 INFO:tasks.workunit.client.0.smithi092.stdout:2/127: dwrite df/d17/f22 [0,4194304] 0 2024-05-06T12:14:29.618 INFO:tasks.workunit.client.0.smithi092.stdout:1/183: mkdir d3/d42 0 2024-05-06T12:14:29.619 INFO:tasks.workunit.client.0.smithi092.stdout:4/174: symlink d9/df/l3e 0 2024-05-06T12:14:29.619 INFO:tasks.workunit.client.0.smithi092.stdout:1/184: write d3/f26 [115014,118572] 0 2024-05-06T12:14:29.622 INFO:tasks.workunit.client.0.smithi092.stdout:6/260: mknod d4/d20/d23/d30/c58 0 2024-05-06T12:14:29.624 INFO:tasks.workunit.client.0.smithi092.stdout:4/175: dread d9/f38 [0,4194304] 0 2024-05-06T12:14:29.625 INFO:tasks.workunit.client.0.smithi092.stdout:8/217: link d9/f29 d9/dc/f51 0 2024-05-06T12:14:29.626 INFO:tasks.workunit.client.0.smithi092.stdout:7/406: creat d9/de/d1f/d37/d48/f8e x:0 0 0 2024-05-06T12:14:29.630 INFO:tasks.workunit.client.0.smithi092.stdout:2/128: symlink df/d17/d1d/l27 0 2024-05-06T12:14:29.633 INFO:tasks.workunit.client.0.smithi092.stdout:8/218: dwrite d9/f35 [0,4194304] 0 2024-05-06T12:14:29.638 INFO:tasks.workunit.client.0.smithi092.stdout:3/211: link d8/d1b/f3e d8/f45 0 2024-05-06T12:14:29.638 INFO:tasks.workunit.client.0.smithi092.stdout:3/212: readlink d8/db/l20 0 2024-05-06T12:14:29.638 INFO:tasks.workunit.client.0.smithi092.stdout:3/213: readlink d8/d9/l3b 0 2024-05-06T12:14:29.640 INFO:tasks.workunit.client.0.smithi092.stdout:7/407: creat d9/de/d10/d17/f8f x:0 0 0 2024-05-06T12:14:29.641 INFO:tasks.workunit.client.0.smithi092.stdout:7/408: write d9/f40 [439523,49799] 0 2024-05-06T12:14:29.644 INFO:tasks.workunit.client.0.smithi092.stdout:1/185: creat d3/d42/f43 x:0 0 0 2024-05-06T12:14:29.646 INFO:tasks.workunit.client.0.smithi092.stdout:2/129: rename df/l10 to df/d15/l28 0 2024-05-06T12:14:29.651 INFO:tasks.workunit.client.0.smithi092.stdout:1/186: dwrite d3/d10/d13/d19/d34/f36 [0,4194304] 0 2024-05-06T12:14:29.654 INFO:tasks.workunit.client.0.smithi092.stdout:7/409: truncate d9/dd/d15/f14 348513 0 2024-05-06T12:14:29.658 INFO:tasks.workunit.client.0.smithi092.stdout:8/219: fsync d9/f29 0 2024-05-06T12:14:29.662 INFO:tasks.workunit.client.0.smithi092.stdout:1/187: dwrite d3/d10/d13/d19/f40 [0,4194304] 0 2024-05-06T12:14:29.662 INFO:tasks.workunit.client.0.smithi092.stdout:8/220: dread d9/f36 [0,4194304] 0 2024-05-06T12:14:29.663 INFO:tasks.workunit.client.0.smithi092.stdout:7/410: creat d9/de/d10/d17/d21/d38/f90 x:0 0 0 2024-05-06T12:14:29.670 INFO:tasks.workunit.client.0.smithi092.stdout:7/411: dread d9/de/d1f/f4c [0,4194304] 0 2024-05-06T12:14:29.670 INFO:tasks.workunit.client.0.smithi092.stdout:8/221: fsync d9/f25 0 2024-05-06T12:14:29.671 INFO:tasks.workunit.client.0.smithi092.stdout:1/188: symlink d3/d10/d13/d19/d21/l44 0 2024-05-06T12:14:29.679 INFO:tasks.workunit.client.0.smithi092.stdout:4/176: truncate d9/dd/f16 3028776 0 2024-05-06T12:14:29.679 INFO:tasks.workunit.client.0.smithi092.stdout:7/412: dwrite d9/de/d2d/f8a [0,4194304] 0 2024-05-06T12:14:29.683 INFO:tasks.workunit.client.0.smithi092.stdout:8/222: rename d9/c19 to d9/d21/d37/c52 0 2024-05-06T12:14:29.686 INFO:tasks.workunit.client.0.smithi092.stdout:7/413: dread - d9/de/d1f/d37/d39/f59 zero size 2024-05-06T12:14:29.687 INFO:tasks.workunit.client.0.smithi092.stdout:9/165: dwrite d16/d1e/f26 [0,4194304] 0 2024-05-06T12:14:29.691 INFO:tasks.workunit.client.0.smithi092.stdout:6/261: dwrite d4/d20/f31 [0,4194304] 0 2024-05-06T12:14:29.699 INFO:tasks.workunit.client.0.smithi092.stdout:4/177: dwrite f5 [0,4194304] 0 2024-05-06T12:14:29.700 INFO:tasks.workunit.client.0.smithi092.stdout:2/130: dwrite f1 [0,4194304] 0 2024-05-06T12:14:29.702 INFO:tasks.workunit.client.0.smithi092.stdout:4/178: dread - d9/f34 zero size 2024-05-06T12:14:29.706 INFO:tasks.workunit.client.0.smithi092.stdout:4/179: chown f5 107164 1 2024-05-06T12:14:29.709 INFO:tasks.workunit.client.0.smithi092.stdout:4/180: write d9/d27/f31 [667774,60673] 0 2024-05-06T12:14:29.709 INFO:tasks.workunit.client.0.smithi092.stdout:4/181: chown d9/fc 53085 1 2024-05-06T12:14:29.710 INFO:tasks.workunit.client.0.smithi092.stdout:4/182: stat d9/dd 0 2024-05-06T12:14:29.710 INFO:tasks.workunit.client.0.smithi092.stdout:7/414: rmdir d9/de/d2d 39 2024-05-06T12:14:29.712 INFO:tasks.workunit.client.0.smithi092.stdout:4/183: write d9/dd/f23 [117540,7060] 0 2024-05-06T12:14:29.715 INFO:tasks.workunit.client.0.smithi092.stdout:9/166: rename d16/f21 to d16/d2c/d2f/f31 0 2024-05-06T12:14:29.725 INFO:tasks.workunit.client.0.smithi092.stdout:1/189: dwrite d3/d10/d13/d19/f3a [0,4194304] 0 2024-05-06T12:14:29.726 INFO:tasks.workunit.client.0.smithi092.stdout:1/190: write d3/f20 [664326,46404] 0 2024-05-06T12:14:29.727 INFO:tasks.workunit.client.0.smithi092.stdout:1/191: fdatasync d3/d42/f43 0 2024-05-06T12:14:29.730 INFO:tasks.workunit.client.0.smithi092.stdout:9/167: mkdir d16/d1c/d32 0 2024-05-06T12:14:29.739 INFO:tasks.workunit.client.0.smithi092.stdout:8/223: getdents d9/d21/d37/d46 0 2024-05-06T12:14:29.740 INFO:tasks.workunit.client.0.smithi092.stdout:2/131: dread fd [0,4194304] 0 2024-05-06T12:14:29.740 INFO:tasks.workunit.client.0.smithi092.stdout:6/262: write d4/d4c/d52/f4a [3361978,69972] 0 2024-05-06T12:14:29.740 INFO:tasks.workunit.client.0.smithi092.stdout:8/224: chown d9/dc/ff 3510 1 2024-05-06T12:14:29.741 INFO:tasks.workunit.client.0.smithi092.stdout:2/132: write fe [266533,19909] 0 2024-05-06T12:14:29.742 INFO:tasks.workunit.client.0.smithi092.stdout:4/184: creat d9/f3f x:0 0 0 2024-05-06T12:14:29.742 INFO:tasks.workunit.client.0.smithi092.stdout:6/263: fsync d4/d4c/d52/f56 0 2024-05-06T12:14:29.743 INFO:tasks.workunit.client.0.smithi092.stdout:7/415: dread f1 [4194304,4194304] 0 2024-05-06T12:14:29.744 INFO:tasks.workunit.client.0.smithi092.stdout:7/416: read - d9/f5a zero size 2024-05-06T12:14:29.744 INFO:tasks.workunit.client.0.smithi092.stdout:4/185: read d9/dd/f17 [1140981,94144] 0 2024-05-06T12:14:29.745 INFO:tasks.workunit.client.0.smithi092.stdout:7/417: dread - d9/de/f50 zero size 2024-05-06T12:14:29.745 INFO:tasks.workunit.client.0.smithi092.stdout:9/168: getdents d16/d2c/d2f 0 2024-05-06T12:14:29.747 INFO:tasks.workunit.client.0.smithi092.stdout:9/169: write d16/f1f [513655,100231] 0 2024-05-06T12:14:29.747 INFO:tasks.workunit.client.0.smithi092.stdout:9/170: truncate d16/f17 4903705 0 2024-05-06T12:14:29.748 INFO:tasks.workunit.client.0.smithi092.stdout:6/264: creat d4/d20/d23/d30/f59 x:0 0 0 2024-05-06T12:14:29.749 INFO:tasks.workunit.client.0.smithi092.stdout:9/171: chown d16/f2a 5 1 2024-05-06T12:14:29.750 INFO:tasks.workunit.client.0.smithi092.stdout:7/418: read d9/de/d2d/f8a [3484404,26959] 0 2024-05-06T12:14:29.752 INFO:tasks.workunit.client.0.smithi092.stdout:8/225: link d9/fe d9/dc/f53 0 2024-05-06T12:14:29.754 INFO:tasks.workunit.client.0.smithi092.stdout:7/419: chown f3 1389189 1 2024-05-06T12:14:29.759 INFO:tasks.workunit.client.0.smithi092.stdout:7/420: dread - d9/de/d10/d17/d21/d38/f90 zero size 2024-05-06T12:14:29.759 INFO:tasks.workunit.client.0.smithi092.stdout:6/265: dwrite d4/d4c/d52/f4a [0,4194304] 0 2024-05-06T12:14:29.761 INFO:tasks.workunit.client.0.smithi092.stdout:1/192: dwrite d3/f12 [4194304,4194304] 0 2024-05-06T12:14:29.761 INFO:tasks.workunit.client.0.smithi092.stdout:2/133: dread f5 [0,4194304] 0 2024-05-06T12:14:29.762 INFO:tasks.workunit.client.0.smithi092.stdout:1/193: chown d3/f12 144 1 2024-05-06T12:14:29.765 INFO:tasks.workunit.client.0.smithi092.stdout:7/421: rename d9/de/d2d/c4e to d9/de/d10/d17/c91 0 2024-05-06T12:14:29.765 INFO:tasks.workunit.client.0.smithi092.stdout:7/422: dread - d9/f80 zero size 2024-05-06T12:14:29.767 INFO:tasks.workunit.client.0.smithi092.stdout:1/194: creat d3/d10/f45 x:0 0 0 2024-05-06T12:14:29.767 INFO:tasks.workunit.client.0.smithi092.stdout:2/134: getdents df/d1b 0 2024-05-06T12:14:29.768 INFO:tasks.workunit.client.0.smithi092.stdout:7/423: dread - d9/dd/d23/f7d zero size 2024-05-06T12:14:29.769 INFO:tasks.workunit.client.0.smithi092.stdout:2/135: readlink df/d17/l20 0 2024-05-06T12:14:29.770 INFO:tasks.workunit.client.0.smithi092.stdout:1/195: chown d3/d10/d13/f3b 2 1 2024-05-06T12:14:29.770 INFO:tasks.workunit.client.0.smithi092.stdout:2/136: chown c2 60 1 2024-05-06T12:14:29.773 INFO:tasks.workunit.client.0.smithi092.stdout:2/137: symlink df/d17/l29 0 2024-05-06T12:14:29.777 INFO:tasks.workunit.client.0.smithi092.stdout:7/424: dwrite d9/de/d1f/d37/d48/f8e [0,4194304] 0 2024-05-06T12:14:29.777 INFO:tasks.workunit.client.0.smithi092.stdout:2/138: creat df/d17/f2a x:0 0 0 2024-05-06T12:14:29.778 INFO:tasks.workunit.client.0.smithi092.stdout:2/139: chown df/d17/l29 871512 1 2024-05-06T12:14:29.779 INFO:tasks.workunit.client.0.smithi092.stdout:7/425: chown d9/dd/d15/l7f 1 1 2024-05-06T12:14:29.779 INFO:tasks.workunit.client.0.smithi092.stdout:8/226: dread f0 [0,4194304] 0 2024-05-06T12:14:29.782 INFO:tasks.workunit.client.0.smithi092.stdout:2/140: unlink c0 0 2024-05-06T12:14:29.785 INFO:tasks.workunit.client.0.smithi092.stdout:8/227: dread d9/f35 [0,4194304] 0 2024-05-06T12:14:29.785 INFO:tasks.workunit.client.0.smithi092.stdout:7/426: fdatasync d9/de/d2d/f8a 0 2024-05-06T12:14:29.786 INFO:tasks.workunit.client.0.smithi092.stdout:8/228: readlink d9/dc/l3c 0 2024-05-06T12:14:29.786 INFO:tasks.workunit.client.0.smithi092.stdout:7/427: dread - d9/f80 zero size 2024-05-06T12:14:29.792 INFO:tasks.workunit.client.0.smithi092.stdout:8/229: unlink d9/dc/c3e 0 2024-05-06T12:14:29.793 INFO:tasks.workunit.client.0.smithi092.stdout:2/141: getdents df/d15 0 2024-05-06T12:14:29.795 INFO:tasks.workunit.client.0.smithi092.stdout:7/428: link d9/dd/f43 d9/de/d10/d17/d6a/f92 0 2024-05-06T12:14:29.796 INFO:tasks.workunit.client.0.smithi092.stdout:8/230: mkdir d9/d21/d54 0 2024-05-06T12:14:29.796 INFO:tasks.workunit.client.0.smithi092.stdout:7/429: read - d9/de/d10/d17/d21/d38/f89 zero size 2024-05-06T12:14:29.797 INFO:tasks.workunit.client.0.smithi092.stdout:8/231: chown d9/dc/f2d 4867972 1 2024-05-06T12:14:29.797 INFO:tasks.workunit.client.0.smithi092.stdout:2/142: creat df/d15/f2b x:0 0 0 2024-05-06T12:14:29.798 INFO:tasks.workunit.client.0.smithi092.stdout:7/430: write d9/de/f2a [718582,109775] 0 2024-05-06T12:14:29.798 INFO:tasks.workunit.client.0.smithi092.stdout:8/232: chown d9/d21/d37/l4f 0 1 2024-05-06T12:14:29.799 INFO:tasks.workunit.client.0.smithi092.stdout:8/233: rename d9 to d9/d21/d37/d55 22 2024-05-06T12:14:29.800 INFO:tasks.workunit.client.0.smithi092.stdout:2/143: creat df/f2c x:0 0 0 2024-05-06T12:14:29.800 INFO:tasks.workunit.client.0.smithi092.stdout:2/144: stat df/d17/d1d 0 2024-05-06T12:14:29.854 INFO:tasks.workunit.client.0.smithi092.stdout:8/234: rmdir d9/d21 39 2024-05-06T12:14:29.856 INFO:tasks.workunit.client.0.smithi092.stdout:2/145: truncate df/d17/d1d/f21 10368 0 2024-05-06T12:14:29.857 INFO:tasks.workunit.client.0.smithi092.stdout:8/235: symlink d9/d21/d37/l56 0 2024-05-06T12:14:29.863 INFO:tasks.workunit.client.0.smithi092.stdout:8/236: read d9/dc/f2d [3303224,89722] 0 2024-05-06T12:14:29.863 INFO:tasks.workunit.client.0.smithi092.stdout:2/146: dwrite df/d15/f2b [0,4194304] 0 2024-05-06T12:14:29.868 INFO:tasks.workunit.client.0.smithi092.stdout:2/147: rename df/d17/d1d/l27 to df/d15/l2d 0 2024-05-06T12:14:29.869 INFO:tasks.workunit.client.0.smithi092.stdout:8/237: symlink d9/dc/l57 0 2024-05-06T12:14:29.873 INFO:tasks.workunit.client.0.smithi092.stdout:2/148: getdents df 0 2024-05-06T12:14:29.874 INFO:tasks.workunit.client.0.smithi092.stdout:8/238: getdents d9/d21/d37 0 2024-05-06T12:14:29.880 INFO:tasks.workunit.client.0.smithi092.stdout:2/149: rename df/d17/f18 to df/d17/d1d/f2e 0 2024-05-06T12:14:29.883 INFO:tasks.workunit.client.0.smithi092.stdout:2/150: rmdir df/d17/d1d 39 2024-05-06T12:14:29.883 INFO:tasks.workunit.client.0.smithi092.stdout:8/239: write d9/dc/f2d [642633,84642] 0 2024-05-06T12:14:29.886 INFO:tasks.workunit.client.0.smithi092.stdout:8/240: unlink d9/f34 0 2024-05-06T12:14:29.887 INFO:tasks.workunit.client.0.smithi092.stdout:2/151: symlink df/d17/d1d/l2f 0 2024-05-06T12:14:29.889 INFO:tasks.workunit.client.0.smithi092.stdout:2/152: rename df/d15/f2b to df/d15/f30 0 2024-05-06T12:14:29.890 INFO:tasks.workunit.client.0.smithi092.stdout:8/241: link d9/f33 d9/d21/f58 0 2024-05-06T12:14:29.892 INFO:tasks.workunit.client.0.smithi092.stdout:8/242: creat d9/d21/d37/f59 x:0 0 0 2024-05-06T12:14:29.992 INFO:tasks.workunit.client.0.smithi092.stdout:0/243: sync 2024-05-06T12:14:29.992 INFO:tasks.workunit.client.0.smithi092.stdout:5/237: sync 2024-05-06T12:14:29.993 INFO:tasks.workunit.client.0.smithi092.stdout:5/238: chown d4/d5/c15 16712988 1 2024-05-06T12:14:29.993 INFO:tasks.workunit.client.0.smithi092.stdout:5/239: stat d4/d34/d28/d4e/d5d/l45 0 2024-05-06T12:14:29.994 INFO:tasks.workunit.client.0.smithi092.stdout:5/240: chown d4/d59 110284032 1 2024-05-06T12:14:29.995 INFO:tasks.workunit.client.0.smithi092.stdout:0/244: mkdir d7/dc/d28/d54/d1e/d56 0 2024-05-06T12:14:29.995 INFO:tasks.workunit.client.0.smithi092.stdout:0/245: fdatasync d7/f9 0 2024-05-06T12:14:29.996 INFO:tasks.workunit.client.0.smithi092.stdout:5/241: creat d4/d5/d10/d12/f63 x:0 0 0 2024-05-06T12:14:29.997 INFO:tasks.workunit.client.0.smithi092.stdout:5/242: chown d4/d5/d10/d31/c5f 53494 1 2024-05-06T12:14:30.002 INFO:tasks.workunit.client.0.smithi092.stdout:5/243: rename d4/d5/d10/d12/d1f/d20/d23 to d4/d5/d10/d31/d53/d64 0 2024-05-06T12:14:30.002 INFO:tasks.workunit.client.0.smithi092.stdout:0/246: dwrite d7/dc/d28/d54/d1e/f22 [0,4194304] 0 2024-05-06T12:14:30.004 INFO:tasks.workunit.client.0.smithi092.stdout:0/247: rmdir d7/dc/d28/d54/d1e/d26/d3a 39 2024-05-06T12:14:30.151 INFO:tasks.workunit.client.0.smithi092.stdout:3/214: sync 2024-05-06T12:14:30.165 INFO:tasks.workunit.client.0.smithi092.stdout:3/215: dread d8/d24/d33/d39/f44 [0,4194304] 0 2024-05-06T12:14:30.166 INFO:tasks.workunit.client.0.smithi092.stdout:3/216: creat d8/d9/f46 x:0 0 0 2024-05-06T12:14:30.167 INFO:tasks.workunit.client.0.smithi092.stdout:3/217: chown d8/d24/f31 7 1 2024-05-06T12:14:30.170 INFO:tasks.workunit.client.0.smithi092.stdout:3/218: link d8/d9/c1e d8/d24/d27/d3a/d3d/c47 0 2024-05-06T12:14:30.227 INFO:tasks.workunit.client.0.smithi092.stdout:4/186: sync 2024-05-06T12:14:30.228 INFO:tasks.workunit.client.0.smithi092.stdout:4/187: readlink d9/df/l3e 0 2024-05-06T12:14:30.230 INFO:tasks.workunit.client.0.smithi092.stdout:4/188: getdents d9/d27 0 2024-05-06T12:14:30.233 INFO:tasks.workunit.client.0.smithi092.stdout:4/189: write d9/dd/f16 [4001865,100983] 0 2024-05-06T12:14:30.234 INFO:tasks.workunit.client.0.smithi092.stdout:4/190: truncate d9/dd/f3b 114175 0 2024-05-06T12:14:30.242 INFO:tasks.workunit.client.0.smithi092.stdout:4/191: dwrite d9/fc [4194304,4194304] 0 2024-05-06T12:14:30.242 INFO:tasks.workunit.client.0.smithi092.stdout:4/192: fdatasync f5 0 2024-05-06T12:14:30.250 INFO:tasks.workunit.client.0.smithi092.stdout:4/193: getdents d9/dd 0 2024-05-06T12:14:30.250 INFO:tasks.workunit.client.0.smithi092.stdout:4/194: write d9/df/f14 [4465780,106264] 0 2024-05-06T12:14:30.250 INFO:tasks.workunit.client.0.smithi092.stdout:4/195: symlink d9/l40 0 2024-05-06T12:14:30.251 INFO:tasks.workunit.client.0.smithi092.stdout:4/196: creat d9/d27/f41 x:0 0 0 2024-05-06T12:14:30.253 INFO:tasks.workunit.client.0.smithi092.stdout:4/197: mkdir d9/d2b/d37/d42 0 2024-05-06T12:14:30.258 INFO:tasks.workunit.client.0.smithi092.stdout:4/198: dwrite d9/d27/f31 [0,4194304] 0 2024-05-06T12:14:30.258 INFO:tasks.workunit.client.0.smithi092.stdout:4/199: stat c6 0 2024-05-06T12:14:30.260 INFO:tasks.workunit.client.0.smithi092.stdout:4/200: unlink d9/df/c2d 0 2024-05-06T12:14:30.262 INFO:tasks.workunit.client.0.smithi092.stdout:3/219: sync 2024-05-06T12:14:30.262 INFO:tasks.workunit.client.0.smithi092.stdout:2/153: sync 2024-05-06T12:14:30.263 INFO:tasks.workunit.client.0.smithi092.stdout:0/248: sync 2024-05-06T12:14:30.263 INFO:tasks.workunit.client.0.smithi092.stdout:9/172: sync 2024-05-06T12:14:30.263 INFO:tasks.workunit.client.0.smithi092.stdout:6/266: sync 2024-05-06T12:14:30.263 INFO:tasks.workunit.client.0.smithi092.stdout:5/244: sync 2024-05-06T12:14:30.263 INFO:tasks.workunit.client.0.smithi092.stdout:8/243: sync 2024-05-06T12:14:30.263 INFO:tasks.workunit.client.0.smithi092.stdout:7/431: sync 2024-05-06T12:14:30.263 INFO:tasks.workunit.client.0.smithi092.stdout:1/196: sync 2024-05-06T12:14:30.264 INFO:tasks.workunit.client.0.smithi092.stdout:6/267: fsync d4/f1a 0 2024-05-06T12:14:30.264 INFO:tasks.workunit.client.0.smithi092.stdout:4/201: link d9/f24 d9/f43 0 2024-05-06T12:14:30.265 INFO:tasks.workunit.client.0.smithi092.stdout:3/220: write d8/d24/f31 [395565,3333] 0 2024-05-06T12:14:30.265 INFO:tasks.workunit.client.0.smithi092.stdout:2/154: chown df/f11 25 1 2024-05-06T12:14:30.266 INFO:tasks.workunit.client.0.smithi092.stdout:7/432: fsync f3 0 2024-05-06T12:14:30.268 INFO:tasks.workunit.client.0.smithi092.stdout:6/268: rename d4/l15 to d4/d20/d22/l5a 0 2024-05-06T12:14:30.270 INFO:tasks.workunit.client.0.smithi092.stdout:3/221: unlink d8/d24/f31 0 2024-05-06T12:14:30.271 INFO:tasks.workunit.client.0.smithi092.stdout:9/173: mkdir d16/d1c/d32/d33 0 2024-05-06T12:14:30.273 INFO:tasks.workunit.client.0.smithi092.stdout:3/222: write d8/d24/d27/f2f [4165994,34732] 0 2024-05-06T12:14:30.275 INFO:tasks.workunit.client.0.smithi092.stdout:9/174: symlink d16/d2c/d2f/l34 0 2024-05-06T12:14:30.278 INFO:tasks.workunit.client.0.smithi092.stdout:1/197: read - d3/d10/d13/f3b zero size 2024-05-06T12:14:30.278 INFO:tasks.workunit.client.0.smithi092.stdout:3/223: fsync d8/fa 0 2024-05-06T12:14:30.278 INFO:tasks.workunit.client.0.smithi092.stdout:9/175: chown d16/d1e/f2e 149 1 2024-05-06T12:14:30.278 INFO:tasks.workunit.client.0.smithi092.stdout:0/249: stat d7/dc/d28/d54/l44 0 2024-05-06T12:14:30.279 INFO:tasks.workunit.client.0.smithi092.stdout:2/155: getdents df/d17/d1d 0 2024-05-06T12:14:30.280 INFO:tasks.workunit.client.0.smithi092.stdout:6/269: dwrite d4/d4c/d52/f4a [4194304,4194304] 0 2024-05-06T12:14:30.282 INFO:tasks.workunit.client.0.smithi092.stdout:5/245: mknod d4/d34/d28/d4e/d5d/c65 0 2024-05-06T12:14:30.283 INFO:tasks.workunit.client.0.smithi092.stdout:0/250: stat d7/dc/d28/d54/d12 0 2024-05-06T12:14:30.283 INFO:tasks.workunit.client.0.smithi092.stdout:1/198: write d3/d10/d13/d19/d21/f22 [2686954,40056] 0 2024-05-06T12:14:30.288 INFO:tasks.workunit.client.0.smithi092.stdout:7/433: mkdir d9/de/d1f/d37/d48/d93 0 2024-05-06T12:14:30.288 INFO:tasks.workunit.client.0.smithi092.stdout:9/176: creat d16/d2c/f35 x:0 0 0 2024-05-06T12:14:30.289 INFO:tasks.workunit.client.0.smithi092.stdout:7/434: fdatasync d9/de/d10/f27 0 2024-05-06T12:14:30.289 INFO:tasks.workunit.client.0.smithi092.stdout:8/244: mkdir d9/d5a 0 2024-05-06T12:14:30.291 INFO:tasks.workunit.client.0.smithi092.stdout:9/177: dread - d16/d1e/f2e zero size 2024-05-06T12:14:30.296 INFO:tasks.workunit.client.0.smithi092.stdout:6/270: dwrite d4/d20/d23/f46 [0,4194304] 0 2024-05-06T12:14:30.297 INFO:tasks.workunit.client.0.smithi092.stdout:0/251: creat d7/dc/d28/d54/d1e/d39/f57 x:0 0 0 2024-05-06T12:14:30.298 INFO:tasks.workunit.client.0.smithi092.stdout:5/246: symlink d4/d34/d28/d4e/l66 0 2024-05-06T12:14:30.299 INFO:tasks.workunit.client.0.smithi092.stdout:9/178: dread d16/f1b [0,4194304] 0 2024-05-06T12:14:30.302 INFO:tasks.workunit.client.0.smithi092.stdout:3/224: mkdir d8/d24/d27/d3a/d3d/d48 0 2024-05-06T12:14:30.304 INFO:tasks.workunit.client.0.smithi092.stdout:1/199: dwrite d3/d10/d13/f14 [0,4194304] 0 2024-05-06T12:14:30.304 INFO:tasks.workunit.client.0.smithi092.stdout:7/435: mknod d9/dd/d23/c94 0 2024-05-06T12:14:30.306 INFO:tasks.workunit.client.0.smithi092.stdout:6/271: write d4/d20/f1c [4578075,35283] 0 2024-05-06T12:14:30.306 INFO:tasks.workunit.client.0.smithi092.stdout:8/245: dread f6 [0,4194304] 0 2024-05-06T12:14:30.310 INFO:tasks.workunit.client.0.smithi092.stdout:9/179: read d16/d1e/f26 [4024536,70225] 0 2024-05-06T12:14:30.310 INFO:tasks.workunit.client.0.smithi092.stdout:4/202: dread d9/f43 [0,4194304] 0 2024-05-06T12:14:30.310 INFO:tasks.workunit.client.0.smithi092.stdout:5/247: mkdir d4/d34/d28/d4e/d67 0 2024-05-06T12:14:30.311 INFO:tasks.workunit.client.0.smithi092.stdout:6/272: readlink d4/l10 0 2024-05-06T12:14:30.312 INFO:tasks.workunit.client.0.smithi092.stdout:8/246: fsync d9/dc/f16 0 2024-05-06T12:14:30.314 INFO:tasks.workunit.client.0.smithi092.stdout:0/252: mknod d7/dc/d28/d54/d12/c58 0 2024-05-06T12:14:30.315 INFO:tasks.workunit.client.0.smithi092.stdout:1/200: rmdir d3/d10/d13/d19/d21 39 2024-05-06T12:14:30.321 INFO:tasks.workunit.client.0.smithi092.stdout:9/180: rename d16/d1e/f26 to d16/d2c/d2f/f36 0 2024-05-06T12:14:30.324 INFO:tasks.workunit.client.0.smithi092.stdout:2/156: dwrite df/d17/d1d/f21 [0,4194304] 0 2024-05-06T12:14:30.324 INFO:tasks.workunit.client.0.smithi092.stdout:3/225: dwrite d8/d9/f46 [0,4194304] 0 2024-05-06T12:14:30.326 INFO:tasks.workunit.client.0.smithi092.stdout:9/181: write f6 [3163560,99786] 0 2024-05-06T12:14:30.328 INFO:tasks.workunit.client.0.smithi092.stdout:4/203: dwrite d9/dd/f16 [0,4194304] 0 2024-05-06T12:14:30.331 INFO:tasks.workunit.client.0.smithi092.stdout:6/273: unlink d4/cd 0 2024-05-06T12:14:30.332 INFO:tasks.workunit.client.0.smithi092.stdout:8/247: symlink d9/l5b 0 2024-05-06T12:14:30.335 INFO:tasks.workunit.client.0.smithi092.stdout:2/157: dwrite df/f2c [0,4194304] 0 2024-05-06T12:14:30.336 INFO:tasks.workunit.client.0.smithi092.stdout:3/226: stat d8/d24/c21 0 2024-05-06T12:14:30.336 INFO:tasks.workunit.client.0.smithi092.stdout:2/158: write f6 [508412,4549] 0 2024-05-06T12:14:30.337 INFO:tasks.workunit.client.0.smithi092.stdout:3/227: stat d8/f45 0 2024-05-06T12:14:30.339 INFO:tasks.workunit.client.0.smithi092.stdout:0/253: symlink d7/l59 0 2024-05-06T12:14:30.340 INFO:tasks.workunit.client.0.smithi092.stdout:2/159: read df/f1f [723031,46526] 0 2024-05-06T12:14:30.341 INFO:tasks.workunit.client.0.smithi092.stdout:1/201: mkdir d3/d10/d13/d46 0 2024-05-06T12:14:30.342 INFO:tasks.workunit.client.0.smithi092.stdout:4/204: dwrite d9/f34 [0,4194304] 0 2024-05-06T12:14:30.343 INFO:tasks.workunit.client.0.smithi092.stdout:7/436: link d9/dd/d23/l33 d9/de/d10/d17/d21/d2e/l95 0 2024-05-06T12:14:30.344 INFO:tasks.workunit.client.0.smithi092.stdout:4/205: write d9/f38 [3777999,107435] 0 2024-05-06T12:14:30.345 INFO:tasks.workunit.client.0.smithi092.stdout:8/248: mkdir d9/d21/d37/d46/d5c 0 2024-05-06T12:14:30.347 INFO:tasks.workunit.client.0.smithi092.stdout:4/206: write d9/dd/f17 [3702745,61044] 0 2024-05-06T12:14:30.348 INFO:tasks.workunit.client.0.smithi092.stdout:2/160: symlink df/d1b/l31 0 2024-05-06T12:14:30.351 INFO:tasks.workunit.client.0.smithi092.stdout:0/254: rename d7/dc/d28/d54/c37 to d7/d43/c5a 0 2024-05-06T12:14:30.353 INFO:tasks.workunit.client.0.smithi092.stdout:1/202: dread d3/d10/d13/d19/f3a [0,4194304] 0 2024-05-06T12:14:30.357 INFO:tasks.workunit.client.0.smithi092.stdout:6/274: creat d4/f5b x:0 0 0 2024-05-06T12:14:30.360 INFO:tasks.workunit.client.0.smithi092.stdout:4/207: dwrite d9/d2b/f32 [4194304,4194304] 0 2024-05-06T12:14:30.360 INFO:tasks.workunit.client.0.smithi092.stdout:0/255: symlink d7/dc/d28/d54/d1e/d39/l5b 0 2024-05-06T12:14:30.361 INFO:tasks.workunit.client.0.smithi092.stdout:4/208: write d9/df/f15 [1040627,21589] 0 2024-05-06T12:14:30.363 INFO:tasks.workunit.client.0.smithi092.stdout:4/209: read - d9/d2b/f39 zero size 2024-05-06T12:14:30.366 INFO:tasks.workunit.client.0.smithi092.stdout:6/275: chown d4/d20/c43 114351007 1 2024-05-06T12:14:30.367 INFO:tasks.workunit.client.0.smithi092.stdout:2/161: dread df/d17/f19 [0,4194304] 0 2024-05-06T12:14:30.367 INFO:tasks.workunit.client.0.smithi092.stdout:2/162: chown df/c14 47228 1 2024-05-06T12:14:30.368 INFO:tasks.workunit.client.0.smithi092.stdout:2/163: fdatasync df/d17/f22 0 2024-05-06T12:14:30.368 INFO:tasks.workunit.client.0.smithi092.stdout:2/164: chown df/d15 1413 1 2024-05-06T12:14:30.369 INFO:tasks.workunit.client.0.smithi092.stdout:1/203: mkdir d3/d10/d13/d19/d21/d47 0 2024-05-06T12:14:30.370 INFO:tasks.workunit.client.0.smithi092.stdout:1/204: fsync d3/d10/d13/f1f 0 2024-05-06T12:14:30.371 INFO:tasks.workunit.client.0.smithi092.stdout:8/249: creat d9/dc/f5d x:0 0 0 2024-05-06T12:14:30.372 INFO:tasks.workunit.client.0.smithi092.stdout:4/210: creat d9/d2b/d37/f44 x:0 0 0 2024-05-06T12:14:30.373 INFO:tasks.workunit.client.0.smithi092.stdout:6/276: unlink d4/d20/d23/c2d 0 2024-05-06T12:14:30.373 INFO:tasks.workunit.client.0.smithi092.stdout:4/211: fsync d9/f21 0 2024-05-06T12:14:30.378 INFO:tasks.workunit.client.0.smithi092.stdout:2/165: dwrite f1 [0,4194304] 0 2024-05-06T12:14:30.378 INFO:tasks.workunit.client.0.smithi092.stdout:7/437: dread d9/de/f2a [0,4194304] 0 2024-05-06T12:14:30.380 INFO:tasks.workunit.client.0.smithi092.stdout:8/250: creat d9/d21/d37/d46/f5e x:0 0 0 2024-05-06T12:14:30.383 INFO:tasks.workunit.client.0.smithi092.stdout:6/277: creat d4/d4c/f5c x:0 0 0 2024-05-06T12:14:30.386 INFO:tasks.workunit.client.0.smithi092.stdout:2/166: dread df/d17/d1d/f21 [0,4194304] 0 2024-05-06T12:14:30.386 INFO:tasks.workunit.client.0.smithi092.stdout:5/248: truncate d4/d34/f56 2188129 0 2024-05-06T12:14:30.389 INFO:tasks.workunit.client.0.smithi092.stdout:5/249: chown d4/d5/d10/d12/f18 0 1 2024-05-06T12:14:30.390 INFO:tasks.workunit.client.0.smithi092.stdout:8/251: dwrite d9/d21/d37/d46/f5e [0,4194304] 0 2024-05-06T12:14:30.390 INFO:tasks.workunit.client.0.smithi092.stdout:4/212: link d9/df/l1b d9/d2b/d37/d42/l45 0 2024-05-06T12:14:30.391 INFO:tasks.workunit.client.0.smithi092.stdout:4/213: read f5 [1059648,55679] 0 2024-05-06T12:14:30.392 INFO:tasks.workunit.client.0.smithi092.stdout:2/167: symlink df/d1b/l32 0 2024-05-06T12:14:30.394 INFO:tasks.workunit.client.0.smithi092.stdout:5/250: creat d4/f68 x:0 0 0 2024-05-06T12:14:30.396 INFO:tasks.workunit.client.0.smithi092.stdout:5/251: write d4/d5/d30/f46 [4778074,63504] 0 2024-05-06T12:14:30.397 INFO:tasks.workunit.client.0.smithi092.stdout:4/214: creat d9/df/f46 x:0 0 0 2024-05-06T12:14:30.398 INFO:tasks.workunit.client.0.smithi092.stdout:2/168: mkdir df/d17/d1d/d33 0 2024-05-06T12:14:30.402 INFO:tasks.workunit.client.0.smithi092.stdout:5/252: fsync d4/d34/d29/d54/f62 0 2024-05-06T12:14:30.404 INFO:tasks.workunit.client.0.smithi092.stdout:4/215: dwrite d9/dd/f3b [0,4194304] 0 2024-05-06T12:14:30.404 INFO:tasks.workunit.client.0.smithi092.stdout:3/228: truncate d8/db/d14/f18 2522857 0 2024-05-06T12:14:30.406 INFO:tasks.workunit.client.0.smithi092.stdout:3/229: stat d8/db/l2d 0 2024-05-06T12:14:30.406 INFO:tasks.workunit.client.0.smithi092.stdout:8/252: mkdir d9/dc/d4a/d5f 0 2024-05-06T12:14:30.406 INFO:tasks.workunit.client.0.smithi092.stdout:0/256: rmdir d7 39 2024-05-06T12:14:30.410 INFO:tasks.workunit.client.0.smithi092.stdout:2/169: dwrite df/d17/d1d/f2e [0,4194304] 0 2024-05-06T12:14:30.411 INFO:tasks.workunit.client.0.smithi092.stdout:4/216: stat d9/c33 0 2024-05-06T12:14:30.412 INFO:tasks.workunit.client.0.smithi092.stdout:4/217: write d9/d27/f41 [152580,52264] 0 2024-05-06T12:14:30.417 INFO:tasks.workunit.client.0.smithi092.stdout:3/230: rmdir d8/d24/d27/d3a/d3d 39 2024-05-06T12:14:30.418 INFO:tasks.workunit.client.0.smithi092.stdout:3/231: write d8/f12 [3754546,35904] 0 2024-05-06T12:14:30.420 INFO:tasks.workunit.client.0.smithi092.stdout:1/205: write d3/fa [212622,104212] 0 2024-05-06T12:14:30.420 INFO:tasks.workunit.client.0.smithi092.stdout:8/253: mknod d9/c60 0 2024-05-06T12:14:30.422 INFO:tasks.workunit.client.0.smithi092.stdout:4/218: fdatasync d9/f1c 0 2024-05-06T12:14:30.429 INFO:tasks.workunit.client.0.smithi092.stdout:8/254: dread d9/f36 [0,4194304] 0 2024-05-06T12:14:30.430 INFO:tasks.workunit.client.0.smithi092.stdout:0/257: dwrite d7/dc/d28/d54/d1e/d39/f47 [4194304,4194304] 0 2024-05-06T12:14:30.431 INFO:tasks.workunit.client.0.smithi092.stdout:5/253: link d4/d5/c38 d4/d5/d10/d31/c69 0 2024-05-06T12:14:30.434 INFO:tasks.workunit.client.0.smithi092.stdout:2/170: dwrite fd [0,4194304] 0 2024-05-06T12:14:30.436 INFO:tasks.workunit.client.0.smithi092.stdout:0/258: chown d7/dc/d28/d54/d1e/d26/l4b 161274591 1 2024-05-06T12:14:30.436 INFO:tasks.workunit.client.0.smithi092.stdout:5/254: dread - d4/d5/d30/f3c zero size 2024-05-06T12:14:30.438 INFO:tasks.workunit.client.0.smithi092.stdout:0/259: truncate d7/d43/f49 400995 0 2024-05-06T12:14:30.439 INFO:tasks.workunit.client.0.smithi092.stdout:5/255: readlink d4/d5/d10/d31/l3d 0 2024-05-06T12:14:30.441 INFO:tasks.workunit.client.0.smithi092.stdout:4/219: dread d9/fc [4194304,4194304] 0 2024-05-06T12:14:30.442 INFO:tasks.workunit.client.0.smithi092.stdout:5/256: write d4/d5/d10/d12/d37/f51 [283569,96497] 0 2024-05-06T12:14:30.442 INFO:tasks.workunit.client.0.smithi092.stdout:4/220: write d9/d2b/f32 [4559359,36201] 0 2024-05-06T12:14:30.443 INFO:tasks.workunit.client.0.smithi092.stdout:3/232: rename d8/d24/d33/d39/d43 to d8/d24/d27/d3a/d3d/d49 0 2024-05-06T12:14:30.445 INFO:tasks.workunit.client.0.smithi092.stdout:8/255: getdents d9/d5a 0 2024-05-06T12:14:30.448 INFO:tasks.workunit.client.0.smithi092.stdout:2/171: mknod df/d17/c34 0 2024-05-06T12:14:30.449 INFO:tasks.workunit.client.0.smithi092.stdout:0/260: dwrite f6 [0,4194304] 0 2024-05-06T12:14:30.464 INFO:tasks.workunit.client.0.smithi092.stdout:2/172: creat df/d17/d1d/d33/f35 x:0 0 0 2024-05-06T12:14:30.465 INFO:tasks.workunit.client.0.smithi092.stdout:2/173: fsync fe 0 2024-05-06T12:14:30.465 INFO:tasks.workunit.client.0.smithi092.stdout:3/233: creat d8/d24/d27/f4a x:0 0 0 2024-05-06T12:14:30.468 INFO:tasks.workunit.client.0.smithi092.stdout:4/221: rename d9/dd/f3b to d9/f47 0 2024-05-06T12:14:30.472 INFO:tasks.workunit.client.0.smithi092.stdout:8/256: link d9/c20 d9/d21/d54/c61 0 2024-05-06T12:14:30.472 INFO:tasks.workunit.client.0.smithi092.stdout:2/174: creat df/d15/f36 x:0 0 0 2024-05-06T12:14:30.473 INFO:tasks.workunit.client.0.smithi092.stdout:3/234: truncate d8/f45 280644 0 2024-05-06T12:14:30.476 INFO:tasks.workunit.client.0.smithi092.stdout:0/261: dwrite f1 [0,4194304] 0 2024-05-06T12:14:30.476 INFO:tasks.workunit.client.0.smithi092.stdout:3/235: dread - d8/d24/d27/f4a zero size 2024-05-06T12:14:30.477 INFO:tasks.workunit.client.0.smithi092.stdout:4/222: dread d9/df/f14 [0,4194304] 0 2024-05-06T12:14:30.478 INFO:tasks.workunit.client.0.smithi092.stdout:3/236: chown d8/db/d14 399345 1 2024-05-06T12:14:30.479 INFO:tasks.workunit.client.0.smithi092.stdout:0/262: write d7/dc/d28/d54/d1e/d26/f2b [5149693,50950] 0 2024-05-06T12:14:30.479 INFO:tasks.workunit.client.0.smithi092.stdout:5/257: getdents d4 0 2024-05-06T12:14:30.480 INFO:tasks.workunit.client.0.smithi092.stdout:3/237: readlink d8/d9/l3b 0 2024-05-06T12:14:30.481 INFO:tasks.workunit.client.0.smithi092.stdout:8/257: chown d9/l15 253 1 2024-05-06T12:14:30.482 INFO:tasks.workunit.client.0.smithi092.stdout:2/175: rename df/d17/f19 to df/f37 0 2024-05-06T12:14:30.486 INFO:tasks.workunit.client.0.smithi092.stdout:0/263: dwrite d7/dc/f29 [0,4194304] 0 2024-05-06T12:14:30.487 INFO:tasks.workunit.client.0.smithi092.stdout:5/258: mknod d4/d34/d28/c6a 0 2024-05-06T12:14:30.488 INFO:tasks.workunit.client.0.smithi092.stdout:5/259: fdatasync d4/d5/d30/f46 0 2024-05-06T12:14:30.492 INFO:tasks.workunit.client.0.smithi092.stdout:3/238: rmdir d8/d24/d33/d39 39 2024-05-06T12:14:30.494 INFO:tasks.workunit.client.0.smithi092.stdout:2/176: mknod df/c38 0 2024-05-06T12:14:30.495 INFO:tasks.workunit.client.0.smithi092.stdout:0/264: dwrite d7/dc/d28/f2e [0,4194304] 0 2024-05-06T12:14:30.496 INFO:tasks.workunit.client.0.smithi092.stdout:4/223: rename d9/d2b/d37/d42 to d9/df/d48 0 2024-05-06T12:14:30.497 INFO:tasks.workunit.client.0.smithi092.stdout:4/224: chown d9/df/c26 2 1 2024-05-06T12:14:30.501 INFO:tasks.workunit.client.0.smithi092.stdout:5/260: mkdir d4/d34/d28/d6b 0 2024-05-06T12:14:30.506 INFO:tasks.workunit.client.0.smithi092.stdout:4/225: creat d9/d27/f49 x:0 0 0 2024-05-06T12:14:30.508 INFO:tasks.workunit.client.0.smithi092.stdout:4/226: read - d9/d2b/f39 zero size 2024-05-06T12:14:30.508 INFO:tasks.workunit.client.0.smithi092.stdout:8/258: dwrite d9/f36 [0,4194304] 0 2024-05-06T12:14:30.508 INFO:tasks.workunit.client.0.smithi092.stdout:5/261: rename d4/d5/d10/d12/d37 to d4/d34/d29/d6c 0 2024-05-06T12:14:30.509 INFO:tasks.workunit.client.0.smithi092.stdout:5/262: chown d4/f4c 86 1 2024-05-06T12:14:30.511 INFO:tasks.workunit.client.0.smithi092.stdout:2/177: link fe df/f39 0 2024-05-06T12:14:30.512 INFO:tasks.workunit.client.0.smithi092.stdout:2/178: dread - df/d17/f2a zero size 2024-05-06T12:14:30.514 INFO:tasks.workunit.client.0.smithi092.stdout:8/259: rmdir d9/d21 39 2024-05-06T12:14:30.515 INFO:tasks.workunit.client.0.smithi092.stdout:8/260: write d9/f18 [787131,31515] 0 2024-05-06T12:14:30.517 INFO:tasks.workunit.client.0.smithi092.stdout:4/227: rename d9/d2b/d37 to d9/d27/d4a 0 2024-05-06T12:14:30.519 INFO:tasks.workunit.client.0.smithi092.stdout:9/182: sync 2024-05-06T12:14:30.521 INFO:tasks.workunit.client.0.smithi092.stdout:5/263: rmdir d4/d34 39 2024-05-06T12:14:30.522 INFO:tasks.workunit.client.0.smithi092.stdout:2/179: symlink df/l3a 0 2024-05-06T12:14:30.523 INFO:tasks.workunit.client.0.smithi092.stdout:5/264: truncate d4/d5/d10/d31/d53/d64/f5a 501870 0 2024-05-06T12:14:30.525 INFO:tasks.workunit.client.0.smithi092.stdout:0/265: truncate d7/dc/d28/d54/f11 3945618 0 2024-05-06T12:14:30.528 INFO:tasks.workunit.client.0.smithi092.stdout:3/239: dread d8/f45 [0,4194304] 0 2024-05-06T12:14:30.533 INFO:tasks.workunit.client.0.smithi092.stdout:9/183: mkdir d16/d1c/d32/d33/d37 0 2024-05-06T12:14:30.535 INFO:tasks.workunit.client.0.smithi092.stdout:4/228: dwrite d9/f22 [0,4194304] 0 2024-05-06T12:14:30.538 INFO:tasks.workunit.client.0.smithi092.stdout:2/180: truncate df/d17/f22 198742 0 2024-05-06T12:14:30.544 INFO:tasks.workunit.client.0.smithi092.stdout:5/265: dwrite d4/d34/d29/d6c/f44 [0,4194304] 0 2024-05-06T12:14:30.546 INFO:tasks.workunit.client.0.smithi092.stdout:4/229: readlink d9/df/l1a 0 2024-05-06T12:14:30.547 INFO:tasks.workunit.client.0.smithi092.stdout:0/266: truncate d7/dc/f29 427108 0 2024-05-06T12:14:30.548 INFO:tasks.workunit.client.0.smithi092.stdout:5/266: write d4/f68 [118467,68588] 0 2024-05-06T12:14:30.550 INFO:tasks.workunit.client.0.smithi092.stdout:9/184: dread d16/d1c/f2b [0,4194304] 0 2024-05-06T12:14:30.551 INFO:tasks.workunit.client.0.smithi092.stdout:8/261: dwrite d9/dc/f53 [0,4194304] 0 2024-05-06T12:14:30.551 INFO:tasks.workunit.client.0.smithi092.stdout:5/267: readlink d4/d34/l11 0 2024-05-06T12:14:30.551 INFO:tasks.workunit.client.0.smithi092.stdout:3/240: rename d8/d9 to d8/d24/d33/d4b 0 2024-05-06T12:14:30.553 INFO:tasks.workunit.client.0.smithi092.stdout:2/181: truncate df/f1f 1479876 0 2024-05-06T12:14:30.555 INFO:tasks.workunit.client.0.smithi092.stdout:8/262: truncate d9/fe 4327152 0 2024-05-06T12:14:30.556 INFO:tasks.workunit.client.0.smithi092.stdout:3/241: write d8/f12 [380417,122143] 0 2024-05-06T12:14:30.559 INFO:tasks.workunit.client.0.smithi092.stdout:2/182: readlink df/d15/l2d 0 2024-05-06T12:14:30.561 INFO:tasks.workunit.client.0.smithi092.stdout:2/183: write df/f12 [799986,65943] 0 2024-05-06T12:14:30.561 INFO:tasks.workunit.client.0.smithi092.stdout:0/267: dwrite d7/f4f [0,4194304] 0 2024-05-06T12:14:30.561 INFO:tasks.workunit.client.0.smithi092.stdout:5/268: rename d4/d5/d30/c35 to d4/d34/d28/c6d 0 2024-05-06T12:14:30.566 INFO:tasks.workunit.client.0.smithi092.stdout:8/263: write d9/f42 [1989905,29544] 0 2024-05-06T12:14:30.568 INFO:tasks.workunit.client.0.smithi092.stdout:2/184: symlink df/d15/l3b 0 2024-05-06T12:14:30.571 INFO:tasks.workunit.client.0.smithi092.stdout:4/230: getdents d9/df/d48 0 2024-05-06T12:14:30.572 INFO:tasks.workunit.client.0.smithi092.stdout:9/185: getdents d16/d1e 0 2024-05-06T12:14:30.572 INFO:tasks.workunit.client.0.smithi092.stdout:9/186: readlink l15 0 2024-05-06T12:14:30.574 INFO:tasks.workunit.client.0.smithi092.stdout:3/242: rename d8/d24/d27/d3a to d8/db/d4c 0 2024-05-06T12:14:30.576 INFO:tasks.workunit.client.0.smithi092.stdout:4/231: write d9/df/f14 [1683293,58717] 0 2024-05-06T12:14:30.577 INFO:tasks.workunit.client.0.smithi092.stdout:8/264: mkdir d9/d21/d37/d46/d5c/d62 0 2024-05-06T12:14:30.579 INFO:tasks.workunit.client.0.smithi092.stdout:3/243: fdatasync d8/d1b/f2b 0 2024-05-06T12:14:30.580 INFO:tasks.workunit.client.0.smithi092.stdout:9/187: dwrite d16/d2c/f35 [0,4194304] 0 2024-05-06T12:14:30.582 INFO:tasks.workunit.client.0.smithi092.stdout:2/185: getdents df 0 2024-05-06T12:14:30.584 INFO:tasks.workunit.client.0.smithi092.stdout:4/232: rename d9/l40 to d9/d2b/l4b 0 2024-05-06T12:14:30.586 INFO:tasks.workunit.client.0.smithi092.stdout:0/268: write d7/d43/d1f/f4a [3068835,119609] 0 2024-05-06T12:14:30.587 INFO:tasks.workunit.client.0.smithi092.stdout:9/188: creat d16/d1c/d32/f38 x:0 0 0 2024-05-06T12:14:30.588 INFO:tasks.workunit.client.0.smithi092.stdout:9/189: fsync d16/d1c/f27 0 2024-05-06T12:14:30.588 INFO:tasks.workunit.client.0.smithi092.stdout:2/186: fsync df/d17/d1d/f21 0 2024-05-06T12:14:30.590 INFO:tasks.workunit.client.0.smithi092.stdout:8/265: creat d9/d21/d54/f63 x:0 0 0 2024-05-06T12:14:30.595 INFO:tasks.workunit.client.0.smithi092.stdout:0/269: creat d7/dc/f5c x:0 0 0 2024-05-06T12:14:30.597 INFO:tasks.workunit.client.0.smithi092.stdout:3/244: creat d8/db/f4d x:0 0 0 2024-05-06T12:14:30.598 INFO:tasks.workunit.client.0.smithi092.stdout:8/266: write d9/f35 [3404655,25887] 0 2024-05-06T12:14:30.600 INFO:tasks.workunit.client.0.smithi092.stdout:0/270: mkdir d7/dc/d28/d54/d1e/d5d 0 2024-05-06T12:14:30.600 INFO:tasks.workunit.client.0.smithi092.stdout:9/190: dwrite d16/d1e/f28 [0,4194304] 0 2024-05-06T12:14:30.601 INFO:tasks.workunit.client.0.smithi092.stdout:3/245: write d8/f12 [2037180,79358] 0 2024-05-06T12:14:30.604 INFO:tasks.workunit.client.0.smithi092.stdout:4/233: dwrite d9/fc [0,4194304] 0 2024-05-06T12:14:30.606 INFO:tasks.workunit.client.0.smithi092.stdout:0/271: chown d7/dc/d28/d54/f11 43650 1 2024-05-06T12:14:30.607 INFO:tasks.workunit.client.0.smithi092.stdout:0/272: chown d7/dc/d31 97411 1 2024-05-06T12:14:30.607 INFO:tasks.workunit.client.0.smithi092.stdout:3/246: fdatasync d8/d24/d33/d39/f44 0 2024-05-06T12:14:30.609 INFO:tasks.workunit.client.0.smithi092.stdout:4/234: unlink d9/df/l36 0 2024-05-06T12:14:30.609 INFO:tasks.workunit.client.0.smithi092.stdout:0/273: chown d7/dc/d28/d54/d1e/f41 7 1 2024-05-06T12:14:30.609 INFO:tasks.workunit.client.0.smithi092.stdout:0/274: fsync d7/f4f 0 2024-05-06T12:14:30.611 INFO:tasks.workunit.client.0.smithi092.stdout:8/267: rename d9/d21/d37/c52 to d9/dc/c64 0 2024-05-06T12:14:30.612 INFO:tasks.workunit.client.0.smithi092.stdout:9/191: link d16/d2c/l2d d16/d1c/l39 0 2024-05-06T12:14:30.613 INFO:tasks.workunit.client.0.smithi092.stdout:3/247: read d8/fa [1857599,13595] 0 2024-05-06T12:14:30.613 INFO:tasks.workunit.client.0.smithi092.stdout:9/192: dread - d16/f29 zero size 2024-05-06T12:14:30.618 INFO:tasks.workunit.client.0.smithi092.stdout:8/268: read d9/dc/ff [2658668,92466] 0 2024-05-06T12:14:30.621 INFO:tasks.workunit.client.0.smithi092.stdout:8/269: mknod d9/d5a/c65 0 2024-05-06T12:14:30.621 INFO:tasks.workunit.client.0.smithi092.stdout:3/248: dwrite d8/db/f4d [0,4194304] 0 2024-05-06T12:14:30.624 INFO:tasks.workunit.client.0.smithi092.stdout:8/270: creat d9/d21/d54/f66 x:0 0 0 2024-05-06T12:14:30.626 INFO:tasks.workunit.client.0.smithi092.stdout:3/249: creat d8/db/d4c/d3d/f4e x:0 0 0 2024-05-06T12:14:30.627 INFO:tasks.workunit.client.0.smithi092.stdout:3/250: dread - d8/d1b/f2b zero size 2024-05-06T12:14:30.631 INFO:tasks.workunit.client.0.smithi092.stdout:3/251: creat d8/db/d4c/d3d/d49/f4f x:0 0 0 2024-05-06T12:14:30.632 INFO:tasks.workunit.client.0.smithi092.stdout:0/275: dread d7/dc/d28/d54/d1e/f41 [0,4194304] 0 2024-05-06T12:14:30.633 INFO:tasks.workunit.client.0.smithi092.stdout:3/252: dread - d8/db/d4c/d3d/f4e zero size 2024-05-06T12:14:30.633 INFO:tasks.workunit.client.0.smithi092.stdout:8/271: dwrite d9/dc/f51 [0,4194304] 0 2024-05-06T12:14:30.634 INFO:tasks.workunit.client.0.smithi092.stdout:0/276: write d7/f9 [2132721,69161] 0 2024-05-06T12:14:30.637 INFO:tasks.workunit.client.0.smithi092.stdout:8/272: creat d9/d5a/f67 x:0 0 0 2024-05-06T12:14:30.638 INFO:tasks.workunit.client.0.smithi092.stdout:3/253: creat d8/db/d14/f50 x:0 0 0 2024-05-06T12:14:30.644 INFO:tasks.workunit.client.0.smithi092.stdout:9/193: dwrite d16/d2c/d2f/f36 [0,4194304] 0 2024-05-06T12:14:30.644 INFO:tasks.workunit.client.0.smithi092.stdout:0/277: getdents d7/dc/d28/d54/d1e/d26/d3a 0 2024-05-06T12:14:30.644 INFO:tasks.workunit.client.0.smithi092.stdout:3/254: dread d8/d24/f32 [0,4194304] 0 2024-05-06T12:14:30.647 INFO:tasks.workunit.client.0.smithi092.stdout:3/255: dread - d8/d24/d27/f2e zero size 2024-05-06T12:14:30.649 INFO:tasks.workunit.client.0.smithi092.stdout:3/256: readlink d8/d24/d33/d4b/l3b 0 2024-05-06T12:14:30.649 INFO:tasks.workunit.client.0.smithi092.stdout:8/273: dwrite d9/dc/f5d [0,4194304] 0 2024-05-06T12:14:30.649 INFO:tasks.workunit.client.0.smithi092.stdout:3/257: stat d8/db/c28 0 2024-05-06T12:14:30.649 INFO:tasks.workunit.client.0.smithi092.stdout:8/274: stat d9/c60 0 2024-05-06T12:14:30.650 INFO:tasks.workunit.client.0.smithi092.stdout:0/278: mknod d7/dc/d28/d54/d1e/d46/c5e 0 2024-05-06T12:14:30.655 INFO:tasks.workunit.client.0.smithi092.stdout:0/279: rmdir d7/dc/d28/d54/d1e/d56 0 2024-05-06T12:14:30.657 INFO:tasks.workunit.client.0.smithi092.stdout:0/280: creat d7/dc/d28/d54/d1e/d26/d3a/f5f x:0 0 0 2024-05-06T12:14:30.671 INFO:tasks.workunit.client.0.smithi092.stdout:8/275: fdatasync d9/dc/f5d 0 2024-05-06T12:14:30.671 INFO:tasks.workunit.client.0.smithi092.stdout:8/276: chown d9/f33 5 1 2024-05-06T12:14:30.675 INFO:tasks.workunit.client.0.smithi092.stdout:8/277: dread d9/d21/d37/d46/f5e [0,4194304] 0 2024-05-06T12:14:30.678 INFO:tasks.workunit.client.0.smithi092.stdout:8/278: creat d9/d21/d37/d46/d5c/d62/f68 x:0 0 0 2024-05-06T12:14:30.678 INFO:tasks.workunit.client.0.smithi092.stdout:8/279: fdatasync d9/f42 0 2024-05-06T12:14:30.680 INFO:tasks.workunit.client.0.smithi092.stdout:8/280: read d9/dc/ff [1259762,6794] 0 2024-05-06T12:14:30.683 INFO:tasks.workunit.client.0.smithi092.stdout:8/281: link d9/dc/l3c d9/d21/d37/d46/d5c/d62/l69 0 2024-05-06T12:14:30.684 INFO:tasks.workunit.client.0.smithi092.stdout:8/282: write d9/dc/f4d [860778,123914] 0 2024-05-06T12:14:30.686 INFO:tasks.workunit.client.0.smithi092.stdout:8/283: readlink d9/l11 0 2024-05-06T12:14:30.689 INFO:tasks.workunit.client.0.smithi092.stdout:8/284: getdents d9/dc 0 2024-05-06T12:14:30.691 INFO:tasks.workunit.client.0.smithi092.stdout:8/285: chown d9/d21/c32 3 1 2024-05-06T12:14:30.691 INFO:tasks.workunit.client.0.smithi092.stdout:8/286: readlink d9/d21/d37/l43 0 2024-05-06T12:14:30.693 INFO:tasks.workunit.client.0.smithi092.stdout:8/287: symlink d9/d21/d54/l6a 0 2024-05-06T12:14:30.697 INFO:tasks.workunit.client.0.smithi092.stdout:8/288: rename d9/d21/d54/f66 to d9/dc/d4a/f6b 0 2024-05-06T12:14:30.699 INFO:tasks.workunit.client.0.smithi092.stdout:8/289: symlink d9/d21/d37/d46/d5c/d62/l6c 0 2024-05-06T12:14:30.700 INFO:tasks.workunit.client.0.smithi092.stdout:8/290: chown f0 3 1 2024-05-06T12:14:30.702 INFO:tasks.workunit.client.0.smithi092.stdout:8/291: mkdir d9/d6d 0 2024-05-06T12:14:30.704 INFO:tasks.workunit.client.0.smithi092.stdout:8/292: getdents d9/d21/d37 0 2024-05-06T12:14:30.705 INFO:tasks.workunit.client.0.smithi092.stdout:8/293: readlink d9/d21/l24 0 2024-05-06T12:14:30.706 INFO:tasks.workunit.client.0.smithi092.stdout:8/294: write d9/dc/d4a/f6b [750087,124182] 0 2024-05-06T12:14:30.710 INFO:tasks.workunit.client.0.smithi092.stdout:8/295: rename d9/fe to d9/d21/d37/d46/d5c/d62/f6e 0 2024-05-06T12:14:30.719 INFO:tasks.workunit.client.0.smithi092.stdout:8/296: dread d9/f18 [0,4194304] 0 2024-05-06T12:14:30.722 INFO:tasks.workunit.client.0.smithi092.stdout:8/297: creat d9/d21/f6f x:0 0 0 2024-05-06T12:14:30.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:30 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:30.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:30 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:30.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:30 smithi092 ceph-mon[26954]: pgmap v17: 65 pgs: 65 active+clean; 439 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 3.7 MiB/s rd, 39 MiB/s wr, 387 op/s 2024-05-06T12:14:30.723 INFO:tasks.workunit.client.0.smithi092.stdout:8/298: fsync d9/f29 0 2024-05-06T12:14:30.725 INFO:tasks.workunit.client.0.smithi092.stdout:8/299: rename f6 to d9/dc/d4a/d5f/f70 0 2024-05-06T12:14:30.727 INFO:tasks.workunit.client.0.smithi092.stdout:8/300: rename d9/d6d to d9/d21/d37/d46/d5c/d71 0 2024-05-06T12:14:30.729 INFO:tasks.workunit.client.0.smithi092.stdout:8/301: mkdir d9/d21/d72 0 2024-05-06T12:14:30.730 INFO:tasks.workunit.client.0.smithi092.stdout:8/302: symlink d9/d21/d37/l73 0 2024-05-06T12:14:30.732 INFO:tasks.workunit.client.0.smithi092.stdout:8/303: truncate d9/d21/d37/d46/d5c/d62/f68 233976 0 2024-05-06T12:14:30.732 INFO:tasks.workunit.client.0.smithi092.stdout:8/304: write d9/d21/d54/f63 [836137,92328] 0 2024-05-06T12:14:30.734 INFO:tasks.workunit.client.0.smithi092.stdout:8/305: mkdir d9/d21/d37/d74 0 2024-05-06T12:14:30.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:30 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:30.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:30 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:30.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:30 smithi190 ceph-mon[33798]: pgmap v17: 65 pgs: 65 active+clean; 439 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 3.7 MiB/s rd, 39 MiB/s wr, 387 op/s 2024-05-06T12:14:30.938 INFO:tasks.workunit.client.0.smithi092.stdout:7/438: sync 2024-05-06T12:14:30.938 INFO:tasks.workunit.client.0.smithi092.stdout:6/278: sync 2024-05-06T12:14:30.939 INFO:tasks.workunit.client.0.smithi092.stdout:7/439: chown d9/de/d10/d17/d21/d2e/f32 29 1 2024-05-06T12:14:30.944 INFO:tasks.workunit.client.0.smithi092.stdout:6/279: dread d4/f19 [0,4194304] 0 2024-05-06T12:14:30.945 INFO:tasks.workunit.client.0.smithi092.stdout:7/440: dwrite d9/de/d1f/d37/d48/d65/f66 [0,4194304] 0 2024-05-06T12:14:30.946 INFO:tasks.workunit.client.0.smithi092.stdout:6/280: mkdir d4/d24/d38/d5d 0 2024-05-06T12:14:30.947 INFO:tasks.workunit.client.0.smithi092.stdout:7/441: mknod d9/de/d10/c96 0 2024-05-06T12:14:30.949 INFO:tasks.workunit.client.0.smithi092.stdout:6/281: dread d4/f19 [0,4194304] 0 2024-05-06T12:14:30.952 INFO:tasks.workunit.client.0.smithi092.stdout:6/282: unlink d4/d20/d23/d30/f59 0 2024-05-06T12:14:30.954 INFO:tasks.workunit.client.0.smithi092.stdout:6/283: creat d4/d20/d23/d30/f5e x:0 0 0 2024-05-06T12:14:30.955 INFO:tasks.workunit.client.0.smithi092.stdout:6/284: readlink d4/d24/l41 0 2024-05-06T12:14:30.956 INFO:tasks.workunit.client.0.smithi092.stdout:6/285: symlink d4/d24/d26/l5f 0 2024-05-06T12:14:30.958 INFO:tasks.workunit.client.0.smithi092.stdout:6/286: symlink d4/d24/d4e/l60 0 2024-05-06T12:14:30.958 INFO:tasks.workunit.client.0.smithi092.stdout:6/287: dread - d4/d4c/d52/f56 zero size 2024-05-06T12:14:30.959 INFO:tasks.workunit.client.0.smithi092.stdout:6/288: stat d4/d24/d26/l27 0 2024-05-06T12:14:30.963 INFO:tasks.workunit.client.0.smithi092.stdout:7/442: dread d9/de/d2d/f8a [0,4194304] 0 2024-05-06T12:14:30.965 INFO:tasks.workunit.client.0.smithi092.stdout:7/443: unlink d9/de/f2a 0 2024-05-06T12:14:30.971 INFO:tasks.workunit.client.0.smithi092.stdout:6/289: write d4/fb [5081365,30917] 0 2024-05-06T12:14:30.973 INFO:tasks.workunit.client.0.smithi092.stdout:6/290: write d4/d4c/f4f [147694,18251] 0 2024-05-06T12:14:30.973 INFO:tasks.workunit.client.0.smithi092.stdout:7/444: truncate d9/dd/f85 3775545 0 2024-05-06T12:14:30.973 INFO:tasks.workunit.client.0.smithi092.stdout:6/291: fdatasync d4/d24/f55 0 2024-05-06T12:14:30.975 INFO:tasks.workunit.client.0.smithi092.stdout:6/292: truncate d4/f19 1465597 0 2024-05-06T12:14:30.977 INFO:tasks.workunit.client.0.smithi092.stdout:6/293: rmdir d4/d20/d23 39 2024-05-06T12:14:30.978 INFO:tasks.workunit.client.0.smithi092.stdout:6/294: write d4/f11 [4244067,90944] 0 2024-05-06T12:14:30.979 INFO:tasks.workunit.client.0.smithi092.stdout:6/295: write d4/d20/f3b [116669,9758] 0 2024-05-06T12:14:30.981 INFO:tasks.workunit.client.0.smithi092.stdout:6/296: dread - d4/d4c/d52/f56 zero size 2024-05-06T12:14:30.983 INFO:tasks.workunit.client.0.smithi092.stdout:6/297: mknod d4/d24/d38/c61 0 2024-05-06T12:14:30.983 INFO:tasks.workunit.client.0.smithi092.stdout:7/445: dwrite f3 [0,4194304] 0 2024-05-06T12:14:30.984 INFO:tasks.workunit.client.0.smithi092.stdout:7/446: write d9/f5a [143776,117744] 0 2024-05-06T12:14:30.988 INFO:tasks.workunit.client.0.smithi092.stdout:9/194: sync 2024-05-06T12:14:30.989 INFO:tasks.workunit.client.0.smithi092.stdout:3/258: sync 2024-05-06T12:14:30.989 INFO:tasks.workunit.client.0.smithi092.stdout:8/306: sync 2024-05-06T12:14:30.989 INFO:tasks.workunit.client.0.smithi092.stdout:4/235: sync 2024-05-06T12:14:30.989 INFO:tasks.workunit.client.0.smithi092.stdout:1/206: sync 2024-05-06T12:14:30.989 INFO:tasks.workunit.client.0.smithi092.stdout:2/187: sync 2024-05-06T12:14:30.989 INFO:tasks.workunit.client.0.smithi092.stdout:0/281: sync 2024-05-06T12:14:30.989 INFO:tasks.workunit.client.0.smithi092.stdout:5/269: sync 2024-05-06T12:14:30.989 INFO:tasks.workunit.client.0.smithi092.stdout:7/447: truncate d9/dd/d15/f14 1131638 0 2024-05-06T12:14:30.990 INFO:tasks.workunit.client.0.smithi092.stdout:7/448: fdatasync d9/f6f 0 2024-05-06T12:14:30.991 INFO:tasks.workunit.client.0.smithi092.stdout:3/259: dread - d8/d24/d27/f4a zero size 2024-05-06T12:14:30.991 INFO:tasks.workunit.client.0.smithi092.stdout:1/207: unlink d3/d10/d13/d19/f3a 0 2024-05-06T12:14:30.992 INFO:tasks.workunit.client.0.smithi092.stdout:1/208: dread - d3/f3d zero size 2024-05-06T12:14:30.992 INFO:tasks.workunit.client.0.smithi092.stdout:8/307: read d9/d21/f58 [4153423,46806] 0 2024-05-06T12:14:30.993 INFO:tasks.workunit.client.0.smithi092.stdout:8/308: fsync d9/f2a 0 2024-05-06T12:14:30.994 INFO:tasks.workunit.client.0.smithi092.stdout:5/270: symlink d4/d5/l6e 0 2024-05-06T12:14:30.995 INFO:tasks.workunit.client.0.smithi092.stdout:9/195: creat d16/d2c/d2f/f3a x:0 0 0 2024-05-06T12:14:30.995 INFO:tasks.workunit.client.0.smithi092.stdout:6/298: write d4/d20/d23/d30/f33 [634438,83456] 0 2024-05-06T12:14:30.996 INFO:tasks.workunit.client.0.smithi092.stdout:9/196: dread - d16/f29 zero size 2024-05-06T12:14:30.996 INFO:tasks.workunit.client.0.smithi092.stdout:2/188: mkdir df/d17/d1d/d33/d3c 0 2024-05-06T12:14:30.997 INFO:tasks.workunit.client.0.smithi092.stdout:6/299: write d4/f3d [634122,72460] 0 2024-05-06T12:14:30.999 INFO:tasks.workunit.client.0.smithi092.stdout:7/449: mkdir d9/de/d1f/d37/d48/d97 0 2024-05-06T12:14:31.000 INFO:tasks.workunit.client.0.smithi092.stdout:1/209: symlink d3/d10/l48 0 2024-05-06T12:14:31.002 INFO:tasks.workunit.client.0.smithi092.stdout:8/309: mkdir d9/d21/d37/d46/d5c/d71/d75 0 2024-05-06T12:14:31.005 INFO:tasks.workunit.client.0.smithi092.stdout:1/210: write d3/d10/d13/d19/f1e [2846770,103458] 0 2024-05-06T12:14:31.005 INFO:tasks.workunit.client.0.smithi092.stdout:7/450: truncate d9/de/d10/d17/d21/f3e 1000409 0 2024-05-06T12:14:31.007 INFO:tasks.workunit.client.0.smithi092.stdout:6/300: mkdir d4/d4c/d52/d62 0 2024-05-06T12:14:31.009 INFO:tasks.workunit.client.0.smithi092.stdout:0/282: dwrite d7/dc/d28/d54/d1e/d26/d3a/f52 [0,4194304] 0 2024-05-06T12:14:31.011 INFO:tasks.workunit.client.0.smithi092.stdout:9/197: link d16/d1c/f30 d16/d1c/d32/d33/f3b 0 2024-05-06T12:14:31.011 INFO:tasks.workunit.client.0.smithi092.stdout:1/211: read - d3/d10/d13/d19/f31 zero size 2024-05-06T12:14:31.014 INFO:tasks.workunit.client.0.smithi092.stdout:4/236: getdents d9/d27/d4a 0 2024-05-06T12:14:31.014 INFO:tasks.workunit.client.0.smithi092.stdout:9/198: write d16/f29 [485075,130927] 0 2024-05-06T12:14:31.016 INFO:tasks.workunit.client.0.smithi092.stdout:1/212: write d3/f12 [8276383,121914] 0 2024-05-06T12:14:31.018 INFO:tasks.workunit.client.0.smithi092.stdout:8/310: dwrite d9/d21/d54/f63 [0,4194304] 0 2024-05-06T12:14:31.020 INFO:tasks.workunit.client.0.smithi092.stdout:7/451: creat d9/de/d1f/d37/d48/d65/f98 x:0 0 0 2024-05-06T12:14:31.021 INFO:tasks.workunit.client.0.smithi092.stdout:5/271: dread d4/d5/d30/f46 [0,4194304] 0 2024-05-06T12:14:31.021 INFO:tasks.workunit.client.0.smithi092.stdout:7/452: stat d9/lf 0 2024-05-06T12:14:31.023 INFO:tasks.workunit.client.0.smithi092.stdout:6/301: rmdir d4 39 2024-05-06T12:14:31.024 INFO:tasks.workunit.client.0.smithi092.stdout:9/199: dread d16/d2c/d2f/f36 [0,4194304] 0 2024-05-06T12:14:31.025 INFO:tasks.workunit.client.0.smithi092.stdout:1/213: unlink d3/d10/d13/d19/l41 0 2024-05-06T12:14:31.026 INFO:tasks.workunit.client.0.smithi092.stdout:9/200: write d16/d1e/f28 [1851047,80152] 0 2024-05-06T12:14:31.026 INFO:tasks.workunit.client.0.smithi092.stdout:0/283: symlink d7/l60 0 2024-05-06T12:14:31.027 INFO:tasks.workunit.client.0.smithi092.stdout:9/201: write f6 [1476884,85660] 0 2024-05-06T12:14:31.028 INFO:tasks.workunit.client.0.smithi092.stdout:8/311: rmdir d9/d21/d54 39 2024-05-06T12:14:31.029 INFO:tasks.workunit.client.0.smithi092.stdout:5/272: symlink d4/d34/l6f 0 2024-05-06T12:14:31.030 INFO:tasks.workunit.client.0.smithi092.stdout:5/273: write d4/f4c [330007,117117] 0 2024-05-06T12:14:31.033 INFO:tasks.workunit.client.0.smithi092.stdout:7/453: rename d9/dd/d23 to d9/de/d10/d17/d99 0 2024-05-06T12:14:31.037 INFO:tasks.workunit.client.0.smithi092.stdout:6/302: dwrite d4/f3d [0,4194304] 0 2024-05-06T12:14:31.040 INFO:tasks.workunit.client.0.smithi092.stdout:0/284: creat d7/d43/f61 x:0 0 0 2024-05-06T12:14:31.041 INFO:tasks.workunit.client.0.smithi092.stdout:2/189: truncate df/f12 30179 0 2024-05-06T12:14:31.045 INFO:tasks.workunit.client.0.smithi092.stdout:0/285: chown d7/dc/d28/d54/d1e/d5d 11 1 2024-05-06T12:14:31.046 INFO:tasks.workunit.client.0.smithi092.stdout:9/202: creat d16/d1c/d32/f3c x:0 0 0 2024-05-06T12:14:31.047 INFO:tasks.workunit.client.0.smithi092.stdout:5/274: symlink d4/d5/d10/d31/l70 0 2024-05-06T12:14:31.048 INFO:tasks.workunit.client.0.smithi092.stdout:1/214: rename d3/c7 to d3/d10/c49 0 2024-05-06T12:14:31.049 INFO:tasks.workunit.client.0.smithi092.stdout:5/275: stat d4/d34/d28/d4e/f58 0 2024-05-06T12:14:31.051 INFO:tasks.workunit.client.0.smithi092.stdout:4/237: truncate d9/fc 6681210 0 2024-05-06T12:14:31.053 INFO:tasks.workunit.client.0.smithi092.stdout:7/454: mkdir d9/d62/d9a 0 2024-05-06T12:14:31.053 INFO:tasks.workunit.client.0.smithi092.stdout:6/303: dwrite d4/d20/f1c [0,4194304] 0 2024-05-06T12:14:31.054 INFO:tasks.workunit.client.0.smithi092.stdout:5/276: chown d4/d34/l14 728 1 2024-05-06T12:14:31.055 INFO:tasks.workunit.client.0.smithi092.stdout:8/312: creat d9/f76 x:0 0 0 2024-05-06T12:14:31.057 INFO:tasks.workunit.client.0.smithi092.stdout:0/286: mknod d7/dc/d28/d54/d1e/d5d/c62 0 2024-05-06T12:14:31.057 INFO:tasks.workunit.client.0.smithi092.stdout:2/190: dwrite df/f1e [0,4194304] 0 2024-05-06T12:14:31.057 INFO:tasks.workunit.client.0.smithi092.stdout:4/238: truncate d9/df/f14 5186525 0 2024-05-06T12:14:31.058 INFO:tasks.workunit.client.0.smithi092.stdout:9/203: getdents d16/d1e 0 2024-05-06T12:14:31.059 INFO:tasks.workunit.client.0.smithi092.stdout:7/455: creat d9/dd/f9b x:0 0 0 2024-05-06T12:14:31.059 INFO:tasks.workunit.client.0.smithi092.stdout:5/277: read - d4/d34/d28/d4e/d4b/f4d zero size 2024-05-06T12:14:31.060 INFO:tasks.workunit.client.0.smithi092.stdout:4/239: readlink d9/df/l1a 0 2024-05-06T12:14:31.064 INFO:tasks.workunit.client.0.smithi092.stdout:7/456: dread - d9/de/d1f/d37/f60 zero size 2024-05-06T12:14:31.064 INFO:tasks.workunit.client.0.smithi092.stdout:4/240: write d9/d2b/f32 [8549347,93954] 0 2024-05-06T12:14:31.070 INFO:tasks.workunit.client.0.smithi092.stdout:4/241: write d9/d27/f49 [956078,45231] 0 2024-05-06T12:14:31.070 INFO:tasks.workunit.client.0.smithi092.stdout:1/215: dwrite d3/d10/d13/f3f [0,4194304] 0 2024-05-06T12:14:31.073 INFO:tasks.workunit.client.0.smithi092.stdout:9/204: rename d16/d1c/d32 to d16/d2c/d2f/d3d 0 2024-05-06T12:14:31.077 INFO:tasks.workunit.client.0.smithi092.stdout:0/287: dwrite d7/dc/d28/d54/d12/f1b [0,4194304] 0 2024-05-06T12:14:31.077 INFO:tasks.workunit.client.0.smithi092.stdout:2/191: creat df/d17/d1d/d33/d3c/f3d x:0 0 0 2024-05-06T12:14:31.078 INFO:tasks.workunit.client.0.smithi092.stdout:7/457: readlink d9/de/d10/d17/d21/d2e/l95 0 2024-05-06T12:14:31.081 INFO:tasks.workunit.client.0.smithi092.stdout:6/304: link d4/c7 d4/d20/c63 0 2024-05-06T12:14:31.082 INFO:tasks.workunit.client.0.smithi092.stdout:9/205: creat d16/d1e/f3e x:0 0 0 2024-05-06T12:14:31.084 INFO:tasks.workunit.client.0.smithi092.stdout:1/216: symlink d3/d10/d13/l4a 0 2024-05-06T12:14:31.085 INFO:tasks.workunit.client.0.smithi092.stdout:1/217: dread - d3/f2c zero size 2024-05-06T12:14:31.087 INFO:tasks.workunit.client.0.smithi092.stdout:7/458: mkdir d9/de/d10/d17/d21/d2e/d9c 0 2024-05-06T12:14:31.090 INFO:tasks.workunit.client.0.smithi092.stdout:6/305: rename d4/d20/d22/c42 to d4/d24/d26/c64 0 2024-05-06T12:14:31.092 INFO:tasks.workunit.client.0.smithi092.stdout:0/288: getdents d7/dc/d28/d54/d1e/d26/d4e 0 2024-05-06T12:14:31.092 INFO:tasks.workunit.client.0.smithi092.stdout:6/306: read d4/d4c/f4f [94433,27814] 0 2024-05-06T12:14:31.093 INFO:tasks.workunit.client.0.smithi092.stdout:0/289: chown d7/c20 5682428 1 2024-05-06T12:14:31.094 INFO:tasks.workunit.client.0.smithi092.stdout:1/218: creat d3/d10/d13/d19/d34/f4b x:0 0 0 2024-05-06T12:14:31.095 INFO:tasks.workunit.client.0.smithi092.stdout:7/459: rmdir d9/dd 39 2024-05-06T12:14:31.095 INFO:tasks.workunit.client.0.smithi092.stdout:7/460: chown d9/f5a 29101000 1 2024-05-06T12:14:31.096 INFO:tasks.workunit.client.0.smithi092.stdout:7/461: dread - d9/de/d10/d17/d99/f7d zero size 2024-05-06T12:14:31.098 INFO:tasks.workunit.client.0.smithi092.stdout:2/192: creat df/d17/f3e x:0 0 0 2024-05-06T12:14:31.098 INFO:tasks.workunit.client.0.smithi092.stdout:2/193: chown df/d15 1148626 1 2024-05-06T12:14:31.099 INFO:tasks.workunit.client.0.smithi092.stdout:2/194: fsync df/d17/d1d/d33/f35 0 2024-05-06T12:14:31.100 INFO:tasks.workunit.client.0.smithi092.stdout:4/242: fsync d9/f38 0 2024-05-06T12:14:31.101 INFO:tasks.workunit.client.0.smithi092.stdout:2/195: write fd [4848536,99370] 0 2024-05-06T12:14:31.103 INFO:tasks.workunit.client.0.smithi092.stdout:0/290: unlink d7/c1d 0 2024-05-06T12:14:31.104 INFO:tasks.workunit.client.0.smithi092.stdout:8/313: dread d9/f42 [0,4194304] 0 2024-05-06T12:14:31.104 INFO:tasks.workunit.client.0.smithi092.stdout:0/291: readlink d7/l60 0 2024-05-06T12:14:31.107 INFO:tasks.workunit.client.0.smithi092.stdout:2/196: dread df/d17/d1d/f2e [0,4194304] 0 2024-05-06T12:14:31.110 INFO:tasks.workunit.client.0.smithi092.stdout:5/278: dread d4/d5/f3b [4194304,4194304] 0 2024-05-06T12:14:31.110 INFO:tasks.workunit.client.0.smithi092.stdout:6/307: getdents d4/d4c/d52/d62 0 2024-05-06T12:14:31.112 INFO:tasks.workunit.client.0.smithi092.stdout:0/292: dwrite d7/d43/f2c [0,4194304] 0 2024-05-06T12:14:31.112 INFO:tasks.workunit.client.0.smithi092.stdout:4/243: creat d9/d27/d4a/f4c x:0 0 0 2024-05-06T12:14:31.113 INFO:tasks.workunit.client.0.smithi092.stdout:5/279: dread - d4/d5/d10/d12/f63 zero size 2024-05-06T12:14:31.114 INFO:tasks.workunit.client.0.smithi092.stdout:3/260: sync 2024-05-06T12:14:31.114 INFO:tasks.workunit.client.0.smithi092.stdout:9/206: dread f6 [0,4194304] 0 2024-05-06T12:14:31.116 INFO:tasks.workunit.client.0.smithi092.stdout:1/219: rmdir d3 39 2024-05-06T12:14:31.123 INFO:tasks.workunit.client.0.smithi092.stdout:9/207: dwrite d16/d2c/d2f/f3a [0,4194304] 0 2024-05-06T12:14:31.123 INFO:tasks.workunit.client.0.smithi092.stdout:4/244: creat d9/d27/f4d x:0 0 0 2024-05-06T12:14:31.123 INFO:tasks.workunit.client.0.smithi092.stdout:4/245: fdatasync f5 0 2024-05-06T12:14:31.124 INFO:tasks.workunit.client.0.smithi092.stdout:0/293: mkdir d7/dc/d28/d54/d1e/d46/d63 0 2024-05-06T12:14:31.126 INFO:tasks.workunit.client.0.smithi092.stdout:0/294: fsync d7/dc/d28/d54/d12/f1b 0 2024-05-06T12:14:31.126 INFO:tasks.workunit.client.0.smithi092.stdout:5/280: creat d4/d34/d28/d4e/f71 x:0 0 0 2024-05-06T12:14:31.127 INFO:tasks.workunit.client.0.smithi092.stdout:3/261: mknod d8/d24/d33/d4b/c51 0 2024-05-06T12:14:31.127 INFO:tasks.workunit.client.0.smithi092.stdout:8/314: getdents d9/d21/d72 0 2024-05-06T12:14:31.128 INFO:tasks.workunit.client.0.smithi092.stdout:8/315: chown d9/d21/d37/d74 87831878 1 2024-05-06T12:14:31.129 INFO:tasks.workunit.client.0.smithi092.stdout:3/262: truncate d8/d24/d27/f2e 752562 0 2024-05-06T12:14:31.130 INFO:tasks.workunit.client.0.smithi092.stdout:7/462: link d9/de/d10/f27 d9/de/d10/d17/d6a/f9d 0 2024-05-06T12:14:31.132 INFO:tasks.workunit.client.0.smithi092.stdout:6/308: symlink d4/d20/d23/l65 0 2024-05-06T12:14:31.133 INFO:tasks.workunit.client.0.smithi092.stdout:9/208: mkdir d16/d2c/d3f 0 2024-05-06T12:14:31.135 INFO:tasks.workunit.client.0.smithi092.stdout:4/246: unlink d9/df/c1f 0 2024-05-06T12:14:31.135 INFO:tasks.workunit.client.0.smithi092.stdout:4/247: chown d9/f38 2 1 2024-05-06T12:14:31.137 INFO:tasks.workunit.client.0.smithi092.stdout:5/281: chown d4/d5/d10/f17 247570 1 2024-05-06T12:14:31.138 INFO:tasks.workunit.client.0.smithi092.stdout:0/295: mkdir d7/dc/d28/d64 0 2024-05-06T12:14:31.140 INFO:tasks.workunit.client.0.smithi092.stdout:1/220: rename d3/d10/l48 to d3/d10/l4c 0 2024-05-06T12:14:31.143 INFO:tasks.workunit.client.0.smithi092.stdout:1/221: dread - d3/f2c zero size 2024-05-06T12:14:31.144 INFO:tasks.workunit.client.0.smithi092.stdout:5/282: truncate d4/d5/d10/d31/d53/d64/f5e 468620 0 2024-05-06T12:14:31.146 INFO:tasks.workunit.client.0.smithi092.stdout:6/309: mkdir d4/d24/d66 0 2024-05-06T12:14:31.147 INFO:tasks.workunit.client.0.smithi092.stdout:5/283: readlink d4/d5/lf 0 2024-05-06T12:14:31.148 INFO:tasks.workunit.client.0.smithi092.stdout:4/248: dwrite d9/d27/f49 [0,4194304] 0 2024-05-06T12:14:31.150 INFO:tasks.workunit.client.0.smithi092.stdout:1/222: dread - d3/d10/d13/d19/f31 zero size 2024-05-06T12:14:31.154 INFO:tasks.workunit.client.0.smithi092.stdout:8/316: mknod d9/d21/c77 0 2024-05-06T12:14:31.154 INFO:tasks.workunit.client.0.smithi092.stdout:4/249: fdatasync d9/d2b/f32 0 2024-05-06T12:14:31.156 INFO:tasks.workunit.client.0.smithi092.stdout:0/296: dwrite d7/d43/f49 [0,4194304] 0 2024-05-06T12:14:31.156 INFO:tasks.workunit.client.0.smithi092.stdout:3/263: creat d8/db/d4c/d40/f52 x:0 0 0 2024-05-06T12:14:31.161 INFO:tasks.workunit.client.0.smithi092.stdout:7/463: write d9/de/d2d/f8a [2160907,121549] 0 2024-05-06T12:14:31.161 INFO:tasks.workunit.client.0.smithi092.stdout:3/264: chown d8/d24/d33/d4b/f41 0 1 2024-05-06T12:14:31.161 INFO:tasks.workunit.client.0.smithi092.stdout:9/209: dread d16/f1f [0,4194304] 0 2024-05-06T12:14:31.163 INFO:tasks.workunit.client.0.smithi092.stdout:8/317: creat d9/d5a/f78 x:0 0 0 2024-05-06T12:14:31.164 INFO:tasks.workunit.client.0.smithi092.stdout:2/197: dwrite df/f12 [0,4194304] 0 2024-05-06T12:14:31.167 INFO:tasks.workunit.client.0.smithi092.stdout:5/284: rename d4/d5/d10/d31/d53/d64/f5a to d4/d34/d28/f72 0 2024-05-06T12:14:31.172 INFO:tasks.workunit.client.0.smithi092.stdout:7/464: dwrite d9/f80 [0,4194304] 0 2024-05-06T12:14:31.173 INFO:tasks.workunit.client.0.smithi092.stdout:7/465: write d9/de/f50 [963620,52133] 0 2024-05-06T12:14:31.177 INFO:tasks.workunit.client.0.smithi092.stdout:3/265: unlink d8/d24/d33/d4b/f17 0 2024-05-06T12:14:31.178 INFO:tasks.workunit.client.0.smithi092.stdout:2/198: symlink df/d17/d1d/l3f 0 2024-05-06T12:14:31.180 INFO:tasks.workunit.client.0.smithi092.stdout:0/297: mknod d7/dc/d28/d64/c65 0 2024-05-06T12:14:31.182 INFO:tasks.workunit.client.0.smithi092.stdout:9/210: dread f7 [0,4194304] 0 2024-05-06T12:14:31.182 INFO:tasks.workunit.client.0.smithi092.stdout:6/310: link d4/d24/f39 d4/d20/d23/f67 0 2024-05-06T12:14:31.184 INFO:tasks.workunit.client.0.smithi092.stdout:8/318: mkdir d9/d21/d54/d79 0 2024-05-06T12:14:31.185 INFO:tasks.workunit.client.0.smithi092.stdout:3/266: symlink d8/db/d4c/d40/l53 0 2024-05-06T12:14:31.187 INFO:tasks.workunit.client.0.smithi092.stdout:3/267: chown d8/db/l2d 1 1 2024-05-06T12:14:31.187 INFO:tasks.workunit.client.0.smithi092.stdout:2/199: chown df/f1f 40 1 2024-05-06T12:14:31.187 INFO:tasks.workunit.client.0.smithi092.stdout:9/211: dread f3 [0,4194304] 0 2024-05-06T12:14:31.188 INFO:tasks.workunit.client.0.smithi092.stdout:5/285: mkdir d4/d34/d29/d54/d73 0 2024-05-06T12:14:31.190 INFO:tasks.workunit.client.0.smithi092.stdout:7/466: symlink d9/de/d10/d17/d21/d2e/d9c/l9e 0 2024-05-06T12:14:31.191 INFO:tasks.workunit.client.0.smithi092.stdout:9/212: chown d16/d1c/l1d 27731 1 2024-05-06T12:14:31.193 INFO:tasks.workunit.client.0.smithi092.stdout:6/311: mknod d4/d24/d54/c68 0 2024-05-06T12:14:31.194 INFO:tasks.workunit.client.0.smithi092.stdout:2/200: dread df/f1f [0,4194304] 0 2024-05-06T12:14:31.194 INFO:tasks.workunit.client.0.smithi092.stdout:7/467: dread - d9/de/d10/d17/d21/d38/f89 zero size 2024-05-06T12:14:31.195 INFO:tasks.workunit.client.0.smithi092.stdout:3/268: symlink d8/db/d4c/d3d/d48/l54 0 2024-05-06T12:14:31.195 INFO:tasks.workunit.client.0.smithi092.stdout:0/298: link d7/l60 d7/dc/d28/d54/d1e/d5d/l66 0 2024-05-06T12:14:31.197 INFO:tasks.workunit.client.0.smithi092.stdout:8/319: dwrite d9/f42 [4194304,4194304] 0 2024-05-06T12:14:31.197 INFO:tasks.workunit.client.0.smithi092.stdout:0/299: readlink d7/l10 0 2024-05-06T12:14:31.200 INFO:tasks.workunit.client.0.smithi092.stdout:9/213: truncate d16/f1b 5958 0 2024-05-06T12:14:31.201 INFO:tasks.workunit.client.0.smithi092.stdout:5/286: dwrite d4/d5/f1d [0,4194304] 0 2024-05-06T12:14:31.202 INFO:tasks.workunit.client.0.smithi092.stdout:5/287: chown d4/d5/d10/d31/f4a 11 1 2024-05-06T12:14:31.202 INFO:tasks.workunit.client.0.smithi092.stdout:4/250: rename d9/df/l3e to d9/dd/l4e 0 2024-05-06T12:14:31.205 INFO:tasks.workunit.client.0.smithi092.stdout:2/201: symlink df/d17/d1d/l40 0 2024-05-06T12:14:31.205 INFO:tasks.workunit.client.0.smithi092.stdout:2/202: chown df/f2c 233128 1 2024-05-06T12:14:31.206 INFO:tasks.workunit.client.0.smithi092.stdout:1/223: truncate d3/f12 6429778 0 2024-05-06T12:14:31.211 INFO:tasks.workunit.client.0.smithi092.stdout:8/320: mkdir d9/d21/d37/d46/d5c/d62/d7a 0 2024-05-06T12:14:31.215 INFO:tasks.workunit.client.0.smithi092.stdout:1/224: dwrite d3/d10/d13/d19/d34/f4b [0,4194304] 0 2024-05-06T12:14:31.215 INFO:tasks.workunit.client.0.smithi092.stdout:6/312: rename d4/d20/d22/f45 to d4/d24/d38/f69 0 2024-05-06T12:14:31.216 INFO:tasks.workunit.client.0.smithi092.stdout:1/225: readlink d3/d10/d13/d19/d21/l44 0 2024-05-06T12:14:31.217 INFO:tasks.workunit.client.0.smithi092.stdout:6/313: write d4/d20/d23/d30/f36 [605690,57170] 0 2024-05-06T12:14:31.217 INFO:tasks.workunit.client.0.smithi092.stdout:4/251: creat d9/d27/f4f x:0 0 0 2024-05-06T12:14:31.219 INFO:tasks.workunit.client.0.smithi092.stdout:1/226: write d3/d10/d13/f3b [1027981,71608] 0 2024-05-06T12:14:31.220 INFO:tasks.workunit.client.0.smithi092.stdout:1/227: dread - d3/d10/f45 zero size 2024-05-06T12:14:31.223 INFO:tasks.workunit.client.0.smithi092.stdout:9/214: mkdir d16/d2c/d2f/d3d/d33/d37/d40 0 2024-05-06T12:14:31.226 INFO:tasks.workunit.client.0.smithi092.stdout:8/321: creat d9/d21/d54/f7b x:0 0 0 2024-05-06T12:14:31.227 INFO:tasks.workunit.client.0.smithi092.stdout:8/322: write d9/f36 [4627411,71545] 0 2024-05-06T12:14:31.229 INFO:tasks.workunit.client.0.smithi092.stdout:8/323: read - d9/d21/d54/f7b zero size 2024-05-06T12:14:31.229 INFO:tasks.workunit.client.0.smithi092.stdout:5/288: link d4/d5/f1d d4/d34/d28/f74 0 2024-05-06T12:14:31.230 INFO:tasks.workunit.client.0.smithi092.stdout:5/289: fdatasync d4/d5/d10/d12/f63 0 2024-05-06T12:14:31.231 INFO:tasks.workunit.client.0.smithi092.stdout:4/252: creat d9/d27/f50 x:0 0 0 2024-05-06T12:14:31.233 INFO:tasks.workunit.client.0.smithi092.stdout:2/203: link df/d17/f3e df/d17/d1d/f41 0 2024-05-06T12:14:31.238 INFO:tasks.workunit.client.0.smithi092.stdout:6/314: dread d4/d20/d23/f46 [0,4194304] 0 2024-05-06T12:14:31.239 INFO:tasks.workunit.client.0.smithi092.stdout:7/468: write d9/de/d10/f12 [6002954,88510] 0 2024-05-06T12:14:31.240 INFO:tasks.workunit.client.0.smithi092.stdout:6/315: fdatasync d4/d20/f1c 0 2024-05-06T12:14:31.241 INFO:tasks.workunit.client.0.smithi092.stdout:0/300: dread d7/d43/f1c [0,4194304] 0 2024-05-06T12:14:31.242 INFO:tasks.workunit.client.0.smithi092.stdout:5/290: dwrite d4/d5/d10/d31/d53/d64/f48 [0,4194304] 0 2024-05-06T12:14:31.248 INFO:tasks.workunit.client.0.smithi092.stdout:6/316: dwrite d4/d24/f3e [0,4194304] 0 2024-05-06T12:14:31.248 INFO:tasks.workunit.client.0.smithi092.stdout:3/269: link d8/db/d4c/d3d/c47 d8/d24/d33/d4b/c55 0 2024-05-06T12:14:31.249 INFO:tasks.workunit.client.0.smithi092.stdout:6/317: chown d4/l18 7527799 1 2024-05-06T12:14:31.249 INFO:tasks.workunit.client.0.smithi092.stdout:9/215: mknod d16/d2c/d3f/c41 0 2024-05-06T12:14:31.253 INFO:tasks.workunit.client.0.smithi092.stdout:7/469: rmdir d9/de/d10/d17/d21/d2e/d9c 39 2024-05-06T12:14:31.256 INFO:tasks.workunit.client.0.smithi092.stdout:5/291: write d4/d34/d28/f74 [4720659,92778] 0 2024-05-06T12:14:31.258 INFO:tasks.workunit.client.0.smithi092.stdout:5/292: write d4/d5/d30/f61 [365374,74639] 0 2024-05-06T12:14:31.259 INFO:tasks.workunit.client.0.smithi092.stdout:8/324: link d9/f1b d9/d5a/f7c 0 2024-05-06T12:14:31.262 INFO:tasks.workunit.client.0.smithi092.stdout:8/325: readlink d9/dc/l57 0 2024-05-06T12:14:31.263 INFO:tasks.workunit.client.0.smithi092.stdout:8/326: stat d9/dc/d4a 0 2024-05-06T12:14:31.263 INFO:tasks.workunit.client.0.smithi092.stdout:7/470: dwrite d9/de/d10/d17/d21/d2e/f4f [0,4194304] 0 2024-05-06T12:14:31.264 INFO:tasks.workunit.client.0.smithi092.stdout:5/293: write d4/d5/d10/d31/d53/d64/f25 [201869,296] 0 2024-05-06T12:14:31.267 INFO:tasks.workunit.client.0.smithi092.stdout:3/270: rename d8/d24/f16 to d8/d24/d33/d4b/f56 0 2024-05-06T12:14:31.269 INFO:tasks.workunit.client.0.smithi092.stdout:8/327: fdatasync d9/dc/f51 0 2024-05-06T12:14:31.269 INFO:tasks.workunit.client.0.smithi092.stdout:9/216: creat d16/d1e/f42 x:0 0 0 2024-05-06T12:14:31.270 INFO:tasks.workunit.client.0.smithi092.stdout:0/301: getdents d7/dc/d28/d54/d1e/d46/d63 0 2024-05-06T12:14:31.271 INFO:tasks.workunit.client.0.smithi092.stdout:4/253: symlink d9/df/d48/l51 0 2024-05-06T12:14:31.274 INFO:tasks.workunit.client.0.smithi092.stdout:1/228: dwrite d3/f12 [4194304,4194304] 0 2024-05-06T12:14:31.278 INFO:tasks.workunit.client.0.smithi092.stdout:9/217: dread f3 [0,4194304] 0 2024-05-06T12:14:31.278 INFO:tasks.workunit.client.0.smithi092.stdout:1/229: stat d3/d10/d13/l4a 0 2024-05-06T12:14:31.278 INFO:tasks.workunit.client.0.smithi092.stdout:3/271: unlink d8/d24/d27/c29 0 2024-05-06T12:14:31.280 INFO:tasks.workunit.client.0.smithi092.stdout:7/471: creat d9/de/d10/d17/d21/d2e/d87/f9f x:0 0 0 2024-05-06T12:14:31.281 INFO:tasks.workunit.client.0.smithi092.stdout:1/230: write d3/fa [938138,105106] 0 2024-05-06T12:14:31.284 INFO:tasks.workunit.client.0.smithi092.stdout:2/204: getdents df/d17/d1d/d33/d3c 0 2024-05-06T12:14:31.285 INFO:tasks.workunit.client.0.smithi092.stdout:6/318: dwrite d4/f3d [4194304,4194304] 0 2024-05-06T12:14:31.285 INFO:tasks.workunit.client.0.smithi092.stdout:0/302: chown d7/dc/f29 159719846 1 2024-05-06T12:14:31.287 INFO:tasks.workunit.client.0.smithi092.stdout:2/205: write f1 [773515,5939] 0 2024-05-06T12:14:31.289 INFO:tasks.workunit.client.0.smithi092.stdout:6/319: chown d4/d20/d22 10043 1 2024-05-06T12:14:31.289 INFO:tasks.workunit.client.0.smithi092.stdout:8/328: rename d9/c20 to d9/dc/c7d 0 2024-05-06T12:14:31.289 INFO:tasks.workunit.client.0.smithi092.stdout:0/303: write d7/f9 [37230,81995] 0 2024-05-06T12:14:31.291 INFO:tasks.workunit.client.0.smithi092.stdout:1/231: creat d3/d10/d13/d19/d21/f4d x:0 0 0 2024-05-06T12:14:31.291 INFO:tasks.workunit.client.0.smithi092.stdout:6/320: chown d4/d20/d22 10188 1 2024-05-06T12:14:31.296 INFO:tasks.workunit.client.0.smithi092.stdout:3/272: dwrite d8/db/d4c/d3d/d49/f4f [0,4194304] 0 2024-05-06T12:14:31.301 INFO:tasks.workunit.client.0.smithi092.stdout:0/304: dread d7/d43/f1c [0,4194304] 0 2024-05-06T12:14:31.302 INFO:tasks.workunit.client.0.smithi092.stdout:2/206: dread df/f12 [0,4194304] 0 2024-05-06T12:14:31.302 INFO:tasks.workunit.client.0.smithi092.stdout:2/207: stat df/d15 0 2024-05-06T12:14:31.302 INFO:tasks.workunit.client.0.smithi092.stdout:9/218: mkdir d16/d2c/d2f/d3d/d33/d37/d40/d43 0 2024-05-06T12:14:31.302 INFO:tasks.workunit.client.0.smithi092.stdout:1/232: dread d3/d10/d13/f3b [0,4194304] 0 2024-05-06T12:14:31.303 INFO:tasks.workunit.client.0.smithi092.stdout:2/208: stat df/d17/d1d/d33 0 2024-05-06T12:14:31.303 INFO:tasks.workunit.client.0.smithi092.stdout:2/209: readlink df/d15/l3b 0 2024-05-06T12:14:31.304 INFO:tasks.workunit.client.0.smithi092.stdout:4/254: creat d9/df/f52 x:0 0 0 2024-05-06T12:14:31.304 INFO:tasks.workunit.client.0.smithi092.stdout:6/321: rmdir d4 39 2024-05-06T12:14:31.306 INFO:tasks.workunit.client.0.smithi092.stdout:5/294: getdents d4/d34/d28/d4e 0 2024-05-06T12:14:31.307 INFO:tasks.workunit.client.0.smithi092.stdout:5/295: chown d4/d34/d28 20090 1 2024-05-06T12:14:31.309 INFO:tasks.workunit.client.0.smithi092.stdout:9/219: unlink f3 0 2024-05-06T12:14:31.312 INFO:tasks.workunit.client.0.smithi092.stdout:7/472: unlink d9/dd/d15/l5e 0 2024-05-06T12:14:31.313 INFO:tasks.workunit.client.0.smithi092.stdout:0/305: creat d7/dc/d28/d54/d1e/d46/f67 x:0 0 0 2024-05-06T12:14:31.314 INFO:tasks.workunit.client.0.smithi092.stdout:0/306: dread - d7/dc/f5c zero size 2024-05-06T12:14:31.314 INFO:tasks.workunit.client.0.smithi092.stdout:4/255: stat d9/dd/l4e 0 2024-05-06T12:14:31.317 INFO:tasks.workunit.client.0.smithi092.stdout:5/296: chown d4/d5/d10/d12/c5b 3118659 1 2024-05-06T12:14:31.318 INFO:tasks.workunit.client.0.smithi092.stdout:9/220: rmdir d16/d2c/d2f/d3d/d33/d37/d40 39 2024-05-06T12:14:31.319 INFO:tasks.workunit.client.0.smithi092.stdout:7/473: creat d9/de/d1f/d37/fa0 x:0 0 0 2024-05-06T12:14:31.320 INFO:tasks.workunit.client.0.smithi092.stdout:0/307: dread d7/d43/f40 [0,4194304] 0 2024-05-06T12:14:31.323 INFO:tasks.workunit.client.0.smithi092.stdout:3/273: dread d8/d24/d33/d39/f44 [0,4194304] 0 2024-05-06T12:14:31.324 INFO:tasks.workunit.client.0.smithi092.stdout:1/233: creat d3/d10/d13/d19/f4e x:0 0 0 2024-05-06T12:14:31.326 INFO:tasks.workunit.client.0.smithi092.stdout:0/308: dread - d7/dc/d28/d54/d1e/d26/d3a/f5f zero size 2024-05-06T12:14:31.327 INFO:tasks.workunit.client.0.smithi092.stdout:3/274: write d8/db/d14/f50 [315786,110538] 0 2024-05-06T12:14:31.327 INFO:tasks.workunit.client.0.smithi092.stdout:6/322: dwrite d4/d4c/d52/f4a [4194304,4194304] 0 2024-05-06T12:14:31.328 INFO:tasks.workunit.client.0.smithi092.stdout:1/234: fsync d3/d10/d13/f3f 0 2024-05-06T12:14:31.331 INFO:tasks.workunit.client.0.smithi092.stdout:8/329: getdents d9/d21/d37/d46/d5c/d71 0 2024-05-06T12:14:31.333 INFO:tasks.workunit.client.0.smithi092.stdout:3/275: readlink d8/db/l20 0 2024-05-06T12:14:31.336 INFO:tasks.workunit.client.0.smithi092.stdout:5/297: mknod d4/d34/d28/d4e/d67/c75 0 2024-05-06T12:14:31.336 INFO:tasks.workunit.client.0.smithi092.stdout:2/210: truncate df/f12 3394212 0 2024-05-06T12:14:31.338 INFO:tasks.workunit.client.0.smithi092.stdout:4/256: dwrite d9/df/f15 [0,4194304] 0 2024-05-06T12:14:31.338 INFO:tasks.workunit.client.0.smithi092.stdout:5/298: stat d4/d34 0 2024-05-06T12:14:31.339 INFO:tasks.workunit.client.0.smithi092.stdout:4/257: chown d9/d2b/c30 12891098 1 2024-05-06T12:14:31.341 INFO:tasks.workunit.client.0.smithi092.stdout:7/474: dread d9/dd/d15/f30 [0,4194304] 0 2024-05-06T12:14:31.343 INFO:tasks.workunit.client.0.smithi092.stdout:3/276: dread d8/db/f4d [0,4194304] 0 2024-05-06T12:14:31.346 INFO:tasks.workunit.client.0.smithi092.stdout:6/323: creat d4/d4c/d52/f6a x:0 0 0 2024-05-06T12:14:31.347 INFO:tasks.workunit.client.0.smithi092.stdout:6/324: write d4/d20/f31 [1485363,89011] 0 2024-05-06T12:14:31.349 INFO:tasks.workunit.client.0.smithi092.stdout:0/309: creat d7/dc/d28/d54/d1e/d5d/f68 x:0 0 0 2024-05-06T12:14:31.351 INFO:tasks.workunit.client.0.smithi092.stdout:1/235: mkdir d3/d10/d13/d19/d21/d47/d4f 0 2024-05-06T12:14:31.352 INFO:tasks.workunit.client.0.smithi092.stdout:1/236: chown d3/d10/d13/d19/f1e 1 1 2024-05-06T12:14:31.352 INFO:tasks.workunit.client.0.smithi092.stdout:8/330: creat d9/d21/d72/f7e x:0 0 0 2024-05-06T12:14:31.354 INFO:tasks.workunit.client.0.smithi092.stdout:4/258: mknod d9/df/d48/c53 0 2024-05-06T12:14:31.355 INFO:tasks.workunit.client.0.smithi092.stdout:7/475: unlink d9/de/d10/d17/d99/l33 0 2024-05-06T12:14:31.355 INFO:tasks.workunit.client.0.smithi092.stdout:9/221: link d16/d1c/c24 d16/d2c/d2f/c44 0 2024-05-06T12:14:31.356 INFO:tasks.workunit.client.0.smithi092.stdout:7/476: stat d9/de/d10/d17/d21/d2e 0 2024-05-06T12:14:31.356 INFO:tasks.workunit.client.0.smithi092.stdout:3/277: mknod d8/db/d4c/d3d/d48/c57 0 2024-05-06T12:14:31.362 INFO:tasks.workunit.client.0.smithi092.stdout:1/237: creat d3/d10/d13/d19/d21/d47/f50 x:0 0 0 2024-05-06T12:14:31.363 INFO:tasks.workunit.client.0.smithi092.stdout:1/238: dread - d3/d10/d13/d19/d21/d47/f50 zero size 2024-05-06T12:14:31.364 INFO:tasks.workunit.client.0.smithi092.stdout:8/331: creat d9/d21/d37/f7f x:0 0 0 2024-05-06T12:14:31.367 INFO:tasks.workunit.client.0.smithi092.stdout:0/310: dread f6 [0,4194304] 0 2024-05-06T12:14:31.369 INFO:tasks.workunit.client.0.smithi092.stdout:7/477: write d9/de/d1f/d37/d48/d65/f73 [249506,68819] 0 2024-05-06T12:14:31.370 INFO:tasks.workunit.client.0.smithi092.stdout:6/325: write d4/d4c/f4f [119079,73033] 0 2024-05-06T12:14:31.371 INFO:tasks.workunit.client.0.smithi092.stdout:3/278: dread d8/d24/d27/f2f [0,4194304] 0 2024-05-06T12:14:31.372 INFO:tasks.workunit.client.0.smithi092.stdout:2/211: creat df/d17/f42 x:0 0 0 2024-05-06T12:14:31.373 INFO:tasks.workunit.client.0.smithi092.stdout:5/299: creat d4/d5/d10/d12/d1f/d20/f76 x:0 0 0 2024-05-06T12:14:31.374 INFO:tasks.workunit.client.0.smithi092.stdout:1/239: write d3/f3d [680023,121899] 0 2024-05-06T12:14:31.376 INFO:tasks.workunit.client.0.smithi092.stdout:1/240: write d3/d10/d13/d19/f1c [470842,117211] 0 2024-05-06T12:14:31.377 INFO:tasks.workunit.client.0.smithi092.stdout:9/222: creat d16/d2c/d2f/f45 x:0 0 0 2024-05-06T12:14:31.378 INFO:tasks.workunit.client.0.smithi092.stdout:9/223: chown d16/d2c/d2f/d3d/f3c 49145401 1 2024-05-06T12:14:31.380 INFO:tasks.workunit.client.0.smithi092.stdout:6/326: rename d4/d20/d23 to d4/d20/d23/d6b 22 2024-05-06T12:14:31.381 INFO:tasks.workunit.client.0.smithi092.stdout:6/327: fdatasync d4/fb 0 2024-05-06T12:14:31.383 INFO:tasks.workunit.client.0.smithi092.stdout:9/224: dread d16/d2c/d2f/f36 [0,4194304] 0 2024-05-06T12:14:31.383 INFO:tasks.workunit.client.0.smithi092.stdout:5/300: unlink d4/d34/d28/f39 0 2024-05-06T12:14:31.384 INFO:tasks.workunit.client.0.smithi092.stdout:8/332: getdents d9/d21/d54/d79 0 2024-05-06T12:14:31.391 INFO:tasks.workunit.client.0.smithi092.stdout:1/241: rmdir d3/d10/d13/d19 39 2024-05-06T12:14:31.393 INFO:tasks.workunit.client.0.smithi092.stdout:0/311: mknod d7/dc/d28/d54/d12/c69 0 2024-05-06T12:14:31.394 INFO:tasks.workunit.client.0.smithi092.stdout:0/312: read d7/dc/d28/d54/d1e/d26/d3a/f52 [2960520,36738] 0 2024-05-06T12:14:31.394 INFO:tasks.workunit.client.0.smithi092.stdout:3/279: rename d8/db/d3f to d8/db/d4c/d3d/d58 0 2024-05-06T12:14:31.396 INFO:tasks.workunit.client.0.smithi092.stdout:2/212: fsync df/d17/f22 0 2024-05-06T12:14:31.397 INFO:tasks.workunit.client.0.smithi092.stdout:6/328: mkdir d4/d4c/d52/d6c 0 2024-05-06T12:14:31.398 INFO:tasks.workunit.client.0.smithi092.stdout:6/329: read d4/d24/f3e [423052,91907] 0 2024-05-06T12:14:31.399 INFO:tasks.workunit.client.0.smithi092.stdout:9/225: creat d16/d2c/d3f/f46 x:0 0 0 2024-05-06T12:14:31.399 INFO:tasks.workunit.client.0.smithi092.stdout:9/226: dread - d16/f2a zero size 2024-05-06T12:14:31.401 INFO:tasks.workunit.client.0.smithi092.stdout:6/330: write d4/d20/d23/d30/f5e [262353,37418] 0 2024-05-06T12:14:31.402 INFO:tasks.workunit.client.0.smithi092.stdout:3/280: dwrite d8/db/d14/f50 [0,4194304] 0 2024-05-06T12:14:31.407 INFO:tasks.workunit.client.0.smithi092.stdout:5/301: mkdir d4/d5/d10/d12/d77 0 2024-05-06T12:14:31.417 INFO:tasks.workunit.client.0.smithi092.stdout:8/333: creat d9/d21/d37/d46/d5c/d71/f80 x:0 0 0 2024-05-06T12:14:31.418 INFO:tasks.workunit.client.0.smithi092.stdout:8/334: write d9/d21/d54/f63 [3571967,11748] 0 2024-05-06T12:14:31.419 INFO:tasks.workunit.client.0.smithi092.stdout:4/259: link d9/l13 d9/d27/l54 0 2024-05-06T12:14:31.421 INFO:tasks.workunit.client.0.smithi092.stdout:4/260: write d9/d27/f50 [938663,77410] 0 2024-05-06T12:14:31.422 INFO:tasks.workunit.client.0.smithi092.stdout:0/313: symlink d7/dc/l6a 0 2024-05-06T12:14:31.423 INFO:tasks.workunit.client.0.smithi092.stdout:0/314: fdatasync d7/dc/d28/d54/d1e/d46/f67 0 2024-05-06T12:14:31.424 INFO:tasks.workunit.client.0.smithi092.stdout:0/315: write d7/dc/d28/d54/d12/f1b [3223537,116494] 0 2024-05-06T12:14:31.427 INFO:tasks.workunit.client.0.smithi092.stdout:2/213: dread df/f12 [0,4194304] 0 2024-05-06T12:14:31.428 INFO:tasks.workunit.client.0.smithi092.stdout:2/214: stat c2 0 2024-05-06T12:14:31.431 INFO:tasks.workunit.client.0.smithi092.stdout:1/242: creat d3/d10/d13/d19/d34/f51 x:0 0 0 2024-05-06T12:14:31.432 INFO:tasks.workunit.client.0.smithi092.stdout:4/261: symlink d9/d27/l55 0 2024-05-06T12:14:31.432 INFO:tasks.workunit.client.0.smithi092.stdout:1/243: dread - d3/d10/d13/d19/d34/f51 zero size 2024-05-06T12:14:31.437 INFO:tasks.workunit.client.0.smithi092.stdout:9/227: rename cf to d16/d2c/d2f/d3d/d33/c47 0 2024-05-06T12:14:31.441 INFO:tasks.workunit.client.0.smithi092.stdout:5/302: mkdir d4/d5/d10/d12/d1f/d78 0 2024-05-06T12:14:31.441 INFO:tasks.workunit.client.0.smithi092.stdout:7/478: creat d9/de/d10/d17/d21/d2e/fa1 x:0 0 0 2024-05-06T12:14:31.443 INFO:tasks.workunit.client.0.smithi092.stdout:4/262: unlink d9/dd/f16 0 2024-05-06T12:14:31.445 INFO:tasks.workunit.client.0.smithi092.stdout:0/316: symlink d7/dc/d28/d54/d1e/d26/d4e/l6b 0 2024-05-06T12:14:31.447 INFO:tasks.workunit.client.0.smithi092.stdout:0/317: write d7/dc/d28/d54/d1e/f22 [378217,15175] 0 2024-05-06T12:14:31.454 INFO:tasks.workunit.client.0.smithi092.stdout:4/263: creat d9/df/d48/f56 x:0 0 0 2024-05-06T12:14:31.458 INFO:tasks.workunit.client.0.smithi092.stdout:1/244: mknod d3/d10/d13/d19/c52 0 2024-05-06T12:14:31.458 INFO:tasks.workunit.client.0.smithi092.stdout:7/479: dwrite f1 [4194304,4194304] 0 2024-05-06T12:14:31.458 INFO:tasks.workunit.client.0.smithi092.stdout:1/245: write d3/d42/f43 [187464,72559] 0 2024-05-06T12:14:31.463 INFO:tasks.workunit.client.0.smithi092.stdout:5/303: getdents d4/d5/d10/d12/d1f/d78 0 2024-05-06T12:14:31.468 INFO:tasks.workunit.client.0.smithi092.stdout:2/215: getdents df/d17/d1d/d33 0 2024-05-06T12:14:31.469 INFO:tasks.workunit.client.0.smithi092.stdout:2/216: readlink df/d15/l28 0 2024-05-06T12:14:31.469 INFO:tasks.workunit.client.0.smithi092.stdout:2/217: readlink df/d17/d1d/l40 0 2024-05-06T12:14:31.470 INFO:tasks.workunit.client.0.smithi092.stdout:2/218: stat df/d17/d1d/d33 0 2024-05-06T12:14:31.472 INFO:tasks.workunit.client.0.smithi092.stdout:8/335: rename d9/dc/f16 to d9/d21/f81 0 2024-05-06T12:14:31.473 INFO:tasks.workunit.client.0.smithi092.stdout:7/480: fsync d9/de/d1f/f61 0 2024-05-06T12:14:31.476 INFO:tasks.workunit.client.0.smithi092.stdout:2/219: rmdir df/d17/d1d 39 2024-05-06T12:14:31.476 INFO:tasks.workunit.client.0.smithi092.stdout:2/220: stat df/f1e 0 2024-05-06T12:14:31.477 INFO:tasks.workunit.client.0.smithi092.stdout:5/304: dread d4/d34/f56 [0,4194304] 0 2024-05-06T12:14:31.479 INFO:tasks.workunit.client.0.smithi092.stdout:3/281: write d8/db/d14/f18 [899807,105790] 0 2024-05-06T12:14:31.483 INFO:tasks.workunit.client.0.smithi092.stdout:0/318: link d7/l60 d7/dc/d31/l6c 0 2024-05-06T12:14:31.488 INFO:tasks.workunit.client.0.smithi092.stdout:1/246: getdents d3/d10/d13/d19/d34 0 2024-05-06T12:14:31.488 INFO:tasks.workunit.client.0.smithi092.stdout:1/247: stat d3/d10/d13/d19/c52 0 2024-05-06T12:14:31.489 INFO:tasks.workunit.client.0.smithi092.stdout:0/319: mknod d7/dc/d28/d54/d1e/d46/c6d 0 2024-05-06T12:14:31.491 INFO:tasks.workunit.client.0.smithi092.stdout:3/282: mknod d8/d24/d33/c59 0 2024-05-06T12:14:31.493 INFO:tasks.workunit.client.0.smithi092.stdout:1/248: creat d3/d10/f53 x:0 0 0 2024-05-06T12:14:31.493 INFO:tasks.workunit.client.0.smithi092.stdout:5/305: link f0 d4/d34/d28/d4e/d67/f79 0 2024-05-06T12:14:31.496 INFO:tasks.workunit.client.0.smithi092.stdout:9/228: dwrite d16/d2c/d2f/d3d/d33/f3b [0,4194304] 0 2024-05-06T12:14:31.497 INFO:tasks.workunit.client.0.smithi092.stdout:3/283: mkdir d8/db/d14/d5a 0 2024-05-06T12:14:31.498 INFO:tasks.workunit.client.0.smithi092.stdout:0/320: mkdir d7/dc/d28/d54/d1e/d46/d63/d6e 0 2024-05-06T12:14:31.500 INFO:tasks.workunit.client.0.smithi092.stdout:1/249: mknod d3/d10/d13/d19/d21/c54 0 2024-05-06T12:14:31.502 INFO:tasks.workunit.client.0.smithi092.stdout:5/306: mknod d4/d5/d10/d12/d77/c7a 0 2024-05-06T12:14:31.503 INFO:tasks.workunit.client.0.smithi092.stdout:6/331: rmdir d4 39 2024-05-06T12:14:31.505 INFO:tasks.workunit.client.0.smithi092.stdout:4/264: truncate d9/f43 647448 0 2024-05-06T12:14:31.507 INFO:tasks.workunit.client.0.smithi092.stdout:3/284: dwrite f3 [0,4194304] 0 2024-05-06T12:14:31.507 INFO:tasks.workunit.client.0.smithi092.stdout:8/336: rmdir d9/d21 39 2024-05-06T12:14:31.510 INFO:tasks.workunit.client.0.smithi092.stdout:0/321: truncate d7/dc/d28/d54/f11 3608011 0 2024-05-06T12:14:31.512 INFO:tasks.workunit.client.0.smithi092.stdout:1/250: dwrite d3/d10/d13/f1f [0,4194304] 0 2024-05-06T12:14:31.517 INFO:tasks.workunit.client.0.smithi092.stdout:7/481: write d9/dd/f43 [793546,124310] 0 2024-05-06T12:14:31.519 INFO:tasks.workunit.client.0.smithi092.stdout:3/285: mkdir d8/d24/d33/d39/d5b 0 2024-05-06T12:14:31.520 INFO:tasks.workunit.client.0.smithi092.stdout:8/337: readlink d9/d21/d37/l43 0 2024-05-06T12:14:31.522 INFO:tasks.workunit.client.0.smithi092.stdout:7/482: fsync d9/f5a 0 2024-05-06T12:14:31.522 INFO:tasks.workunit.client.0.smithi092.stdout:1/251: readlink d3/d10/l4c 0 2024-05-06T12:14:31.523 INFO:tasks.workunit.client.0.smithi092.stdout:0/322: dwrite d7/dc/d28/d54/d1e/d39/f57 [0,4194304] 0 2024-05-06T12:14:31.525 INFO:tasks.workunit.client.0.smithi092.stdout:5/307: mknod d4/d5/d10/c7b 0 2024-05-06T12:14:31.527 INFO:tasks.workunit.client.0.smithi092.stdout:4/265: fdatasync d9/f24 0 2024-05-06T12:14:31.527 INFO:tasks.workunit.client.0.smithi092.stdout:8/338: write d9/d21/d37/f7f [167426,829] 0 2024-05-06T12:14:31.531 INFO:tasks.workunit.client.0.smithi092.stdout:4/266: write d9/d27/f41 [528380,23670] 0 2024-05-06T12:14:31.532 INFO:tasks.workunit.client.0.smithi092.stdout:3/286: chown d8/d24/d33/d4b/c25 206823 1 2024-05-06T12:14:31.533 INFO:tasks.workunit.client.0.smithi092.stdout:6/332: mkdir d4/d24/d38/d5d/d6d 0 2024-05-06T12:14:31.535 INFO:tasks.workunit.client.0.smithi092.stdout:2/221: dwrite df/f37 [0,4194304] 0 2024-05-06T12:14:31.535 INFO:tasks.workunit.client.0.smithi092.stdout:9/229: truncate d16/d1c/f30 1686835 0 2024-05-06T12:14:31.539 INFO:tasks.workunit.client.0.smithi092.stdout:4/267: fdatasync d9/d27/d4a/f4c 0 2024-05-06T12:14:31.540 INFO:tasks.workunit.client.0.smithi092.stdout:2/222: mknod df/d15/c43 0 2024-05-06T12:14:31.542 INFO:tasks.workunit.client.0.smithi092.stdout:2/223: chown df/d17/l29 3 1 2024-05-06T12:14:31.542 INFO:tasks.workunit.client.0.smithi092.stdout:6/333: creat d4/d24/d26/f6e x:0 0 0 2024-05-06T12:14:31.546 INFO:tasks.workunit.client.0.smithi092.stdout:8/339: rename d9/d5a/f67 to d9/d21/d54/d79/f82 0 2024-05-06T12:14:31.548 INFO:tasks.workunit.client.0.smithi092.stdout:1/252: creat d3/d10/d13/d46/f55 x:0 0 0 2024-05-06T12:14:31.549 INFO:tasks.workunit.client.0.smithi092.stdout:2/224: dwrite df/f37 [0,4194304] 0 2024-05-06T12:14:31.550 INFO:tasks.workunit.client.0.smithi092.stdout:4/268: write d9/df/f14 [2341795,63168] 0 2024-05-06T12:14:31.551 INFO:tasks.workunit.client.0.smithi092.stdout:7/483: creat d9/de/fa2 x:0 0 0 2024-05-06T12:14:31.553 INFO:tasks.workunit.client.0.smithi092.stdout:5/308: getdents d4/d5/d10/d12/d1f/d78 0 2024-05-06T12:14:31.560 INFO:tasks.workunit.client.0.smithi092.stdout:6/334: rename d4/d4c/f4f to d4/d20/d23/d30/f6f 0 2024-05-06T12:14:31.561 INFO:tasks.workunit.client.0.smithi092.stdout:4/269: dwrite d9/f43 [0,4194304] 0 2024-05-06T12:14:31.562 INFO:tasks.workunit.client.0.smithi092.stdout:6/335: read - d4/d24/d26/f4b zero size 2024-05-06T12:14:31.563 INFO:tasks.workunit.client.0.smithi092.stdout:1/253: unlink d3/f12 0 2024-05-06T12:14:31.564 INFO:tasks.workunit.client.0.smithi092.stdout:1/254: write d3/f26 [976126,27445] 0 2024-05-06T12:14:31.569 INFO:tasks.workunit.client.0.smithi092.stdout:5/309: unlink d4/d34/d28/d4e/f58 0 2024-05-06T12:14:31.570 INFO:tasks.workunit.client.0.smithi092.stdout:0/323: truncate d7/dc/d28/d54/d1e/d39/f57 606785 0 2024-05-06T12:14:31.571 INFO:tasks.workunit.client.0.smithi092.stdout:3/287: rmdir d8/db/d4c/d3d/d58 0 2024-05-06T12:14:31.574 INFO:tasks.workunit.client.0.smithi092.stdout:8/340: creat d9/d21/d37/d74/f83 x:0 0 0 2024-05-06T12:14:31.576 INFO:tasks.workunit.client.0.smithi092.stdout:4/270: rmdir d9/df/d48 39 2024-05-06T12:14:31.577 INFO:tasks.workunit.client.0.smithi092.stdout:1/255: mknod d3/d10/d13/d19/d21/c56 0 2024-05-06T12:14:31.579 INFO:tasks.workunit.client.0.smithi092.stdout:7/484: creat d9/d62/d9a/fa3 x:0 0 0 2024-05-06T12:14:31.580 INFO:tasks.workunit.client.0.smithi092.stdout:9/230: symlink d16/d2c/d2f/d3d/d33/d37/d40/d43/l48 0 2024-05-06T12:14:31.581 INFO:tasks.workunit.client.0.smithi092.stdout:2/225: dread df/d17/d1d/f2e [0,4194304] 0 2024-05-06T12:14:31.582 INFO:tasks.workunit.client.0.smithi092.stdout:5/310: symlink d4/d34/d28/d4e/d67/l7c 0 2024-05-06T12:14:31.583 INFO:tasks.workunit.client.0.smithi092.stdout:0/324: mkdir d7/dc/d6f 0 2024-05-06T12:14:31.585 INFO:tasks.workunit.client.0.smithi092.stdout:5/311: dread d4/d34/f56 [0,4194304] 0 2024-05-06T12:14:31.585 INFO:tasks.workunit.client.0.smithi092.stdout:4/271: chown d9/df/l3c 416747 1 2024-05-06T12:14:31.586 INFO:tasks.workunit.client.0.smithi092.stdout:5/312: fdatasync d4/d5/d30/f61 0 2024-05-06T12:14:31.587 INFO:tasks.workunit.client.0.smithi092.stdout:5/313: write d4/d5/d10/d12/d1f/d20/f76 [936183,55204] 0 2024-05-06T12:14:31.588 INFO:tasks.workunit.client.0.smithi092.stdout:6/336: link d4/f1a d4/d4c/d52/f70 0 2024-05-06T12:14:31.589 INFO:tasks.workunit.client.0.smithi092.stdout:1/256: chown d3/d10/d13/d19/d21/l28 248 1 2024-05-06T12:14:31.591 INFO:tasks.workunit.client.0.smithi092.stdout:9/231: fdatasync d16/d1c/f30 0 2024-05-06T12:14:31.596 INFO:tasks.workunit.client.0.smithi092.stdout:1/257: dwrite d3/d10/f53 [0,4194304] 0 2024-05-06T12:14:31.598 INFO:tasks.workunit.client.0.smithi092.stdout:1/258: fdatasync d3/d10/d13/d19/f40 0 2024-05-06T12:14:31.598 INFO:tasks.workunit.client.0.smithi092.stdout:4/272: mknod d9/d2b/c57 0 2024-05-06T12:14:31.599 INFO:tasks.workunit.client.0.smithi092.stdout:7/485: dread d9/de/d10/f2b [0,4194304] 0 2024-05-06T12:14:31.601 INFO:tasks.workunit.client.0.smithi092.stdout:4/273: chown d9/l13 219 1 2024-05-06T12:14:31.607 INFO:tasks.workunit.client.0.smithi092.stdout:5/314: dwrite d4/d5/d10/d31/d53/d64/f25 [0,4194304] 0 2024-05-06T12:14:31.608 INFO:tasks.workunit.client.0.smithi092.stdout:3/288: link d8/d1b/f2b d8/db/d4c/f5c 0 2024-05-06T12:14:31.610 INFO:tasks.workunit.client.0.smithi092.stdout:5/315: stat d4/d34/d28/d4e/d67/c75 0 2024-05-06T12:14:31.612 INFO:tasks.workunit.client.0.smithi092.stdout:5/316: truncate d4/d34/d28/d4e/d4b/f4d 1021595 0 2024-05-06T12:14:31.614 INFO:tasks.workunit.client.0.smithi092.stdout:5/317: chown d4/d34/d28/f72 0 1 2024-05-06T12:14:31.616 INFO:tasks.workunit.client.0.smithi092.stdout:5/318: write d4/d34/d28/d4e/f4f [388371,77515] 0 2024-05-06T12:14:31.617 INFO:tasks.workunit.client.0.smithi092.stdout:6/337: creat d4/d4c/d52/d62/f71 x:0 0 0 2024-05-06T12:14:31.619 INFO:tasks.workunit.client.0.smithi092.stdout:0/325: creat d7/f70 x:0 0 0 2024-05-06T12:14:31.621 INFO:tasks.workunit.client.0.smithi092.stdout:1/259: mkdir d3/d10/d13/d19/d57 0 2024-05-06T12:14:31.623 INFO:tasks.workunit.client.0.smithi092.stdout:2/226: dwrite fe [0,4194304] 0 2024-05-06T12:14:31.625 INFO:tasks.workunit.client.0.smithi092.stdout:6/338: mknod d4/d24/d4e/c72 0 2024-05-06T12:14:31.626 INFO:tasks.workunit.client.0.smithi092.stdout:4/274: rename d9/lb to d9/df/d48/l58 0 2024-05-06T12:14:31.626 INFO:tasks.workunit.client.0.smithi092.stdout:6/339: write d4/d24/d26/f6e [265053,107529] 0 2024-05-06T12:14:31.628 INFO:tasks.workunit.client.0.smithi092.stdout:3/289: link d8/db/d4c/d40/f52 d8/db/d4c/d3d/f5d 0 2024-05-06T12:14:31.629 INFO:tasks.workunit.client.0.smithi092.stdout:0/326: rmdir d7/dc/d28/d54 39 2024-05-06T12:14:31.631 INFO:tasks.workunit.client.0.smithi092.stdout:2/227: creat df/d17/d1d/d33/f44 x:0 0 0 2024-05-06T12:14:31.633 INFO:tasks.workunit.client.0.smithi092.stdout:5/319: creat d4/f7d x:0 0 0 2024-05-06T12:14:31.634 INFO:tasks.workunit.client.0.smithi092.stdout:1/260: mkdir d3/d10/d13/d19/d21/d47/d4f/d58 0 2024-05-06T12:14:31.635 INFO:tasks.workunit.client.0.smithi092.stdout:3/290: rmdir d8/d24/d33 39 2024-05-06T12:14:31.637 INFO:tasks.workunit.client.0.smithi092.stdout:2/228: truncate df/f12 3086260 0 2024-05-06T12:14:31.639 INFO:tasks.workunit.client.0.smithi092.stdout:2/229: readlink df/d17/l20 0 2024-05-06T12:14:31.639 INFO:tasks.workunit.client.0.smithi092.stdout:3/291: mkdir d8/db/d14/d5e 0 2024-05-06T12:14:31.640 INFO:tasks.workunit.client.0.smithi092.stdout:7/486: dwrite d9/de/d1f/f4c [0,4194304] 0 2024-05-06T12:14:31.641 INFO:tasks.workunit.client.0.smithi092.stdout:4/275: dwrite d9/d27/f49 [0,4194304] 0 2024-05-06T12:14:31.643 INFO:tasks.workunit.client.0.smithi092.stdout:3/292: write f3 [1760174,80303] 0 2024-05-06T12:14:31.646 INFO:tasks.workunit.client.0.smithi092.stdout:1/261: dread d3/d10/d13/f3b [0,4194304] 0 2024-05-06T12:14:31.646 INFO:tasks.workunit.client.0.smithi092.stdout:5/320: truncate d4/d5/d10/d31/d53/d64/f25 4626189 0 2024-05-06T12:14:31.649 INFO:tasks.workunit.client.0.smithi092.stdout:4/276: mkdir d9/d27/d4a/d59 0 2024-05-06T12:14:31.654 INFO:tasks.workunit.client.0.smithi092.stdout:3/293: mknod d8/d1b/c5f 0 2024-05-06T12:14:31.654 INFO:tasks.workunit.client.0.smithi092.stdout:4/277: fsync d9/d27/f4d 0 2024-05-06T12:14:31.655 INFO:tasks.workunit.client.0.smithi092.stdout:5/321: symlink d4/d34/l7e 0 2024-05-06T12:14:31.655 INFO:tasks.workunit.client.0.smithi092.stdout:0/327: dread d7/dc/d28/d54/d1e/d26/d3a/f52 [0,4194304] 0 2024-05-06T12:14:31.656 INFO:tasks.workunit.client.0.smithi092.stdout:3/294: readlink l6 0 2024-05-06T12:14:31.656 INFO:tasks.workunit.client.0.smithi092.stdout:2/230: dwrite fd [4194304,4194304] 0 2024-05-06T12:14:31.662 INFO:tasks.workunit.client.0.smithi092.stdout:5/322: truncate d4/f68 1156088 0 2024-05-06T12:14:31.664 INFO:tasks.workunit.client.0.smithi092.stdout:3/295: unlink d8/db/f11 0 2024-05-06T12:14:31.668 INFO:tasks.workunit.client.0.smithi092.stdout:6/340: dwrite d4/d20/d23/f28 [0,4194304] 0 2024-05-06T12:14:31.668 INFO:tasks.workunit.client.0.smithi092.stdout:2/231: truncate df/d17/f22 270759 0 2024-05-06T12:14:31.669 INFO:tasks.workunit.client.0.smithi092.stdout:3/296: readlink d8/db/d4c/d40/l53 0 2024-05-06T12:14:31.670 INFO:tasks.workunit.client.0.smithi092.stdout:5/323: stat d4/d5/d10/d12/d77/c7a 0 2024-05-06T12:14:31.671 INFO:tasks.workunit.client.0.smithi092.stdout:7/487: truncate d9/de/d1f/f4c 3929947 0 2024-05-06T12:14:31.672 INFO:tasks.workunit.client.0.smithi092.stdout:1/262: dwrite d3/d10/d13/d19/f40 [0,4194304] 0 2024-05-06T12:14:31.673 INFO:tasks.workunit.client.0.smithi092.stdout:5/324: fdatasync d4/d34/d28/f3a 0 2024-05-06T12:14:31.673 INFO:tasks.workunit.client.0.smithi092.stdout:3/297: fdatasync d8/d24/d27/f2e 0 2024-05-06T12:14:31.678 INFO:tasks.workunit.client.0.smithi092.stdout:0/328: dwrite d7/dc/d28/d54/d1e/d26/f2b [0,4194304] 0 2024-05-06T12:14:31.681 INFO:tasks.workunit.client.0.smithi092.stdout:2/232: mkdir df/d17/d1d/d33/d3c/d45 0 2024-05-06T12:14:31.681 INFO:tasks.workunit.client.0.smithi092.stdout:0/329: write d7/f4f [852761,28053] 0 2024-05-06T12:14:31.682 INFO:tasks.workunit.client.0.smithi092.stdout:0/330: fdatasync f1 0 2024-05-06T12:14:31.682 INFO:tasks.workunit.client.0.smithi092.stdout:5/325: stat d4/d5/d10/d12/d77 0 2024-05-06T12:14:31.683 INFO:tasks.workunit.client.0.smithi092.stdout:7/488: truncate d9/de/d1f/d37/d48/f7e 782843 0 2024-05-06T12:14:31.685 INFO:tasks.workunit.client.0.smithi092.stdout:4/278: link d9/df/d48/l58 d9/df/d48/l5a 0 2024-05-06T12:14:31.688 INFO:tasks.workunit.client.0.smithi092.stdout:7/489: write f1 [7759049,119460] 0 2024-05-06T12:14:31.690 INFO:tasks.workunit.client.0.smithi092.stdout:6/341: dwrite d4/d20/f1c [4194304,4194304] 0 2024-05-06T12:14:31.691 INFO:tasks.workunit.client.0.smithi092.stdout:4/279: mkdir d9/d2b/d5b 0 2024-05-06T12:14:31.691 INFO:tasks.workunit.client.0.smithi092.stdout:7/490: read d9/de/d1f/f4c [3246018,59325] 0 2024-05-06T12:14:31.697 INFO:tasks.workunit.client.0.smithi092.stdout:0/331: mkdir d7/dc/d28/d54/d1e/d39/d71 0 2024-05-06T12:14:31.700 INFO:tasks.workunit.client.0.smithi092.stdout:0/332: fsync d7/d43/f49 0 2024-05-06T12:14:31.702 INFO:tasks.workunit.client.0.smithi092.stdout:4/280: dwrite d9/df/f14 [0,4194304] 0 2024-05-06T12:14:31.702 INFO:tasks.workunit.client.0.smithi092.stdout:6/342: rename d4/d4c/d52/l4d to d4/d24/l73 0 2024-05-06T12:14:31.704 INFO:tasks.workunit.client.0.smithi092.stdout:2/233: rmdir df/d17/d1d/d33/d3c/d45 0 2024-05-06T12:14:31.705 INFO:tasks.workunit.client.0.smithi092.stdout:4/281: readlink d9/l29 0 2024-05-06T12:14:31.708 INFO:tasks.workunit.client.0.smithi092.stdout:7/491: mkdir d9/de/da4 0 2024-05-06T12:14:31.710 INFO:tasks.workunit.client.0.smithi092.stdout:0/333: fsync d7/dc/d28/d54/d1e/d5d/f68 0 2024-05-06T12:14:31.710 INFO:tasks.workunit.client.0.smithi092.stdout:4/282: chown f5 4074098 1 2024-05-06T12:14:31.710 INFO:tasks.workunit.client.0.smithi092.stdout:7/492: readlink d9/lf 0 2024-05-06T12:14:31.710 INFO:tasks.workunit.client.0.smithi092.stdout:6/343: write d4/d4c/d52/f4a [5425367,30003] 0 2024-05-06T12:14:31.712 INFO:tasks.workunit.client.0.smithi092.stdout:1/263: dwrite d3/d10/d13/f3b [0,4194304] 0 2024-05-06T12:14:31.714 INFO:tasks.workunit.client.0.smithi092.stdout:5/326: link d4/d34/d28/d4e/d67/c75 d4/d34/d29/d54/c7f 0 2024-05-06T12:14:31.714 INFO:tasks.workunit.client.0.smithi092.stdout:3/298: dwrite d8/db/d4c/f5c [0,4194304] 0 2024-05-06T12:14:31.719 INFO:tasks.workunit.client.0.smithi092.stdout:1/264: rename d3/d10 to d3/d10/d13/d46/d59 22 2024-05-06T12:14:31.722 INFO:tasks.workunit.client.0.smithi092.stdout:2/234: link df/d17/d1d/d33/f35 df/d17/d1d/f46 0 2024-05-06T12:14:31.722 INFO:tasks.workunit.client.0.smithi092.stdout:6/344: symlink d4/d24/d38/d5d/d6d/l74 0 2024-05-06T12:14:31.725 INFO:tasks.workunit.client.0.smithi092.stdout:3/299: rename d8/fa to d8/db/d4c/d3d/f60 0 2024-05-06T12:14:31.727 INFO:tasks.workunit.client.0.smithi092.stdout:5/327: dread d4/d5/d30/f61 [0,4194304] 0 2024-05-06T12:14:31.729 INFO:tasks.workunit.client.0.smithi092.stdout:4/283: link d9/c10 d9/c5c 0 2024-05-06T12:14:31.730 INFO:tasks.workunit.client.0.smithi092.stdout:6/345: truncate d4/d24/d26/f6e 1316271 0 2024-05-06T12:14:31.732 INFO:tasks.workunit.client.0.smithi092.stdout:0/334: dread d7/dc/d28/d54/d1e/d26/d3a/f52 [0,4194304] 0 2024-05-06T12:14:31.732 INFO:tasks.workunit.client.0.smithi092.stdout:2/235: truncate df/d17/d1d/d33/f35 232243 0 2024-05-06T12:14:31.733 INFO:tasks.workunit.client.0.smithi092.stdout:6/346: dread - d4/d24/d38/f69 zero size 2024-05-06T12:14:31.734 INFO:tasks.workunit.client.0.smithi092.stdout:1/265: creat d3/d10/d13/d19/f5a x:0 0 0 2024-05-06T12:14:31.735 INFO:tasks.workunit.client.0.smithi092.stdout:2/236: readlink df/d1b/l31 0 2024-05-06T12:14:31.738 INFO:tasks.workunit.client.0.smithi092.stdout:6/347: dread - d4/d24/d26/f4b zero size 2024-05-06T12:14:31.738 INFO:tasks.workunit.client.0.smithi092.stdout:7/493: dwrite d9/de/d10/d17/f8f [0,4194304] 0 2024-05-06T12:14:31.738 INFO:tasks.workunit.client.0.smithi092.stdout:0/335: write d7/d43/f61 [474105,21648] 0 2024-05-06T12:14:31.739 INFO:tasks.workunit.client.0.smithi092.stdout:1/266: fsync d3/d10/d13/d19/f1e 0 2024-05-06T12:14:31.742 INFO:tasks.workunit.client.0.smithi092.stdout:5/328: dwrite d4/d5/f2a [4194304,4194304] 0 2024-05-06T12:14:31.744 INFO:tasks.workunit.client.0.smithi092.stdout:2/237: creat df/d17/d1d/d33/f47 x:0 0 0 2024-05-06T12:14:31.745 INFO:tasks.workunit.client.0.smithi092.stdout:6/348: read d4/f11 [752702,83312] 0 2024-05-06T12:14:31.745 INFO:tasks.workunit.client.0.smithi092.stdout:0/336: fdatasync d7/dc/d28/d54/d1e/f41 0 2024-05-06T12:14:31.746 INFO:tasks.workunit.client.0.smithi092.stdout:2/238: dread - df/d15/f36 zero size 2024-05-06T12:14:31.751 INFO:tasks.workunit.client.0.smithi092.stdout:7/494: dwrite d9/de/d1f/f35 [0,4194304] 0 2024-05-06T12:14:31.752 INFO:tasks.workunit.client.0.smithi092.stdout:1/267: truncate d3/d10/d13/d19/d21/d47/f50 541316 0 2024-05-06T12:14:31.754 INFO:tasks.workunit.client.0.smithi092.stdout:1/268: chown d3/d42 84540357 1 2024-05-06T12:14:31.755 INFO:tasks.workunit.client.0.smithi092.stdout:4/284: truncate d9/df/f14 3195938 0 2024-05-06T12:14:31.758 INFO:tasks.workunit.client.0.smithi092.stdout:7/495: chown d9/de/d1f/f52 54203 1 2024-05-06T12:14:31.759 INFO:tasks.workunit.client.0.smithi092.stdout:0/337: creat d7/dc/d28/d54/d1e/d46/d63/f72 x:0 0 0 2024-05-06T12:14:31.762 INFO:tasks.workunit.client.0.smithi092.stdout:5/329: link d4/d5/d10/d12/d1f/d20/l2d d4/d34/d28/d4e/d4b/l80 0 2024-05-06T12:14:31.763 INFO:tasks.workunit.client.0.smithi092.stdout:1/269: chown d3/d10/d13/d19/d21/c2d 32785 1 2024-05-06T12:14:31.763 INFO:tasks.workunit.client.0.smithi092.stdout:7/496: chown d9/de/d10/d17/l6c 11 1 2024-05-06T12:14:31.765 INFO:tasks.workunit.client.0.smithi092.stdout:0/338: mknod d7/dc/d28/d54/d1e/c73 0 2024-05-06T12:14:31.765 INFO:tasks.workunit.client.0.smithi092.stdout:2/239: dwrite df/d17/f42 [0,4194304] 0 2024-05-06T12:14:31.767 INFO:tasks.workunit.client.0.smithi092.stdout:5/330: symlink d4/d5/d10/d12/l81 0 2024-05-06T12:14:31.767 INFO:tasks.workunit.client.0.smithi092.stdout:6/349: dwrite d4/d24/d26/f4b [0,4194304] 0 2024-05-06T12:14:31.770 INFO:tasks.workunit.client.0.smithi092.stdout:1/270: truncate d3/d10/d13/d19/d21/f2f 520446 0 2024-05-06T12:14:31.772 INFO:tasks.workunit.client.0.smithi092.stdout:6/350: readlink d4/d20/l47 0 2024-05-06T12:14:31.773 INFO:tasks.workunit.client.0.smithi092.stdout:2/240: creat df/d1b/f48 x:0 0 0 2024-05-06T12:14:31.776 INFO:tasks.workunit.client.0.smithi092.stdout:0/339: chown d7/dc/d28/d54/d1e/c48 75167968 1 2024-05-06T12:14:31.779 INFO:tasks.workunit.client.0.smithi092.stdout:7/497: dwrite d9/de/d1f/d37/d39/f59 [0,4194304] 0 2024-05-06T12:14:31.780 INFO:tasks.workunit.client.0.smithi092.stdout:5/331: rename d4/d5/d10/d12 to d4/d34/d28/d4e/d5d/d82 0 2024-05-06T12:14:31.782 INFO:tasks.workunit.client.0.smithi092.stdout:1/271: symlink d3/d10/d13/d19/d21/l5b 0 2024-05-06T12:14:31.786 INFO:tasks.workunit.client.0.smithi092.stdout:5/332: creat d4/d59/f83 x:0 0 0 2024-05-06T12:14:31.787 INFO:tasks.workunit.client.0.smithi092.stdout:1/272: symlink d3/d10/d13/d19/d21/d47/d4f/l5c 0 2024-05-06T12:14:31.789 INFO:tasks.workunit.client.0.smithi092.stdout:7/498: rename d9/dd/d15 to d9/da5 0 2024-05-06T12:14:31.791 INFO:tasks.workunit.client.0.smithi092.stdout:2/241: dwrite df/d15/f30 [0,4194304] 0 2024-05-06T12:14:31.791 INFO:tasks.workunit.client.0.smithi092.stdout:5/333: stat d4/d5/d10/d31/c69 0 2024-05-06T12:14:31.793 INFO:tasks.workunit.client.0.smithi092.stdout:7/499: write d9/fb [787634,89023] 0 2024-05-06T12:14:31.794 INFO:tasks.workunit.client.0.smithi092.stdout:1/273: unlink d3/d10/l30 0 2024-05-06T12:14:31.794 INFO:tasks.workunit.client.0.smithi092.stdout:0/340: dread d7/d43/f61 [0,4194304] 0 2024-05-06T12:14:31.796 INFO:tasks.workunit.client.0.smithi092.stdout:2/242: write df/d17/f42 [4261906,71511] 0 2024-05-06T12:14:31.798 INFO:tasks.workunit.client.0.smithi092.stdout:6/351: dread d4/d20/f1b [4194304,4194304] 0 2024-05-06T12:14:31.799 INFO:tasks.workunit.client.0.smithi092.stdout:7/500: mkdir d9/de/d10/d17/d99/da6 0 2024-05-06T12:14:31.800 INFO:tasks.workunit.client.0.smithi092.stdout:6/352: stat d4/d24/d54 0 2024-05-06T12:14:31.802 INFO:tasks.workunit.client.0.smithi092.stdout:0/341: rename d7/dc/d28/d64/c65 to d7/dc/d6f/c74 0 2024-05-06T12:14:31.803 INFO:tasks.workunit.client.0.smithi092.stdout:0/342: fdatasync d7/d43/f49 0 2024-05-06T12:14:31.805 INFO:tasks.workunit.client.0.smithi092.stdout:0/343: write d7/f9 [2478980,118858] 0 2024-05-06T12:14:31.806 INFO:tasks.workunit.client.0.smithi092.stdout:6/353: creat d4/d4c/f75 x:0 0 0 2024-05-06T12:14:31.812 INFO:tasks.workunit.client.0.smithi092.stdout:6/354: read d4/d4c/d52/f70 [1796333,42045] 0 2024-05-06T12:14:31.813 INFO:tasks.workunit.client.0.smithi092.stdout:1/274: dread d3/d10/d13/d19/d34/f36 [0,4194304] 0 2024-05-06T12:14:31.813 INFO:tasks.workunit.client.0.smithi092.stdout:7/501: creat d9/de/d10/d17/d99/da6/fa7 x:0 0 0 2024-05-06T12:14:31.815 INFO:tasks.workunit.client.0.smithi092.stdout:1/275: write d3/d10/d13/d19/f31 [545728,62975] 0 2024-05-06T12:14:31.819 INFO:tasks.workunit.client.0.smithi092.stdout:7/502: mknod d9/de/d1f/d37/d39/ca8 0 2024-05-06T12:14:31.821 INFO:tasks.workunit.client.0.smithi092.stdout:2/243: write df/f11 [983815,75978] 0 2024-05-06T12:14:31.822 INFO:tasks.workunit.client.0.smithi092.stdout:0/344: getdents d7/d43/d1f 0 2024-05-06T12:14:31.823 INFO:tasks.workunit.client.0.smithi092.stdout:5/334: dwrite d4/d34/d29/d54/f62 [0,4194304] 0 2024-05-06T12:14:31.824 INFO:tasks.workunit.client.0.smithi092.stdout:2/244: chown df/d17/d1d/d33/f44 60 1 2024-05-06T12:14:31.825 INFO:tasks.workunit.client.0.smithi092.stdout:0/345: dread - d7/dc/d28/d54/d1e/d5d/f68 zero size 2024-05-06T12:14:31.827 INFO:tasks.workunit.client.0.smithi092.stdout:7/503: rename d9/da5/l5c to d9/de/da4/la9 0 2024-05-06T12:14:31.828 INFO:tasks.workunit.client.0.smithi092.stdout:6/355: write d4/d24/f3e [1709520,60754] 0 2024-05-06T12:14:31.829 INFO:tasks.workunit.client.0.smithi092.stdout:0/346: mknod d7/dc/d28/d54/d1e/d46/c75 0 2024-05-06T12:14:31.832 INFO:tasks.workunit.client.0.smithi092.stdout:2/245: rename df/d17/d1d/d33/d3c/f3d to df/d17/d1d/f49 0 2024-05-06T12:14:31.833 INFO:tasks.workunit.client.0.smithi092.stdout:2/246: write df/d17/f2a [906144,123722] 0 2024-05-06T12:14:31.834 INFO:tasks.workunit.client.0.smithi092.stdout:5/335: dread d4/d5/d30/f46 [4194304,4194304] 0 2024-05-06T12:14:31.835 INFO:tasks.workunit.client.0.smithi092.stdout:1/276: link d3/f17 d3/d10/d13/d19/f5d 0 2024-05-06T12:14:31.836 INFO:tasks.workunit.client.0.smithi092.stdout:5/336: write d4/d5/d30/f46 [5179529,17914] 0 2024-05-06T12:14:31.837 INFO:tasks.workunit.client.0.smithi092.stdout:0/347: creat d7/dc/d28/d54/d1e/d5d/f76 x:0 0 0 2024-05-06T12:14:31.837 INFO:tasks.workunit.client.0.smithi092.stdout:1/277: stat d3/d10/d13/d19/c25 0 2024-05-06T12:14:31.838 INFO:tasks.workunit.client.0.smithi092.stdout:1/278: stat d3/d10/f45 0 2024-05-06T12:14:31.839 INFO:tasks.workunit.client.0.smithi092.stdout:6/356: rename d4/f5b to d4/f76 0 2024-05-06T12:14:31.840 INFO:tasks.workunit.client.0.smithi092.stdout:6/357: chown d4/d20/l47 25 1 2024-05-06T12:14:31.840 INFO:tasks.workunit.client.0.smithi092.stdout:5/337: creat d4/d34/d28/d4e/d4b/f84 x:0 0 0 2024-05-06T12:14:31.842 INFO:tasks.workunit.client.0.smithi092.stdout:5/338: chown d4/d34/d28/d4e/d5d/l2f 176558 1 2024-05-06T12:14:31.843 INFO:tasks.workunit.client.0.smithi092.stdout:0/348: unlink d7/d43/d1f/f3c 0 2024-05-06T12:14:31.847 INFO:tasks.workunit.client.0.smithi092.stdout:6/358: creat d4/d24/d54/f77 x:0 0 0 2024-05-06T12:14:31.849 INFO:tasks.workunit.client.0.smithi092.stdout:1/279: getdents d3/d10/d13/d19/d57 0 2024-05-06T12:14:31.851 INFO:tasks.workunit.client.0.smithi092.stdout:6/359: write d4/d20/d23/f46 [2915903,108040] 0 2024-05-06T12:14:31.851 INFO:tasks.workunit.client.0.smithi092.stdout:7/504: dwrite d9/da5/f14 [0,4194304] 0 2024-05-06T12:14:31.853 INFO:tasks.workunit.client.0.smithi092.stdout:0/349: rename d7/dc/d28/d54/d1e/d46/c75 to d7/dc/d28/d54/c77 0 2024-05-06T12:14:31.855 INFO:tasks.workunit.client.0.smithi092.stdout:6/360: write d4/d20/d23/d30/f33 [39009,107975] 0 2024-05-06T12:14:31.855 INFO:tasks.workunit.client.0.smithi092.stdout:5/339: truncate d4/d5/d10/f17 193834 0 2024-05-06T12:14:31.859 INFO:tasks.workunit.client.0.smithi092.stdout:7/505: read - d9/de/d10/d17/d6a/f81 zero size 2024-05-06T12:14:31.860 INFO:tasks.workunit.client.0.smithi092.stdout:2/247: dread f1 [0,4194304] 0 2024-05-06T12:14:31.861 INFO:tasks.workunit.client.0.smithi092.stdout:6/361: symlink d4/d24/d4e/l78 0 2024-05-06T12:14:31.865 INFO:tasks.workunit.client.0.smithi092.stdout:6/362: creat d4/d24/d54/f79 x:0 0 0 2024-05-06T12:14:31.868 INFO:tasks.workunit.client.0.smithi092.stdout:0/350: link d7/dc/d28/d54/d1e/f3e d7/dc/d28/d64/f78 0 2024-05-06T12:14:31.870 INFO:tasks.workunit.client.0.smithi092.stdout:0/351: stat d7/d43/l34 0 2024-05-06T12:14:31.872 INFO:tasks.workunit.client.0.smithi092.stdout:2/248: dwrite df/d17/d1d/f49 [0,4194304] 0 2024-05-06T12:14:31.873 INFO:tasks.workunit.client.0.smithi092.stdout:2/249: chown df/c38 1 1 2024-05-06T12:14:31.874 INFO:tasks.workunit.client.0.smithi092.stdout:6/363: dwrite d4/f3d [4194304,4194304] 0 2024-05-06T12:14:31.880 INFO:tasks.workunit.client.0.smithi092.stdout:6/364: creat d4/d20/d22/f7a x:0 0 0 2024-05-06T12:14:31.881 INFO:tasks.workunit.client.0.smithi092.stdout:0/352: getdents d7/d43/d1f 0 2024-05-06T12:14:31.885 INFO:tasks.workunit.client.0.smithi092.stdout:2/250: write df/f12 [176105,27478] 0 2024-05-06T12:14:31.886 INFO:tasks.workunit.client.0.smithi092.stdout:5/340: dwrite d4/d5/d10/f17 [0,4194304] 0 2024-05-06T12:14:31.886 INFO:tasks.workunit.client.0.smithi092.stdout:0/353: write d7/dc/d28/d54/d1e/f22 [2794845,40722] 0 2024-05-06T12:14:31.887 INFO:tasks.workunit.client.0.smithi092.stdout:2/251: chown df/d1b/l31 49356625 1 2024-05-06T12:14:31.889 INFO:tasks.workunit.client.0.smithi092.stdout:0/354: stat d7/dc/d28/d54/d1e/c45 0 2024-05-06T12:14:31.889 INFO:tasks.workunit.client.0.smithi092.stdout:6/365: dread d4/d4c/d52/f4a [4194304,4194304] 0 2024-05-06T12:14:31.890 INFO:tasks.workunit.client.0.smithi092.stdout:0/355: dread - d7/f70 zero size 2024-05-06T12:14:31.897 INFO:tasks.workunit.client.0.smithi092.stdout:0/356: unlink d7/dc/d28/d54/f11 0 2024-05-06T12:14:31.898 INFO:tasks.workunit.client.0.smithi092.stdout:6/366: dwrite d4/d20/f3b [0,4194304] 0 2024-05-06T12:14:31.906 INFO:tasks.workunit.client.0.smithi092.stdout:6/367: dwrite d4/d20/f1b [0,4194304] 0 2024-05-06T12:14:31.908 INFO:tasks.workunit.client.0.smithi092.stdout:5/341: truncate d4/d5/d10/d31/d53/d64/f48 3368729 0 2024-05-06T12:14:31.909 INFO:tasks.workunit.client.0.smithi092.stdout:6/368: mkdir d4/d24/d38/d5d/d6d/d7b 0 2024-05-06T12:14:31.910 INFO:tasks.workunit.client.0.smithi092.stdout:0/357: truncate d7/dc/d28/f2e 2978430 0 2024-05-06T12:14:31.911 INFO:tasks.workunit.client.0.smithi092.stdout:6/369: fdatasync d4/d24/d54/f77 0 2024-05-06T12:14:31.912 INFO:tasks.workunit.client.0.smithi092.stdout:0/358: mknod d7/dc/c79 0 2024-05-06T12:14:31.912 INFO:tasks.workunit.client.0.smithi092.stdout:6/370: stat d4/c12 0 2024-05-06T12:14:31.913 INFO:tasks.workunit.client.0.smithi092.stdout:0/359: readlink d7/dc/d28/d54/d1e/d46/l4c 0 2024-05-06T12:14:31.914 INFO:tasks.workunit.client.0.smithi092.stdout:5/342: read d4/d5/f1d [180415,68414] 0 2024-05-06T12:14:31.915 INFO:tasks.workunit.client.0.smithi092.stdout:0/360: creat d7/dc/d31/f7a x:0 0 0 2024-05-06T12:14:31.917 INFO:tasks.workunit.client.0.smithi092.stdout:5/343: creat d4/d59/f85 x:0 0 0 2024-05-06T12:14:31.917 INFO:tasks.workunit.client.0.smithi092.stdout:5/344: chown d4/d34/l6f 1829042467 1 2024-05-06T12:14:31.918 INFO:tasks.workunit.client.0.smithi092.stdout:5/345: chown d4/d34/d29/d6c/l50 3534 1 2024-05-06T12:14:31.918 INFO:tasks.workunit.client.0.smithi092.stdout:0/361: readlink d7/dc/d28/d54/d1e/d5d/l66 0 2024-05-06T12:14:31.920 INFO:tasks.workunit.client.0.smithi092.stdout:5/346: symlink d4/d34/d29/d6c/l86 0 2024-05-06T12:14:31.922 INFO:tasks.workunit.client.0.smithi092.stdout:5/347: read d4/d34/d28/d4e/d5d/d82/f18 [22157,41914] 0 2024-05-06T12:14:31.922 INFO:tasks.workunit.client.0.smithi092.stdout:0/362: unlink d7/d43/c5a 0 2024-05-06T12:14:31.923 INFO:tasks.workunit.client.0.smithi092.stdout:0/363: write d7/dc/d28/d54/d1e/d26/f36 [423126,96564] 0 2024-05-06T12:14:31.931 INFO:tasks.workunit.client.0.smithi092.stdout:0/364: dwrite d7/f70 [0,4194304] 0 2024-05-06T12:14:31.931 INFO:tasks.workunit.client.0.smithi092.stdout:6/371: truncate d4/d4c/d52/f4a 4515713 0 2024-05-06T12:14:31.932 INFO:tasks.workunit.client.0.smithi092.stdout:0/365: fdatasync d7/dc/d28/f4d 0 2024-05-06T12:14:31.935 INFO:tasks.workunit.client.0.smithi092.stdout:0/366: stat d7/dc/d28/d54/d12/f1b 0 2024-05-06T12:14:31.936 INFO:tasks.workunit.client.0.smithi092.stdout:6/372: dwrite d4/d20/d23/f46 [0,4194304] 0 2024-05-06T12:14:31.937 INFO:tasks.workunit.client.0.smithi092.stdout:0/367: creat d7/dc/d28/d54/d1e/d46/f7b x:0 0 0 2024-05-06T12:14:31.937 INFO:tasks.workunit.client.0.smithi092.stdout:6/373: dread - d4/d20/d22/f7a zero size 2024-05-06T12:14:31.939 INFO:tasks.workunit.client.0.smithi092.stdout:0/368: write d7/dc/d28/d54/d1e/d26/f2b [1726831,26319] 0 2024-05-06T12:14:31.945 INFO:tasks.workunit.client.0.smithi092.stdout:6/374: link d4/d24/f39 d4/d20/d22/f7c 0 2024-05-06T12:14:31.948 INFO:tasks.workunit.client.0.smithi092.stdout:6/375: truncate d4/f19 832141 0 2024-05-06T12:14:31.948 INFO:tasks.workunit.client.0.smithi092.stdout:6/376: chown d4 902000111 1 2024-05-06T12:14:31.951 INFO:tasks.workunit.client.0.smithi092.stdout:6/377: mknod d4/d20/d23/c7d 0 2024-05-06T12:14:31.953 INFO:tasks.workunit.client.0.smithi092.stdout:6/378: rmdir d4/d24/d26 39 2024-05-06T12:14:31.954 INFO:tasks.workunit.client.0.smithi092.stdout:6/379: write d4/d20/d23/d30/f33 [1187310,3740] 0 2024-05-06T12:14:31.956 INFO:tasks.workunit.client.0.smithi092.stdout:0/369: write d7/dc/d28/d64/f78 [4795802,107200] 0 2024-05-06T12:14:31.958 INFO:tasks.workunit.client.0.smithi092.stdout:6/380: creat d4/d24/d66/f7e x:0 0 0 2024-05-06T12:14:31.958 INFO:tasks.workunit.client.0.smithi092.stdout:6/381: chown d4/d20 4263196 1 2024-05-06T12:14:31.960 INFO:tasks.workunit.client.0.smithi092.stdout:6/382: chown d4/d20/d23/d30 1 1 2024-05-06T12:14:31.964 INFO:tasks.workunit.client.0.smithi092.stdout:0/370: link d7/dc/d28/d54/d1e/d39/f57 d7/dc/d28/d54/d1e/d5d/f7c 0 2024-05-06T12:14:31.965 INFO:tasks.workunit.client.0.smithi092.stdout:6/383: creat d4/d24/d38/d5d/d6d/d7b/f7f x:0 0 0 2024-05-06T12:14:31.968 INFO:tasks.workunit.client.0.smithi092.stdout:6/384: creat d4/d4c/d52/f80 x:0 0 0 2024-05-06T12:14:31.970 INFO:tasks.workunit.client.0.smithi092.stdout:6/385: mknod d4/d4c/c81 0 2024-05-06T12:14:31.971 INFO:tasks.workunit.client.0.smithi092.stdout:0/371: dwrite d7/dc/d31/f7a [0,4194304] 0 2024-05-06T12:14:31.972 INFO:tasks.workunit.client.0.smithi092.stdout:6/386: mknod d4/c82 0 2024-05-06T12:14:31.980 INFO:tasks.workunit.client.0.smithi092.stdout:6/387: write d4/f6 [1866985,113347] 0 2024-05-06T12:14:31.984 INFO:tasks.workunit.client.0.smithi092.stdout:6/388: write d4/f1a [168190,18273] 0 2024-05-06T12:14:31.999 INFO:tasks.workunit.client.0.smithi092.stdout:6/389: dread d4/f17 [0,4194304] 0 2024-05-06T12:14:32.001 INFO:tasks.workunit.client.0.smithi092.stdout:6/390: creat d4/d20/f83 x:0 0 0 2024-05-06T12:14:32.002 INFO:tasks.workunit.client.0.smithi092.stdout:6/391: creat d4/d24/d38/d5d/d6d/f84 x:0 0 0 2024-05-06T12:14:32.003 INFO:tasks.workunit.client.0.smithi092.stdout:6/392: fdatasync d4/d24/d26/f4b 0 2024-05-06T12:14:32.004 INFO:tasks.workunit.client.0.smithi092.stdout:6/393: write d4/d24/d38/d5d/d6d/d7b/f7f [558910,53386] 0 2024-05-06T12:14:32.005 INFO:tasks.workunit.client.0.smithi092.stdout:6/394: chown d4/f1a 1685655480 1 2024-05-06T12:14:32.007 INFO:tasks.workunit.client.0.smithi092.stdout:6/395: mknod d4/d20/d23/d30/c85 0 2024-05-06T12:14:32.044 INFO:tasks.workunit.client.0.smithi092.stdout:6/396: unlink d4/d20/f3b 0 2024-05-06T12:14:32.048 INFO:tasks.workunit.client.0.smithi092.stdout:6/397: creat d4/d24/d54/f86 x:0 0 0 2024-05-06T12:14:32.048 INFO:tasks.workunit.client.0.smithi092.stdout:6/398: dread - d4/d4c/d52/f80 zero size 2024-05-06T12:14:32.050 INFO:tasks.workunit.client.0.smithi092.stdout:6/399: mkdir d4/d24/d38/d5d/d6d/d87 0 2024-05-06T12:14:32.055 INFO:tasks.workunit.client.0.smithi092.stdout:6/400: getdents d4/d24/d54 0 2024-05-06T12:14:32.056 INFO:tasks.workunit.client.0.smithi092.stdout:6/401: dread - d4/d20/f83 zero size 2024-05-06T12:14:32.056 INFO:tasks.workunit.client.0.smithi092.stdout:6/402: chown d4/d20/f31 0 1 2024-05-06T12:14:32.056 INFO:tasks.workunit.client.0.smithi092.stdout:6/403: stat d4/d20/d23 0 2024-05-06T12:14:32.174 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:31 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:32.174 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:31 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:32.174 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:31 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:14:32.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:31 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:32.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:31 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:32.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:31 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:14:32.256 INFO:tasks.workunit.client.0.smithi092.stdout:8/341: sync 2024-05-06T12:14:32.257 INFO:tasks.workunit.client.0.smithi092.stdout:8/342: rmdir d9/d21/d37/d46/d5c 39 2024-05-06T12:14:32.258 INFO:tasks.workunit.client.0.smithi092.stdout:8/343: symlink d9/d21/d54/l84 0 2024-05-06T12:14:32.259 INFO:tasks.workunit.client.0.smithi092.stdout:8/344: mknod d9/c85 0 2024-05-06T12:14:32.260 INFO:tasks.workunit.client.0.smithi092.stdout:8/345: rename d9/d21/d54 to d9/d21/d54/d79/d86 22 2024-05-06T12:14:32.276 INFO:tasks.workunit.client.0.smithi092.stdout:8/346: dread d9/dc/f2d [0,4194304] 0 2024-05-06T12:14:32.278 INFO:tasks.workunit.client.0.smithi092.stdout:8/347: fdatasync d9/dc/ff 0 2024-05-06T12:14:32.281 INFO:tasks.workunit.client.0.smithi092.stdout:8/348: creat d9/dc/f87 x:0 0 0 2024-05-06T12:14:32.282 INFO:tasks.workunit.client.0.smithi092.stdout:8/349: rmdir d9/d21/d37/d46/d5c 39 2024-05-06T12:14:32.284 INFO:tasks.workunit.client.0.smithi092.stdout:8/350: creat d9/d21/f88 x:0 0 0 2024-05-06T12:14:32.290 INFO:tasks.workunit.client.0.smithi092.stdout:8/351: dwrite d9/dc/f51 [4194304,4194304] 0 2024-05-06T12:14:32.291 INFO:tasks.workunit.client.0.smithi092.stdout:8/352: readlink d9/d21/l22 0 2024-05-06T12:14:32.293 INFO:tasks.workunit.client.0.smithi092.stdout:8/353: symlink d9/d21/d37/d46/l89 0 2024-05-06T12:14:32.304 INFO:tasks.workunit.client.0.smithi092.stdout:8/354: dread d9/f33 [0,4194304] 0 2024-05-06T12:14:32.309 INFO:tasks.workunit.client.0.smithi092.stdout:8/355: truncate d9/d21/d37/d46/f5e 2908608 0 2024-05-06T12:14:32.311 INFO:tasks.workunit.client.0.smithi092.stdout:8/356: mkdir d9/d21/d54/d79/d8a 0 2024-05-06T12:14:32.313 INFO:tasks.workunit.client.0.smithi092.stdout:3/300: sync 2024-05-06T12:14:32.314 INFO:tasks.workunit.client.0.smithi092.stdout:5/348: sync 2024-05-06T12:14:32.314 INFO:tasks.workunit.client.0.smithi092.stdout:7/506: sync 2024-05-06T12:14:32.314 INFO:tasks.workunit.client.0.smithi092.stdout:4/285: sync 2024-05-06T12:14:32.314 INFO:tasks.workunit.client.0.smithi092.stdout:6/404: sync 2024-05-06T12:14:32.314 INFO:tasks.workunit.client.0.smithi092.stdout:2/252: sync 2024-05-06T12:14:32.314 INFO:tasks.workunit.client.0.smithi092.stdout:9/232: sync 2024-05-06T12:14:32.314 INFO:tasks.workunit.client.0.smithi092.stdout:1/280: sync 2024-05-06T12:14:32.314 INFO:tasks.workunit.client.0.smithi092.stdout:0/372: sync 2024-05-06T12:14:32.314 INFO:tasks.workunit.client.0.smithi092.stdout:8/357: unlink d9/d21/f38 0 2024-05-06T12:14:32.315 INFO:tasks.workunit.client.0.smithi092.stdout:2/253: stat df/f37 0 2024-05-06T12:14:32.315 INFO:tasks.workunit.client.0.smithi092.stdout:4/286: readlink d9/l1e 0 2024-05-06T12:14:32.316 INFO:tasks.workunit.client.0.smithi092.stdout:6/405: rmdir d4/d24/d54 39 2024-05-06T12:14:32.318 INFO:tasks.workunit.client.0.smithi092.stdout:8/358: truncate d9/dc/f5d 5146323 0 2024-05-06T12:14:32.318 INFO:tasks.workunit.client.0.smithi092.stdout:7/507: fdatasync d9/de/d10/f27 0 2024-05-06T12:14:32.319 INFO:tasks.workunit.client.0.smithi092.stdout:8/359: chown d9/d21/d37 1232 1 2024-05-06T12:14:32.320 INFO:tasks.workunit.client.0.smithi092.stdout:5/349: creat d4/d34/d28/d4e/f87 x:0 0 0 2024-05-06T12:14:32.321 INFO:tasks.workunit.client.0.smithi092.stdout:9/233: mknod d16/d2c/c49 0 2024-05-06T12:14:32.323 INFO:tasks.workunit.client.0.smithi092.stdout:5/350: write d4/d34/d28/d4e/f4f [418383,71490] 0 2024-05-06T12:14:32.327 INFO:tasks.workunit.client.0.smithi092.stdout:2/254: dwrite f1 [4194304,4194304] 0 2024-05-06T12:14:32.327 INFO:tasks.workunit.client.0.smithi092.stdout:5/351: write d4/d34/d29/d6c/f51 [410492,45666] 0 2024-05-06T12:14:32.328 INFO:tasks.workunit.client.0.smithi092.stdout:0/373: truncate f1 4062415 0 2024-05-06T12:14:32.329 INFO:tasks.workunit.client.0.smithi092.stdout:0/374: fsync d7/dc/f5c 0 2024-05-06T12:14:32.331 INFO:tasks.workunit.client.0.smithi092.stdout:8/360: write d9/dc/d4a/f6b [1307752,63269] 0 2024-05-06T12:14:32.332 INFO:tasks.workunit.client.0.smithi092.stdout:3/301: write d8/db/d4c/d40/f52 [282618,71106] 0 2024-05-06T12:14:32.333 INFO:tasks.workunit.client.0.smithi092.stdout:9/234: symlink d16/d2c/d3f/l4a 0 2024-05-06T12:14:32.337 INFO:tasks.workunit.client.0.smithi092.stdout:9/235: fdatasync d16/d2c/d2f/d3d/d33/f3b 0 2024-05-06T12:14:32.344 INFO:tasks.workunit.client.0.smithi092.stdout:0/375: dwrite d7/dc/d28/d54/d1e/d5d/f76 [0,4194304] 0 2024-05-06T12:14:32.347 INFO:tasks.workunit.client.0.smithi092.stdout:1/281: link d3/d10/d13/d19/d21/c56 d3/d10/d13/d19/d57/c5e 0 2024-05-06T12:14:32.348 INFO:tasks.workunit.client.0.smithi092.stdout:2/255: write f5 [2991507,71766] 0 2024-05-06T12:14:32.350 INFO:tasks.workunit.client.0.smithi092.stdout:9/236: dread d16/f1b [0,4194304] 0 2024-05-06T12:14:32.350 INFO:tasks.workunit.client.0.smithi092.stdout:3/302: symlink d8/db/l61 0 2024-05-06T12:14:32.351 INFO:tasks.workunit.client.0.smithi092.stdout:8/361: dread d9/dc/d4a/d5f/f70 [0,4194304] 0 2024-05-06T12:14:32.352 INFO:tasks.workunit.client.0.smithi092.stdout:4/287: getdents d9/df/d48 0 2024-05-06T12:14:32.354 INFO:tasks.workunit.client.0.smithi092.stdout:5/352: rename d4/d59/f85 to d4/f88 0 2024-05-06T12:14:32.356 INFO:tasks.workunit.client.0.smithi092.stdout:4/288: write d9/f3f [792493,53022] 0 2024-05-06T12:14:32.356 INFO:tasks.workunit.client.0.smithi092.stdout:0/376: dwrite d7/dc/d28/d54/d1e/f22 [0,4194304] 0 2024-05-06T12:14:32.360 INFO:tasks.workunit.client.0.smithi092.stdout:4/289: write d9/d27/f49 [2316713,130402] 0 2024-05-06T12:14:32.367 INFO:tasks.workunit.client.0.smithi092.stdout:8/362: dwrite d9/d21/d72/f7e [0,4194304] 0 2024-05-06T12:14:32.367 INFO:tasks.workunit.client.0.smithi092.stdout:5/353: dread d4/d5/d30/f3c [0,4194304] 0 2024-05-06T12:14:32.367 INFO:tasks.workunit.client.0.smithi092.stdout:3/303: dwrite d8/db/d4c/d3d/f5d [0,4194304] 0 2024-05-06T12:14:32.374 INFO:tasks.workunit.client.0.smithi092.stdout:3/304: dwrite d8/f12 [0,4194304] 0 2024-05-06T12:14:32.375 INFO:tasks.workunit.client.0.smithi092.stdout:3/305: chown d8/db/l2d 209200116 1 2024-05-06T12:14:32.412 INFO:tasks.workunit.client.0.smithi092.stdout:7/508: getdents d9 0 2024-05-06T12:14:32.414 INFO:tasks.workunit.client.0.smithi092.stdout:2/256: unlink df/d17/d1d/d33/f35 0 2024-05-06T12:14:32.415 INFO:tasks.workunit.client.0.smithi092.stdout:9/237: mkdir d16/d2c/d4b 0 2024-05-06T12:14:32.419 INFO:tasks.workunit.client.0.smithi092.stdout:0/377: fdatasync d7/dc/d28/d54/d1e/d26/d3a/f52 0 2024-05-06T12:14:32.420 INFO:tasks.workunit.client.0.smithi092.stdout:0/378: chown d7/dc/d28/d54/d1e/d26/d4e/l6b 4496942 1 2024-05-06T12:14:32.420 INFO:tasks.workunit.client.0.smithi092.stdout:8/363: rmdir d9/d21/d37/d74 39 2024-05-06T12:14:32.426 INFO:tasks.workunit.client.0.smithi092.stdout:5/354: dwrite d4/d5/f1d [0,4194304] 0 2024-05-06T12:14:32.427 INFO:tasks.workunit.client.0.smithi092.stdout:7/509: rename d9/l19 to d9/laa 0 2024-05-06T12:14:32.430 INFO:tasks.workunit.client.0.smithi092.stdout:9/238: unlink d16/d2c/d2f/l34 0 2024-05-06T12:14:32.432 INFO:tasks.workunit.client.0.smithi092.stdout:0/379: creat d7/dc/d28/d64/f7d x:0 0 0 2024-05-06T12:14:32.433 INFO:tasks.workunit.client.0.smithi092.stdout:3/306: write d8/db/d14/f50 [4194749,42199] 0 2024-05-06T12:14:32.438 INFO:tasks.workunit.client.0.smithi092.stdout:9/239: mknod d16/d1c/c4c 0 2024-05-06T12:14:32.440 INFO:tasks.workunit.client.0.smithi092.stdout:0/380: mknod d7/d43/d1f/c7e 0 2024-05-06T12:14:32.441 INFO:tasks.workunit.client.0.smithi092.stdout:8/364: rmdir d9/d21/d37/d74 39 2024-05-06T12:14:32.442 INFO:tasks.workunit.client.0.smithi092.stdout:5/355: dread d4/d34/d29/d54/f62 [0,4194304] 0 2024-05-06T12:14:32.444 INFO:tasks.workunit.client.0.smithi092.stdout:7/510: dwrite d9/de/d10/d17/d99/f7d [0,4194304] 0 2024-05-06T12:14:32.444 INFO:tasks.workunit.client.0.smithi092.stdout:2/257: getdents df/d17 0 2024-05-06T12:14:32.446 INFO:tasks.workunit.client.0.smithi092.stdout:9/240: read fc [3015759,22650] 0 2024-05-06T12:14:32.446 INFO:tasks.workunit.client.0.smithi092.stdout:0/381: chown d7/dc/d28/d54/d1e/d5d/f7c 45497294 1 2024-05-06T12:14:32.449 INFO:tasks.workunit.client.0.smithi092.stdout:0/382: chown d7/dc/d28/d54/d1e/d26/d4e/l6b 152420554 1 2024-05-06T12:14:32.452 INFO:tasks.workunit.client.0.smithi092.stdout:2/258: creat df/d17/f4a x:0 0 0 2024-05-06T12:14:32.453 INFO:tasks.workunit.client.0.smithi092.stdout:8/365: dwrite d9/dc/d4a/f6b [0,4194304] 0 2024-05-06T12:14:32.455 INFO:tasks.workunit.client.0.smithi092.stdout:7/511: dread - d9/de/d10/d17/d6a/f6d zero size 2024-05-06T12:14:32.456 INFO:tasks.workunit.client.0.smithi092.stdout:9/241: readlink d16/d2c/l2d 0 2024-05-06T12:14:32.458 INFO:tasks.workunit.client.0.smithi092.stdout:2/259: dread f1 [4194304,4194304] 0 2024-05-06T12:14:32.459 INFO:tasks.workunit.client.0.smithi092.stdout:6/406: sync 2024-05-06T12:14:32.462 INFO:tasks.workunit.client.0.smithi092.stdout:8/366: rename d9/d21/l24 to d9/d5a/l8b 0 2024-05-06T12:14:32.462 INFO:tasks.workunit.client.0.smithi092.stdout:5/356: truncate d4/f9 3686710 0 2024-05-06T12:14:32.464 INFO:tasks.workunit.client.0.smithi092.stdout:9/242: creat d16/d2c/d3f/f4d x:0 0 0 2024-05-06T12:14:32.465 INFO:tasks.workunit.client.0.smithi092.stdout:9/243: stat d16/d2c/d4b 0 2024-05-06T12:14:32.465 INFO:tasks.workunit.client.0.smithi092.stdout:3/307: link d8/d24/d33/d4b/f41 d8/d24/d33/f62 0 2024-05-06T12:14:32.466 INFO:tasks.workunit.client.0.smithi092.stdout:3/308: fsync d8/f12 0 2024-05-06T12:14:32.467 INFO:tasks.workunit.client.0.smithi092.stdout:7/512: mknod d9/de/d10/d17/d21/d38/cab 0 2024-05-06T12:14:32.469 INFO:tasks.workunit.client.0.smithi092.stdout:6/407: dwrite d4/d20/d23/f28 [0,4194304] 0 2024-05-06T12:14:32.469 INFO:tasks.workunit.client.0.smithi092.stdout:0/383: mkdir d7/dc/d28/d54/d12/d7f 0 2024-05-06T12:14:32.469 INFO:tasks.workunit.client.0.smithi092.stdout:9/244: chown d16/d2c/d2f/d3d/d33/d37/d40/d43 22 1 2024-05-06T12:14:32.472 INFO:tasks.workunit.client.0.smithi092.stdout:9/245: readlink d16/l18 0 2024-05-06T12:14:32.472 INFO:tasks.workunit.client.0.smithi092.stdout:6/408: dread - d4/d4c/f5c zero size 2024-05-06T12:14:32.475 INFO:tasks.workunit.client.0.smithi092.stdout:5/357: dwrite d4/d34/d29/d6c/f51 [0,4194304] 0 2024-05-06T12:14:32.479 INFO:tasks.workunit.client.0.smithi092.stdout:7/513: unlink d9/de/f50 0 2024-05-06T12:14:32.480 INFO:tasks.workunit.client.0.smithi092.stdout:8/367: fdatasync d9/d21/d37/d46/f5e 0 2024-05-06T12:14:32.481 INFO:tasks.workunit.client.0.smithi092.stdout:3/309: dwrite d8/d24/d33/d39/f44 [0,4194304] 0 2024-05-06T12:14:32.486 INFO:tasks.workunit.client.0.smithi092.stdout:9/246: mknod d16/d2c/d2f/d3d/d33/d37/d40/d43/c4e 0 2024-05-06T12:14:32.488 INFO:tasks.workunit.client.0.smithi092.stdout:5/358: creat d4/d34/d28/d4e/d5d/d82/d77/f89 x:0 0 0 2024-05-06T12:14:32.503 INFO:tasks.workunit.client.0.smithi092.stdout:6/409: dread d4/f11 [0,4194304] 0 2024-05-06T12:14:32.504 INFO:tasks.workunit.client.0.smithi092.stdout:6/410: read d4/d20/f1c [2253302,25850] 0 2024-05-06T12:14:32.509 INFO:tasks.workunit.client.0.smithi092.stdout:6/411: dwrite d4/d24/f3e [4194304,4194304] 0 2024-05-06T12:14:32.515 INFO:tasks.workunit.client.0.smithi092.stdout:8/368: unlink d9/d5a/f7c 0 2024-05-06T12:14:32.520 INFO:tasks.workunit.client.0.smithi092.stdout:9/247: fdatasync d16/f1b 0 2024-05-06T12:14:32.521 INFO:tasks.workunit.client.0.smithi092.stdout:8/369: dread d9/dc/d4a/d5f/f70 [0,4194304] 0 2024-05-06T12:14:32.521 INFO:tasks.workunit.client.0.smithi092.stdout:9/248: write d16/d1e/f3e [175623,89264] 0 2024-05-06T12:14:32.521 INFO:tasks.workunit.client.0.smithi092.stdout:8/370: chown d9/d21/d37/l39 56056801 1 2024-05-06T12:14:32.521 INFO:tasks.workunit.client.0.smithi092.stdout:5/359: creat d4/d5/d10/d31/d53/d64/f8a x:0 0 0 2024-05-06T12:14:32.522 INFO:tasks.workunit.client.0.smithi092.stdout:6/412: stat d4/d20/c63 0 2024-05-06T12:14:32.523 INFO:tasks.workunit.client.0.smithi092.stdout:7/514: creat d9/d62/d9a/fac x:0 0 0 2024-05-06T12:14:32.525 INFO:tasks.workunit.client.0.smithi092.stdout:7/515: chown d9/de/d10/d17/d99/da6 0 1 2024-05-06T12:14:32.527 INFO:tasks.workunit.client.0.smithi092.stdout:9/249: dwrite d16/d1e/f42 [0,4194304] 0 2024-05-06T12:14:32.533 INFO:tasks.workunit.client.0.smithi092.stdout:5/360: rename d4/f88 to d4/d5/d10/d31/d53/d64/f8b 0 2024-05-06T12:14:32.535 INFO:tasks.workunit.client.0.smithi092.stdout:0/384: getdents d7/dc/d28/d54 0 2024-05-06T12:14:32.535 INFO:tasks.workunit.client.0.smithi092.stdout:0/385: readlink d7/dc/d31/l6c 0 2024-05-06T12:14:32.537 INFO:tasks.workunit.client.0.smithi092.stdout:7/516: creat d9/de/d2d/fad x:0 0 0 2024-05-06T12:14:32.537 INFO:tasks.workunit.client.0.smithi092.stdout:0/386: write d7/dc/d28/d54/d1e/d26/f2b [37954,47447] 0 2024-05-06T12:14:32.537 INFO:tasks.workunit.client.0.smithi092.stdout:2/260: link df/c14 df/d17/d1d/c4b 0 2024-05-06T12:14:32.538 INFO:tasks.workunit.client.0.smithi092.stdout:0/387: stat d7/dc/d28/d54/d1e/c73 0 2024-05-06T12:14:32.540 INFO:tasks.workunit.client.0.smithi092.stdout:9/250: unlink d16/d1c/l1d 0 2024-05-06T12:14:32.545 INFO:tasks.workunit.client.0.smithi092.stdout:2/261: unlink df/f1f 0 2024-05-06T12:14:32.546 INFO:tasks.workunit.client.0.smithi092.stdout:7/517: write d9/de/d10/d17/d21/f5d [829510,367] 0 2024-05-06T12:14:32.548 INFO:tasks.workunit.client.0.smithi092.stdout:8/371: truncate d9/f36 1289736 0 2024-05-06T12:14:32.550 INFO:tasks.workunit.client.0.smithi092.stdout:9/251: unlink d16/d1c/f30 0 2024-05-06T12:14:32.550 INFO:tasks.workunit.client.0.smithi092.stdout:6/413: truncate d4/d4c/d52/f70 3330499 0 2024-05-06T12:14:32.552 INFO:tasks.workunit.client.0.smithi092.stdout:7/518: mknod d9/de/d10/d17/cae 0 2024-05-06T12:14:32.554 INFO:tasks.workunit.client.0.smithi092.stdout:0/388: fdatasync d7/dc/d28/f2e 0 2024-05-06T12:14:32.555 INFO:tasks.workunit.client.0.smithi092.stdout:0/389: write d7/dc/d28/d54/d1e/d5d/f68 [591764,12212] 0 2024-05-06T12:14:32.558 INFO:tasks.workunit.client.0.smithi092.stdout:6/414: creat d4/d24/d66/f88 x:0 0 0 2024-05-06T12:14:32.563 INFO:tasks.workunit.client.0.smithi092.stdout:1/282: sync 2024-05-06T12:14:32.564 INFO:tasks.workunit.client.0.smithi092.stdout:1/283: write d3/d10/d13/d19/f40 [3492310,34756] 0 2024-05-06T12:14:32.566 INFO:tasks.workunit.client.0.smithi092.stdout:5/361: truncate d4/d5/f3b 3814857 0 2024-05-06T12:14:32.567 INFO:tasks.workunit.client.0.smithi092.stdout:5/362: truncate d4/f7d 222729 0 2024-05-06T12:14:32.567 INFO:tasks.workunit.client.0.smithi092.stdout:6/415: mknod d4/d4c/c89 0 2024-05-06T12:14:32.571 INFO:tasks.workunit.client.0.smithi092.stdout:1/284: creat d3/d10/d13/d19/d21/d47/f5f x:0 0 0 2024-05-06T12:14:32.576 INFO:tasks.workunit.client.0.smithi092.stdout:8/372: dwrite f0 [0,4194304] 0 2024-05-06T12:14:32.588 INFO:tasks.workunit.client.0.smithi092.stdout:8/373: unlink d9/dc/d4a/d5f/f70 0 2024-05-06T12:14:32.588 INFO:tasks.workunit.client.0.smithi092.stdout:6/416: getdents d4/d4c/d52/d6c 0 2024-05-06T12:14:32.591 INFO:tasks.workunit.client.0.smithi092.stdout:0/390: getdents d7/dc/d28/d54/d1e/d5d 0 2024-05-06T12:14:32.592 INFO:tasks.workunit.client.0.smithi092.stdout:2/262: truncate df/d17/d1d/f49 3522501 0 2024-05-06T12:14:32.594 INFO:tasks.workunit.client.0.smithi092.stdout:5/363: write d4/d34/f56 [352563,129840] 0 2024-05-06T12:14:32.595 INFO:tasks.workunit.client.0.smithi092.stdout:6/417: mknod d4/d24/d66/c8a 0 2024-05-06T12:14:32.596 INFO:tasks.workunit.client.0.smithi092.stdout:5/364: chown d4/d5/d10/d31/f4a 161 1 2024-05-06T12:14:32.596 INFO:tasks.workunit.client.0.smithi092.stdout:0/391: fsync d7/dc/d28/d54/d1e/d26/d3a/f52 0 2024-05-06T12:14:32.600 INFO:tasks.workunit.client.0.smithi092.stdout:1/285: dwrite d3/d10/d13/d19/f5d [0,4194304] 0 2024-05-06T12:14:32.601 INFO:tasks.workunit.client.0.smithi092.stdout:1/286: fdatasync d3/f2c 0 2024-05-06T12:14:32.603 INFO:tasks.workunit.client.0.smithi092.stdout:1/287: write d3/d10/f53 [606007,6394] 0 2024-05-06T12:14:32.605 INFO:tasks.workunit.client.0.smithi092.stdout:1/288: write d3/f20 [646367,109330] 0 2024-05-06T12:14:32.608 INFO:tasks.workunit.client.0.smithi092.stdout:6/418: mknod d4/d24/c8b 0 2024-05-06T12:14:32.609 INFO:tasks.workunit.client.0.smithi092.stdout:6/419: dread - d4/d20/d22/f7a zero size 2024-05-06T12:14:32.613 INFO:tasks.workunit.client.0.smithi092.stdout:0/392: write d7/dc/f29 [463036,115975] 0 2024-05-06T12:14:32.613 INFO:tasks.workunit.client.0.smithi092.stdout:8/374: dread d9/f18 [0,4194304] 0 2024-05-06T12:14:32.614 INFO:tasks.workunit.client.0.smithi092.stdout:8/375: write d9/d21/f6f [652856,74912] 0 2024-05-06T12:14:32.616 INFO:tasks.workunit.client.0.smithi092.stdout:0/393: fdatasync d7/d43/f61 0 2024-05-06T12:14:32.617 INFO:tasks.workunit.client.0.smithi092.stdout:0/394: chown d7/dc/d28 2634 1 2024-05-06T12:14:32.618 INFO:tasks.workunit.client.0.smithi092.stdout:8/376: mknod d9/dc/c8c 0 2024-05-06T12:14:32.619 INFO:tasks.workunit.client.0.smithi092.stdout:0/395: mkdir d7/dc/d28/d54/d1e/d46/d63/d80 0 2024-05-06T12:14:32.621 INFO:tasks.workunit.client.0.smithi092.stdout:8/377: readlink d9/l50 0 2024-05-06T12:14:32.623 INFO:tasks.workunit.client.0.smithi092.stdout:2/263: truncate df/d17/f42 1872236 0 2024-05-06T12:14:32.623 INFO:tasks.workunit.client.0.smithi092.stdout:5/365: dread d4/d5/d10/d31/d53/d64/f25 [0,4194304] 0 2024-05-06T12:14:32.627 INFO:tasks.workunit.client.0.smithi092.stdout:6/420: dread d4/f6 [0,4194304] 0 2024-05-06T12:14:32.627 INFO:tasks.workunit.client.0.smithi092.stdout:2/264: read fd [7311945,91011] 0 2024-05-06T12:14:32.627 INFO:tasks.workunit.client.0.smithi092.stdout:8/378: rename d9/f35 to d9/d21/d37/d74/f8d 0 2024-05-06T12:14:32.629 INFO:tasks.workunit.client.0.smithi092.stdout:0/396: link d7/d43/l32 d7/dc/d6f/l81 0 2024-05-06T12:14:32.630 INFO:tasks.workunit.client.0.smithi092.stdout:5/366: dwrite d4/d5/d30/f46 [0,4194304] 0 2024-05-06T12:14:32.631 INFO:tasks.workunit.client.0.smithi092.stdout:0/397: write d7/dc/d28/d64/f78 [4164973,50839] 0 2024-05-06T12:14:32.632 INFO:tasks.workunit.client.0.smithi092.stdout:0/398: write d7/d43/f49 [3662557,75799] 0 2024-05-06T12:14:32.633 INFO:tasks.workunit.client.0.smithi092.stdout:6/421: mkdir d4/d24/d38/d8c 0 2024-05-06T12:14:32.637 INFO:tasks.workunit.client.0.smithi092.stdout:8/379: mknod d9/d21/d72/c8e 0 2024-05-06T12:14:32.641 INFO:tasks.workunit.client.0.smithi092.stdout:6/422: symlink d4/d4c/d52/d62/l8d 0 2024-05-06T12:14:32.647 INFO:tasks.workunit.client.0.smithi092.stdout:0/399: creat d7/dc/d28/d54/d12/f82 x:0 0 0 2024-05-06T12:14:32.648 INFO:tasks.workunit.client.0.smithi092.stdout:6/423: dwrite d4/d4c/d52/f56 [0,4194304] 0 2024-05-06T12:14:32.650 INFO:tasks.workunit.client.0.smithi092.stdout:0/400: write d7/dc/d28/d54/d1e/d26/f2b [5668713,109944] 0 2024-05-06T12:14:32.654 INFO:tasks.workunit.client.0.smithi092.stdout:8/380: write d9/f1b [750939,65710] 0 2024-05-06T12:14:32.656 INFO:tasks.workunit.client.0.smithi092.stdout:0/401: chown d7/dc/d28/d54/d1e/d26/d4e/l6b 136671869 1 2024-05-06T12:14:32.656 INFO:tasks.workunit.client.0.smithi092.stdout:6/424: fsync d4/d4c/d52/f70 0 2024-05-06T12:14:32.656 INFO:tasks.workunit.client.0.smithi092.stdout:2/265: dwrite df/d17/d1d/f46 [0,4194304] 0 2024-05-06T12:14:32.658 INFO:tasks.workunit.client.0.smithi092.stdout:0/402: dread - d7/dc/d28/d54/f3b zero size 2024-05-06T12:14:32.658 INFO:tasks.workunit.client.0.smithi092.stdout:2/266: stat df/d15/c1a 0 2024-05-06T12:14:32.659 INFO:tasks.workunit.client.0.smithi092.stdout:6/425: fdatasync d4/d24/d66/f7e 0 2024-05-06T12:14:32.659 INFO:tasks.workunit.client.0.smithi092.stdout:2/267: readlink df/d15/l28 0 2024-05-06T12:14:32.662 INFO:tasks.workunit.client.0.smithi092.stdout:2/268: creat df/d15/f4c x:0 0 0 2024-05-06T12:14:32.664 INFO:tasks.workunit.client.0.smithi092.stdout:8/381: getdents d9/d21 0 2024-05-06T12:14:32.666 INFO:tasks.workunit.client.0.smithi092.stdout:6/426: creat d4/f8e x:0 0 0 2024-05-06T12:14:32.667 INFO:tasks.workunit.client.0.smithi092.stdout:2/269: mkdir df/d17/d1d/d33/d4d 0 2024-05-06T12:14:32.668 INFO:tasks.workunit.client.0.smithi092.stdout:8/382: stat d9/d21/d37/d46/d5c/d62/l6c 0 2024-05-06T12:14:32.671 INFO:tasks.workunit.client.0.smithi092.stdout:4/290: sync 2024-05-06T12:14:32.672 INFO:tasks.workunit.client.0.smithi092.stdout:6/427: dread d4/d20/f1b [0,4194304] 0 2024-05-06T12:14:32.674 INFO:tasks.workunit.client.0.smithi092.stdout:4/291: chown d9/df/f46 0 1 2024-05-06T12:14:32.677 INFO:tasks.workunit.client.0.smithi092.stdout:8/383: dwrite d9/d21/d37/d46/d5c/d71/f80 [0,4194304] 0 2024-05-06T12:14:32.679 INFO:tasks.workunit.client.0.smithi092.stdout:0/403: dread d7/f4f [0,4194304] 0 2024-05-06T12:14:32.680 INFO:tasks.workunit.client.0.smithi092.stdout:2/270: truncate df/f37 1869432 0 2024-05-06T12:14:32.681 INFO:tasks.workunit.client.0.smithi092.stdout:6/428: link d4/d20/l21 d4/d24/d26/l8f 0 2024-05-06T12:14:32.682 INFO:tasks.workunit.client.0.smithi092.stdout:0/404: truncate d7/d43/f49 4628331 0 2024-05-06T12:14:32.684 INFO:tasks.workunit.client.0.smithi092.stdout:8/384: dread d9/f1b [0,4194304] 0 2024-05-06T12:14:32.684 INFO:tasks.workunit.client.0.smithi092.stdout:0/405: read d7/dc/d28/d54/d1e/d26/d3a/f52 [1667036,81016] 0 2024-05-06T12:14:32.685 INFO:tasks.workunit.client.0.smithi092.stdout:2/271: readlink df/d17/d1d/l3f 0 2024-05-06T12:14:32.686 INFO:tasks.workunit.client.0.smithi092.stdout:0/406: chown d7/dc/d28/d54/c77 84 1 2024-05-06T12:14:32.687 INFO:tasks.workunit.client.0.smithi092.stdout:6/429: rmdir d4/d24/d38/d5d/d6d/d7b 39 2024-05-06T12:14:32.689 INFO:tasks.workunit.client.0.smithi092.stdout:0/407: write f6 [2764738,3600] 0 2024-05-06T12:14:32.689 INFO:tasks.workunit.client.0.smithi092.stdout:6/430: mknod d4/d4c/c90 0 2024-05-06T12:14:32.693 INFO:tasks.workunit.client.0.smithi092.stdout:2/272: creat df/d17/d1d/f4e x:0 0 0 2024-05-06T12:14:32.695 INFO:tasks.workunit.client.0.smithi092.stdout:6/431: mkdir d4/d4c/d91 0 2024-05-06T12:14:32.695 INFO:tasks.workunit.client.0.smithi092.stdout:8/385: dwrite d9/d21/d37/f3b [4194304,4194304] 0 2024-05-06T12:14:32.697 INFO:tasks.workunit.client.0.smithi092.stdout:0/408: rmdir d7/dc/d28/d54/d1e/d39 39 2024-05-06T12:14:32.698 INFO:tasks.workunit.client.0.smithi092.stdout:4/292: dread d9/df/f15 [0,4194304] 0 2024-05-06T12:14:32.699 INFO:tasks.workunit.client.0.smithi092.stdout:0/409: write d7/dc/d28/d54/d1e/d46/f7b [814244,45510] 0 2024-05-06T12:14:32.700 INFO:tasks.workunit.client.0.smithi092.stdout:4/293: stat d9/d2b/c2f 0 2024-05-06T12:14:32.701 INFO:tasks.workunit.client.0.smithi092.stdout:4/294: write d9/dd/f23 [1097916,92143] 0 2024-05-06T12:14:32.701 INFO:tasks.workunit.client.0.smithi092.stdout:0/410: chown d7/d43/d1f/c7e 0 1 2024-05-06T12:14:32.706 INFO:tasks.workunit.client.0.smithi092.stdout:2/273: creat df/d17/d1d/f4f x:0 0 0 2024-05-06T12:14:32.709 INFO:tasks.workunit.client.0.smithi092.stdout:8/386: creat d9/d21/d37/f8f x:0 0 0 2024-05-06T12:14:32.710 INFO:tasks.workunit.client.0.smithi092.stdout:2/274: mkdir df/d15/d50 0 2024-05-06T12:14:32.711 INFO:tasks.workunit.client.0.smithi092.stdout:6/432: link d4/c82 d4/c92 0 2024-05-06T12:14:32.711 INFO:tasks.workunit.client.0.smithi092.stdout:8/387: write d9/dc/d4a/f6b [3758325,62678] 0 2024-05-06T12:14:32.712 INFO:tasks.workunit.client.0.smithi092.stdout:6/433: write d4/d4c/d52/f56 [4660610,105617] 0 2024-05-06T12:14:32.713 INFO:tasks.workunit.client.0.smithi092.stdout:4/295: mknod d9/c5d 0 2024-05-06T12:14:32.717 INFO:tasks.workunit.client.0.smithi092.stdout:8/388: dread d9/f33 [0,4194304] 0 2024-05-06T12:14:32.718 INFO:tasks.workunit.client.0.smithi092.stdout:6/434: mkdir d4/d4c/d91/d93 0 2024-05-06T12:14:32.719 INFO:tasks.workunit.client.0.smithi092.stdout:2/275: dread df/d17/f2a [0,4194304] 0 2024-05-06T12:14:32.719 INFO:tasks.workunit.client.0.smithi092.stdout:8/389: read d9/d21/d72/f7e [2496040,55066] 0 2024-05-06T12:14:32.720 INFO:tasks.workunit.client.0.smithi092.stdout:2/276: readlink df/l3a 0 2024-05-06T12:14:32.721 INFO:tasks.workunit.client.0.smithi092.stdout:4/296: rename d9/d27/l35 to d9/l5e 0 2024-05-06T12:14:32.723 INFO:tasks.workunit.client.0.smithi092.stdout:6/435: fsync d4/f11 0 2024-05-06T12:14:32.724 INFO:tasks.workunit.client.0.smithi092.stdout:8/390: rename d9/d5a/c65 to d9/d21/c90 0 2024-05-06T12:14:32.726 INFO:tasks.workunit.client.0.smithi092.stdout:2/277: dwrite df/d17/d1d/f4e [0,4194304] 0 2024-05-06T12:14:32.727 INFO:tasks.workunit.client.0.smithi092.stdout:4/297: creat d9/dd/f5f x:0 0 0 2024-05-06T12:14:32.730 INFO:tasks.workunit.client.0.smithi092.stdout:0/411: truncate d7/d43/f2c 1120275 0 2024-05-06T12:14:32.733 INFO:tasks.workunit.client.0.smithi092.stdout:0/412: write d7/dc/d28/d64/f78 [5546728,93496] 0 2024-05-06T12:14:32.734 INFO:tasks.workunit.client.0.smithi092.stdout:6/436: creat d4/d24/d38/d8c/f94 x:0 0 0 2024-05-06T12:14:32.736 INFO:tasks.workunit.client.0.smithi092.stdout:2/278: mkdir df/d17/d51 0 2024-05-06T12:14:32.736 INFO:tasks.workunit.client.0.smithi092.stdout:4/298: dwrite d9/df/f52 [0,4194304] 0 2024-05-06T12:14:32.737 INFO:tasks.workunit.client.0.smithi092.stdout:2/279: write df/f2c [2181569,106335] 0 2024-05-06T12:14:32.743 INFO:tasks.workunit.client.0.smithi092.stdout:4/299: dwrite d9/fc [0,4194304] 0 2024-05-06T12:14:32.745 INFO:tasks.workunit.client.0.smithi092.stdout:6/437: mknod d4/d20/d23/c95 0 2024-05-06T12:14:32.751 INFO:tasks.workunit.client.0.smithi092.stdout:6/438: truncate d4/f17 4017608 0 2024-05-06T12:14:32.756 INFO:tasks.workunit.client.0.smithi092.stdout:6/439: symlink d4/d24/d38/d5d/d6d/d7b/l96 0 2024-05-06T12:14:32.757 INFO:tasks.workunit.client.0.smithi092.stdout:4/300: getdents d9/d2b 0 2024-05-06T12:14:32.758 INFO:tasks.workunit.client.0.smithi092.stdout:6/440: mknod d4/d24/c97 0 2024-05-06T12:14:32.759 INFO:tasks.workunit.client.0.smithi092.stdout:4/301: getdents d9/d27/d4a/d59 0 2024-05-06T12:14:32.760 INFO:tasks.workunit.client.0.smithi092.stdout:4/302: write d9/dd/f23 [1509014,92377] 0 2024-05-06T12:14:32.762 INFO:tasks.workunit.client.0.smithi092.stdout:6/441: getdents d4/d4c/d91 0 2024-05-06T12:14:32.763 INFO:tasks.workunit.client.0.smithi092.stdout:6/442: write d4/d4c/d52/f56 [1941705,108521] 0 2024-05-06T12:14:32.767 INFO:tasks.workunit.client.0.smithi092.stdout:6/443: truncate d4/d20/f1c 6801032 0 2024-05-06T12:14:32.768 INFO:tasks.workunit.client.0.smithi092.stdout:4/303: dwrite d9/df/d48/f56 [0,4194304] 0 2024-05-06T12:14:32.771 INFO:tasks.workunit.client.0.smithi092.stdout:8/391: write d9/d21/d72/f7e [379182,281] 0 2024-05-06T12:14:32.772 INFO:tasks.workunit.client.0.smithi092.stdout:8/392: dread - d9/f2a zero size 2024-05-06T12:14:32.774 INFO:tasks.workunit.client.0.smithi092.stdout:2/280: truncate df/d17/d1d/f46 2428962 0 2024-05-06T12:14:32.776 INFO:tasks.workunit.client.0.smithi092.stdout:0/413: dread d7/dc/d28/d54/d1e/d39/f47 [0,4194304] 0 2024-05-06T12:14:32.780 INFO:tasks.workunit.client.0.smithi092.stdout:8/393: read d9/d21/f6f [558060,45436] 0 2024-05-06T12:14:32.786 INFO:tasks.workunit.client.0.smithi092.stdout:0/414: mkdir d7/dc/d28/d83 0 2024-05-06T12:14:32.790 INFO:tasks.workunit.client.0.smithi092.stdout:0/415: read - d7/dc/d28/d54/d1e/d26/f53 zero size 2024-05-06T12:14:32.791 INFO:tasks.workunit.client.0.smithi092.stdout:2/281: dwrite df/d17/f2a [0,4194304] 0 2024-05-06T12:14:32.791 INFO:tasks.workunit.client.0.smithi092.stdout:6/444: write d4/d24/f39 [21463,5021] 0 2024-05-06T12:14:32.792 INFO:tasks.workunit.client.0.smithi092.stdout:6/445: write d4/d20/d23/f28 [1853863,119201] 0 2024-05-06T12:14:32.797 INFO:tasks.workunit.client.0.smithi092.stdout:0/416: mknod d7/dc/d6f/c84 0 2024-05-06T12:14:32.799 INFO:tasks.workunit.client.0.smithi092.stdout:8/394: mknod d9/d21/d37/d46/d5c/d62/d7a/c91 0 2024-05-06T12:14:32.800 INFO:tasks.workunit.client.0.smithi092.stdout:4/304: dread d9/d27/f31 [0,4194304] 0 2024-05-06T12:14:32.801 INFO:tasks.workunit.client.0.smithi092.stdout:6/446: dwrite d4/d24/d26/f4b [0,4194304] 0 2024-05-06T12:14:32.804 INFO:tasks.workunit.client.0.smithi092.stdout:4/305: chown d9/l5e 9705 1 2024-05-06T12:14:32.804 INFO:tasks.workunit.client.0.smithi092.stdout:6/447: write d4/d4c/d52/f56 [1422950,3064] 0 2024-05-06T12:14:32.807 INFO:tasks.workunit.client.0.smithi092.stdout:4/306: fsync d9/fc 0 2024-05-06T12:14:32.809 INFO:tasks.workunit.client.0.smithi092.stdout:0/417: dwrite d7/dc/d28/d54/d1e/f3e [0,4194304] 0 2024-05-06T12:14:32.812 INFO:tasks.workunit.client.0.smithi092.stdout:4/307: rename d9/c33 to d9/dd/c60 0 2024-05-06T12:14:32.815 INFO:tasks.workunit.client.0.smithi092.stdout:3/310: sync 2024-05-06T12:14:32.815 INFO:tasks.workunit.client.0.smithi092.stdout:9/252: sync 2024-05-06T12:14:32.815 INFO:tasks.workunit.client.0.smithi092.stdout:0/418: read d7/d43/d1f/f4a [199847,47593] 0 2024-05-06T12:14:32.816 INFO:tasks.workunit.client.0.smithi092.stdout:4/308: symlink d9/dd/l61 0 2024-05-06T12:14:32.821 INFO:tasks.workunit.client.0.smithi092.stdout:3/311: unlink d8/db/d14/f18 0 2024-05-06T12:14:32.822 INFO:tasks.workunit.client.0.smithi092.stdout:6/448: getdents d4/d24/d26 0 2024-05-06T12:14:32.823 INFO:tasks.workunit.client.0.smithi092.stdout:9/253: getdents d16/d2c/d4b 0 2024-05-06T12:14:32.823 INFO:tasks.workunit.client.0.smithi092.stdout:3/312: stat d8/d24/c26 0 2024-05-06T12:14:32.823 INFO:tasks.workunit.client.0.smithi092.stdout:4/309: dread d9/d27/f31 [0,4194304] 0 2024-05-06T12:14:32.825 INFO:tasks.workunit.client.0.smithi092.stdout:8/395: dwrite d9/f36 [0,4194304] 0 2024-05-06T12:14:32.826 INFO:tasks.workunit.client.0.smithi092.stdout:4/310: chown d9/dd/l4e 53020570 1 2024-05-06T12:14:32.826 INFO:tasks.workunit.client.0.smithi092.stdout:3/313: truncate d8/f12 4445156 0 2024-05-06T12:14:32.827 INFO:tasks.workunit.client.0.smithi092.stdout:6/449: dread d4/d24/f39 [0,4194304] 0 2024-05-06T12:14:32.827 INFO:tasks.workunit.client.0.smithi092.stdout:8/396: fdatasync d9/dc/f5d 0 2024-05-06T12:14:32.828 INFO:tasks.workunit.client.0.smithi092.stdout:4/311: stat d9/d27/d4a 0 2024-05-06T12:14:32.828 INFO:tasks.workunit.client.0.smithi092.stdout:0/419: getdents d7/dc/d28/d54/d12 0 2024-05-06T12:14:32.830 INFO:tasks.workunit.client.0.smithi092.stdout:0/420: write d7/dc/d28/d64/f7d [931304,105407] 0 2024-05-06T12:14:32.832 INFO:tasks.workunit.client.0.smithi092.stdout:4/312: rmdir d9/dd 39 2024-05-06T12:14:32.835 INFO:tasks.workunit.client.0.smithi092.stdout:0/421: dread d7/dc/d31/f7a [0,4194304] 0 2024-05-06T12:14:32.837 INFO:tasks.workunit.client.0.smithi092.stdout:0/422: read d7/dc/d28/d54/d1e/d5d/f68 [162265,21659] 0 2024-05-06T12:14:32.844 INFO:tasks.workunit.client.0.smithi092.stdout:4/313: mkdir d9/df/d62 0 2024-05-06T12:14:32.847 INFO:tasks.workunit.client.0.smithi092.stdout:8/397: creat d9/d21/d37/d46/f92 x:0 0 0 2024-05-06T12:14:32.848 INFO:tasks.workunit.client.0.smithi092.stdout:3/314: dwrite d8/db/d4c/d3d/f60 [0,4194304] 0 2024-05-06T12:14:32.849 INFO:tasks.workunit.client.0.smithi092.stdout:6/450: getdents d4/d24/d66 0 2024-05-06T12:14:32.850 INFO:tasks.workunit.client.0.smithi092.stdout:0/423: dwrite d7/f4f [0,4194304] 0 2024-05-06T12:14:32.852 INFO:tasks.workunit.client.0.smithi092.stdout:9/254: getdents d16/d2c/d2f/d3d/d33/d37/d40/d43 0 2024-05-06T12:14:32.852 INFO:tasks.workunit.client.0.smithi092.stdout:0/424: chown d7/l8 1093 1 2024-05-06T12:14:32.855 INFO:tasks.workunit.client.0.smithi092.stdout:4/314: unlink d9/df/l1a 0 2024-05-06T12:14:32.857 INFO:tasks.workunit.client.0.smithi092.stdout:8/398: dread d9/f33 [0,4194304] 0 2024-05-06T12:14:32.858 INFO:tasks.workunit.client.0.smithi092.stdout:3/315: fsync d8/d24/d27/f2f 0 2024-05-06T12:14:32.863 INFO:tasks.workunit.client.0.smithi092.stdout:4/315: creat d9/df/d62/f63 x:0 0 0 2024-05-06T12:14:32.863 INFO:tasks.workunit.client.0.smithi092.stdout:8/399: mkdir d9/d21/d37/d46/d5c/d93 0 2024-05-06T12:14:32.864 INFO:tasks.workunit.client.0.smithi092.stdout:9/255: dwrite d16/d1e/f42 [4194304,4194304] 0 2024-05-06T12:14:32.865 INFO:tasks.workunit.client.0.smithi092.stdout:0/425: rename d7/dc/d28/d54/c77 to d7/c85 0 2024-05-06T12:14:32.866 INFO:tasks.workunit.client.0.smithi092.stdout:6/451: dread d4/f17 [0,4194304] 0 2024-05-06T12:14:32.869 INFO:tasks.workunit.client.0.smithi092.stdout:9/256: creat d16/d2c/f4f x:0 0 0 2024-05-06T12:14:32.869 INFO:tasks.workunit.client.0.smithi092.stdout:0/426: read - d7/dc/d28/d54/f3b zero size 2024-05-06T12:14:32.872 INFO:tasks.workunit.client.0.smithi092.stdout:3/316: rename d8/d24/f32 to d8/d24/f63 0 2024-05-06T12:14:32.873 INFO:tasks.workunit.client.0.smithi092.stdout:6/452: write d4/d20/d23/d30/f33 [2621701,20560] 0 2024-05-06T12:14:32.874 INFO:tasks.workunit.client.0.smithi092.stdout:9/257: stat d16/d2c/d2f/d3d 0 2024-05-06T12:14:32.877 INFO:tasks.workunit.client.0.smithi092.stdout:9/258: readlink d16/d2c/d3f/l4a 0 2024-05-06T12:14:32.877 INFO:tasks.workunit.client.0.smithi092.stdout:4/316: dwrite d9/f21 [4194304,4194304] 0 2024-05-06T12:14:32.878 INFO:tasks.workunit.client.0.smithi092.stdout:0/427: mknod d7/dc/d28/d54/d1e/d39/c86 0 2024-05-06T12:14:32.879 INFO:tasks.workunit.client.0.smithi092.stdout:8/400: dwrite d9/dc/ff [0,4194304] 0 2024-05-06T12:14:32.880 INFO:tasks.workunit.client.0.smithi092.stdout:0/428: chown d7/dc/d28/d54/d12/c58 29 1 2024-05-06T12:14:32.882 INFO:tasks.workunit.client.0.smithi092.stdout:3/317: creat d8/d24/d33/f64 x:0 0 0 2024-05-06T12:14:32.884 INFO:tasks.workunit.client.0.smithi092.stdout:0/429: write d7/d43/f40 [1532521,99224] 0 2024-05-06T12:14:32.885 INFO:tasks.workunit.client.0.smithi092.stdout:0/430: fdatasync d7/dc/d28/d54/d1e/d26/f2b 0 2024-05-06T12:14:32.886 INFO:tasks.workunit.client.0.smithi092.stdout:8/401: creat d9/d21/d37/d74/f94 x:0 0 0 2024-05-06T12:14:32.892 INFO:tasks.workunit.client.0.smithi092.stdout:8/402: write d9/f18 [2957668,91693] 0 2024-05-06T12:14:32.895 INFO:tasks.workunit.client.0.smithi092.stdout:6/453: rename d4/d20/d23/d30/c58 to d4/d24/d38/d5d/d6d/c98 0 2024-05-06T12:14:32.897 INFO:tasks.workunit.client.0.smithi092.stdout:6/454: write d4/d4c/d52/f6a [378707,103014] 0 2024-05-06T12:14:32.899 INFO:tasks.workunit.client.0.smithi092.stdout:3/318: rename d8/d24/f63 to d8/d24/f65 0 2024-05-06T12:14:32.899 INFO:tasks.workunit.client.0.smithi092.stdout:0/431: creat d7/dc/d28/d54/d1e/d46/d63/d6e/f87 x:0 0 0 2024-05-06T12:14:32.901 INFO:tasks.workunit.client.0.smithi092.stdout:8/403: mknod d9/d21/d37/d46/d5c/d71/c95 0 2024-05-06T12:14:32.905 INFO:tasks.workunit.client.0.smithi092.stdout:0/432: creat d7/f88 x:0 0 0 2024-05-06T12:14:32.905 INFO:tasks.workunit.client.0.smithi092.stdout:4/317: dwrite d9/d27/d4a/f4c [0,4194304] 0 2024-05-06T12:14:32.913 INFO:tasks.workunit.client.0.smithi092.stdout:0/433: creat d7/dc/d28/d64/f89 x:0 0 0 2024-05-06T12:14:32.914 INFO:tasks.workunit.client.0.smithi092.stdout:0/434: dread - d7/dc/d28/f4d zero size 2024-05-06T12:14:32.919 INFO:tasks.workunit.client.0.smithi092.stdout:3/319: dread d8/d1b/f2b [0,4194304] 0 2024-05-06T12:14:32.921 INFO:tasks.workunit.client.0.smithi092.stdout:0/435: truncate d7/dc/d28/f2e 2225556 0 2024-05-06T12:14:32.922 INFO:tasks.workunit.client.0.smithi092.stdout:8/404: dread d9/d21/d37/d46/f5e [0,4194304] 0 2024-05-06T12:14:32.922 INFO:tasks.workunit.client.0.smithi092.stdout:6/455: dwrite d4/d4c/d52/f4a [0,4194304] 0 2024-05-06T12:14:32.923 INFO:tasks.workunit.client.0.smithi092.stdout:4/318: dwrite d9/df/f46 [0,4194304] 0 2024-05-06T12:14:32.923 INFO:tasks.workunit.client.0.smithi092.stdout:3/320: mkdir d8/db/d4c/d3d/d48/d66 0 2024-05-06T12:14:32.925 INFO:tasks.workunit.client.0.smithi092.stdout:4/319: stat d9/d27 0 2024-05-06T12:14:32.927 INFO:tasks.workunit.client.0.smithi092.stdout:6/456: mknod d4/d24/d26/c99 0 2024-05-06T12:14:32.928 INFO:tasks.workunit.client.0.smithi092.stdout:0/436: rename d7/dc/d28/d54/d12/l33 to d7/dc/l8a 0 2024-05-06T12:14:32.930 INFO:tasks.workunit.client.0.smithi092.stdout:8/405: creat d9/d21/d54/d79/f96 x:0 0 0 2024-05-06T12:14:32.932 INFO:tasks.workunit.client.0.smithi092.stdout:0/437: symlink d7/dc/d28/d54/d1e/d39/l8b 0 2024-05-06T12:14:32.934 INFO:tasks.workunit.client.0.smithi092.stdout:0/438: chown d7/dc/d28/d54/d1e/d46/l4c 0 1 2024-05-06T12:14:32.934 INFO:tasks.workunit.client.0.smithi092.stdout:4/320: creat d9/d27/d4a/d59/f64 x:0 0 0 2024-05-06T12:14:32.935 INFO:tasks.workunit.client.0.smithi092.stdout:6/457: symlink d4/d24/d38/d5d/d6d/d87/l9a 0 2024-05-06T12:14:32.938 INFO:tasks.workunit.client.0.smithi092.stdout:0/439: creat d7/d43/f8c x:0 0 0 2024-05-06T12:14:32.944 INFO:tasks.workunit.client.0.smithi092.stdout:3/321: dwrite d8/d1b/f3e [0,4194304] 0 2024-05-06T12:14:32.945 INFO:tasks.workunit.client.0.smithi092.stdout:3/322: chown d8/d24/d33/d39/d5b 0 1 2024-05-06T12:14:32.946 INFO:tasks.workunit.client.0.smithi092.stdout:6/458: getdents d4/d24 0 2024-05-06T12:14:32.948 INFO:tasks.workunit.client.0.smithi092.stdout:3/323: unlink d8/l19 0 2024-05-06T12:14:32.954 INFO:tasks.workunit.client.0.smithi092.stdout:0/440: dread d7/d43/f40 [0,4194304] 0 2024-05-06T12:14:32.956 INFO:tasks.workunit.client.0.smithi092.stdout:0/441: creat d7/dc/f8d x:0 0 0 2024-05-06T12:14:32.959 INFO:tasks.workunit.client.0.smithi092.stdout:4/321: rename d9/d27/d4a/d59 to d9/d2b/d65 0 2024-05-06T12:14:32.961 INFO:tasks.workunit.client.0.smithi092.stdout:4/322: mknod d9/d27/c66 0 2024-05-06T12:14:32.962 INFO:tasks.workunit.client.0.smithi092.stdout:8/406: truncate d9/f42 5260752 0 2024-05-06T12:14:32.964 INFO:tasks.workunit.client.0.smithi092.stdout:8/407: write d9/d21/d37/d46/d5c/d71/f80 [3973316,84264] 0 2024-05-06T12:14:32.965 INFO:tasks.workunit.client.0.smithi092.stdout:8/408: readlink d9/d21/d37/d46/l49 0 2024-05-06T12:14:32.967 INFO:tasks.workunit.client.0.smithi092.stdout:6/459: truncate d4/d24/d26/f32 144357 0 2024-05-06T12:14:32.970 INFO:tasks.workunit.client.0.smithi092.stdout:4/323: link d9/d27/d4a/f44 d9/df/f67 0 2024-05-06T12:14:32.970 INFO:tasks.workunit.client.0.smithi092.stdout:3/324: write d8/d24/d33/f62 [40140,114893] 0 2024-05-06T12:14:32.970 INFO:tasks.workunit.client.0.smithi092.stdout:4/324: chown d9 466902628 1 2024-05-06T12:14:32.972 INFO:tasks.workunit.client.0.smithi092.stdout:8/409: rename d9/d21/d37/d74/f8d to d9/dc/d4a/f97 0 2024-05-06T12:14:32.975 INFO:tasks.workunit.client.0.smithi092.stdout:6/460: dwrite d4/f8e [0,4194304] 0 2024-05-06T12:14:32.976 INFO:tasks.workunit.client.0.smithi092.stdout:3/325: mkdir d8/db/d4c/d40/d67 0 2024-05-06T12:14:32.981 INFO:tasks.workunit.client.0.smithi092.stdout:8/410: symlink d9/d21/d37/d46/d5c/l98 0 2024-05-06T12:14:32.982 INFO:tasks.workunit.client.0.smithi092.stdout:3/326: symlink d8/db/d4c/d3d/l68 0 2024-05-06T12:14:32.985 INFO:tasks.workunit.client.0.smithi092.stdout:3/327: fsync d8/d24/d33/f62 0 2024-05-06T12:14:32.985 INFO:tasks.workunit.client.0.smithi092.stdout:6/461: dread d4/d20/f1b [0,4194304] 0 2024-05-06T12:14:32.987 INFO:tasks.workunit.client.0.smithi092.stdout:4/325: dwrite d9/f47 [0,4194304] 0 2024-05-06T12:14:32.989 INFO:tasks.workunit.client.0.smithi092.stdout:8/411: rename d9/c85 to d9/d21/d37/d46/d5c/c99 0 2024-05-06T12:14:32.990 INFO:tasks.workunit.client.0.smithi092.stdout:3/328: read d8/d24/d33/d4b/f46 [783616,129881] 0 2024-05-06T12:14:32.992 INFO:tasks.workunit.client.0.smithi092.stdout:4/326: creat d9/df/d62/f68 x:0 0 0 2024-05-06T12:14:32.994 INFO:tasks.workunit.client.0.smithi092.stdout:8/412: creat d9/d5a/f9a x:0 0 0 2024-05-06T12:14:33.000 INFO:tasks.workunit.client.0.smithi092.stdout:4/327: creat d9/df/d48/f69 x:0 0 0 2024-05-06T12:14:33.001 INFO:tasks.workunit.client.0.smithi092.stdout:8/413: rmdir d9/dc 39 2024-05-06T12:14:33.003 INFO:tasks.workunit.client.0.smithi092.stdout:4/328: creat d9/d2b/f6a x:0 0 0 2024-05-06T12:14:33.006 INFO:tasks.workunit.client.0.smithi092.stdout:4/329: mknod d9/c6b 0 2024-05-06T12:14:33.007 INFO:tasks.workunit.client.0.smithi092.stdout:6/462: truncate d4/d20/d23/f46 1114938 0 2024-05-06T12:14:33.008 INFO:tasks.workunit.client.0.smithi092.stdout:4/330: unlink d9/df/d48/c53 0 2024-05-06T12:14:33.010 INFO:tasks.workunit.client.0.smithi092.stdout:4/331: symlink d9/d2b/l6c 0 2024-05-06T12:14:33.020 INFO:tasks.workunit.client.0.smithi092.stdout:8/414: dread d9/dc/d4a/f97 [0,4194304] 0 2024-05-06T12:14:33.020 INFO:tasks.workunit.client.0.smithi092.stdout:8/415: chown d9/dc/c8c 513556998 1 2024-05-06T12:14:33.051 INFO:tasks.workunit.client.0.smithi092.stdout:5/367: sync 2024-05-06T12:14:33.051 INFO:tasks.workunit.client.0.smithi092.stdout:1/289: sync 2024-05-06T12:14:33.051 INFO:tasks.workunit.client.0.smithi092.stdout:7/519: sync 2024-05-06T12:14:33.052 INFO:tasks.workunit.client.0.smithi092.stdout:3/329: truncate d8/d24/d27/f42 13734 0 2024-05-06T12:14:33.054 INFO:tasks.workunit.client.0.smithi092.stdout:6/463: unlink d4/d24/d4e/l57 0 2024-05-06T12:14:33.055 INFO:tasks.workunit.client.0.smithi092.stdout:4/332: write d9/f34 [4622514,10399] 0 2024-05-06T12:14:33.055 INFO:tasks.workunit.client.0.smithi092.stdout:8/416: mknod d9/d21/d54/d79/c9b 0 2024-05-06T12:14:33.057 INFO:tasks.workunit.client.0.smithi092.stdout:8/417: read d9/d21/d37/d46/d5c/d62/f68 [59261,43084] 0 2024-05-06T12:14:33.060 INFO:tasks.workunit.client.0.smithi092.stdout:3/330: creat d8/db/d4c/d3d/d49/f69 x:0 0 0 2024-05-06T12:14:33.061 INFO:tasks.workunit.client.0.smithi092.stdout:3/331: write f4 [5095573,104043] 0 2024-05-06T12:14:33.062 INFO:tasks.workunit.client.0.smithi092.stdout:4/333: rmdir d9/d27/d4a 39 2024-05-06T12:14:33.067 INFO:tasks.workunit.client.0.smithi092.stdout:7/520: dwrite d9/de/d10/d17/d21/f49 [0,4194304] 0 2024-05-06T12:14:33.071 INFO:tasks.workunit.client.0.smithi092.stdout:5/368: dread d4/d5/d30/f40 [0,4194304] 0 2024-05-06T12:14:33.075 INFO:tasks.workunit.client.0.smithi092.stdout:6/464: dread d4/d20/f31 [0,4194304] 0 2024-05-06T12:14:33.076 INFO:tasks.workunit.client.0.smithi092.stdout:6/465: stat d4/d20/d22 0 2024-05-06T12:14:33.079 INFO:tasks.workunit.client.0.smithi092.stdout:8/418: creat d9/d21/d37/d46/d5c/d71/d75/f9c x:0 0 0 2024-05-06T12:14:33.079 INFO:tasks.workunit.client.0.smithi092.stdout:7/521: creat d9/de/d10/d17/faf x:0 0 0 2024-05-06T12:14:33.080 INFO:tasks.workunit.client.0.smithi092.stdout:5/369: creat d4/d34/d28/d4e/d4b/f8c x:0 0 0 2024-05-06T12:14:33.082 INFO:tasks.workunit.client.0.smithi092.stdout:4/334: mknod d9/df/c6d 0 2024-05-06T12:14:33.082 INFO:tasks.workunit.client.0.smithi092.stdout:5/370: write d4/d34/d28/d4e/d4b/f84 [255046,48557] 0 2024-05-06T12:14:33.083 INFO:tasks.workunit.client.0.smithi092.stdout:1/290: getdents d3/d10 0 2024-05-06T12:14:33.084 INFO:tasks.workunit.client.0.smithi092.stdout:4/335: write d9/d27/f50 [1734376,19223] 0 2024-05-06T12:14:33.084 INFO:tasks.workunit.client.0.smithi092.stdout:6/466: chown d4/d24/d26/l27 157 1 2024-05-06T12:14:33.085 INFO:tasks.workunit.client.0.smithi092.stdout:4/336: stat d9/d2b/f6a 0 2024-05-06T12:14:33.092 INFO:tasks.workunit.client.0.smithi092.stdout:3/332: rename d8/d1b/f1f to d8/d24/d33/f6a 0 2024-05-06T12:14:33.093 INFO:tasks.workunit.client.0.smithi092.stdout:8/419: mkdir d9/d9d 0 2024-05-06T12:14:33.095 INFO:tasks.workunit.client.0.smithi092.stdout:6/467: dwrite d4/d4c/d52/d62/f71 [0,4194304] 0 2024-05-06T12:14:33.095 INFO:tasks.workunit.client.0.smithi092.stdout:7/522: readlink d9/de/d10/d17/d21/d2e/l41 0 2024-05-06T12:14:33.096 INFO:tasks.workunit.client.0.smithi092.stdout:4/337: dwrite d9/d2b/d65/f64 [0,4194304] 0 2024-05-06T12:14:33.103 INFO:tasks.workunit.client.0.smithi092.stdout:1/291: creat d3/d10/d13/f60 x:0 0 0 2024-05-06T12:14:33.103 INFO:tasks.workunit.client.0.smithi092.stdout:7/523: dread d9/de/d10/d17/d21/f49 [0,4194304] 0 2024-05-06T12:14:33.106 INFO:tasks.workunit.client.0.smithi092.stdout:4/338: write d9/d27/f31 [2597642,65067] 0 2024-05-06T12:14:33.108 INFO:tasks.workunit.client.0.smithi092.stdout:5/371: creat d4/d34/d28/d4e/d5d/d82/d1f/d20/f8d x:0 0 0 2024-05-06T12:14:33.109 INFO:tasks.workunit.client.0.smithi092.stdout:7/524: write d9/de/d1f/f52 [810979,54614] 0 2024-05-06T12:14:33.112 INFO:tasks.workunit.client.0.smithi092.stdout:3/333: dwrite d8/d24/d33/d4b/f46 [4194304,4194304] 0 2024-05-06T12:14:33.114 INFO:tasks.workunit.client.0.smithi092.stdout:5/372: dread d4/d5/d10/d31/d53/d64/f5e [0,4194304] 0 2024-05-06T12:14:33.115 INFO:tasks.workunit.client.0.smithi092.stdout:4/339: chown d9/f43 0 1 2024-05-06T12:14:33.115 INFO:tasks.workunit.client.0.smithi092.stdout:6/468: symlink d4/d4c/d91/d93/l9b 0 2024-05-06T12:14:33.120 INFO:tasks.workunit.client.0.smithi092.stdout:1/292: dwrite d3/d10/d13/f3b [0,4194304] 0 2024-05-06T12:14:33.122 INFO:tasks.workunit.client.0.smithi092.stdout:3/334: rename d8/db/c28 to d8/d24/d33/d39/c6b 0 2024-05-06T12:14:33.123 INFO:tasks.workunit.client.0.smithi092.stdout:5/373: stat f0 0 2024-05-06T12:14:33.124 INFO:tasks.workunit.client.0.smithi092.stdout:3/335: write d8/d1b/f3e [4781660,81962] 0 2024-05-06T12:14:33.124 INFO:tasks.workunit.client.0.smithi092.stdout:8/420: truncate d9/f36 3196256 0 2024-05-06T12:14:33.125 INFO:tasks.workunit.client.0.smithi092.stdout:5/374: write d4/f7d [312487,119799] 0 2024-05-06T12:14:33.127 INFO:tasks.workunit.client.0.smithi092.stdout:4/340: dread d9/d27/f41 [0,4194304] 0 2024-05-06T12:14:33.129 INFO:tasks.workunit.client.0.smithi092.stdout:3/336: truncate d8/d24/d27/f4a 450362 0 2024-05-06T12:14:33.131 INFO:tasks.workunit.client.0.smithi092.stdout:3/337: write d8/f45 [3948601,116470] 0 2024-05-06T12:14:33.131 INFO:tasks.workunit.client.0.smithi092.stdout:1/293: creat d3/d10/d13/d46/f61 x:0 0 0 2024-05-06T12:14:33.136 INFO:tasks.workunit.client.0.smithi092.stdout:5/375: symlink d4/d34/d28/d4e/l8e 0 2024-05-06T12:14:33.136 INFO:tasks.workunit.client.0.smithi092.stdout:4/341: read d9/f3f [364547,62229] 0 2024-05-06T12:14:33.138 INFO:tasks.workunit.client.0.smithi092.stdout:7/525: dread d9/da5/f30 [0,4194304] 0 2024-05-06T12:14:33.140 INFO:tasks.workunit.client.0.smithi092.stdout:1/294: creat d3/d10/d13/d19/d21/d47/d4f/d58/f62 x:0 0 0 2024-05-06T12:14:33.142 INFO:tasks.workunit.client.0.smithi092.stdout:6/469: dwrite d4/d20/d23/d30/f6f [0,4194304] 0 2024-05-06T12:14:33.142 INFO:tasks.workunit.client.0.smithi092.stdout:8/421: rename d9/f29 to d9/dc/f9e 0 2024-05-06T12:14:33.143 INFO:tasks.workunit.client.0.smithi092.stdout:7/526: fsync d9/de/d1f/d37/fa0 0 2024-05-06T12:14:33.146 INFO:tasks.workunit.client.0.smithi092.stdout:3/338: dwrite d8/d24/d33/f64 [0,4194304] 0 2024-05-06T12:14:33.149 INFO:tasks.workunit.client.0.smithi092.stdout:2/282: sync 2024-05-06T12:14:33.151 INFO:tasks.workunit.client.0.smithi092.stdout:4/342: getdents d9/d2b/d5b 0 2024-05-06T12:14:33.153 INFO:tasks.workunit.client.0.smithi092.stdout:8/422: mkdir d9/d5a/d9f 0 2024-05-06T12:14:33.153 INFO:tasks.workunit.client.0.smithi092.stdout:4/343: chown d9/d2b/d5b 707523284 1 2024-05-06T12:14:33.156 INFO:tasks.workunit.client.0.smithi092.stdout:6/470: creat d4/d24/f9c x:0 0 0 2024-05-06T12:14:33.158 INFO:tasks.workunit.client.0.smithi092.stdout:7/527: dwrite d9/de/d2d/f3a [0,4194304] 0 2024-05-06T12:14:33.159 INFO:tasks.workunit.client.0.smithi092.stdout:2/283: rmdir df 39 2024-05-06T12:14:33.160 INFO:tasks.workunit.client.0.smithi092.stdout:6/471: chown d4/d4c/d52/f70 44 1 2024-05-06T12:14:33.160 INFO:tasks.workunit.client.0.smithi092.stdout:1/295: dwrite d3/d10/d13/d19/d21/d47/f5f [0,4194304] 0 2024-05-06T12:14:33.161 INFO:tasks.workunit.client.0.smithi092.stdout:7/528: chown d9/de/d10/d17/d21/f49 3818792 1 2024-05-06T12:14:33.163 INFO:tasks.workunit.client.0.smithi092.stdout:7/529: chown d9/de/d10/d17/d6a/f9d 84 1 2024-05-06T12:14:33.169 INFO:tasks.workunit.client.0.smithi092.stdout:8/423: creat d9/d9d/fa0 x:0 0 0 2024-05-06T12:14:33.171 INFO:tasks.workunit.client.0.smithi092.stdout:5/376: dwrite d4/d34/d28/d4e/d67/f79 [0,4194304] 0 2024-05-06T12:14:33.174 INFO:tasks.workunit.client.0.smithi092.stdout:8/424: write d9/d21/d37/d46/f92 [769158,45668] 0 2024-05-06T12:14:33.174 INFO:tasks.workunit.client.0.smithi092.stdout:7/530: creat d9/de/d1f/d37/fb0 x:0 0 0 2024-05-06T12:14:33.176 INFO:tasks.workunit.client.0.smithi092.stdout:3/339: creat d8/db/f6c x:0 0 0 2024-05-06T12:14:33.180 INFO:tasks.workunit.client.0.smithi092.stdout:3/340: dread d8/d1b/f3e [0,4194304] 0 2024-05-06T12:14:33.181 INFO:tasks.workunit.client.0.smithi092.stdout:5/377: mknod d4/d5/c8f 0 2024-05-06T12:14:33.182 INFO:tasks.workunit.client.0.smithi092.stdout:4/344: truncate d9/fc 3124730 0 2024-05-06T12:14:33.183 INFO:tasks.workunit.client.0.smithi092.stdout:5/378: fdatasync d4/d34/d28/d4e/d4b/f8c 0 2024-05-06T12:14:33.184 INFO:tasks.workunit.client.0.smithi092.stdout:9/259: sync 2024-05-06T12:14:33.184 INFO:tasks.workunit.client.0.smithi092.stdout:0/442: sync 2024-05-06T12:14:33.184 INFO:tasks.workunit.client.0.smithi092.stdout:1/296: dread d3/d10/d13/d19/d21/f24 [0,4194304] 0 2024-05-06T12:14:33.184 INFO:tasks.workunit.client.0.smithi092.stdout:8/425: fdatasync d9/dc/d4a/f97 0 2024-05-06T12:14:33.185 INFO:tasks.workunit.client.0.smithi092.stdout:6/472: truncate d4/d24/d66/f88 845179 0 2024-05-06T12:14:33.186 INFO:tasks.workunit.client.0.smithi092.stdout:2/284: fsync df/d17/d1d/f46 0 2024-05-06T12:14:33.187 INFO:tasks.workunit.client.0.smithi092.stdout:9/260: dread - d16/d2c/d3f/f46 zero size 2024-05-06T12:14:33.187 INFO:tasks.workunit.client.0.smithi092.stdout:7/531: mknod d9/de/cb1 0 2024-05-06T12:14:33.190 INFO:tasks.workunit.client.0.smithi092.stdout:3/341: creat d8/d24/d33/d39/d5b/f6d x:0 0 0 2024-05-06T12:14:33.194 INFO:tasks.workunit.client.0.smithi092.stdout:6/473: fsync d4/d20/f1b 0 2024-05-06T12:14:33.195 INFO:tasks.workunit.client.0.smithi092.stdout:4/345: dwrite d9/f47 [0,4194304] 0 2024-05-06T12:14:33.197 INFO:tasks.workunit.client.0.smithi092.stdout:8/426: write d9/d21/f6f [1001160,60444] 0 2024-05-06T12:14:33.198 INFO:tasks.workunit.client.0.smithi092.stdout:6/474: dread - d4/f76 zero size 2024-05-06T12:14:33.198 INFO:tasks.workunit.client.0.smithi092.stdout:0/443: rename d7/dc/d6f/c74 to d7/d43/c8e 0 2024-05-06T12:14:33.199 INFO:tasks.workunit.client.0.smithi092.stdout:1/297: dwrite d3/f17 [0,4194304] 0 2024-05-06T12:14:33.199 INFO:tasks.workunit.client.0.smithi092.stdout:2/285: symlink df/d17/d1d/l52 0 2024-05-06T12:14:33.202 INFO:tasks.workunit.client.0.smithi092.stdout:3/342: mknod d8/db/d4c/d3d/d49/c6e 0 2024-05-06T12:14:33.202 INFO:tasks.workunit.client.0.smithi092.stdout:7/532: read - d9/de/d10/d17/d21/d2e/d87/f9f zero size 2024-05-06T12:14:33.202 INFO:tasks.workunit.client.0.smithi092.stdout:8/427: fsync d9/d9d/fa0 0 2024-05-06T12:14:33.203 INFO:tasks.workunit.client.0.smithi092.stdout:0/444: write d7/f88 [740092,38141] 0 2024-05-06T12:14:33.205 INFO:tasks.workunit.client.0.smithi092.stdout:6/475: mkdir d4/d4c/d91/d93/d9d 0 2024-05-06T12:14:33.206 INFO:tasks.workunit.client.0.smithi092.stdout:9/261: dwrite d16/d2c/d2f/d3d/f3c [0,4194304] 0 2024-05-06T12:14:33.208 INFO:tasks.workunit.client.0.smithi092.stdout:2/286: mkdir df/d17/d53 0 2024-05-06T12:14:33.208 INFO:tasks.workunit.client.0.smithi092.stdout:3/343: chown d8/d24/d33/d4b/c10 33 1 2024-05-06T12:14:33.210 INFO:tasks.workunit.client.0.smithi092.stdout:8/428: write d9/f1b [527082,72563] 0 2024-05-06T12:14:33.212 INFO:tasks.workunit.client.0.smithi092.stdout:8/429: dread - d9/d5a/f78 zero size 2024-05-06T12:14:33.213 INFO:tasks.workunit.client.0.smithi092.stdout:7/533: mknod d9/de/d10/d17/d21/d38/cb2 0 2024-05-06T12:14:33.213 INFO:tasks.workunit.client.0.smithi092.stdout:8/430: stat d9/f2a 0 2024-05-06T12:14:33.216 INFO:tasks.workunit.client.0.smithi092.stdout:5/379: truncate d4/d5/f2a 2899035 0 2024-05-06T12:14:33.216 INFO:tasks.workunit.client.0.smithi092.stdout:7/534: truncate d9/d62/d9a/fac 886574 0 2024-05-06T12:14:33.221 INFO:tasks.workunit.client.0.smithi092.stdout:9/262: creat d16/d2c/d2f/d3d/d33/d37/d40/d43/f50 x:0 0 0 2024-05-06T12:14:33.221 INFO:tasks.workunit.client.0.smithi092.stdout:4/346: write d9/df/f67 [777032,60646] 0 2024-05-06T12:14:33.221 INFO:tasks.workunit.client.0.smithi092.stdout:0/445: dread d7/dc/d28/d54/d1e/d5d/f68 [0,4194304] 0 2024-05-06T12:14:33.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:32 smithi092 ceph-mon[26954]: pgmap v18: 65 pgs: 65 active+clean; 439 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 3.0 MiB/s rd, 38 MiB/s wr, 310 op/s 2024-05-06T12:14:33.222 INFO:tasks.workunit.client.0.smithi092.stdout:8/431: truncate d9/d21/f3f 753224 0 2024-05-06T12:14:33.223 INFO:tasks.workunit.client.0.smithi092.stdout:2/287: mknod df/d17/d1d/d33/d4d/c54 0 2024-05-06T12:14:33.224 INFO:tasks.workunit.client.0.smithi092.stdout:1/298: link d3/l16 d3/d42/l63 0 2024-05-06T12:14:33.225 INFO:tasks.workunit.client.0.smithi092.stdout:0/446: fdatasync d7/dc/d28/f4d 0 2024-05-06T12:14:33.225 INFO:tasks.workunit.client.0.smithi092.stdout:8/432: read d9/f18 [2390625,118366] 0 2024-05-06T12:14:33.226 INFO:tasks.workunit.client.0.smithi092.stdout:1/299: readlink d3/l2b 0 2024-05-06T12:14:33.226 INFO:tasks.workunit.client.0.smithi092.stdout:5/380: rename d4/d34/d28/d4e/l8e to d4/d34/d29/d54/l90 0 2024-05-06T12:14:33.227 INFO:tasks.workunit.client.0.smithi092.stdout:4/347: stat d9/f38 0 2024-05-06T12:14:33.229 INFO:tasks.workunit.client.0.smithi092.stdout:3/344: creat d8/d24/f6f x:0 0 0 2024-05-06T12:14:33.232 INFO:tasks.workunit.client.0.smithi092.stdout:0/447: dread - d7/dc/d28/d64/f89 zero size 2024-05-06T12:14:33.233 INFO:tasks.workunit.client.0.smithi092.stdout:2/288: chown df/d17/c34 49 1 2024-05-06T12:14:33.235 INFO:tasks.workunit.client.0.smithi092.stdout:4/348: creat d9/d2b/f6e x:0 0 0 2024-05-06T12:14:33.237 INFO:tasks.workunit.client.0.smithi092.stdout:6/476: dwrite d4/d24/d38/f69 [0,4194304] 0 2024-05-06T12:14:33.238 INFO:tasks.workunit.client.0.smithi092.stdout:7/535: rename d9/de/d10/d17/d21/d2e/f32 to d9/de/d10/d17/d99/fb3 0 2024-05-06T12:14:33.245 INFO:tasks.workunit.client.0.smithi092.stdout:9/263: dwrite d16/d1e/f2e [0,4194304] 0 2024-05-06T12:14:33.246 INFO:tasks.workunit.client.0.smithi092.stdout:1/300: symlink d3/d10/d13/d19/d21/d47/d4f/d58/l64 0 2024-05-06T12:14:33.246 INFO:tasks.workunit.client.0.smithi092.stdout:5/381: truncate d4/d34/d28/d4e/d5d/d82/d77/f89 471575 0 2024-05-06T12:14:33.246 INFO:tasks.workunit.client.0.smithi092.stdout:6/477: fdatasync d4/d24/d38/d5d/d6d/d7b/f7f 0 2024-05-06T12:14:33.253 INFO:tasks.workunit.client.0.smithi092.stdout:6/478: dread - d4/d24/d38/d8c/f94 zero size 2024-05-06T12:14:33.256 INFO:tasks.workunit.client.0.smithi092.stdout:8/433: dwrite d9/dc/f4d [0,4194304] 0 2024-05-06T12:14:33.256 INFO:tasks.workunit.client.0.smithi092.stdout:3/345: dwrite d8/d24/d33/d4b/f41 [4194304,4194304] 0 2024-05-06T12:14:33.262 INFO:tasks.workunit.client.0.smithi092.stdout:1/301: mkdir d3/d10/d13/d19/d21/d47/d4f/d58/d65 0 2024-05-06T12:14:33.262 INFO:tasks.workunit.client.0.smithi092.stdout:0/448: dread - d7/dc/d28/d54/d12/f82 zero size 2024-05-06T12:14:33.264 INFO:tasks.workunit.client.0.smithi092.stdout:4/349: mknod d9/d27/d4a/c6f 0 2024-05-06T12:14:33.264 INFO:tasks.workunit.client.0.smithi092.stdout:9/264: creat d16/d2c/d2f/d3d/d33/d37/d40/d43/f51 x:0 0 0 2024-05-06T12:14:33.267 INFO:tasks.workunit.client.0.smithi092.stdout:8/434: truncate d9/d9d/fa0 956065 0 2024-05-06T12:14:33.267 INFO:tasks.workunit.client.0.smithi092.stdout:7/536: dwrite d9/de/d1f/d37/f60 [0,4194304] 0 2024-05-06T12:14:33.269 INFO:tasks.workunit.client.0.smithi092.stdout:2/289: truncate df/f11 4166619 0 2024-05-06T12:14:33.271 INFO:tasks.workunit.client.0.smithi092.stdout:4/350: read d9/df/f15 [1285845,126705] 0 2024-05-06T12:14:33.272 INFO:tasks.workunit.client.0.smithi092.stdout:6/479: write d4/d24/f39 [897591,48435] 0 2024-05-06T12:14:33.274 INFO:tasks.workunit.client.0.smithi092.stdout:1/302: creat d3/d10/d13/d19/d34/f66 x:0 0 0 2024-05-06T12:14:33.274 INFO:tasks.workunit.client.0.smithi092.stdout:7/537: write d9/f80 [264759,54778] 0 2024-05-06T12:14:33.276 INFO:tasks.workunit.client.0.smithi092.stdout:9/265: rename d16/d1e/f3e to d16/d1c/f52 0 2024-05-06T12:14:33.276 INFO:tasks.workunit.client.0.smithi092.stdout:7/538: readlink d9/da5/l3c 0 2024-05-06T12:14:33.278 INFO:tasks.workunit.client.0.smithi092.stdout:5/382: dwrite d4/d34/d28/d4e/d4b/f84 [0,4194304] 0 2024-05-06T12:14:33.278 INFO:tasks.workunit.client.0.smithi092.stdout:2/290: creat df/f55 x:0 0 0 2024-05-06T12:14:33.280 INFO:tasks.workunit.client.0.smithi092.stdout:6/480: rmdir d4/d24/d66 39 2024-05-06T12:14:33.280 INFO:tasks.workunit.client.0.smithi092.stdout:9/266: truncate d16/d2c/f4f 137098 0 2024-05-06T12:14:33.281 INFO:tasks.workunit.client.0.smithi092.stdout:7/539: chown d9/de/d1f/d37/d48/d65/f7c 2849 1 2024-05-06T12:14:33.282 INFO:tasks.workunit.client.0.smithi092.stdout:7/540: readlink d9/de/d2d/l58 0 2024-05-06T12:14:33.283 INFO:tasks.workunit.client.0.smithi092.stdout:1/303: mknod d3/d10/d13/d19/d34/c67 0 2024-05-06T12:14:33.284 INFO:tasks.workunit.client.0.smithi092.stdout:2/291: symlink df/d15/l56 0 2024-05-06T12:14:33.285 INFO:tasks.workunit.client.0.smithi092.stdout:1/304: readlink d3/d10/l29 0 2024-05-06T12:14:33.285 INFO:tasks.workunit.client.0.smithi092.stdout:5/383: mknod d4/d5/d30/c91 0 2024-05-06T12:14:33.287 INFO:tasks.workunit.client.0.smithi092.stdout:7/541: write d9/de/d1f/d37/d48/d65/f66 [1458722,109901] 0 2024-05-06T12:14:33.289 INFO:tasks.workunit.client.0.smithi092.stdout:4/351: creat d9/f70 x:0 0 0 2024-05-06T12:14:33.293 INFO:tasks.workunit.client.0.smithi092.stdout:4/352: write d9/df/f46 [2474858,83296] 0 2024-05-06T12:14:33.293 INFO:tasks.workunit.client.0.smithi092.stdout:0/449: rename d7/dc/d28/d83 to d7/dc/d28/d54/d12/d8f 0 2024-05-06T12:14:33.295 INFO:tasks.workunit.client.0.smithi092.stdout:9/267: dwrite d16/d2c/d2f/d3d/d33/d37/d40/d43/f50 [0,4194304] 0 2024-05-06T12:14:33.297 INFO:tasks.workunit.client.0.smithi092.stdout:0/450: readlink d7/dc/d28/d54/l51 0 2024-05-06T12:14:33.299 INFO:tasks.workunit.client.0.smithi092.stdout:7/542: write d9/dd/f9b [230560,126116] 0 2024-05-06T12:14:33.300 INFO:tasks.workunit.client.0.smithi092.stdout:6/481: dwrite d4/f6 [0,4194304] 0 2024-05-06T12:14:33.300 INFO:tasks.workunit.client.0.smithi092.stdout:9/268: write d16/d1e/f28 [3801082,63905] 0 2024-05-06T12:14:33.300 INFO:tasks.workunit.client.0.smithi092.stdout:2/292: creat df/d17/d51/f57 x:0 0 0 2024-05-06T12:14:33.306 INFO:tasks.workunit.client.0.smithi092.stdout:4/353: chown d9/dd/l4e 7261 1 2024-05-06T12:14:33.307 INFO:tasks.workunit.client.0.smithi092.stdout:5/384: creat d4/d34/d29/d54/d73/f92 x:0 0 0 2024-05-06T12:14:33.307 INFO:tasks.workunit.client.0.smithi092.stdout:4/354: chown d9/f43 573381 1 2024-05-06T12:14:33.307 INFO:tasks.workunit.client.0.smithi092.stdout:8/435: truncate d9/dc/ff 2222954 0 2024-05-06T12:14:33.309 INFO:tasks.workunit.client.0.smithi092.stdout:0/451: dwrite d7/f9 [4194304,4194304] 0 2024-05-06T12:14:33.312 INFO:tasks.workunit.client.0.smithi092.stdout:1/305: link d3/d10/d13/d19/f5a d3/d10/d13/d46/f68 0 2024-05-06T12:14:33.315 INFO:tasks.workunit.client.0.smithi092.stdout:1/306: chown d3/lb 877 1 2024-05-06T12:14:33.316 INFO:tasks.workunit.client.0.smithi092.stdout:2/293: dread - df/d17/f3e zero size 2024-05-06T12:14:33.318 INFO:tasks.workunit.client.0.smithi092.stdout:7/543: unlink d9/de/d1f/f4c 0 2024-05-06T12:14:33.322 INFO:tasks.workunit.client.0.smithi092.stdout:5/385: write d4/d34/d28/f74 [4057265,5514] 0 2024-05-06T12:14:33.322 INFO:tasks.workunit.client.0.smithi092.stdout:4/355: creat d9/d27/d4a/f71 x:0 0 0 2024-05-06T12:14:33.323 INFO:tasks.workunit.client.0.smithi092.stdout:0/452: mkdir d7/dc/d28/d54/d1e/d46/d63/d6e/d90 0 2024-05-06T12:14:33.331 INFO:tasks.workunit.client.0.smithi092.stdout:5/386: mknod d4/d5/d10/d31/c93 0 2024-05-06T12:14:33.331 INFO:tasks.workunit.client.0.smithi092.stdout:6/482: dwrite d4/d24/d66/f88 [0,4194304] 0 2024-05-06T12:14:33.333 INFO:tasks.workunit.client.0.smithi092.stdout:4/356: write d9/d27/f41 [684567,103866] 0 2024-05-06T12:14:33.335 INFO:tasks.workunit.client.0.smithi092.stdout:0/453: mknod d7/dc/d28/d54/d1e/d46/c91 0 2024-05-06T12:14:33.339 INFO:tasks.workunit.client.0.smithi092.stdout:1/307: link d3/d10/d13/d19/d21/d47/d4f/d58/f62 d3/d10/d13/d19/d21/d47/d4f/d58/f69 0 2024-05-06T12:14:33.341 INFO:tasks.workunit.client.0.smithi092.stdout:7/544: chown d9/de/d10/d17/d21/d2e/d9c 0 1 2024-05-06T12:14:33.344 INFO:tasks.workunit.client.0.smithi092.stdout:4/357: dwrite d9/df/f46 [4194304,4194304] 0 2024-05-06T12:14:33.345 INFO:tasks.workunit.client.0.smithi092.stdout:0/454: dread d7/dc/d28/d54/d1e/f3e [0,4194304] 0 2024-05-06T12:14:33.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:32 smithi190 ceph-mon[33798]: pgmap v18: 65 pgs: 65 active+clean; 439 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 3.0 MiB/s rd, 38 MiB/s wr, 310 op/s 2024-05-06T12:14:33.354 INFO:tasks.workunit.client.0.smithi092.stdout:9/269: rename c13 to d16/d2c/d2f/d3d/d33/c53 0 2024-05-06T12:14:33.355 INFO:tasks.workunit.client.0.smithi092.stdout:8/436: link d9/c60 d9/dc/ca1 0 2024-05-06T12:14:33.356 INFO:tasks.workunit.client.0.smithi092.stdout:5/387: unlink d4/d5/d10/c42 0 2024-05-06T12:14:33.357 INFO:tasks.workunit.client.0.smithi092.stdout:4/358: truncate d9/fc 2790704 0 2024-05-06T12:14:33.360 INFO:tasks.workunit.client.0.smithi092.stdout:2/294: rename df/d15/c43 to df/d17/d51/c58 0 2024-05-06T12:14:33.361 INFO:tasks.workunit.client.0.smithi092.stdout:9/270: write d16/d2c/d2f/f36 [2718358,40771] 0 2024-05-06T12:14:33.362 INFO:tasks.workunit.client.0.smithi092.stdout:2/295: chown df/d17/d1d/d33/f47 10507 1 2024-05-06T12:14:33.364 INFO:tasks.workunit.client.0.smithi092.stdout:4/359: write d9/f21 [8955806,677] 0 2024-05-06T12:14:33.365 INFO:tasks.workunit.client.0.smithi092.stdout:5/388: write d4/d5/d10/d31/d53/d64/f8b [977304,63887] 0 2024-05-06T12:14:33.369 INFO:tasks.workunit.client.0.smithi092.stdout:7/545: rename d9/de/d10/d17/l6c to d9/de/d10/d17/d21/d2e/d9c/lb4 0 2024-05-06T12:14:33.371 INFO:tasks.workunit.client.0.smithi092.stdout:9/271: dread d16/d2c/d2f/d3d/f3c [0,4194304] 0 2024-05-06T12:14:33.375 INFO:tasks.workunit.client.0.smithi092.stdout:7/546: write d9/de/d1f/d37/d48/d65/f7c [387372,68630] 0 2024-05-06T12:14:33.375 INFO:tasks.workunit.client.0.smithi092.stdout:9/272: write d16/d1e/f42 [5276185,108000] 0 2024-05-06T12:14:33.375 INFO:tasks.workunit.client.0.smithi092.stdout:4/360: dwrite d9/f70 [0,4194304] 0 2024-05-06T12:14:33.375 INFO:tasks.workunit.client.0.smithi092.stdout:6/483: truncate d4/d20/d23/d30/f6f 294367 0 2024-05-06T12:14:33.378 INFO:tasks.workunit.client.0.smithi092.stdout:9/273: fsync d16/d2c/d3f/f46 0 2024-05-06T12:14:33.380 INFO:tasks.workunit.client.0.smithi092.stdout:5/389: dwrite d4/f9 [0,4194304] 0 2024-05-06T12:14:33.380 INFO:tasks.workunit.client.0.smithi092.stdout:0/455: creat d7/dc/d28/d54/d1e/d26/f92 x:0 0 0 2024-05-06T12:14:33.380 INFO:tasks.workunit.client.0.smithi092.stdout:1/308: truncate d3/fa 960195 0 2024-05-06T12:14:33.381 INFO:tasks.workunit.client.0.smithi092.stdout:9/274: fdatasync d16/d2c/d2f/f3a 0 2024-05-06T12:14:33.382 INFO:tasks.workunit.client.0.smithi092.stdout:8/437: creat d9/fa2 x:0 0 0 2024-05-06T12:14:33.382 INFO:tasks.workunit.client.0.smithi092.stdout:4/361: write d9/d27/d4a/f71 [532815,129293] 0 2024-05-06T12:14:33.383 INFO:tasks.workunit.client.0.smithi092.stdout:1/309: chown d3/f3d 1 1 2024-05-06T12:14:33.385 INFO:tasks.workunit.client.0.smithi092.stdout:2/296: mkdir df/d15/d50/d59 0 2024-05-06T12:14:33.387 INFO:tasks.workunit.client.0.smithi092.stdout:2/297: fsync df/d1b/f48 0 2024-05-06T12:14:33.389 INFO:tasks.workunit.client.0.smithi092.stdout:6/484: stat d4/d24/c8b 0 2024-05-06T12:14:33.390 INFO:tasks.workunit.client.0.smithi092.stdout:0/456: read d7/dc/d28/d54/d1e/d5d/f7c [558880,126744] 0 2024-05-06T12:14:33.392 INFO:tasks.workunit.client.0.smithi092.stdout:7/547: rename d9/de/d10/d17/d6a/f92 to d9/de/d10/d17/fb5 0 2024-05-06T12:14:33.392 INFO:tasks.workunit.client.0.smithi092.stdout:6/485: write d4/d20/d23/d30/f5e [490514,25280] 0 2024-05-06T12:14:33.396 INFO:tasks.workunit.client.0.smithi092.stdout:5/390: dwrite d4/d34/d28/d4e/d4b/f8c [0,4194304] 0 2024-05-06T12:14:33.403 INFO:tasks.workunit.client.0.smithi092.stdout:9/275: creat d16/d2c/d4b/f54 x:0 0 0 2024-05-06T12:14:33.404 INFO:tasks.workunit.client.0.smithi092.stdout:5/391: dread d4/d34/d28/d4e/d4b/f84 [0,4194304] 0 2024-05-06T12:14:33.406 INFO:tasks.workunit.client.0.smithi092.stdout:9/276: read fc [3783281,40995] 0 2024-05-06T12:14:33.407 INFO:tasks.workunit.client.0.smithi092.stdout:6/486: creat d4/d20/d22/f9e x:0 0 0 2024-05-06T12:14:33.411 INFO:tasks.workunit.client.0.smithi092.stdout:0/457: dwrite d7/d43/d1f/f4a [0,4194304] 0 2024-05-06T12:14:33.414 INFO:tasks.workunit.client.0.smithi092.stdout:8/438: creat d9/d21/d37/fa3 x:0 0 0 2024-05-06T12:14:33.419 INFO:tasks.workunit.client.0.smithi092.stdout:7/548: dwrite d9/f6f [0,4194304] 0 2024-05-06T12:14:33.419 INFO:tasks.workunit.client.0.smithi092.stdout:2/298: link df/d15/l56 df/d15/d50/d59/l5a 0 2024-05-06T12:14:33.423 INFO:tasks.workunit.client.0.smithi092.stdout:8/439: write d9/dc/d4a/f6b [146140,114986] 0 2024-05-06T12:14:33.423 INFO:tasks.workunit.client.0.smithi092.stdout:1/310: rename d3/f17 to d3/f6a 0 2024-05-06T12:14:33.427 INFO:tasks.workunit.client.0.smithi092.stdout:6/487: dwrite d4/d4c/d52/d62/f71 [0,4194304] 0 2024-05-06T12:14:33.429 INFO:tasks.workunit.client.0.smithi092.stdout:1/311: dread - d3/d10/d13/d19/d34/f66 zero size 2024-05-06T12:14:33.430 INFO:tasks.workunit.client.0.smithi092.stdout:0/458: read d7/dc/d28/d54/d1e/d26/d3a/f52 [3249135,59157] 0 2024-05-06T12:14:33.431 INFO:tasks.workunit.client.0.smithi092.stdout:1/312: read - d3/d10/d13/d46/f68 zero size 2024-05-06T12:14:33.434 INFO:tasks.workunit.client.0.smithi092.stdout:0/459: fsync d7/dc/d28/d54/d1e/d46/d63/d6e/f87 0 2024-05-06T12:14:33.435 INFO:tasks.workunit.client.0.smithi092.stdout:7/549: mkdir d9/de/d10/d17/d99/db6 0 2024-05-06T12:14:33.436 INFO:tasks.workunit.client.0.smithi092.stdout:6/488: read d4/d20/d23/f67 [863295,112058] 0 2024-05-06T12:14:33.438 INFO:tasks.workunit.client.0.smithi092.stdout:0/460: truncate d7/dc/f5c 700927 0 2024-05-06T12:14:33.438 INFO:tasks.workunit.client.0.smithi092.stdout:4/362: dread d9/f22 [0,4194304] 0 2024-05-06T12:14:33.439 INFO:tasks.workunit.client.0.smithi092.stdout:0/461: write d7/d43/f49 [3974418,100011] 0 2024-05-06T12:14:33.441 INFO:tasks.workunit.client.0.smithi092.stdout:1/313: rename d3/d10/d13/d19/d21/l5b to d3/d10/d13/d46/l6b 0 2024-05-06T12:14:33.442 INFO:tasks.workunit.client.0.smithi092.stdout:5/392: write d4/d5/d10/d31/d53/d64/f5e [791092,117228] 0 2024-05-06T12:14:33.451 INFO:tasks.workunit.client.0.smithi092.stdout:9/277: getdents d16/d2c/d2f/d3d/d33/d37/d40 0 2024-05-06T12:14:33.453 INFO:tasks.workunit.client.0.smithi092.stdout:5/393: write d4/d5/d10/d31/d53/d64/f25 [1398243,73545] 0 2024-05-06T12:14:33.453 INFO:tasks.workunit.client.0.smithi092.stdout:5/394: chown d4/d34 3 1 2024-05-06T12:14:33.459 INFO:tasks.workunit.client.0.smithi092.stdout:9/278: creat d16/d1e/f55 x:0 0 0 2024-05-06T12:14:33.459 INFO:tasks.workunit.client.0.smithi092.stdout:2/299: dread df/f39 [0,4194304] 0 2024-05-06T12:14:33.460 INFO:tasks.workunit.client.0.smithi092.stdout:5/395: mkdir d4/d34/d28/d4e/d5d/d94 0 2024-05-06T12:14:33.462 INFO:tasks.workunit.client.0.smithi092.stdout:0/462: creat d7/dc/d28/d54/d12/f93 x:0 0 0 2024-05-06T12:14:33.462 INFO:tasks.workunit.client.0.smithi092.stdout:1/314: truncate d3/f26 299326 0 2024-05-06T12:14:33.466 INFO:tasks.workunit.client.0.smithi092.stdout:7/550: link d9/de/da4/la9 d9/de/d10/d17/d99/da6/lb7 0 2024-05-06T12:14:33.469 INFO:tasks.workunit.client.0.smithi092.stdout:0/463: rename d7/c20 to d7/dc/d28/d54/d1e/d39/c94 0 2024-05-06T12:14:33.470 INFO:tasks.workunit.client.0.smithi092.stdout:8/440: dread d9/dc/f51 [4194304,4194304] 0 2024-05-06T12:14:33.473 INFO:tasks.workunit.client.0.smithi092.stdout:5/396: dwrite d4/d5/d10/d31/d53/d64/f5e [0,4194304] 0 2024-05-06T12:14:33.475 INFO:tasks.workunit.client.0.smithi092.stdout:7/551: symlink d9/de/da4/lb8 0 2024-05-06T12:14:33.476 INFO:tasks.workunit.client.0.smithi092.stdout:1/315: dwrite d3/d10/f37 [0,4194304] 0 2024-05-06T12:14:33.477 INFO:tasks.workunit.client.0.smithi092.stdout:4/363: dread d9/df/d48/f56 [0,4194304] 0 2024-05-06T12:14:33.478 INFO:tasks.workunit.client.0.smithi092.stdout:9/279: creat d16/d2c/d2f/d3d/d33/f56 x:0 0 0 2024-05-06T12:14:33.478 INFO:tasks.workunit.client.0.smithi092.stdout:0/464: write d7/d43/f40 [176514,124198] 0 2024-05-06T12:14:33.482 INFO:tasks.workunit.client.0.smithi092.stdout:2/300: write df/d17/d1d/f2e [4476366,72152] 0 2024-05-06T12:14:33.483 INFO:tasks.workunit.client.0.smithi092.stdout:9/280: write d16/f29 [122221,122500] 0 2024-05-06T12:14:33.485 INFO:tasks.workunit.client.0.smithi092.stdout:0/465: chown d7/dc/d6f/c84 14929877 1 2024-05-06T12:14:33.485 INFO:tasks.workunit.client.0.smithi092.stdout:1/316: write d3/d10/d13/d19/f5d [1181535,90795] 0 2024-05-06T12:14:33.488 INFO:tasks.workunit.client.0.smithi092.stdout:6/489: dwrite d4/d24/f39 [0,4194304] 0 2024-05-06T12:14:33.488 INFO:tasks.workunit.client.0.smithi092.stdout:5/397: symlink d4/l95 0 2024-05-06T12:14:33.489 INFO:tasks.workunit.client.0.smithi092.stdout:2/301: dread - df/d17/d1d/d33/f44 zero size 2024-05-06T12:14:33.489 INFO:tasks.workunit.client.0.smithi092.stdout:1/317: chown d3/d10/d13/d19/d21/f2f 1 1 2024-05-06T12:14:33.490 INFO:tasks.workunit.client.0.smithi092.stdout:7/552: unlink d9/de/d10/d17/d21/d2e/f4f 0 2024-05-06T12:14:33.493 INFO:tasks.workunit.client.0.smithi092.stdout:8/441: rename d9/f1b to d9/dc/fa4 0 2024-05-06T12:14:33.495 INFO:tasks.workunit.client.0.smithi092.stdout:0/466: creat d7/dc/d28/d54/d1e/d26/d3a/f95 x:0 0 0 2024-05-06T12:14:33.497 INFO:tasks.workunit.client.0.smithi092.stdout:7/553: truncate d9/de/d10/d17/d21/f3e 1373025 0 2024-05-06T12:14:33.497 INFO:tasks.workunit.client.0.smithi092.stdout:8/442: dread d9/d21/f3f [0,4194304] 0 2024-05-06T12:14:33.497 INFO:tasks.workunit.client.0.smithi092.stdout:6/490: chown d4/d24/d4e 0 1 2024-05-06T12:14:33.498 INFO:tasks.workunit.client.0.smithi092.stdout:5/398: creat d4/d5/f96 x:0 0 0 2024-05-06T12:14:33.499 INFO:tasks.workunit.client.0.smithi092.stdout:0/467: truncate d7/dc/d28/d54/d1e/d46/f67 989284 0 2024-05-06T12:14:33.500 INFO:tasks.workunit.client.0.smithi092.stdout:9/281: mkdir d16/d2c/d2f/d3d/d33/d57 0 2024-05-06T12:14:33.502 INFO:tasks.workunit.client.0.smithi092.stdout:9/282: chown d16/d2c/d2f/d3d/d33/d37/d40/d43/f50 57045366 1 2024-05-06T12:14:33.505 INFO:tasks.workunit.client.0.smithi092.stdout:7/554: write d9/de/d10/d17/d21/f8d [1094600,118268] 0 2024-05-06T12:14:33.507 INFO:tasks.workunit.client.0.smithi092.stdout:5/399: unlink d4/d5/d10/c7b 0 2024-05-06T12:14:33.507 INFO:tasks.workunit.client.0.smithi092.stdout:7/555: stat d9/de/d10/d17/d21/d38/c55 0 2024-05-06T12:14:33.507 INFO:tasks.workunit.client.0.smithi092.stdout:7/556: stat d9/de/c44 0 2024-05-06T12:14:33.508 INFO:tasks.workunit.client.0.smithi092.stdout:4/364: dread d9/dd/f17 [0,4194304] 0 2024-05-06T12:14:33.508 INFO:tasks.workunit.client.0.smithi092.stdout:1/318: rename d3/d10/c1b to d3/d10/d13/d19/d21/c6c 0 2024-05-06T12:14:33.509 INFO:tasks.workunit.client.0.smithi092.stdout:7/557: fsync d9/de/d1f/f52 0 2024-05-06T12:14:33.510 INFO:tasks.workunit.client.0.smithi092.stdout:9/283: mkdir d16/d1c/d58 0 2024-05-06T12:14:33.511 INFO:tasks.workunit.client.0.smithi092.stdout:5/400: mknod d4/d5/d10/d31/c97 0 2024-05-06T12:14:33.513 INFO:tasks.workunit.client.0.smithi092.stdout:5/401: truncate d4/d5/f96 37382 0 2024-05-06T12:14:33.516 INFO:tasks.workunit.client.0.smithi092.stdout:9/284: write d16/f1f [4617603,74973] 0 2024-05-06T12:14:33.518 INFO:tasks.workunit.client.0.smithi092.stdout:6/491: dread d4/d20/d23/f46 [0,4194304] 0 2024-05-06T12:14:33.519 INFO:tasks.workunit.client.0.smithi092.stdout:1/319: dwrite d3/d10/d13/d19/d21/f2f [0,4194304] 0 2024-05-06T12:14:33.520 INFO:tasks.workunit.client.0.smithi092.stdout:0/468: rename d7/dc/d28/d54/d1e/d46/d63/d6e to d7/dc/d28/d54/d1e/d26/d96 0 2024-05-06T12:14:33.523 INFO:tasks.workunit.client.0.smithi092.stdout:7/558: creat d9/de/d10/d17/d6a/fb9 x:0 0 0 2024-05-06T12:14:33.525 INFO:tasks.workunit.client.0.smithi092.stdout:7/559: read d9/de/d10/d17/d21/f8d [681932,86823] 0 2024-05-06T12:14:33.527 INFO:tasks.workunit.client.0.smithi092.stdout:1/320: symlink d3/d10/d13/d19/d21/d47/d4f/d58/l6d 0 2024-05-06T12:14:33.529 INFO:tasks.workunit.client.0.smithi092.stdout:0/469: mkdir d7/dc/d97 0 2024-05-06T12:14:33.530 INFO:tasks.workunit.client.0.smithi092.stdout:9/285: creat d16/d2c/d2f/d3d/d33/d57/f59 x:0 0 0 2024-05-06T12:14:33.531 INFO:tasks.workunit.client.0.smithi092.stdout:9/286: write d16/f17 [1016204,3465] 0 2024-05-06T12:14:33.533 INFO:tasks.workunit.client.0.smithi092.stdout:6/492: mkdir d4/d24/d9f 0 2024-05-06T12:14:33.536 INFO:tasks.workunit.client.0.smithi092.stdout:0/470: dwrite d7/dc/d28/d54/d1e/d26/d3a/f5f [0,4194304] 0 2024-05-06T12:14:33.537 INFO:tasks.workunit.client.0.smithi092.stdout:0/471: fdatasync d7/d43/f61 0 2024-05-06T12:14:33.537 INFO:tasks.workunit.client.0.smithi092.stdout:0/472: chown d7/f88 235 1 2024-05-06T12:14:33.538 INFO:tasks.workunit.client.0.smithi092.stdout:1/321: unlink d3/d10/d13/d19/d21/f22 0 2024-05-06T12:14:33.540 INFO:tasks.workunit.client.0.smithi092.stdout:1/322: write d3/d10/d13/f60 [7958,90418] 0 2024-05-06T12:14:33.544 INFO:tasks.workunit.client.0.smithi092.stdout:4/365: truncate d9/f21 7570393 0 2024-05-06T12:14:33.545 INFO:tasks.workunit.client.0.smithi092.stdout:5/402: getdents d4/d34/d29/d54/d73 0 2024-05-06T12:14:33.550 INFO:tasks.workunit.client.0.smithi092.stdout:9/287: unlink d16/d2c/d2f/d3d/d33/c53 0 2024-05-06T12:14:33.551 INFO:tasks.workunit.client.0.smithi092.stdout:9/288: read d16/d2c/d2f/d3d/d33/d37/d40/d43/f50 [2310976,93182] 0 2024-05-06T12:14:33.554 INFO:tasks.workunit.client.0.smithi092.stdout:6/493: mkdir d4/d4c/d52/d6c/da0 0 2024-05-06T12:14:33.556 INFO:tasks.workunit.client.0.smithi092.stdout:7/560: link d9/de/d10/d17/d21/d2e/d9c/l9e d9/de/d1f/d37/d48/lba 0 2024-05-06T12:14:33.557 INFO:tasks.workunit.client.0.smithi092.stdout:0/473: symlink d7/dc/d28/l98 0 2024-05-06T12:14:33.558 INFO:tasks.workunit.client.0.smithi092.stdout:1/323: mknod d3/d10/d13/d19/d21/d47/d4f/d58/d65/c6e 0 2024-05-06T12:14:33.558 INFO:tasks.workunit.client.0.smithi092.stdout:0/474: chown d7/dc/d28/d54/d1e/d26/d4e 3 1 2024-05-06T12:14:33.559 INFO:tasks.workunit.client.0.smithi092.stdout:0/475: fdatasync d7/dc/d28/d54/f3b 0 2024-05-06T12:14:33.560 INFO:tasks.workunit.client.0.smithi092.stdout:0/476: chown d7/dc/f5c 1263083 1 2024-05-06T12:14:33.560 INFO:tasks.workunit.client.0.smithi092.stdout:4/366: rmdir d9 39 2024-05-06T12:14:33.564 INFO:tasks.workunit.client.0.smithi092.stdout:7/561: truncate d9/da5/f30 890451 0 2024-05-06T12:14:33.566 INFO:tasks.workunit.client.0.smithi092.stdout:0/477: dread d7/dc/f29 [0,4194304] 0 2024-05-06T12:14:33.573 INFO:tasks.workunit.client.0.smithi092.stdout:5/403: truncate d4/d5/d30/f3c 651531 0 2024-05-06T12:14:33.573 INFO:tasks.workunit.client.0.smithi092.stdout:5/404: dread - d4/d59/f83 zero size 2024-05-06T12:14:33.574 INFO:tasks.workunit.client.0.smithi092.stdout:5/405: dread - d4/d34/d28/d4e/f87 zero size 2024-05-06T12:14:33.577 INFO:tasks.workunit.client.0.smithi092.stdout:7/562: write d9/f5a [712657,114729] 0 2024-05-06T12:14:33.577 INFO:tasks.workunit.client.0.smithi092.stdout:9/289: link d16/d1c/l39 d16/d1c/d58/l5a 0 2024-05-06T12:14:33.578 INFO:tasks.workunit.client.0.smithi092.stdout:0/478: dread d7/dc/d28/d54/d1e/d39/f47 [0,4194304] 0 2024-05-06T12:14:33.580 INFO:tasks.workunit.client.0.smithi092.stdout:6/494: link d4/l3f d4/d4c/la1 0 2024-05-06T12:14:33.581 INFO:tasks.workunit.client.0.smithi092.stdout:9/290: chown d16/d2c/d3f/l4a 1 1 2024-05-06T12:14:33.582 INFO:tasks.workunit.client.0.smithi092.stdout:5/406: mknod d4/d5/d10/d31/c98 0 2024-05-06T12:14:33.582 INFO:tasks.workunit.client.0.smithi092.stdout:9/291: write d16/d2c/d2f/f45 [473558,79847] 0 2024-05-06T12:14:33.583 INFO:tasks.workunit.client.0.smithi092.stdout:4/367: symlink d9/d27/l72 0 2024-05-06T12:14:33.584 INFO:tasks.workunit.client.0.smithi092.stdout:4/368: fdatasync d9/d27/f31 0 2024-05-06T12:14:33.584 INFO:tasks.workunit.client.0.smithi092.stdout:9/292: rename d16/d2c to d16/d2c/d4b/d5b 22 2024-05-06T12:14:33.585 INFO:tasks.workunit.client.0.smithi092.stdout:5/407: stat d4/d34/d28/d4e/d5d/d82/d1f/d20/f8d 0 2024-05-06T12:14:33.586 INFO:tasks.workunit.client.0.smithi092.stdout:4/369: chown d9/df/f67 27405 1 2024-05-06T12:14:33.587 INFO:tasks.workunit.client.0.smithi092.stdout:4/370: fsync d9/d27/f4d 0 2024-05-06T12:14:33.588 INFO:tasks.workunit.client.0.smithi092.stdout:6/495: symlink d4/d20/d22/la2 0 2024-05-06T12:14:33.590 INFO:tasks.workunit.client.0.smithi092.stdout:9/293: symlink d16/d2c/d4b/l5c 0 2024-05-06T12:14:33.596 INFO:tasks.workunit.client.0.smithi092.stdout:5/408: read d4/d5/f2a [2663049,108691] 0 2024-05-06T12:14:33.596 INFO:tasks.workunit.client.0.smithi092.stdout:9/294: write d16/d2c/f4f [365226,18852] 0 2024-05-06T12:14:33.598 INFO:tasks.workunit.client.0.smithi092.stdout:7/563: dwrite d9/de/d10/d17/d6a/f7a [0,4194304] 0 2024-05-06T12:14:33.599 INFO:tasks.workunit.client.0.smithi092.stdout:4/371: mknod d9/df/c73 0 2024-05-06T12:14:33.600 INFO:tasks.workunit.client.0.smithi092.stdout:1/324: dwrite d3/f5 [0,4194304] 0 2024-05-06T12:14:33.600 INFO:tasks.workunit.client.0.smithi092.stdout:7/564: readlink d9/da5/l3c 0 2024-05-06T12:14:33.601 INFO:tasks.workunit.client.0.smithi092.stdout:0/479: dread d7/dc/d28/d54/d1e/d26/f2b [0,4194304] 0 2024-05-06T12:14:33.602 INFO:tasks.workunit.client.0.smithi092.stdout:6/496: getdents d4/d24/d66 0 2024-05-06T12:14:33.607 INFO:tasks.workunit.client.0.smithi092.stdout:7/565: stat d9/da5/l7f 0 2024-05-06T12:14:33.608 INFO:tasks.workunit.client.0.smithi092.stdout:4/372: dwrite d9/d27/f50 [0,4194304] 0 2024-05-06T12:14:33.610 INFO:tasks.workunit.client.0.smithi092.stdout:7/566: write d9/de/d1f/d37/d48/d65/f73 [1227209,35208] 0 2024-05-06T12:14:33.612 INFO:tasks.workunit.client.0.smithi092.stdout:9/295: write f6 [533315,111645] 0 2024-05-06T12:14:33.613 INFO:tasks.workunit.client.0.smithi092.stdout:5/409: mkdir d4/d34/d28/d4e/d5d/d82/d1f/d20/d99 0 2024-05-06T12:14:33.615 INFO:tasks.workunit.client.0.smithi092.stdout:6/497: creat d4/d24/d38/d5d/fa3 x:0 0 0 2024-05-06T12:14:33.617 INFO:tasks.workunit.client.0.smithi092.stdout:0/480: mknod d7/dc/d28/d54/d1e/c99 0 2024-05-06T12:14:33.621 INFO:tasks.workunit.client.0.smithi092.stdout:0/481: write d7/f88 [751458,65272] 0 2024-05-06T12:14:33.623 INFO:tasks.workunit.client.0.smithi092.stdout:0/482: readlink d7/dc/d28/d54/l51 0 2024-05-06T12:14:33.623 INFO:tasks.workunit.client.0.smithi092.stdout:5/410: symlink d4/d5/d30/l9a 0 2024-05-06T12:14:33.625 INFO:tasks.workunit.client.0.smithi092.stdout:6/498: dwrite d4/d20/f83 [0,4194304] 0 2024-05-06T12:14:33.627 INFO:tasks.workunit.client.0.smithi092.stdout:0/483: write d7/dc/d28/d54/d1e/d26/f36 [1395400,26133] 0 2024-05-06T12:14:33.629 INFO:tasks.workunit.client.0.smithi092.stdout:7/567: link d9/de/d1f/d37/d48/d65/l84 d9/de/d10/d17/d99/da6/lbb 0 2024-05-06T12:14:33.630 INFO:tasks.workunit.client.0.smithi092.stdout:4/373: getdents d9/df/d62 0 2024-05-06T12:14:33.633 INFO:tasks.workunit.client.0.smithi092.stdout:4/374: write d9/f47 [2550143,127557] 0 2024-05-06T12:14:33.637 INFO:tasks.workunit.client.0.smithi092.stdout:7/568: mknod d9/de/d10/d17/d21/d2e/d9c/cbc 0 2024-05-06T12:14:33.637 INFO:tasks.workunit.client.0.smithi092.stdout:4/375: dread - d9/df/d62/f68 zero size 2024-05-06T12:14:33.639 INFO:tasks.workunit.client.0.smithi092.stdout:0/484: dwrite d7/f4f [0,4194304] 0 2024-05-06T12:14:33.640 INFO:tasks.workunit.client.0.smithi092.stdout:4/376: write d9/d27/f4f [903452,12316] 0 2024-05-06T12:14:33.641 INFO:tasks.workunit.client.0.smithi092.stdout:1/325: truncate d3/d10/d13/d19/d34/f36 1459996 0 2024-05-06T12:14:33.643 INFO:tasks.workunit.client.0.smithi092.stdout:7/569: chown d9/de/d10/d17/d21/c56 431546918 1 2024-05-06T12:14:33.644 INFO:tasks.workunit.client.0.smithi092.stdout:6/499: link d4/d20/d23/d30/f6f d4/d24/d26/fa4 0 2024-05-06T12:14:33.649 INFO:tasks.workunit.client.0.smithi092.stdout:0/485: fdatasync d7/dc/d28/d54/d1e/d39/f57 0 2024-05-06T12:14:33.650 INFO:tasks.workunit.client.0.smithi092.stdout:6/500: dread d4/d20/d23/f46 [0,4194304] 0 2024-05-06T12:14:33.651 INFO:tasks.workunit.client.0.smithi092.stdout:5/411: dread d4/d5/d30/f3c [0,4194304] 0 2024-05-06T12:14:33.652 INFO:tasks.workunit.client.0.smithi092.stdout:7/570: mknod d9/de/d10/d17/d21/d38/cbd 0 2024-05-06T12:14:33.654 INFO:tasks.workunit.client.0.smithi092.stdout:9/296: dread f7 [0,4194304] 0 2024-05-06T12:14:33.656 INFO:tasks.workunit.client.0.smithi092.stdout:1/326: dwrite d3/d10/d13/d46/f55 [0,4194304] 0 2024-05-06T12:14:33.661 INFO:tasks.workunit.client.0.smithi092.stdout:4/377: write d9/dd/f19 [1799838,31511] 0 2024-05-06T12:14:33.662 INFO:tasks.workunit.client.0.smithi092.stdout:7/571: write d9/dd/f43 [251757,32835] 0 2024-05-06T12:14:33.662 INFO:tasks.workunit.client.0.smithi092.stdout:1/327: unlink d3/d10/d13/d19/d21/d47/d4f/d58/l6d 0 2024-05-06T12:14:33.664 INFO:tasks.workunit.client.0.smithi092.stdout:5/412: creat d4/f9b x:0 0 0 2024-05-06T12:14:33.664 INFO:tasks.workunit.client.0.smithi092.stdout:9/297: dwrite f6 [0,4194304] 0 2024-05-06T12:14:33.667 INFO:tasks.workunit.client.0.smithi092.stdout:7/572: write f3 [4052718,66531] 0 2024-05-06T12:14:33.667 INFO:tasks.workunit.client.0.smithi092.stdout:6/501: creat d4/d24/d38/d5d/d6d/fa5 x:0 0 0 2024-05-06T12:14:33.670 INFO:tasks.workunit.client.0.smithi092.stdout:5/413: dread d4/d5/d30/f3c [0,4194304] 0 2024-05-06T12:14:33.671 INFO:tasks.workunit.client.0.smithi092.stdout:7/573: chown d9/de/d1f/d37/d48/d65 125283 1 2024-05-06T12:14:33.673 INFO:tasks.workunit.client.0.smithi092.stdout:1/328: mknod d3/d10/d13/d19/d21/d47/c6f 0 2024-05-06T12:14:33.673 INFO:tasks.workunit.client.0.smithi092.stdout:4/378: read d9/f38 [918583,98070] 0 2024-05-06T12:14:33.679 INFO:tasks.workunit.client.0.smithi092.stdout:7/574: creat d9/de/d10/d17/d21/d2e/d9c/fbe x:0 0 0 2024-05-06T12:14:33.679 INFO:tasks.workunit.client.0.smithi092.stdout:1/329: write d3/d10/d13/d19/f2e [233222,589] 0 2024-05-06T12:14:33.680 INFO:tasks.workunit.client.0.smithi092.stdout:4/379: creat d9/df/d48/f74 x:0 0 0 2024-05-06T12:14:33.681 INFO:tasks.workunit.client.0.smithi092.stdout:9/298: dwrite d16/d2c/d2f/d3d/d33/d37/d40/d43/f51 [0,4194304] 0 2024-05-06T12:14:33.683 INFO:tasks.workunit.client.0.smithi092.stdout:5/414: readlink d4/d34/d28/d4e/d5d/d82/d1f/d20/l2d 0 2024-05-06T12:14:33.684 INFO:tasks.workunit.client.0.smithi092.stdout:5/415: write d4/d34/d28/f74 [4627583,51188] 0 2024-05-06T12:14:33.686 INFO:tasks.workunit.client.0.smithi092.stdout:4/380: creat d9/d27/d4a/f75 x:0 0 0 2024-05-06T12:14:33.691 INFO:tasks.workunit.client.0.smithi092.stdout:1/330: chown d3/d10/d13/d19/d34/f36 91930872 1 2024-05-06T12:14:33.692 INFO:tasks.workunit.client.0.smithi092.stdout:0/486: dread d7/dc/d28/d54/d1e/d26/d3a/f52 [0,4194304] 0 2024-05-06T12:14:33.692 INFO:tasks.workunit.client.0.smithi092.stdout:5/416: creat d4/d34/d29/d54/f9c x:0 0 0 2024-05-06T12:14:33.694 INFO:tasks.workunit.client.0.smithi092.stdout:4/381: rename d9/fc to d9/d27/d4a/f76 0 2024-05-06T12:14:33.699 INFO:tasks.workunit.client.0.smithi092.stdout:9/299: dwrite fc [4194304,4194304] 0 2024-05-06T12:14:33.699 INFO:tasks.workunit.client.0.smithi092.stdout:5/417: mknod d4/d34/d28/d4e/d5d/c9d 0 2024-05-06T12:14:33.699 INFO:tasks.workunit.client.0.smithi092.stdout:7/575: dwrite d9/de/d10/d17/fb5 [0,4194304] 0 2024-05-06T12:14:33.702 INFO:tasks.workunit.client.0.smithi092.stdout:4/382: creat d9/d2b/f77 x:0 0 0 2024-05-06T12:14:33.703 INFO:tasks.workunit.client.0.smithi092.stdout:0/487: mknod d7/dc/d28/d54/d1e/d39/c9a 0 2024-05-06T12:14:33.706 INFO:tasks.workunit.client.0.smithi092.stdout:1/331: dread d3/d10/d13/d19/f5d [0,4194304] 0 2024-05-06T12:14:33.708 INFO:tasks.workunit.client.0.smithi092.stdout:1/332: readlink d3/l2b 0 2024-05-06T12:14:33.712 INFO:tasks.workunit.client.0.smithi092.stdout:9/300: write d16/d2c/d2f/f31 [770804,87820] 0 2024-05-06T12:14:33.712 INFO:tasks.workunit.client.0.smithi092.stdout:4/383: chown d9/df/l2a 11 1 2024-05-06T12:14:33.714 INFO:tasks.workunit.client.0.smithi092.stdout:4/384: chown d9/d27/d4a/f75 18 1 2024-05-06T12:14:33.714 INFO:tasks.workunit.client.0.smithi092.stdout:0/488: creat d7/dc/f9b x:0 0 0 2024-05-06T12:14:33.714 INFO:tasks.workunit.client.0.smithi092.stdout:4/385: stat d9/f70 0 2024-05-06T12:14:33.716 INFO:tasks.workunit.client.0.smithi092.stdout:0/489: rmdir d7/dc/d28 39 2024-05-06T12:14:33.717 INFO:tasks.workunit.client.0.smithi092.stdout:5/418: rmdir d4 39 2024-05-06T12:14:33.719 INFO:tasks.workunit.client.0.smithi092.stdout:1/333: creat d3/f70 x:0 0 0 2024-05-06T12:14:33.720 INFO:tasks.workunit.client.0.smithi092.stdout:0/490: dread - d7/dc/d28/d54/d12/f82 zero size 2024-05-06T12:14:33.723 INFO:tasks.workunit.client.0.smithi092.stdout:5/419: stat d4/d34/d28/d4e/f4f 0 2024-05-06T12:14:33.723 INFO:tasks.workunit.client.0.smithi092.stdout:5/420: chown d4/d5/d10/l52 734160 1 2024-05-06T12:14:33.725 INFO:tasks.workunit.client.0.smithi092.stdout:4/386: link f5 d9/df/f78 0 2024-05-06T12:14:33.729 INFO:tasks.workunit.client.0.smithi092.stdout:4/387: readlink d9/l29 0 2024-05-06T12:14:33.733 INFO:tasks.workunit.client.0.smithi092.stdout:5/421: rename d4/d5/d10/d31/d53/d64/f48 to d4/d5/d10/d31/d53/f9e 0 2024-05-06T12:14:33.734 INFO:tasks.workunit.client.0.smithi092.stdout:5/422: fsync d4/f7d 0 2024-05-06T12:14:33.735 INFO:tasks.workunit.client.0.smithi092.stdout:4/388: write f5 [2603353,32788] 0 2024-05-06T12:14:33.737 INFO:tasks.workunit.client.0.smithi092.stdout:4/389: truncate d9/df/d48/f56 4952708 0 2024-05-06T12:14:33.738 INFO:tasks.workunit.client.0.smithi092.stdout:4/390: dread - d9/d2b/f77 zero size 2024-05-06T12:14:33.738 INFO:tasks.workunit.client.0.smithi092.stdout:0/491: link d7/dc/d28/d54/d1e/d26/d3a/f52 d7/dc/f9c 0 2024-05-06T12:14:33.739 INFO:tasks.workunit.client.0.smithi092.stdout:1/334: write d3/d10/d13/d19/d34/f36 [1228415,104299] 0 2024-05-06T12:14:33.740 INFO:tasks.workunit.client.0.smithi092.stdout:0/492: dread - d7/dc/d28/d54/d1e/d46/d63/f72 zero size 2024-05-06T12:14:33.742 INFO:tasks.workunit.client.0.smithi092.stdout:0/493: dread d7/dc/f5c [0,4194304] 0 2024-05-06T12:14:33.744 INFO:tasks.workunit.client.0.smithi092.stdout:1/335: symlink d3/d10/d13/d19/l71 0 2024-05-06T12:14:33.749 INFO:tasks.workunit.client.0.smithi092.stdout:5/423: truncate d4/d34/d28/f74 3323340 0 2024-05-06T12:14:33.750 INFO:tasks.workunit.client.0.smithi092.stdout:5/424: chown d4/d34/d28/d4e/d5d/d82/l16 1605805747 1 2024-05-06T12:14:33.750 INFO:tasks.workunit.client.0.smithi092.stdout:1/336: unlink d3/d10/d13/d19/c25 0 2024-05-06T12:14:33.753 INFO:tasks.workunit.client.0.smithi092.stdout:5/425: unlink d4/d34/d28/d4e/d4b/f84 0 2024-05-06T12:14:33.754 INFO:tasks.workunit.client.0.smithi092.stdout:1/337: mkdir d3/d10/d13/d19/d21/d72 0 2024-05-06T12:14:33.754 INFO:tasks.workunit.client.0.smithi092.stdout:4/391: truncate d9/f24 223246 0 2024-05-06T12:14:33.755 INFO:tasks.workunit.client.0.smithi092.stdout:1/338: dread - d3/d10/f45 zero size 2024-05-06T12:14:33.755 INFO:tasks.workunit.client.0.smithi092.stdout:1/339: dread - d3/f2c zero size 2024-05-06T12:14:33.756 INFO:tasks.workunit.client.0.smithi092.stdout:1/340: fdatasync d3/f70 0 2024-05-06T12:14:33.758 INFO:tasks.workunit.client.0.smithi092.stdout:4/392: mknod d9/df/d48/c79 0 2024-05-06T12:14:33.758 INFO:tasks.workunit.client.0.smithi092.stdout:0/494: write d7/dc/d28/d54/d1e/d26/d3a/f52 [4888722,14122] 0 2024-05-06T12:14:33.761 INFO:tasks.workunit.client.0.smithi092.stdout:4/393: symlink d9/d2b/l7a 0 2024-05-06T12:14:33.762 INFO:tasks.workunit.client.0.smithi092.stdout:0/495: dread d7/f9 [4194304,4194304] 0 2024-05-06T12:14:33.766 INFO:tasks.workunit.client.0.smithi092.stdout:4/394: mkdir d9/df/d62/d7b 0 2024-05-06T12:14:33.768 INFO:tasks.workunit.client.0.smithi092.stdout:1/341: link d3/c33 d3/d10/d13/d19/d21/c73 0 2024-05-06T12:14:33.771 INFO:tasks.workunit.client.0.smithi092.stdout:4/395: mkdir d9/d2b/d7c 0 2024-05-06T12:14:33.775 INFO:tasks.workunit.client.0.smithi092.stdout:0/496: write d7/dc/d28/d54/d1e/f3e [3493345,41364] 0 2024-05-06T12:14:33.776 INFO:tasks.workunit.client.0.smithi092.stdout:1/342: dwrite d3/d10/d13/d19/d21/d47/f50 [0,4194304] 0 2024-05-06T12:14:33.777 INFO:tasks.workunit.client.0.smithi092.stdout:4/396: creat d9/d2b/f7d x:0 0 0 2024-05-06T12:14:33.779 INFO:tasks.workunit.client.0.smithi092.stdout:1/343: fsync d3/d10/d13/d19/d34/f51 0 2024-05-06T12:14:33.780 INFO:tasks.workunit.client.0.smithi092.stdout:0/497: creat d7/dc/d28/d54/d1e/d46/d63/f9d x:0 0 0 2024-05-06T12:14:33.781 INFO:tasks.workunit.client.0.smithi092.stdout:1/344: stat d3/d10/d13/d19/d21/d72 0 2024-05-06T12:14:33.781 INFO:tasks.workunit.client.0.smithi092.stdout:5/426: dread d4/d34/d28/f74 [0,4194304] 0 2024-05-06T12:14:33.786 INFO:tasks.workunit.client.0.smithi092.stdout:4/397: dwrite d9/df/d62/f63 [0,4194304] 0 2024-05-06T12:14:33.786 INFO:tasks.workunit.client.0.smithi092.stdout:1/345: link d3/c18 d3/d42/c74 0 2024-05-06T12:14:33.788 INFO:tasks.workunit.client.0.smithi092.stdout:4/398: write d9/f47 [995450,118934] 0 2024-05-06T12:14:33.788 INFO:tasks.workunit.client.0.smithi092.stdout:0/498: truncate d7/dc/d28/d54/d12/f14 4227020 0 2024-05-06T12:14:33.790 INFO:tasks.workunit.client.0.smithi092.stdout:4/399: creat d9/dd/f7e x:0 0 0 2024-05-06T12:14:33.792 INFO:tasks.workunit.client.0.smithi092.stdout:1/346: rename d3/d10/d13/d19/l2a to d3/d10/d13/d19/d57/l75 0 2024-05-06T12:14:33.793 INFO:tasks.workunit.client.0.smithi092.stdout:0/499: rmdir d7/dc/d28/d54/d1e/d26/d4e 39 2024-05-06T12:14:33.795 INFO:tasks.workunit.client.0.smithi092.stdout:1/347: creat d3/d42/f76 x:0 0 0 2024-05-06T12:14:33.798 INFO:tasks.workunit.client.0.smithi092.stdout:4/400: link d9/d27/f31 d9/d27/f7f 0 2024-05-06T12:14:33.798 INFO:tasks.workunit.client.0.smithi092.stdout:4/401: truncate f5 4629799 0 2024-05-06T12:14:33.799 INFO:tasks.workunit.client.0.smithi092.stdout:1/348: creat d3/d10/d13/d19/d21/f77 x:0 0 0 2024-05-06T12:14:33.800 INFO:tasks.workunit.client.0.smithi092.stdout:4/402: write d9/dd/f23 [1750029,63215] 0 2024-05-06T12:14:33.802 INFO:tasks.workunit.client.0.smithi092.stdout:0/500: rename d7/d43 to d7/dc/d9e 0 2024-05-06T12:14:33.803 INFO:tasks.workunit.client.0.smithi092.stdout:1/349: stat d3/fa 0 2024-05-06T12:14:33.805 INFO:tasks.workunit.client.0.smithi092.stdout:4/403: chown d9/df/c26 6527115 1 2024-05-06T12:14:33.806 INFO:tasks.workunit.client.0.smithi092.stdout:0/501: unlink d7/dc/d9e/d1f/f4a 0 2024-05-06T12:14:33.807 INFO:tasks.workunit.client.0.smithi092.stdout:1/350: creat d3/d10/d13/d19/d21/d47/d4f/d58/f78 x:0 0 0 2024-05-06T12:14:33.808 INFO:tasks.workunit.client.0.smithi092.stdout:5/427: dread d4/d5/f2a [0,4194304] 0 2024-05-06T12:14:33.809 INFO:tasks.workunit.client.0.smithi092.stdout:4/404: creat d9/dd/f80 x:0 0 0 2024-05-06T12:14:33.810 INFO:tasks.workunit.client.0.smithi092.stdout:5/428: write d4/d34/d28/d4e/d4b/f4d [916027,94892] 0 2024-05-06T12:14:33.814 INFO:tasks.workunit.client.0.smithi092.stdout:0/502: symlink d7/dc/l9f 0 2024-05-06T12:14:33.815 INFO:tasks.workunit.client.0.smithi092.stdout:1/351: symlink d3/d10/d13/l79 0 2024-05-06T12:14:33.816 INFO:tasks.workunit.client.0.smithi092.stdout:0/503: fdatasync d7/dc/d28/d54/d12/f93 0 2024-05-06T12:14:33.817 INFO:tasks.workunit.client.0.smithi092.stdout:5/429: mkdir d4/d34/d29/d54/d9f 0 2024-05-06T12:14:33.818 INFO:tasks.workunit.client.0.smithi092.stdout:5/430: chown d4/d5/d30 1 1 2024-05-06T12:14:33.819 INFO:tasks.workunit.client.0.smithi092.stdout:0/504: truncate d7/dc/d28/d54/d1e/d39/f47 9076630 0 2024-05-06T12:14:33.819 INFO:tasks.workunit.client.0.smithi092.stdout:1/352: unlink d3/d10/d13/d19/f1e 0 2024-05-06T12:14:33.821 INFO:tasks.workunit.client.0.smithi092.stdout:1/353: read - d3/d10/d13/d19/d21/d47/d4f/d58/f78 zero size 2024-05-06T12:14:33.822 INFO:tasks.workunit.client.0.smithi092.stdout:1/354: write d3/d10/d13/d19/f40 [4885203,73386] 0 2024-05-06T12:14:33.823 INFO:tasks.workunit.client.0.smithi092.stdout:0/505: creat d7/dc/d28/d54/d1e/d26/d4e/fa0 x:0 0 0 2024-05-06T12:14:33.825 INFO:tasks.workunit.client.0.smithi092.stdout:1/355: creat d3/d10/d13/d19/d21/d47/f7a x:0 0 0 2024-05-06T12:14:33.825 INFO:tasks.workunit.client.0.smithi092.stdout:4/405: dread d9/f34 [0,4194304] 0 2024-05-06T12:14:33.826 INFO:tasks.workunit.client.0.smithi092.stdout:0/506: unlink d7/dc/d28/d64/f7d 0 2024-05-06T12:14:33.832 INFO:tasks.workunit.client.0.smithi092.stdout:0/507: link d7/dc/d28/d54/d1e/d46/f7b d7/dc/d6f/fa1 0 2024-05-06T12:14:33.832 INFO:tasks.workunit.client.0.smithi092.stdout:4/406: getdents d9/d2b/d65 0 2024-05-06T12:14:33.833 INFO:tasks.workunit.client.0.smithi092.stdout:0/508: fdatasync d7/dc/d28/d54/d1e/d26/d4e/fa0 0 2024-05-06T12:14:33.835 INFO:tasks.workunit.client.0.smithi092.stdout:0/509: mkdir d7/dc/d6f/da2 0 2024-05-06T12:14:33.836 INFO:tasks.workunit.client.0.smithi092.stdout:0/510: stat d7/f9 0 2024-05-06T12:14:33.837 INFO:tasks.workunit.client.0.smithi092.stdout:0/511: truncate d7/f70 4804814 0 2024-05-06T12:14:33.838 INFO:tasks.workunit.client.0.smithi092.stdout:4/407: dwrite d9/f22 [0,4194304] 0 2024-05-06T12:14:33.839 INFO:tasks.workunit.client.0.smithi092.stdout:4/408: chown d9/dd/f5f 28571 1 2024-05-06T12:14:33.840 INFO:tasks.workunit.client.0.smithi092.stdout:4/409: stat d9/d27/f31 0 2024-05-06T12:14:33.842 INFO:tasks.workunit.client.0.smithi092.stdout:4/410: chown d9/df/d48/f56 4050639 1 2024-05-06T12:14:33.843 INFO:tasks.workunit.client.0.smithi092.stdout:4/411: fdatasync d9/f47 0 2024-05-06T12:14:33.844 INFO:tasks.workunit.client.0.smithi092.stdout:1/356: dread d3/d10/d13/d19/f1c [0,4194304] 0 2024-05-06T12:14:33.845 INFO:tasks.workunit.client.0.smithi092.stdout:4/412: fsync d9/df/d62/f68 0 2024-05-06T12:14:33.846 INFO:tasks.workunit.client.0.smithi092.stdout:0/512: dwrite d7/dc/d28/d54/d1e/d26/f36 [0,4194304] 0 2024-05-06T12:14:33.848 INFO:tasks.workunit.client.0.smithi092.stdout:1/357: unlink d3/d10/d13/d19/l23 0 2024-05-06T12:14:33.850 INFO:tasks.workunit.client.0.smithi092.stdout:4/413: symlink d9/d2b/d5b/l81 0 2024-05-06T12:14:33.851 INFO:tasks.workunit.client.0.smithi092.stdout:0/513: symlink d7/dc/la3 0 2024-05-06T12:14:33.852 INFO:tasks.workunit.client.0.smithi092.stdout:1/358: truncate d3/d10/d13/d19/f31 1195091 0 2024-05-06T12:14:33.853 INFO:tasks.workunit.client.0.smithi092.stdout:0/514: write d7/f70 [3511590,78890] 0 2024-05-06T12:14:33.854 INFO:tasks.workunit.client.0.smithi092.stdout:0/515: chown d7/dc/d28/d54/d1e/c48 203731 1 2024-05-06T12:14:33.856 INFO:tasks.workunit.client.0.smithi092.stdout:1/359: unlink d3/f26 0 2024-05-06T12:14:33.856 INFO:tasks.workunit.client.0.smithi092.stdout:1/360: readlink d3/d10/d13/l4a 0 2024-05-06T12:14:33.857 INFO:tasks.workunit.client.0.smithi092.stdout:1/361: chown d3/d10/d13/d19/f5d 133900 1 2024-05-06T12:14:33.858 INFO:tasks.workunit.client.0.smithi092.stdout:1/362: rmdir d3/d10 39 2024-05-06T12:14:33.859 INFO:tasks.workunit.client.0.smithi092.stdout:1/363: write d3/f70 [846383,1490] 0 2024-05-06T12:14:33.862 INFO:tasks.workunit.client.0.smithi092.stdout:0/516: link d7/c3d d7/dc/d28/d54/d1e/d5d/ca4 0 2024-05-06T12:14:33.864 INFO:tasks.workunit.client.0.smithi092.stdout:1/364: creat d3/d10/d13/d46/f7b x:0 0 0 2024-05-06T12:14:33.865 INFO:tasks.workunit.client.0.smithi092.stdout:1/365: chown d3/d10/d13/d19/d21/l28 0 1 2024-05-06T12:14:33.866 INFO:tasks.workunit.client.0.smithi092.stdout:0/517: unlink d7/dc/d28/d54/d1e/d46/f67 0 2024-05-06T12:14:33.867 INFO:tasks.workunit.client.0.smithi092.stdout:0/518: dread - d7/dc/d28/d54/d12/f82 zero size 2024-05-06T12:14:33.868 INFO:tasks.workunit.client.0.smithi092.stdout:1/366: symlink d3/d10/d13/d19/d21/l7c 0 2024-05-06T12:14:33.869 INFO:tasks.workunit.client.0.smithi092.stdout:1/367: write d3/d10/d13/f14 [4028600,115208] 0 2024-05-06T12:14:33.872 INFO:tasks.workunit.client.0.smithi092.stdout:1/368: read d3/d10/d13/d19/d21/d47/f5f [2678552,63293] 0 2024-05-06T12:14:33.873 INFO:tasks.workunit.client.0.smithi092.stdout:1/369: write d3/f3e [1020578,49716] 0 2024-05-06T12:14:33.874 INFO:tasks.workunit.client.0.smithi092.stdout:1/370: chown d3/d10/d13/d19/d21/d47/f50 0 1 2024-05-06T12:14:33.875 INFO:tasks.workunit.client.0.smithi092.stdout:1/371: dread - d3/d10/d13/d46/f68 zero size 2024-05-06T12:14:33.877 INFO:tasks.workunit.client.0.smithi092.stdout:1/372: write d3/d10/d13/d19/d21/f2f [2310287,129354] 0 2024-05-06T12:14:33.883 INFO:tasks.workunit.client.0.smithi092.stdout:0/519: dread d7/dc/d28/d54/d1e/d5d/f76 [0,4194304] 0 2024-05-06T12:14:33.884 INFO:tasks.workunit.client.0.smithi092.stdout:4/414: truncate d9/d2b/d65/f64 2019929 0 2024-05-06T12:14:33.886 INFO:tasks.workunit.client.0.smithi092.stdout:1/373: dwrite d3/d10/d13/d46/f7b [0,4194304] 0 2024-05-06T12:14:33.887 INFO:tasks.workunit.client.0.smithi092.stdout:0/520: mkdir d7/dc/d6f/da5 0 2024-05-06T12:14:33.890 INFO:tasks.workunit.client.0.smithi092.stdout:0/521: write d7/dc/d28/d54/d1e/d46/d63/f72 [840954,115348] 0 2024-05-06T12:14:33.893 INFO:tasks.workunit.client.0.smithi092.stdout:4/415: getdents d9/d2b/d5b 0 2024-05-06T12:14:33.894 INFO:tasks.workunit.client.0.smithi092.stdout:4/416: fdatasync d9/df/d62/f63 0 2024-05-06T12:14:33.895 INFO:tasks.workunit.client.0.smithi092.stdout:3/346: sync 2024-05-06T12:14:33.896 INFO:tasks.workunit.client.0.smithi092.stdout:3/347: chown d8/d24/d33/d4b/c10 484 1 2024-05-06T12:14:33.898 INFO:tasks.workunit.client.0.smithi092.stdout:1/374: getdents d3/d10/d13/d19/d21/d47/d4f/d58 0 2024-05-06T12:14:33.901 INFO:tasks.workunit.client.0.smithi092.stdout:0/522: rename d7/dc/f29 to d7/fa6 0 2024-05-06T12:14:33.902 INFO:tasks.workunit.client.0.smithi092.stdout:3/348: creat d8/d24/d33/d39/f70 x:0 0 0 2024-05-06T12:14:33.903 INFO:tasks.workunit.client.0.smithi092.stdout:3/349: stat d8/d24/d33/d4b/f41 0 2024-05-06T12:14:33.904 INFO:tasks.workunit.client.0.smithi092.stdout:3/350: fdatasync d8/db/d4c/d3d/f4e 0 2024-05-06T12:14:33.907 INFO:tasks.workunit.client.0.smithi092.stdout:3/351: chown d8/d24/d33/f64 302 1 2024-05-06T12:14:33.907 INFO:tasks.workunit.client.0.smithi092.stdout:4/417: getdents d9/d27/d4a 0 2024-05-06T12:14:33.907 INFO:tasks.workunit.client.0.smithi092.stdout:0/523: dread d7/dc/d28/d54/d1e/d26/f2b [0,4194304] 0 2024-05-06T12:14:33.911 INFO:tasks.workunit.client.0.smithi092.stdout:0/524: mknod d7/dc/d28/d54/d1e/d26/d4e/ca7 0 2024-05-06T12:14:33.912 INFO:tasks.workunit.client.0.smithi092.stdout:1/375: dwrite d3/d10/d13/d19/f5d [4194304,4194304] 0 2024-05-06T12:14:33.920 INFO:tasks.workunit.client.0.smithi092.stdout:3/352: dwrite d8/d24/d33/d4b/f46 [4194304,4194304] 0 2024-05-06T12:14:33.920 INFO:tasks.workunit.client.0.smithi092.stdout:4/418: dwrite d9/d2b/f7d [0,4194304] 0 2024-05-06T12:14:33.922 INFO:tasks.workunit.client.0.smithi092.stdout:4/419: chown d9/d27/d4a/f44 1 1 2024-05-06T12:14:33.925 INFO:tasks.workunit.client.0.smithi092.stdout:1/376: write d3/fa [692536,11014] 0 2024-05-06T12:14:33.925 INFO:tasks.workunit.client.0.smithi092.stdout:0/525: write d7/dc/d9e/f2c [767694,46517] 0 2024-05-06T12:14:33.929 INFO:tasks.workunit.client.0.smithi092.stdout:3/353: write d8/db/d4c/d3d/f5d [5189798,58613] 0 2024-05-06T12:14:33.930 INFO:tasks.workunit.client.0.smithi092.stdout:1/377: mknod d3/d10/d13/d19/d21/c7d 0 2024-05-06T12:14:33.931 INFO:tasks.workunit.client.0.smithi092.stdout:0/526: getdents d7/dc/d6f/da2 0 2024-05-06T12:14:33.934 INFO:tasks.workunit.client.0.smithi092.stdout:4/420: dwrite d9/d27/f41 [0,4194304] 0 2024-05-06T12:14:33.935 INFO:tasks.workunit.client.0.smithi092.stdout:0/527: mkdir d7/dc/d28/d54/d1e/d39/da8 0 2024-05-06T12:14:33.936 INFO:tasks.workunit.client.0.smithi092.stdout:4/421: write d9/f22 [146651,99296] 0 2024-05-06T12:14:33.943 INFO:tasks.workunit.client.0.smithi092.stdout:1/378: dwrite d3/f3e [0,4194304] 0 2024-05-06T12:14:33.944 INFO:tasks.workunit.client.0.smithi092.stdout:0/528: dread d7/dc/d28/d54/d1e/d39/f47 [8388608,4194304] 0 2024-05-06T12:14:33.946 INFO:tasks.workunit.client.0.smithi092.stdout:3/354: dwrite d8/d24/d33/d4b/f41 [0,4194304] 0 2024-05-06T12:14:33.948 INFO:tasks.workunit.client.0.smithi092.stdout:1/379: unlink d3/f8 0 2024-05-06T12:14:33.948 INFO:tasks.workunit.client.0.smithi092.stdout:0/529: fsync d7/dc/d31/f7a 0 2024-05-06T12:14:33.950 INFO:tasks.workunit.client.0.smithi092.stdout:1/380: chown d3/d10/d13/d46/f7b 1155098 1 2024-05-06T12:14:33.951 INFO:tasks.workunit.client.0.smithi092.stdout:3/355: write d8/d24/d33/d4b/f56 [216501,89293] 0 2024-05-06T12:14:33.951 INFO:tasks.workunit.client.0.smithi092.stdout:1/381: write d3/d10/d13/f3f [2112142,65902] 0 2024-05-06T12:14:33.952 INFO:tasks.workunit.client.0.smithi092.stdout:0/530: mknod d7/dc/d28/d64/ca9 0 2024-05-06T12:14:33.956 INFO:tasks.workunit.client.0.smithi092.stdout:1/382: read d3/d10/d13/d19/f40 [2838737,14224] 0 2024-05-06T12:14:33.957 INFO:tasks.workunit.client.0.smithi092.stdout:3/356: fdatasync d8/d24/d27/f42 0 2024-05-06T12:14:33.957 INFO:tasks.workunit.client.0.smithi092.stdout:0/531: symlink d7/dc/d28/d54/d1e/d26/d4e/laa 0 2024-05-06T12:14:33.960 INFO:tasks.workunit.client.0.smithi092.stdout:1/383: mknod d3/d10/d13/d19/d21/d47/d4f/c7e 0 2024-05-06T12:14:33.961 INFO:tasks.workunit.client.0.smithi092.stdout:0/532: creat d7/dc/d28/d54/d1e/d26/d96/fab x:0 0 0 2024-05-06T12:14:33.963 INFO:tasks.workunit.client.0.smithi092.stdout:1/384: fsync d3/d10/d13/d19/d21/d47/f5f 0 2024-05-06T12:14:33.963 INFO:tasks.workunit.client.0.smithi092.stdout:0/533: fdatasync d7/dc/d28/d54/d1e/d26/d96/f87 0 2024-05-06T12:14:33.967 INFO:tasks.workunit.client.0.smithi092.stdout:1/385: read d3/d10/f53 [2431099,43482] 0 2024-05-06T12:14:33.968 INFO:tasks.workunit.client.0.smithi092.stdout:1/386: dread - d3/d10/d13/d19/d21/f4d zero size 2024-05-06T12:14:33.971 INFO:tasks.workunit.client.0.smithi092.stdout:1/387: mkdir d3/d10/d13/d19/d21/d47/d4f/d7f 0 2024-05-06T12:14:33.975 INFO:tasks.workunit.client.0.smithi092.stdout:4/422: write d9/d27/d4a/f76 [660298,97929] 0 2024-05-06T12:14:33.979 INFO:tasks.workunit.client.0.smithi092.stdout:4/423: mknod d9/d27/d4a/c82 0 2024-05-06T12:14:33.980 INFO:tasks.workunit.client.0.smithi092.stdout:6/502: sync 2024-05-06T12:14:33.981 INFO:tasks.workunit.client.0.smithi092.stdout:8/443: sync 2024-05-06T12:14:33.981 INFO:tasks.workunit.client.0.smithi092.stdout:2/302: sync 2024-05-06T12:14:33.982 INFO:tasks.workunit.client.0.smithi092.stdout:0/534: dread d7/dc/d9e/f2c [0,4194304] 0 2024-05-06T12:14:33.982 INFO:tasks.workunit.client.0.smithi092.stdout:1/388: dwrite d3/d10/d13/d19/d21/f24 [0,4194304] 0 2024-05-06T12:14:33.983 INFO:tasks.workunit.client.0.smithi092.stdout:4/424: mkdir d9/d2b/d65/d83 0 2024-05-06T12:14:33.985 INFO:tasks.workunit.client.0.smithi092.stdout:2/303: creat df/f5b x:0 0 0 2024-05-06T12:14:33.985 INFO:tasks.workunit.client.0.smithi092.stdout:3/357: dread d8/f45 [0,4194304] 0 2024-05-06T12:14:33.986 INFO:tasks.workunit.client.0.smithi092.stdout:4/425: chown d9/d2b/d5b/l81 3888151 1 2024-05-06T12:14:33.987 INFO:tasks.workunit.client.0.smithi092.stdout:3/358: write d8/db/d14/f50 [4730663,91366] 0 2024-05-06T12:14:33.988 INFO:tasks.workunit.client.0.smithi092.stdout:6/503: creat d4/d4c/d52/d62/fa6 x:0 0 0 2024-05-06T12:14:33.991 INFO:tasks.workunit.client.0.smithi092.stdout:6/504: write d4/d20/f1b [8285371,45318] 0 2024-05-06T12:14:33.992 INFO:tasks.workunit.client.0.smithi092.stdout:1/389: dwrite d3/f5 [0,4194304] 0 2024-05-06T12:14:33.993 INFO:tasks.workunit.client.0.smithi092.stdout:8/444: mkdir d9/d21/d37/d46/d5c/d62/da5 0 2024-05-06T12:14:33.993 INFO:tasks.workunit.client.0.smithi092.stdout:0/535: rmdir d7/dc/d9e 39 2024-05-06T12:14:33.996 INFO:tasks.workunit.client.0.smithi092.stdout:2/304: mkdir df/d5c 0 2024-05-06T12:14:33.996 INFO:tasks.workunit.client.0.smithi092.stdout:1/390: fdatasync d3/d10/d13/d46/f61 0 2024-05-06T12:14:33.997 INFO:tasks.workunit.client.0.smithi092.stdout:6/505: write d4/d24/d38/d8c/f94 [605180,51983] 0 2024-05-06T12:14:33.997 INFO:tasks.workunit.client.0.smithi092.stdout:2/305: stat df/d1b/f48 0 2024-05-06T12:14:33.998 INFO:tasks.workunit.client.0.smithi092.stdout:2/306: dread - df/f55 zero size 2024-05-06T12:14:33.999 INFO:tasks.workunit.client.0.smithi092.stdout:4/426: rename d9/d27/d4a/f71 to d9/d27/d4a/f84 0 2024-05-06T12:14:33.999 INFO:tasks.workunit.client.0.smithi092.stdout:2/307: readlink df/d17/d1d/l3f 0 2024-05-06T12:14:33.999 INFO:tasks.workunit.client.0.smithi092.stdout:1/391: fdatasync d3/d10/d13/d19/f4e 0 2024-05-06T12:14:34.001 INFO:tasks.workunit.client.0.smithi092.stdout:2/308: chown df/d15/c1a 253 1 2024-05-06T12:14:34.002 INFO:tasks.workunit.client.0.smithi092.stdout:6/506: read d4/d20/d23/d30/f5e [147522,712] 0 2024-05-06T12:14:34.005 INFO:tasks.workunit.client.0.smithi092.stdout:0/536: rename d7/dc/d28/d54/d1e/d26/f2b to d7/dc/d6f/da2/fac 0 2024-05-06T12:14:34.007 INFO:tasks.workunit.client.0.smithi092.stdout:0/537: write d7/dc/d28/d54/d12/f82 [1041453,38210] 0 2024-05-06T12:14:34.009 INFO:tasks.workunit.client.0.smithi092.stdout:0/538: write d7/dc/d28/d54/d1e/f41 [3182919,42696] 0 2024-05-06T12:14:34.011 INFO:tasks.workunit.client.0.smithi092.stdout:6/507: unlink d4/l10 0 2024-05-06T12:14:34.013 INFO:tasks.workunit.client.0.smithi092.stdout:6/508: chown d4/d24/d38/d5d/d6d/d87 2021576157 1 2024-05-06T12:14:34.013 INFO:tasks.workunit.client.0.smithi092.stdout:3/359: truncate d8/db/d4c/d3d/d49/f4f 2661377 0 2024-05-06T12:14:34.019 INFO:tasks.workunit.client.0.smithi092.stdout:0/539: dwrite d7/dc/d28/d54/d1e/d26/d96/f87 [0,4194304] 0 2024-05-06T12:14:34.019 INFO:tasks.workunit.client.0.smithi092.stdout:8/445: dread f0 [0,4194304] 0 2024-05-06T12:14:34.020 INFO:tasks.workunit.client.0.smithi092.stdout:4/427: link d9/f1c d9/d2b/d5b/f85 0 2024-05-06T12:14:34.022 INFO:tasks.workunit.client.0.smithi092.stdout:8/446: write d9/d21/d37/fa3 [742253,71635] 0 2024-05-06T12:14:34.022 INFO:tasks.workunit.client.0.smithi092.stdout:4/428: chown d9/d27/l55 297647339 1 2024-05-06T12:14:34.027 INFO:tasks.workunit.client.0.smithi092.stdout:6/509: rename d4/c12 to d4/d20/d23/ca7 0 2024-05-06T12:14:34.028 INFO:tasks.workunit.client.0.smithi092.stdout:0/540: dread d7/dc/d28/d54/d1e/d26/d96/f87 [0,4194304] 0 2024-05-06T12:14:34.031 INFO:tasks.workunit.client.0.smithi092.stdout:8/447: mkdir d9/d9d/da6 0 2024-05-06T12:14:34.034 INFO:tasks.workunit.client.0.smithi092.stdout:4/429: creat d9/f86 x:0 0 0 2024-05-06T12:14:34.035 INFO:tasks.workunit.client.0.smithi092.stdout:8/448: chown d9/d21/d37/d46/d5c/d71/c95 445 1 2024-05-06T12:14:34.037 INFO:tasks.workunit.client.0.smithi092.stdout:0/541: dwrite d7/dc/f9b [0,4194304] 0 2024-05-06T12:14:34.038 INFO:tasks.workunit.client.0.smithi092.stdout:4/430: creat d9/dd/f87 x:0 0 0 2024-05-06T12:14:34.042 INFO:tasks.workunit.client.0.smithi092.stdout:8/449: symlink d9/d21/d37/d46/d5c/la7 0 2024-05-06T12:14:34.043 INFO:tasks.workunit.client.0.smithi092.stdout:3/360: dwrite d8/d24/d27/f42 [0,4194304] 0 2024-05-06T12:14:34.046 INFO:tasks.workunit.client.0.smithi092.stdout:8/450: fsync d9/f42 0 2024-05-06T12:14:34.051 INFO:tasks.workunit.client.0.smithi092.stdout:8/451: stat d9/d21/d54/c61 0 2024-05-06T12:14:34.051 INFO:tasks.workunit.client.0.smithi092.stdout:0/542: getdents d7/dc/d28/d54/d1e/d26/d4e 0 2024-05-06T12:14:34.058 INFO:tasks.workunit.client.0.smithi092.stdout:4/431: dwrite d9/d2b/f6e [0,4194304] 0 2024-05-06T12:14:34.059 INFO:tasks.workunit.client.0.smithi092.stdout:3/361: dwrite d8/d24/d33/f62 [0,4194304] 0 2024-05-06T12:14:34.061 INFO:tasks.workunit.client.0.smithi092.stdout:3/362: fdatasync d8/db/d14/f50 0 2024-05-06T12:14:34.065 INFO:tasks.workunit.client.0.smithi092.stdout:0/543: write d7/dc/d9e/f1c [2662220,13160] 0 2024-05-06T12:14:34.065 INFO:tasks.workunit.client.0.smithi092.stdout:8/452: mkdir d9/dc/d4a/d5f/da8 0 2024-05-06T12:14:34.069 INFO:tasks.workunit.client.0.smithi092.stdout:4/432: dwrite d9/dd/f23 [0,4194304] 0 2024-05-06T12:14:34.071 INFO:tasks.workunit.client.0.smithi092.stdout:8/453: creat d9/d5a/fa9 x:0 0 0 2024-05-06T12:14:34.075 INFO:tasks.workunit.client.0.smithi092.stdout:8/454: mknod d9/d21/d37/d46/d5c/d62/d7a/caa 0 2024-05-06T12:14:34.090 INFO:tasks.workunit.client.0.smithi092.stdout:0/544: write d7/dc/d28/d54/d1e/d5d/f68 [248093,73693] 0 2024-05-06T12:14:34.091 INFO:tasks.workunit.client.0.smithi092.stdout:8/455: fsync d9/d5a/fa9 0 2024-05-06T12:14:34.091 INFO:tasks.workunit.client.0.smithi092.stdout:4/433: write d9/f21 [3793780,63409] 0 2024-05-06T12:14:34.092 INFO:tasks.workunit.client.0.smithi092.stdout:0/545: dread - d7/dc/d28/d54/d12/f93 zero size 2024-05-06T12:14:34.093 INFO:tasks.workunit.client.0.smithi092.stdout:0/546: fsync d7/dc/d9e/f8c 0 2024-05-06T12:14:34.094 INFO:tasks.workunit.client.0.smithi092.stdout:0/547: chown d7/dc/d9e/f8c 251 1 2024-05-06T12:14:34.094 INFO:tasks.workunit.client.0.smithi092.stdout:4/434: rename d9/c6b to d9/d2b/d7c/c88 0 2024-05-06T12:14:34.095 INFO:tasks.workunit.client.0.smithi092.stdout:4/435: chown c7 0 1 2024-05-06T12:14:34.096 INFO:tasks.workunit.client.0.smithi092.stdout:8/456: getdents d9/d21/d37/d46/d5c/d71/d75 0 2024-05-06T12:14:34.097 INFO:tasks.workunit.client.0.smithi092.stdout:4/436: mkdir d9/df/d48/d89 0 2024-05-06T12:14:34.098 INFO:tasks.workunit.client.0.smithi092.stdout:8/457: fdatasync d9/d21/d37/fa3 0 2024-05-06T12:14:34.099 INFO:tasks.workunit.client.0.smithi092.stdout:8/458: truncate d9/dc/f87 367640 0 2024-05-06T12:14:34.100 INFO:tasks.workunit.client.0.smithi092.stdout:4/437: rmdir d9/df/d48/d89 0 2024-05-06T12:14:34.102 INFO:tasks.workunit.client.0.smithi092.stdout:4/438: symlink d9/d2b/d5b/l8a 0 2024-05-06T12:14:34.104 INFO:tasks.workunit.client.0.smithi092.stdout:4/439: unlink d9/df/f67 0 2024-05-06T12:14:34.105 INFO:tasks.workunit.client.0.smithi092.stdout:8/459: getdents d9/d21/d37 0 2024-05-06T12:14:34.109 INFO:tasks.workunit.client.0.smithi092.stdout:8/460: mknod d9/d21/d72/cab 0 2024-05-06T12:14:34.111 INFO:tasks.workunit.client.0.smithi092.stdout:4/440: link d9/l3d d9/df/d48/l8b 0 2024-05-06T12:14:34.114 INFO:tasks.workunit.client.0.smithi092.stdout:8/461: link d9/d21/d54/d79/f96 d9/d21/d37/d46/d5c/d62/fac 0 2024-05-06T12:14:34.115 INFO:tasks.workunit.client.0.smithi092.stdout:8/462: symlink d9/d21/d54/d79/lad 0 2024-05-06T12:14:34.122 INFO:tasks.workunit.client.0.smithi092.stdout:8/463: dwrite d9/d21/d37/fa3 [0,4194304] 0 2024-05-06T12:14:34.124 INFO:tasks.workunit.client.0.smithi092.stdout:4/441: dread d9/d27/d4a/f76 [0,4194304] 0 2024-05-06T12:14:34.130 INFO:tasks.workunit.client.0.smithi092.stdout:8/464: rename d9/fa2 to d9/d21/d37/d46/fae 0 2024-05-06T12:14:34.133 INFO:tasks.workunit.client.0.smithi092.stdout:8/465: mknod d9/d21/d54/d79/caf 0 2024-05-06T12:14:34.138 INFO:tasks.workunit.client.0.smithi092.stdout:8/466: dwrite d9/d21/f6f [0,4194304] 0 2024-05-06T12:14:34.139 INFO:tasks.workunit.client.0.smithi092.stdout:8/467: chown d9/f76 224871 1 2024-05-06T12:14:34.140 INFO:tasks.workunit.client.0.smithi092.stdout:8/468: chown d9/d21/d37/l73 0 1 2024-05-06T12:14:34.145 INFO:tasks.workunit.client.0.smithi092.stdout:8/469: dwrite d9/dc/f9e [4194304,4194304] 0 2024-05-06T12:14:34.146 INFO:tasks.workunit.client.0.smithi092.stdout:4/442: dread f5 [0,4194304] 0 2024-05-06T12:14:34.149 INFO:tasks.workunit.client.0.smithi092.stdout:8/470: rmdir d9/d9d 39 2024-05-06T12:14:34.152 INFO:tasks.workunit.client.0.smithi092.stdout:4/443: creat d9/d2b/d65/d83/f8c x:0 0 0 2024-05-06T12:14:34.158 INFO:tasks.workunit.client.0.smithi092.stdout:8/471: dwrite d9/dc/f53 [0,4194304] 0 2024-05-06T12:14:34.158 INFO:tasks.workunit.client.0.smithi092.stdout:4/444: read d9/f24 [215007,127428] 0 2024-05-06T12:14:34.159 INFO:tasks.workunit.client.0.smithi092.stdout:8/472: chown d9/d21/d54/d79/d8a 3 1 2024-05-06T12:14:34.160 INFO:tasks.workunit.client.0.smithi092.stdout:4/445: stat d9/d2b/c30 0 2024-05-06T12:14:34.163 INFO:tasks.workunit.client.0.smithi092.stdout:4/446: creat d9/d2b/d5b/f8d x:0 0 0 2024-05-06T12:14:34.164 INFO:tasks.workunit.client.0.smithi092.stdout:4/447: mknod d9/df/d62/c8e 0 2024-05-06T12:14:34.165 INFO:tasks.workunit.client.0.smithi092.stdout:4/448: write d9/f70 [1418215,65619] 0 2024-05-06T12:14:34.167 INFO:tasks.workunit.client.0.smithi092.stdout:4/449: creat d9/d2b/f8f x:0 0 0 2024-05-06T12:14:34.170 INFO:tasks.workunit.client.0.smithi092.stdout:8/473: write d9/d21/d37/d46/d5c/d62/f68 [1258872,96125] 0 2024-05-06T12:14:34.171 INFO:tasks.workunit.client.0.smithi092.stdout:8/474: stat d9/d21/d54/l6a 0 2024-05-06T12:14:34.172 INFO:tasks.workunit.client.0.smithi092.stdout:4/450: dread d9/f3f [0,4194304] 0 2024-05-06T12:14:34.174 INFO:tasks.workunit.client.0.smithi092.stdout:4/451: mkdir d9/dd/d90 0 2024-05-06T12:14:34.178 INFO:tasks.workunit.client.0.smithi092.stdout:8/475: dwrite d9/f2a [0,4194304] 0 2024-05-06T12:14:34.178 INFO:tasks.workunit.client.0.smithi092.stdout:4/452: creat d9/d2b/d65/d83/f91 x:0 0 0 2024-05-06T12:14:34.184 INFO:tasks.workunit.client.0.smithi092.stdout:4/453: link d9/d27/d4a/f76 d9/d27/f92 0 2024-05-06T12:14:34.186 INFO:tasks.workunit.client.0.smithi092.stdout:8/476: rename d9/d21/d37/d46/d5c/c99 to d9/cb0 0 2024-05-06T12:14:34.189 INFO:tasks.workunit.client.0.smithi092.stdout:8/477: symlink d9/d9d/lb1 0 2024-05-06T12:14:34.189 INFO:tasks.workunit.client.0.smithi092.stdout:4/454: link d9/d27/c66 d9/dd/d90/c93 0 2024-05-06T12:14:34.192 INFO:tasks.workunit.client.0.smithi092.stdout:8/478: unlink d9/d21/c90 0 2024-05-06T12:14:34.193 INFO:tasks.workunit.client.0.smithi092.stdout:8/479: readlink d9/dc/l57 0 2024-05-06T12:14:34.194 INFO:tasks.workunit.client.0.smithi092.stdout:4/455: getdents d9/df 0 2024-05-06T12:14:34.195 INFO:tasks.workunit.client.0.smithi092.stdout:4/456: chown d9/dd/f7e 6476409 1 2024-05-06T12:14:34.197 INFO:tasks.workunit.client.0.smithi092.stdout:8/480: creat d9/d21/d54/d79/d8a/fb2 x:0 0 0 2024-05-06T12:14:34.197 INFO:tasks.workunit.client.0.smithi092.stdout:4/457: creat d9/dd/f94 x:0 0 0 2024-05-06T12:14:34.199 INFO:tasks.workunit.client.0.smithi092.stdout:4/458: stat d9/df/d48/l5a 0 2024-05-06T12:14:34.200 INFO:tasks.workunit.client.0.smithi092.stdout:8/481: mknod d9/d21/d37/d46/d5c/d62/da5/cb3 0 2024-05-06T12:14:34.201 INFO:tasks.workunit.client.0.smithi092.stdout:8/482: fsync d9/dc/f9e 0 2024-05-06T12:14:34.202 INFO:tasks.workunit.client.0.smithi092.stdout:4/459: creat d9/d2b/d7c/f95 x:0 0 0 2024-05-06T12:14:34.202 INFO:tasks.workunit.client.0.smithi092.stdout:8/483: write d9/dc/f5d [3170499,55385] 0 2024-05-06T12:14:34.206 INFO:tasks.workunit.client.0.smithi092.stdout:8/484: creat d9/d21/fb4 x:0 0 0 2024-05-06T12:14:34.208 INFO:tasks.workunit.client.0.smithi092.stdout:4/460: dwrite d9/df/f46 [0,4194304] 0 2024-05-06T12:14:34.209 INFO:tasks.workunit.client.0.smithi092.stdout:4/461: fsync d9/d27/f41 0 2024-05-06T12:14:34.210 INFO:tasks.workunit.client.0.smithi092.stdout:4/462: chown d9/d27/d4a/c82 259831849 1 2024-05-06T12:14:34.213 INFO:tasks.workunit.client.0.smithi092.stdout:4/463: symlink d9/d2b/d65/l96 0 2024-05-06T12:14:34.214 INFO:tasks.workunit.client.0.smithi092.stdout:8/485: link d9/d21/d54/l84 d9/d9d/lb5 0 2024-05-06T12:14:34.215 INFO:tasks.workunit.client.0.smithi092.stdout:4/464: symlink d9/d2b/d7c/l97 0 2024-05-06T12:14:34.218 INFO:tasks.workunit.client.0.smithi092.stdout:4/465: creat d9/d2b/d5b/f98 x:0 0 0 2024-05-06T12:14:34.221 INFO:tasks.workunit.client.0.smithi092.stdout:4/466: dread d9/d27/f41 [0,4194304] 0 2024-05-06T12:14:34.222 INFO:tasks.workunit.client.0.smithi092.stdout:4/467: stat d9/df/d48/l5a 0 2024-05-06T12:14:34.230 INFO:tasks.workunit.client.0.smithi092.stdout:4/468: dwrite d9/df/f78 [0,4194304] 0 2024-05-06T12:14:34.237 INFO:tasks.workunit.client.0.smithi092.stdout:7/576: sync 2024-05-06T12:14:34.238 INFO:tasks.workunit.client.0.smithi092.stdout:7/577: chown d9/de/d10/d17/d6a/f6d 898 1 2024-05-06T12:14:34.239 INFO:tasks.workunit.client.0.smithi092.stdout:7/578: stat d9/de/d10/d17/d21/d38/cbd 0 2024-05-06T12:14:34.242 INFO:tasks.workunit.client.0.smithi092.stdout:7/579: creat d9/de/d1f/d37/d48/d97/fbf x:0 0 0 2024-05-06T12:14:34.351 INFO:tasks.workunit.client.0.smithi092.stdout:5/431: sync 2024-05-06T12:14:34.351 INFO:tasks.workunit.client.0.smithi092.stdout:9/301: sync 2024-05-06T12:14:34.352 INFO:tasks.workunit.client.0.smithi092.stdout:9/302: chown d16/f17 514 1 2024-05-06T12:14:34.353 INFO:tasks.workunit.client.0.smithi092.stdout:9/303: symlink d16/d2c/l5d 0 2024-05-06T12:14:34.355 INFO:tasks.workunit.client.0.smithi092.stdout:9/304: creat d16/d2c/d3f/f5e x:0 0 0 2024-05-06T12:14:34.355 INFO:tasks.workunit.client.0.smithi092.stdout:5/432: link d4/d5/d30/c32 d4/d34/d29/d54/d73/ca0 0 2024-05-06T12:14:34.357 INFO:tasks.workunit.client.0.smithi092.stdout:5/433: mkdir d4/d5/d10/da1 0 2024-05-06T12:14:34.358 INFO:tasks.workunit.client.0.smithi092.stdout:9/305: fdatasync d16/d2c/d2f/d3d/d33/f3b 0 2024-05-06T12:14:34.360 INFO:tasks.workunit.client.0.smithi092.stdout:5/434: creat d4/d34/d29/d54/fa2 x:0 0 0 2024-05-06T12:14:34.361 INFO:tasks.workunit.client.0.smithi092.stdout:9/306: readlink d16/d1c/d58/l5a 0 2024-05-06T12:14:34.362 INFO:tasks.workunit.client.0.smithi092.stdout:5/435: mkdir d4/d34/d29/da3 0 2024-05-06T12:14:34.363 INFO:tasks.workunit.client.0.smithi092.stdout:5/436: fdatasync d4/d34/d29/d6c/f51 0 2024-05-06T12:14:34.364 INFO:tasks.workunit.client.0.smithi092.stdout:9/307: creat d16/d2c/d2f/d3d/d33/d37/d40/f5f x:0 0 0 2024-05-06T12:14:34.365 INFO:tasks.workunit.client.0.smithi092.stdout:9/308: write d16/d2c/d3f/f4d [448754,97967] 0 2024-05-06T12:14:34.366 INFO:tasks.workunit.client.0.smithi092.stdout:9/309: stat d16/d2c/d2f/c44 0 2024-05-06T12:14:34.367 INFO:tasks.workunit.client.0.smithi092.stdout:9/310: write d16/d2c/d3f/f46 [49560,122799] 0 2024-05-06T12:14:34.369 INFO:tasks.workunit.client.0.smithi092.stdout:5/437: creat d4/d34/d28/d4e/d5d/d82/d1f/d20/fa4 x:0 0 0 2024-05-06T12:14:34.372 INFO:tasks.workunit.client.0.smithi092.stdout:5/438: read - d4/d34/d28/d4e/d5d/d82/d1f/d20/fa4 zero size 2024-05-06T12:14:34.374 INFO:tasks.workunit.client.0.smithi092.stdout:9/311: dwrite d16/d2c/d2f/f3a [4194304,4194304] 0 2024-05-06T12:14:34.375 INFO:tasks.workunit.client.0.smithi092.stdout:5/439: getdents d4/d34/d28/d4e/d5d/d82/d1f/d20/d99 0 2024-05-06T12:14:34.383 INFO:tasks.workunit.client.0.smithi092.stdout:9/312: dwrite d16/d2c/d4b/f54 [0,4194304] 0 2024-05-06T12:14:34.384 INFO:tasks.workunit.client.0.smithi092.stdout:5/440: dwrite d4/d34/d28/d4e/d5d/d82/d1f/d20/f8d [0,4194304] 0 2024-05-06T12:14:34.385 INFO:tasks.workunit.client.0.smithi092.stdout:9/313: write d16/d2c/d2f/f36 [2858016,83527] 0 2024-05-06T12:14:34.388 INFO:tasks.workunit.client.0.smithi092.stdout:9/314: unlink d16/d2c/d2f/c44 0 2024-05-06T12:14:34.390 INFO:tasks.workunit.client.0.smithi092.stdout:5/441: mknod d4/d34/d28/d4e/d5d/d82/d1f/d20/ca5 0 2024-05-06T12:14:34.391 INFO:tasks.workunit.client.0.smithi092.stdout:9/315: creat d16/d2c/d3f/f60 x:0 0 0 2024-05-06T12:14:34.391 INFO:tasks.workunit.client.0.smithi092.stdout:5/442: write d4/d34/d28/f3a [4982037,32904] 0 2024-05-06T12:14:34.454 INFO:tasks.workunit.client.0.smithi092.stdout:9/316: creat d16/d2c/d2f/d3d/f61 x:0 0 0 2024-05-06T12:14:34.455 INFO:tasks.workunit.client.0.smithi092.stdout:9/317: fdatasync d16/d1e/f2e 0 2024-05-06T12:14:34.455 INFO:tasks.workunit.client.0.smithi092.stdout:9/318: stat d16/d1e/f2e 0 2024-05-06T12:14:34.457 INFO:tasks.workunit.client.0.smithi092.stdout:5/443: creat d4/d5/d10/fa6 x:0 0 0 2024-05-06T12:14:34.462 INFO:tasks.workunit.client.0.smithi092.stdout:9/319: dwrite d16/f1f [0,4194304] 0 2024-05-06T12:14:34.467 INFO:tasks.workunit.client.0.smithi092.stdout:9/320: link f7 d16/d2c/d2f/d3d/f62 0 2024-05-06T12:14:34.478 INFO:tasks.workunit.client.0.smithi092.stdout:9/321: truncate d16/d1e/f28 2389806 0 2024-05-06T12:14:34.483 INFO:tasks.workunit.client.0.smithi092.stdout:9/322: dwrite d16/d2c/d2f/f31 [0,4194304] 0 2024-05-06T12:14:34.484 INFO:tasks.workunit.client.0.smithi092.stdout:9/323: mknod d16/d2c/c63 0 2024-05-06T12:14:34.490 INFO:tasks.workunit.client.0.smithi092.stdout:9/324: dwrite d16/d2c/d2f/d3d/d33/f56 [0,4194304] 0 2024-05-06T12:14:34.491 INFO:tasks.workunit.client.0.smithi092.stdout:1/392: sync 2024-05-06T12:14:34.491 INFO:tasks.workunit.client.0.smithi092.stdout:9/325: stat d16/d1e/f55 0 2024-05-06T12:14:34.498 INFO:tasks.workunit.client.0.smithi092.stdout:1/393: truncate d3/d10/d13/d19/f40 446895 0 2024-05-06T12:14:34.503 INFO:tasks.workunit.client.0.smithi092.stdout:1/394: dread d3/f20 [0,4194304] 0 2024-05-06T12:14:34.504 INFO:tasks.workunit.client.0.smithi092.stdout:1/395: chown d3/d10/d13/d19/d21/d47/d4f/d58/l64 64915 1 2024-05-06T12:14:34.505 INFO:tasks.workunit.client.0.smithi092.stdout:1/396: dread - d3/d10/d13/d19/d34/f51 zero size 2024-05-06T12:14:34.518 INFO:tasks.workunit.client.0.smithi092.stdout:1/397: dread d3/d10/d13/d19/d34/f36 [0,4194304] 0 2024-05-06T12:14:34.518 INFO:tasks.workunit.client.0.smithi092.stdout:1/398: chown d3/d10/d13/f1f 2870513 1 2024-05-06T12:14:34.519 INFO:tasks.workunit.client.0.smithi092.stdout:1/399: dread - d3/d10/d13/d19/f4e zero size 2024-05-06T12:14:34.533 INFO:tasks.workunit.client.0.smithi092.stdout:1/400: dread d3/d10/d13/f1f [0,4194304] 0 2024-05-06T12:14:34.535 INFO:tasks.workunit.client.0.smithi092.stdout:1/401: getdents d3/d10/d13/d19/d21/d47/d4f/d7f 0 2024-05-06T12:14:34.535 INFO:tasks.workunit.client.0.smithi092.stdout:1/402: chown d3/d10/d13/c1a 490 1 2024-05-06T12:14:34.537 INFO:tasks.workunit.client.0.smithi092.stdout:1/403: write d3/d10/d13/d19/d34/f36 [496447,33568] 0 2024-05-06T12:14:34.539 INFO:tasks.workunit.client.0.smithi092.stdout:1/404: symlink d3/d10/d13/d19/d21/d47/d4f/d58/d65/l80 0 2024-05-06T12:14:34.540 INFO:tasks.workunit.client.0.smithi092.stdout:1/405: rmdir d3/d10/d13/d46 39 2024-05-06T12:14:34.542 INFO:tasks.workunit.client.0.smithi092.stdout:1/406: dread d3/f20 [0,4194304] 0 2024-05-06T12:14:34.545 INFO:tasks.workunit.client.0.smithi092.stdout:1/407: mkdir d3/d10/d13/d19/d21/d47/d4f/d58/d65/d81 0 2024-05-06T12:14:34.553 INFO:tasks.workunit.client.0.smithi092.stdout:1/408: truncate d3/d10/d13/d19/f40 394386 0 2024-05-06T12:14:34.579 INFO:tasks.workunit.client.0.smithi092.stdout:2/309: sync 2024-05-06T12:14:34.579 INFO:tasks.workunit.client.0.smithi092.stdout:6/510: sync 2024-05-06T12:14:34.580 INFO:tasks.workunit.client.0.smithi092.stdout:2/310: truncate df/f5b 32351 0 2024-05-06T12:14:34.583 INFO:tasks.workunit.client.0.smithi092.stdout:2/311: symlink df/d17/d1d/d33/d4d/l5d 0 2024-05-06T12:14:34.585 INFO:tasks.workunit.client.0.smithi092.stdout:6/511: dwrite d4/f8e [0,4194304] 0 2024-05-06T12:14:34.585 INFO:tasks.workunit.client.0.smithi092.stdout:2/312: symlink df/l5e 0 2024-05-06T12:14:34.587 INFO:tasks.workunit.client.0.smithi092.stdout:6/512: chown d4/d24/d38/d5d/d6d/l74 1479 1 2024-05-06T12:14:34.587 INFO:tasks.workunit.client.0.smithi092.stdout:6/513: stat d4/d20/d23/c7d 0 2024-05-06T12:14:34.588 INFO:tasks.workunit.client.0.smithi092.stdout:6/514: stat d4/d20/d22/f7a 0 2024-05-06T12:14:34.591 INFO:tasks.workunit.client.0.smithi092.stdout:6/515: mknod d4/ca8 0 2024-05-06T12:14:34.593 INFO:tasks.workunit.client.0.smithi092.stdout:6/516: mkdir d4/d4c/da9 0 2024-05-06T12:14:34.597 INFO:tasks.workunit.client.0.smithi092.stdout:6/517: link d4/d24/f39 d4/d24/d38/faa 0 2024-05-06T12:14:34.603 INFO:tasks.workunit.client.0.smithi092.stdout:6/518: dwrite d4/d24/f55 [0,4194304] 0 2024-05-06T12:14:34.609 INFO:tasks.workunit.client.0.smithi092.stdout:6/519: write d4/d20/d23/d30/f6f [538655,72831] 0 2024-05-06T12:14:34.611 INFO:tasks.workunit.client.0.smithi092.stdout:2/313: dread df/f2c [0,4194304] 0 2024-05-06T12:14:34.612 INFO:tasks.workunit.client.0.smithi092.stdout:6/520: rmdir d4/d4c/d52/d62 39 2024-05-06T12:14:34.613 INFO:tasks.workunit.client.0.smithi092.stdout:6/521: write d4/d24/d26/f4b [2533131,29000] 0 2024-05-06T12:14:34.614 INFO:tasks.workunit.client.0.smithi092.stdout:2/314: symlink df/d17/l5f 0 2024-05-06T12:14:34.615 INFO:tasks.workunit.client.0.smithi092.stdout:2/315: dread - df/d17/d1d/d33/f47 zero size 2024-05-06T12:14:34.615 INFO:tasks.workunit.client.0.smithi092.stdout:6/522: symlink d4/d24/d38/lab 0 2024-05-06T12:14:34.616 INFO:tasks.workunit.client.0.smithi092.stdout:6/523: dread - d4/d4c/f75 zero size 2024-05-06T12:14:34.618 INFO:tasks.workunit.client.0.smithi092.stdout:2/316: truncate df/f37 1886302 0 2024-05-06T12:14:34.620 INFO:tasks.workunit.client.0.smithi092.stdout:6/524: symlink d4/d24/d38/d5d/d6d/lac 0 2024-05-06T12:14:34.621 INFO:tasks.workunit.client.0.smithi092.stdout:6/525: chown d4/d20/c1f 24 1 2024-05-06T12:14:34.624 INFO:tasks.workunit.client.0.smithi092.stdout:6/526: dread d4/f8e [0,4194304] 0 2024-05-06T12:14:34.628 INFO:tasks.workunit.client.0.smithi092.stdout:6/527: dread d4/d20/d23/d30/f36 [0,4194304] 0 2024-05-06T12:14:34.631 INFO:tasks.workunit.client.0.smithi092.stdout:6/528: rename d4/f3d to d4/d4c/d52/d62/fad 0 2024-05-06T12:14:34.637 INFO:tasks.workunit.client.0.smithi092.stdout:6/529: dwrite d4/d24/d38/d5d/d6d/fa5 [0,4194304] 0 2024-05-06T12:14:34.640 INFO:tasks.workunit.client.0.smithi092.stdout:6/530: mkdir d4/d24/d26/dae 0 2024-05-06T12:14:34.645 INFO:tasks.workunit.client.0.smithi092.stdout:6/531: mkdir d4/d24/d38/d5d/daf 0 2024-05-06T12:14:34.646 INFO:tasks.workunit.client.0.smithi092.stdout:6/532: write d4/d24/d54/f77 [200324,46403] 0 2024-05-06T12:14:34.647 INFO:tasks.workunit.client.0.smithi092.stdout:6/533: chown d4/d24/d54/f86 122 1 2024-05-06T12:14:34.651 INFO:tasks.workunit.client.0.smithi092.stdout:6/534: rename d4/d24/d38/d5d/d6d/d87 to d4/d24/db0 0 2024-05-06T12:14:34.653 INFO:tasks.workunit.client.0.smithi092.stdout:6/535: mknod d4/d24/d38/d8c/cb1 0 2024-05-06T12:14:34.656 INFO:tasks.workunit.client.0.smithi092.stdout:6/536: creat d4/d20/fb2 x:0 0 0 2024-05-06T12:14:34.656 INFO:tasks.workunit.client.0.smithi092.stdout:6/537: chown d4/d24/d38/d8c/f94 8657 1 2024-05-06T12:14:34.658 INFO:tasks.workunit.client.0.smithi092.stdout:6/538: mknod d4/d4c/da9/cb3 0 2024-05-06T12:14:34.662 INFO:tasks.workunit.client.0.smithi092.stdout:6/539: link d4/c9 d4/d4c/d52/d6c/da0/cb4 0 2024-05-06T12:14:34.664 INFO:tasks.workunit.client.0.smithi092.stdout:6/540: mknod d4/d24/d66/cb5 0 2024-05-06T12:14:34.665 INFO:tasks.workunit.client.0.smithi092.stdout:6/541: chown d4/d4c/la1 20331030 1 2024-05-06T12:14:34.675 INFO:tasks.workunit.client.0.smithi092.stdout:6/542: dwrite d4/f1a [0,4194304] 0 2024-05-06T12:14:34.678 INFO:tasks.workunit.client.0.smithi092.stdout:6/543: rmdir d4/d24/d38/d5d 39 2024-05-06T12:14:34.681 INFO:tasks.workunit.client.0.smithi092.stdout:6/544: creat d4/d4c/d91/fb6 x:0 0 0 2024-05-06T12:14:34.683 INFO:tasks.workunit.client.0.smithi092.stdout:6/545: unlink d4/d20/d23/f67 0 2024-05-06T12:14:34.684 INFO:tasks.workunit.client.0.smithi092.stdout:6/546: truncate d4/d20/d22/f7a 536405 0 2024-05-06T12:14:34.685 INFO:tasks.workunit.client.0.smithi092.stdout:6/547: stat d4/d20/f31 0 2024-05-06T12:14:34.701 INFO:tasks.workunit.client.0.smithi092.stdout:0/548: sync 2024-05-06T12:14:34.701 INFO:tasks.workunit.client.0.smithi092.stdout:3/363: sync 2024-05-06T12:14:34.704 INFO:tasks.workunit.client.0.smithi092.stdout:0/549: mkdir d7/dc/d28/d54/d12/d8f/dad 0 2024-05-06T12:14:34.705 INFO:tasks.workunit.client.0.smithi092.stdout:0/550: stat d7/f4f 0 2024-05-06T12:14:34.706 INFO:tasks.workunit.client.0.smithi092.stdout:0/551: chown d7/dc/d28/d54/d1e/d26/d3a/f5f 1 1 2024-05-06T12:14:34.707 INFO:tasks.workunit.client.0.smithi092.stdout:0/552: fsync d7/dc/d9e/f40 0 2024-05-06T12:14:34.711 INFO:tasks.workunit.client.0.smithi092.stdout:0/553: creat d7/dc/d9e/fae x:0 0 0 2024-05-06T12:14:34.713 INFO:tasks.workunit.client.0.smithi092.stdout:3/364: dwrite d8/d1b/f3e [4194304,4194304] 0 2024-05-06T12:14:34.715 INFO:tasks.workunit.client.0.smithi092.stdout:0/554: unlink d7/dc/f8d 0 2024-05-06T12:14:34.718 INFO:tasks.workunit.client.0.smithi092.stdout:0/555: fsync d7/dc/d28/d54/d1e/d5d/f68 0 2024-05-06T12:14:34.719 INFO:tasks.workunit.client.0.smithi092.stdout:3/365: creat d8/db/d4c/d40/d67/f71 x:0 0 0 2024-05-06T12:14:34.722 INFO:tasks.workunit.client.0.smithi092.stdout:0/556: creat d7/dc/d28/d54/d1e/d26/faf x:0 0 0 2024-05-06T12:14:34.726 INFO:tasks.workunit.client.0.smithi092.stdout:3/366: dwrite d8/d24/d27/f2e [0,4194304] 0 2024-05-06T12:14:34.731 INFO:tasks.workunit.client.0.smithi092.stdout:3/367: mkdir d8/db/d72 0 2024-05-06T12:14:34.737 INFO:tasks.workunit.client.0.smithi092.stdout:0/557: write d7/dc/d28/d54/d1e/d46/f7b [394760,8150] 0 2024-05-06T12:14:34.739 INFO:tasks.workunit.client.0.smithi092.stdout:3/368: dwrite d8/d24/d33/d4b/f46 [4194304,4194304] 0 2024-05-06T12:14:34.740 INFO:tasks.workunit.client.0.smithi092.stdout:0/558: read d7/dc/f9c [4168810,109031] 0 2024-05-06T12:14:34.742 INFO:tasks.workunit.client.0.smithi092.stdout:3/369: read d8/d24/d27/f2e [1854800,9480] 0 2024-05-06T12:14:34.744 INFO:tasks.workunit.client.0.smithi092.stdout:0/559: symlink d7/dc/d6f/lb0 0 2024-05-06T12:14:34.745 INFO:tasks.workunit.client.0.smithi092.stdout:3/370: mkdir d8/d1b/d73 0 2024-05-06T12:14:34.747 INFO:tasks.workunit.client.0.smithi092.stdout:0/560: write d7/dc/d28/d54/d1e/d26/d96/fab [731428,97478] 0 2024-05-06T12:14:34.750 INFO:tasks.workunit.client.0.smithi092.stdout:0/561: truncate d7/dc/d28/d54/d1e/d5d/f76 4646712 0 2024-05-06T12:14:34.753 INFO:tasks.workunit.client.0.smithi092.stdout:0/562: symlink d7/dc/d6f/lb1 0 2024-05-06T12:14:34.754 INFO:tasks.workunit.client.0.smithi092.stdout:3/371: dwrite d8/d24/d27/f2e [0,4194304] 0 2024-05-06T12:14:34.756 INFO:tasks.workunit.client.0.smithi092.stdout:3/372: chown d8/d24/d33/d4b/c25 0 1 2024-05-06T12:14:34.758 INFO:tasks.workunit.client.0.smithi092.stdout:0/563: rename d7/dc/d9e/d1f to d7/dc/d28/d54/d12/d7f/db2 0 2024-05-06T12:14:34.759 INFO:tasks.workunit.client.0.smithi092.stdout:3/373: mkdir d8/d24/d33/d4b/d74 0 2024-05-06T12:14:34.759 INFO:tasks.workunit.client.0.smithi092.stdout:0/564: stat d7/l59 0 2024-05-06T12:14:34.760 INFO:tasks.workunit.client.0.smithi092.stdout:0/565: write d7/dc/d9e/f49 [5256469,91363] 0 2024-05-06T12:14:34.761 INFO:tasks.workunit.client.0.smithi092.stdout:3/374: mknod d8/db/d72/c75 0 2024-05-06T12:14:34.763 INFO:tasks.workunit.client.0.smithi092.stdout:0/566: creat d7/dc/d28/d54/d1e/d46/fb3 x:0 0 0 2024-05-06T12:14:34.764 INFO:tasks.workunit.client.0.smithi092.stdout:3/375: mknod d8/db/d14/d5e/c76 0 2024-05-06T12:14:34.764 INFO:tasks.workunit.client.0.smithi092.stdout:3/376: read - d8/d24/f6f zero size 2024-05-06T12:14:34.766 INFO:tasks.workunit.client.0.smithi092.stdout:0/567: mknod d7/dc/d28/d54/d1e/d46/d63/d80/cb4 0 2024-05-06T12:14:34.769 INFO:tasks.workunit.client.0.smithi092.stdout:3/377: link d8/db/d14/f50 d8/d24/f77 0 2024-05-06T12:14:34.769 INFO:tasks.workunit.client.0.smithi092.stdout:0/568: mkdir d7/dc/d28/d54/d12/d7f/db5 0 2024-05-06T12:14:34.772 INFO:tasks.workunit.client.0.smithi092.stdout:0/569: link d7/dc/d28/d54/d1e/d26/d4e/l6b d7/dc/d6f/da5/lb6 0 2024-05-06T12:14:34.775 INFO:tasks.workunit.client.0.smithi092.stdout:0/570: getdents d7/dc/d31 0 2024-05-06T12:14:34.777 INFO:tasks.workunit.client.0.smithi092.stdout:5/444: sync 2024-05-06T12:14:34.777 INFO:tasks.workunit.client.0.smithi092.stdout:7/580: sync 2024-05-06T12:14:34.778 INFO:tasks.workunit.client.0.smithi092.stdout:9/326: sync 2024-05-06T12:14:34.778 INFO:tasks.workunit.client.0.smithi092.stdout:8/486: sync 2024-05-06T12:14:34.778 INFO:tasks.workunit.client.0.smithi092.stdout:4/469: sync 2024-05-06T12:14:34.778 INFO:tasks.workunit.client.0.smithi092.stdout:1/409: sync 2024-05-06T12:14:34.778 INFO:tasks.workunit.client.0.smithi092.stdout:0/571: unlink d7/dc/d28/d54/d1e/d39/c86 0 2024-05-06T12:14:34.779 INFO:tasks.workunit.client.0.smithi092.stdout:4/470: read - d9/d2b/f77 zero size 2024-05-06T12:14:34.779 INFO:tasks.workunit.client.0.smithi092.stdout:7/581: readlink d9/la 0 2024-05-06T12:14:34.780 INFO:tasks.workunit.client.0.smithi092.stdout:1/410: write d3/d10/d13/d19/d34/f66 [251973,12369] 0 2024-05-06T12:14:34.782 INFO:tasks.workunit.client.0.smithi092.stdout:0/572: write d7/dc/d28/d54/f3b [563194,42037] 0 2024-05-06T12:14:34.783 INFO:tasks.workunit.client.0.smithi092.stdout:5/445: mkdir d4/d34/d28/d4e/d5d/d82/d77/da7 0 2024-05-06T12:14:34.788 INFO:tasks.workunit.client.0.smithi092.stdout:4/471: dwrite d9/d2b/d65/d83/f91 [0,4194304] 0 2024-05-06T12:14:34.788 INFO:tasks.workunit.client.0.smithi092.stdout:4/472: chown d9/df/c26 31721671 1 2024-05-06T12:14:34.791 INFO:tasks.workunit.client.0.smithi092.stdout:3/378: dread d8/db/d4c/d3d/f60 [0,4194304] 0 2024-05-06T12:14:34.792 INFO:tasks.workunit.client.0.smithi092.stdout:3/379: readlink d8/d24/d33/d4b/l35 0 2024-05-06T12:14:34.824 INFO:tasks.workunit.client.0.smithi092.stdout:7/582: symlink d9/de/d10/d17/d99/lc0 0 2024-05-06T12:14:34.827 INFO:tasks.workunit.client.0.smithi092.stdout:0/573: mknod d7/dc/d28/d54/d12/d7f/cb7 0 2024-05-06T12:14:34.829 INFO:tasks.workunit.client.0.smithi092.stdout:5/446: mknod d4/d34/d28/d4e/d5d/d82/d77/ca8 0 2024-05-06T12:14:34.832 INFO:tasks.workunit.client.0.smithi092.stdout:3/380: unlink d8/db/d4c/d3d/f4e 0 2024-05-06T12:14:34.832 INFO:tasks.workunit.client.0.smithi092.stdout:3/381: fdatasync d8/d24/d33/d39/f44 0 2024-05-06T12:14:34.833 INFO:tasks.workunit.client.0.smithi092.stdout:3/382: chown d8/db/d4c/d3d 0 1 2024-05-06T12:14:34.835 INFO:tasks.workunit.client.0.smithi092.stdout:3/383: truncate d8/db/d14/f50 5185619 0 2024-05-06T12:14:34.836 INFO:tasks.workunit.client.0.smithi092.stdout:9/327: write f7 [394297,79760] 0 2024-05-06T12:14:34.837 INFO:tasks.workunit.client.0.smithi092.stdout:9/328: write f7 [314311,61233] 0 2024-05-06T12:14:34.838 INFO:tasks.workunit.client.0.smithi092.stdout:4/473: symlink d9/l99 0 2024-05-06T12:14:34.840 INFO:tasks.workunit.client.0.smithi092.stdout:0/574: dread d7/fa6 [0,4194304] 0 2024-05-06T12:14:34.843 INFO:tasks.workunit.client.0.smithi092.stdout:7/583: mknod d9/de/d10/d17/d21/d2e/d87/cc1 0 2024-05-06T12:14:34.844 INFO:tasks.workunit.client.0.smithi092.stdout:5/447: creat d4/d34/d29/da3/fa9 x:0 0 0 2024-05-06T12:14:34.846 INFO:tasks.workunit.client.0.smithi092.stdout:4/474: mknod d9/d2b/d65/d83/c9a 0 2024-05-06T12:14:34.847 INFO:tasks.workunit.client.0.smithi092.stdout:7/584: read d9/de/d1f/d37/d48/f8e [2553643,10732] 0 2024-05-06T12:14:34.848 INFO:tasks.workunit.client.0.smithi092.stdout:9/329: truncate d16/d2c/d2f/d3d/d33/d37/d40/d43/f50 4629495 0 2024-05-06T12:14:34.848 INFO:tasks.workunit.client.0.smithi092.stdout:1/411: dwrite d3/d10/d13/d19/f40 [0,4194304] 0 2024-05-06T12:14:34.857 INFO:tasks.workunit.client.0.smithi092.stdout:9/330: creat d16/d2c/d2f/d3d/d33/d57/f64 x:0 0 0 2024-05-06T12:14:34.859 INFO:tasks.workunit.client.0.smithi092.stdout:3/384: dread d8/d1b/f2b [0,4194304] 0 2024-05-06T12:14:34.862 INFO:tasks.workunit.client.0.smithi092.stdout:9/331: creat d16/d2c/f65 x:0 0 0 2024-05-06T12:14:34.862 INFO:tasks.workunit.client.0.smithi092.stdout:5/448: dwrite d4/d34/d28/d4e/d5d/d82/d1f/d20/f8d [4194304,4194304] 0 2024-05-06T12:14:34.865 INFO:tasks.workunit.client.0.smithi092.stdout:4/475: dwrite d9/f86 [0,4194304] 0 2024-05-06T12:14:34.867 INFO:tasks.workunit.client.0.smithi092.stdout:3/385: unlink d8/d24/d33/d4b/c1a 0 2024-05-06T12:14:34.874 INFO:tasks.workunit.client.0.smithi092.stdout:4/476: write d9/dd/f5f [501177,64298] 0 2024-05-06T12:14:34.874 INFO:tasks.workunit.client.0.smithi092.stdout:0/575: truncate f6 1807120 0 2024-05-06T12:14:34.874 INFO:tasks.workunit.client.0.smithi092.stdout:1/412: dwrite d3/d10/d13/d19/f2e [0,4194304] 0 2024-05-06T12:14:34.876 INFO:tasks.workunit.client.0.smithi092.stdout:4/477: fsync d9/df/f14 0 2024-05-06T12:14:34.882 INFO:tasks.workunit.client.0.smithi092.stdout:7/585: dwrite d9/de/d10/f27 [0,4194304] 0 2024-05-06T12:14:34.882 INFO:tasks.workunit.client.1.smithi190.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-05-06T12:14:34.883 INFO:tasks.workunit.client.0.smithi092.stdout:9/332: rename d16/d2c/d2f/f3a to d16/d2c/d2f/d3d/d33/f66 0 2024-05-06T12:14:34.887 INFO:tasks.workunit.client.0.smithi092.stdout:5/449: symlink d4/d34/d28/d4e/d5d/d82/d1f/d20/d99/laa 0 2024-05-06T12:14:34.890 INFO:tasks.workunit.client.0.smithi092.stdout:7/586: rmdir d9 39 2024-05-06T12:14:34.891 INFO:tasks.workunit.client.0.smithi092.stdout:3/386: dwrite d8/d24/f77 [4194304,4194304] 0 2024-05-06T12:14:34.892 INFO:tasks.workunit.client.0.smithi092.stdout:5/450: stat d4/d5/d10/d31/d53/d64/l47 0 2024-05-06T12:14:34.895 INFO:tasks.workunit.client.0.smithi092.stdout:4/478: creat d9/dd/d90/f9b x:0 0 0 2024-05-06T12:14:34.895 INFO:tasks.workunit.client.0.smithi092.stdout:1/413: truncate d3/d10/d13/d46/f68 342197 0 2024-05-06T12:14:34.897 INFO:tasks.workunit.client.0.smithi092.stdout:9/333: dwrite d16/d2c/d2f/d3d/f38 [0,4194304] 0 2024-05-06T12:14:34.898 INFO:tasks.workunit.client.0.smithi092.stdout:1/414: readlink d3/d10/d13/d19/d21/l7c 0 2024-05-06T12:14:34.899 INFO:tasks.workunit.client.0.smithi092.stdout:5/451: symlink d4/d5/d10/d31/d53/d64/lab 0 2024-05-06T12:14:34.901 INFO:tasks.workunit.client.0.smithi092.stdout:4/479: mkdir d9/d27/d4a/d9c 0 2024-05-06T12:14:34.901 INFO:tasks.workunit.client.0.smithi092.stdout:5/452: write d4/d59/f83 [324062,99234] 0 2024-05-06T12:14:34.902 INFO:tasks.workunit.client.0.smithi092.stdout:5/453: chown d4/d5/d10/d31/d53/d64/f8a 1442856 1 2024-05-06T12:14:34.903 INFO:tasks.workunit.client.0.smithi092.stdout:7/587: rmdir d9/de/d1f/d37/d48/d65 39 2024-05-06T12:14:34.904 INFO:tasks.workunit.client.0.smithi092.stdout:7/588: read d9/dd/f9b [255730,58594] 0 2024-05-06T12:14:34.906 INFO:tasks.workunit.client.0.smithi092.stdout:9/334: creat d16/d2c/d2f/d3d/f67 x:0 0 0 2024-05-06T12:14:34.908 INFO:tasks.workunit.client.0.smithi092.stdout:9/335: dread - d16/d2c/d2f/d3d/f61 zero size 2024-05-06T12:14:34.911 INFO:tasks.workunit.client.0.smithi092.stdout:4/480: dwrite d9/dd/f5f [0,4194304] 0 2024-05-06T12:14:34.912 INFO:tasks.workunit.client.0.smithi092.stdout:5/454: write d4/d34/d28/f74 [4118652,92416] 0 2024-05-06T12:14:34.916 INFO:tasks.workunit.client.1.smithi190.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-05-06T12:14:34.917 INFO:tasks.workunit.client.1.smithi190.stderr:+ make 2024-05-06T12:14:34.918 INFO:tasks.workunit.client.0.smithi092.stdout:6/548: sync 2024-05-06T12:14:34.918 INFO:tasks.workunit.client.0.smithi092.stdout:2/317: sync 2024-05-06T12:14:34.920 INFO:tasks.workunit.client.0.smithi092.stdout:0/576: dread d7/f70 [0,4194304] 0 2024-05-06T12:14:34.922 INFO:tasks.workunit.client.0.smithi092.stdout:3/387: fsync d8/db/d14/f50 0 2024-05-06T12:14:34.924 INFO:tasks.workunit.client.0.smithi092.stdout:4/481: link d9/df/f15 d9/df/d62/f9d 0 2024-05-06T12:14:34.924 INFO:tasks.workunit.client.0.smithi092.stdout:5/455: dread d4/d34/d28/d4e/d4b/f4d [0,4194304] 0 2024-05-06T12:14:34.926 INFO:tasks.workunit.client.0.smithi092.stdout:1/415: getdents d3/d10/d13/d19/d21/d47/d4f/d58/d65 0 2024-05-06T12:14:34.927 INFO:tasks.workunit.client.0.smithi092.stdout:9/336: dwrite d16/d2c/d2f/d3d/d33/d37/d40/d43/f51 [0,4194304] 0 2024-05-06T12:14:34.928 INFO:tasks.workunit.client.0.smithi092.stdout:2/318: mkdir df/d5c/d60 0 2024-05-06T12:14:34.929 INFO:tasks.workunit.client.0.smithi092.stdout:4/482: write d9/d27/f4f [1286137,6212] 0 2024-05-06T12:14:34.931 INFO:tasks.workunit.client.0.smithi092.stdout:0/577: truncate d7/dc/d6f/da2/fac 6778938 0 2024-05-06T12:14:34.932 INFO:tasks.workunit.client.0.smithi092.stdout:5/456: creat d4/d34/d28/fac x:0 0 0 2024-05-06T12:14:34.934 INFO:tasks.workunit.client.0.smithi092.stdout:9/337: symlink d16/d2c/d2f/l68 0 2024-05-06T12:14:34.935 INFO:tasks.workunit.client.0.smithi092.stdout:6/549: dwrite d4/d4c/d52/f80 [0,4194304] 0 2024-05-06T12:14:34.935 INFO:tasks.workunit.client.0.smithi092.stdout:7/589: truncate d9/de/d10/d17/d6a/f7a 29546 0 2024-05-06T12:14:34.936 INFO:tasks.workunit.client.0.smithi092.stdout:0/578: write d7/dc/d28/d54/d1e/d46/d63/f9d [641683,93876] 0 2024-05-06T12:14:34.938 INFO:tasks.workunit.client.0.smithi092.stdout:1/416: mkdir d3/d10/d13/d19/d21/d47/d4f/d58/d82 0 2024-05-06T12:14:34.941 INFO:tasks.workunit.client.0.smithi092.stdout:9/338: fdatasync d16/d1e/f25 0 2024-05-06T12:14:34.942 INFO:tasks.workunit.client.0.smithi092.stdout:5/457: symlink d4/d5/d10/d31/d53/d64/lad 0 2024-05-06T12:14:34.943 INFO:tasks.workunit.client.0.smithi092.stdout:9/339: truncate d16/d1c/f52 1273910 0 2024-05-06T12:14:34.944 INFO:tasks.workunit.client.0.smithi092.stdout:6/550: stat d4/d24/d26/f32 0 2024-05-06T12:14:34.945 INFO:tasks.workunit.client.0.smithi092.stdout:5/458: write d4/d34/d28/d4e/f4f [1148748,27892] 0 2024-05-06T12:14:34.948 INFO:tasks.workunit.client.0.smithi092.stdout:4/483: truncate d9/dd/f19 2846699 0 2024-05-06T12:14:34.954 INFO:tasks.workunit.client.0.smithi092.stdout:9/340: dwrite d16/d2c/d4b/f54 [0,4194304] 0 2024-05-06T12:14:34.958 INFO:tasks.workunit.client.0.smithi092.stdout:4/484: fdatasync d9/df/f14 0 2024-05-06T12:14:34.958 INFO:tasks.workunit.client.0.smithi092.stdout:4/485: readlink d9/d2b/d5b/l8a 0 2024-05-06T12:14:34.958 INFO:tasks.workunit.client.0.smithi092.stdout:7/590: rmdir d9/de/d75 0 2024-05-06T12:14:34.958 INFO:tasks.workunit.client.0.smithi092.stdout:2/319: getdents df/d1b 0 2024-05-06T12:14:34.959 INFO:tasks.workunit.client.0.smithi092.stdout:4/486: creat d9/dd/f9e x:0 0 0 2024-05-06T12:14:34.959 INFO:tasks.workunit.client.0.smithi092.stdout:7/591: rmdir d9/dd 39 2024-05-06T12:14:34.961 INFO:tasks.workunit.client.0.smithi092.stdout:9/341: symlink d16/d2c/d2f/d3d/d33/l69 0 2024-05-06T12:14:34.962 INFO:tasks.workunit.client.0.smithi092.stdout:9/342: chown d16/d1c/f27 0 1 2024-05-06T12:14:34.967 INFO:tasks.workunit.client.0.smithi092.stdout:4/487: dwrite d9/df/d62/f63 [0,4194304] 0 2024-05-06T12:14:34.967 INFO:tasks.workunit.client.0.smithi092.stdout:9/343: write d16/d2c/d2f/d3d/d33/f66 [8414335,2066] 0 2024-05-06T12:14:34.971 INFO:tasks.workunit.client.0.smithi092.stdout:9/344: creat d16/d1e/f6a x:0 0 0 2024-05-06T12:14:34.972 INFO:tasks.workunit.client.0.smithi092.stdout:7/592: dwrite d9/de/d1f/d37/d48/d65/f98 [0,4194304] 0 2024-05-06T12:14:34.972 INFO:tasks.workunit.client.0.smithi092.stdout:7/593: stat d9/de/d10/d17/cae 0 2024-05-06T12:14:34.974 INFO:tasks.workunit.client.0.smithi092.stdout:4/488: rename d9/d2b/d65/d83/f8c to d9/d27/f9f 0 2024-05-06T12:14:34.977 INFO:tasks.workunit.client.0.smithi092.stdout:4/489: rename d9/dd/c3a to d9/df/d48/ca0 0 2024-05-06T12:14:34.978 INFO:tasks.workunit.client.0.smithi092.stdout:1/417: dread d3/d10/d13/d46/f55 [0,4194304] 0 2024-05-06T12:14:34.979 INFO:tasks.workunit.client.0.smithi092.stdout:9/345: mkdir d16/d2c/d2f/d3d/d33/d6b 0 2024-05-06T12:14:34.980 INFO:tasks.workunit.client.0.smithi092.stdout:7/594: mknod d9/de/d1f/d37/d48/cc2 0 2024-05-06T12:14:34.980 INFO:tasks.workunit.client.0.smithi092.stdout:9/346: stat d16/d2c/d2f/d3d/f3c 0 2024-05-06T12:14:34.981 INFO:tasks.workunit.client.0.smithi092.stdout:9/347: chown d16/d2c/d2f/d3d/f67 103 1 2024-05-06T12:14:34.983 INFO:tasks.workunit.client.0.smithi092.stdout:7/595: truncate d9/de/d10/d17/d6a/fb9 896236 0 2024-05-06T12:14:34.984 INFO:tasks.workunit.client.0.smithi092.stdout:1/418: chown d3/d10/d13/d19/d21/c56 5897 1 2024-05-06T12:14:34.985 INFO:tasks.workunit.client.0.smithi092.stdout:9/348: symlink d16/d2c/d2f/d3d/d33/d57/l6c 0 2024-05-06T12:14:34.987 INFO:tasks.workunit.client.0.smithi092.stdout:7/596: mkdir d9/de/d10/d17/d99/dc3 0 2024-05-06T12:14:34.993 INFO:tasks.workunit.client.0.smithi092.stdout:9/349: dwrite d16/d2c/d2f/f31 [0,4194304] 0 2024-05-06T12:14:34.996 INFO:tasks.workunit.client.0.smithi092.stdout:9/350: symlink d16/d1e/l6d 0 2024-05-06T12:14:34.999 INFO:tasks.workunit.client.0.smithi092.stdout:9/351: mknod d16/d2c/d2f/d3d/d33/d37/c6e 0 2024-05-06T12:14:35.000 INFO:tasks.workunit.client.0.smithi092.stdout:9/352: chown d16/d2c/d2f/f31 1 1 2024-05-06T12:14:35.003 INFO:tasks.workunit.client.0.smithi092.stdout:7/597: getdents d9/de/d1f/d37/d48 0 2024-05-06T12:14:35.004 INFO:tasks.workunit.client.0.smithi092.stdout:4/490: write d9/d27/f50 [4698800,127673] 0 2024-05-06T12:14:35.007 INFO:tasks.workunit.client.0.smithi092.stdout:7/598: mknod d9/de/d10/d17/cc4 0 2024-05-06T12:14:35.007 INFO:tasks.workunit.client.0.smithi092.stdout:9/353: dwrite d16/d2c/f35 [0,4194304] 0 2024-05-06T12:14:35.009 INFO:tasks.workunit.client.0.smithi092.stdout:4/491: creat d9/dd/fa1 x:0 0 0 2024-05-06T12:14:35.013 INFO:tasks.workunit.client.0.smithi092.stdout:4/492: chown d9/l3d 8 1 2024-05-06T12:14:35.014 INFO:tasks.workunit.client.0.smithi092.stdout:1/419: dwrite d3/d10/d13/d19/d21/d47/f5f [0,4194304] 0 2024-05-06T12:14:35.022 INFO:tasks.workunit.client.0.smithi092.stdout:7/599: dwrite d9/de/d1f/d37/d48/d65/f6b [0,4194304] 0 2024-05-06T12:14:35.023 INFO:tasks.workunit.client.0.smithi092.stdout:9/354: dwrite d16/d1c/f2b [4194304,4194304] 0 2024-05-06T12:14:35.026 INFO:tasks.workunit.client.0.smithi092.stdout:1/420: link d3/d10/d13/f14 d3/d10/d13/d19/d21/d47/d4f/d58/d65/d81/f83 0 2024-05-06T12:14:35.033 INFO:tasks.workunit.client.0.smithi092.stdout:7/600: dwrite d9/de/d2d/fad [0,4194304] 0 2024-05-06T12:14:35.036 INFO:tasks.workunit.client.0.smithi092.stdout:9/355: rename d16/d2c/f4f to d16/d2c/d2f/d3d/d33/d37/f6f 0 2024-05-06T12:14:35.038 INFO:tasks.workunit.client.0.smithi092.stdout:4/493: truncate d9/df/d62/f63 1927458 0 2024-05-06T12:14:35.039 INFO:tasks.workunit.client.0.smithi092.stdout:4/494: stat d9/f86 0 2024-05-06T12:14:35.040 INFO:tasks.workunit.client.0.smithi092.stdout:4/495: write d9/d27/f50 [3470546,94321] 0 2024-05-06T12:14:35.041 INFO:tasks.workunit.client.0.smithi092.stdout:9/356: dread d16/d2c/f35 [0,4194304] 0 2024-05-06T12:14:35.043 INFO:tasks.workunit.client.0.smithi092.stdout:1/421: rename d3/d10/d13/d19/d21/d47 to d3/d42/d84 0 2024-05-06T12:14:35.044 INFO:tasks.workunit.client.0.smithi092.stdout:7/601: creat d9/de/d10/d17/d99/db6/fc5 x:0 0 0 2024-05-06T12:14:35.047 INFO:tasks.workunit.client.0.smithi092.stdout:1/422: creat d3/d42/f85 x:0 0 0 2024-05-06T12:14:35.048 INFO:tasks.workunit.client.0.smithi092.stdout:8/487: sync 2024-05-06T12:14:35.048 INFO:tasks.workunit.client.0.smithi092.stdout:4/496: mkdir d9/da2 0 2024-05-06T12:14:35.049 INFO:tasks.workunit.client.0.smithi092.stdout:9/357: dwrite d16/d2c/d2f/d3d/d33/d37/d40/f5f [0,4194304] 0 2024-05-06T12:14:35.051 INFO:tasks.workunit.client.0.smithi092.stdout:1/423: truncate d3/d42/f85 838497 0 2024-05-06T12:14:35.052 INFO:tasks.workunit.client.0.smithi092.stdout:8/488: write d9/d21/d37/f3b [8231269,89769] 0 2024-05-06T12:14:35.052 INFO:tasks.workunit.client.0.smithi092.stdout:9/358: stat d16/d2c/l2d 0 2024-05-06T12:14:35.052 INFO:tasks.workunit.client.0.smithi092.stdout:4/497: read d9/df/f46 [6291692,69373] 0 2024-05-06T12:14:35.052 INFO:tasks.workunit.client.0.smithi092.stdout:7/602: symlink d9/de/d10/d17/d21/d2e/d9c/lc6 0 2024-05-06T12:14:35.061 INFO:tasks.workunit.client.0.smithi092.stdout:4/498: creat d9/df/d48/fa3 x:0 0 0 2024-05-06T12:14:35.063 INFO:tasks.workunit.client.0.smithi092.stdout:1/424: rename d3/d10/d13/d19/f2e to d3/d10/d13/d19/d21/d72/f86 0 2024-05-06T12:14:35.063 INFO:tasks.workunit.client.0.smithi092.stdout:9/359: dread d16/f29 [0,4194304] 0 2024-05-06T12:14:35.071 INFO:tasks.workunit.client.0.smithi092.stdout:4/499: dread d9/df/f14 [0,4194304] 0 2024-05-06T12:14:35.072 INFO:tasks.workunit.client.0.smithi092.stdout:8/489: creat d9/fb6 x:0 0 0 2024-05-06T12:14:35.073 INFO:tasks.workunit.client.0.smithi092.stdout:9/360: creat d16/d1e/f70 x:0 0 0 2024-05-06T12:14:35.078 INFO:tasks.workunit.client.0.smithi092.stdout:9/361: dwrite d16/f17 [0,4194304] 0 2024-05-06T12:14:35.080 INFO:tasks.workunit.client.0.smithi092.stdout:9/362: write d16/d2c/d3f/f5e [641741,51185] 0 2024-05-06T12:14:35.108 INFO:tasks.workunit.client.1.smithi190.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-05-06T12:14:35.169 INFO:tasks.workunit.client.0.smithi092.stdout:7/603: truncate d9/de/d10/d17/d6a/f9d 1328616 0 2024-05-06T12:14:35.169 INFO:tasks.workunit.client.0.smithi092.stdout:1/425: truncate d3/d10/d13/d19/d34/f4b 4117725 0 2024-05-06T12:14:35.169 INFO:tasks.workunit.client.0.smithi092.stdout:8/490: truncate d9/d21/d37/d46/d5c/d62/f68 272722 0 2024-05-06T12:14:35.169 INFO:tasks.workunit.client.0.smithi092.stdout:4/500: link d9/df/d48/l51 d9/d2b/d7c/la4 0 2024-05-06T12:14:35.171 INFO:tasks.workunit.client.0.smithi092.stdout:8/491: symlink d9/d21/d37/lb7 0 2024-05-06T12:14:35.171 INFO:tasks.workunit.client.0.smithi092.stdout:4/501: truncate d9/d27/f4d 1006612 0 2024-05-06T12:14:35.173 INFO:tasks.workunit.client.0.smithi092.stdout:8/492: chown d9/l5b 11494 1 2024-05-06T12:14:35.174 INFO:tasks.workunit.client.0.smithi092.stdout:1/426: rename d3/d10/f45 to d3/d10/d13/d19/f87 0 2024-05-06T12:14:35.175 INFO:tasks.workunit.client.0.smithi092.stdout:9/363: dwrite d16/d2c/d2f/d3d/d33/f3b [0,4194304] 0 2024-05-06T12:14:35.175 INFO:tasks.workunit.client.0.smithi092.stdout:1/427: stat d3/f70 0 2024-05-06T12:14:35.180 INFO:tasks.workunit.client.0.smithi092.stdout:7/604: dwrite d9/de/d10/d17/d99/fb3 [0,4194304] 0 2024-05-06T12:14:35.183 INFO:tasks.workunit.client.0.smithi092.stdout:1/428: mkdir d3/d10/d13/d19/d21/d88 0 2024-05-06T12:14:35.184 INFO:tasks.workunit.client.0.smithi092.stdout:1/429: chown d3/d10/d13/d46/f55 129 1 2024-05-06T12:14:35.187 INFO:tasks.workunit.client.0.smithi092.stdout:4/502: rename d9/d2b/d7c/c88 to d9/d27/d4a/ca5 0 2024-05-06T12:14:35.190 INFO:tasks.workunit.client.0.smithi092.stdout:1/430: mkdir d3/d10/d13/d19/d34/d89 0 2024-05-06T12:14:35.195 INFO:tasks.workunit.client.0.smithi092.stdout:1/431: truncate d3/d42/d84/d4f/d58/f69 514484 0 2024-05-06T12:14:35.195 INFO:tasks.workunit.client.0.smithi092.stdout:4/503: dwrite d9/df/d48/f69 [0,4194304] 0 2024-05-06T12:14:35.196 INFO:tasks.workunit.client.0.smithi092.stdout:9/364: getdents d16/d1c 0 2024-05-06T12:14:35.199 INFO:tasks.workunit.client.0.smithi092.stdout:4/504: mkdir d9/dd/da6 0 2024-05-06T12:14:35.203 INFO:tasks.workunit.client.0.smithi092.stdout:9/365: dwrite d16/d2c/d2f/d3d/d33/f66 [4194304,4194304] 0 2024-05-06T12:14:35.206 INFO:tasks.workunit.client.0.smithi092.stdout:4/505: mknod d9/dd/d90/ca7 0 2024-05-06T12:14:35.207 INFO:tasks.workunit.client.0.smithi092.stdout:4/506: fsync d9/d27/f49 0 2024-05-06T12:14:35.209 INFO:tasks.workunit.client.0.smithi092.stdout:9/366: chown d16/d2c/d2f/d3d/d33/d37/d40/d43/f50 368659 1 2024-05-06T12:14:35.211 INFO:tasks.workunit.client.0.smithi092.stdout:4/507: mkdir d9/d27/da8 0 2024-05-06T12:14:35.212 INFO:tasks.workunit.client.0.smithi092.stdout:4/508: rmdir d9/d2b/d65/d83 39 2024-05-06T12:14:35.214 INFO:tasks.workunit.client.0.smithi092.stdout:9/367: read d16/d2c/d2f/d3d/f62 [3034972,100206] 0 2024-05-06T12:14:35.214 INFO:tasks.workunit.client.0.smithi092.stdout:4/509: creat d9/d2b/d5b/fa9 x:0 0 0 2024-05-06T12:14:35.215 INFO:tasks.workunit.client.0.smithi092.stdout:4/510: fdatasync d9/d2b/f8f 0 2024-05-06T12:14:35.218 INFO:tasks.workunit.client.0.smithi092.stdout:8/493: dread d9/d21/d37/d46/d5c/d71/f80 [0,4194304] 0 2024-05-06T12:14:35.221 INFO:tasks.workunit.client.0.smithi092.stdout:4/511: getdents d9/dd/d90 0 2024-05-06T12:14:35.223 INFO:tasks.workunit.client.0.smithi092.stdout:8/494: creat d9/d21/d54/d79/d8a/fb8 x:0 0 0 2024-05-06T12:14:35.224 INFO:tasks.workunit.client.0.smithi092.stdout:8/495: chown d9/d21/d72/c8e 70 1 2024-05-06T12:14:35.225 INFO:tasks.workunit.client.0.smithi092.stdout:4/512: rename d9/d27/d4a/c6f to d9/caa 0 2024-05-06T12:14:35.226 INFO:tasks.workunit.client.0.smithi092.stdout:3/388: sync 2024-05-06T12:14:35.227 INFO:tasks.workunit.client.0.smithi092.stdout:8/496: mknod d9/dc/d4a/d5f/da8/cb9 0 2024-05-06T12:14:35.228 INFO:tasks.workunit.client.0.smithi092.stdout:9/368: truncate d16/d2c/d2f/d3d/d33/f56 1205748 0 2024-05-06T12:14:35.229 INFO:tasks.workunit.client.0.smithi092.stdout:4/513: mkdir d9/d2b/d7c/dab 0 2024-05-06T12:14:35.230 INFO:tasks.workunit.client.0.smithi092.stdout:3/389: symlink d8/db/d4c/d40/d67/l78 0 2024-05-06T12:14:35.230 INFO:tasks.workunit.client.0.smithi092.stdout:8/497: dread - d9/d21/d37/d74/f83 zero size 2024-05-06T12:14:35.233 INFO:tasks.workunit.client.0.smithi092.stdout:3/390: rename d8/d24/c22 to d8/db/d14/d5e/c79 0 2024-05-06T12:14:35.235 INFO:tasks.workunit.client.0.smithi092.stdout:9/369: link d16/f29 d16/d2c/d2f/d3d/d33/f71 0 2024-05-06T12:14:35.236 INFO:tasks.workunit.client.0.smithi092.stdout:8/498: rmdir d9/d21/d37/d46/d5c/d62 39 2024-05-06T12:14:35.245 INFO:tasks.workunit.client.0.smithi092.stdout:3/391: truncate d8/db/d14/f50 2984213 0 2024-05-06T12:14:35.245 INFO:tasks.workunit.client.0.smithi092.stdout:4/514: write d9/df/d62/f63 [2915286,20568] 0 2024-05-06T12:14:35.245 INFO:tasks.workunit.client.0.smithi092.stdout:3/392: fsync f3 0 2024-05-06T12:14:35.246 INFO:tasks.workunit.client.0.smithi092.stdout:9/370: unlink d16/d1c/c23 0 2024-05-06T12:14:35.254 INFO:tasks.workunit.client.0.smithi092.stdout:3/393: dwrite d8/d24/d27/f4a [0,4194304] 0 2024-05-06T12:14:35.254 INFO:tasks.workunit.client.0.smithi092.stdout:4/515: dwrite d9/d2b/d5b/fa9 [0,4194304] 0 2024-05-06T12:14:35.257 INFO:tasks.workunit.client.0.smithi092.stdout:8/499: getdents d9/d21/d37/d74 0 2024-05-06T12:14:35.260 INFO:tasks.workunit.client.0.smithi092.stdout:3/394: mkdir d8/db/d4c/d40/d7a 0 2024-05-06T12:14:35.262 INFO:tasks.workunit.client.0.smithi092.stdout:4/516: rename d9/df/d48/fa3 to d9/d2b/fac 0 2024-05-06T12:14:35.264 INFO:tasks.workunit.client.0.smithi092.stdout:9/371: creat d16/d2c/d2f/d3d/d33/d37/f72 x:0 0 0 2024-05-06T12:14:35.268 INFO:tasks.workunit.client.0.smithi092.stdout:3/395: write d8/db/d4c/f5c [2146394,17691] 0 2024-05-06T12:14:35.268 INFO:tasks.workunit.client.0.smithi092.stdout:9/372: dread d16/d2c/d3f/f46 [0,4194304] 0 2024-05-06T12:14:35.268 INFO:tasks.workunit.client.0.smithi092.stdout:4/517: fdatasync d9/d27/f9f 0 2024-05-06T12:14:35.269 INFO:tasks.workunit.client.0.smithi092.stdout:9/373: write f6 [4997604,96375] 0 2024-05-06T12:14:35.272 INFO:tasks.workunit.client.0.smithi092.stdout:4/518: mkdir d9/dd/da6/dad 0 2024-05-06T12:14:35.274 INFO:tasks.workunit.client.0.smithi092.stdout:3/396: rename d8/db/d4c/f5c to d8/db/d14/d5a/f7b 0 2024-05-06T12:14:35.275 INFO:tasks.workunit.client.0.smithi092.stdout:3/397: chown d8/db 15 1 2024-05-06T12:14:35.275 INFO:tasks.workunit.client.0.smithi092.stdout:3/398: write f3 [568116,77882] 0 2024-05-06T12:14:35.277 INFO:tasks.workunit.client.0.smithi092.stdout:4/519: fsync d9/d27/f31 0 2024-05-06T12:14:35.279 INFO:tasks.workunit.client.0.smithi092.stdout:8/500: dread d9/d21/d54/f63 [0,4194304] 0 2024-05-06T12:14:35.281 INFO:tasks.workunit.client.0.smithi092.stdout:3/399: symlink d8/db/d4c/l7c 0 2024-05-06T12:14:35.282 INFO:tasks.workunit.client.0.smithi092.stdout:4/520: mknod d9/cae 0 2024-05-06T12:14:35.283 INFO:tasks.workunit.client.0.smithi092.stdout:4/521: dread - d9/d27/d4a/f75 zero size 2024-05-06T12:14:35.284 INFO:tasks.workunit.client.0.smithi092.stdout:3/400: mknod d8/d24/d33/d4b/c7d 0 2024-05-06T12:14:35.286 INFO:tasks.workunit.client.0.smithi092.stdout:8/501: symlink d9/d21/d37/d46/d5c/d62/d7a/lba 0 2024-05-06T12:14:35.288 INFO:tasks.workunit.client.0.smithi092.stdout:4/522: rmdir d9/d27/d4a/d9c 0 2024-05-06T12:14:35.290 INFO:tasks.workunit.client.0.smithi092.stdout:4/523: mkdir d9/d2b/d5b/daf 0 2024-05-06T12:14:35.293 INFO:tasks.workunit.client.0.smithi092.stdout:4/524: truncate d9/dd/f19 2686624 0 2024-05-06T12:14:35.295 INFO:tasks.workunit.client.0.smithi092.stdout:9/374: dwrite d16/d2c/d2f/d3d/d33/f56 [0,4194304] 0 2024-05-06T12:14:35.296 INFO:tasks.workunit.client.0.smithi092.stdout:9/375: fsync d16/d1c/f27 0 2024-05-06T12:14:35.297 INFO:tasks.workunit.client.0.smithi092.stdout:9/376: chown d16/d2c/d2f 15 1 2024-05-06T12:14:35.297 INFO:tasks.workunit.client.0.smithi092.stdout:4/525: creat d9/df/fb0 x:0 0 0 2024-05-06T12:14:35.297 INFO:tasks.workunit.client.0.smithi092.stdout:9/377: read - d16/d1e/f70 zero size 2024-05-06T12:14:35.306 INFO:tasks.workunit.client.0.smithi092.stdout:9/378: dwrite d16/d2c/d2f/d3d/f38 [0,4194304] 0 2024-05-06T12:14:35.308 INFO:tasks.workunit.client.0.smithi092.stdout:4/526: rename c6 to d9/df/d48/cb1 0 2024-05-06T12:14:35.310 INFO:tasks.workunit.client.0.smithi092.stdout:9/379: symlink d16/d2c/d2f/d3d/l73 0 2024-05-06T12:14:35.312 INFO:tasks.workunit.client.0.smithi092.stdout:4/527: rename d9/d27/d4a/ca5 to d9/d27/da8/cb2 0 2024-05-06T12:14:35.312 INFO:tasks.workunit.client.0.smithi092.stdout:4/528: readlink d9/df/d48/l51 0 2024-05-06T12:14:35.313 INFO:tasks.workunit.client.0.smithi092.stdout:9/380: write d16/d2c/d2f/d3d/f62 [1949441,4397] 0 2024-05-06T12:14:35.315 INFO:tasks.workunit.client.0.smithi092.stdout:4/529: mkdir d9/d27/da8/db3 0 2024-05-06T12:14:35.317 INFO:tasks.workunit.client.0.smithi092.stdout:9/381: mknod d16/d2c/d2f/d3d/d33/c74 0 2024-05-06T12:14:35.319 INFO:tasks.workunit.client.0.smithi092.stdout:9/382: mkdir d16/d2c/d2f/d75 0 2024-05-06T12:14:35.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:34 smithi190 ceph-mon[33798]: pgmap v19: 65 pgs: 65 active+clean; 961 MiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 20 MiB/s rd, 115 MiB/s wr, 513 op/s 2024-05-06T12:14:35.348 INFO:tasks.workunit.client.0.smithi092.stdout:0/579: sync 2024-05-06T12:14:35.350 INFO:tasks.workunit.client.0.smithi092.stdout:4/530: dwrite d9/d2b/f8f [0,4194304] 0 2024-05-06T12:14:35.351 INFO:tasks.workunit.client.0.smithi092.stdout:4/531: fsync d9/d2b/f77 0 2024-05-06T12:14:35.352 INFO:tasks.workunit.client.0.smithi092.stdout:9/383: mkdir d16/d2c/d4b/d76 0 2024-05-06T12:14:35.353 INFO:tasks.workunit.client.0.smithi092.stdout:0/580: readlink d7/dc/d6f/da5/lb6 0 2024-05-06T12:14:35.353 INFO:tasks.workunit.client.0.smithi092.stdout:0/581: fsync d7/dc/d28/d54/f3b 0 2024-05-06T12:14:35.354 INFO:tasks.workunit.client.0.smithi092.stdout:4/532: chown d9/df/d62/f68 18 1 2024-05-06T12:14:35.358 INFO:tasks.workunit.client.0.smithi092.stdout:0/582: mknod d7/dc/d28/d54/d1e/d5d/cb8 0 2024-05-06T12:14:35.360 INFO:tasks.workunit.client.0.smithi092.stdout:0/583: write d7/dc/f9b [5031611,20454] 0 2024-05-06T12:14:35.363 INFO:tasks.workunit.client.0.smithi092.stdout:4/533: dwrite d9/d2b/d5b/fa9 [0,4194304] 0 2024-05-06T12:14:35.364 INFO:tasks.workunit.client.0.smithi092.stdout:4/534: chown d9/df/d48/l51 921 1 2024-05-06T12:14:35.366 INFO:tasks.workunit.client.0.smithi092.stdout:0/584: rename d7/dc/d28/d54/d1e/f22 to d7/dc/d28/d54/d1e/d46/d63/fb9 0 2024-05-06T12:14:35.374 INFO:tasks.workunit.client.0.smithi092.stdout:9/384: dwrite d16/d1e/f28 [0,4194304] 0 2024-05-06T12:14:35.378 INFO:tasks.workunit.client.0.smithi092.stdout:9/385: getdents d16/d2c/d2f/d3d/d33 0 2024-05-06T12:14:35.379 INFO:tasks.workunit.client.0.smithi092.stdout:9/386: write f7 [4721095,49802] 0 2024-05-06T12:14:35.380 INFO:tasks.workunit.client.0.smithi092.stdout:9/387: truncate d16/d1e/f70 108054 0 2024-05-06T12:14:35.384 INFO:tasks.workunit.client.0.smithi092.stdout:9/388: creat d16/d2c/d2f/d3d/d33/d6b/f77 x:0 0 0 2024-05-06T12:14:35.385 INFO:tasks.workunit.client.0.smithi092.stdout:0/585: dwrite d7/dc/d28/d54/d1e/d39/f57 [0,4194304] 0 2024-05-06T12:14:35.387 INFO:tasks.workunit.client.0.smithi092.stdout:9/389: write d16/d1e/f28 [1641570,125603] 0 2024-05-06T12:14:35.388 INFO:tasks.workunit.client.0.smithi092.stdout:0/586: read d7/dc/f5c [86664,38766] 0 2024-05-06T12:14:35.389 INFO:tasks.workunit.client.0.smithi092.stdout:9/390: mkdir d16/d2c/d2f/d3d/d33/d37/d40/d78 0 2024-05-06T12:14:35.392 INFO:tasks.workunit.client.0.smithi092.stdout:9/391: chown d16/d2c/d2f/d3d/d33/d37/d40/d43/f51 90 1 2024-05-06T12:14:35.394 INFO:tasks.workunit.client.0.smithi092.stdout:0/587: dwrite d7/dc/d9e/f40 [0,4194304] 0 2024-05-06T12:14:35.395 INFO:tasks.workunit.client.0.smithi092.stdout:9/392: truncate d16/d2c/d2f/d3d/d33/f66 8519021 0 2024-05-06T12:14:35.398 INFO:tasks.workunit.client.0.smithi092.stdout:9/393: creat d16/d2c/d2f/d75/f79 x:0 0 0 2024-05-06T12:14:35.405 INFO:tasks.workunit.client.0.smithi092.stdout:0/588: dwrite d7/dc/d28/d54/d1e/d5d/f76 [0,4194304] 0 2024-05-06T12:14:35.406 INFO:tasks.workunit.client.0.smithi092.stdout:9/394: dwrite d16/d1e/f28 [0,4194304] 0 2024-05-06T12:14:35.409 INFO:tasks.workunit.client.0.smithi092.stdout:9/395: symlink d16/d2c/d2f/d3d/d33/d57/l7a 0 2024-05-06T12:14:35.410 INFO:tasks.workunit.client.0.smithi092.stdout:0/589: creat d7/dc/d28/d54/d1e/d26/d96/fba x:0 0 0 2024-05-06T12:14:35.411 INFO:tasks.workunit.client.0.smithi092.stdout:9/396: readlink l15 0 2024-05-06T12:14:35.411 INFO:tasks.workunit.client.0.smithi092.stdout:9/397: readlink d16/d2c/d3f/l4a 0 2024-05-06T12:14:35.413 INFO:tasks.workunit.client.0.smithi092.stdout:0/590: creat d7/dc/d28/d54/d1e/d26/d4e/fbb x:0 0 0 2024-05-06T12:14:35.415 INFO:tasks.workunit.client.0.smithi092.stdout:9/398: mkdir d16/d1e/d7b 0 2024-05-06T12:14:35.417 INFO:tasks.workunit.client.0.smithi092.stdout:9/399: fsync d16/d2c/d2f/d3d/d33/d37/d40/d43/f50 0 2024-05-06T12:14:35.422 INFO:tasks.workunit.client.0.smithi092.stdout:0/591: dread d7/dc/d9e/f2c [0,4194304] 0 2024-05-06T12:14:35.423 INFO:tasks.workunit.client.0.smithi092.stdout:9/400: dwrite fc [4194304,4194304] 0 2024-05-06T12:14:35.424 INFO:tasks.workunit.client.0.smithi092.stdout:9/401: dread - d16/d1c/f27 zero size 2024-05-06T12:14:35.427 INFO:tasks.workunit.client.0.smithi092.stdout:9/402: creat d16/d1c/f7c x:0 0 0 2024-05-06T12:14:35.430 INFO:tasks.workunit.client.0.smithi092.stdout:9/403: creat d16/d2c/d2f/d3d/f7d x:0 0 0 2024-05-06T12:14:35.434 INFO:tasks.workunit.client.0.smithi092.stdout:0/592: dread d7/dc/d9e/f1c [0,4194304] 0 2024-05-06T12:14:35.436 INFO:tasks.workunit.client.0.smithi092.stdout:0/593: mkdir d7/dc/d28/d54/d12/d7f/db2/dbc 0 2024-05-06T12:14:35.438 INFO:tasks.workunit.client.0.smithi092.stdout:0/594: mknod d7/dc/d28/d54/d1e/cbd 0 2024-05-06T12:14:35.439 INFO:tasks.workunit.client.0.smithi092.stdout:0/595: chown d7/dc/d28/d54/d1e/d5d/c62 1345791 1 2024-05-06T12:14:35.440 INFO:tasks.workunit.client.0.smithi092.stdout:9/404: truncate f7 167248 0 2024-05-06T12:14:35.441 INFO:tasks.workunit.client.0.smithi092.stdout:9/405: chown d16/d2c/d4b/l5c 27 1 2024-05-06T12:14:35.442 INFO:tasks.workunit.client.0.smithi092.stdout:0/596: creat d7/dc/d28/d54/d12/d8f/fbe x:0 0 0 2024-05-06T12:14:35.444 INFO:tasks.workunit.client.0.smithi092.stdout:0/597: mknod d7/dc/d28/d54/d1e/d46/d63/d80/cbf 0 2024-05-06T12:14:35.447 INFO:tasks.workunit.client.0.smithi092.stdout:9/406: dwrite d16/d2c/d3f/f5e [0,4194304] 0 2024-05-06T12:14:35.449 INFO:tasks.workunit.client.0.smithi092.stdout:9/407: mkdir d16/d2c/d2f/d3d/d33/d7e 0 2024-05-06T12:14:35.450 INFO:tasks.workunit.client.0.smithi092.stdout:9/408: mknod d16/d2c/d2f/c7f 0 2024-05-06T12:14:35.451 INFO:tasks.workunit.client.0.smithi092.stdout:9/409: creat d16/d2c/f80 x:0 0 0 2024-05-06T12:14:35.461 INFO:tasks.workunit.client.0.smithi092.stdout:0/598: dread d7/dc/d28/d54/d1e/d26/d3a/f52 [0,4194304] 0 2024-05-06T12:14:35.465 INFO:tasks.workunit.client.0.smithi092.stdout:9/410: creat d16/d2c/d2f/d3d/d33/d6b/f81 x:0 0 0 2024-05-06T12:14:35.468 INFO:tasks.workunit.client.0.smithi092.stdout:0/599: dwrite d7/dc/d28/d54/d12/d8f/fbe [0,4194304] 0 2024-05-06T12:14:35.470 INFO:tasks.workunit.client.0.smithi092.stdout:9/411: unlink d16/d1c/f27 0 2024-05-06T12:14:35.471 INFO:tasks.workunit.client.0.smithi092.stdout:9/412: chown d16/f1f 6 1 2024-05-06T12:14:35.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:34 smithi092 ceph-mon[26954]: pgmap v19: 65 pgs: 65 active+clean; 961 MiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 20 MiB/s rd, 115 MiB/s wr, 513 op/s 2024-05-06T12:14:35.477 INFO:tasks.workunit.client.0.smithi092.stdout:6/551: sync 2024-05-06T12:14:35.478 INFO:tasks.workunit.client.0.smithi092.stdout:9/413: rename d16/d2c/d2f/d75/f79 to d16/d2c/d2f/d3d/d33/d57/f82 0 2024-05-06T12:14:35.478 INFO:tasks.workunit.client.0.smithi092.stdout:9/414: readlink d16/d2c/d2f/d3d/d33/d57/l7a 0 2024-05-06T12:14:35.480 INFO:tasks.workunit.client.0.smithi092.stdout:6/552: getdents d4/d24/d26/dae 0 2024-05-06T12:14:35.480 INFO:tasks.workunit.client.0.smithi092.stdout:0/600: truncate d7/dc/d28/d54/d1e/d26/f36 1418471 0 2024-05-06T12:14:35.481 INFO:tasks.workunit.client.0.smithi092.stdout:6/553: truncate d4/d20/d22/f7a 1441351 0 2024-05-06T12:14:35.482 INFO:tasks.workunit.client.0.smithi092.stdout:9/415: stat d16/d2c/d2f/d3d/d33/d37 0 2024-05-06T12:14:35.482 INFO:tasks.workunit.client.0.smithi092.stdout:6/554: readlink d4/d4c/d52/d62/l8d 0 2024-05-06T12:14:35.482 INFO:tasks.workunit.client.0.smithi092.stdout:0/601: creat d7/dc/d9e/fc0 x:0 0 0 2024-05-06T12:14:35.484 INFO:tasks.workunit.client.0.smithi092.stdout:6/555: write d4/d24/d38/f69 [396301,110584] 0 2024-05-06T12:14:35.486 INFO:tasks.workunit.client.0.smithi092.stdout:6/556: mknod d4/d20/d22/cb7 0 2024-05-06T12:14:35.487 INFO:tasks.workunit.client.0.smithi092.stdout:0/602: dread d7/dc/f9c [0,4194304] 0 2024-05-06T12:14:35.488 INFO:tasks.workunit.client.0.smithi092.stdout:0/603: stat d7/dc/d28/d64 0 2024-05-06T12:14:35.488 INFO:tasks.workunit.client.0.smithi092.stdout:6/557: read d4/d24/d26/fa4 [65191,66559] 0 2024-05-06T12:14:35.489 INFO:tasks.workunit.client.0.smithi092.stdout:6/558: read - d4/d4c/f5c zero size 2024-05-06T12:14:35.491 INFO:tasks.workunit.client.0.smithi092.stdout:0/604: fdatasync d7/dc/d9e/f1c 0 2024-05-06T12:14:35.493 INFO:tasks.workunit.client.0.smithi092.stdout:9/416: truncate d16/d1e/f42 5269024 0 2024-05-06T12:14:35.494 INFO:tasks.workunit.client.0.smithi092.stdout:0/605: symlink d7/dc/d28/lc1 0 2024-05-06T12:14:35.496 INFO:tasks.workunit.client.0.smithi092.stdout:6/559: dwrite d4/fb [4194304,4194304] 0 2024-05-06T12:14:35.498 INFO:tasks.workunit.client.0.smithi092.stdout:6/560: write d4/d24/f9c [219907,158] 0 2024-05-06T12:14:35.499 INFO:tasks.workunit.client.0.smithi092.stdout:0/606: rename d7/fa6 to d7/dc/d6f/fc2 0 2024-05-06T12:14:35.500 INFO:tasks.workunit.client.0.smithi092.stdout:0/607: readlink d7/dc/l9f 0 2024-05-06T12:14:35.504 INFO:tasks.workunit.client.0.smithi092.stdout:9/417: dwrite d16/d2c/f35 [4194304,4194304] 0 2024-05-06T12:14:35.504 INFO:tasks.workunit.client.0.smithi092.stdout:0/608: chown d7/dc/d28/d54/d1e/c25 33674 1 2024-05-06T12:14:35.511 INFO:tasks.workunit.client.0.smithi092.stdout:9/418: creat d16/d2c/d4b/d76/f83 x:0 0 0 2024-05-06T12:14:35.514 INFO:tasks.workunit.client.0.smithi092.stdout:9/419: fsync d16/f29 0 2024-05-06T12:14:35.517 INFO:tasks.workunit.client.0.smithi092.stdout:9/420: chown d16/d2c/d2f/d3d/f62 35 1 2024-05-06T12:14:35.517 INFO:tasks.workunit.client.0.smithi092.stdout:9/421: fdatasync d16/d2c/d2f/d3d/f61 0 2024-05-06T12:14:35.518 INFO:tasks.workunit.client.0.smithi092.stdout:9/422: write d16/d2c/d2f/d3d/d33/d6b/f81 [109315,38704] 0 2024-05-06T12:14:35.520 INFO:tasks.workunit.client.0.smithi092.stdout:9/423: read d16/d2c/d2f/d3d/d33/d37/f6f [87324,29803] 0 2024-05-06T12:14:35.527 INFO:tasks.workunit.client.0.smithi092.stdout:9/424: dwrite d16/d2c/d2f/f45 [0,4194304] 0 2024-05-06T12:14:35.527 INFO:tasks.workunit.client.0.smithi092.stdout:9/425: readlink l15 0 2024-05-06T12:14:35.529 INFO:tasks.workunit.client.0.smithi092.stdout:6/561: dread d4/d20/f83 [0,4194304] 0 2024-05-06T12:14:35.532 INFO:tasks.workunit.client.0.smithi092.stdout:0/609: dwrite d7/dc/d28/d54/d1e/d26/f36 [0,4194304] 0 2024-05-06T12:14:35.533 INFO:tasks.workunit.client.0.smithi092.stdout:9/426: truncate d16/d2c/d2f/d3d/d33/d37/f6f 1178026 0 2024-05-06T12:14:35.536 INFO:tasks.workunit.client.0.smithi092.stdout:0/610: creat d7/dc/d28/d54/d1e/d26/d96/fc3 x:0 0 0 2024-05-06T12:14:35.537 INFO:tasks.workunit.client.0.smithi092.stdout:9/427: symlink d16/d2c/d2f/d3d/d33/d6b/l84 0 2024-05-06T12:14:35.539 INFO:tasks.workunit.client.0.smithi092.stdout:9/428: dread - d16/d2c/f65 zero size 2024-05-06T12:14:35.541 INFO:tasks.workunit.client.0.smithi092.stdout:9/429: symlink d16/d2c/d2f/d75/l85 0 2024-05-06T12:14:35.542 INFO:tasks.workunit.client.0.smithi092.stdout:9/430: creat d16/d1e/f86 x:0 0 0 2024-05-06T12:14:35.545 INFO:tasks.workunit.client.0.smithi092.stdout:9/431: creat d16/d2c/d2f/d3d/d33/f87 x:0 0 0 2024-05-06T12:14:35.546 INFO:tasks.workunit.client.0.smithi092.stdout:9/432: write d16/f17 [2859110,109533] 0 2024-05-06T12:14:35.548 INFO:tasks.workunit.client.0.smithi092.stdout:9/433: write d16/d2c/f80 [925093,110380] 0 2024-05-06T12:14:35.549 INFO:tasks.workunit.client.0.smithi092.stdout:9/434: chown d16/d2c/d2f/d3d/l73 22 1 2024-05-06T12:14:35.550 INFO:tasks.workunit.client.0.smithi092.stdout:9/435: dread d16/d2c/d3f/f46 [0,4194304] 0 2024-05-06T12:14:35.551 INFO:tasks.workunit.client.0.smithi092.stdout:0/611: dread d7/dc/d28/d64/f78 [4194304,4194304] 0 2024-05-06T12:14:35.558 INFO:tasks.workunit.client.0.smithi092.stdout:6/562: dread d4/d20/f1c [0,4194304] 0 2024-05-06T12:14:35.560 INFO:tasks.workunit.client.0.smithi092.stdout:6/563: fdatasync d4/d20/d22/f7c 0 2024-05-06T12:14:35.561 INFO:tasks.workunit.client.0.smithi092.stdout:6/564: fsync d4/d20/d22/f7a 0 2024-05-06T12:14:35.596 INFO:tasks.workunit.client.0.smithi092.stdout:5/459: sync 2024-05-06T12:14:35.599 INFO:tasks.workunit.client.0.smithi092.stdout:0/612: dread d7/dc/d28/d54/d1e/d46/d63/fb9 [0,4194304] 0 2024-05-06T12:14:35.600 INFO:tasks.workunit.client.0.smithi092.stdout:5/460: truncate d4/d34/d28/d4e/d5d/d82/d1f/d20/f3e 728259 0 2024-05-06T12:14:35.602 INFO:tasks.workunit.client.0.smithi092.stdout:0/613: symlink d7/dc/d28/d64/lc4 0 2024-05-06T12:14:35.604 INFO:tasks.workunit.client.0.smithi092.stdout:5/461: rename d4/d34/d28/d4e/d5d/d82/d1f/d20/d99 to d4/d34/dae 0 2024-05-06T12:14:35.606 INFO:tasks.workunit.client.0.smithi092.stdout:0/614: dread d7/dc/d28/d54/d1e/d46/d63/fb9 [0,4194304] 0 2024-05-06T12:14:35.607 INFO:tasks.workunit.client.0.smithi092.stdout:5/462: truncate d4/d34/d29/d54/f62 1442457 0 2024-05-06T12:14:35.608 INFO:tasks.workunit.client.0.smithi092.stdout:0/615: getdents d7/dc/d28/d54/d12/d7f/db5 0 2024-05-06T12:14:35.609 INFO:tasks.workunit.client.0.smithi092.stdout:0/616: fdatasync f1 0 2024-05-06T12:14:35.610 INFO:tasks.workunit.client.0.smithi092.stdout:0/617: read - d7/dc/d9e/fc0 zero size 2024-05-06T12:14:35.616 INFO:tasks.workunit.client.0.smithi092.stdout:0/618: truncate d7/f9 1668576 0 2024-05-06T12:14:35.617 INFO:tasks.workunit.client.0.smithi092.stdout:0/619: mkdir d7/dc/d28/d54/d1e/d39/da8/dc5 0 2024-05-06T12:14:35.619 INFO:tasks.workunit.client.0.smithi092.stdout:0/620: mkdir d7/dc/d6f/da5/dc6 0 2024-05-06T12:14:35.620 INFO:tasks.workunit.client.0.smithi092.stdout:0/621: chown d7/dc/d28/d54/d1e/d39/f57 0 1 2024-05-06T12:14:35.621 INFO:tasks.workunit.client.0.smithi092.stdout:0/622: mknod d7/dc/d97/cc7 0 2024-05-06T12:14:35.623 INFO:tasks.workunit.client.0.smithi092.stdout:0/623: creat d7/dc/d28/d54/d12/d7f/db2/dbc/fc8 x:0 0 0 2024-05-06T12:14:35.623 INFO:tasks.workunit.client.0.smithi092.stdout:0/624: readlink d7/l59 0 2024-05-06T12:14:35.630 INFO:tasks.workunit.client.0.smithi092.stdout:0/625: dread f6 [0,4194304] 0 2024-05-06T12:14:35.631 INFO:tasks.workunit.client.0.smithi092.stdout:0/626: fdatasync d7/dc/f30 0 2024-05-06T12:14:35.719 INFO:tasks.workunit.client.0.smithi092.stdout:3/401: sync 2024-05-06T12:14:35.719 INFO:tasks.workunit.client.0.smithi092.stdout:8/502: sync 2024-05-06T12:14:35.719 INFO:tasks.workunit.client.0.smithi092.stdout:1/432: sync 2024-05-06T12:14:35.719 INFO:tasks.workunit.client.0.smithi092.stdout:2/320: sync 2024-05-06T12:14:35.720 INFO:tasks.workunit.client.0.smithi092.stdout:4/535: sync 2024-05-06T12:14:35.720 INFO:tasks.workunit.client.0.smithi092.stdout:7/605: sync 2024-05-06T12:14:35.720 INFO:tasks.workunit.client.0.smithi092.stdout:2/321: chown df/d17/d1d 2140 1 2024-05-06T12:14:35.721 INFO:tasks.workunit.client.0.smithi092.stdout:3/402: write d8/d24/f6f [963279,123741] 0 2024-05-06T12:14:35.722 INFO:tasks.workunit.client.0.smithi092.stdout:4/536: fdatasync d9/df/f15 0 2024-05-06T12:14:35.723 INFO:tasks.workunit.client.0.smithi092.stdout:3/403: readlink d8/db/d4c/l7c 0 2024-05-06T12:14:35.724 INFO:tasks.workunit.client.0.smithi092.stdout:2/322: creat df/f61 x:0 0 0 2024-05-06T12:14:35.728 INFO:tasks.workunit.client.0.smithi092.stdout:1/433: rename d3/d10/d13/f3f to d3/d42/d84/d4f/d58/d65/d81/f8a 0 2024-05-06T12:14:35.734 INFO:tasks.workunit.client.0.smithi092.stdout:4/537: mkdir d9/df/d62/d7b/db4 0 2024-05-06T12:14:35.737 INFO:tasks.workunit.client.0.smithi092.stdout:1/434: getdents d3/d42/d84/d4f/d7f 0 2024-05-06T12:14:35.738 INFO:tasks.workunit.client.0.smithi092.stdout:2/323: dread df/d17/f2a [0,4194304] 0 2024-05-06T12:14:35.739 INFO:tasks.workunit.client.0.smithi092.stdout:4/538: rename d9/d27/f9f to d9/d2b/d5b/fb5 0 2024-05-06T12:14:35.739 INFO:tasks.workunit.client.0.smithi092.stdout:8/503: link d9/d21/f58 d9/dc/fbb 0 2024-05-06T12:14:35.740 INFO:tasks.workunit.client.0.smithi092.stdout:2/324: dread df/f5b [0,4194304] 0 2024-05-06T12:14:35.741 INFO:tasks.workunit.client.0.smithi092.stdout:4/539: truncate d9/d2b/d7c/f95 142246 0 2024-05-06T12:14:35.741 INFO:tasks.workunit.client.0.smithi092.stdout:2/325: dread - df/d15/f4c zero size 2024-05-06T12:14:35.742 INFO:tasks.workunit.client.0.smithi092.stdout:1/435: mkdir d3/d10/d13/d19/d21/d88/d8b 0 2024-05-06T12:14:35.743 INFO:tasks.workunit.client.0.smithi092.stdout:3/404: dread d8/d24/f77 [0,4194304] 0 2024-05-06T12:14:35.744 INFO:tasks.workunit.client.0.smithi092.stdout:8/504: mknod d9/d21/d37/d46/d5c/d62/cbc 0 2024-05-06T12:14:35.745 INFO:tasks.workunit.client.0.smithi092.stdout:4/540: rmdir d9/d2b/d7c 39 2024-05-06T12:14:35.747 INFO:tasks.workunit.client.0.smithi092.stdout:1/436: rename d3/f20 to d3/d10/d13/d19/d21/d88/d8b/f8c 0 2024-05-06T12:14:35.749 INFO:tasks.workunit.client.0.smithi092.stdout:3/405: symlink d8/db/d4c/d40/d67/l7e 0 2024-05-06T12:14:35.750 INFO:tasks.workunit.client.0.smithi092.stdout:7/606: truncate d9/de/d2d/fad 1834759 0 2024-05-06T12:14:35.772 INFO:tasks.workunit.client.1.smithi190.stderr:++ readlink -f fsstress 2024-05-06T12:14:35.773 INFO:tasks.workunit.client.1.smithi190.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-05-06T12:14:35.774 INFO:tasks.workunit.client.1.smithi190.stderr:+ popd 2024-05-06T12:14:35.774 INFO:tasks.workunit.client.1.smithi190.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-05-06T12:14:35.775 INFO:tasks.workunit.client.1.smithi190.stderr:+ popd 2024-05-06T12:14:35.775 INFO:tasks.workunit.client.1.smithi190.stdout:~/cephtest/mnt.1/client.1/tmp 2024-05-06T12:14:35.775 INFO:tasks.workunit.client.1.smithi190.stderr:++ mktemp -d -p . 2024-05-06T12:14:35.778 INFO:tasks.workunit.client.1.smithi190.stderr:+ T=./tmp.n2MO3EjBDS 2024-05-06T12:14:35.778 INFO:tasks.workunit.client.1.smithi190.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.n2MO3EjBDS -l 1 -n 1000 -p 10 -v 2024-05-06T12:14:35.781 INFO:tasks.workunit.client.0.smithi092.stdout:1/437: mkdir d3/d42/d84/d4f/d58/d65/d81/d8d 0 2024-05-06T12:14:35.781 INFO:tasks.workunit.client.0.smithi092.stdout:3/406: mknod d8/db/d14/c7f 0 2024-05-06T12:14:35.782 INFO:tasks.workunit.client.1.smithi190.stdout:seed = 1714627512 2024-05-06T12:14:35.782 INFO:tasks.workunit.client.0.smithi092.stdout:1/438: chown d3/d42/d84/d4f 0 1 2024-05-06T12:14:35.783 INFO:tasks.workunit.client.0.smithi092.stdout:7/607: rmdir d9/de/da4 39 2024-05-06T12:14:35.785 INFO:tasks.workunit.client.0.smithi092.stdout:1/439: write d3/d42/d84/f5f [4294300,66986] 0 2024-05-06T12:14:35.786 INFO:tasks.workunit.client.1.smithi190.stdout:0/0: fsync - no filename 2024-05-06T12:14:35.786 INFO:tasks.workunit.client.0.smithi092.stdout:3/407: creat d8/db/d4c/d40/d67/f80 x:0 0 0 2024-05-06T12:14:35.787 INFO:tasks.workunit.client.1.smithi190.stdout:0/1: creat f0 x:0 0 0 2024-05-06T12:14:35.788 INFO:tasks.workunit.client.1.smithi190.stdout:0/2: chown f0 23890 1 2024-05-06T12:14:35.789 INFO:tasks.workunit.client.1.smithi190.stdout:2/0: truncate - no filename 2024-05-06T12:14:35.789 INFO:tasks.workunit.client.1.smithi190.stdout:2/1: write - no filename 2024-05-06T12:14:35.789 INFO:tasks.workunit.client.1.smithi190.stdout:2/2: dwrite - no filename 2024-05-06T12:14:35.789 INFO:tasks.workunit.client.1.smithi190.stdout:2/3: chown . 73697494 1 2024-05-06T12:14:35.790 INFO:tasks.workunit.client.1.smithi190.stdout:2/4: fsync - no filename 2024-05-06T12:14:35.790 INFO:tasks.workunit.client.1.smithi190.stdout:2/5: fsync - no filename 2024-05-06T12:14:35.790 INFO:tasks.workunit.client.1.smithi190.stdout:2/6: write - no filename 2024-05-06T12:14:35.790 INFO:tasks.workunit.client.1.smithi190.stdout:2/7: dwrite - no filename 2024-05-06T12:14:35.790 INFO:tasks.workunit.client.1.smithi190.stdout:2/8: truncate - no filename 2024-05-06T12:14:35.790 INFO:tasks.workunit.client.1.smithi190.stdout:2/9: chown . 314113 1 2024-05-06T12:14:35.791 INFO:tasks.workunit.client.0.smithi092.stdout:3/408: read d8/d24/d33/d4b/f56 [1370865,108628] 0 2024-05-06T12:14:35.792 INFO:tasks.workunit.client.1.smithi190.stdout:0/3: unlink f0 0 2024-05-06T12:14:35.792 INFO:tasks.workunit.client.1.smithi190.stdout:0/4: unlink - no file 2024-05-06T12:14:35.792 INFO:tasks.workunit.client.1.smithi190.stdout:0/5: rename - no filename 2024-05-06T12:14:35.792 INFO:tasks.workunit.client.1.smithi190.stdout:0/6: dread - no filename 2024-05-06T12:14:35.792 INFO:tasks.workunit.client.1.smithi190.stdout:1/0: getdents . 0 2024-05-06T12:14:35.792 INFO:tasks.workunit.client.1.smithi190.stdout:1/1: write - no filename 2024-05-06T12:14:35.792 INFO:tasks.workunit.client.1.smithi190.stdout:1/2: write - no filename 2024-05-06T12:14:35.792 INFO:tasks.workunit.client.1.smithi190.stdout:3/0: write - no filename 2024-05-06T12:14:35.793 INFO:tasks.workunit.client.1.smithi190.stdout:3/1: rename - no filename 2024-05-06T12:14:35.793 INFO:tasks.workunit.client.1.smithi190.stdout:0/7: mknod c1 0 2024-05-06T12:14:35.793 INFO:tasks.workunit.client.0.smithi092.stdout:4/541: dread f5 [0,4194304] 0 2024-05-06T12:14:35.793 INFO:tasks.workunit.client.0.smithi092.stdout:7/608: dwrite d9/da5/f14 [0,4194304] 0 2024-05-06T12:14:35.794 INFO:tasks.workunit.client.0.smithi092.stdout:1/440: rename d3/f3e to d3/d42/d84/f8e 0 2024-05-06T12:14:35.795 INFO:tasks.workunit.client.1.smithi190.stdout:2/10: symlink l0 0 2024-05-06T12:14:35.796 INFO:tasks.workunit.client.1.smithi190.stdout:1/3: creat f0 x:0 0 0 2024-05-06T12:14:35.796 INFO:tasks.workunit.client.0.smithi092.stdout:3/409: dread d8/db/d14/f50 [0,4194304] 0 2024-05-06T12:14:35.796 INFO:tasks.workunit.client.0.smithi092.stdout:1/441: write d3/d10/d13/d19/d21/f77 [49828,90536] 0 2024-05-06T12:14:35.797 INFO:tasks.workunit.client.1.smithi190.stdout:0/8: mknod c2 0 2024-05-06T12:14:35.797 INFO:tasks.workunit.client.1.smithi190.stdout:0/9: dread - no filename 2024-05-06T12:14:35.797 INFO:tasks.workunit.client.1.smithi190.stdout:0/10: dread - no filename 2024-05-06T12:14:35.797 INFO:tasks.workunit.client.1.smithi190.stdout:0/11: truncate - no filename 2024-05-06T12:14:35.797 INFO:tasks.workunit.client.1.smithi190.stdout:0/12: dread - no filename 2024-05-06T12:14:35.797 INFO:tasks.workunit.client.1.smithi190.stdout:0/13: readlink - no filename 2024-05-06T12:14:35.798 INFO:tasks.workunit.client.1.smithi190.stdout:0/14: dread - no filename 2024-05-06T12:14:35.799 INFO:tasks.workunit.client.0.smithi092.stdout:3/410: write d8/d24/f6f [525083,110193] 0 2024-05-06T12:14:35.799 INFO:tasks.workunit.client.1.smithi190.stdout:3/2: mkdir d0 0 2024-05-06T12:14:35.800 INFO:tasks.workunit.client.0.smithi092.stdout:3/411: readlink d8/db/l2d 0 2024-05-06T12:14:35.800 INFO:tasks.workunit.client.1.smithi190.stdout:2/11: symlink l1 0 2024-05-06T12:14:35.800 INFO:tasks.workunit.client.1.smithi190.stdout:2/12: fdatasync - no filename 2024-05-06T12:14:35.801 INFO:tasks.workunit.client.1.smithi190.stdout:0/15: creat f3 x:0 0 0 2024-05-06T12:14:35.802 INFO:tasks.workunit.client.1.smithi190.stdout:6/0: dwrite - no filename 2024-05-06T12:14:35.803 INFO:tasks.workunit.client.1.smithi190.stdout:4/0: creat f0 x:0 0 0 2024-05-06T12:14:35.804 INFO:tasks.workunit.client.0.smithi092.stdout:4/542: mkdir d9/df/d62/d7b/db6 0 2024-05-06T12:14:35.804 INFO:tasks.workunit.client.0.smithi092.stdout:8/505: truncate d9/dc/f2d 3063742 0 2024-05-06T12:14:35.804 INFO:tasks.workunit.client.1.smithi190.stdout:2/13: mkdir d2 0 2024-05-06T12:14:35.805 INFO:tasks.workunit.client.1.smithi190.stdout:2/14: dwrite - no filename 2024-05-06T12:14:35.806 INFO:tasks.workunit.client.1.smithi190.stdout:0/16: mknod c4 0 2024-05-06T12:14:35.806 INFO:tasks.workunit.client.1.smithi190.stdout:5/0: creat f0 x:0 0 0 2024-05-06T12:14:35.809 INFO:tasks.workunit.client.1.smithi190.stdout:7/0: chown . 12764438 1 2024-05-06T12:14:35.810 INFO:tasks.workunit.client.1.smithi190.stdout:4/1: unlink f0 0 2024-05-06T12:14:35.811 INFO:tasks.workunit.client.1.smithi190.stdout:4/2: read - no filename 2024-05-06T12:14:35.811 INFO:tasks.workunit.client.1.smithi190.stdout:4/3: chown . 8920628 1 2024-05-06T12:14:35.812 INFO:tasks.workunit.client.0.smithi092.stdout:4/543: mknod d9/dd/d90/cb7 0 2024-05-06T12:14:35.813 INFO:tasks.workunit.client.1.smithi190.stdout:6/1: mknod c0 0 2024-05-06T12:14:35.814 INFO:tasks.workunit.client.0.smithi092.stdout:8/506: mkdir d9/d21/d37/dbd 0 2024-05-06T12:14:35.816 INFO:tasks.workunit.client.1.smithi190.stdout:8/0: readlink - no filename 2024-05-06T12:14:35.816 INFO:tasks.workunit.client.1.smithi190.stdout:8/1: dwrite - no filename 2024-05-06T12:14:35.817 INFO:tasks.workunit.client.0.smithi092.stdout:1/442: rename d3/d42/d84/d4f/d58/d82 to d3/d8f 0 2024-05-06T12:14:35.817 INFO:tasks.workunit.client.1.smithi190.stdout:3/3: rmdir d0 0 2024-05-06T12:14:35.818 INFO:tasks.workunit.client.0.smithi092.stdout:9/436: sync 2024-05-06T12:14:35.818 INFO:tasks.workunit.client.1.smithi190.stdout:5/1: dwrite f0 [0,4194304] 0 2024-05-06T12:14:35.819 INFO:tasks.workunit.client.0.smithi092.stdout:7/609: getdents d9/de/d10 0 2024-05-06T12:14:35.819 INFO:tasks.workunit.client.1.smithi190.stdout:5/2: chown f0 22 1 2024-05-06T12:14:35.819 INFO:tasks.workunit.client.1.smithi190.stdout:0/17: link f3 f5 0 2024-05-06T12:14:35.821 INFO:tasks.workunit.client.1.smithi190.stdout:0/18: chown c4 19178294 1 2024-05-06T12:14:35.821 INFO:tasks.workunit.client.1.smithi190.stdout:4/4: creat f1 x:0 0 0 2024-05-06T12:14:35.821 INFO:tasks.workunit.client.0.smithi092.stdout:4/544: dwrite d9/d2b/d5b/f8d [0,4194304] 0 2024-05-06T12:14:35.821 INFO:tasks.workunit.client.0.smithi092.stdout:3/412: dread d8/d24/d33/f62 [4194304,4194304] 0 2024-05-06T12:14:35.823 INFO:tasks.workunit.client.1.smithi190.stdout:6/2: creat f1 x:0 0 0 2024-05-06T12:14:35.824 INFO:tasks.workunit.client.0.smithi092.stdout:7/610: write d9/de/d10/d17/d99/db6/fc5 [167737,69717] 0 2024-05-06T12:14:35.825 INFO:tasks.workunit.client.1.smithi190.stdout:7/1: mkdir d0 0 2024-05-06T12:14:35.827 INFO:tasks.workunit.client.1.smithi190.stdout:6/3: truncate f1 104161 0 2024-05-06T12:14:35.829 INFO:tasks.workunit.client.1.smithi190.stdout:3/4: symlink l1 0 2024-05-06T12:14:35.829 INFO:tasks.workunit.client.1.smithi190.stdout:9/0: chown . 7030 1 2024-05-06T12:14:35.830 INFO:tasks.workunit.client.1.smithi190.stdout:9/1: dwrite - no filename 2024-05-06T12:14:35.830 INFO:tasks.workunit.client.1.smithi190.stdout:9/2: truncate - no filename 2024-05-06T12:14:35.830 INFO:tasks.workunit.client.1.smithi190.stdout:6/4: chown c0 12 1 2024-05-06T12:14:35.830 INFO:tasks.workunit.client.1.smithi190.stdout:2/15: link l0 d2/l3 0 2024-05-06T12:14:35.832 INFO:tasks.workunit.client.1.smithi190.stdout:5/3: symlink l1 0 2024-05-06T12:14:35.835 INFO:tasks.workunit.client.0.smithi092.stdout:9/437: write d16/d2c/d2f/d3d/d33/f66 [8445803,40179] 0 2024-05-06T12:14:35.835 INFO:tasks.workunit.client.0.smithi092.stdout:1/443: dwrite d3/d10/d13/d19/f4e [0,4194304] 0 2024-05-06T12:14:35.836 INFO:tasks.workunit.client.1.smithi190.stdout:4/5: mkdir d2 0 2024-05-06T12:14:35.838 INFO:tasks.workunit.client.0.smithi092.stdout:4/545: dwrite d9/f86 [0,4194304] 0 2024-05-06T12:14:35.838 INFO:tasks.workunit.client.0.smithi092.stdout:4/546: chown d9/dd/da6 21 1 2024-05-06T12:14:35.840 INFO:tasks.workunit.client.0.smithi092.stdout:4/547: write d9/f22 [4977528,8582] 0 2024-05-06T12:14:35.841 INFO:tasks.workunit.client.1.smithi190.stdout:8/2: mknod c0 0 2024-05-06T12:14:35.841 INFO:tasks.workunit.client.1.smithi190.stdout:8/3: dwrite - no filename 2024-05-06T12:14:35.841 INFO:tasks.workunit.client.1.smithi190.stdout:8/4: write - no filename 2024-05-06T12:14:35.841 INFO:tasks.workunit.client.0.smithi092.stdout:9/438: mkdir d16/d2c/d88 0 2024-05-06T12:14:35.843 INFO:tasks.workunit.client.1.smithi190.stdout:3/5: creat f2 x:0 0 0 2024-05-06T12:14:35.844 INFO:tasks.workunit.client.0.smithi092.stdout:9/439: fsync d16/d1c/f2b 0 2024-05-06T12:14:35.844 INFO:tasks.workunit.client.1.smithi190.stdout:0/19: dwrite f5 [0,4194304] 0 2024-05-06T12:14:35.845 INFO:tasks.workunit.client.1.smithi190.stdout:3/6: write f2 [588255,118475] 0 2024-05-06T12:14:35.846 INFO:tasks.workunit.client.1.smithi190.stdout:6/5: creat f2 x:0 0 0 2024-05-06T12:14:35.847 INFO:tasks.workunit.client.0.smithi092.stdout:7/611: link d9/de/d10/d17/d21/d38/cb2 d9/de/d1f/cc7 0 2024-05-06T12:14:35.847 INFO:tasks.workunit.client.0.smithi092.stdout:9/440: readlink d16/d1c/d58/l5a 0 2024-05-06T12:14:35.848 INFO:tasks.workunit.client.0.smithi092.stdout:8/507: dread d9/f18 [0,4194304] 0 2024-05-06T12:14:35.850 INFO:tasks.workunit.client.0.smithi092.stdout:1/444: unlink d3/d10/d13/d46/l6b 0 2024-05-06T12:14:35.851 INFO:tasks.workunit.client.0.smithi092.stdout:1/445: chown d3/f5 6168509 1 2024-05-06T12:14:35.852 INFO:tasks.workunit.client.1.smithi190.stdout:5/4: mkdir d2 0 2024-05-06T12:14:35.852 INFO:tasks.workunit.client.0.smithi092.stdout:4/548: symlink d9/d2b/d65/d83/lb8 0 2024-05-06T12:14:35.854 INFO:tasks.workunit.client.1.smithi190.stdout:8/5: symlink l1 0 2024-05-06T12:14:35.854 INFO:tasks.workunit.client.1.smithi190.stdout:8/6: write - no filename 2024-05-06T12:14:35.855 INFO:tasks.workunit.client.0.smithi092.stdout:9/441: dread d16/d2c/f35 [0,4194304] 0 2024-05-06T12:14:35.856 INFO:tasks.workunit.client.1.smithi190.stdout:0/20: creat f6 x:0 0 0 2024-05-06T12:14:35.856 INFO:tasks.workunit.client.0.smithi092.stdout:4/549: chown d9/d2b/d5b/daf 12968745 1 2024-05-06T12:14:35.857 INFO:tasks.workunit.client.0.smithi092.stdout:8/508: dread d9/dc/f5d [4194304,4194304] 0 2024-05-06T12:14:35.858 INFO:tasks.workunit.client.1.smithi190.stdout:6/6: symlink l3 0 2024-05-06T12:14:35.858 INFO:tasks.workunit.client.1.smithi190.stdout:6/7: rmdir - no directory 2024-05-06T12:14:35.858 INFO:tasks.workunit.client.1.smithi190.stdout:9/3: getdents . 0 2024-05-06T12:14:35.859 INFO:tasks.workunit.client.1.smithi190.stdout:6/8: truncate f1 109896 0 2024-05-06T12:14:35.859 INFO:tasks.workunit.client.1.smithi190.stdout:2/16: mknod d2/c4 0 2024-05-06T12:14:35.859 INFO:tasks.workunit.client.1.smithi190.stdout:2/17: write - no filename 2024-05-06T12:14:35.859 INFO:tasks.workunit.client.1.smithi190.stdout:2/18: write - no filename 2024-05-06T12:14:35.861 INFO:tasks.workunit.client.0.smithi092.stdout:1/446: dwrite d3/f2c [0,4194304] 0 2024-05-06T12:14:35.861 INFO:tasks.workunit.client.1.smithi190.stdout:4/6: mknod d2/c3 0 2024-05-06T12:14:35.862 INFO:tasks.workunit.client.0.smithi092.stdout:1/447: fdatasync d3/d42/f76 0 2024-05-06T12:14:35.862 INFO:tasks.workunit.client.1.smithi190.stdout:8/7: mknod c2 0 2024-05-06T12:14:35.862 INFO:tasks.workunit.client.0.smithi092.stdout:3/413: dread d8/d24/d33/f6a [0,4194304] 0 2024-05-06T12:14:35.863 INFO:tasks.workunit.client.1.smithi190.stdout:8/8: chown l1 683627504 1 2024-05-06T12:14:35.863 INFO:tasks.workunit.client.1.smithi190.stdout:8/9: truncate - no filename 2024-05-06T12:14:35.863 INFO:tasks.workunit.client.0.smithi092.stdout:1/448: fdatasync d3/d42/d84/f50 0 2024-05-06T12:14:35.864 INFO:tasks.workunit.client.0.smithi092.stdout:8/509: creat d9/d5a/d9f/fbe x:0 0 0 2024-05-06T12:14:35.864 INFO:tasks.workunit.client.0.smithi092.stdout:7/612: dread d9/f40 [0,4194304] 0 2024-05-06T12:14:35.867 INFO:tasks.workunit.client.1.smithi190.stdout:5/5: creat d2/f3 x:0 0 0 2024-05-06T12:14:35.867 INFO:tasks.workunit.client.0.smithi092.stdout:9/442: getdents d16/d1c 0 2024-05-06T12:14:35.868 INFO:tasks.workunit.client.1.smithi190.stdout:8/10: mkdir d3 0 2024-05-06T12:14:35.868 INFO:tasks.workunit.client.1.smithi190.stdout:8/11: dread - no filename 2024-05-06T12:14:35.869 INFO:tasks.workunit.client.0.smithi092.stdout:8/510: symlink d9/d21/d72/lbf 0 2024-05-06T12:14:35.869 INFO:tasks.workunit.client.1.smithi190.stdout:9/4: creat f0 x:0 0 0 2024-05-06T12:14:35.870 INFO:tasks.workunit.client.1.smithi190.stdout:9/5: truncate f0 541129 0 2024-05-06T12:14:35.870 INFO:tasks.workunit.client.1.smithi190.stdout:9/6: readlink - no filename 2024-05-06T12:14:35.871 INFO:tasks.workunit.client.1.smithi190.stdout:5/6: mknod d2/c4 0 2024-05-06T12:14:35.871 INFO:tasks.workunit.client.1.smithi190.stdout:5/7: dread - d2/f3 zero size 2024-05-06T12:14:35.872 INFO:tasks.workunit.client.1.smithi190.stdout:9/7: write f0 [746001,62140] 0 2024-05-06T12:14:35.872 INFO:tasks.workunit.client.0.smithi092.stdout:4/550: getdents d9/d2b/d7c 0 2024-05-06T12:14:35.873 INFO:tasks.workunit.client.0.smithi092.stdout:7/613: creat d9/de/d10/d17/d99/db6/fc8 x:0 0 0 2024-05-06T12:14:35.873 INFO:tasks.workunit.client.1.smithi190.stdout:8/12: mknod d3/c4 0 2024-05-06T12:14:35.873 INFO:tasks.workunit.client.1.smithi190.stdout:8/13: dread - no filename 2024-05-06T12:14:35.873 INFO:tasks.workunit.client.1.smithi190.stdout:8/14: readlink l1 0 2024-05-06T12:14:35.876 INFO:tasks.workunit.client.1.smithi190.stdout:0/21: fdatasync f5 0 2024-05-06T12:14:35.876 INFO:tasks.workunit.client.1.smithi190.stdout:9/8: creat f1 x:0 0 0 2024-05-06T12:14:35.876 INFO:tasks.workunit.client.0.smithi092.stdout:3/414: dread d8/db/d4c/d40/f52 [4194304,4194304] 0 2024-05-06T12:14:35.876 INFO:tasks.workunit.client.0.smithi092.stdout:1/449: dwrite d3/d42/f85 [0,4194304] 0 2024-05-06T12:14:35.877 INFO:tasks.workunit.client.0.smithi092.stdout:8/511: mkdir d9/d21/dc0 0 2024-05-06T12:14:35.877 INFO:tasks.workunit.client.1.smithi190.stdout:9/9: write f1 [177914,92477] 0 2024-05-06T12:14:35.877 INFO:tasks.workunit.client.1.smithi190.stdout:8/15: rmdir d3 39 2024-05-06T12:14:35.877 INFO:tasks.workunit.client.1.smithi190.stdout:8/16: readlink l1 0 2024-05-06T12:14:35.879 INFO:tasks.workunit.client.1.smithi190.stdout:9/10: creat f2 x:0 0 0 2024-05-06T12:14:35.880 INFO:tasks.workunit.client.0.smithi092.stdout:5/463: sync 2024-05-06T12:14:35.880 INFO:tasks.workunit.client.0.smithi092.stdout:6/565: sync 2024-05-06T12:14:35.880 INFO:tasks.workunit.client.1.smithi190.stdout:3/7: dread f2 [0,4194304] 0 2024-05-06T12:14:35.881 INFO:tasks.workunit.client.1.smithi190.stdout:9/11: creat f3 x:0 0 0 2024-05-06T12:14:35.882 INFO:tasks.workunit.client.0.smithi092.stdout:7/614: truncate d9/de/d10/d17/d99/da6/fa7 897243 0 2024-05-06T12:14:35.883 INFO:tasks.workunit.client.0.smithi092.stdout:5/464: chown d4/d34/d29 444007 1 2024-05-06T12:14:35.884 INFO:tasks.workunit.client.0.smithi092.stdout:3/415: creat d8/d1b/f81 x:0 0 0 2024-05-06T12:14:35.885 INFO:tasks.workunit.client.1.smithi190.stdout:9/12: dread f0 [0,4194304] 0 2024-05-06T12:14:35.887 INFO:tasks.workunit.client.0.smithi092.stdout:9/443: dwrite d16/d2c/d2f/d3d/d33/d57/f82 [0,4194304] 0 2024-05-06T12:14:35.888 INFO:tasks.workunit.client.1.smithi190.stdout:9/13: rename f2 to f4 0 2024-05-06T12:14:35.889 INFO:tasks.workunit.client.0.smithi092.stdout:1/450: symlink d3/d42/d84/d4f/d58/l90 0 2024-05-06T12:14:35.889 INFO:tasks.workunit.client.1.smithi190.stdout:9/14: chown f0 24008 1 2024-05-06T12:14:35.890 INFO:tasks.workunit.client.0.smithi092.stdout:6/566: unlink d4/d24/d38/d8c/f94 0 2024-05-06T12:14:35.890 INFO:tasks.workunit.client.1.smithi190.stdout:9/15: creat f5 x:0 0 0 2024-05-06T12:14:35.891 INFO:tasks.workunit.client.1.smithi190.stdout:9/16: readlink - no filename 2024-05-06T12:14:35.891 INFO:tasks.workunit.client.0.smithi092.stdout:4/551: link d9/f1c d9/dd/fb9 0 2024-05-06T12:14:35.891 INFO:tasks.workunit.client.0.smithi092.stdout:7/615: read d9/de/d1f/d37/d48/d65/f66 [4078324,105991] 0 2024-05-06T12:14:35.892 INFO:tasks.workunit.client.0.smithi092.stdout:5/465: rmdir d4/d34/d28/d4e/d5d/d82/d1f/d20 39 2024-05-06T12:14:35.892 INFO:tasks.workunit.client.1.smithi190.stdout:9/17: creat f6 x:0 0 0 2024-05-06T12:14:35.893 INFO:tasks.workunit.client.0.smithi092.stdout:4/552: truncate d9/d2b/d65/d83/f91 4253752 0 2024-05-06T12:14:35.894 INFO:tasks.workunit.client.0.smithi092.stdout:4/553: fdatasync d9/d27/f50 0 2024-05-06T12:14:35.894 INFO:tasks.workunit.client.0.smithi092.stdout:8/512: rename d9/d5a/f9a to d9/dc/fc1 0 2024-05-06T12:14:35.894 INFO:tasks.workunit.client.1.smithi190.stdout:9/18: truncate f1 1207920 0 2024-05-06T12:14:35.895 INFO:tasks.workunit.client.1.smithi190.stdout:9/19: rmdir - no directory 2024-05-06T12:14:35.896 INFO:tasks.workunit.client.1.smithi190.stdout:9/20: rename f1 to f7 0 2024-05-06T12:14:35.899 INFO:tasks.workunit.client.0.smithi092.stdout:6/567: unlink d4/d20/c43 0 2024-05-06T12:14:35.899 INFO:tasks.workunit.client.1.smithi190.stdout:9/21: symlink l8 0 2024-05-06T12:14:35.899 INFO:tasks.workunit.client.0.smithi092.stdout:6/568: stat d4/d20/d22/f7c 0 2024-05-06T12:14:35.900 INFO:tasks.workunit.client.1.smithi190.stdout:9/22: mknod c9 0 2024-05-06T12:14:35.900 INFO:tasks.workunit.client.1.smithi190.stdout:9/23: readlink l8 0 2024-05-06T12:14:35.901 INFO:tasks.workunit.client.0.smithi092.stdout:1/451: creat d3/d10/d13/d19/d21/d88/f91 x:0 0 0 2024-05-06T12:14:35.901 INFO:tasks.workunit.client.1.smithi190.stdout:9/24: symlink la 0 2024-05-06T12:14:35.901 INFO:tasks.workunit.client.1.smithi190.stdout:9/25: rmdir - no directory 2024-05-06T12:14:35.902 INFO:tasks.workunit.client.0.smithi092.stdout:7/616: mknod d9/de/d1f/d37/d48/d97/cc9 0 2024-05-06T12:14:35.903 INFO:tasks.workunit.client.1.smithi190.stdout:9/26: mkdir db 0 2024-05-06T12:14:35.906 INFO:tasks.workunit.client.1.smithi190.stdout:9/27: link f7 db/fc 0 2024-05-06T12:14:35.907 INFO:tasks.workunit.client.1.smithi190.stdout:9/28: chown f3 1 1 2024-05-06T12:14:35.908 INFO:tasks.workunit.client.0.smithi092.stdout:8/513: truncate d9/d21/d37/d46/d5c/d62/f6e 4846134 0 2024-05-06T12:14:35.909 INFO:tasks.workunit.client.1.smithi190.stdout:9/29: mkdir db/dd 0 2024-05-06T12:14:35.910 INFO:tasks.workunit.client.0.smithi092.stdout:8/514: write d9/d5a/f78 [472943,112530] 0 2024-05-06T12:14:35.910 INFO:tasks.workunit.client.1.smithi190.stdout:9/30: write f3 [514384,121806] 0 2024-05-06T12:14:35.911 INFO:tasks.workunit.client.0.smithi092.stdout:6/569: creat d4/d24/d26/fb8 x:0 0 0 2024-05-06T12:14:35.912 INFO:tasks.workunit.client.0.smithi092.stdout:7/617: creat d9/de/d2d/fca x:0 0 0 2024-05-06T12:14:35.912 INFO:tasks.workunit.client.0.smithi092.stdout:2/326: sync 2024-05-06T12:14:35.912 INFO:tasks.workunit.client.0.smithi092.stdout:0/627: sync 2024-05-06T12:14:35.913 INFO:tasks.workunit.client.1.smithi190.stdout:9/31: symlink db/le 0 2024-05-06T12:14:35.913 INFO:tasks.workunit.client.0.smithi092.stdout:5/466: symlink d4/d34/d28/d4e/d5d/d82/d1f/d78/laf 0 2024-05-06T12:14:35.913 INFO:tasks.workunit.client.0.smithi092.stdout:6/570: chown d4/d24/d66/f88 8870845 1 2024-05-06T12:14:35.914 INFO:tasks.workunit.client.1.smithi190.stdout:9/32: mkdir db/dd/df 0 2024-05-06T12:14:35.915 INFO:tasks.workunit.client.1.smithi190.stdout:9/33: chown db/le 1 1 2024-05-06T12:14:35.916 INFO:tasks.workunit.client.0.smithi092.stdout:9/444: link d16/d2c/d2f/f31 d16/d2c/d2f/d3d/d33/d37/d40/d43/f89 0 2024-05-06T12:14:35.916 INFO:tasks.workunit.client.1.smithi190.stdout:9/34: mknod db/c10 0 2024-05-06T12:14:35.918 INFO:tasks.workunit.client.1.smithi190.stdout:9/35: write f7 [1328330,23534] 0 2024-05-06T12:14:35.919 INFO:tasks.workunit.client.0.smithi092.stdout:7/618: unlink d9/de/d2d/fca 0 2024-05-06T12:14:35.919 INFO:tasks.workunit.client.1.smithi190.stdout:9/36: chown f6 437189583 1 2024-05-06T12:14:35.919 INFO:tasks.workunit.client.0.smithi092.stdout:4/554: dread d9/d2b/d5b/f85 [0,4194304] 0 2024-05-06T12:14:35.920 INFO:tasks.workunit.client.0.smithi092.stdout:3/416: truncate d8/f12 2804307 0 2024-05-06T12:14:35.921 INFO:tasks.workunit.client.0.smithi092.stdout:1/452: dread d3/d10/d13/d19/d21/d72/f86 [0,4194304] 0 2024-05-06T12:14:35.921 INFO:tasks.workunit.client.1.smithi190.stdout:9/37: creat db/f11 x:0 0 0 2024-05-06T12:14:35.923 INFO:tasks.workunit.client.0.smithi092.stdout:3/417: chown d8/db/d14/c7f 14 1 2024-05-06T12:14:35.924 INFO:tasks.workunit.client.0.smithi092.stdout:0/628: unlink d7/dc/d28/d54/d12/d8f/fbe 0 2024-05-06T12:14:35.924 INFO:tasks.workunit.client.0.smithi092.stdout:1/453: truncate d3/d10/d13/d46/f61 622883 0 2024-05-06T12:14:35.925 INFO:tasks.workunit.client.1.smithi190.stdout:9/38: link c9 db/dd/df/c12 0 2024-05-06T12:14:35.925 INFO:tasks.workunit.client.1.smithi190.stdout:9/39: stat f5 0 2024-05-06T12:14:35.925 INFO:tasks.workunit.client.0.smithi092.stdout:0/629: write d7/dc/d9e/fc0 [71302,85430] 0 2024-05-06T12:14:35.926 INFO:tasks.workunit.client.1.smithi190.stdout:9/40: write f3 [1179815,98744] 0 2024-05-06T12:14:35.928 INFO:tasks.workunit.client.0.smithi092.stdout:5/467: dread d4/d34/d28/f74 [0,4194304] 0 2024-05-06T12:14:35.928 INFO:tasks.workunit.client.1.smithi190.stdout:9/41: creat db/f13 x:0 0 0 2024-05-06T12:14:35.928 INFO:tasks.workunit.client.1.smithi190.stdout:9/42: truncate f4 950372 0 2024-05-06T12:14:35.929 INFO:tasks.workunit.client.0.smithi092.stdout:6/571: unlink d4/d20/c40 0 2024-05-06T12:14:35.930 INFO:tasks.workunit.client.0.smithi092.stdout:6/572: chown d4/d24/d38/lab 1859715 1 2024-05-06T12:14:35.931 INFO:tasks.workunit.client.1.smithi190.stdout:9/43: mknod db/dd/df/c14 0 2024-05-06T12:14:35.933 INFO:tasks.workunit.client.0.smithi092.stdout:4/555: creat d9/d2b/d65/d83/fba x:0 0 0 2024-05-06T12:14:35.935 INFO:tasks.workunit.client.1.smithi190.stdout:9/44: dread db/fc [0,4194304] 0 2024-05-06T12:14:35.936 INFO:tasks.workunit.client.0.smithi092.stdout:8/515: creat d9/fc2 x:0 0 0 2024-05-06T12:14:35.936 INFO:tasks.workunit.client.0.smithi092.stdout:1/454: unlink d3/l2b 0 2024-05-06T12:14:35.937 INFO:tasks.workunit.client.1.smithi190.stdout:9/45: dread f7 [0,4194304] 0 2024-05-06T12:14:35.938 INFO:tasks.workunit.client.0.smithi092.stdout:9/445: mkdir d16/d2c/d2f/d3d/d33/d37/d8a 0 2024-05-06T12:14:35.940 INFO:tasks.workunit.client.0.smithi092.stdout:2/327: creat df/d17/f62 x:0 0 0 2024-05-06T12:14:35.943 INFO:tasks.workunit.client.0.smithi092.stdout:5/468: mkdir d4/d34/d29/d54/db0 0 2024-05-06T12:14:35.945 INFO:tasks.workunit.client.0.smithi092.stdout:3/418: mknod d8/d24/d27/c82 0 2024-05-06T12:14:35.945 INFO:tasks.workunit.client.0.smithi092.stdout:9/446: dwrite d16/d1c/f7c [0,4194304] 0 2024-05-06T12:14:35.948 INFO:tasks.workunit.client.0.smithi092.stdout:0/630: write f1 [2182112,1082] 0 2024-05-06T12:14:35.950 INFO:tasks.workunit.client.0.smithi092.stdout:7/619: truncate d9/f80 4075646 0 2024-05-06T12:14:35.954 INFO:tasks.workunit.client.0.smithi092.stdout:5/469: symlink d4/d34/dae/lb1 0 2024-05-06T12:14:35.956 INFO:tasks.workunit.client.0.smithi092.stdout:6/573: rename d4/d20/d23/l65 to d4/d24/d38/d5d/d6d/d7b/lb9 0 2024-05-06T12:14:35.957 INFO:tasks.workunit.client.0.smithi092.stdout:9/447: dwrite d16/d2c/d3f/f60 [0,4194304] 0 2024-05-06T12:14:35.960 INFO:tasks.workunit.client.0.smithi092.stdout:6/574: dread - d4/d20/fb2 zero size 2024-05-06T12:14:35.960 INFO:tasks.workunit.client.0.smithi092.stdout:8/516: fdatasync d9/d21/f81 0 2024-05-06T12:14:35.964 INFO:tasks.workunit.client.0.smithi092.stdout:0/631: dwrite d7/dc/d28/f4d [0,4194304] 0 2024-05-06T12:14:35.966 INFO:tasks.workunit.client.0.smithi092.stdout:5/470: truncate d4/d5/d10/d31/d53/d64/f8b 1085010 0 2024-05-06T12:14:35.968 INFO:tasks.workunit.client.1.smithi190.stdout:0/22: fsync f5 0 2024-05-06T12:14:35.968 INFO:tasks.workunit.client.1.smithi190.stdout:0/23: fsync f5 0 2024-05-06T12:14:35.969 INFO:tasks.workunit.client.0.smithi092.stdout:8/517: truncate d9/d21/d37/d46/d5c/d71/f80 4607419 0 2024-05-06T12:14:35.970 INFO:tasks.workunit.client.1.smithi190.stdout:0/24: creat f7 x:0 0 0 2024-05-06T12:14:35.971 INFO:tasks.workunit.client.0.smithi092.stdout:2/328: dread df/f1e [0,4194304] 0 2024-05-06T12:14:35.971 INFO:tasks.workunit.client.1.smithi190.stdout:0/25: write f7 [914337,111985] 0 2024-05-06T12:14:35.972 INFO:tasks.workunit.client.1.smithi190.stdout:0/26: fdatasync f6 0 2024-05-06T12:14:35.975 INFO:tasks.workunit.client.0.smithi092.stdout:4/556: rmdir d9/df/d62/d7b/db4 0 2024-05-06T12:14:35.975 INFO:tasks.workunit.client.1.smithi190.stdout:0/27: symlink l8 0 2024-05-06T12:14:35.976 INFO:tasks.workunit.client.0.smithi092.stdout:3/419: rename d8/db/d14/d5e/c76 to d8/d1b/c83 0 2024-05-06T12:14:35.977 INFO:tasks.workunit.client.0.smithi092.stdout:1/455: write d3/d10/d13/d19/f5a [824544,79071] 0 2024-05-06T12:14:35.977 INFO:tasks.workunit.client.0.smithi092.stdout:0/632: dread d7/dc/d28/d54/d1e/d5d/f68 [0,4194304] 0 2024-05-06T12:14:35.978 INFO:tasks.workunit.client.0.smithi092.stdout:9/448: creat d16/d2c/d3f/f8b x:0 0 0 2024-05-06T12:14:35.978 INFO:tasks.workunit.client.0.smithi092.stdout:3/420: chown d8/db/f4d 2 1 2024-05-06T12:14:35.979 INFO:tasks.workunit.client.0.smithi092.stdout:6/575: mknod d4/d24/db0/cba 0 2024-05-06T12:14:35.981 INFO:tasks.workunit.client.0.smithi092.stdout:6/576: write d4/d4c/d52/f70 [4495803,35421] 0 2024-05-06T12:14:35.981 INFO:tasks.workunit.client.1.smithi190.stdout:5/8: rmdir d2 39 2024-05-06T12:14:35.982 INFO:tasks.workunit.client.0.smithi092.stdout:2/329: truncate df/d17/f3e 572571 0 2024-05-06T12:14:35.983 INFO:tasks.workunit.client.1.smithi190.stdout:8/17: rename l1 to d3/l5 0 2024-05-06T12:14:35.984 INFO:tasks.workunit.client.0.smithi092.stdout:4/557: mkdir d9/df/d62/dbb 0 2024-05-06T12:14:35.985 INFO:tasks.workunit.client.0.smithi092.stdout:0/633: mkdir d7/dc/d97/dc9 0 2024-05-06T12:14:35.986 INFO:tasks.workunit.client.1.smithi190.stdout:8/18: creat d3/f6 x:0 0 0 2024-05-06T12:14:35.987 INFO:tasks.workunit.client.0.smithi092.stdout:1/456: rmdir d3/d10/d13/d19/d21/d88 39 2024-05-06T12:14:35.988 INFO:tasks.workunit.client.0.smithi092.stdout:9/449: dwrite d16/d2c/d2f/d3d/d33/d37/f72 [0,4194304] 0 2024-05-06T12:14:35.988 INFO:tasks.workunit.client.1.smithi190.stdout:5/9: creat d2/f5 x:0 0 0 2024-05-06T12:14:35.990 INFO:tasks.workunit.client.1.smithi190.stdout:3/8: truncate f2 593783 0 2024-05-06T12:14:35.990 INFO:tasks.workunit.client.0.smithi092.stdout:8/518: getdents d9/d21/d37/dbd 0 2024-05-06T12:14:35.994 INFO:tasks.workunit.client.0.smithi092.stdout:3/421: mknod d8/db/d4c/d40/d67/c84 0 2024-05-06T12:14:35.994 INFO:tasks.workunit.client.0.smithi092.stdout:3/422: chown d8/db/d4c/d40 15444799 1 2024-05-06T12:14:35.995 INFO:tasks.workunit.client.1.smithi190.stdout:8/19: dwrite d3/f6 [0,4194304] 0 2024-05-06T12:14:35.996 INFO:tasks.workunit.client.1.smithi190.stdout:3/9: chown f2 21368 1 2024-05-06T12:14:35.998 INFO:tasks.workunit.client.1.smithi190.stdout:8/20: symlink d3/l7 0 2024-05-06T12:14:35.998 INFO:tasks.workunit.client.0.smithi092.stdout:0/634: creat d7/dc/d28/d54/d1e/d39/fca x:0 0 0 2024-05-06T12:14:35.999 INFO:tasks.workunit.client.0.smithi092.stdout:0/635: stat d7/f88 0 2024-05-06T12:14:35.999 INFO:tasks.workunit.client.0.smithi092.stdout:5/471: symlink d4/d34/d28/d4e/d5d/d82/d1f/d20/lb2 0 2024-05-06T12:14:36.001 INFO:tasks.workunit.client.1.smithi190.stdout:8/21: mkdir d3/d8 0 2024-05-06T12:14:36.002 INFO:tasks.workunit.client.0.smithi092.stdout:1/457: creat d3/d10/d13/d46/f92 x:0 0 0 2024-05-06T12:14:36.004 INFO:tasks.workunit.client.0.smithi092.stdout:9/450: rename d16/d2c/f65 to d16/d2c/d2f/d3d/d33/d37/d40/d43/f8c 0 2024-05-06T12:14:36.004 INFO:tasks.workunit.client.1.smithi190.stdout:8/22: rename c0 to d3/c9 0 2024-05-06T12:14:36.005 INFO:tasks.workunit.client.0.smithi092.stdout:7/620: link d9/de/d10/d17/d6a/f9d d9/de/d1f/d37/d48/d97/fcb 0 2024-05-06T12:14:36.007 INFO:tasks.workunit.client.0.smithi092.stdout:6/577: mknod d4/d4c/d91/d93/d9d/cbb 0 2024-05-06T12:14:36.008 INFO:tasks.workunit.client.0.smithi092.stdout:6/578: fdatasync d4/d4c/d52/d62/f71 0 2024-05-06T12:14:36.008 INFO:tasks.workunit.client.1.smithi190.stdout:8/23: dread d3/f6 [0,4194304] 0 2024-05-06T12:14:36.009 INFO:tasks.workunit.client.0.smithi092.stdout:4/558: mkdir d9/d2b/d65/dbc 0 2024-05-06T12:14:36.010 INFO:tasks.workunit.client.1.smithi190.stdout:8/24: mknod d3/ca 0 2024-05-06T12:14:36.011 INFO:tasks.workunit.client.0.smithi092.stdout:0/636: rename d7/dc/d28/d54/d1e/d5d/cb8 to d7/dc/d28/d54/ccb 0 2024-05-06T12:14:36.012 INFO:tasks.workunit.client.1.smithi190.stdout:8/25: write d3/f6 [981138,67705] 0 2024-05-06T12:14:36.012 INFO:tasks.workunit.client.0.smithi092.stdout:8/519: mkdir d9/d21/d37/dbd/dc3 0 2024-05-06T12:14:36.013 INFO:tasks.workunit.client.1.smithi190.stdout:9/46: fsync db/f13 0 2024-05-06T12:14:36.013 INFO:tasks.workunit.client.1.smithi190.stdout:9/47: dread - f5 zero size 2024-05-06T12:14:36.014 INFO:tasks.workunit.client.0.smithi092.stdout:7/621: fdatasync d9/de/d10/d17/d99/f83 0 2024-05-06T12:14:36.014 INFO:tasks.workunit.client.0.smithi092.stdout:3/423: dwrite d8/db/d4c/d40/f52 [0,4194304] 0 2024-05-06T12:14:36.015 INFO:tasks.workunit.client.1.smithi190.stdout:8/26: rename c2 to d3/cb 0 2024-05-06T12:14:36.016 INFO:tasks.workunit.client.1.smithi190.stdout:9/48: fsync db/fc 0 2024-05-06T12:14:36.016 INFO:tasks.workunit.client.1.smithi190.stdout:9/49: readlink l8 0 2024-05-06T12:14:36.017 INFO:tasks.workunit.client.1.smithi190.stdout:9/50: chown f7 499797465 1 2024-05-06T12:14:36.018 INFO:tasks.workunit.client.1.smithi190.stdout:0/28: truncate f7 57118 0 2024-05-06T12:14:36.019 INFO:tasks.workunit.client.0.smithi092.stdout:9/451: mkdir d16/d1e/d7b/d8d 0 2024-05-06T12:14:36.019 INFO:tasks.workunit.client.1.smithi190.stdout:9/51: symlink db/dd/l15 0 2024-05-06T12:14:36.020 INFO:tasks.workunit.client.1.smithi190.stdout:8/27: fdatasync d3/f6 0 2024-05-06T12:14:36.020 INFO:tasks.workunit.client.1.smithi190.stdout:9/52: read f7 [1227588,61810] 0 2024-05-06T12:14:36.020 INFO:tasks.workunit.client.0.smithi092.stdout:0/637: dread d7/dc/d28/d54/d1e/d5d/f76 [0,4194304] 0 2024-05-06T12:14:36.020 INFO:tasks.workunit.client.0.smithi092.stdout:8/520: fdatasync d9/d21/f88 0 2024-05-06T12:14:36.021 INFO:tasks.workunit.client.1.smithi190.stdout:9/53: write f6 [531916,72075] 0 2024-05-06T12:14:36.022 INFO:tasks.workunit.client.1.smithi190.stdout:0/29: mkdir d9 0 2024-05-06T12:14:36.023 INFO:tasks.workunit.client.0.smithi092.stdout:8/521: stat d9/d21/d72/lbf 0 2024-05-06T12:14:36.024 INFO:tasks.workunit.client.1.smithi190.stdout:5/10: rmdir d2 39 2024-05-06T12:14:36.024 INFO:tasks.workunit.client.0.smithi092.stdout:2/330: link df/d17/d1d/f41 df/d17/d1d/d33/d3c/f63 0 2024-05-06T12:14:36.025 INFO:tasks.workunit.client.0.smithi092.stdout:7/622: dwrite d9/de/d10/d17/d21/d38/f89 [0,4194304] 0 2024-05-06T12:14:36.025 INFO:tasks.workunit.client.1.smithi190.stdout:3/10: read f2 [136859,6196] 0 2024-05-06T12:14:36.025 INFO:tasks.workunit.client.1.smithi190.stdout:8/28: dread d3/f6 [0,4194304] 0 2024-05-06T12:14:36.026 INFO:tasks.workunit.client.1.smithi190.stdout:8/29: write d3/f6 [3771132,461] 0 2024-05-06T12:14:36.027 INFO:tasks.workunit.client.0.smithi092.stdout:2/331: write df/d17/f4a [56762,13380] 0 2024-05-06T12:14:36.028 INFO:tasks.workunit.client.0.smithi092.stdout:7/623: truncate d9/de/d1f/d37/fa0 39871 0 2024-05-06T12:14:36.028 INFO:tasks.workunit.client.0.smithi092.stdout:6/579: mknod d4/d24/d38/d5d/d6d/cbc 0 2024-05-06T12:14:36.029 INFO:tasks.workunit.client.1.smithi190.stdout:9/54: mkdir db/dd/d16 0 2024-05-06T12:14:36.032 INFO:tasks.workunit.client.1.smithi190.stdout:8/30: dread d3/f6 [0,4194304] 0 2024-05-06T12:14:36.032 INFO:tasks.workunit.client.0.smithi092.stdout:5/472: rename d4/d34/d28/d4e/d5d/d82/d1f/d20/f3e to d4/d34/d29/d54/d9f/fb3 0 2024-05-06T12:14:36.036 INFO:tasks.workunit.client.0.smithi092.stdout:9/452: mknod d16/d2c/d2f/d75/c8e 0 2024-05-06T12:14:36.037 INFO:tasks.workunit.client.0.smithi092.stdout:7/624: dwrite d9/de/d10/d17/d21/f5d [0,4194304] 0 2024-05-06T12:14:36.037 INFO:tasks.workunit.client.0.smithi092.stdout:5/473: dread d4/f7d [0,4194304] 0 2024-05-06T12:14:36.038 INFO:tasks.workunit.client.1.smithi190.stdout:5/11: symlink d2/l6 0 2024-05-06T12:14:36.039 INFO:tasks.workunit.client.1.smithi190.stdout:5/12: readlink d2/l6 0 2024-05-06T12:14:36.039 INFO:tasks.workunit.client.0.smithi092.stdout:2/332: mknod df/d1b/c64 0 2024-05-06T12:14:36.039 INFO:tasks.workunit.client.1.smithi190.stdout:8/31: creat d3/fc x:0 0 0 2024-05-06T12:14:36.040 INFO:tasks.workunit.client.1.smithi190.stdout:8/32: dread - d3/fc zero size 2024-05-06T12:14:36.041 INFO:tasks.workunit.client.0.smithi092.stdout:5/474: read - d4/d5/d10/d31/d53/d64/f8a zero size 2024-05-06T12:14:36.041 INFO:tasks.workunit.client.0.smithi092.stdout:1/458: getdents d3/d10/d13 0 2024-05-06T12:14:36.042 INFO:tasks.workunit.client.1.smithi190.stdout:8/33: dread - d3/fc zero size 2024-05-06T12:14:36.043 INFO:tasks.workunit.client.1.smithi190.stdout:8/34: chown d3 525 1 2024-05-06T12:14:36.043 INFO:tasks.workunit.client.0.smithi092.stdout:5/475: write d4/d34/f56 [2647841,19586] 0 2024-05-06T12:14:36.043 INFO:tasks.workunit.client.1.smithi190.stdout:9/55: link f6 db/dd/f17 0 2024-05-06T12:14:36.045 INFO:tasks.workunit.client.0.smithi092.stdout:4/559: getdents d9/df 0 2024-05-06T12:14:36.047 INFO:tasks.workunit.client.1.smithi190.stdout:8/35: chown d3/c9 32026 1 2024-05-06T12:14:36.048 INFO:tasks.workunit.client.0.smithi092.stdout:0/638: mkdir d7/dc/d28/d54/d12/d7f/db5/dcc 0 2024-05-06T12:14:36.050 INFO:tasks.workunit.client.0.smithi092.stdout:8/522: creat d9/d21/d37/d46/d5c/d93/fc4 x:0 0 0 2024-05-06T12:14:36.050 INFO:tasks.workunit.client.1.smithi190.stdout:9/56: dread db/dd/f17 [0,4194304] 0 2024-05-06T12:14:36.051 INFO:tasks.workunit.client.1.smithi190.stdout:5/13: dwrite d2/f3 [0,4194304] 0 2024-05-06T12:14:36.051 INFO:tasks.workunit.client.0.smithi092.stdout:7/625: creat d9/de/d10/d17/d21/d2e/d9c/fcc x:0 0 0 2024-05-06T12:14:36.051 INFO:tasks.workunit.client.1.smithi190.stdout:0/30: dread f7 [0,4194304] 0 2024-05-06T12:14:36.055 INFO:tasks.workunit.client.1.smithi190.stdout:8/36: creat d3/d8/fd x:0 0 0 2024-05-06T12:14:36.056 INFO:tasks.workunit.client.0.smithi092.stdout:5/476: symlink d4/d34/d29/lb4 0 2024-05-06T12:14:36.057 INFO:tasks.workunit.client.0.smithi092.stdout:3/424: getdents d8/db/d4c 0 2024-05-06T12:14:36.058 INFO:tasks.workunit.client.1.smithi190.stdout:9/57: stat c9 0 2024-05-06T12:14:36.058 INFO:tasks.workunit.client.0.smithi092.stdout:9/453: symlink d16/d1c/d58/l8f 0 2024-05-06T12:14:36.059 INFO:tasks.workunit.client.0.smithi092.stdout:0/639: dwrite d7/dc/d9e/f1c [0,4194304] 0 2024-05-06T12:14:36.060 INFO:tasks.workunit.client.1.smithi190.stdout:5/14: dread d2/f3 [0,4194304] 0 2024-05-06T12:14:36.060 INFO:tasks.workunit.client.1.smithi190.stdout:9/58: dread f6 [0,4194304] 0 2024-05-06T12:14:36.066 INFO:tasks.workunit.client.0.smithi092.stdout:1/459: rename d3/c33 to d3/d42/d84/d4f/d58/c93 0 2024-05-06T12:14:36.123 INFO:tasks.workunit.client.1.smithi190.stdout:0/31: mknod d9/ca 0 2024-05-06T12:14:36.123 INFO:tasks.workunit.client.1.smithi190.stdout:3/11: dwrite f2 [0,4194304] 0 2024-05-06T12:14:36.123 INFO:tasks.workunit.client.1.smithi190.stdout:3/12: rmdir - no directory 2024-05-06T12:14:36.123 INFO:tasks.workunit.client.1.smithi190.stdout:5/15: rename l1 to d2/l7 0 2024-05-06T12:14:36.123 INFO:tasks.workunit.client.1.smithi190.stdout:9/59: creat db/dd/d16/f18 x:0 0 0 2024-05-06T12:14:36.123 INFO:tasks.workunit.client.1.smithi190.stdout:5/16: creat d2/f8 x:0 0 0 2024-05-06T12:14:36.123 INFO:tasks.workunit.client.1.smithi190.stdout:0/32: rename f6 to d9/fb 0 2024-05-06T12:14:36.123 INFO:tasks.workunit.client.1.smithi190.stdout:0/33: chown c4 7 1 2024-05-06T12:14:36.123 INFO:tasks.workunit.client.1.smithi190.stdout:0/34: truncate f7 393938 0 2024-05-06T12:14:36.123 INFO:tasks.workunit.client.1.smithi190.stdout:0/35: mkdir d9/dc 0 2024-05-06T12:14:36.123 INFO:tasks.workunit.client.1.smithi190.stdout:0/36: mknod d9/dc/cd 0 2024-05-06T12:14:36.124 INFO:tasks.workunit.client.1.smithi190.stdout:8/37: dwrite d3/f6 [4194304,4194304] 0 2024-05-06T12:14:36.124 INFO:tasks.workunit.client.1.smithi190.stdout:0/37: chown f3 144477846 1 2024-05-06T12:14:36.124 INFO:tasks.workunit.client.1.smithi190.stdout:8/38: write d3/d8/fd [767498,19214] 0 2024-05-06T12:14:36.124 INFO:tasks.workunit.client.1.smithi190.stdout:0/38: rename f5 to d9/dc/fe 0 2024-05-06T12:14:36.124 INFO:tasks.workunit.client.1.smithi190.stdout:8/39: unlink d3/ca 0 2024-05-06T12:14:36.124 INFO:tasks.workunit.client.1.smithi190.stdout:8/40: write d3/fc [728706,113919] 0 2024-05-06T12:14:36.124 INFO:tasks.workunit.client.1.smithi190.stdout:8/41: chown d3/c4 1 1 2024-05-06T12:14:36.124 INFO:tasks.workunit.client.1.smithi190.stdout:8/42: write d3/fc [855689,16252] 0 2024-05-06T12:14:36.124 INFO:tasks.workunit.client.1.smithi190.stdout:8/43: chown d3/f6 2 1 2024-05-06T12:14:36.124 INFO:tasks.workunit.client.1.smithi190.stdout:8/44: chown d3 54 1 2024-05-06T12:14:36.124 INFO:tasks.workunit.client.1.smithi190.stdout:5/17: getdents d2 0 2024-05-06T12:14:36.124 INFO:tasks.workunit.client.1.smithi190.stdout:3/13: truncate f2 1412232 0 2024-05-06T12:14:36.124 INFO:tasks.workunit.client.1.smithi190.stdout:9/60: truncate f3 196596 0 2024-05-06T12:14:36.124 INFO:tasks.workunit.client.1.smithi190.stdout:8/45: dwrite d3/d8/fd [0,4194304] 0 2024-05-06T12:14:36.124 INFO:tasks.workunit.client.1.smithi190.stdout:5/18: dread f0 [0,4194304] 0 2024-05-06T12:14:36.125 INFO:tasks.workunit.client.1.smithi190.stdout:9/61: mkdir db/dd/d16/d19 0 2024-05-06T12:14:36.125 INFO:tasks.workunit.client.1.smithi190.stdout:0/39: truncate d9/dc/fe 448038 0 2024-05-06T12:14:36.125 INFO:tasks.workunit.client.1.smithi190.stdout:8/46: creat d3/fe x:0 0 0 2024-05-06T12:14:36.125 INFO:tasks.workunit.client.1.smithi190.stdout:3/14: write f2 [442858,109437] 0 2024-05-06T12:14:36.125 INFO:tasks.workunit.client.1.smithi190.stdout:9/62: symlink db/dd/df/l1a 0 2024-05-06T12:14:36.126 INFO:tasks.workunit.client.0.smithi092.stdout:6/580: dwrite d4/d24/d38/faa [0,4194304] 0 2024-05-06T12:14:36.126 INFO:tasks.workunit.client.0.smithi092.stdout:8/523: dwrite d9/d21/d37/d46/d5c/d93/fc4 [0,4194304] 0 2024-05-06T12:14:36.126 INFO:tasks.workunit.client.0.smithi092.stdout:0/640: stat d7/dc/d28/d54/d1e/d46/d63/d80/cb4 0 2024-05-06T12:14:36.126 INFO:tasks.workunit.client.0.smithi092.stdout:4/560: rename d9/f38 to d9/d2b/d65/d83/fbd 0 2024-05-06T12:14:36.126 INFO:tasks.workunit.client.0.smithi092.stdout:5/477: symlink d4/d34/d28/d4e/d5d/d82/d1f/d78/lb5 0 2024-05-06T12:14:36.126 INFO:tasks.workunit.client.0.smithi092.stdout:4/561: readlink d9/dd/l61 0 2024-05-06T12:14:36.126 INFO:tasks.workunit.client.0.smithi092.stdout:5/478: dread - d4/d34/d29/d54/fa2 zero size 2024-05-06T12:14:36.126 INFO:tasks.workunit.client.0.smithi092.stdout:2/333: dwrite df/d17/d1d/f49 [0,4194304] 0 2024-05-06T12:14:36.126 INFO:tasks.workunit.client.0.smithi092.stdout:2/334: write df/d17/d1d/d33/f44 [601086,31516] 0 2024-05-06T12:14:36.126 INFO:tasks.workunit.client.0.smithi092.stdout:9/454: mknod d16/d2c/d2f/d3d/d33/d37/d40/d43/c90 0 2024-05-06T12:14:36.126 INFO:tasks.workunit.client.0.smithi092.stdout:0/641: chown d7/dc/l8a 0 1 2024-05-06T12:14:36.126 INFO:tasks.workunit.client.0.smithi092.stdout:6/581: write d4/d4c/d52/d62/fad [71467,75418] 0 2024-05-06T12:14:36.126 INFO:tasks.workunit.client.0.smithi092.stdout:8/524: mkdir d9/d21/d37/dbd/dc5 0 2024-05-06T12:14:36.126 INFO:tasks.workunit.client.0.smithi092.stdout:9/455: chown fc 6531 1 2024-05-06T12:14:36.127 INFO:tasks.workunit.client.0.smithi092.stdout:5/479: symlink d4/d34/d29/lb6 0 2024-05-06T12:14:36.127 INFO:tasks.workunit.client.0.smithi092.stdout:1/460: truncate d3/d10/d13/d19/d21/d88/f91 877823 0 2024-05-06T12:14:36.127 INFO:tasks.workunit.client.0.smithi092.stdout:0/642: mkdir d7/dc/d31/dcd 0 2024-05-06T12:14:36.127 INFO:tasks.workunit.client.0.smithi092.stdout:9/456: unlink d16/d1e/f55 0 2024-05-06T12:14:36.127 INFO:tasks.workunit.client.0.smithi092.stdout:8/525: creat d9/d5a/d9f/fc6 x:0 0 0 2024-05-06T12:14:36.127 INFO:tasks.workunit.client.0.smithi092.stdout:4/562: rmdir d9/dd/da6/dad 0 2024-05-06T12:14:36.127 INFO:tasks.workunit.client.0.smithi092.stdout:1/461: readlink d3/d10/d13/d19/d21/l28 0 2024-05-06T12:14:36.127 INFO:tasks.workunit.client.0.smithi092.stdout:6/582: creat d4/d24/d26/dae/fbd x:0 0 0 2024-05-06T12:14:36.127 INFO:tasks.workunit.client.0.smithi092.stdout:5/480: dwrite d4/d59/f83 [0,4194304] 0 2024-05-06T12:14:36.127 INFO:tasks.workunit.client.0.smithi092.stdout:4/563: creat d9/d27/fbe x:0 0 0 2024-05-06T12:14:36.128 INFO:tasks.workunit.client.0.smithi092.stdout:8/526: creat d9/d21/d37/d46/d5c/d62/da5/fc7 x:0 0 0 2024-05-06T12:14:36.128 INFO:tasks.workunit.client.0.smithi092.stdout:6/583: write d4/d4c/d52/f4a [199943,86583] 0 2024-05-06T12:14:36.128 INFO:tasks.workunit.client.0.smithi092.stdout:8/527: dread - d9/d21/f88 zero size 2024-05-06T12:14:36.128 INFO:tasks.workunit.client.0.smithi092.stdout:9/457: dwrite d16/d1e/f28 [0,4194304] 0 2024-05-06T12:14:36.128 INFO:tasks.workunit.client.0.smithi092.stdout:4/564: write d9/df/d62/f63 [2497558,53626] 0 2024-05-06T12:14:36.128 INFO:tasks.workunit.client.0.smithi092.stdout:9/458: chown d16/d2c/d2f 16780190 1 2024-05-06T12:14:36.128 INFO:tasks.workunit.client.0.smithi092.stdout:1/462: rename d3/d10/l4c to d3/d10/d13/d19/d21/l94 0 2024-05-06T12:14:36.128 INFO:tasks.workunit.client.0.smithi092.stdout:7/626: dread d9/de/f34 [0,4194304] 0 2024-05-06T12:14:36.128 INFO:tasks.workunit.client.0.smithi092.stdout:9/459: stat d16/d2c/d2f/d3d/d33 0 2024-05-06T12:14:36.128 INFO:tasks.workunit.client.0.smithi092.stdout:1/463: fdatasync d3/d42/f76 0 2024-05-06T12:14:36.128 INFO:tasks.workunit.client.0.smithi092.stdout:8/528: truncate d9/dc/f51 7200017 0 2024-05-06T12:14:36.128 INFO:tasks.workunit.client.0.smithi092.stdout:6/584: dwrite d4/d4c/d91/fb6 [0,4194304] 0 2024-05-06T12:14:36.128 INFO:tasks.workunit.client.0.smithi092.stdout:6/585: read d4/fb [5313949,107290] 0 2024-05-06T12:14:36.129 INFO:tasks.workunit.client.0.smithi092.stdout:9/460: unlink d16/d2c/d4b/f54 0 2024-05-06T12:14:36.129 INFO:tasks.workunit.client.1.smithi190.stdout:8/47: symlink d3/d8/lf 0 2024-05-06T12:14:36.129 INFO:tasks.workunit.client.1.smithi190.stdout:0/40: mkdir d9/df 0 2024-05-06T12:14:36.129 INFO:tasks.workunit.client.1.smithi190.stdout:3/15: write f2 [915433,38715] 0 2024-05-06T12:14:36.130 INFO:tasks.workunit.client.1.smithi190.stdout:8/48: write d3/f6 [7750477,64175] 0 2024-05-06T12:14:36.130 INFO:tasks.workunit.client.1.smithi190.stdout:3/16: write f2 [201625,19666] 0 2024-05-06T12:14:36.130 INFO:tasks.workunit.client.1.smithi190.stdout:8/49: chown d3/d8/fd 1487 1 2024-05-06T12:14:36.130 INFO:tasks.workunit.client.1.smithi190.stdout:8/50: fdatasync d3/fc 0 2024-05-06T12:14:36.132 INFO:tasks.workunit.client.1.smithi190.stdout:9/63: rename db/dd/d16/f18 to db/dd/df/f1b 0 2024-05-06T12:14:36.134 INFO:tasks.workunit.client.1.smithi190.stdout:0/41: write d9/fb [357373,103080] 0 2024-05-06T12:14:36.134 INFO:tasks.workunit.client.1.smithi190.stdout:9/64: write db/fc [425858,52377] 0 2024-05-06T12:14:36.134 INFO:tasks.workunit.client.1.smithi190.stdout:3/17: mknod c3 0 2024-05-06T12:14:36.135 INFO:tasks.workunit.client.0.smithi092.stdout:1/464: rename d3/d42/f76 to d3/d10/d13/d19/d34/d89/f95 0 2024-05-06T12:14:36.135 INFO:tasks.workunit.client.0.smithi092.stdout:1/465: chown d3/d10/d13/f3b 1 1 2024-05-06T12:14:36.138 INFO:tasks.workunit.client.1.smithi190.stdout:8/51: rename d3/fe to d3/d8/f10 0 2024-05-06T12:14:36.138 INFO:tasks.workunit.client.1.smithi190.stdout:9/65: dread db/dd/f17 [0,4194304] 0 2024-05-06T12:14:36.140 INFO:tasks.workunit.client.1.smithi190.stdout:8/52: chown d3/fc 1033128417 1 2024-05-06T12:14:36.141 INFO:tasks.workunit.client.0.smithi092.stdout:3/425: truncate d8/d24/d33/f6a 1175152 0 2024-05-06T12:14:36.143 INFO:tasks.workunit.client.1.smithi190.stdout:9/66: mknod db/dd/df/c1c 0 2024-05-06T12:14:36.144 INFO:tasks.workunit.client.0.smithi092.stdout:1/466: symlink d3/d10/d13/d19/d34/d89/l96 0 2024-05-06T12:14:36.146 INFO:tasks.workunit.client.1.smithi190.stdout:9/67: chown db/fc 458 1 2024-05-06T12:14:36.146 INFO:tasks.workunit.client.1.smithi190.stdout:0/42: dwrite d9/fb [0,4194304] 0 2024-05-06T12:14:36.147 INFO:tasks.workunit.client.1.smithi190.stdout:3/18: dwrite f2 [0,4194304] 0 2024-05-06T12:14:36.149 INFO:tasks.workunit.client.0.smithi092.stdout:8/529: creat d9/dc/fc8 x:0 0 0 2024-05-06T12:14:36.149 INFO:tasks.workunit.client.0.smithi092.stdout:2/335: dwrite df/f5b [0,4194304] 0 2024-05-06T12:14:36.150 INFO:tasks.workunit.client.0.smithi092.stdout:8/530: write d9/fb6 [65078,27230] 0 2024-05-06T12:14:36.151 INFO:tasks.workunit.client.1.smithi190.stdout:3/19: mknod c4 0 2024-05-06T12:14:36.152 INFO:tasks.workunit.client.0.smithi092.stdout:3/426: unlink d8/d24/d33/d4b/c51 0 2024-05-06T12:14:36.157 INFO:tasks.workunit.client.0.smithi092.stdout:0/643: truncate d7/dc/d9e/f49 1077752 0 2024-05-06T12:14:36.158 INFO:tasks.workunit.client.0.smithi092.stdout:6/586: link d4/d20/d23/c7d d4/d4c/d91/d93/d9d/cbe 0 2024-05-06T12:14:36.158 INFO:tasks.workunit.client.1.smithi190.stdout:8/53: dwrite d3/d8/f10 [0,4194304] 0 2024-05-06T12:14:36.159 INFO:tasks.workunit.client.0.smithi092.stdout:6/587: read d4/d24/d38/faa [3080873,67719] 0 2024-05-06T12:14:36.160 INFO:tasks.workunit.client.0.smithi092.stdout:8/531: rmdir d9/dc/d4a/d5f/da8 39 2024-05-06T12:14:36.160 INFO:tasks.workunit.client.0.smithi092.stdout:6/588: stat d4/d24/d4e/l60 0 2024-05-06T12:14:36.161 INFO:tasks.workunit.client.0.smithi092.stdout:6/589: chown d4/d20/c1f 6 1 2024-05-06T12:14:36.162 INFO:tasks.workunit.client.0.smithi092.stdout:6/590: chown d4/d20/d23/d30/f5e 1 1 2024-05-06T12:14:36.162 INFO:tasks.workunit.client.1.smithi190.stdout:3/20: unlink c3 0 2024-05-06T12:14:36.163 INFO:tasks.workunit.client.1.smithi190.stdout:8/54: fsync d3/fc 0 2024-05-06T12:14:36.163 INFO:tasks.workunit.client.0.smithi092.stdout:6/591: stat d4/d24/d4e/c72 0 2024-05-06T12:14:36.164 INFO:tasks.workunit.client.0.smithi092.stdout:0/644: rmdir d7/dc/d6f/da5 39 2024-05-06T12:14:36.164 INFO:tasks.workunit.client.1.smithi190.stdout:3/21: chown l1 18086 1 2024-05-06T12:14:36.166 INFO:tasks.workunit.client.0.smithi092.stdout:2/336: creat df/d15/d50/d59/f65 x:0 0 0 2024-05-06T12:14:36.168 INFO:tasks.workunit.client.0.smithi092.stdout:4/565: truncate d9/d2b/f8f 2064126 0 2024-05-06T12:14:36.169 INFO:tasks.workunit.client.1.smithi190.stdout:8/55: rename d3 to d3/d8/d11 22 2024-05-06T12:14:36.170 INFO:tasks.workunit.client.0.smithi092.stdout:4/566: write d9/df/d62/f68 [757576,51601] 0 2024-05-06T12:14:36.170 INFO:tasks.workunit.client.1.smithi190.stdout:9/68: dwrite db/dd/df/f1b [0,4194304] 0 2024-05-06T12:14:36.171 INFO:tasks.workunit.client.0.smithi092.stdout:6/592: creat d4/d4c/d91/d93/d9d/fbf x:0 0 0 2024-05-06T12:14:36.173 INFO:tasks.workunit.client.0.smithi092.stdout:7/627: dread f3 [0,4194304] 0 2024-05-06T12:14:36.174 INFO:tasks.workunit.client.1.smithi190.stdout:0/43: dread f3 [0,4194304] 0 2024-05-06T12:14:36.175 INFO:tasks.workunit.client.0.smithi092.stdout:0/645: rename d7/f70 to d7/dc/d28/d54/d1e/d26/d4e/fce 0 2024-05-06T12:14:36.175 INFO:tasks.workunit.client.0.smithi092.stdout:7/628: chown d9/de/d10/d17/d21/d2e 209 1 2024-05-06T12:14:36.176 INFO:tasks.workunit.client.1.smithi190.stdout:8/56: creat d3/f12 x:0 0 0 2024-05-06T12:14:36.177 INFO:tasks.workunit.client.0.smithi092.stdout:2/337: creat df/d5c/f66 x:0 0 0 2024-05-06T12:14:36.177 INFO:tasks.workunit.client.1.smithi190.stdout:3/22: dread f2 [0,4194304] 0 2024-05-06T12:14:36.181 INFO:tasks.workunit.client.1.smithi190.stdout:3/23: mkdir d5 0 2024-05-06T12:14:36.182 INFO:tasks.workunit.client.0.smithi092.stdout:0/646: unlink d7/f4f 0 2024-05-06T12:14:36.186 INFO:tasks.workunit.client.1.smithi190.stdout:3/24: unlink c4 0 2024-05-06T12:14:36.186 INFO:tasks.workunit.client.1.smithi190.stdout:3/25: rename d5 to d5/d6 22 2024-05-06T12:14:36.186 INFO:tasks.workunit.client.0.smithi092.stdout:0/647: chown d7/dc/d28/d54/d12/f93 281522 1 2024-05-06T12:14:36.187 INFO:tasks.workunit.client.0.smithi092.stdout:6/593: rename d4/d4c/d52/d62 to d4/d24/d26/dae/dc0 0 2024-05-06T12:14:36.187 INFO:tasks.workunit.client.0.smithi092.stdout:2/338: rmdir df/d1b 39 2024-05-06T12:14:36.187 INFO:tasks.workunit.client.0.smithi092.stdout:7/629: mknod d9/de/d10/d17/d21/d2e/d9c/ccd 0 2024-05-06T12:14:36.189 INFO:tasks.workunit.client.0.smithi092.stdout:2/339: chown df/d1b/l31 3 1 2024-05-06T12:14:36.190 INFO:tasks.workunit.client.1.smithi190.stdout:3/26: dwrite f2 [0,4194304] 0 2024-05-06T12:14:36.191 INFO:tasks.workunit.client.0.smithi092.stdout:0/648: fdatasync d7/dc/d6f/fc2 0 2024-05-06T12:14:36.191 INFO:tasks.workunit.client.0.smithi092.stdout:7/630: dread d9/de/d10/d17/d21/d38/f89 [0,4194304] 0 2024-05-06T12:14:36.193 INFO:tasks.workunit.client.0.smithi092.stdout:7/631: chown d9/de/d1f/d37/d48/d65/f66 3 1 2024-05-06T12:14:36.194 INFO:tasks.workunit.client.1.smithi190.stdout:3/27: mkdir d5/d7 0 2024-05-06T12:14:36.195 INFO:tasks.workunit.client.0.smithi092.stdout:2/340: read df/f39 [2407298,45849] 0 2024-05-06T12:14:36.195 INFO:tasks.workunit.client.0.smithi092.stdout:6/594: getdents d4/d20/d23 0 2024-05-06T12:14:36.196 INFO:tasks.workunit.client.1.smithi190.stdout:3/28: getdents d5/d7 0 2024-05-06T12:14:36.197 INFO:tasks.workunit.client.0.smithi092.stdout:0/649: creat d7/dc/d28/d54/d12/d8f/dad/fcf x:0 0 0 2024-05-06T12:14:36.198 INFO:tasks.workunit.client.0.smithi092.stdout:0/650: dread - d7/dc/d28/d64/f89 zero size 2024-05-06T12:14:36.200 INFO:tasks.workunit.client.0.smithi092.stdout:7/632: mknod d9/de/cce 0 2024-05-06T12:14:36.201 INFO:tasks.workunit.client.1.smithi190.stdout:3/29: rename l1 to d5/l8 0 2024-05-06T12:14:36.201 INFO:tasks.workunit.client.0.smithi092.stdout:1/467: dread d3/d10/d13/f3b [0,4194304] 0 2024-05-06T12:14:36.202 INFO:tasks.workunit.client.0.smithi092.stdout:7/633: chown d9/d62/d9a/fac 409529302 1 2024-05-06T12:14:36.203 INFO:tasks.workunit.client.1.smithi190.stdout:3/30: symlink d5/d7/l9 0 2024-05-06T12:14:36.203 INFO:tasks.workunit.client.0.smithi092.stdout:6/595: fdatasync d4/d24/f3e 0 2024-05-06T12:14:36.204 INFO:tasks.workunit.client.0.smithi092.stdout:4/567: dread d9/d27/f4f [0,4194304] 0 2024-05-06T12:14:36.204 INFO:tasks.workunit.client.0.smithi092.stdout:0/651: creat d7/dc/d6f/da2/fd0 x:0 0 0 2024-05-06T12:14:36.205 INFO:tasks.workunit.client.0.smithi092.stdout:9/461: truncate d16/d2c/d2f/f36 128804 0 2024-05-06T12:14:36.205 INFO:tasks.workunit.client.1.smithi190.stdout:3/31: mkdir d5/d7/da 0 2024-05-06T12:14:36.207 INFO:tasks.workunit.client.0.smithi092.stdout:8/532: dread d9/dc/f5d [0,4194304] 0 2024-05-06T12:14:36.208 INFO:tasks.workunit.client.0.smithi092.stdout:9/462: stat d16/d2c/d2f/d3d/d33/f87 0 2024-05-06T12:14:36.208 INFO:tasks.workunit.client.0.smithi092.stdout:2/341: fsync df/f37 0 2024-05-06T12:14:36.210 INFO:tasks.workunit.client.0.smithi092.stdout:7/634: mkdir d9/de/d1f/dcf 0 2024-05-06T12:14:36.210 INFO:tasks.workunit.client.1.smithi190.stdout:3/32: dwrite f2 [0,4194304] 0 2024-05-06T12:14:36.212 INFO:tasks.workunit.client.1.smithi190.stdout:3/33: mkdir d5/d7/db 0 2024-05-06T12:14:36.214 INFO:tasks.workunit.client.1.smithi190.stdout:8/57: rmdir d3 39 2024-05-06T12:14:36.215 INFO:tasks.workunit.client.0.smithi092.stdout:6/596: dwrite d4/d4c/f5c [0,4194304] 0 2024-05-06T12:14:36.217 INFO:tasks.workunit.client.1.smithi190.stdout:0/44: truncate f3 1417365 0 2024-05-06T12:14:36.220 INFO:tasks.workunit.client.0.smithi092.stdout:1/468: unlink d3/l16 0 2024-05-06T12:14:36.220 INFO:tasks.workunit.client.0.smithi092.stdout:6/597: write d4/d24/f3e [2758302,85673] 0 2024-05-06T12:14:36.220 INFO:tasks.workunit.client.0.smithi092.stdout:0/652: dwrite d7/dc/d28/d54/d1e/d26/d96/fba [0,4194304] 0 2024-05-06T12:14:36.220 INFO:tasks.workunit.client.1.smithi190.stdout:9/69: truncate f4 885280 0 2024-05-06T12:14:36.221 INFO:tasks.workunit.client.0.smithi092.stdout:4/568: mkdir d9/d27/da8/db3/dbf 0 2024-05-06T12:14:36.221 INFO:tasks.workunit.client.0.smithi092.stdout:3/427: write d8/d24/f65 [860044,119895] 0 2024-05-06T12:14:36.224 INFO:tasks.workunit.client.0.smithi092.stdout:1/469: chown d3/d10/d13/d19/d21/d88/d8b/f8c 255 1 2024-05-06T12:14:36.224 INFO:tasks.workunit.client.1.smithi190.stdout:0/45: dread f7 [0,4194304] 0 2024-05-06T12:14:36.224 INFO:tasks.workunit.client.0.smithi092.stdout:3/428: read - d8/db/d4c/d40/d67/f80 zero size 2024-05-06T12:14:36.225 INFO:tasks.workunit.client.1.smithi190.stdout:8/58: dread d3/f6 [4194304,4194304] 0 2024-05-06T12:14:36.225 INFO:tasks.workunit.client.1.smithi190.stdout:0/46: chown f7 32045 1 2024-05-06T12:14:36.225 INFO:tasks.workunit.client.1.smithi190.stdout:3/34: rmdir d5/d7/da 0 2024-05-06T12:14:36.226 INFO:tasks.workunit.client.0.smithi092.stdout:7/635: unlink f1 0 2024-05-06T12:14:36.227 INFO:tasks.workunit.client.0.smithi092.stdout:1/470: fdatasync d3/d42/d84/f50 0 2024-05-06T12:14:36.227 INFO:tasks.workunit.client.1.smithi190.stdout:8/59: mkdir d3/d13 0 2024-05-06T12:14:36.228 INFO:tasks.workunit.client.1.smithi190.stdout:8/60: write d3/f12 [943664,49272] 0 2024-05-06T12:14:36.229 INFO:tasks.workunit.client.1.smithi190.stdout:8/61: readlink d3/d8/lf 0 2024-05-06T12:14:36.230 INFO:tasks.workunit.client.1.smithi190.stdout:3/35: link f2 d5/fc 0 2024-05-06T12:14:36.231 INFO:tasks.workunit.client.0.smithi092.stdout:2/342: creat df/d17/d51/f67 x:0 0 0 2024-05-06T12:14:36.231 INFO:tasks.workunit.client.1.smithi190.stdout:0/47: creat d9/df/f10 x:0 0 0 2024-05-06T12:14:36.233 INFO:tasks.workunit.client.1.smithi190.stdout:8/62: creat d3/d8/f14 x:0 0 0 2024-05-06T12:14:36.235 INFO:tasks.workunit.client.1.smithi190.stdout:3/36: write f2 [720714,96855] 0 2024-05-06T12:14:36.236 INFO:tasks.workunit.client.1.smithi190.stdout:0/48: fsync f7 0 2024-05-06T12:14:36.238 INFO:tasks.workunit.client.1.smithi190.stdout:8/63: creat d3/d8/f15 x:0 0 0 2024-05-06T12:14:36.239 INFO:tasks.workunit.client.0.smithi092.stdout:7/636: symlink d9/de/d10/d17/d99/db6/ld0 0 2024-05-06T12:14:36.241 INFO:tasks.workunit.client.0.smithi092.stdout:1/471: creat d3/d42/d84/d4f/f97 x:0 0 0 2024-05-06T12:14:36.241 INFO:tasks.workunit.client.1.smithi190.stdout:3/37: rename f2 to d5/d7/db/fd 0 2024-05-06T12:14:36.243 INFO:tasks.workunit.client.1.smithi190.stdout:8/64: chown d3/l5 32188 1 2024-05-06T12:14:36.244 INFO:tasks.workunit.client.1.smithi190.stdout:0/49: dwrite d9/df/f10 [0,4194304] 0 2024-05-06T12:14:36.245 INFO:tasks.workunit.client.1.smithi190.stdout:0/50: write d9/df/f10 [3949305,17505] 0 2024-05-06T12:14:36.246 INFO:tasks.workunit.client.0.smithi092.stdout:0/653: creat d7/dc/d28/d54/d1e/d39/da8/dc5/fd1 x:0 0 0 2024-05-06T12:14:36.246 INFO:tasks.workunit.client.1.smithi190.stdout:0/51: stat d9/ca 0 2024-05-06T12:14:36.247 INFO:tasks.workunit.client.0.smithi092.stdout:8/533: link d9/c4b d9/d21/d54/d79/cc9 0 2024-05-06T12:14:36.247 INFO:tasks.workunit.client.1.smithi190.stdout:3/38: rmdir d5/d7/db 39 2024-05-06T12:14:36.248 INFO:tasks.workunit.client.0.smithi092.stdout:1/472: dwrite d3/d42/d84/f7a [0,4194304] 0 2024-05-06T12:14:36.251 INFO:tasks.workunit.client.0.smithi092.stdout:9/463: rename d16/d2c/d2f/f36 to d16/d2c/f91 0 2024-05-06T12:14:36.255 INFO:tasks.workunit.client.0.smithi092.stdout:8/534: dread d9/d21/d37/d46/d5c/d71/f80 [4194304,4194304] 0 2024-05-06T12:14:36.257 INFO:tasks.workunit.client.1.smithi190.stdout:8/65: dwrite d3/d8/f10 [0,4194304] 0 2024-05-06T12:14:36.259 INFO:tasks.workunit.client.0.smithi092.stdout:0/654: rename d7/dc/d6f/da2/fac to d7/dc/d6f/da2/fd2 0 2024-05-06T12:14:36.261 INFO:tasks.workunit.client.1.smithi190.stdout:0/52: rename f7 to d9/dc/f11 0 2024-05-06T12:14:36.261 INFO:tasks.workunit.client.0.smithi092.stdout:0/655: write d7/dc/d28/d54/d1e/d26/d96/fba [629205,4149] 0 2024-05-06T12:14:36.261 INFO:tasks.workunit.client.1.smithi190.stdout:3/39: dread d5/d7/db/fd [0,4194304] 0 2024-05-06T12:14:36.262 INFO:tasks.workunit.client.0.smithi092.stdout:6/598: dread d4/d24/d38/d5d/d6d/fa5 [0,4194304] 0 2024-05-06T12:14:36.265 INFO:tasks.workunit.client.0.smithi092.stdout:9/464: dread - d16/d2c/d2f/d3d/d33/d6b/f77 zero size 2024-05-06T12:14:36.266 INFO:tasks.workunit.client.0.smithi092.stdout:7/637: creat d9/de/d10/d17/d6a/fd1 x:0 0 0 2024-05-06T12:14:36.267 INFO:tasks.workunit.client.0.smithi092.stdout:7/638: readlink d9/de/l1c 0 2024-05-06T12:14:36.267 INFO:tasks.workunit.client.1.smithi190.stdout:3/40: creat d5/d7/db/fe x:0 0 0 2024-05-06T12:14:36.268 INFO:tasks.workunit.client.0.smithi092.stdout:2/343: getdents df/d17/d1d/d33/d4d 0 2024-05-06T12:14:36.269 INFO:tasks.workunit.client.0.smithi092.stdout:8/535: rmdir d9/d21/d37/d46/d5c/d71/d75 39 2024-05-06T12:14:36.271 INFO:tasks.workunit.client.1.smithi190.stdout:3/41: creat d5/ff x:0 0 0 2024-05-06T12:14:36.274 INFO:tasks.workunit.client.1.smithi190.stdout:8/66: dwrite d3/f6 [0,4194304] 0 2024-05-06T12:14:36.276 INFO:tasks.workunit.client.0.smithi092.stdout:0/656: mkdir d7/dc/d28/d54/d12/d8f/dad/dd3 0 2024-05-06T12:14:36.276 INFO:tasks.workunit.client.1.smithi190.stdout:3/42: mkdir d5/d7/db/d10 0 2024-05-06T12:14:36.276 INFO:tasks.workunit.client.1.smithi190.stdout:8/67: dread - d3/d8/f14 zero size 2024-05-06T12:14:36.277 INFO:tasks.workunit.client.1.smithi190.stdout:0/53: dwrite d9/dc/f11 [0,4194304] 0 2024-05-06T12:14:36.277 INFO:tasks.workunit.client.0.smithi092.stdout:8/536: dwrite d9/d5a/d9f/fbe [0,4194304] 0 2024-05-06T12:14:36.281 INFO:tasks.workunit.client.0.smithi092.stdout:3/429: write d8/db/d14/f50 [3997699,62848] 0 2024-05-06T12:14:36.282 INFO:tasks.workunit.client.1.smithi190.stdout:9/70: truncate f4 871819 0 2024-05-06T12:14:36.282 INFO:tasks.workunit.client.1.smithi190.stdout:9/71: fdatasync f5 0 2024-05-06T12:14:36.285 INFO:tasks.workunit.client.1.smithi190.stdout:9/72: dread - db/f11 zero size 2024-05-06T12:14:36.285 INFO:tasks.workunit.client.1.smithi190.stdout:8/68: creat d3/d8/f16 x:0 0 0 2024-05-06T12:14:36.287 INFO:tasks.workunit.client.0.smithi092.stdout:4/569: rename d9/d2b/d65/d83/fba to d9/fc0 0 2024-05-06T12:14:36.287 INFO:tasks.workunit.client.1.smithi190.stdout:3/43: mkdir d5/d7/db/d10/d11 0 2024-05-06T12:14:36.288 INFO:tasks.workunit.client.1.smithi190.stdout:3/44: fsync d5/ff 0 2024-05-06T12:14:36.289 INFO:tasks.workunit.client.0.smithi092.stdout:1/473: link d3/d10/d13/f1f d3/f98 0 2024-05-06T12:14:36.292 INFO:tasks.workunit.client.1.smithi190.stdout:3/45: fdatasync d5/fc 0 2024-05-06T12:14:36.292 INFO:tasks.workunit.client.1.smithi190.stdout:8/69: getdents d3/d13 0 2024-05-06T12:14:36.294 INFO:tasks.workunit.client.0.smithi092.stdout:7/639: mkdir d9/de/d10/d17/d99/dc3/dd2 0 2024-05-06T12:14:36.295 INFO:tasks.workunit.client.0.smithi092.stdout:7/640: chown d9/da5/c2f 14655746 1 2024-05-06T12:14:36.295 INFO:tasks.workunit.client.0.smithi092.stdout:3/430: chown d8/db/d14/d5e/c79 2571668 1 2024-05-06T12:14:36.296 INFO:tasks.workunit.client.1.smithi190.stdout:3/46: dread d5/fc [0,4194304] 0 2024-05-06T12:14:36.296 INFO:tasks.workunit.client.1.smithi190.stdout:8/70: link d3/f12 d3/f17 0 2024-05-06T12:14:36.299 INFO:tasks.workunit.client.1.smithi190.stdout:3/47: symlink d5/d7/db/d10/l12 0 2024-05-06T12:14:36.299 INFO:tasks.workunit.client.1.smithi190.stdout:3/48: write d5/fc [4239050,7594] 0 2024-05-06T12:14:36.305 INFO:tasks.workunit.client.0.smithi092.stdout:1/474: unlink d3/c18 0 2024-05-06T12:14:36.306 INFO:tasks.workunit.client.0.smithi092.stdout:6/599: getdents d4/d24/d26 0 2024-05-06T12:14:36.306 INFO:tasks.workunit.client.1.smithi190.stdout:3/49: dwrite d5/ff [0,4194304] 0 2024-05-06T12:14:36.307 INFO:tasks.workunit.client.0.smithi092.stdout:8/537: creat d9/d21/d37/d46/d5c/d71/fca x:0 0 0 2024-05-06T12:14:36.307 INFO:tasks.workunit.client.0.smithi092.stdout:6/600: stat d4/d20/d22/f9e 0 2024-05-06T12:14:36.307 INFO:tasks.workunit.client.0.smithi092.stdout:7/641: symlink d9/dd/ld3 0 2024-05-06T12:14:36.309 INFO:tasks.workunit.client.0.smithi092.stdout:2/344: getdents df/d17 0 2024-05-06T12:14:36.310 INFO:tasks.workunit.client.1.smithi190.stdout:3/50: creat d5/d7/db/d10/d11/f13 x:0 0 0 2024-05-06T12:14:36.310 INFO:tasks.workunit.client.1.smithi190.stdout:3/51: chown d5/d7/db/fe 19 1 2024-05-06T12:14:36.311 INFO:tasks.workunit.client.0.smithi092.stdout:3/431: dwrite d8/d24/d27/f2f [4194304,4194304] 0 2024-05-06T12:14:36.311 INFO:tasks.workunit.client.0.smithi092.stdout:2/345: readlink df/l3a 0 2024-05-06T12:14:36.311 INFO:tasks.workunit.client.0.smithi092.stdout:6/601: fdatasync d4/d24/d26/f4b 0 2024-05-06T12:14:36.312 INFO:tasks.workunit.client.1.smithi190.stdout:3/52: truncate d5/d7/db/fe 410525 0 2024-05-06T12:14:36.312 INFO:tasks.workunit.client.0.smithi092.stdout:7/642: write d9/de/d10/d17/fb5 [4107038,61643] 0 2024-05-06T12:14:36.313 INFO:tasks.workunit.client.1.smithi190.stdout:0/54: truncate d9/dc/f11 1330902 0 2024-05-06T12:14:36.317 INFO:tasks.workunit.client.1.smithi190.stdout:8/71: truncate d3/d8/f10 788546 0 2024-05-06T12:14:36.320 INFO:tasks.workunit.client.0.smithi092.stdout:3/432: dwrite d8/d1b/f3e [0,4194304] 0 2024-05-06T12:14:36.321 INFO:tasks.workunit.client.0.smithi092.stdout:3/433: write d8/d24/f77 [3476193,26033] 0 2024-05-06T12:14:36.321 INFO:tasks.workunit.client.0.smithi092.stdout:0/657: truncate d7/dc/d28/d54/d12/f82 618906 0 2024-05-06T12:14:36.321 INFO:tasks.workunit.client.1.smithi190.stdout:9/73: dwrite db/dd/f17 [0,4194304] 0 2024-05-06T12:14:36.322 INFO:tasks.workunit.client.1.smithi190.stdout:8/72: creat d3/f18 x:0 0 0 2024-05-06T12:14:36.323 INFO:tasks.workunit.client.0.smithi092.stdout:9/465: truncate d16/d2c/d2f/d3d/d33/f3b 2847258 0 2024-05-06T12:14:36.323 INFO:tasks.workunit.client.1.smithi190.stdout:8/73: chown d3 128044 1 2024-05-06T12:14:36.323 INFO:tasks.workunit.client.0.smithi092.stdout:0/658: stat d7/dc/d28/d54/d1e/d26/d3a/f52 0 2024-05-06T12:14:36.324 INFO:tasks.workunit.client.1.smithi190.stdout:8/74: chown d3/c4 0 1 2024-05-06T12:14:36.325 INFO:tasks.workunit.client.0.smithi092.stdout:0/659: chown d7/dc/d28/d54/d12/d8f 2 1 2024-05-06T12:14:36.326 INFO:tasks.workunit.client.0.smithi092.stdout:6/602: mknod d4/d4c/d52/d6c/cc1 0 2024-05-06T12:14:36.326 INFO:tasks.workunit.client.0.smithi092.stdout:0/660: write d7/dc/d9e/f1c [4803040,66526] 0 2024-05-06T12:14:36.326 INFO:tasks.workunit.client.1.smithi190.stdout:8/75: write d3/f6 [745358,11499] 0 2024-05-06T12:14:36.327 INFO:tasks.workunit.client.0.smithi092.stdout:1/475: rmdir d3/d42/d84/d4f/d58/d65/d81 39 2024-05-06T12:14:36.328 INFO:tasks.workunit.client.1.smithi190.stdout:3/53: rename d5/d7/db/d10 to d5/d14 0 2024-05-06T12:14:36.328 INFO:tasks.workunit.client.0.smithi092.stdout:4/570: write d9/d2b/f8f [1526726,115516] 0 2024-05-06T12:14:36.329 INFO:tasks.workunit.client.1.smithi190.stdout:3/54: read - d5/d14/d11/f13 zero size 2024-05-06T12:14:36.332 INFO:tasks.workunit.client.0.smithi092.stdout:8/538: creat d9/d21/d37/d46/d5c/d71/fcb x:0 0 0 2024-05-06T12:14:36.334 INFO:tasks.workunit.client.1.smithi190.stdout:9/74: mknod db/dd/c1d 0 2024-05-06T12:14:36.334 INFO:tasks.workunit.client.0.smithi092.stdout:9/466: mkdir d16/d1c/d58/d92 0 2024-05-06T12:14:36.335 INFO:tasks.workunit.client.0.smithi092.stdout:8/539: write d9/d21/fb4 [82022,120326] 0 2024-05-06T12:14:36.337 INFO:tasks.workunit.client.0.smithi092.stdout:6/603: mknod d4/d4c/d91/cc2 0 2024-05-06T12:14:36.337 INFO:tasks.workunit.client.0.smithi092.stdout:1/476: mknod d3/d10/d13/d46/c99 0 2024-05-06T12:14:36.338 INFO:tasks.workunit.client.1.smithi190.stdout:8/76: symlink d3/d13/l19 0 2024-05-06T12:14:36.341 INFO:tasks.workunit.client.1.smithi190.stdout:0/55: dwrite d9/dc/f11 [0,4194304] 0 2024-05-06T12:14:36.342 INFO:tasks.workunit.client.1.smithi190.stdout:3/55: getdents d5/d14 0 2024-05-06T12:14:36.343 INFO:tasks.workunit.client.1.smithi190.stdout:0/56: fdatasync d9/df/f10 0 2024-05-06T12:14:36.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:35 smithi190 ceph-mon[33798]: pgmap v20: 65 pgs: 65 active+clean; 961 MiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 19 MiB/s rd, 113 MiB/s wr, 370 op/s 2024-05-06T12:14:36.349 INFO:tasks.workunit.client.1.smithi190.stdout:0/57: symlink d9/dc/l12 0 2024-05-06T12:14:36.350 INFO:tasks.workunit.client.1.smithi190.stdout:9/75: dwrite db/f13 [0,4194304] 0 2024-05-06T12:14:36.353 INFO:tasks.workunit.client.1.smithi190.stdout:3/56: dread d5/fc [0,4194304] 0 2024-05-06T12:14:36.356 INFO:tasks.workunit.client.1.smithi190.stdout:9/76: rename db/fc to db/dd/d16/f1e 0 2024-05-06T12:14:36.358 INFO:tasks.workunit.client.1.smithi190.stdout:0/58: rename d9/dc/f11 to d9/dc/f13 0 2024-05-06T12:14:36.360 INFO:tasks.workunit.client.1.smithi190.stdout:9/77: creat db/f1f x:0 0 0 2024-05-06T12:14:36.361 INFO:tasks.workunit.client.1.smithi190.stdout:9/78: chown db/f1f 57772 1 2024-05-06T12:14:36.362 INFO:tasks.workunit.client.1.smithi190.stdout:0/59: write d9/dc/f13 [1711035,121288] 0 2024-05-06T12:14:36.363 INFO:tasks.workunit.client.1.smithi190.stdout:0/60: write d9/df/f10 [3349642,115378] 0 2024-05-06T12:14:36.364 INFO:tasks.workunit.client.1.smithi190.stdout:0/61: rmdir d9 39 2024-05-06T12:14:36.367 INFO:tasks.workunit.client.1.smithi190.stdout:0/62: creat d9/df/f14 x:0 0 0 2024-05-06T12:14:36.367 INFO:tasks.workunit.client.1.smithi190.stdout:0/63: chown d9/dc 257 1 2024-05-06T12:14:36.368 INFO:tasks.workunit.client.1.smithi190.stdout:0/64: creat d9/df/f15 x:0 0 0 2024-05-06T12:14:36.369 INFO:tasks.workunit.client.0.smithi092.stdout:0/661: mkdir d7/dc/dd4 0 2024-05-06T12:14:36.370 INFO:tasks.workunit.client.0.smithi092.stdout:6/604: chown d4/d24/c34 5651494 1 2024-05-06T12:14:36.370 INFO:tasks.workunit.client.0.smithi092.stdout:0/662: read d7/dc/d28/d54/d1e/d5d/f68 [529407,26011] 0 2024-05-06T12:14:36.372 INFO:tasks.workunit.client.1.smithi190.stdout:0/65: rename f3 to d9/f16 0 2024-05-06T12:14:36.373 INFO:tasks.workunit.client.1.smithi190.stdout:0/66: chown d9/dc/cd 54157 1 2024-05-06T12:14:36.373 INFO:tasks.workunit.client.0.smithi092.stdout:3/434: rmdir d8/db/d4c/d3d/d48/d66 0 2024-05-06T12:14:36.374 INFO:tasks.workunit.client.1.smithi190.stdout:0/67: write d9/fb [2932708,28825] 0 2024-05-06T12:14:36.375 INFO:tasks.workunit.client.0.smithi092.stdout:3/435: write d8/db/d4c/d3d/d49/f69 [495477,60799] 0 2024-05-06T12:14:36.375 INFO:tasks.workunit.client.0.smithi092.stdout:8/540: creat d9/d21/dc0/fcc x:0 0 0 2024-05-06T12:14:36.377 INFO:tasks.workunit.client.0.smithi092.stdout:9/467: rmdir d16/d2c/d2f/d3d 39 2024-05-06T12:14:36.377 INFO:tasks.workunit.client.1.smithi190.stdout:0/68: rmdir d9/dc 39 2024-05-06T12:14:36.377 INFO:tasks.workunit.client.0.smithi092.stdout:9/468: fsync d16/f2a 0 2024-05-06T12:14:36.378 INFO:tasks.workunit.client.1.smithi190.stdout:9/79: getdents db/dd 0 2024-05-06T12:14:36.379 INFO:tasks.workunit.client.0.smithi092.stdout:2/346: write df/d17/d1d/d33/d3c/f63 [779136,102303] 0 2024-05-06T12:14:36.380 INFO:tasks.workunit.client.0.smithi092.stdout:3/436: mknod d8/db/d4c/d40/d67/c85 0 2024-05-06T12:14:36.380 INFO:tasks.workunit.client.0.smithi092.stdout:2/347: chown df/d17/f22 554509717 1 2024-05-06T12:14:36.381 INFO:tasks.workunit.client.0.smithi092.stdout:0/663: truncate d7/dc/d28/d54/d12/f1b 5034510 0 2024-05-06T12:14:36.381 INFO:tasks.workunit.client.0.smithi092.stdout:8/541: mknod d9/d21/d37/d46/d5c/d93/ccd 0 2024-05-06T12:14:36.382 INFO:tasks.workunit.client.0.smithi092.stdout:0/664: fsync d7/dc/d6f/fa1 0 2024-05-06T12:14:36.382 INFO:tasks.workunit.client.0.smithi092.stdout:6/605: mkdir d4/d4c/d52/d6c/da0/dc3 0 2024-05-06T12:14:36.383 INFO:tasks.workunit.client.1.smithi190.stdout:8/77: write d3/d8/f10 [1523039,67130] 0 2024-05-06T12:14:36.383 INFO:tasks.workunit.client.1.smithi190.stdout:9/80: dread f0 [0,4194304] 0 2024-05-06T12:14:36.383 INFO:tasks.workunit.client.0.smithi092.stdout:0/665: fdatasync d7/dc/d28/d54/d12/f1b 0 2024-05-06T12:14:36.384 INFO:tasks.workunit.client.1.smithi190.stdout:0/69: mknod d9/dc/c17 0 2024-05-06T12:14:36.384 INFO:tasks.workunit.client.1.smithi190.stdout:0/70: readlink l8 0 2024-05-06T12:14:36.384 INFO:tasks.workunit.client.0.smithi092.stdout:6/606: write d4/d24/f55 [1856936,121821] 0 2024-05-06T12:14:36.385 INFO:tasks.workunit.client.1.smithi190.stdout:8/78: creat d3/d8/f1a x:0 0 0 2024-05-06T12:14:36.385 INFO:tasks.workunit.client.0.smithi092.stdout:9/469: dread - d16/d2c/d2f/d3d/f7d zero size 2024-05-06T12:14:36.386 INFO:tasks.workunit.client.0.smithi092.stdout:3/437: symlink d8/db/d4c/d40/l86 0 2024-05-06T12:14:36.387 INFO:tasks.workunit.client.1.smithi190.stdout:0/71: symlink d9/df/l18 0 2024-05-06T12:14:36.389 INFO:tasks.workunit.client.0.smithi092.stdout:2/348: creat df/d17/d1d/d33/f68 x:0 0 0 2024-05-06T12:14:36.390 INFO:tasks.workunit.client.0.smithi092.stdout:6/607: dread d4/d4c/d52/f6a [0,4194304] 0 2024-05-06T12:14:36.390 INFO:tasks.workunit.client.0.smithi092.stdout:2/349: write df/f12 [2363444,24035] 0 2024-05-06T12:14:36.390 INFO:tasks.workunit.client.1.smithi190.stdout:8/79: rename d3/d8/f15 to d3/d8/f1b 0 2024-05-06T12:14:36.391 INFO:tasks.workunit.client.0.smithi092.stdout:6/608: chown d4/d24/d66 53246149 1 2024-05-06T12:14:36.392 INFO:tasks.workunit.client.0.smithi092.stdout:4/571: write d9/dd/fb9 [4591495,63727] 0 2024-05-06T12:14:36.392 INFO:tasks.workunit.client.0.smithi092.stdout:8/542: dwrite d9/d21/f81 [0,4194304] 0 2024-05-06T12:14:36.393 INFO:tasks.workunit.client.1.smithi190.stdout:9/81: rmdir db/dd/d16/d19 0 2024-05-06T12:14:36.395 INFO:tasks.workunit.client.0.smithi092.stdout:8/543: chown d9/d21/d37/d46/d5c/d62/fac 0 1 2024-05-06T12:14:36.395 INFO:tasks.workunit.client.0.smithi092.stdout:6/609: truncate d4/d24/d26/dae/dc0/f71 5120925 0 2024-05-06T12:14:36.396 INFO:tasks.workunit.client.1.smithi190.stdout:8/80: rename d3/d8/f14 to d3/f1c 0 2024-05-06T12:14:36.398 INFO:tasks.workunit.client.1.smithi190.stdout:9/82: symlink db/dd/d16/l20 0 2024-05-06T12:14:36.398 INFO:tasks.workunit.client.0.smithi092.stdout:0/666: mknod d7/dc/d28/d54/d12/d7f/db2/cd5 0 2024-05-06T12:14:36.399 INFO:tasks.workunit.client.1.smithi190.stdout:0/72: getdents d9 0 2024-05-06T12:14:36.400 INFO:tasks.workunit.client.1.smithi190.stdout:9/83: mkdir db/d21 0 2024-05-06T12:14:36.402 INFO:tasks.workunit.client.1.smithi190.stdout:8/81: truncate d3/fc 591004 0 2024-05-06T12:14:36.403 INFO:tasks.workunit.client.1.smithi190.stdout:8/82: truncate d3/f18 73757 0 2024-05-06T12:14:36.404 INFO:tasks.workunit.client.1.smithi190.stdout:0/73: creat d9/df/f19 x:0 0 0 2024-05-06T12:14:36.404 INFO:tasks.workunit.client.1.smithi190.stdout:9/84: rmdir db/dd 39 2024-05-06T12:14:36.405 INFO:tasks.workunit.client.1.smithi190.stdout:8/83: write d3/f18 [862349,31995] 0 2024-05-06T12:14:36.407 INFO:tasks.workunit.client.1.smithi190.stdout:8/84: write d3/d8/fd [2430413,70584] 0 2024-05-06T12:14:36.407 INFO:tasks.workunit.client.1.smithi190.stdout:8/85: chown d3/d13 82701296 1 2024-05-06T12:14:36.412 INFO:tasks.workunit.client.1.smithi190.stdout:0/74: creat d9/f1a x:0 0 0 2024-05-06T12:14:36.414 INFO:tasks.workunit.client.1.smithi190.stdout:8/86: dwrite d3/f18 [0,4194304] 0 2024-05-06T12:14:36.415 INFO:tasks.workunit.client.1.smithi190.stdout:8/87: chown d3/f17 155627 1 2024-05-06T12:14:36.448 INFO:tasks.workunit.client.0.smithi092.stdout:4/572: fsync d9/df/f15 0 2024-05-06T12:14:36.448 INFO:tasks.workunit.client.0.smithi092.stdout:4/573: stat d9/dd/le 0 2024-05-06T12:14:36.449 INFO:tasks.workunit.client.0.smithi092.stdout:4/574: stat d9/df/d48/f74 0 2024-05-06T12:14:36.454 INFO:tasks.workunit.client.0.smithi092.stdout:8/544: symlink d9/d21/d72/lce 0 2024-05-06T12:14:36.457 INFO:tasks.workunit.client.0.smithi092.stdout:3/438: symlink d8/db/l87 0 2024-05-06T12:14:36.457 INFO:tasks.workunit.client.0.smithi092.stdout:3/439: chown d8/db/d4c/d40/d67 130547507 1 2024-05-06T12:14:36.458 INFO:tasks.workunit.client.0.smithi092.stdout:6/610: mkdir d4/d24/d26/dc4 0 2024-05-06T12:14:36.459 INFO:tasks.workunit.client.0.smithi092.stdout:6/611: stat d4/d20/d22 0 2024-05-06T12:14:36.460 INFO:tasks.workunit.client.0.smithi092.stdout:0/667: fsync f6 0 2024-05-06T12:14:36.463 INFO:tasks.workunit.client.0.smithi092.stdout:9/470: fdatasync d16/d2c/d2f/d3d/d33/f71 0 2024-05-06T12:14:36.467 INFO:tasks.workunit.client.0.smithi092.stdout:9/471: dread d16/d2c/d2f/d3d/d33/f71 [0,4194304] 0 2024-05-06T12:14:36.469 INFO:tasks.workunit.client.0.smithi092.stdout:9/472: fdatasync d16/d2c/d2f/d3d/d33/d37/d40/f5f 0 2024-05-06T12:14:36.469 INFO:tasks.workunit.client.0.smithi092.stdout:8/545: rmdir d9/d21/d37/d46/d5c/d62/d7a 39 2024-05-06T12:14:36.471 INFO:tasks.workunit.client.0.smithi092.stdout:6/612: symlink d4/d24/d26/dae/lc5 0 2024-05-06T12:14:36.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:35 smithi092 ceph-mon[26954]: pgmap v20: 65 pgs: 65 active+clean; 961 MiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 19 MiB/s rd, 113 MiB/s wr, 370 op/s 2024-05-06T12:14:36.474 INFO:tasks.workunit.client.0.smithi092.stdout:4/575: truncate d9/f47 356736 0 2024-05-06T12:14:36.478 INFO:tasks.workunit.client.0.smithi092.stdout:6/613: mkdir d4/d24/d9f/dc6 0 2024-05-06T12:14:36.480 INFO:tasks.workunit.client.0.smithi092.stdout:8/546: rename d9/c17 to d9/d21/d37/d46/d5c/d71/d75/ccf 0 2024-05-06T12:14:36.480 INFO:tasks.workunit.client.0.smithi092.stdout:0/668: mknod d7/dc/d6f/da5/dc6/cd6 0 2024-05-06T12:14:36.482 INFO:tasks.workunit.client.0.smithi092.stdout:4/576: creat d9/d2b/d65/fc1 x:0 0 0 2024-05-06T12:14:36.485 INFO:tasks.workunit.client.0.smithi092.stdout:8/547: mknod d9/dc/d4a/d5f/da8/cd0 0 2024-05-06T12:14:36.486 INFO:tasks.workunit.client.0.smithi092.stdout:0/669: rmdir d7/dc/d28/d54/d12/d8f/dad/dd3 0 2024-05-06T12:14:36.488 INFO:tasks.workunit.client.0.smithi092.stdout:0/670: mkdir d7/dc/d28/d54/d1e/d26/d4e/dd7 0 2024-05-06T12:14:36.489 INFO:tasks.workunit.client.0.smithi092.stdout:8/548: getdents d9/d21 0 2024-05-06T12:14:36.492 INFO:tasks.workunit.client.0.smithi092.stdout:8/549: rmdir d9/d21/d37/d46/d5c/d62/da5 39 2024-05-06T12:14:36.493 INFO:tasks.workunit.client.0.smithi092.stdout:0/671: getdents d7/dc/d28/d54/d1e/d46/d63/d80 0 2024-05-06T12:14:36.495 INFO:tasks.workunit.client.0.smithi092.stdout:8/550: creat d9/dc/d4a/d5f/da8/fd1 x:0 0 0 2024-05-06T12:14:36.495 INFO:tasks.workunit.client.0.smithi092.stdout:8/551: chown d9/d21/l3a 1402242 1 2024-05-06T12:14:36.497 INFO:tasks.workunit.client.0.smithi092.stdout:0/672: creat d7/dc/d28/d54/d12/fd8 x:0 0 0 2024-05-06T12:14:36.498 INFO:tasks.workunit.client.0.smithi092.stdout:0/673: chown d7/dc/d28/d64/f89 18 1 2024-05-06T12:14:36.499 INFO:tasks.workunit.client.0.smithi092.stdout:4/577: dread d9/d27/d4a/f44 [0,4194304] 0 2024-05-06T12:14:36.500 INFO:tasks.workunit.client.0.smithi092.stdout:0/674: creat d7/dc/d97/fd9 x:0 0 0 2024-05-06T12:14:36.502 INFO:tasks.workunit.client.0.smithi092.stdout:4/578: creat d9/fc2 x:0 0 0 2024-05-06T12:14:36.504 INFO:tasks.workunit.client.0.smithi092.stdout:4/579: mknod d9/d2b/d65/dbc/cc3 0 2024-05-06T12:14:36.506 INFO:tasks.workunit.client.0.smithi092.stdout:0/675: dwrite d7/dc/d28/d54/d1e/d5d/f68 [0,4194304] 0 2024-05-06T12:14:36.508 INFO:tasks.workunit.client.0.smithi092.stdout:0/676: fdatasync d7/dc/d9e/fc0 0 2024-05-06T12:14:36.541 INFO:tasks.workunit.client.1.smithi190.stdout:9/85: write f3 [196059,45606] 0 2024-05-06T12:14:36.560 INFO:tasks.workunit.client.0.smithi092.stdout:2/350: write df/d17/d1d/f46 [2444422,114547] 0 2024-05-06T12:14:36.577 INFO:tasks.workunit.client.0.smithi092.stdout:1/477: write d3/d42/d84/d4f/d58/f62 [695777,124290] 0 2024-05-06T12:14:36.578 INFO:tasks.workunit.client.0.smithi092.stdout:1/478: write d3/d10/d13/d19/d21/d88/f91 [520785,64487] 0 2024-05-06T12:14:36.580 INFO:tasks.workunit.client.0.smithi092.stdout:1/479: chown d3/d10/d13/d19/d21/f24 12303 1 2024-05-06T12:14:36.580 INFO:tasks.workunit.client.0.smithi092.stdout:1/480: chown d3/d10/d13/d46 29037918 1 2024-05-06T12:14:36.584 INFO:tasks.workunit.client.0.smithi092.stdout:1/481: write d3/d42/d84/f8e [182080,113878] 0 2024-05-06T12:14:36.593 INFO:tasks.workunit.client.0.smithi092.stdout:9/473: creat d16/d2c/d2f/d3d/d33/d37/f93 x:0 0 0 2024-05-06T12:14:36.595 INFO:tasks.workunit.client.0.smithi092.stdout:3/440: symlink d8/d24/l88 0 2024-05-06T12:14:36.596 INFO:tasks.workunit.client.0.smithi092.stdout:3/441: fdatasync d8/d24/d33/d39/f70 0 2024-05-06T12:14:36.596 INFO:tasks.workunit.client.0.smithi092.stdout:9/474: mkdir d16/d94 0 2024-05-06T12:14:36.597 INFO:tasks.workunit.client.0.smithi092.stdout:3/442: stat d8/db/d4c/d3d/d49/f69 0 2024-05-06T12:14:36.598 INFO:tasks.workunit.client.0.smithi092.stdout:3/443: write d8/db/d4c/d3d/f5d [1653918,34574] 0 2024-05-06T12:14:36.599 INFO:tasks.workunit.client.0.smithi092.stdout:5/481: sync 2024-05-06T12:14:36.601 INFO:tasks.workunit.client.0.smithi092.stdout:5/482: write d4/d5/d10/fa6 [807548,97664] 0 2024-05-06T12:14:36.602 INFO:tasks.workunit.client.0.smithi092.stdout:4/580: rename d9/d2b/f6e to d9/da2/fc4 0 2024-05-06T12:14:36.603 INFO:tasks.workunit.client.0.smithi092.stdout:4/581: write d9/dd/f87 [606493,69046] 0 2024-05-06T12:14:36.604 INFO:tasks.workunit.client.0.smithi092.stdout:5/483: rmdir d4/d34/d29/d54/d73 39 2024-05-06T12:14:36.605 INFO:tasks.workunit.client.0.smithi092.stdout:8/552: creat d9/d21/d37/d46/fd2 x:0 0 0 2024-05-06T12:14:36.605 INFO:tasks.workunit.client.0.smithi092.stdout:3/444: symlink d8/d24/d33/d4b/d74/l89 0 2024-05-06T12:14:36.606 INFO:tasks.workunit.client.0.smithi092.stdout:8/553: dread - d9/d21/dc0/fcc zero size 2024-05-06T12:14:36.608 INFO:tasks.workunit.client.0.smithi092.stdout:2/351: write df/f37 [2758893,61988] 0 2024-05-06T12:14:36.613 INFO:tasks.workunit.client.0.smithi092.stdout:4/582: mkdir d9/d27/dc5 0 2024-05-06T12:14:36.616 INFO:tasks.workunit.client.0.smithi092.stdout:2/352: mknod df/d17/d1d/d33/d3c/c69 0 2024-05-06T12:14:36.617 INFO:tasks.workunit.client.0.smithi092.stdout:8/554: truncate d9/dc/fbb 460451 0 2024-05-06T12:14:36.625 INFO:tasks.workunit.client.0.smithi092.stdout:8/555: link d9/d21/f6f d9/fd3 0 2024-05-06T12:14:36.627 INFO:tasks.workunit.client.0.smithi092.stdout:9/475: dwrite d16/d1e/f25 [0,4194304] 0 2024-05-06T12:14:36.629 INFO:tasks.workunit.client.0.smithi092.stdout:3/445: rmdir d8 39 2024-05-06T12:14:36.629 INFO:tasks.workunit.client.0.smithi092.stdout:8/556: dread d9/dc/f53 [4194304,4194304] 0 2024-05-06T12:14:36.629 INFO:tasks.workunit.client.0.smithi092.stdout:5/484: dread d4/d34/d28/d4e/d5d/d82/d1f/d20/f8d [0,4194304] 0 2024-05-06T12:14:36.629 INFO:tasks.workunit.client.0.smithi092.stdout:9/476: write fc [4495737,124583] 0 2024-05-06T12:14:36.635 INFO:tasks.workunit.client.0.smithi092.stdout:2/353: truncate df/f5b 2604291 0 2024-05-06T12:14:36.639 INFO:tasks.workunit.client.0.smithi092.stdout:5/485: rename d4/d5/d10/d31/l70 to d4/d5/d10/lb7 0 2024-05-06T12:14:36.640 INFO:tasks.workunit.client.0.smithi092.stdout:5/486: write d4/d5/d10/d31/d53/d64/f25 [1364830,19036] 0 2024-05-06T12:14:36.641 INFO:tasks.workunit.client.0.smithi092.stdout:9/477: getdents d16/d1e/d7b 0 2024-05-06T12:14:36.642 INFO:tasks.workunit.client.0.smithi092.stdout:3/446: getdents d8/d24 0 2024-05-06T12:14:36.645 INFO:tasks.workunit.client.0.smithi092.stdout:5/487: mknod d4/d5/d10/d31/cb8 0 2024-05-06T12:14:36.648 INFO:tasks.workunit.client.0.smithi092.stdout:3/447: rename d8/db/d4c/d3d/d49/f69 to d8/db/d4c/d3d/d49/f8a 0 2024-05-06T12:14:36.649 INFO:tasks.workunit.client.0.smithi092.stdout:9/478: dwrite d16/d2c/d2f/d3d/f67 [0,4194304] 0 2024-05-06T12:14:36.654 INFO:tasks.workunit.client.0.smithi092.stdout:5/488: dwrite d4/d34/d29/d54/f9c [0,4194304] 0 2024-05-06T12:14:36.656 INFO:tasks.workunit.client.0.smithi092.stdout:9/479: mknod d16/d2c/d2f/d3d/d33/d37/d40/d78/c95 0 2024-05-06T12:14:36.663 INFO:tasks.workunit.client.0.smithi092.stdout:5/489: getdents d4/d5/d30 0 2024-05-06T12:14:36.664 INFO:tasks.workunit.client.0.smithi092.stdout:5/490: readlink d4/d34/l7e 0 2024-05-06T12:14:36.665 INFO:tasks.workunit.client.0.smithi092.stdout:3/448: write d8/f12 [2849592,66685] 0 2024-05-06T12:14:36.666 INFO:tasks.workunit.client.0.smithi092.stdout:5/491: fsync d4/d34/d29/d6c/f44 0 2024-05-06T12:14:36.668 INFO:tasks.workunit.client.0.smithi092.stdout:3/449: write d8/db/d4c/d3d/f60 [3987597,54504] 0 2024-05-06T12:14:36.672 INFO:tasks.workunit.client.0.smithi092.stdout:5/492: dread d4/d5/f1d [0,4194304] 0 2024-05-06T12:14:36.676 INFO:tasks.workunit.client.0.smithi092.stdout:9/480: dwrite d16/d2c/d2f/d3d/f62 [0,4194304] 0 2024-05-06T12:14:36.678 INFO:tasks.workunit.client.0.smithi092.stdout:3/450: dwrite d8/f12 [0,4194304] 0 2024-05-06T12:14:36.681 INFO:tasks.workunit.client.0.smithi092.stdout:9/481: creat d16/d2c/d3f/f96 x:0 0 0 2024-05-06T12:14:36.683 INFO:tasks.workunit.client.0.smithi092.stdout:5/493: truncate d4/d34/d29/d6c/f51 4001699 0 2024-05-06T12:14:36.684 INFO:tasks.workunit.client.0.smithi092.stdout:3/451: symlink d8/db/d4c/d3d/d49/l8b 0 2024-05-06T12:14:36.685 INFO:tasks.workunit.client.0.smithi092.stdout:5/494: rmdir d4/d34/d28/d4e 39 2024-05-06T12:14:36.686 INFO:tasks.workunit.client.0.smithi092.stdout:9/482: symlink d16/d2c/d2f/d3d/d33/d7e/l97 0 2024-05-06T12:14:36.689 INFO:tasks.workunit.client.0.smithi092.stdout:5/495: dread d4/d5/d30/f40 [0,4194304] 0 2024-05-06T12:14:36.693 INFO:tasks.workunit.client.0.smithi092.stdout:5/496: rename d4/d5/d10/d31/d53/d64/l47 to d4/lb9 0 2024-05-06T12:14:36.694 INFO:tasks.workunit.client.0.smithi092.stdout:5/497: truncate d4/d5/f96 453771 0 2024-05-06T12:14:36.695 INFO:tasks.workunit.client.1.smithi190.stdout:9/86: fdatasync db/dd/d16/f1e 0 2024-05-06T12:14:36.696 INFO:tasks.workunit.client.1.smithi190.stdout:9/87: dread - db/f1f zero size 2024-05-06T12:14:36.696 INFO:tasks.workunit.client.1.smithi190.stdout:8/88: getdents d3/d13 0 2024-05-06T12:14:36.697 INFO:tasks.workunit.client.1.smithi190.stdout:9/88: stat db/dd/df 0 2024-05-06T12:14:36.697 INFO:tasks.workunit.client.1.smithi190.stdout:8/89: dread - d3/d8/f1a zero size 2024-05-06T12:14:36.697 INFO:tasks.workunit.client.0.smithi092.stdout:9/483: write d16/d2c/d2f/f31 [4855187,48780] 0 2024-05-06T12:14:36.698 INFO:tasks.workunit.client.1.smithi190.stdout:0/75: creat d9/f1b x:0 0 0 2024-05-06T12:14:36.698 INFO:tasks.workunit.client.0.smithi092.stdout:9/484: stat d16/d2c/l5d 0 2024-05-06T12:14:36.699 INFO:tasks.workunit.client.0.smithi092.stdout:9/485: fsync d16/d2c/d2f/d3d/f67 0 2024-05-06T12:14:36.699 INFO:tasks.workunit.client.1.smithi190.stdout:9/89: mkdir db/d21/d22 0 2024-05-06T12:14:36.701 INFO:tasks.workunit.client.1.smithi190.stdout:0/76: write d9/df/f10 [5212864,53405] 0 2024-05-06T12:14:36.703 INFO:tasks.workunit.client.1.smithi190.stdout:8/90: rename d3/c4 to d3/d8/c1d 0 2024-05-06T12:14:36.704 INFO:tasks.workunit.client.1.smithi190.stdout:8/91: write d3/d8/fd [1302806,15791] 0 2024-05-06T12:14:36.705 INFO:tasks.workunit.client.1.smithi190.stdout:8/92: chown d3/d8/lf 10771 1 2024-05-06T12:14:36.706 INFO:tasks.workunit.client.1.smithi190.stdout:8/93: write d3/d8/f16 [353742,76219] 0 2024-05-06T12:14:36.706 INFO:tasks.workunit.client.1.smithi190.stdout:0/77: creat d9/df/f1c x:0 0 0 2024-05-06T12:14:36.708 INFO:tasks.workunit.client.1.smithi190.stdout:0/78: dread - d9/df/f19 zero size 2024-05-06T12:14:36.708 INFO:tasks.workunit.client.1.smithi190.stdout:0/79: stat l8 0 2024-05-06T12:14:36.709 INFO:tasks.workunit.client.1.smithi190.stdout:9/90: mkdir db/d23 0 2024-05-06T12:14:36.710 INFO:tasks.workunit.client.1.smithi190.stdout:9/91: chown db/f13 1964035 1 2024-05-06T12:14:36.711 INFO:tasks.workunit.client.1.smithi190.stdout:8/94: dread d3/f17 [0,4194304] 0 2024-05-06T12:14:36.711 INFO:tasks.workunit.client.1.smithi190.stdout:8/95: stat d3/d8/f10 0 2024-05-06T12:14:36.712 INFO:tasks.workunit.client.1.smithi190.stdout:9/92: creat db/dd/d16/f24 x:0 0 0 2024-05-06T12:14:36.714 INFO:tasks.workunit.client.1.smithi190.stdout:8/96: symlink d3/l1e 0 2024-05-06T12:14:36.716 INFO:tasks.workunit.client.1.smithi190.stdout:0/80: link d9/df/f10 d9/f1d 0 2024-05-06T12:14:36.717 INFO:tasks.workunit.client.1.smithi190.stdout:9/93: mkdir db/d23/d25 0 2024-05-06T12:14:36.718 INFO:tasks.workunit.client.1.smithi190.stdout:9/94: write db/dd/f17 [2398584,24730] 0 2024-05-06T12:14:36.719 INFO:tasks.workunit.client.1.smithi190.stdout:9/95: truncate db/f11 338348 0 2024-05-06T12:14:36.721 INFO:tasks.workunit.client.1.smithi190.stdout:0/81: unlink d9/df/f14 0 2024-05-06T12:14:36.722 INFO:tasks.workunit.client.1.smithi190.stdout:0/82: write d9/df/f15 [797415,104382] 0 2024-05-06T12:14:36.728 INFO:tasks.workunit.client.1.smithi190.stdout:9/96: dwrite db/dd/df/f1b [4194304,4194304] 0 2024-05-06T12:14:36.729 INFO:tasks.workunit.client.1.smithi190.stdout:9/97: readlink la 0 2024-05-06T12:14:36.736 INFO:tasks.workunit.client.1.smithi190.stdout:9/98: creat db/f26 x:0 0 0 2024-05-06T12:14:36.739 INFO:tasks.workunit.client.1.smithi190.stdout:9/99: rename db/le to db/d23/d25/l27 0 2024-05-06T12:14:36.742 INFO:tasks.workunit.client.1.smithi190.stdout:9/100: unlink f0 0 2024-05-06T12:14:36.744 INFO:tasks.workunit.client.1.smithi190.stdout:9/101: symlink db/dd/df/l28 0 2024-05-06T12:14:36.745 INFO:tasks.workunit.client.1.smithi190.stdout:9/102: write f5 [333175,96757] 0 2024-05-06T12:14:36.748 INFO:tasks.workunit.client.1.smithi190.stdout:9/103: mkdir db/d23/d29 0 2024-05-06T12:14:36.750 INFO:tasks.workunit.client.1.smithi190.stdout:9/104: creat db/d23/f2a x:0 0 0 2024-05-06T12:14:36.753 INFO:tasks.workunit.client.1.smithi190.stdout:9/105: creat db/f2b x:0 0 0 2024-05-06T12:14:36.754 INFO:tasks.workunit.client.1.smithi190.stdout:9/106: truncate db/f26 240341 0 2024-05-06T12:14:36.762 INFO:tasks.workunit.client.1.smithi190.stdout:9/107: dwrite db/f2b [0,4194304] 0 2024-05-06T12:14:36.762 INFO:tasks.workunit.client.0.smithi092.stdout:5/498: rmdir d4/d34/d28/d4e/d5d/d82/d77/da7 0 2024-05-06T12:14:36.763 INFO:tasks.workunit.client.0.smithi092.stdout:7/643: sync 2024-05-06T12:14:36.763 INFO:tasks.workunit.client.0.smithi092.stdout:6/614: sync 2024-05-06T12:14:36.764 INFO:tasks.workunit.client.1.smithi190.stdout:9/108: creat db/dd/df/f2c x:0 0 0 2024-05-06T12:14:36.768 INFO:tasks.workunit.client.1.smithi190.stdout:9/109: rename db/c10 to db/dd/c2d 0 2024-05-06T12:14:36.771 INFO:tasks.workunit.client.0.smithi092.stdout:7/644: rmdir d9/de/d10/d17/d99/da6 39 2024-05-06T12:14:36.773 INFO:tasks.workunit.client.0.smithi092.stdout:9/486: truncate f6 4597561 0 2024-05-06T12:14:36.774 INFO:tasks.workunit.client.0.smithi092.stdout:9/487: chown d16/d1c 286956215 1 2024-05-06T12:14:36.774 INFO:tasks.workunit.client.0.smithi092.stdout:6/615: mknod d4/d24/d26/dae/dc0/cc7 0 2024-05-06T12:14:36.777 INFO:tasks.workunit.client.1.smithi190.stdout:9/110: dwrite db/d23/f2a [0,4194304] 0 2024-05-06T12:14:36.777 INFO:tasks.workunit.client.0.smithi092.stdout:6/616: creat d4/d4c/fc8 x:0 0 0 2024-05-06T12:14:36.780 INFO:tasks.workunit.client.1.smithi190.stdout:9/111: rmdir db/d23/d25 39 2024-05-06T12:14:36.785 INFO:tasks.workunit.client.0.smithi092.stdout:5/499: link d4/d5/f3b d4/d5/d10/fba 0 2024-05-06T12:14:36.788 INFO:tasks.workunit.client.0.smithi092.stdout:6/617: dwrite d4/d4c/d52/f4a [4194304,4194304] 0 2024-05-06T12:14:36.791 INFO:tasks.workunit.client.0.smithi092.stdout:7/645: dwrite d9/de/d10/d17/d21/d38/f90 [0,4194304] 0 2024-05-06T12:14:36.791 INFO:tasks.workunit.client.1.smithi190.stdout:8/97: write d3/f12 [1403552,63750] 0 2024-05-06T12:14:36.793 INFO:tasks.workunit.client.0.smithi092.stdout:6/618: write d4/d20/d23/d30/f33 [3141087,60163] 0 2024-05-06T12:14:36.793 INFO:tasks.workunit.client.1.smithi190.stdout:8/98: rmdir d3 39 2024-05-06T12:14:36.794 INFO:tasks.workunit.client.0.smithi092.stdout:5/500: fdatasync d4/d5/d10/fba 0 2024-05-06T12:14:36.796 INFO:tasks.workunit.client.0.smithi092.stdout:5/501: read d4/d34/d28/d4e/d5d/d82/d1f/d20/f8d [3929496,56501] 0 2024-05-06T12:14:36.797 INFO:tasks.workunit.client.1.smithi190.stdout:8/99: rmdir d3/d8 39 2024-05-06T12:14:36.799 INFO:tasks.workunit.client.0.smithi092.stdout:9/488: truncate d16/d2c/d2f/d3d/d33/d57/f82 542868 0 2024-05-06T12:14:36.800 INFO:tasks.workunit.client.1.smithi190.stdout:8/100: getdents d3/d13 0 2024-05-06T12:14:36.801 INFO:tasks.workunit.client.0.smithi092.stdout:6/619: creat d4/d24/d26/dae/dc0/fc9 x:0 0 0 2024-05-06T12:14:36.802 INFO:tasks.workunit.client.1.smithi190.stdout:8/101: readlink d3/d8/lf 0 2024-05-06T12:14:36.803 INFO:tasks.workunit.client.0.smithi092.stdout:6/620: chown d4/d24/d26/dae/dc0/l8d 595773439 1 2024-05-06T12:14:36.803 INFO:tasks.workunit.client.0.smithi092.stdout:9/489: mknod d16/d2c/d2f/d3d/d33/d7e/c98 0 2024-05-06T12:14:36.804 INFO:tasks.workunit.client.0.smithi092.stdout:5/502: dwrite d4/d34/d28/d4e/d4b/f8c [0,4194304] 0 2024-05-06T12:14:36.804 INFO:tasks.workunit.client.1.smithi190.stdout:0/83: truncate d9/dc/f13 1424581 0 2024-05-06T12:14:36.806 INFO:tasks.workunit.client.0.smithi092.stdout:6/621: truncate d4/d24/d38/f69 4348130 0 2024-05-06T12:14:36.806 INFO:tasks.workunit.client.0.smithi092.stdout:7/646: rename d9/de/d10/d17/d6a/f6d to d9/de/d10/d17/d21/fd4 0 2024-05-06T12:14:36.806 INFO:tasks.workunit.client.1.smithi190.stdout:0/84: chown d9/f16 3233 1 2024-05-06T12:14:36.807 INFO:tasks.workunit.client.0.smithi092.stdout:9/490: symlink d16/d2c/d2f/d75/l99 0 2024-05-06T12:14:36.807 INFO:tasks.workunit.client.0.smithi092.stdout:6/622: chown d4/d20/c63 1 1 2024-05-06T12:14:36.809 INFO:tasks.workunit.client.0.smithi092.stdout:7/647: write d9/de/d10/d17/d21/d2e/d9c/fbe [601066,80104] 0 2024-05-06T12:14:36.809 INFO:tasks.workunit.client.1.smithi190.stdout:8/102: dwrite d3/d8/f10 [0,4194304] 0 2024-05-06T12:14:36.810 INFO:tasks.workunit.client.0.smithi092.stdout:5/503: mkdir d4/d34/d29/d54/db0/dbb 0 2024-05-06T12:14:36.812 INFO:tasks.workunit.client.1.smithi190.stdout:0/85: symlink d9/l1e 0 2024-05-06T12:14:36.813 INFO:tasks.workunit.client.1.smithi190.stdout:0/86: readlink d9/df/l18 0 2024-05-06T12:14:36.815 INFO:tasks.workunit.client.0.smithi092.stdout:6/623: creat d4/d24/d26/dc4/fca x:0 0 0 2024-05-06T12:14:36.816 INFO:tasks.workunit.client.0.smithi092.stdout:7/648: mknod d9/de/d10/d17/d21/d38/cd5 0 2024-05-06T12:14:36.817 INFO:tasks.workunit.client.1.smithi190.stdout:0/87: truncate d9/df/f15 1485203 0 2024-05-06T12:14:36.819 INFO:tasks.workunit.client.0.smithi092.stdout:5/504: symlink d4/d5/d10/da1/lbc 0 2024-05-06T12:14:36.821 INFO:tasks.workunit.client.0.smithi092.stdout:6/624: rmdir d4/d4c/da9 39 2024-05-06T12:14:36.821 INFO:tasks.workunit.client.1.smithi190.stdout:8/103: dwrite d3/d8/fd [4194304,4194304] 0 2024-05-06T12:14:36.823 INFO:tasks.workunit.client.0.smithi092.stdout:9/491: rename d16/d1e/f42 to d16/d2c/d2f/d3d/d33/f9a 0 2024-05-06T12:14:36.823 INFO:tasks.workunit.client.1.smithi190.stdout:0/88: rename d9/df/f1c to d9/dc/f1f 0 2024-05-06T12:14:36.829 INFO:tasks.workunit.client.0.smithi092.stdout:9/492: rename d16/d2c/d3f/f5e to d16/d2c/d2f/d3d/d33/d37/d40/d43/f9b 0 2024-05-06T12:14:36.830 INFO:tasks.workunit.client.1.smithi190.stdout:8/104: link d3/f12 d3/d8/f1f 0 2024-05-06T12:14:36.831 INFO:tasks.workunit.client.0.smithi092.stdout:5/505: dwrite d4/d34/d28/f74 [0,4194304] 0 2024-05-06T12:14:36.833 INFO:tasks.workunit.client.1.smithi190.stdout:8/105: dread d3/d8/f16 [0,4194304] 0 2024-05-06T12:14:36.833 INFO:tasks.workunit.client.1.smithi190.stdout:8/106: dread - d3/f1c zero size 2024-05-06T12:14:36.836 INFO:tasks.workunit.client.1.smithi190.stdout:8/107: symlink d3/l20 0 2024-05-06T12:14:36.836 INFO:tasks.workunit.client.0.smithi092.stdout:5/506: mknod d4/d34/d28/d4e/d5d/d82/d77/cbd 0 2024-05-06T12:14:36.836 INFO:tasks.workunit.client.1.smithi190.stdout:8/108: dread - d3/f1c zero size 2024-05-06T12:14:36.837 INFO:tasks.workunit.client.1.smithi190.stdout:0/89: rmdir d9/df 39 2024-05-06T12:14:36.838 INFO:tasks.workunit.client.0.smithi092.stdout:5/507: fdatasync d4/d34/d28/d4e/d5d/d82/d1f/d20/f76 0 2024-05-06T12:14:36.839 INFO:tasks.workunit.client.0.smithi092.stdout:5/508: chown d4/d34/d28 0 1 2024-05-06T12:14:36.842 INFO:tasks.workunit.client.0.smithi092.stdout:5/509: mknod d4/d34/d28/d4e/d5d/d82/d1f/cbe 0 2024-05-06T12:14:36.843 INFO:tasks.workunit.client.0.smithi092.stdout:5/510: dread - d4/d34/d29/d54/fa2 zero size 2024-05-06T12:14:36.844 INFO:tasks.workunit.client.1.smithi190.stdout:0/90: rename d9/df/f19 to d9/dc/f20 0 2024-05-06T12:14:36.844 INFO:tasks.workunit.client.1.smithi190.stdout:0/91: dread - d9/dc/f1f zero size 2024-05-06T12:14:36.847 INFO:tasks.workunit.client.0.smithi092.stdout:9/493: write d16/f29 [1352091,8310] 0 2024-05-06T12:14:36.847 INFO:tasks.workunit.client.0.smithi092.stdout:7/649: dread d9/de/d1f/d37/d48/f8e [0,4194304] 0 2024-05-06T12:14:36.848 INFO:tasks.workunit.client.1.smithi190.stdout:0/92: dread d9/f1d [0,4194304] 0 2024-05-06T12:14:36.849 INFO:tasks.workunit.client.0.smithi092.stdout:9/494: fsync d16/d2c/d2f/d3d/d33/d37/f93 0 2024-05-06T12:14:36.849 INFO:tasks.workunit.client.0.smithi092.stdout:7/650: write d9/dd/f43 [2838263,23040] 0 2024-05-06T12:14:36.850 INFO:tasks.workunit.client.0.smithi092.stdout:9/495: readlink d16/d2c/d2f/d3d/d33/d7e/l97 0 2024-05-06T12:14:36.852 INFO:tasks.workunit.client.0.smithi092.stdout:6/625: dread d4/f1a [0,4194304] 0 2024-05-06T12:14:36.852 INFO:tasks.workunit.client.0.smithi092.stdout:7/651: creat d9/de/d1f/dcf/fd6 x:0 0 0 2024-05-06T12:14:36.853 INFO:tasks.workunit.client.1.smithi190.stdout:8/109: getdents d3 0 2024-05-06T12:14:36.854 INFO:tasks.workunit.client.0.smithi092.stdout:6/626: fdatasync d4/d24/d26/dae/dc0/fad 0 2024-05-06T12:14:36.854 INFO:tasks.workunit.client.1.smithi190.stdout:0/93: dwrite d9/fb [0,4194304] 0 2024-05-06T12:14:36.856 INFO:tasks.workunit.client.0.smithi092.stdout:6/627: fsync d4/d20/f31 0 2024-05-06T12:14:36.860 INFO:tasks.workunit.client.0.smithi092.stdout:6/628: creat d4/d24/d38/fcb x:0 0 0 2024-05-06T12:14:36.863 INFO:tasks.workunit.client.0.smithi092.stdout:6/629: creat d4/d24/d26/dae/dc0/fcc x:0 0 0 2024-05-06T12:14:36.864 INFO:tasks.workunit.client.0.smithi092.stdout:6/630: write d4/d24/d54/f86 [335968,123985] 0 2024-05-06T12:14:36.866 INFO:tasks.workunit.client.0.smithi092.stdout:6/631: write d4/f76 [833227,104124] 0 2024-05-06T12:14:36.869 INFO:tasks.workunit.client.1.smithi190.stdout:8/110: creat d3/d13/f21 x:0 0 0 2024-05-06T12:14:36.871 INFO:tasks.workunit.client.1.smithi190.stdout:8/111: mknod d3/c22 0 2024-05-06T12:14:36.871 INFO:tasks.workunit.client.0.smithi092.stdout:6/632: dwrite d4/d20/d23/f28 [0,4194304] 0 2024-05-06T12:14:36.874 INFO:tasks.workunit.client.0.smithi092.stdout:6/633: fdatasync d4/d20/d23/d30/f36 0 2024-05-06T12:14:36.878 INFO:tasks.workunit.client.0.smithi092.stdout:6/634: mknod d4/d4c/da9/ccd 0 2024-05-06T12:14:36.880 INFO:tasks.workunit.client.0.smithi092.stdout:6/635: mknod d4/d4c/d91/d93/cce 0 2024-05-06T12:14:36.882 INFO:tasks.workunit.client.0.smithi092.stdout:6/636: rename d4/d24/f9c to d4/d24/d38/d5d/fcf 0 2024-05-06T12:14:36.888 INFO:tasks.workunit.client.0.smithi092.stdout:6/637: dwrite d4/d24/d26/f6e [0,4194304] 0 2024-05-06T12:14:36.892 INFO:tasks.workunit.client.1.smithi190.stdout:8/112: fsync d3/d13/f21 0 2024-05-06T12:14:36.893 INFO:tasks.workunit.client.0.smithi092.stdout:0/677: sync 2024-05-06T12:14:36.893 INFO:tasks.workunit.client.0.smithi092.stdout:6/638: creat d4/d24/d38/d5d/fd0 x:0 0 0 2024-05-06T12:14:36.894 INFO:tasks.workunit.client.1.smithi190.stdout:0/94: truncate d9/f1d 3362889 0 2024-05-06T12:14:36.895 INFO:tasks.workunit.client.0.smithi092.stdout:7/652: dread d9/de/d1f/d37/d39/f59 [0,4194304] 0 2024-05-06T12:14:36.897 INFO:tasks.workunit.client.1.smithi190.stdout:0/95: mkdir d9/df/d21 0 2024-05-06T12:14:36.897 INFO:tasks.workunit.client.0.smithi092.stdout:0/678: mkdir d7/dc/d28/d54/d12/d7f/dda 0 2024-05-06T12:14:36.898 INFO:tasks.workunit.client.0.smithi092.stdout:6/639: creat d4/d24/d9f/dc6/fd1 x:0 0 0 2024-05-06T12:14:36.898 INFO:tasks.workunit.client.0.smithi092.stdout:7/653: truncate d9/de/d1f/d37/f74 43822 0 2024-05-06T12:14:36.899 INFO:tasks.workunit.client.1.smithi190.stdout:8/113: mknod d3/d8/c23 0 2024-05-06T12:14:36.901 INFO:tasks.workunit.client.1.smithi190.stdout:8/114: chown d3/d8/f1f 13983 1 2024-05-06T12:14:36.901 INFO:tasks.workunit.client.1.smithi190.stdout:0/96: dread d9/f16 [0,4194304] 0 2024-05-06T12:14:36.903 INFO:tasks.workunit.client.1.smithi190.stdout:0/97: truncate d9/f1b 662612 0 2024-05-06T12:14:36.906 INFO:tasks.workunit.client.0.smithi092.stdout:7/654: rename d9/de/d10/d17/d21/d2e/d9c/cbc to d9/de/da4/cd7 0 2024-05-06T12:14:36.908 INFO:tasks.workunit.client.1.smithi190.stdout:8/115: creat d3/f24 x:0 0 0 2024-05-06T12:14:36.911 INFO:tasks.workunit.client.1.smithi190.stdout:8/116: mkdir d3/d8/d25 0 2024-05-06T12:14:36.912 INFO:tasks.workunit.client.0.smithi092.stdout:7/655: rename d9/de/d1f/d37/d48/d97/fcb to d9/de/d10/d17/d99/dc3/dd2/fd8 0 2024-05-06T12:14:36.913 INFO:tasks.workunit.client.1.smithi190.stdout:8/117: creat d3/f26 x:0 0 0 2024-05-06T12:14:36.915 INFO:tasks.workunit.client.1.smithi190.stdout:8/118: unlink d3/d8/lf 0 2024-05-06T12:14:36.916 INFO:tasks.workunit.client.1.smithi190.stdout:8/119: write d3/f24 [588650,57929] 0 2024-05-06T12:14:36.917 INFO:tasks.workunit.client.0.smithi092.stdout:9/496: dread d16/d2c/d2f/d3d/d33/d37/d40/d43/f51 [0,4194304] 0 2024-05-06T12:14:36.918 INFO:tasks.workunit.client.0.smithi092.stdout:5/511: dread d4/d5/d10/d31/d53/f9e [0,4194304] 0 2024-05-06T12:14:36.918 INFO:tasks.workunit.client.0.smithi092.stdout:7/656: rename d9/de/d10/d17/d21/d2e/l41 to d9/ld9 0 2024-05-06T12:14:36.919 INFO:tasks.workunit.client.1.smithi190.stdout:8/120: creat d3/d8/f27 x:0 0 0 2024-05-06T12:14:36.920 INFO:tasks.workunit.client.0.smithi092.stdout:9/497: write d16/f1f [4929535,7662] 0 2024-05-06T12:14:36.920 INFO:tasks.workunit.client.1.smithi190.stdout:8/121: mknod d3/d13/c28 0 2024-05-06T12:14:36.922 INFO:tasks.workunit.client.0.smithi092.stdout:7/657: creat d9/de/d10/d17/d99/db6/fda x:0 0 0 2024-05-06T12:14:36.923 INFO:tasks.workunit.client.0.smithi092.stdout:6/640: dread d4/d24/d66/f88 [0,4194304] 0 2024-05-06T12:14:36.924 INFO:tasks.workunit.client.0.smithi092.stdout:9/498: mknod d16/d2c/d2f/d3d/d33/d37/d8a/c9c 0 2024-05-06T12:14:36.925 INFO:tasks.workunit.client.0.smithi092.stdout:5/512: dread d4/f4c [0,4194304] 0 2024-05-06T12:14:36.926 INFO:tasks.workunit.client.1.smithi190.stdout:8/122: link d3/f24 d3/d8/d25/f29 0 2024-05-06T12:14:36.926 INFO:tasks.workunit.client.0.smithi092.stdout:7/658: mknod d9/de/d1f/d37/d48/d65/cdb 0 2024-05-06T12:14:36.927 INFO:tasks.workunit.client.1.smithi190.stdout:8/123: truncate d3/d8/f1a 426992 0 2024-05-06T12:14:36.927 INFO:tasks.workunit.client.1.smithi190.stdout:8/124: stat d3/d13 0 2024-05-06T12:14:36.928 INFO:tasks.workunit.client.1.smithi190.stdout:8/125: fdatasync d3/f26 0 2024-05-06T12:14:36.930 INFO:tasks.workunit.client.0.smithi092.stdout:1/482: sync 2024-05-06T12:14:36.930 INFO:tasks.workunit.client.0.smithi092.stdout:4/583: sync 2024-05-06T12:14:36.931 INFO:tasks.workunit.client.0.smithi092.stdout:7/659: creat d9/de/d10/d17/d21/d2e/d9c/fdc x:0 0 0 2024-05-06T12:14:36.932 INFO:tasks.workunit.client.1.smithi190.stdout:8/126: mknod d3/c2a 0 2024-05-06T12:14:36.933 INFO:tasks.workunit.client.1.smithi190.stdout:0/98: dwrite d9/dc/f13 [0,4194304] 0 2024-05-06T12:14:36.936 INFO:tasks.workunit.client.0.smithi092.stdout:9/499: dwrite d16/d2c/d3f/f96 [0,4194304] 0 2024-05-06T12:14:36.936 INFO:tasks.workunit.client.0.smithi092.stdout:6/641: dwrite d4/d4c/d52/f56 [4194304,4194304] 0 2024-05-06T12:14:36.937 INFO:tasks.workunit.client.1.smithi190.stdout:8/127: mkdir d3/d8/d2b 0 2024-05-06T12:14:36.941 INFO:tasks.workunit.client.1.smithi190.stdout:0/99: symlink d9/dc/l22 0 2024-05-06T12:14:36.942 INFO:tasks.workunit.client.0.smithi092.stdout:9/500: write d16/d2c/f80 [1229030,73334] 0 2024-05-06T12:14:36.947 INFO:tasks.workunit.client.0.smithi092.stdout:9/501: chown d16/d2c/d2f/d3d/f67 666 1 2024-05-06T12:14:36.947 INFO:tasks.workunit.client.0.smithi092.stdout:0/679: dwrite d7/f9 [0,4194304] 0 2024-05-06T12:14:36.947 INFO:tasks.workunit.client.1.smithi190.stdout:0/100: creat d9/df/d21/f23 x:0 0 0 2024-05-06T12:14:36.948 INFO:tasks.workunit.client.0.smithi092.stdout:1/483: dwrite d3/d42/d84/f7a [0,4194304] 0 2024-05-06T12:14:36.957 INFO:tasks.workunit.client.0.smithi092.stdout:0/680: creat d7/dc/d28/d54/d1e/d26/d4e/dd7/fdb x:0 0 0 2024-05-06T12:14:36.959 INFO:tasks.workunit.client.0.smithi092.stdout:7/660: dread d9/de/d10/f25 [0,4194304] 0 2024-05-06T12:14:36.960 INFO:tasks.workunit.client.0.smithi092.stdout:0/681: symlink d7/dc/d6f/da2/ldc 0 2024-05-06T12:14:36.960 INFO:tasks.workunit.client.0.smithi092.stdout:7/661: stat d9/de/c44 0 2024-05-06T12:14:36.961 INFO:tasks.workunit.client.0.smithi092.stdout:9/502: dwrite d16/d2c/d2f/d3d/f62 [0,4194304] 0 2024-05-06T12:14:36.962 INFO:tasks.workunit.client.0.smithi092.stdout:0/682: dread - d7/dc/d28/d54/d1e/d26/faf zero size 2024-05-06T12:14:36.963 INFO:tasks.workunit.client.0.smithi092.stdout:0/683: fdatasync d7/dc/d28/d54/d12/f1b 0 2024-05-06T12:14:36.964 INFO:tasks.workunit.client.0.smithi092.stdout:7/662: creat d9/de/d1f/d37/fdd x:0 0 0 2024-05-06T12:14:36.966 INFO:tasks.workunit.client.0.smithi092.stdout:9/503: creat d16/d2c/d3f/f9d x:0 0 0 2024-05-06T12:14:36.969 INFO:tasks.workunit.client.0.smithi092.stdout:0/684: link d7/dc/d9e/fc0 d7/dc/d28/d54/fdd 0 2024-05-06T12:14:36.970 INFO:tasks.workunit.client.0.smithi092.stdout:0/685: readlink d7/dc/d6f/da2/ldc 0 2024-05-06T12:14:36.971 INFO:tasks.workunit.client.0.smithi092.stdout:9/504: mkdir d16/d2c/d2f/d3d/d33/d6b/d9e 0 2024-05-06T12:14:36.973 INFO:tasks.workunit.client.1.smithi190.stdout:0/101: getdents d9/df/d21 0 2024-05-06T12:14:36.974 INFO:tasks.workunit.client.1.smithi190.stdout:0/102: mknod d9/df/d21/c24 0 2024-05-06T12:14:36.974 INFO:tasks.workunit.client.0.smithi092.stdout:0/686: creat d7/dc/d28/d54/d12/d7f/db2/fde x:0 0 0 2024-05-06T12:14:36.975 INFO:tasks.workunit.client.1.smithi190.stdout:0/103: mknod d9/df/d21/c25 0 2024-05-06T12:14:36.976 INFO:tasks.workunit.client.1.smithi190.stdout:0/104: fdatasync d9/df/f15 0 2024-05-06T12:14:36.977 INFO:tasks.workunit.client.0.smithi092.stdout:9/505: write d16/d2c/d2f/d3d/d33/f56 [5023135,62280] 0 2024-05-06T12:14:36.977 INFO:tasks.workunit.client.0.smithi092.stdout:0/687: fdatasync d7/dc/d28/d54/f3b 0 2024-05-06T12:14:36.979 INFO:tasks.workunit.client.0.smithi092.stdout:9/506: fdatasync d16/d2c/f80 0 2024-05-06T12:14:36.981 INFO:tasks.workunit.client.0.smithi092.stdout:6/642: dwrite d4/d20/d22/f7c [4194304,4194304] 0 2024-05-06T12:14:36.981 INFO:tasks.workunit.client.0.smithi092.stdout:4/584: dwrite d9/d2b/f39 [0,4194304] 0 2024-05-06T12:14:36.985 INFO:tasks.workunit.client.0.smithi092.stdout:9/507: readlink d16/d2c/d2f/d3d/d33/l69 0 2024-05-06T12:14:36.986 INFO:tasks.workunit.client.0.smithi092.stdout:4/585: rmdir d9/d2b/d7c 39 2024-05-06T12:14:36.986 INFO:tasks.workunit.client.1.smithi190.stdout:0/105: dwrite d9/df/f10 [0,4194304] 0 2024-05-06T12:14:36.989 INFO:tasks.workunit.client.0.smithi092.stdout:0/688: rename d7/dc/d9e/fae to d7/dc/d28/d54/d12/d8f/dad/fdf 0 2024-05-06T12:14:36.990 INFO:tasks.workunit.client.0.smithi092.stdout:0/689: chown d7/dc/d6f 4375830 1 2024-05-06T12:14:36.992 INFO:tasks.workunit.client.1.smithi190.stdout:0/106: creat d9/f26 x:0 0 0 2024-05-06T12:14:36.992 INFO:tasks.workunit.client.0.smithi092.stdout:1/484: dread d3/d42/d84/d4f/d58/d65/d81/f83 [0,4194304] 0 2024-05-06T12:14:36.993 INFO:tasks.workunit.client.0.smithi092.stdout:6/643: rename d4/d24/d9f/dc6/fd1 to d4/d4c/da9/fd2 0 2024-05-06T12:14:36.994 INFO:tasks.workunit.client.1.smithi190.stdout:0/107: truncate d9/dc/f1f 957321 0 2024-05-06T12:14:36.995 INFO:tasks.workunit.client.0.smithi092.stdout:1/485: stat d3/d10/d13/d19/d21/f77 0 2024-05-06T12:14:36.998 INFO:tasks.workunit.client.0.smithi092.stdout:4/586: creat d9/d2b/d7c/dab/fc6 x:0 0 0 2024-05-06T12:14:37.002 INFO:tasks.workunit.client.0.smithi092.stdout:4/587: chown d9/d2b/d5b/f8d 196447 1 2024-05-06T12:14:37.002 INFO:tasks.workunit.client.0.smithi092.stdout:1/486: dread d3/d10/d13/f3b [0,4194304] 0 2024-05-06T12:14:37.002 INFO:tasks.workunit.client.0.smithi092.stdout:9/508: rename d16/d2c/d2f/d3d/l73 to d16/d2c/d2f/d3d/d33/d57/l9f 0 2024-05-06T12:14:37.002 INFO:tasks.workunit.client.0.smithi092.stdout:9/509: fdatasync d16/d2c/d3f/f60 0 2024-05-06T12:14:37.003 INFO:tasks.workunit.client.0.smithi092.stdout:9/510: chown d16/d2c/d2f/d3d/d33/d37/d40/d43/f8c 259630 1 2024-05-06T12:14:37.006 INFO:tasks.workunit.client.0.smithi092.stdout:1/487: mkdir d3/d10/d13/d19/d21/d72/d9a 0 2024-05-06T12:14:37.009 INFO:tasks.workunit.client.0.smithi092.stdout:0/690: link d7/dc/d28/d54/d1e/c25 d7/dc/d28/d54/d1e/d46/d63/ce0 0 2024-05-06T12:14:37.011 INFO:tasks.workunit.client.0.smithi092.stdout:9/511: mkdir d16/d94/da0 0 2024-05-06T12:14:37.011 INFO:tasks.workunit.client.0.smithi092.stdout:9/512: readlink l15 0 2024-05-06T12:14:37.012 INFO:tasks.workunit.client.0.smithi092.stdout:6/644: dread d4/d20/d23/d30/f6f [0,4194304] 0 2024-05-06T12:14:37.013 INFO:tasks.workunit.client.0.smithi092.stdout:0/691: symlink d7/dc/d28/d54/d12/d8f/dad/le1 0 2024-05-06T12:14:37.013 INFO:tasks.workunit.client.0.smithi092.stdout:1/488: dwrite d3/d10/d13/d19/d21/f77 [0,4194304] 0 2024-05-06T12:14:37.014 INFO:tasks.workunit.client.0.smithi092.stdout:9/513: chown d16/d2c/d2f/d3d/d33/d7e/l97 24047930 1 2024-05-06T12:14:37.015 INFO:tasks.workunit.client.0.smithi092.stdout:6/645: write d4/d24/d26/dae/dc0/fc9 [262333,17636] 0 2024-05-06T12:14:37.015 INFO:tasks.workunit.client.0.smithi092.stdout:9/514: chown d16/c19 127650 1 2024-05-06T12:14:37.016 INFO:tasks.workunit.client.1.smithi190.stdout:0/108: write d9/f16 [762131,65541] 0 2024-05-06T12:14:37.018 INFO:tasks.workunit.client.1.smithi190.stdout:0/109: fsync d9/f16 0 2024-05-06T12:14:37.019 INFO:tasks.workunit.client.0.smithi092.stdout:1/489: mknod d3/d10/d13/d19/d34/d89/c9b 0 2024-05-06T12:14:37.022 INFO:tasks.workunit.client.1.smithi190.stdout:0/110: dread d9/dc/fe [0,4194304] 0 2024-05-06T12:14:37.022 INFO:tasks.workunit.client.0.smithi092.stdout:0/692: rename d7/dc/d9e/f2c to d7/dc/d28/d54/d1e/d26/d96/fe2 0 2024-05-06T12:14:37.024 INFO:tasks.workunit.client.0.smithi092.stdout:9/515: dwrite d16/d2c/d4b/d76/f83 [0,4194304] 0 2024-05-06T12:14:37.026 INFO:tasks.workunit.client.0.smithi092.stdout:1/490: dread d3/d10/d13/f3b [0,4194304] 0 2024-05-06T12:14:37.029 INFO:tasks.workunit.client.1.smithi190.stdout:0/111: dwrite d9/df/d21/f23 [0,4194304] 0 2024-05-06T12:14:37.031 INFO:tasks.workunit.client.0.smithi092.stdout:1/491: write d3/d42/d84/f5f [3885182,59638] 0 2024-05-06T12:14:37.036 INFO:tasks.workunit.client.1.smithi190.stdout:0/112: link d9/f26 d9/f27 0 2024-05-06T12:14:37.037 INFO:tasks.workunit.client.0.smithi092.stdout:9/516: rename d16/d2c/d2f/d3d/d33/d37/d40 to d16/d1e/da1 0 2024-05-06T12:14:37.037 INFO:tasks.workunit.client.0.smithi092.stdout:4/588: dwrite d9/d27/f92 [0,4194304] 0 2024-05-06T12:14:37.037 INFO:tasks.workunit.client.0.smithi092.stdout:0/693: dwrite d7/dc/d9e/f61 [0,4194304] 0 2024-05-06T12:14:37.038 INFO:tasks.workunit.client.1.smithi190.stdout:0/113: mkdir d9/dc/d28 0 2024-05-06T12:14:37.039 INFO:tasks.workunit.client.0.smithi092.stdout:1/492: unlink d3/d10/d13/d19/d34/c67 0 2024-05-06T12:14:37.040 INFO:tasks.workunit.client.0.smithi092.stdout:9/517: write d16/d2c/d3f/f4d [135809,16461] 0 2024-05-06T12:14:37.040 INFO:tasks.workunit.client.0.smithi092.stdout:0/694: write d7/dc/f9b [779001,92312] 0 2024-05-06T12:14:37.042 INFO:tasks.workunit.client.1.smithi190.stdout:0/114: link c4 d9/df/c29 0 2024-05-06T12:14:37.042 INFO:tasks.workunit.client.0.smithi092.stdout:1/493: write d3/d10/d13/f60 [953186,103368] 0 2024-05-06T12:14:37.044 INFO:tasks.workunit.client.0.smithi092.stdout:0/695: mkdir d7/dc/d28/d54/d1e/d26/d3a/de3 0 2024-05-06T12:14:37.050 INFO:tasks.workunit.client.0.smithi092.stdout:0/696: dwrite d7/dc/d9e/f1c [0,4194304] 0 2024-05-06T12:14:37.051 INFO:tasks.workunit.client.1.smithi190.stdout:0/115: dwrite d9/f1d [0,4194304] 0 2024-05-06T12:14:37.055 INFO:tasks.workunit.client.0.smithi092.stdout:4/589: read d9/df/f52 [361798,85588] 0 2024-05-06T12:14:37.055 INFO:tasks.workunit.client.0.smithi092.stdout:0/697: symlink d7/dc/d28/d54/d1e/d26/d96/d90/le4 0 2024-05-06T12:14:37.056 INFO:tasks.workunit.client.0.smithi092.stdout:0/698: fdatasync d7/dc/d28/d54/d1e/d26/d4e/fbb 0 2024-05-06T12:14:37.061 INFO:tasks.workunit.client.1.smithi190.stdout:0/116: dwrite d9/f1b [0,4194304] 0 2024-05-06T12:14:37.063 INFO:tasks.workunit.client.0.smithi092.stdout:4/590: dwrite d9/dd/f9e [0,4194304] 0 2024-05-06T12:14:37.077 INFO:tasks.workunit.client.0.smithi092.stdout:1/494: dread d3/fa [0,4194304] 0 2024-05-06T12:14:37.078 INFO:tasks.workunit.client.0.smithi092.stdout:1/495: stat d3/d10/d13/d19/d57/c5e 0 2024-05-06T12:14:37.083 INFO:tasks.workunit.client.0.smithi092.stdout:4/591: creat d9/d2b/fc7 x:0 0 0 2024-05-06T12:14:37.084 INFO:tasks.workunit.client.0.smithi092.stdout:1/496: mkdir d3/d10/d13/d19/d21/d72/d9a/d9c 0 2024-05-06T12:14:37.084 INFO:tasks.workunit.client.0.smithi092.stdout:4/592: chown d9/f70 265794 1 2024-05-06T12:14:37.084 INFO:tasks.workunit.client.0.smithi092.stdout:4/593: chown d9/l99 51 1 2024-05-06T12:14:37.084 INFO:tasks.workunit.client.0.smithi092.stdout:1/497: write d3/d42/d84/f8e [4611689,28472] 0 2024-05-06T12:14:37.087 INFO:tasks.workunit.client.0.smithi092.stdout:4/594: creat d9/df/d62/d7b/fc8 x:0 0 0 2024-05-06T12:14:37.087 INFO:tasks.workunit.client.0.smithi092.stdout:4/595: readlink d9/df/d48/l8b 0 2024-05-06T12:14:37.090 INFO:tasks.workunit.client.0.smithi092.stdout:4/596: creat d9/df/d62/dbb/fc9 x:0 0 0 2024-05-06T12:14:37.092 INFO:tasks.workunit.client.0.smithi092.stdout:4/597: mkdir d9/dd/da6/dca 0 2024-05-06T12:14:37.093 INFO:tasks.workunit.client.1.smithi190.stdout:0/117: dwrite d9/f27 [0,4194304] 0 2024-05-06T12:14:37.097 INFO:tasks.workunit.client.0.smithi092.stdout:0/699: truncate d7/dc/d28/f4d 2391801 0 2024-05-06T12:14:37.098 INFO:tasks.workunit.client.1.smithi190.stdout:0/118: dread d9/f1d [0,4194304] 0 2024-05-06T12:14:37.099 INFO:tasks.workunit.client.0.smithi092.stdout:0/700: symlink d7/dc/d28/d54/d1e/d39/da8/le5 0 2024-05-06T12:14:37.099 INFO:tasks.workunit.client.1.smithi190.stdout:0/119: rmdir d9/dc 39 2024-05-06T12:14:37.101 INFO:tasks.workunit.client.0.smithi092.stdout:0/701: read d7/dc/d28/d54/d1e/d26/d3a/f52 [3881509,50148] 0 2024-05-06T12:14:37.102 INFO:tasks.workunit.client.0.smithi092.stdout:1/498: fsync d3/d42/f43 0 2024-05-06T12:14:37.102 INFO:tasks.workunit.client.1.smithi190.stdout:0/120: link d9/f1b d9/df/d21/f2a 0 2024-05-06T12:14:37.103 INFO:tasks.workunit.client.1.smithi190.stdout:0/121: read d9/f1d [93085,112163] 0 2024-05-06T12:14:37.103 INFO:tasks.workunit.client.1.smithi190.stdout:0/122: read d9/f1b [671515,115579] 0 2024-05-06T12:14:37.104 INFO:tasks.workunit.client.0.smithi092.stdout:0/702: stat d7/dc/d28/d54/d1e/d46/d63/d80/cbf 0 2024-05-06T12:14:37.106 INFO:tasks.workunit.client.0.smithi092.stdout:1/499: unlink d3/d10/d13/d19/d21/f77 0 2024-05-06T12:14:37.106 INFO:tasks.workunit.client.1.smithi190.stdout:0/123: dread d9/df/d21/f2a [0,4194304] 0 2024-05-06T12:14:37.108 INFO:tasks.workunit.client.1.smithi190.stdout:0/124: write d9/df/d21/f23 [31554,126181] 0 2024-05-06T12:14:37.108 INFO:tasks.workunit.client.1.smithi190.stdout:0/125: chown d9/f26 15966881 1 2024-05-06T12:14:37.109 INFO:tasks.workunit.client.1.smithi190.stdout:0/126: fdatasync d9/f16 0 2024-05-06T12:14:37.109 INFO:tasks.workunit.client.0.smithi092.stdout:1/500: unlink d3/d10/d13/f14 0 2024-05-06T12:14:37.112 INFO:tasks.workunit.client.0.smithi092.stdout:0/703: dwrite d7/dc/f9c [0,4194304] 0 2024-05-06T12:14:37.113 INFO:tasks.workunit.client.0.smithi092.stdout:4/598: write d9/f47 [56918,114510] 0 2024-05-06T12:14:37.114 INFO:tasks.workunit.client.1.smithi190.stdout:0/127: getdents d9/df/d21 0 2024-05-06T12:14:37.115 INFO:tasks.workunit.client.1.smithi190.stdout:0/128: write d9/df/d21/f23 [4292645,58023] 0 2024-05-06T12:14:37.116 INFO:tasks.workunit.client.1.smithi190.stdout:0/129: stat d9/df/l18 0 2024-05-06T12:14:37.117 INFO:tasks.workunit.client.0.smithi092.stdout:0/704: rmdir d7/dc/d28/d54/d1e/d26/d4e 39 2024-05-06T12:14:37.120 INFO:tasks.workunit.client.0.smithi092.stdout:0/705: mkdir d7/dc/d31/de6 0 2024-05-06T12:14:37.121 INFO:tasks.workunit.client.0.smithi092.stdout:4/599: mknod d9/d2b/d5b/daf/ccb 0 2024-05-06T12:14:37.121 INFO:tasks.workunit.client.0.smithi092.stdout:4/600: chown d9/d2b/d65/l96 54536 1 2024-05-06T12:14:37.122 INFO:tasks.workunit.client.0.smithi092.stdout:0/706: symlink d7/dc/d97/le7 0 2024-05-06T12:14:37.123 INFO:tasks.workunit.client.0.smithi092.stdout:4/601: write d9/d27/f92 [1066580,31077] 0 2024-05-06T12:14:37.125 INFO:tasks.workunit.client.1.smithi190.stdout:0/130: write d9/f1b [4899606,88376] 0 2024-05-06T12:14:37.128 INFO:tasks.workunit.client.1.smithi190.stdout:0/131: symlink d9/dc/d28/l2b 0 2024-05-06T12:14:37.128 INFO:tasks.workunit.client.0.smithi092.stdout:1/501: write d3/d10/d13/d19/f5d [6885069,57694] 0 2024-05-06T12:14:37.129 INFO:tasks.workunit.client.1.smithi190.stdout:0/132: unlink d9/f27 0 2024-05-06T12:14:37.130 INFO:tasks.workunit.client.0.smithi092.stdout:1/502: chown d3/d10/d13/d19/d21/l44 13 1 2024-05-06T12:14:37.131 INFO:tasks.workunit.client.0.smithi092.stdout:1/503: chown d3/d42/d84/f7a 28 1 2024-05-06T12:14:37.133 INFO:tasks.workunit.client.1.smithi190.stdout:0/133: rename d9/f16 to d9/f2c 0 2024-05-06T12:14:37.135 INFO:tasks.workunit.client.0.smithi092.stdout:0/707: dread d7/dc/d28/d54/d1e/d46/f7b [0,4194304] 0 2024-05-06T12:14:37.135 INFO:tasks.workunit.client.1.smithi190.stdout:0/134: dread d9/dc/f1f [0,4194304] 0 2024-05-06T12:14:37.135 INFO:tasks.workunit.client.1.smithi190.stdout:0/135: fdatasync d9/dc/f13 0 2024-05-06T12:14:37.137 INFO:tasks.workunit.client.0.smithi092.stdout:0/708: write d7/dc/d28/d54/d1e/d26/faf [335353,58003] 0 2024-05-06T12:14:37.137 INFO:tasks.workunit.client.1.smithi190.stdout:0/136: mknod d9/c2d 0 2024-05-06T12:14:37.138 INFO:tasks.workunit.client.0.smithi092.stdout:0/709: chown d7/dc/d28/d54/d12/d7f/db2/dbc 7 1 2024-05-06T12:14:37.139 INFO:tasks.workunit.client.1.smithi190.stdout:0/137: dread d9/dc/fe [0,4194304] 0 2024-05-06T12:14:37.141 INFO:tasks.workunit.client.1.smithi190.stdout:0/138: rmdir d9/df 39 2024-05-06T12:14:37.142 INFO:tasks.workunit.client.1.smithi190.stdout:0/139: dread d9/dc/fe [0,4194304] 0 2024-05-06T12:14:37.146 INFO:tasks.workunit.client.0.smithi092.stdout:0/710: dwrite d7/dc/d28/d54/d12/d7f/db2/fde [0,4194304] 0 2024-05-06T12:14:37.147 INFO:tasks.workunit.client.0.smithi092.stdout:4/602: dread d9/d27/d4a/f4c [0,4194304] 0 2024-05-06T12:14:37.150 INFO:tasks.workunit.client.1.smithi190.stdout:0/140: dwrite d9/df/f15 [0,4194304] 0 2024-05-06T12:14:37.150 INFO:tasks.workunit.client.0.smithi092.stdout:4/603: symlink d9/d2b/d7c/lcc 0 2024-05-06T12:14:37.151 INFO:tasks.workunit.client.1.smithi190.stdout:0/141: chown d9/f2c 6917333 1 2024-05-06T12:14:37.152 INFO:tasks.workunit.client.1.smithi190.stdout:0/142: write d9/df/d21/f23 [4700014,75928] 0 2024-05-06T12:14:37.154 INFO:tasks.workunit.client.0.smithi092.stdout:4/604: creat d9/d27/da8/db3/dbf/fcd x:0 0 0 2024-05-06T12:14:37.155 INFO:tasks.workunit.client.1.smithi190.stdout:0/143: creat d9/df/f2e x:0 0 0 2024-05-06T12:14:37.155 INFO:tasks.workunit.client.0.smithi092.stdout:1/504: dread d3/d10/d13/d19/f1c [0,4194304] 0 2024-05-06T12:14:37.157 INFO:tasks.workunit.client.1.smithi190.stdout:0/144: creat d9/f2f x:0 0 0 2024-05-06T12:14:37.159 INFO:tasks.workunit.client.0.smithi092.stdout:8/557: sync 2024-05-06T12:14:37.159 INFO:tasks.workunit.client.0.smithi092.stdout:2/354: sync 2024-05-06T12:14:37.159 INFO:tasks.workunit.client.0.smithi092.stdout:3/452: sync 2024-05-06T12:14:37.160 INFO:tasks.workunit.client.0.smithi092.stdout:2/355: dread - df/f61 zero size 2024-05-06T12:14:37.161 INFO:tasks.workunit.client.1.smithi190.stdout:0/145: link l8 d9/df/d21/l30 0 2024-05-06T12:14:37.161 INFO:tasks.workunit.client.1.smithi190.stdout:0/146: readlink d9/dc/l22 0 2024-05-06T12:14:37.162 INFO:tasks.workunit.client.0.smithi092.stdout:8/558: mknod d9/d21/cd4 0 2024-05-06T12:14:37.162 INFO:tasks.workunit.client.1.smithi190.stdout:0/147: stat d9/ca 0 2024-05-06T12:14:37.163 INFO:tasks.workunit.client.1.smithi190.stdout:0/148: fdatasync d9/fb 0 2024-05-06T12:14:37.166 INFO:tasks.workunit.client.0.smithi092.stdout:3/453: read d8/d1b/f2b [3202496,115343] 0 2024-05-06T12:14:37.167 INFO:tasks.workunit.client.0.smithi092.stdout:8/559: truncate d9/dc/f2d 2121490 0 2024-05-06T12:14:37.168 INFO:tasks.workunit.client.0.smithi092.stdout:2/356: dwrite df/d17/f62 [0,4194304] 0 2024-05-06T12:14:37.170 INFO:tasks.workunit.client.0.smithi092.stdout:3/454: readlink d8/db/d4c/d40/d67/l7e 0 2024-05-06T12:14:37.171 INFO:tasks.workunit.client.0.smithi092.stdout:0/711: dread d7/dc/d28/f2e [0,4194304] 0 2024-05-06T12:14:37.171 INFO:tasks.workunit.client.0.smithi092.stdout:3/455: stat d8/db/d4c/d3d/f60 0 2024-05-06T12:14:37.173 INFO:tasks.workunit.client.0.smithi092.stdout:2/357: write df/d17/d1d/d33/d3c/f63 [1457987,90248] 0 2024-05-06T12:14:37.174 INFO:tasks.workunit.client.0.smithi092.stdout:8/560: unlink d9/dc/c40 0 2024-05-06T12:14:37.174 INFO:tasks.workunit.client.0.smithi092.stdout:1/505: dread d3/d10/d13/d19/d34/f36 [0,4194304] 0 2024-05-06T12:14:37.177 INFO:tasks.workunit.client.0.smithi092.stdout:3/456: symlink d8/db/d4c/d3d/d49/l8c 0 2024-05-06T12:14:37.178 INFO:tasks.workunit.client.0.smithi092.stdout:3/457: write d8/db/d4c/d40/f52 [1174594,124505] 0 2024-05-06T12:14:37.178 INFO:tasks.workunit.client.0.smithi092.stdout:3/458: chown d8/db/d4c 1 1 2024-05-06T12:14:37.180 INFO:tasks.workunit.client.0.smithi092.stdout:2/358: mkdir df/d15/d6a 0 2024-05-06T12:14:37.180 INFO:tasks.workunit.client.0.smithi092.stdout:3/459: fsync d8/d24/d33/d4b/f41 0 2024-05-06T12:14:37.181 INFO:tasks.workunit.client.0.smithi092.stdout:2/359: chown df/d17/d1d/d33/d4d/l5d 22285 1 2024-05-06T12:14:37.185 INFO:tasks.workunit.client.0.smithi092.stdout:3/460: symlink d8/db/d4c/d3d/d49/l8d 0 2024-05-06T12:14:37.186 INFO:tasks.workunit.client.0.smithi092.stdout:1/506: getdents d3/d10/d13/d19/d21/d72/d9a/d9c 0 2024-05-06T12:14:37.187 INFO:tasks.workunit.client.0.smithi092.stdout:8/561: creat d9/d21/d37/dbd/fd5 x:0 0 0 2024-05-06T12:14:37.189 INFO:tasks.workunit.client.0.smithi092.stdout:1/507: stat d3/d42/f43 0 2024-05-06T12:14:37.191 INFO:tasks.workunit.client.0.smithi092.stdout:2/360: dwrite df/d17/d51/f57 [0,4194304] 0 2024-05-06T12:14:37.191 INFO:tasks.workunit.client.0.smithi092.stdout:0/712: dwrite d7/dc/f30 [0,4194304] 0 2024-05-06T12:14:37.193 INFO:tasks.workunit.client.0.smithi092.stdout:2/361: stat df/d17/d1d/l3f 0 2024-05-06T12:14:37.193 INFO:tasks.workunit.client.0.smithi092.stdout:0/713: readlink d7/dc/d6f/lb0 0 2024-05-06T12:14:37.195 INFO:tasks.workunit.client.0.smithi092.stdout:1/508: symlink d3/d10/d13/d46/l9d 0 2024-05-06T12:14:37.197 INFO:tasks.workunit.client.0.smithi092.stdout:0/714: unlink d7/dc/f30 0 2024-05-06T12:14:37.198 INFO:tasks.workunit.client.0.smithi092.stdout:8/562: creat d9/d21/d37/d46/d5c/d62/da5/fd6 x:0 0 0 2024-05-06T12:14:37.200 INFO:tasks.workunit.client.0.smithi092.stdout:1/509: mknod d3/d10/d13/d19/d21/d88/c9e 0 2024-05-06T12:14:37.200 INFO:tasks.workunit.client.0.smithi092.stdout:0/715: mknod d7/dc/d28/d54/d12/d7f/db2/dbc/ce8 0 2024-05-06T12:14:37.201 INFO:tasks.workunit.client.0.smithi092.stdout:8/563: truncate d9/f25 69212 0 2024-05-06T12:14:37.202 INFO:tasks.workunit.client.0.smithi092.stdout:1/510: mkdir d3/d42/d84/d9f 0 2024-05-06T12:14:37.203 INFO:tasks.workunit.client.0.smithi092.stdout:0/716: mknod d7/dc/d28/d54/ce9 0 2024-05-06T12:14:37.209 INFO:tasks.workunit.client.0.smithi092.stdout:1/511: dwrite d3/d10/d13/d19/f5a [0,4194304] 0 2024-05-06T12:14:37.210 INFO:tasks.workunit.client.0.smithi092.stdout:1/512: readlink d3/d42/d84/d4f/d58/d65/l80 0 2024-05-06T12:14:37.230 INFO:tasks.workunit.client.0.smithi092.stdout:1/513: dread d3/d10/f37 [0,4194304] 0 2024-05-06T12:14:37.231 INFO:tasks.workunit.client.0.smithi092.stdout:1/514: fdatasync d3/d10/d13/d19/f4e 0 2024-05-06T12:14:37.232 INFO:tasks.workunit.client.0.smithi092.stdout:1/515: chown d3/d10/d13/d19/d34/d89 6 1 2024-05-06T12:14:37.233 INFO:tasks.workunit.client.0.smithi092.stdout:1/516: write d3/d10/d13/d19/d21/f24 [3480475,20201] 0 2024-05-06T12:14:37.235 INFO:tasks.workunit.client.0.smithi092.stdout:1/517: creat d3/d10/d13/d19/d21/d88/fa0 x:0 0 0 2024-05-06T12:14:37.237 INFO:tasks.workunit.client.0.smithi092.stdout:1/518: getdents d3/d10/d13/d19/d21/d72/d9a/d9c 0 2024-05-06T12:14:37.238 INFO:tasks.workunit.client.0.smithi092.stdout:1/519: write d3/d10/f37 [3562013,18757] 0 2024-05-06T12:14:37.291 INFO:tasks.workunit.client.0.smithi092.stdout:5/513: sync 2024-05-06T12:14:37.303 INFO:tasks.workunit.client.0.smithi092.stdout:2/362: dread df/d17/d1d/d33/d3c/f63 [0,4194304] 0 2024-05-06T12:14:37.306 INFO:tasks.workunit.client.0.smithi092.stdout:4/605: dread d9/f21 [4194304,4194304] 0 2024-05-06T12:14:37.314 INFO:tasks.workunit.client.0.smithi092.stdout:5/514: dread d4/d5/d10/fa6 [0,4194304] 0 2024-05-06T12:14:37.314 INFO:tasks.workunit.client.0.smithi092.stdout:4/606: rename d9/l13 to d9/lce 0 2024-05-06T12:14:37.316 INFO:tasks.workunit.client.0.smithi092.stdout:5/515: rmdir d4/d34/d28 39 2024-05-06T12:14:37.321 INFO:tasks.workunit.client.0.smithi092.stdout:4/607: dread d9/d27/d4a/f84 [0,4194304] 0 2024-05-06T12:14:37.321 INFO:tasks.workunit.client.0.smithi092.stdout:5/516: dread d4/d5/d10/d31/d53/f9e [0,4194304] 0 2024-05-06T12:14:37.323 INFO:tasks.workunit.client.0.smithi092.stdout:1/520: write d3/d10/d13/f1f [4536441,44000] 0 2024-05-06T12:14:37.324 INFO:tasks.workunit.client.0.smithi092.stdout:2/363: write df/f5b [3565081,70131] 0 2024-05-06T12:14:37.325 INFO:tasks.workunit.client.0.smithi092.stdout:4/608: truncate d9/da2/fc4 4220992 0 2024-05-06T12:14:37.326 INFO:tasks.workunit.client.0.smithi092.stdout:1/521: creat d3/d10/d13/d46/fa1 x:0 0 0 2024-05-06T12:14:37.327 INFO:tasks.workunit.client.0.smithi092.stdout:8/564: read d9/f25 [11562,112942] 0 2024-05-06T12:14:37.333 INFO:tasks.workunit.client.0.smithi092.stdout:3/461: dread d8/d24/d27/f2e [0,4194304] 0 2024-05-06T12:14:37.336 INFO:tasks.workunit.client.0.smithi092.stdout:3/462: read f3 [1741573,13662] 0 2024-05-06T12:14:37.337 INFO:tasks.workunit.client.0.smithi092.stdout:0/717: dread d7/dc/d28/d54/d1e/d5d/f7c [0,4194304] 0 2024-05-06T12:14:37.341 INFO:tasks.workunit.client.0.smithi092.stdout:3/463: truncate d8/db/d4c/d3d/d49/f8a 1590016 0 2024-05-06T12:14:37.344 INFO:tasks.workunit.client.0.smithi092.stdout:0/718: write d7/dc/d28/d54/d1e/d39/da8/dc5/fd1 [659222,74186] 0 2024-05-06T12:14:37.346 INFO:tasks.workunit.client.0.smithi092.stdout:2/364: dread df/d17/d1d/f2e [0,4194304] 0 2024-05-06T12:14:37.349 INFO:tasks.workunit.client.0.smithi092.stdout:1/522: dwrite d3/d42/f43 [0,4194304] 0 2024-05-06T12:14:37.351 INFO:tasks.workunit.client.0.smithi092.stdout:4/609: dwrite d9/df/f14 [0,4194304] 0 2024-05-06T12:14:37.351 INFO:tasks.workunit.client.0.smithi092.stdout:0/719: mknod d7/cea 0 2024-05-06T12:14:37.351 INFO:tasks.workunit.client.0.smithi092.stdout:8/565: dwrite d9/f76 [0,4194304] 0 2024-05-06T12:14:37.353 INFO:tasks.workunit.client.0.smithi092.stdout:4/610: read - d9/df/d48/f74 zero size 2024-05-06T12:14:37.355 INFO:tasks.workunit.client.0.smithi092.stdout:2/365: write df/d17/d1d/f2e [5321099,52726] 0 2024-05-06T12:14:37.356 INFO:tasks.workunit.client.0.smithi092.stdout:3/464: link d8/db/d14/d5a/f7b d8/db/d4c/d3d/f8e 0 2024-05-06T12:14:37.360 INFO:tasks.workunit.client.0.smithi092.stdout:4/611: rmdir d9/d27/d4a 39 2024-05-06T12:14:37.363 INFO:tasks.workunit.client.0.smithi092.stdout:0/720: rename d7/dc/d9e/f1c to d7/dc/d28/d54/d1e/d46/d63/feb 0 2024-05-06T12:14:37.365 INFO:tasks.workunit.client.0.smithi092.stdout:1/523: symlink d3/d42/d84/d4f/d7f/la2 0 2024-05-06T12:14:37.369 INFO:tasks.workunit.client.0.smithi092.stdout:4/612: truncate d9/d27/f49 1881829 0 2024-05-06T12:14:37.369 INFO:tasks.workunit.client.0.smithi092.stdout:3/465: dwrite d8/d24/d33/d4b/f46 [4194304,4194304] 0 2024-05-06T12:14:37.370 INFO:tasks.workunit.client.0.smithi092.stdout:2/366: dwrite df/d5c/f66 [0,4194304] 0 2024-05-06T12:14:37.372 INFO:tasks.workunit.client.0.smithi092.stdout:8/566: getdents d9/dc 0 2024-05-06T12:14:37.380 INFO:tasks.workunit.client.0.smithi092.stdout:2/367: rename df/f39 to df/d17/f6b 0 2024-05-06T12:14:37.381 INFO:tasks.workunit.client.0.smithi092.stdout:8/567: truncate d9/f2a 4252552 0 2024-05-06T12:14:37.387 INFO:tasks.workunit.client.0.smithi092.stdout:1/524: dwrite d3/d10/d13/d19/f4e [0,4194304] 0 2024-05-06T12:14:37.389 INFO:tasks.workunit.client.0.smithi092.stdout:0/721: dwrite d7/dc/d28/d54/d1e/d26/d4e/fa0 [0,4194304] 0 2024-05-06T12:14:37.392 INFO:tasks.workunit.client.0.smithi092.stdout:4/613: dwrite d9/d2b/f39 [0,4194304] 0 2024-05-06T12:14:37.395 INFO:tasks.workunit.client.0.smithi092.stdout:1/525: mknod d3/d10/ca3 0 2024-05-06T12:14:37.397 INFO:tasks.workunit.client.0.smithi092.stdout:4/614: stat d9/d2b/f6a 0 2024-05-06T12:14:37.399 INFO:tasks.workunit.client.0.smithi092.stdout:8/568: dwrite d9/dc/d4a/f97 [0,4194304] 0 2024-05-06T12:14:37.401 INFO:tasks.workunit.client.0.smithi092.stdout:4/615: truncate d9/dd/f9e 4542066 0 2024-05-06T12:14:37.404 INFO:tasks.workunit.client.0.smithi092.stdout:4/616: symlink d9/d27/da8/db3/dbf/lcf 0 2024-05-06T12:14:37.406 INFO:tasks.workunit.client.0.smithi092.stdout:8/569: symlink d9/d21/d37/ld7 0 2024-05-06T12:14:37.409 INFO:tasks.workunit.client.0.smithi092.stdout:2/368: dwrite df/d17/d51/f57 [0,4194304] 0 2024-05-06T12:14:37.414 INFO:tasks.workunit.client.0.smithi092.stdout:2/369: chown df/d15/l56 817 1 2024-05-06T12:14:37.417 INFO:tasks.workunit.client.0.smithi092.stdout:0/722: dread d7/dc/d6f/fc2 [0,4194304] 0 2024-05-06T12:14:37.417 INFO:tasks.workunit.client.0.smithi092.stdout:4/617: dwrite d9/d27/d4a/f75 [0,4194304] 0 2024-05-06T12:14:37.417 INFO:tasks.workunit.client.0.smithi092.stdout:8/570: rename d9/d21/fb4 to d9/d21/d54/d79/fd8 0 2024-05-06T12:14:37.423 INFO:tasks.workunit.client.0.smithi092.stdout:2/370: truncate df/d17/d1d/d33/f44 1265278 0 2024-05-06T12:14:37.423 INFO:tasks.workunit.client.0.smithi092.stdout:8/571: mkdir d9/d5a/dd9 0 2024-05-06T12:14:37.423 INFO:tasks.workunit.client.0.smithi092.stdout:0/723: write d7/dc/d28/d54/d1e/d26/d4e/fbb [240802,47315] 0 2024-05-06T12:14:37.426 INFO:tasks.workunit.client.0.smithi092.stdout:9/518: sync 2024-05-06T12:14:37.426 INFO:tasks.workunit.client.0.smithi092.stdout:6/646: sync 2024-05-06T12:14:37.426 INFO:tasks.workunit.client.0.smithi092.stdout:7/663: sync 2024-05-06T12:14:37.429 INFO:tasks.workunit.client.0.smithi092.stdout:1/526: dread d3/f70 [0,4194304] 0 2024-05-06T12:14:37.430 INFO:tasks.workunit.client.0.smithi092.stdout:6/647: rmdir d4/d24/d4e 39 2024-05-06T12:14:37.431 INFO:tasks.workunit.client.0.smithi092.stdout:3/466: dread d8/d24/d27/f2f [4194304,4194304] 0 2024-05-06T12:14:37.433 INFO:tasks.workunit.client.0.smithi092.stdout:9/519: unlink d16/d2c/f80 0 2024-05-06T12:14:37.435 INFO:tasks.workunit.client.0.smithi092.stdout:6/648: symlink d4/d24/d38/d8c/ld3 0 2024-05-06T12:14:37.436 INFO:tasks.workunit.client.0.smithi092.stdout:3/467: write d8/db/d14/f50 [4163887,48424] 0 2024-05-06T12:14:37.438 INFO:tasks.workunit.client.0.smithi092.stdout:1/527: mkdir d3/d10/d13/d19/d21/d72/d9a/da4 0 2024-05-06T12:14:37.439 INFO:tasks.workunit.client.0.smithi092.stdout:2/371: dwrite df/d17/d1d/f41 [0,4194304] 0 2024-05-06T12:14:37.439 INFO:tasks.workunit.client.0.smithi092.stdout:6/649: chown d4/d4c/d91/d93/cce 1 1 2024-05-06T12:14:37.441 INFO:tasks.workunit.client.0.smithi092.stdout:9/520: creat d16/d2c/d2f/d3d/d33/d6b/fa2 x:0 0 0 2024-05-06T12:14:37.442 INFO:tasks.workunit.client.0.smithi092.stdout:2/372: write df/d5c/f66 [284535,76867] 0 2024-05-06T12:14:37.446 INFO:tasks.workunit.client.0.smithi092.stdout:1/528: creat d3/d10/d13/d19/d34/d89/fa5 x:0 0 0 2024-05-06T12:14:37.449 INFO:tasks.workunit.client.0.smithi092.stdout:9/521: mknod d16/d2c/d4b/ca3 0 2024-05-06T12:14:37.451 INFO:tasks.workunit.client.0.smithi092.stdout:8/572: dread d9/dc/d4a/f6b [0,4194304] 0 2024-05-06T12:14:37.451 INFO:tasks.workunit.client.0.smithi092.stdout:3/468: dwrite d8/db/f4d [0,4194304] 0 2024-05-06T12:14:37.453 INFO:tasks.workunit.client.0.smithi092.stdout:6/650: symlink d4/d24/d38/d5d/d6d/ld4 0 2024-05-06T12:14:37.453 INFO:tasks.workunit.client.0.smithi092.stdout:0/724: truncate d7/f9 3939053 0 2024-05-06T12:14:37.454 INFO:tasks.workunit.client.0.smithi092.stdout:8/573: write d9/dc/d4a/d5f/da8/fd1 [323099,19057] 0 2024-05-06T12:14:37.454 INFO:tasks.workunit.client.0.smithi092.stdout:6/651: fsync d4/d4c/f75 0 2024-05-06T12:14:37.457 INFO:tasks.workunit.client.0.smithi092.stdout:2/373: symlink df/d17/d1d/l6c 0 2024-05-06T12:14:37.463 INFO:tasks.workunit.client.0.smithi092.stdout:1/529: rename d3/d10/d13/l4a to d3/d42/d84/d9f/la6 0 2024-05-06T12:14:37.464 INFO:tasks.workunit.client.0.smithi092.stdout:7/664: dread d9/de/d10/d17/d21/d38/f89 [0,4194304] 0 2024-05-06T12:14:37.464 INFO:tasks.workunit.client.0.smithi092.stdout:0/725: fsync d7/dc/d28/d54/d1e/f41 0 2024-05-06T12:14:37.466 INFO:tasks.workunit.client.0.smithi092.stdout:3/469: link d8/f12 d8/db/d14/d5a/f8f 0 2024-05-06T12:14:37.472 INFO:tasks.workunit.client.0.smithi092.stdout:6/652: mknod d4/d20/cd5 0 2024-05-06T12:14:37.473 INFO:tasks.workunit.client.0.smithi092.stdout:7/665: creat d9/de/d1f/dcf/fde x:0 0 0 2024-05-06T12:14:37.474 INFO:tasks.workunit.client.0.smithi092.stdout:0/726: dwrite d7/dc/d28/d54/d1e/d26/d4e/fa0 [4194304,4194304] 0 2024-05-06T12:14:37.478 INFO:tasks.workunit.client.0.smithi092.stdout:9/522: dwrite d16/d2c/d2f/f45 [4194304,4194304] 0 2024-05-06T12:14:37.480 INFO:tasks.workunit.client.0.smithi092.stdout:2/374: creat df/f6d x:0 0 0 2024-05-06T12:14:37.480 INFO:tasks.workunit.client.0.smithi092.stdout:8/574: write d9/d21/d37/f59 [465053,17778] 0 2024-05-06T12:14:37.487 INFO:tasks.workunit.client.0.smithi092.stdout:3/470: mkdir d8/db/d4c/d40/d7a/d90 0 2024-05-06T12:14:37.488 INFO:tasks.workunit.client.0.smithi092.stdout:9/523: dwrite d16/d2c/d2f/d3d/d33/d6b/f77 [0,4194304] 0 2024-05-06T12:14:37.490 INFO:tasks.workunit.client.0.smithi092.stdout:6/653: symlink d4/ld6 0 2024-05-06T12:14:37.491 INFO:tasks.workunit.client.0.smithi092.stdout:2/375: fsync df/d15/f30 0 2024-05-06T12:14:37.492 INFO:tasks.workunit.client.0.smithi092.stdout:6/654: write d4/d24/d38/f69 [494634,37199] 0 2024-05-06T12:14:37.493 INFO:tasks.workunit.client.0.smithi092.stdout:7/666: mknod d9/de/d10/d17/d6a/cdf 0 2024-05-06T12:14:37.494 INFO:tasks.workunit.client.0.smithi092.stdout:9/524: dread f7 [0,4194304] 0 2024-05-06T12:14:37.494 INFO:tasks.workunit.client.0.smithi092.stdout:3/471: dread d8/db/f4d [0,4194304] 0 2024-05-06T12:14:37.498 INFO:tasks.workunit.client.0.smithi092.stdout:2/376: symlink df/d5c/l6e 0 2024-05-06T12:14:37.500 INFO:tasks.workunit.client.0.smithi092.stdout:0/727: creat d7/dc/d28/d54/d1e/fec x:0 0 0 2024-05-06T12:14:37.500 INFO:tasks.workunit.client.0.smithi092.stdout:2/377: chown df/d17/d1d/f49 1255 1 2024-05-06T12:14:37.504 INFO:tasks.workunit.client.0.smithi092.stdout:2/378: write df/d15/d50/d59/f65 [718932,103153] 0 2024-05-06T12:14:37.505 INFO:tasks.workunit.client.0.smithi092.stdout:6/655: truncate d4/d20/d23/d30/f6f 111260 0 2024-05-06T12:14:37.506 INFO:tasks.workunit.client.0.smithi092.stdout:9/525: dwrite d16/d2c/d2f/f45 [4194304,4194304] 0 2024-05-06T12:14:37.506 INFO:tasks.workunit.client.0.smithi092.stdout:3/472: dwrite d8/d24/d33/d4b/f46 [0,4194304] 0 2024-05-06T12:14:37.510 INFO:tasks.workunit.client.0.smithi092.stdout:7/667: getdents d9/de/d1f 0 2024-05-06T12:14:37.511 INFO:tasks.workunit.client.0.smithi092.stdout:0/728: truncate d7/dc/d28/d64/f78 3290960 0 2024-05-06T12:14:37.512 INFO:tasks.workunit.client.0.smithi092.stdout:3/473: dread - d8/d24/d33/d39/f70 zero size 2024-05-06T12:14:37.515 INFO:tasks.workunit.client.0.smithi092.stdout:0/729: write d7/dc/d31/f7a [3744652,8078] 0 2024-05-06T12:14:37.516 INFO:tasks.workunit.client.0.smithi092.stdout:9/526: mkdir d16/d1e/d7b/d8d/da4 0 2024-05-06T12:14:37.519 INFO:tasks.workunit.client.0.smithi092.stdout:7/668: truncate d9/de/d2d/f3a 5194547 0 2024-05-06T12:14:37.520 INFO:tasks.workunit.client.0.smithi092.stdout:6/656: dwrite d4/d24/f3e [8388608,4194304] 0 2024-05-06T12:14:37.520 INFO:tasks.workunit.client.0.smithi092.stdout:3/474: mkdir d8/d24/d33/d91 0 2024-05-06T12:14:37.522 INFO:tasks.workunit.client.0.smithi092.stdout:0/730: creat d7/dc/d28/d54/d1e/d26/d96/fed x:0 0 0 2024-05-06T12:14:37.523 INFO:tasks.workunit.client.0.smithi092.stdout:1/530: truncate d3/d10/d13/d19/d34/f36 100633 0 2024-05-06T12:14:37.525 INFO:tasks.workunit.client.0.smithi092.stdout:0/731: stat d7/dc/d28/d54/d1e/d46/d63/fb9 0 2024-05-06T12:14:37.525 INFO:tasks.workunit.client.0.smithi092.stdout:1/531: read - d3/d10/d13/d19/d34/d89/fa5 zero size 2024-05-06T12:14:37.527 INFO:tasks.workunit.client.0.smithi092.stdout:1/532: stat d3/d42/d84 0 2024-05-06T12:14:37.529 INFO:tasks.workunit.client.0.smithi092.stdout:0/732: read - d7/dc/d28/d54/d1e/d46/fb3 zero size 2024-05-06T12:14:37.529 INFO:tasks.workunit.client.0.smithi092.stdout:9/527: dread d16/d2c/d2f/d3d/d33/d6b/f77 [0,4194304] 0 2024-05-06T12:14:37.530 INFO:tasks.workunit.client.0.smithi092.stdout:7/669: mknod d9/de/d1f/d37/d48/d65/ce0 0 2024-05-06T12:14:37.532 INFO:tasks.workunit.client.0.smithi092.stdout:1/533: read d3/d10/d13/d19/f31 [500434,102359] 0 2024-05-06T12:14:37.532 INFO:tasks.workunit.client.0.smithi092.stdout:7/670: write d9/de/d10/d17/d99/db6/fc8 [52075,106022] 0 2024-05-06T12:14:37.535 INFO:tasks.workunit.client.0.smithi092.stdout:1/534: chown d3/d10/ca3 671852519 1 2024-05-06T12:14:37.536 INFO:tasks.workunit.client.0.smithi092.stdout:6/657: symlink d4/d4c/d52/d6c/da0/dc3/ld7 0 2024-05-06T12:14:37.536 INFO:tasks.workunit.client.0.smithi092.stdout:7/671: stat d9/de/d10/d17/d21/d2e/d9c/fbe 0 2024-05-06T12:14:37.539 INFO:tasks.workunit.client.0.smithi092.stdout:9/528: dwrite d16/d2c/d3f/f4d [0,4194304] 0 2024-05-06T12:14:37.542 INFO:tasks.workunit.client.0.smithi092.stdout:0/733: getdents d7/dc/d28/d54/d12/d7f/db2 0 2024-05-06T12:14:37.544 INFO:tasks.workunit.client.0.smithi092.stdout:9/529: creat d16/d2c/d2f/d3d/d33/d37/d8a/fa5 x:0 0 0 2024-05-06T12:14:37.544 INFO:tasks.workunit.client.0.smithi092.stdout:9/530: readlink l15 0 2024-05-06T12:14:37.544 INFO:tasks.workunit.client.0.smithi092.stdout:9/531: chown d16/d1c/f7c 6 1 2024-05-06T12:14:37.546 INFO:tasks.workunit.client.0.smithi092.stdout:6/658: mkdir d4/d24/d38/d5d/d6d/dd8 0 2024-05-06T12:14:37.547 INFO:tasks.workunit.client.0.smithi092.stdout:1/535: mknod d3/d10/d13/d19/d21/d72/d9a/d9c/ca7 0 2024-05-06T12:14:37.549 INFO:tasks.workunit.client.0.smithi092.stdout:0/734: creat d7/dc/d28/d64/fee x:0 0 0 2024-05-06T12:14:37.551 INFO:tasks.workunit.client.0.smithi092.stdout:9/532: symlink d16/d2c/d2f/d75/la6 0 2024-05-06T12:14:37.552 INFO:tasks.workunit.client.0.smithi092.stdout:3/475: dread d8/d24/d27/f42 [0,4194304] 0 2024-05-06T12:14:37.554 INFO:tasks.workunit.client.0.smithi092.stdout:1/536: fsync d3/d42/f85 0 2024-05-06T12:14:37.559 INFO:tasks.workunit.client.0.smithi092.stdout:0/735: dread d7/f9 [0,4194304] 0 2024-05-06T12:14:37.559 INFO:tasks.workunit.client.0.smithi092.stdout:0/736: chown d7/dc/d31 8162 1 2024-05-06T12:14:37.562 INFO:tasks.workunit.client.0.smithi092.stdout:9/533: mkdir d16/d94/da0/da7 0 2024-05-06T12:14:37.563 INFO:tasks.workunit.client.0.smithi092.stdout:6/659: link d4/d20/c1d d4/d20/d22/cd9 0 2024-05-06T12:14:37.564 INFO:tasks.workunit.client.0.smithi092.stdout:9/534: stat d16/d2c/d2f/d3d/d33/d57/f59 0 2024-05-06T12:14:37.564 INFO:tasks.workunit.client.0.smithi092.stdout:7/672: dwrite d9/de/d10/f12 [0,4194304] 0 2024-05-06T12:14:37.622 INFO:tasks.workunit.client.0.smithi092.stdout:1/537: unlink d3/d10/d13/c1a 0 2024-05-06T12:14:37.623 INFO:tasks.workunit.client.0.smithi092.stdout:3/476: creat d8/d24/d33/f92 x:0 0 0 2024-05-06T12:14:37.623 INFO:tasks.workunit.client.0.smithi092.stdout:0/737: symlink d7/dc/d31/de6/lef 0 2024-05-06T12:14:37.624 INFO:tasks.workunit.client.0.smithi092.stdout:3/477: chown f4 11500376 1 2024-05-06T12:14:37.624 INFO:tasks.workunit.client.0.smithi092.stdout:6/660: rename d4/c7 to d4/d4c/d52/d6c/cda 0 2024-05-06T12:14:37.626 INFO:tasks.workunit.client.0.smithi092.stdout:3/478: truncate d8/db/f6c 481250 0 2024-05-06T12:14:37.627 INFO:tasks.workunit.client.0.smithi092.stdout:1/538: creat d3/d42/d84/d4f/fa8 x:0 0 0 2024-05-06T12:14:37.627 INFO:tasks.workunit.client.0.smithi092.stdout:3/479: chown d8/db/d4c/d3d/d49 31 1 2024-05-06T12:14:37.698 INFO:tasks.workunit.client.0.smithi092.stdout:6/661: symlink d4/d24/d9f/ldb 0 2024-05-06T12:14:37.704 INFO:tasks.workunit.client.0.smithi092.stdout:7/673: write d9/de/d10/d17/d99/f83 [560960,110128] 0 2024-05-06T12:14:37.707 INFO:tasks.workunit.client.0.smithi092.stdout:6/662: mkdir d4/d4c/d52/ddc 0 2024-05-06T12:14:37.709 INFO:tasks.workunit.client.0.smithi092.stdout:9/535: truncate d16/d2c/d2f/f31 4746588 0 2024-05-06T12:14:37.710 INFO:tasks.workunit.client.0.smithi092.stdout:3/480: truncate d8/d24/f65 956013 0 2024-05-06T12:14:37.711 INFO:tasks.workunit.client.0.smithi092.stdout:7/674: dread d9/de/d1f/d37/d48/d65/f7c [0,4194304] 0 2024-05-06T12:14:37.711 INFO:tasks.workunit.client.0.smithi092.stdout:1/539: getdents d3/d42/d84/d4f/d58/d65/d81/d8d 0 2024-05-06T12:14:37.711 INFO:tasks.workunit.client.0.smithi092.stdout:0/738: dwrite d7/dc/d28/d54/d1e/d5d/f76 [0,4194304] 0 2024-05-06T12:14:37.713 INFO:tasks.workunit.client.0.smithi092.stdout:3/481: write d8/d24/d27/f4a [3177154,88621] 0 2024-05-06T12:14:37.714 INFO:tasks.workunit.client.0.smithi092.stdout:0/739: stat d7/dc/d28/d54/d12/d7f/db2/dbc 0 2024-05-06T12:14:37.714 INFO:tasks.workunit.client.0.smithi092.stdout:9/536: creat d16/d1c/d58/d92/fa8 x:0 0 0 2024-05-06T12:14:37.718 INFO:tasks.workunit.client.0.smithi092.stdout:1/540: read - d3/d10/d13/d19/f87 zero size 2024-05-06T12:14:37.719 INFO:tasks.workunit.client.0.smithi092.stdout:7/675: rename d9/de/d10/d17/d6a/fd1 to d9/d62/fe1 0 2024-05-06T12:14:37.720 INFO:tasks.workunit.client.0.smithi092.stdout:3/482: fdatasync d8/db/d4c/d3d/d49/f8a 0 2024-05-06T12:14:37.721 INFO:tasks.workunit.client.0.smithi092.stdout:3/483: chown d8/db/f6c 6 1 2024-05-06T12:14:37.723 INFO:tasks.workunit.client.0.smithi092.stdout:7/676: dread d9/de/d10/d17/d99/db6/fc5 [0,4194304] 0 2024-05-06T12:14:37.723 INFO:tasks.workunit.client.0.smithi092.stdout:0/740: mkdir d7/dc/d28/d54/d1e/d46/df0 0 2024-05-06T12:14:37.725 INFO:tasks.workunit.client.0.smithi092.stdout:7/677: chown d9/de/d10/d17/d21/d38/cbd 3 1 2024-05-06T12:14:37.725 INFO:tasks.workunit.client.0.smithi092.stdout:6/663: getdents d4/d24/d9f 0 2024-05-06T12:14:37.730 INFO:tasks.workunit.client.0.smithi092.stdout:8/575: sync 2024-05-06T12:14:37.730 INFO:tasks.workunit.client.0.smithi092.stdout:4/618: sync 2024-05-06T12:14:37.730 INFO:tasks.workunit.client.0.smithi092.stdout:5/517: sync 2024-05-06T12:14:37.730 INFO:tasks.workunit.client.0.smithi092.stdout:0/741: creat d7/dc/d28/d54/d12/d7f/db5/ff1 x:0 0 0 2024-05-06T12:14:37.731 INFO:tasks.workunit.client.0.smithi092.stdout:8/576: chown d9/fc2 1 1 2024-05-06T12:14:37.731 INFO:tasks.workunit.client.0.smithi092.stdout:5/518: chown d4/d5/d10/d31/c69 163 1 2024-05-06T12:14:37.733 INFO:tasks.workunit.client.0.smithi092.stdout:8/577: write d9/d21/d37/d46/d5c/d62/da5/fc7 [629262,70852] 0 2024-05-06T12:14:37.734 INFO:tasks.workunit.client.0.smithi092.stdout:8/578: chown d9/d5a/f78 593326 1 2024-05-06T12:14:37.738 INFO:tasks.workunit.client.0.smithi092.stdout:4/619: mknod d9/df/d48/cd0 0 2024-05-06T12:14:37.740 INFO:tasks.workunit.client.0.smithi092.stdout:1/541: creat d3/d10/d13/d19/fa9 x:0 0 0 2024-05-06T12:14:37.741 INFO:tasks.workunit.client.0.smithi092.stdout:9/537: truncate d16/d2c/d4b/d76/f83 1678885 0 2024-05-06T12:14:37.742 INFO:tasks.workunit.client.0.smithi092.stdout:5/519: dread d4/d34/d29/d54/d9f/fb3 [0,4194304] 0 2024-05-06T12:14:37.744 INFO:tasks.workunit.client.0.smithi092.stdout:8/579: chown d9/d21/f81 113243 1 2024-05-06T12:14:37.744 INFO:tasks.workunit.client.0.smithi092.stdout:3/484: dread d8/f12 [0,4194304] 0 2024-05-06T12:14:37.748 INFO:tasks.workunit.client.0.smithi092.stdout:9/538: symlink d16/d1c/la9 0 2024-05-06T12:14:37.751 INFO:tasks.workunit.client.0.smithi092.stdout:8/580: unlink d9/dc/d4a/d5f/da8/fd1 0 2024-05-06T12:14:37.752 INFO:tasks.workunit.client.0.smithi092.stdout:1/542: dwrite d3/d10/d13/d19/d34/f51 [0,4194304] 0 2024-05-06T12:14:37.762 INFO:tasks.workunit.client.0.smithi092.stdout:3/485: creat d8/db/d4c/d40/d7a/d90/f93 x:0 0 0 2024-05-06T12:14:37.766 INFO:tasks.workunit.client.0.smithi092.stdout:0/742: dwrite d7/f9 [0,4194304] 0 2024-05-06T12:14:37.766 INFO:tasks.workunit.client.0.smithi092.stdout:9/539: rename d16/d2c/d4b/l5c to d16/d1e/d7b/d8d/laa 0 2024-05-06T12:14:37.766 INFO:tasks.workunit.client.0.smithi092.stdout:4/620: dwrite d9/d2b/fac [0,4194304] 0 2024-05-06T12:14:37.768 INFO:tasks.workunit.client.0.smithi092.stdout:4/621: write d9/d2b/d7c/dab/fc6 [969219,50550] 0 2024-05-06T12:14:37.771 INFO:tasks.workunit.client.0.smithi092.stdout:7/678: dwrite d9/de/d10/d17/d6a/f7b [0,4194304] 0 2024-05-06T12:14:37.772 INFO:tasks.workunit.client.0.smithi092.stdout:1/543: write d3/d10/d13/f1f [2336947,120044] 0 2024-05-06T12:14:37.774 INFO:tasks.workunit.client.0.smithi092.stdout:4/622: chown d9/df/d48/c79 49 1 2024-05-06T12:14:37.776 INFO:tasks.workunit.client.0.smithi092.stdout:4/623: dread - d9/d2b/f6a zero size 2024-05-06T12:14:37.777 INFO:tasks.workunit.client.0.smithi092.stdout:3/486: creat d8/db/d14/d5a/f94 x:0 0 0 2024-05-06T12:14:37.777 INFO:tasks.workunit.client.0.smithi092.stdout:9/540: mkdir d16/d1c/d58/d92/dab 0 2024-05-06T12:14:37.779 INFO:tasks.workunit.client.0.smithi092.stdout:1/544: fdatasync d3/d10/d13/d19/f87 0 2024-05-06T12:14:37.780 INFO:tasks.workunit.client.0.smithi092.stdout:0/743: chown d7/dc/d28/d54/d1e/c25 71441800 1 2024-05-06T12:14:37.783 INFO:tasks.workunit.client.0.smithi092.stdout:5/520: dwrite d4/d59/f83 [4194304,4194304] 0 2024-05-06T12:14:37.783 INFO:tasks.workunit.client.0.smithi092.stdout:9/541: write d16/d1e/f70 [34997,8796] 0 2024-05-06T12:14:37.787 INFO:tasks.workunit.client.0.smithi092.stdout:4/624: rename d9/df/d48/cb1 to d9/dd/cd1 0 2024-05-06T12:14:37.787 INFO:tasks.workunit.client.0.smithi092.stdout:1/545: write d3/f98 [3865320,37973] 0 2024-05-06T12:14:37.787 INFO:tasks.workunit.client.0.smithi092.stdout:3/487: read d8/d24/d33/d39/f44 [2775774,95929] 0 2024-05-06T12:14:37.787 INFO:tasks.workunit.client.0.smithi092.stdout:0/744: write d7/dc/d28/d54/d12/f93 [619937,115939] 0 2024-05-06T12:14:37.793 INFO:tasks.workunit.client.0.smithi092.stdout:9/542: rmdir d16/d2c/d2f/d3d/d33/d7e 39 2024-05-06T12:14:37.795 INFO:tasks.workunit.client.0.smithi092.stdout:0/745: write d7/dc/d28/d54/d1e/d39/fca [953633,96004] 0 2024-05-06T12:14:37.795 INFO:tasks.workunit.client.0.smithi092.stdout:4/625: creat d9/d27/da8/db3/fd2 x:0 0 0 2024-05-06T12:14:37.796 INFO:tasks.workunit.client.0.smithi092.stdout:3/488: symlink d8/d1b/d73/l95 0 2024-05-06T12:14:37.797 INFO:tasks.workunit.client.0.smithi092.stdout:9/543: readlink d16/d2c/d2f/d3d/d33/d57/l6c 0 2024-05-06T12:14:37.798 INFO:tasks.workunit.client.0.smithi092.stdout:1/546: truncate d3/d10/d13/d46/f92 449842 0 2024-05-06T12:14:37.798 INFO:tasks.workunit.client.0.smithi092.stdout:4/626: chown d9/d2b/d5b/fb5 2905 1 2024-05-06T12:14:37.799 INFO:tasks.workunit.client.0.smithi092.stdout:5/521: dwrite d4/d34/f56 [4194304,4194304] 0 2024-05-06T12:14:37.799 INFO:tasks.workunit.client.0.smithi092.stdout:3/489: dread - d8/d24/d33/d39/d5b/f6d zero size 2024-05-06T12:14:37.801 INFO:tasks.workunit.client.0.smithi092.stdout:4/627: chown d9/df/d62/c8e 7029 1 2024-05-06T12:14:37.802 INFO:tasks.workunit.client.0.smithi092.stdout:1/547: write d3/d10/d13/d46/f7b [5167648,109942] 0 2024-05-06T12:14:37.807 INFO:tasks.workunit.client.0.smithi092.stdout:0/746: rename d7/dc/d28/d54/d12/d7f/dda to d7/dc/d6f/da2/df2 0 2024-05-06T12:14:37.811 INFO:tasks.workunit.client.0.smithi092.stdout:8/581: dread d9/d21/d54/f63 [0,4194304] 0 2024-05-06T12:14:37.812 INFO:tasks.workunit.client.0.smithi092.stdout:0/747: write d7/dc/d28/d54/d12/f1b [257924,84657] 0 2024-05-06T12:14:37.814 INFO:tasks.workunit.client.0.smithi092.stdout:3/490: dwrite d8/d1b/f3e [4194304,4194304] 0 2024-05-06T12:14:37.818 INFO:tasks.workunit.client.0.smithi092.stdout:5/522: rename d4/d34/d29/d54/d9f/fb3 to d4/d34/d29/da3/fbf 0 2024-05-06T12:14:37.818 INFO:tasks.workunit.client.0.smithi092.stdout:9/544: getdents d16/d2c/d2f/d75 0 2024-05-06T12:14:37.818 INFO:tasks.workunit.client.0.smithi092.stdout:0/748: mkdir d7/dc/d28/d54/d1e/d46/df3 0 2024-05-06T12:14:37.821 INFO:tasks.workunit.client.0.smithi092.stdout:3/491: creat d8/db/d4c/d3d/f96 x:0 0 0 2024-05-06T12:14:37.823 INFO:tasks.workunit.client.0.smithi092.stdout:9/545: mknod d16/d2c/d4b/cac 0 2024-05-06T12:14:37.826 INFO:tasks.workunit.client.0.smithi092.stdout:0/749: symlink d7/dc/lf4 0 2024-05-06T12:14:37.829 INFO:tasks.workunit.client.0.smithi092.stdout:4/628: dread d9/d2b/f8f [0,4194304] 0 2024-05-06T12:14:37.830 INFO:tasks.workunit.client.0.smithi092.stdout:1/548: dread d3/f2c [0,4194304] 0 2024-05-06T12:14:37.830 INFO:tasks.workunit.client.0.smithi092.stdout:4/629: stat d9/f21 0 2024-05-06T12:14:37.831 INFO:tasks.workunit.client.0.smithi092.stdout:0/750: symlink d7/dc/d28/d54/d12/d7f/db2/lf5 0 2024-05-06T12:14:37.832 INFO:tasks.workunit.client.0.smithi092.stdout:4/630: readlink d9/df/d48/l8b 0 2024-05-06T12:14:37.833 INFO:tasks.workunit.client.0.smithi092.stdout:8/582: dread d9/dc/ff [0,4194304] 0 2024-05-06T12:14:37.833 INFO:tasks.workunit.client.0.smithi092.stdout:0/751: write d7/dc/d28/d54/d1e/d26/d4e/fa0 [1068259,122233] 0 2024-05-06T12:14:37.837 INFO:tasks.workunit.client.0.smithi092.stdout:0/752: fdatasync d7/dc/d28/d54/d1e/d26/d3a/f52 0 2024-05-06T12:14:37.838 INFO:tasks.workunit.client.0.smithi092.stdout:4/631: dread d9/d27/d4a/f4c [0,4194304] 0 2024-05-06T12:14:37.839 INFO:tasks.workunit.client.0.smithi092.stdout:3/492: write d8/d1b/f2b [2942143,47753] 0 2024-05-06T12:14:37.844 INFO:tasks.workunit.client.0.smithi092.stdout:8/583: link d9/d21/d37/dbd/fd5 d9/d5a/dd9/fda 0 2024-05-06T12:14:37.845 INFO:tasks.workunit.client.0.smithi092.stdout:4/632: symlink d9/d27/da8/ld3 0 2024-05-06T12:14:37.845 INFO:tasks.workunit.client.0.smithi092.stdout:1/549: truncate d3/d10/d13/d19/f5a 2513169 0 2024-05-06T12:14:37.848 INFO:tasks.workunit.client.0.smithi092.stdout:0/753: rename d7/dc/d28/d54/d1e/d46/f7b to d7/dc/d28/d54/d1e/d46/df3/ff6 0 2024-05-06T12:14:37.848 INFO:tasks.workunit.client.0.smithi092.stdout:1/550: dread - d3/d10/d13/d46/fa1 zero size 2024-05-06T12:14:37.848 INFO:tasks.workunit.client.0.smithi092.stdout:5/523: dwrite d4/d34/d29/da3/fbf [0,4194304] 0 2024-05-06T12:14:37.849 INFO:tasks.workunit.client.0.smithi092.stdout:3/493: creat d8/db/d4c/d40/d67/f97 x:0 0 0 2024-05-06T12:14:37.850 INFO:tasks.workunit.client.0.smithi092.stdout:1/551: fdatasync d3/d10/d13/d19/d21/f24 0 2024-05-06T12:14:37.855 INFO:tasks.workunit.client.0.smithi092.stdout:3/494: mkdir d8/d24/d33/d39/d5b/d98 0 2024-05-06T12:14:37.857 INFO:tasks.workunit.client.0.smithi092.stdout:1/552: write d3/d10/d13/d46/f55 [2639577,118798] 0 2024-05-06T12:14:37.857 INFO:tasks.workunit.client.0.smithi092.stdout:5/524: creat d4/d34/d28/d4e/d5d/d82/fc0 x:0 0 0 2024-05-06T12:14:37.858 INFO:tasks.workunit.client.0.smithi092.stdout:8/584: dwrite d9/d21/d54/f63 [0,4194304] 0 2024-05-06T12:14:37.865 INFO:tasks.workunit.client.0.smithi092.stdout:5/525: mknod d4/d34/d28/d4e/d5d/cc1 0 2024-05-06T12:14:37.866 INFO:tasks.workunit.client.0.smithi092.stdout:3/495: dwrite d8/d1b/f3e [4194304,4194304] 0 2024-05-06T12:14:37.867 INFO:tasks.workunit.client.0.smithi092.stdout:4/633: dread d9/d27/f41 [0,4194304] 0 2024-05-06T12:14:37.873 INFO:tasks.workunit.client.0.smithi092.stdout:4/634: write d9/df/d62/dbb/fc9 [697606,106206] 0 2024-05-06T12:14:37.874 INFO:tasks.workunit.client.0.smithi092.stdout:8/585: dwrite d9/d21/d54/d79/d8a/fb2 [0,4194304] 0 2024-05-06T12:14:37.877 INFO:tasks.workunit.client.0.smithi092.stdout:0/754: dwrite d7/dc/d28/f2e [0,4194304] 0 2024-05-06T12:14:37.877 INFO:tasks.workunit.client.0.smithi092.stdout:1/553: link d3/d42/d84/d4f/fa8 d3/d10/d13/faa 0 2024-05-06T12:14:37.880 INFO:tasks.workunit.client.0.smithi092.stdout:3/496: creat d8/d24/d33/d39/f99 x:0 0 0 2024-05-06T12:14:37.880 INFO:tasks.workunit.client.0.smithi092.stdout:1/554: chown d3/d10/d13/d19/f87 7 1 2024-05-06T12:14:37.885 INFO:tasks.workunit.client.0.smithi092.stdout:8/586: write d9/d21/d37/f3b [7863207,131025] 0 2024-05-06T12:14:37.886 INFO:tasks.workunit.client.0.smithi092.stdout:2/379: sync 2024-05-06T12:14:37.887 INFO:tasks.workunit.client.0.smithi092.stdout:8/587: write d9/dc/f87 [595509,40120] 0 2024-05-06T12:14:37.888 INFO:tasks.workunit.client.0.smithi092.stdout:3/497: chown d8/db/d4c/d3d/d49/f4f 1 1 2024-05-06T12:14:37.889 INFO:tasks.workunit.client.0.smithi092.stdout:1/555: symlink d3/d10/d13/d19/d21/d88/d8b/lab 0 2024-05-06T12:14:37.891 INFO:tasks.workunit.client.0.smithi092.stdout:2/380: creat df/d17/d1d/d33/f6f x:0 0 0 2024-05-06T12:14:37.892 INFO:tasks.workunit.client.0.smithi092.stdout:1/556: rmdir d3/d42/d84/d9f 39 2024-05-06T12:14:37.894 INFO:tasks.workunit.client.0.smithi092.stdout:2/381: creat df/d17/d51/f70 x:0 0 0 2024-05-06T12:14:37.894 INFO:tasks.workunit.client.0.smithi092.stdout:8/588: dread d9/d21/d54/f63 [0,4194304] 0 2024-05-06T12:14:37.895 INFO:tasks.workunit.client.0.smithi092.stdout:2/382: stat df/d15/c26 0 2024-05-06T12:14:37.895 INFO:tasks.workunit.client.0.smithi092.stdout:2/383: stat df/d1b 0 2024-05-06T12:14:37.896 INFO:tasks.workunit.client.0.smithi092.stdout:1/557: creat d3/d10/d13/d19/d21/d88/d8b/fac x:0 0 0 2024-05-06T12:14:37.898 INFO:tasks.workunit.client.0.smithi092.stdout:2/384: symlink df/d17/d51/l71 0 2024-05-06T12:14:37.898 INFO:tasks.workunit.client.0.smithi092.stdout:1/558: rmdir d3/d42 39 2024-05-06T12:14:37.899 INFO:tasks.workunit.client.0.smithi092.stdout:2/385: chown df/d17/f42 0 1 2024-05-06T12:14:37.900 INFO:tasks.workunit.client.0.smithi092.stdout:8/589: dread d9/dc/fa4 [0,4194304] 0 2024-05-06T12:14:37.901 INFO:tasks.workunit.client.0.smithi092.stdout:2/386: chown df/d17/d1d/d33/d3c 1231244 1 2024-05-06T12:14:37.906 INFO:tasks.workunit.client.0.smithi092.stdout:2/387: creat df/d1b/f72 x:0 0 0 2024-05-06T12:14:37.907 INFO:tasks.workunit.client.0.smithi092.stdout:5/526: dread d4/d5/d10/d31/d53/d64/f5e [0,4194304] 0 2024-05-06T12:14:37.907 INFO:tasks.workunit.client.0.smithi092.stdout:0/755: dread d7/dc/d9e/f49 [0,4194304] 0 2024-05-06T12:14:37.908 INFO:tasks.workunit.client.0.smithi092.stdout:1/559: rename d3/d10/d13/d19/d34/f4b to d3/d42/d84/d4f/d7f/fad 0 2024-05-06T12:14:37.910 INFO:tasks.workunit.client.0.smithi092.stdout:4/635: write d9/f43 [604413,69176] 0 2024-05-06T12:14:37.911 INFO:tasks.workunit.client.0.smithi092.stdout:2/388: chown df/d17/f2a 0 1 2024-05-06T12:14:37.912 INFO:tasks.workunit.client.0.smithi092.stdout:8/590: dwrite d9/d21/d37/d46/f92 [0,4194304] 0 2024-05-06T12:14:37.918 INFO:tasks.workunit.client.0.smithi092.stdout:3/498: truncate d8/db/d4c/d3d/f60 3861285 0 2024-05-06T12:14:37.988 INFO:tasks.workunit.client.0.smithi092.stdout:1/560: creat d3/d10/d13/d19/fae x:0 0 0 2024-05-06T12:14:37.988 INFO:tasks.workunit.client.0.smithi092.stdout:3/499: dread - d8/db/d4c/d40/d7a/d90/f93 zero size 2024-05-06T12:14:37.988 INFO:tasks.workunit.client.0.smithi092.stdout:2/389: truncate df/d17/f22 1089707 0 2024-05-06T12:14:37.990 INFO:tasks.workunit.client.0.smithi092.stdout:8/591: write d9/d21/d37/dbd/fd5 [1047592,48665] 0 2024-05-06T12:14:37.990 INFO:tasks.workunit.client.0.smithi092.stdout:0/756: dwrite d7/dc/d28/d54/d1e/d39/f47 [4194304,4194304] 0 2024-05-06T12:14:37.993 INFO:tasks.workunit.client.0.smithi092.stdout:1/561: dread - d3/d10/d13/d19/d34/d89/fa5 zero size 2024-05-06T12:14:38.001 INFO:tasks.workunit.client.0.smithi092.stdout:0/757: creat d7/dc/d28/d54/d12/d7f/ff7 x:0 0 0 2024-05-06T12:14:38.001 INFO:tasks.workunit.client.0.smithi092.stdout:5/527: dread d4/d34/f56 [0,4194304] 0 2024-05-06T12:14:38.007 INFO:tasks.workunit.client.0.smithi092.stdout:1/562: dwrite d3/d10/d13/d46/f7b [4194304,4194304] 0 2024-05-06T12:14:38.007 INFO:tasks.workunit.client.0.smithi092.stdout:8/592: write d9/d21/d54/d79/f96 [714792,15232] 0 2024-05-06T12:14:38.008 INFO:tasks.workunit.client.0.smithi092.stdout:8/593: chown d9/d21/d54/l6a 932998255 1 2024-05-06T12:14:38.009 INFO:tasks.workunit.client.0.smithi092.stdout:2/390: write df/d17/f22 [1218291,10718] 0 2024-05-06T12:14:38.009 INFO:tasks.workunit.client.0.smithi092.stdout:0/758: getdents d7/dc/d28/d54/d1e/d39/da8/dc5 0 2024-05-06T12:14:38.012 INFO:tasks.workunit.client.0.smithi092.stdout:1/563: mkdir d3/daf 0 2024-05-06T12:14:38.012 INFO:tasks.workunit.client.0.smithi092.stdout:2/391: chown df/d5c/f66 1312138 1 2024-05-06T12:14:38.012 INFO:tasks.workunit.client.0.smithi092.stdout:0/759: read f1 [1648194,51296] 0 2024-05-06T12:14:38.013 INFO:tasks.workunit.client.0.smithi092.stdout:8/594: write d9/d21/d37/d46/d5c/d62/da5/fd6 [1027154,23574] 0 2024-05-06T12:14:38.016 INFO:tasks.workunit.client.0.smithi092.stdout:1/564: chown d3/d42/f43 8 1 2024-05-06T12:14:38.018 INFO:tasks.workunit.client.0.smithi092.stdout:3/500: dread d8/db/d4c/d3d/f8e [0,4194304] 0 2024-05-06T12:14:38.021 INFO:tasks.workunit.client.0.smithi092.stdout:5/528: dwrite d4/f4c [0,4194304] 0 2024-05-06T12:14:38.022 INFO:tasks.workunit.client.0.smithi092.stdout:0/760: truncate d7/dc/d28/d64/fee 576871 0 2024-05-06T12:14:38.023 INFO:tasks.workunit.client.0.smithi092.stdout:0/761: chown d7/dc/l2a 0 1 2024-05-06T12:14:38.023 INFO:tasks.workunit.client.0.smithi092.stdout:2/392: link df/d17/d1d/f4f df/d5c/f73 0 2024-05-06T12:14:38.025 INFO:tasks.workunit.client.0.smithi092.stdout:1/565: mkdir d3/d10/d13/db0 0 2024-05-06T12:14:38.026 INFO:tasks.workunit.client.0.smithi092.stdout:2/393: chown df/d17/f2a 454941204 1 2024-05-06T12:14:38.027 INFO:tasks.workunit.client.0.smithi092.stdout:8/595: dread d9/d21/d37/d46/d5c/d62/da5/fd6 [0,4194304] 0 2024-05-06T12:14:38.027 INFO:tasks.workunit.client.0.smithi092.stdout:5/529: mkdir d4/d34/d29/da3/dc2 0 2024-05-06T12:14:38.027 INFO:tasks.workunit.client.0.smithi092.stdout:0/762: write d7/dc/d28/d64/f89 [85795,100107] 0 2024-05-06T12:14:38.029 INFO:tasks.workunit.client.0.smithi092.stdout:3/501: mknod d8/db/c9a 0 2024-05-06T12:14:38.031 INFO:tasks.workunit.client.0.smithi092.stdout:1/566: dread d3/d10/d13/d46/f61 [0,4194304] 0 2024-05-06T12:14:38.033 INFO:tasks.workunit.client.0.smithi092.stdout:1/567: write d3/d42/d84/f50 [4991427,47024] 0 2024-05-06T12:14:38.035 INFO:tasks.workunit.client.0.smithi092.stdout:0/763: mkdir d7/dc/dd4/df8 0 2024-05-06T12:14:38.035 INFO:tasks.workunit.client.0.smithi092.stdout:1/568: write d3/d10/d13/d46/f7b [7679287,115230] 0 2024-05-06T12:14:38.036 INFO:tasks.workunit.client.0.smithi092.stdout:3/502: truncate d8/d24/d33/f6a 592683 0 2024-05-06T12:14:38.041 INFO:tasks.workunit.client.0.smithi092.stdout:6/664: sync 2024-05-06T12:14:38.042 INFO:tasks.workunit.client.0.smithi092.stdout:3/503: creat d8/d24/d33/d91/f9b x:0 0 0 2024-05-06T12:14:38.046 INFO:tasks.workunit.client.0.smithi092.stdout:1/569: dwrite d3/d42/d84/f5f [0,4194304] 0 2024-05-06T12:14:38.053 INFO:tasks.workunit.client.0.smithi092.stdout:8/596: dwrite d9/d21/f6f [0,4194304] 0 2024-05-06T12:14:38.056 INFO:tasks.workunit.client.0.smithi092.stdout:3/504: dwrite d8/f12 [0,4194304] 0 2024-05-06T12:14:38.056 INFO:tasks.workunit.client.0.smithi092.stdout:6/665: truncate d4/d24/d38/faa 1698424 0 2024-05-06T12:14:38.056 INFO:tasks.workunit.client.0.smithi092.stdout:8/597: write d9/dc/f87 [32021,75522] 0 2024-05-06T12:14:38.058 INFO:tasks.workunit.client.0.smithi092.stdout:0/764: dread f1 [0,4194304] 0 2024-05-06T12:14:38.058 INFO:tasks.workunit.client.0.smithi092.stdout:6/666: write d4/f17 [6032085,43977] 0 2024-05-06T12:14:38.062 INFO:tasks.workunit.client.0.smithi092.stdout:8/598: mkdir d9/d21/d37/dbd/ddb 0 2024-05-06T12:14:38.065 INFO:tasks.workunit.client.0.smithi092.stdout:2/394: dread df/d17/f6b [0,4194304] 0 2024-05-06T12:14:38.071 INFO:tasks.workunit.client.0.smithi092.stdout:8/599: read d9/fb6 [32175,88022] 0 2024-05-06T12:14:38.076 INFO:tasks.workunit.client.0.smithi092.stdout:0/765: dwrite d7/dc/d28/d54/d1e/d26/d96/fc3 [0,4194304] 0 2024-05-06T12:14:38.078 INFO:tasks.workunit.client.0.smithi092.stdout:8/600: chown d9/d21/d54/c61 249676868 1 2024-05-06T12:14:38.083 INFO:tasks.workunit.client.0.smithi092.stdout:6/667: dwrite d4/d20/d23/d30/f5e [0,4194304] 0 2024-05-06T12:14:38.086 INFO:tasks.workunit.client.0.smithi092.stdout:0/766: mknod d7/dc/d28/d54/d1e/d26/d96/cf9 0 2024-05-06T12:14:38.088 INFO:tasks.workunit.client.0.smithi092.stdout:8/601: read d9/d21/f3f [161978,76160] 0 2024-05-06T12:14:38.089 INFO:tasks.workunit.client.0.smithi092.stdout:8/602: stat d9/d9d/da6 0 2024-05-06T12:14:38.097 INFO:tasks.workunit.client.0.smithi092.stdout:2/395: dwrite df/d17/f42 [0,4194304] 0 2024-05-06T12:14:38.097 INFO:tasks.workunit.client.0.smithi092.stdout:6/668: dwrite d4/d24/d26/dae/dc0/fcc [0,4194304] 0 2024-05-06T12:14:38.104 INFO:tasks.workunit.client.0.smithi092.stdout:8/603: dwrite d9/d21/d54/d79/f96 [0,4194304] 0 2024-05-06T12:14:38.107 INFO:tasks.workunit.client.0.smithi092.stdout:8/604: creat d9/d21/d37/dbd/fdc x:0 0 0 2024-05-06T12:14:38.109 INFO:tasks.workunit.client.0.smithi092.stdout:2/396: dread df/d17/f4a [0,4194304] 0 2024-05-06T12:14:38.112 INFO:tasks.workunit.client.0.smithi092.stdout:2/397: dread - df/d17/d51/f70 zero size 2024-05-06T12:14:38.112 INFO:tasks.workunit.client.0.smithi092.stdout:6/669: link d4/l3f d4/d24/d4e/ldd 0 2024-05-06T12:14:38.114 INFO:tasks.workunit.client.0.smithi092.stdout:2/398: dread - df/d15/f36 zero size 2024-05-06T12:14:38.115 INFO:tasks.workunit.client.0.smithi092.stdout:2/399: dread - df/d17/d1d/f4f zero size 2024-05-06T12:14:38.119 INFO:tasks.workunit.client.0.smithi092.stdout:2/400: mknod df/d17/d51/c74 0 2024-05-06T12:14:38.121 INFO:tasks.workunit.client.0.smithi092.stdout:0/767: truncate d7/dc/d9e/f61 258510 0 2024-05-06T12:14:38.122 INFO:tasks.workunit.client.0.smithi092.stdout:2/401: rename df/d1b/c64 to df/d17/d1d/d33/d3c/c75 0 2024-05-06T12:14:38.123 INFO:tasks.workunit.client.0.smithi092.stdout:0/768: write d7/dc/d28/d54/d1e/d39/da8/dc5/fd1 [362485,9134] 0 2024-05-06T12:14:38.130 INFO:tasks.workunit.client.0.smithi092.stdout:0/769: dread d7/dc/d28/d54/d1e/d5d/f76 [0,4194304] 0 2024-05-06T12:14:38.136 INFO:tasks.workunit.client.0.smithi092.stdout:0/770: rmdir d7/dc/d28/d54/d12/d8f/dad 39 2024-05-06T12:14:38.137 INFO:tasks.workunit.client.0.smithi092.stdout:8/605: dwrite d9/f25 [0,4194304] 0 2024-05-06T12:14:38.139 INFO:tasks.workunit.client.0.smithi092.stdout:6/670: dread d4/d24/d26/dae/dc0/fad [0,4194304] 0 2024-05-06T12:14:38.140 INFO:tasks.workunit.client.0.smithi092.stdout:2/402: dwrite fd [4194304,4194304] 0 2024-05-06T12:14:38.140 INFO:tasks.workunit.client.0.smithi092.stdout:0/771: chown d7/dc/d28/d54/d12/fd8 668 1 2024-05-06T12:14:38.146 INFO:tasks.workunit.client.0.smithi092.stdout:2/403: truncate df/d17/d1d/f41 5075662 0 2024-05-06T12:14:38.146 INFO:tasks.workunit.client.0.smithi092.stdout:8/606: symlink d9/d21/d37/d46/d5c/d71/d75/ldd 0 2024-05-06T12:14:38.150 INFO:tasks.workunit.client.0.smithi092.stdout:2/404: creat df/d15/f76 x:0 0 0 2024-05-06T12:14:38.152 INFO:tasks.workunit.client.0.smithi092.stdout:6/671: dwrite d4/d4c/d91/fb6 [4194304,4194304] 0 2024-05-06T12:14:38.156 INFO:tasks.workunit.client.0.smithi092.stdout:8/607: creat d9/d21/d37/d46/d5c/fde x:0 0 0 2024-05-06T12:14:38.158 INFO:tasks.workunit.client.0.smithi092.stdout:8/608: mkdir d9/dc/d4a/d5f/da8/ddf 0 2024-05-06T12:14:38.170 INFO:tasks.workunit.client.0.smithi092.stdout:0/772: truncate d7/dc/d28/d64/fee 515615 0 2024-05-06T12:14:38.172 INFO:tasks.workunit.client.0.smithi092.stdout:8/609: truncate d9/dc/f4d 1807992 0 2024-05-06T12:14:38.173 INFO:tasks.workunit.client.0.smithi092.stdout:0/773: mknod d7/dc/d28/d54/d12/d7f/db2/dbc/cfa 0 2024-05-06T12:14:38.176 INFO:tasks.workunit.client.0.smithi092.stdout:2/405: dwrite df/d1b/f1c [0,4194304] 0 2024-05-06T12:14:38.187 INFO:tasks.workunit.client.0.smithi092.stdout:8/610: dread d9/f2a [0,4194304] 0 2024-05-06T12:14:38.198 INFO:tasks.workunit.client.0.smithi092.stdout:7/679: sync 2024-05-06T12:14:38.199 INFO:tasks.workunit.client.0.smithi092.stdout:9/546: sync 2024-05-06T12:14:38.200 INFO:tasks.workunit.client.0.smithi092.stdout:7/680: dread - d9/de/d1f/dcf/fd6 zero size 2024-05-06T12:14:38.202 INFO:tasks.workunit.client.0.smithi092.stdout:0/774: mkdir d7/dc/d28/d54/d1e/d26/d96/dfb 0 2024-05-06T12:14:38.205 INFO:tasks.workunit.client.0.smithi092.stdout:8/611: rename d9/d5a/f78 to d9/d21/d37/d46/d5c/d62/da5/fe0 0 2024-05-06T12:14:38.207 INFO:tasks.workunit.client.0.smithi092.stdout:7/681: dread d9/de/d10/d17/d99/db6/fc5 [0,4194304] 0 2024-05-06T12:14:38.208 INFO:tasks.workunit.client.0.smithi092.stdout:0/775: symlink d7/dc/d28/d64/lfc 0 2024-05-06T12:14:38.212 INFO:tasks.workunit.client.0.smithi092.stdout:8/612: chown d9/d21/d72/cab 211332 1 2024-05-06T12:14:38.218 INFO:tasks.workunit.client.0.smithi092.stdout:2/406: getdents df 0 2024-05-06T12:14:38.219 INFO:tasks.workunit.client.0.smithi092.stdout:2/407: readlink df/d17/d1d/l6c 0 2024-05-06T12:14:38.220 INFO:tasks.workunit.client.0.smithi092.stdout:2/408: write df/f61 [853230,33593] 0 2024-05-06T12:14:38.220 INFO:tasks.workunit.client.0.smithi092.stdout:0/776: mknod d7/dc/d28/d54/d1e/d26/d96/dfb/cfd 0 2024-05-06T12:14:38.220 INFO:tasks.workunit.client.0.smithi092.stdout:9/547: dread d16/d2c/d2f/d3d/f67 [0,4194304] 0 2024-05-06T12:14:38.222 INFO:tasks.workunit.client.0.smithi092.stdout:8/613: link d9/dc/d4a/f6b d9/d5a/d9f/fe1 0 2024-05-06T12:14:38.225 INFO:tasks.workunit.client.0.smithi092.stdout:7/682: rename d9/de/d1f/dcf/fde to d9/de/d10/d17/d6a/fe2 0 2024-05-06T12:14:38.230 INFO:tasks.workunit.client.0.smithi092.stdout:0/777: write d7/dc/d28/d54/d1e/d26/d96/f87 [3397633,39336] 0 2024-05-06T12:14:38.230 INFO:tasks.workunit.client.0.smithi092.stdout:9/548: dwrite d16/d2c/d2f/d3d/f38 [0,4194304] 0 2024-05-06T12:14:38.230 INFO:tasks.workunit.client.0.smithi092.stdout:8/614: mknod d9/ce2 0 2024-05-06T12:14:38.230 INFO:tasks.workunit.client.0.smithi092.stdout:7/683: creat d9/de/d2d/fe3 x:0 0 0 2024-05-06T12:14:38.235 INFO:tasks.workunit.client.0.smithi092.stdout:8/615: write d9/d21/f88 [1042845,66562] 0 2024-05-06T12:14:38.239 INFO:tasks.workunit.client.0.smithi092.stdout:0/778: dwrite d7/dc/d28/d54/d1e/d46/d63/f72 [0,4194304] 0 2024-05-06T12:14:38.240 INFO:tasks.workunit.client.0.smithi092.stdout:9/549: rename d16/d2c/d2f/d3d/d33/d37/d8a to d16/d2c/d2f/d3d/d33/d7e/dad 0 2024-05-06T12:14:38.241 INFO:tasks.workunit.client.0.smithi092.stdout:8/616: mknod d9/d21/d37/d46/d5c/d62/da5/ce3 0 2024-05-06T12:14:38.242 INFO:tasks.workunit.client.0.smithi092.stdout:0/779: fsync d7/dc/d28/d54/d12/fd8 0 2024-05-06T12:14:38.243 INFO:tasks.workunit.client.0.smithi092.stdout:8/617: write d9/d21/d54/d79/f96 [3756179,87461] 0 2024-05-06T12:14:38.243 INFO:tasks.workunit.client.0.smithi092.stdout:8/618: chown d9/d21/d37/d46 199337052 1 2024-05-06T12:14:38.260 INFO:tasks.workunit.client.0.smithi092.stdout:7/684: dread d9/fb [0,4194304] 0 2024-05-06T12:14:38.266 INFO:tasks.workunit.client.0.smithi092.stdout:7/685: dwrite d9/de/d10/f2b [4194304,4194304] 0 2024-05-06T12:14:38.270 INFO:tasks.workunit.client.0.smithi092.stdout:7/686: unlink d9/de/d10/d17/d21/d38/c55 0 2024-05-06T12:14:38.272 INFO:tasks.workunit.client.0.smithi092.stdout:7/687: mknod d9/de/d1f/d37/d48/ce4 0 2024-05-06T12:14:38.277 INFO:tasks.workunit.client.0.smithi092.stdout:7/688: dwrite d9/de/d10/d17/d21/f5d [0,4194304] 0 2024-05-06T12:14:38.331 INFO:tasks.workunit.client.0.smithi092.stdout:9/550: write d16/d2c/f91 [944302,603] 0 2024-05-06T12:14:38.332 INFO:tasks.workunit.client.0.smithi092.stdout:9/551: write d16/d1c/f7c [3708913,74660] 0 2024-05-06T12:14:38.334 INFO:tasks.workunit.client.0.smithi092.stdout:9/552: creat d16/d2c/d4b/fae x:0 0 0 2024-05-06T12:14:38.338 INFO:tasks.workunit.client.0.smithi092.stdout:9/553: rename c10 to d16/d1e/da1/d43/caf 0 2024-05-06T12:14:38.340 INFO:tasks.workunit.client.0.smithi092.stdout:0/780: dwrite d7/dc/d28/d54/d1e/f3e [0,4194304] 0 2024-05-06T12:14:38.341 INFO:tasks.workunit.client.0.smithi092.stdout:7/689: truncate d9/dd/f43 2004659 0 2024-05-06T12:14:38.344 INFO:tasks.workunit.client.0.smithi092.stdout:9/554: rename d16/d2c/d2f/d3d/f62 to d16/d1e/d7b/fb0 0 2024-05-06T12:14:38.345 INFO:tasks.workunit.client.0.smithi092.stdout:7/690: write d9/de/d10/d17/d21/d38/f90 [5128237,69204] 0 2024-05-06T12:14:38.347 INFO:tasks.workunit.client.0.smithi092.stdout:9/555: truncate d16/d2c/d2f/d3d/d33/d37/f93 105043 0 2024-05-06T12:14:38.350 INFO:tasks.workunit.client.0.smithi092.stdout:9/556: mkdir d16/d2c/d2f/db1 0 2024-05-06T12:14:38.350 INFO:tasks.workunit.client.0.smithi092.stdout:7/691: getdents d9/de/d10/d17/d6a 0 2024-05-06T12:14:38.355 INFO:tasks.workunit.client.0.smithi092.stdout:7/692: creat d9/de/d10/d17/d21/d38/d77/fe5 x:0 0 0 2024-05-06T12:14:38.357 INFO:tasks.workunit.client.0.smithi092.stdout:7/693: rmdir d9 39 2024-05-06T12:14:38.361 INFO:tasks.workunit.client.0.smithi092.stdout:7/694: symlink d9/de/d1f/d37/le6 0 2024-05-06T12:14:38.362 INFO:tasks.workunit.client.0.smithi092.stdout:0/781: truncate d7/dc/d28/d54/d1e/d5d/f7c 2889944 0 2024-05-06T12:14:38.366 INFO:tasks.workunit.client.0.smithi092.stdout:7/695: symlink d9/de/d10/d17/d99/da6/le7 0 2024-05-06T12:14:38.368 INFO:tasks.workunit.client.0.smithi092.stdout:7/696: chown d9/de/d10/d17/d21/d2e/d87/f9f 376 1 2024-05-06T12:14:38.369 INFO:tasks.workunit.client.0.smithi092.stdout:9/557: dwrite d16/d2c/d2f/f31 [0,4194304] 0 2024-05-06T12:14:38.378 INFO:tasks.workunit.client.0.smithi092.stdout:7/697: creat d9/fe8 x:0 0 0 2024-05-06T12:14:38.381 INFO:tasks.workunit.client.0.smithi092.stdout:9/558: dwrite d16/d2c/d2f/d3d/f7d [0,4194304] 0 2024-05-06T12:14:38.381 INFO:tasks.workunit.client.0.smithi092.stdout:7/698: creat d9/de/d1f/dcf/fe9 x:0 0 0 2024-05-06T12:14:38.383 INFO:tasks.workunit.client.0.smithi092.stdout:0/782: dwrite d7/dc/d28/d54/d12/f14 [0,4194304] 0 2024-05-06T12:14:38.384 INFO:tasks.workunit.client.0.smithi092.stdout:9/559: symlink d16/d94/da0/lb2 0 2024-05-06T12:14:38.390 INFO:tasks.workunit.client.0.smithi092.stdout:9/560: dread d16/d2c/d2f/f31 [0,4194304] 0 2024-05-06T12:14:38.392 INFO:tasks.workunit.client.0.smithi092.stdout:7/699: rename d9/de/d1f/d37/d48/d65/ce0 to d9/de/d1f/d37/d48/d97/cea 0 2024-05-06T12:14:38.394 INFO:tasks.workunit.client.0.smithi092.stdout:7/700: dread - d9/de/d10/d17/d99/db6/fda zero size 2024-05-06T12:14:38.394 INFO:tasks.workunit.client.0.smithi092.stdout:0/783: link d7/c3d d7/dc/d28/d54/d1e/d46/df3/cfe 0 2024-05-06T12:14:38.394 INFO:tasks.workunit.client.0.smithi092.stdout:9/561: write d16/d2c/d2f/d3d/d33/d6b/f77 [4891233,112892] 0 2024-05-06T12:14:38.397 INFO:tasks.workunit.client.0.smithi092.stdout:9/562: symlink d16/d1e/da1/lb3 0 2024-05-06T12:14:38.398 INFO:tasks.workunit.client.0.smithi092.stdout:9/563: stat d16/d1e/f6a 0 2024-05-06T12:14:38.400 INFO:tasks.workunit.client.0.smithi092.stdout:9/564: fdatasync d16/d2c/d3f/f60 0 2024-05-06T12:14:38.400 INFO:tasks.workunit.client.0.smithi092.stdout:0/784: symlink d7/dc/d28/d54/d1e/d46/lff 0 2024-05-06T12:14:38.405 INFO:tasks.workunit.client.0.smithi092.stdout:0/785: mkdir d7/dc/d6f/da5/dc6/d100 0 2024-05-06T12:14:38.405 INFO:tasks.workunit.client.0.smithi092.stdout:7/701: dwrite d9/de/d10/d17/faf [0,4194304] 0 2024-05-06T12:14:38.407 INFO:tasks.workunit.client.0.smithi092.stdout:7/702: chown d9/la 972 1 2024-05-06T12:14:38.407 INFO:tasks.workunit.client.0.smithi092.stdout:4/636: sync 2024-05-06T12:14:38.410 INFO:tasks.workunit.client.0.smithi092.stdout:9/565: dwrite d16/d2c/d3f/f96 [0,4194304] 0 2024-05-06T12:14:38.410 INFO:tasks.workunit.client.0.smithi092.stdout:4/637: dread d9/d2b/d7c/f95 [0,4194304] 0 2024-05-06T12:14:38.411 INFO:tasks.workunit.client.0.smithi092.stdout:4/638: stat d9/df/c26 0 2024-05-06T12:14:38.412 INFO:tasks.workunit.client.0.smithi092.stdout:0/786: rename d7/dc/d28/d54/d1e/d46/c91 to d7/dc/c101 0 2024-05-06T12:14:38.418 INFO:tasks.workunit.client.0.smithi092.stdout:9/566: mkdir d16/d1e/da1/d78/db4 0 2024-05-06T12:14:38.420 INFO:tasks.workunit.client.0.smithi092.stdout:9/567: chown d16/d2c/d2f/d3d/d33/d37/f72 21136077 1 2024-05-06T12:14:38.424 INFO:tasks.workunit.client.0.smithi092.stdout:7/703: link d9/d62/fe1 d9/de/d1f/d37/d39/feb 0 2024-05-06T12:14:38.425 INFO:tasks.workunit.client.0.smithi092.stdout:7/704: dread - d9/de/d10/d17/d21/d2e/d9c/fcc zero size 2024-05-06T12:14:38.426 INFO:tasks.workunit.client.0.smithi092.stdout:4/639: creat d9/df/d62/d7b/db6/fd4 x:0 0 0 2024-05-06T12:14:38.427 INFO:tasks.workunit.client.0.smithi092.stdout:4/640: fdatasync d9/d27/f92 0 2024-05-06T12:14:38.428 INFO:tasks.workunit.client.0.smithi092.stdout:4/641: chown d9/d2b/d7c 239 1 2024-05-06T12:14:38.429 INFO:tasks.workunit.client.0.smithi092.stdout:9/568: creat d16/d94/da0/da7/fb5 x:0 0 0 2024-05-06T12:14:38.429 INFO:tasks.workunit.client.0.smithi092.stdout:9/569: stat d16/l18 0 2024-05-06T12:14:38.434 INFO:tasks.workunit.client.0.smithi092.stdout:4/642: rename d9/f21 to d9/fd5 0 2024-05-06T12:14:38.434 INFO:tasks.workunit.client.0.smithi092.stdout:9/570: rmdir d16/d2c/d2f/d3d 39 2024-05-06T12:14:38.435 INFO:tasks.workunit.client.0.smithi092.stdout:7/705: dwrite d9/de/d2d/fe3 [0,4194304] 0 2024-05-06T12:14:38.441 INFO:tasks.workunit.client.0.smithi092.stdout:9/571: mkdir d16/d2c/d2f/d3d/d33/d7e/dad/db6 0 2024-05-06T12:14:38.444 INFO:tasks.workunit.client.0.smithi092.stdout:7/706: link d9/de/d10/d17/d99/f7d d9/de/d1f/d37/fec 0 2024-05-06T12:14:38.444 INFO:tasks.workunit.client.0.smithi092.stdout:4/643: dwrite d9/df/f78 [0,4194304] 0 2024-05-06T12:14:38.446 INFO:tasks.workunit.client.0.smithi092.stdout:9/572: creat d16/d1e/d7b/d8d/da4/fb7 x:0 0 0 2024-05-06T12:14:38.447 INFO:tasks.workunit.client.0.smithi092.stdout:7/707: mknod d9/d62/ced 0 2024-05-06T12:14:38.448 INFO:tasks.workunit.client.0.smithi092.stdout:4/644: mknod d9/dd/da6/cd6 0 2024-05-06T12:14:38.448 INFO:tasks.workunit.client.0.smithi092.stdout:9/573: symlink d16/d1c/d58/d92/lb8 0 2024-05-06T12:14:38.450 INFO:tasks.workunit.client.0.smithi092.stdout:7/708: dread d9/de/d1f/d37/f74 [0,4194304] 0 2024-05-06T12:14:38.451 INFO:tasks.workunit.client.0.smithi092.stdout:9/574: mknod d16/d1e/d7b/d8d/da4/cb9 0 2024-05-06T12:14:38.456 INFO:tasks.workunit.client.0.smithi092.stdout:4/645: getdents d9/d27/da8/db3/dbf 0 2024-05-06T12:14:38.460 INFO:tasks.workunit.client.0.smithi092.stdout:7/709: dwrite d9/de/d1f/d37/d48/d65/f73 [0,4194304] 0 2024-05-06T12:14:38.460 INFO:tasks.workunit.client.0.smithi092.stdout:9/575: creat d16/d2c/d2f/d3d/d33/d6b/d9e/fba x:0 0 0 2024-05-06T12:14:38.462 INFO:tasks.workunit.client.0.smithi092.stdout:7/710: write d9/de/d1f/d37/fb0 [917920,91642] 0 2024-05-06T12:14:38.464 INFO:tasks.workunit.client.0.smithi092.stdout:4/646: unlink d9/df/d48/l58 0 2024-05-06T12:14:38.465 INFO:tasks.workunit.client.0.smithi092.stdout:9/576: symlink d16/d1e/lbb 0 2024-05-06T12:14:38.467 INFO:tasks.workunit.client.0.smithi092.stdout:9/577: chown d16/d2c/d2f/d3d/d33/f56 0 1 2024-05-06T12:14:38.469 INFO:tasks.workunit.client.0.smithi092.stdout:4/647: symlink d9/d2b/d5b/daf/ld7 0 2024-05-06T12:14:38.469 INFO:tasks.workunit.client.0.smithi092.stdout:9/578: truncate d16/d2c/d2f/d3d/f61 589916 0 2024-05-06T12:14:38.470 INFO:tasks.workunit.client.0.smithi092.stdout:4/648: chown d9/d2b/f7d 40385068 1 2024-05-06T12:14:38.471 INFO:tasks.workunit.client.0.smithi092.stdout:4/649: readlink d9/l5e 0 2024-05-06T12:14:38.472 INFO:tasks.workunit.client.0.smithi092.stdout:4/650: write d9/d2b/f77 [608737,88366] 0 2024-05-06T12:14:38.472 INFO:tasks.workunit.client.0.smithi092.stdout:9/579: creat d16/d1c/d58/d92/dab/fbc x:0 0 0 2024-05-06T12:14:38.555 INFO:tasks.workunit.client.0.smithi092.stdout:0/787: read d7/dc/d28/d54/d1e/d39/da8/dc5/fd1 [308292,98391] 0 2024-05-06T12:14:38.557 INFO:tasks.workunit.client.0.smithi092.stdout:7/711: dread d9/de/d10/d17/d21/d38/f89 [0,4194304] 0 2024-05-06T12:14:38.557 INFO:tasks.workunit.client.0.smithi092.stdout:7/712: stat d9/d62/fe1 0 2024-05-06T12:14:38.557 INFO:tasks.workunit.client.0.smithi092.stdout:7/713: chown d9/d62 2067961 1 2024-05-06T12:14:38.559 INFO:tasks.workunit.client.0.smithi092.stdout:0/788: write d7/dc/d28/d54/d12/d8f/dad/fcf [214042,120130] 0 2024-05-06T12:14:38.562 INFO:tasks.workunit.client.0.smithi092.stdout:7/714: dread d9/de/d2d/fe3 [0,4194304] 0 2024-05-06T12:14:38.563 INFO:tasks.workunit.client.0.smithi092.stdout:9/580: write d16/d2c/d2f/d3d/f3c [4081246,34348] 0 2024-05-06T12:14:38.564 INFO:tasks.workunit.client.0.smithi092.stdout:9/581: fdatasync d16/d2c/d2f/f45 0 2024-05-06T12:14:38.569 INFO:tasks.workunit.client.0.smithi092.stdout:5/530: sync 2024-05-06T12:14:38.573 INFO:tasks.workunit.client.0.smithi092.stdout:0/789: dwrite d7/dc/d28/d54/d1e/d26/d96/fc3 [4194304,4194304] 0 2024-05-06T12:14:38.575 INFO:tasks.workunit.client.0.smithi092.stdout:4/651: dwrite d9/d27/f31 [0,4194304] 0 2024-05-06T12:14:38.575 INFO:tasks.workunit.client.0.smithi092.stdout:5/531: truncate d4/d34/d28/f72 649590 0 2024-05-06T12:14:38.578 INFO:tasks.workunit.client.0.smithi092.stdout:0/790: readlink d7/dc/l9f 0 2024-05-06T12:14:38.578 INFO:tasks.workunit.client.0.smithi092.stdout:9/582: link d16/d1e/l6d d16/d1c/d58/d92/lbd 0 2024-05-06T12:14:38.581 INFO:tasks.workunit.client.0.smithi092.stdout:4/652: dread d9/d27/d4a/f84 [0,4194304] 0 2024-05-06T12:14:38.582 INFO:tasks.workunit.client.0.smithi092.stdout:0/791: creat d7/dc/d28/d54/d1e/d26/d96/dfb/f102 x:0 0 0 2024-05-06T12:14:38.583 INFO:tasks.workunit.client.0.smithi092.stdout:7/715: dread d9/de/d10/d17/d6a/f9d [0,4194304] 0 2024-05-06T12:14:38.583 INFO:tasks.workunit.client.0.smithi092.stdout:4/653: chown d9/dd/d90/f9b 3515342 1 2024-05-06T12:14:38.583 INFO:tasks.workunit.client.0.smithi092.stdout:9/583: mkdir d16/d2c/d2f/d3d/d33/d6b/dbe 0 2024-05-06T12:14:38.584 INFO:tasks.workunit.client.0.smithi092.stdout:5/532: creat d4/d34/d28/fc3 x:0 0 0 2024-05-06T12:14:38.586 INFO:tasks.workunit.client.0.smithi092.stdout:4/654: mkdir d9/d2b/d7c/dab/dd8 0 2024-05-06T12:14:38.588 INFO:tasks.workunit.client.0.smithi092.stdout:9/584: creat d16/d2c/d2f/d3d/d33/d57/fbf x:0 0 0 2024-05-06T12:14:38.589 INFO:tasks.workunit.client.0.smithi092.stdout:7/716: creat d9/de/d10/d17/d21/d38/d77/fee x:0 0 0 2024-05-06T12:14:38.591 INFO:tasks.workunit.client.0.smithi092.stdout:9/585: chown d16/d1e/da1/lb3 661046076 1 2024-05-06T12:14:38.592 INFO:tasks.workunit.client.0.smithi092.stdout:9/586: readlink l15 0 2024-05-06T12:14:38.595 INFO:tasks.workunit.client.0.smithi092.stdout:5/533: dwrite d4/d5/f1d [0,4194304] 0 2024-05-06T12:14:38.600 INFO:tasks.workunit.client.0.smithi092.stdout:9/587: dread d16/d2c/d2f/d3d/d33/d6b/f81 [0,4194304] 0 2024-05-06T12:14:38.601 INFO:tasks.workunit.client.0.smithi092.stdout:0/792: truncate d7/dc/d28/d54/d1e/d26/d96/fba 1184183 0 2024-05-06T12:14:38.603 INFO:tasks.workunit.client.0.smithi092.stdout:7/717: truncate d9/f6f 1598769 0 2024-05-06T12:14:38.607 INFO:tasks.workunit.client.0.smithi092.stdout:0/793: symlink d7/dc/d28/l103 0 2024-05-06T12:14:38.608 INFO:tasks.workunit.client.0.smithi092.stdout:4/655: dread d9/d27/d4a/f4c [0,4194304] 0 2024-05-06T12:14:38.609 INFO:tasks.workunit.client.0.smithi092.stdout:0/794: dread - d7/dc/d28/d54/d1e/d26/d96/fed zero size 2024-05-06T12:14:38.610 INFO:tasks.workunit.client.0.smithi092.stdout:9/588: link f7 d16/d2c/d2f/d75/fc0 0 2024-05-06T12:14:38.611 INFO:tasks.workunit.client.0.smithi092.stdout:9/589: stat d16/d2c/d2f/d3d/d33/d37/f93 0 2024-05-06T12:14:38.612 INFO:tasks.workunit.client.0.smithi092.stdout:4/656: unlink d9/d2b/d65/d83/fbd 0 2024-05-06T12:14:38.617 INFO:tasks.workunit.client.0.smithi092.stdout:0/795: mkdir d7/dc/d28/d54/d12/d104 0 2024-05-06T12:14:38.619 INFO:tasks.workunit.client.0.smithi092.stdout:0/796: read d7/dc/d28/d54/d1e/f3e [3636143,16472] 0 2024-05-06T12:14:38.620 INFO:tasks.workunit.client.0.smithi092.stdout:2/409: sync 2024-05-06T12:14:38.620 INFO:tasks.workunit.client.0.smithi092.stdout:8/619: sync 2024-05-06T12:14:38.620 INFO:tasks.workunit.client.0.smithi092.stdout:1/570: sync 2024-05-06T12:14:38.620 INFO:tasks.workunit.client.0.smithi092.stdout:3/505: sync 2024-05-06T12:14:38.621 INFO:tasks.workunit.client.0.smithi092.stdout:6/672: sync 2024-05-06T12:14:38.623 INFO:tasks.workunit.client.0.smithi092.stdout:4/657: truncate d9/d27/f41 1185427 0 2024-05-06T12:14:38.623 INFO:tasks.workunit.client.0.smithi092.stdout:5/534: dread d4/d34/d28/d4e/d5d/d82/d1f/d20/f8d [4194304,4194304] 0 2024-05-06T12:14:38.624 INFO:tasks.workunit.client.0.smithi092.stdout:8/620: chown d9/d21/d54/c61 10523 1 2024-05-06T12:14:38.624 INFO:tasks.workunit.client.0.smithi092.stdout:4/658: fdatasync d9/d27/fbe 0 2024-05-06T12:14:38.625 INFO:tasks.workunit.client.0.smithi092.stdout:3/506: read - d8/d24/d33/f92 zero size 2024-05-06T12:14:38.626 INFO:tasks.workunit.client.0.smithi092.stdout:3/507: stat d8/d1b/c5f 0 2024-05-06T12:14:38.628 INFO:tasks.workunit.client.0.smithi092.stdout:8/621: dread d9/d21/f3f [0,4194304] 0 2024-05-06T12:14:38.630 INFO:tasks.workunit.client.0.smithi092.stdout:5/535: dread d4/f7d [0,4194304] 0 2024-05-06T12:14:38.632 INFO:tasks.workunit.client.0.smithi092.stdout:6/673: creat d4/d4c/d52/d6c/da0/dc3/fde x:0 0 0 2024-05-06T12:14:38.634 INFO:tasks.workunit.client.0.smithi092.stdout:0/797: creat d7/dc/d28/d54/d12/d104/f105 x:0 0 0 2024-05-06T12:14:38.635 INFO:tasks.workunit.client.0.smithi092.stdout:6/674: fdatasync d4/d24/f3e 0 2024-05-06T12:14:38.635 INFO:tasks.workunit.client.0.smithi092.stdout:9/590: dread d16/d2c/d2f/d3d/d33/f3b [0,4194304] 0 2024-05-06T12:14:38.637 INFO:tasks.workunit.client.0.smithi092.stdout:7/718: dread d9/de/d2d/f3a [0,4194304] 0 2024-05-06T12:14:38.638 INFO:tasks.workunit.client.0.smithi092.stdout:3/508: dwrite d8/db/d14/f50 [0,4194304] 0 2024-05-06T12:14:38.640 INFO:tasks.workunit.client.0.smithi092.stdout:8/622: read d9/f36 [1292102,2677] 0 2024-05-06T12:14:38.642 INFO:tasks.workunit.client.0.smithi092.stdout:8/623: write d9/d21/d37/dbd/fd5 [419678,83964] 0 2024-05-06T12:14:38.644 INFO:tasks.workunit.client.0.smithi092.stdout:2/410: symlink df/d17/d53/l77 0 2024-05-06T12:14:38.645 INFO:tasks.workunit.client.0.smithi092.stdout:5/536: rename d4/d5/d10/d31/d53/d64/f8a to d4/d5/fc4 0 2024-05-06T12:14:38.646 INFO:tasks.workunit.client.0.smithi092.stdout:2/411: dread - df/d15/f4c zero size 2024-05-06T12:14:38.647 INFO:tasks.workunit.client.0.smithi092.stdout:6/675: creat d4/d20/d22/fdf x:0 0 0 2024-05-06T12:14:38.650 INFO:tasks.workunit.client.0.smithi092.stdout:1/571: truncate d3/d10/d13/d19/f5d 6265422 0 2024-05-06T12:14:38.651 INFO:tasks.workunit.client.0.smithi092.stdout:3/509: rmdir d8/d24/d33/d4b 39 2024-05-06T12:14:38.652 INFO:tasks.workunit.client.0.smithi092.stdout:8/624: dread d9/d21/d54/d79/fd8 [0,4194304] 0 2024-05-06T12:14:38.653 INFO:tasks.workunit.client.0.smithi092.stdout:0/798: unlink d7/dc/d28/d54/d1e/d26/d3a/f52 0 2024-05-06T12:14:38.654 INFO:tasks.workunit.client.0.smithi092.stdout:4/659: creat d9/d2b/fd9 x:0 0 0 2024-05-06T12:14:38.657 INFO:tasks.workunit.client.0.smithi092.stdout:2/412: creat df/d15/d50/f78 x:0 0 0 2024-05-06T12:14:38.658 INFO:tasks.workunit.client.0.smithi092.stdout:7/719: truncate d9/f6f 251615 0 2024-05-06T12:14:38.661 INFO:tasks.workunit.client.0.smithi092.stdout:0/799: symlink d7/dc/d28/d54/d1e/d46/df3/l106 0 2024-05-06T12:14:38.663 INFO:tasks.workunit.client.0.smithi092.stdout:6/676: symlink d4/d24/d38/d5d/d6d/le0 0 2024-05-06T12:14:38.665 INFO:tasks.workunit.client.0.smithi092.stdout:1/572: symlink d3/d10/d13/lb1 0 2024-05-06T12:14:38.667 INFO:tasks.workunit.client.0.smithi092.stdout:4/660: symlink d9/d27/lda 0 2024-05-06T12:14:38.668 INFO:tasks.workunit.client.0.smithi092.stdout:0/800: symlink d7/dc/d28/d64/l107 0 2024-05-06T12:14:38.670 INFO:tasks.workunit.client.0.smithi092.stdout:5/537: dread d4/d34/d28/d4e/d4b/f8c [0,4194304] 0 2024-05-06T12:14:38.670 INFO:tasks.workunit.client.0.smithi092.stdout:2/413: link df/d5c/f66 df/f79 0 2024-05-06T12:14:38.672 INFO:tasks.workunit.client.0.smithi092.stdout:2/414: chown c2 843275862 1 2024-05-06T12:14:38.673 INFO:tasks.workunit.client.0.smithi092.stdout:5/538: write d4/d34/d29/d54/f9c [568556,92353] 0 2024-05-06T12:14:38.675 INFO:tasks.workunit.client.0.smithi092.stdout:8/625: creat d9/fe4 x:0 0 0 2024-05-06T12:14:38.681 INFO:tasks.workunit.client.0.smithi092.stdout:0/801: creat d7/dc/d28/d54/d1e/d46/df3/f108 x:0 0 0 2024-05-06T12:14:38.684 INFO:tasks.workunit.client.0.smithi092.stdout:5/539: dwrite d4/f4c [0,4194304] 0 2024-05-06T12:14:38.688 INFO:tasks.workunit.client.0.smithi092.stdout:6/677: dread d4/d4c/d52/f80 [0,4194304] 0 2024-05-06T12:14:38.689 INFO:tasks.workunit.client.0.smithi092.stdout:1/573: dread d3/d10/f37 [0,4194304] 0 2024-05-06T12:14:38.689 INFO:tasks.workunit.client.0.smithi092.stdout:8/626: dwrite d9/d21/d37/f59 [0,4194304] 0 2024-05-06T12:14:38.691 INFO:tasks.workunit.client.0.smithi092.stdout:8/627: write d9/d5a/fa9 [104929,90830] 0 2024-05-06T12:14:38.705 INFO:tasks.workunit.client.0.smithi092.stdout:7/720: rename d9/lf to d9/de/d1f/d37/d48/lef 0 2024-05-06T12:14:38.708 INFO:tasks.workunit.client.0.smithi092.stdout:8/628: mknod d9/dc/d4a/d5f/da8/ce5 0 2024-05-06T12:14:38.710 INFO:tasks.workunit.client.0.smithi092.stdout:8/629: truncate d9/d5a/dd9/fda 2079744 0 2024-05-06T12:14:38.711 INFO:tasks.workunit.client.0.smithi092.stdout:2/415: chown df/d17/f3e 641274 1 2024-05-06T12:14:38.713 INFO:tasks.workunit.client.0.smithi092.stdout:7/721: fdatasync d9/f40 0 2024-05-06T12:14:38.716 INFO:tasks.workunit.client.0.smithi092.stdout:5/540: write d4/d5/f3b [2798740,21558] 0 2024-05-06T12:14:38.717 INFO:tasks.workunit.client.0.smithi092.stdout:4/661: link d9/l5e d9/d2b/d65/ldb 0 2024-05-06T12:14:38.717 INFO:tasks.workunit.client.0.smithi092.stdout:5/541: stat d4/d34/d29/lb6 0 2024-05-06T12:14:38.720 INFO:tasks.workunit.client.0.smithi092.stdout:1/574: dread d3/d10/d13/d19/f4e [0,4194304] 0 2024-05-06T12:14:38.721 INFO:tasks.workunit.client.0.smithi092.stdout:6/678: getdents d4/d24/d9f 0 2024-05-06T12:14:38.728 INFO:tasks.workunit.client.0.smithi092.stdout:2/416: symlink df/d17/l7a 0 2024-05-06T12:14:38.729 INFO:tasks.workunit.client.0.smithi092.stdout:8/630: write d9/d21/d37/f7f [665727,26994] 0 2024-05-06T12:14:38.732 INFO:tasks.workunit.client.0.smithi092.stdout:4/662: link d9/d2b/d5b/f98 d9/d27/d4a/fdc 0 2024-05-06T12:14:38.734 INFO:tasks.workunit.client.0.smithi092.stdout:4/663: write d9/f22 [4923369,13837] 0 2024-05-06T12:14:38.735 INFO:tasks.workunit.client.0.smithi092.stdout:7/722: dwrite d9/d62/d9a/fa3 [0,4194304] 0 2024-05-06T12:14:38.735 INFO:tasks.workunit.client.0.smithi092.stdout:6/679: mknod d4/d24/ce1 0 2024-05-06T12:14:38.736 INFO:tasks.workunit.client.0.smithi092.stdout:7/723: chown d9 19926 1 2024-05-06T12:14:38.736 INFO:tasks.workunit.client.0.smithi092.stdout:6/680: dread - d4/d4c/da9/fd2 zero size 2024-05-06T12:14:38.738 INFO:tasks.workunit.client.0.smithi092.stdout:0/802: link d7/dc/f9b d7/dc/dd4/f109 0 2024-05-06T12:14:38.739 INFO:tasks.workunit.client.0.smithi092.stdout:7/724: write d9/de/d1f/d37/fdd [756376,117415] 0 2024-05-06T12:14:38.743 INFO:tasks.workunit.client.0.smithi092.stdout:5/542: truncate d4/d34/d29/d54/f9c 2451548 0 2024-05-06T12:14:38.743 INFO:tasks.workunit.client.0.smithi092.stdout:1/575: creat d3/fb2 x:0 0 0 2024-05-06T12:14:38.745 INFO:tasks.workunit.client.0.smithi092.stdout:4/664: chown d9/d27/da8/cb2 40 1 2024-05-06T12:14:38.748 INFO:tasks.workunit.client.0.smithi092.stdout:6/681: mkdir d4/d20/de2 0 2024-05-06T12:14:38.751 INFO:tasks.workunit.client.0.smithi092.stdout:4/665: creat d9/df/d62/d7b/db6/fdd x:0 0 0 2024-05-06T12:14:38.751 INFO:tasks.workunit.client.0.smithi092.stdout:0/803: chown d7/dc/d28/l103 22804375 1 2024-05-06T12:14:38.752 INFO:tasks.workunit.client.0.smithi092.stdout:4/666: chown d9/dd/f7e 13186812 1 2024-05-06T12:14:38.752 INFO:tasks.workunit.client.0.smithi092.stdout:8/631: dread d9/f18 [0,4194304] 0 2024-05-06T12:14:38.754 INFO:tasks.workunit.client.0.smithi092.stdout:4/667: write d9/dd/f9e [4619333,91848] 0 2024-05-06T12:14:38.755 INFO:tasks.workunit.client.0.smithi092.stdout:5/543: mkdir d4/d34/d29/d54/dc5 0 2024-05-06T12:14:38.755 INFO:tasks.workunit.client.0.smithi092.stdout:4/668: write d9/df/f78 [978626,67132] 0 2024-05-06T12:14:38.759 INFO:tasks.workunit.client.0.smithi092.stdout:2/417: getdents df/d17/d51 0 2024-05-06T12:14:38.767 INFO:tasks.workunit.client.0.smithi092.stdout:0/804: write d7/dc/d28/d54/fdd [287401,12904] 0 2024-05-06T12:14:38.768 INFO:tasks.workunit.client.0.smithi092.stdout:8/632: dread d9/dc/fbb [0,4194304] 0 2024-05-06T12:14:38.769 INFO:tasks.workunit.client.0.smithi092.stdout:4/669: write d9/d27/d4a/f4c [4361943,106853] 0 2024-05-06T12:14:38.771 INFO:tasks.workunit.client.0.smithi092.stdout:7/725: getdents d9/d62/d9a 0 2024-05-06T12:14:38.772 INFO:tasks.workunit.client.0.smithi092.stdout:2/418: readlink df/d1b/l32 0 2024-05-06T12:14:38.772 INFO:tasks.workunit.client.0.smithi092.stdout:2/419: dread - df/d1b/f48 zero size 2024-05-06T12:14:38.773 INFO:tasks.workunit.client.0.smithi092.stdout:7/726: truncate d9/de/d1f/d37/d39/feb 431784 0 2024-05-06T12:14:38.774 INFO:tasks.workunit.client.0.smithi092.stdout:1/576: getdents d3/d42/d84/d4f/d58/d65/d81 0 2024-05-06T12:14:38.775 INFO:tasks.workunit.client.0.smithi092.stdout:1/577: readlink d3/lb 0 2024-05-06T12:14:38.777 INFO:tasks.workunit.client.0.smithi092.stdout:1/578: write d3/d10/d13/d19/fa9 [642679,23509] 0 2024-05-06T12:14:38.780 INFO:tasks.workunit.client.0.smithi092.stdout:6/682: rename d4/d24/d9f/ldb to d4/le3 0 2024-05-06T12:14:38.781 INFO:tasks.workunit.client.0.smithi092.stdout:4/670: symlink d9/d2b/d65/dbc/lde 0 2024-05-06T12:14:38.783 INFO:tasks.workunit.client.0.smithi092.stdout:8/633: getdents d9/d9d/da6 0 2024-05-06T12:14:38.786 INFO:tasks.workunit.client.0.smithi092.stdout:2/420: creat df/f7b x:0 0 0 2024-05-06T12:14:38.788 INFO:tasks.workunit.client.0.smithi092.stdout:0/805: rename d7/dc/d28/f4d to d7/dc/d97/dc9/f10a 0 2024-05-06T12:14:38.789 INFO:tasks.workunit.client.0.smithi092.stdout:5/544: dwrite d4/d5/f2a [0,4194304] 0 2024-05-06T12:14:38.789 INFO:tasks.workunit.client.0.smithi092.stdout:6/683: write d4/d24/d26/dae/dc0/fcc [2549821,1542] 0 2024-05-06T12:14:38.793 INFO:tasks.workunit.client.0.smithi092.stdout:7/727: dwrite d9/de/d10/d17/d99/da6/fa7 [0,4194304] 0 2024-05-06T12:14:38.794 INFO:tasks.workunit.client.0.smithi092.stdout:2/421: chown df/d5c/l6e 3173886 1 2024-05-06T12:14:38.797 INFO:tasks.workunit.client.0.smithi092.stdout:8/634: mkdir d9/d21/d37/d46/d5c/d62/da5/de6 0 2024-05-06T12:14:38.801 INFO:tasks.workunit.client.0.smithi092.stdout:4/671: dwrite d9/dd/f5f [0,4194304] 0 2024-05-06T12:14:38.803 INFO:tasks.workunit.client.0.smithi092.stdout:5/545: creat d4/d34/d28/d4e/fc6 x:0 0 0 2024-05-06T12:14:38.803 INFO:tasks.workunit.client.0.smithi092.stdout:6/684: fdatasync d4/d24/d54/f79 0 2024-05-06T12:14:38.805 INFO:tasks.workunit.client.0.smithi092.stdout:5/546: dread - d4/d34/d28/fac zero size 2024-05-06T12:14:38.807 INFO:tasks.workunit.client.0.smithi092.stdout:7/728: mkdir d9/de/d10/df0 0 2024-05-06T12:14:38.812 INFO:tasks.workunit.client.0.smithi092.stdout:4/672: rename d9/df/l3c to d9/d27/da8/db3/dbf/ldf 0 2024-05-06T12:14:38.813 INFO:tasks.workunit.client.0.smithi092.stdout:6/685: dwrite d4/d24/d26/f6e [0,4194304] 0 2024-05-06T12:14:38.815 INFO:tasks.workunit.client.0.smithi092.stdout:1/579: truncate d3/d10/d13/d46/f7b 3160388 0 2024-05-06T12:14:38.819 INFO:tasks.workunit.client.0.smithi092.stdout:7/729: rename d9/de/d1f/d37/d48/d65 to d9/de/d10/d17/d99/db6/df1 0 2024-05-06T12:14:38.820 INFO:tasks.workunit.client.0.smithi092.stdout:2/422: mknod df/d17/d1d/c7c 0 2024-05-06T12:14:38.826 INFO:tasks.workunit.client.0.smithi092.stdout:1/580: mkdir d3/d10/db3 0 2024-05-06T12:14:38.826 INFO:tasks.workunit.client.0.smithi092.stdout:0/806: link d7/c85 d7/dc/d28/d54/d1e/d46/df0/c10b 0 2024-05-06T12:14:38.828 INFO:tasks.workunit.client.0.smithi092.stdout:1/581: write d3/d42/d84/d4f/d58/f62 [797426,9539] 0 2024-05-06T12:14:38.829 INFO:tasks.workunit.client.0.smithi092.stdout:2/423: fdatasync df/f6d 0 2024-05-06T12:14:38.831 INFO:tasks.workunit.client.0.smithi092.stdout:2/424: chown df/d17/d51/c58 206479 1 2024-05-06T12:14:38.831 INFO:tasks.workunit.client.0.smithi092.stdout:6/686: symlink d4/d4c/le4 0 2024-05-06T12:14:38.832 INFO:tasks.workunit.client.0.smithi092.stdout:8/635: dread d9/d21/d37/d46/f5e [0,4194304] 0 2024-05-06T12:14:38.832 INFO:tasks.workunit.client.0.smithi092.stdout:7/730: truncate d9/de/d2d/fad 2624462 0 2024-05-06T12:14:38.834 INFO:tasks.workunit.client.0.smithi092.stdout:7/731: write d9/fe8 [786908,52835] 0 2024-05-06T12:14:38.834 INFO:tasks.workunit.client.0.smithi092.stdout:6/687: rmdir d4/d20/d23/d30 39 2024-05-06T12:14:38.836 INFO:tasks.workunit.client.0.smithi092.stdout:0/807: mkdir d7/dc/d6f/da5/dc6/d100/d10c 0 2024-05-06T12:14:38.841 INFO:tasks.workunit.client.0.smithi092.stdout:8/636: rmdir d9/d21/d37/d46/d5c/d62/d7a 39 2024-05-06T12:14:38.842 INFO:tasks.workunit.client.0.smithi092.stdout:8/637: chown d9/dc/l57 102283915 1 2024-05-06T12:14:38.842 INFO:tasks.workunit.client.0.smithi092.stdout:8/638: chown d9/d21/d37/dbd 220 1 2024-05-06T12:14:38.843 INFO:tasks.workunit.client.0.smithi092.stdout:8/639: write d9/d5a/d9f/fc6 [164613,29370] 0 2024-05-06T12:14:38.844 INFO:tasks.workunit.client.0.smithi092.stdout:8/640: chown d9/d21/d37/f7f 368840 1 2024-05-06T12:14:38.845 INFO:tasks.workunit.client.0.smithi092.stdout:1/582: rename d3/d10/d13/d19/d34/f51 to d3/d10/d13/d19/d21/fb4 0 2024-05-06T12:14:38.846 INFO:tasks.workunit.client.0.smithi092.stdout:4/673: truncate d9/d27/d4a/f4c 1730770 0 2024-05-06T12:14:38.847 INFO:tasks.workunit.client.0.smithi092.stdout:4/674: write d9/dd/f23 [3559924,63877] 0 2024-05-06T12:14:38.852 INFO:tasks.workunit.client.0.smithi092.stdout:1/583: dwrite d3/d10/d13/d46/f55 [0,4194304] 0 2024-05-06T12:14:38.853 INFO:tasks.workunit.client.0.smithi092.stdout:1/584: write d3/d10/d13/d19/fa9 [506125,70796] 0 2024-05-06T12:14:38.856 INFO:tasks.workunit.client.0.smithi092.stdout:7/732: link d9/de/d10/d17/d21/d38/f90 d9/de/d10/d17/d21/d2e/ff2 0 2024-05-06T12:14:38.861 INFO:tasks.workunit.client.0.smithi092.stdout:6/688: getdents d4 0 2024-05-06T12:14:38.863 INFO:tasks.workunit.client.0.smithi092.stdout:8/641: write d9/dc/f53 [3379392,6169] 0 2024-05-06T12:14:38.865 INFO:tasks.workunit.client.0.smithi092.stdout:0/808: dwrite d7/dc/f9b [4194304,4194304] 0 2024-05-06T12:14:38.867 INFO:tasks.workunit.client.0.smithi092.stdout:7/733: mkdir d9/de/d10/d17/d99/da6/df3 0 2024-05-06T12:14:38.870 INFO:tasks.workunit.client.0.smithi092.stdout:2/425: rename df/d15/d6a to df/d17/d7d 0 2024-05-06T12:14:38.870 INFO:tasks.workunit.client.0.smithi092.stdout:0/809: truncate d7/dc/d28/d54/d1e/d26/d96/dfb/f102 280994 0 2024-05-06T12:14:38.872 INFO:tasks.workunit.client.0.smithi092.stdout:1/585: fsync d3/d10/d13/f3b 0 2024-05-06T12:14:38.872 INFO:tasks.workunit.client.0.smithi092.stdout:8/642: fdatasync d9/d21/d37/d46/d5c/d71/f80 0 2024-05-06T12:14:38.873 INFO:tasks.workunit.client.0.smithi092.stdout:4/675: creat d9/fe0 x:0 0 0 2024-05-06T12:14:38.874 INFO:tasks.workunit.client.0.smithi092.stdout:1/586: write d3/d10/d13/d19/d21/d88/fa0 [412023,123749] 0 2024-05-06T12:14:38.879 INFO:tasks.workunit.client.0.smithi092.stdout:2/426: mkdir df/d17/d7e 0 2024-05-06T12:14:38.880 INFO:tasks.workunit.client.0.smithi092.stdout:0/810: getdents d7/dc/d28/d54/d12/d7f/db5/dcc 0 2024-05-06T12:14:38.881 INFO:tasks.workunit.client.0.smithi092.stdout:4/676: dwrite d9/dd/f5f [0,4194304] 0 2024-05-06T12:14:38.883 INFO:tasks.workunit.client.0.smithi092.stdout:4/677: stat d9/dd/f19 0 2024-05-06T12:14:38.883 INFO:tasks.workunit.client.0.smithi092.stdout:4/678: stat d9/df/c73 0 2024-05-06T12:14:38.885 INFO:tasks.workunit.client.0.smithi092.stdout:1/587: mknod d3/d10/d13/d19/d21/d72/d9a/d9c/cb5 0 2024-05-06T12:14:38.885 INFO:tasks.workunit.client.0.smithi092.stdout:2/427: rmdir df/d17/d51 39 2024-05-06T12:14:38.888 INFO:tasks.workunit.client.0.smithi092.stdout:4/679: rmdir d9/df 39 2024-05-06T12:14:38.890 INFO:tasks.workunit.client.0.smithi092.stdout:4/680: dread - d9/fc0 zero size 2024-05-06T12:14:38.890 INFO:tasks.workunit.client.0.smithi092.stdout:6/689: dwrite d4/f11 [0,4194304] 0 2024-05-06T12:14:38.892 INFO:tasks.workunit.client.0.smithi092.stdout:6/690: dread - d4/d24/d38/d5d/fd0 zero size 2024-05-06T12:14:38.892 INFO:tasks.workunit.client.0.smithi092.stdout:1/588: fdatasync d3/d10/d13/d19/f1c 0 2024-05-06T12:14:38.894 INFO:tasks.workunit.client.0.smithi092.stdout:8/643: getdents d9/d21/d37/d46 0 2024-05-06T12:14:38.895 INFO:tasks.workunit.client.0.smithi092.stdout:0/811: mknod d7/dc/d28/d54/d12/c10d 0 2024-05-06T12:14:38.900 INFO:tasks.workunit.client.0.smithi092.stdout:7/734: dread d9/de/d10/f12 [0,4194304] 0 2024-05-06T12:14:38.902 INFO:tasks.workunit.client.0.smithi092.stdout:8/644: mkdir d9/d21/d37/d46/d5c/d62/da5/de7 0 2024-05-06T12:14:38.903 INFO:tasks.workunit.client.0.smithi092.stdout:0/812: chown d7/dc/d28/d54/d1e/d26/d4e/fbb 102178712 1 2024-05-06T12:14:38.904 INFO:tasks.workunit.client.0.smithi092.stdout:6/691: chown d4/d20/d23/d30/f5e 309801 1 2024-05-06T12:14:38.905 INFO:tasks.workunit.client.0.smithi092.stdout:4/681: dwrite d9/d2b/f6a [0,4194304] 0 2024-05-06T12:14:38.911 INFO:tasks.workunit.client.0.smithi092.stdout:2/428: dwrite f1 [4194304,4194304] 0 2024-05-06T12:14:38.911 INFO:tasks.workunit.client.0.smithi092.stdout:7/735: rmdir d9/de/d1f/d37/d39 39 2024-05-06T12:14:38.911 INFO:tasks.workunit.client.0.smithi092.stdout:1/589: dwrite d3/d10/d13/f60 [0,4194304] 0 2024-05-06T12:14:38.914 INFO:tasks.workunit.client.0.smithi092.stdout:6/692: write d4/d24/d38/d5d/fcf [469119,103865] 0 2024-05-06T12:14:38.915 INFO:tasks.workunit.client.0.smithi092.stdout:1/590: truncate d3/fb2 299562 0 2024-05-06T12:14:38.917 INFO:tasks.workunit.client.0.smithi092.stdout:6/693: chown d4/d20/d22/f7a 23898 1 2024-05-06T12:14:38.919 INFO:tasks.workunit.client.0.smithi092.stdout:0/813: rename d7/dc/d28/d64/f89 to d7/dc/d28/d54/d1e/d26/d3a/f10e 0 2024-05-06T12:14:38.919 INFO:tasks.workunit.client.0.smithi092.stdout:7/736: rmdir d9/de/d10/d17/d21/d2e/d87 39 2024-05-06T12:14:38.921 INFO:tasks.workunit.client.0.smithi092.stdout:2/429: truncate df/d17/d1d/f4f 911056 0 2024-05-06T12:14:38.923 INFO:tasks.workunit.client.0.smithi092.stdout:2/430: readlink df/d17/d1d/l6c 0 2024-05-06T12:14:38.925 INFO:tasks.workunit.client.0.smithi092.stdout:4/682: dwrite d9/d27/f4f [0,4194304] 0 2024-05-06T12:14:38.927 INFO:tasks.workunit.client.0.smithi092.stdout:1/591: write d3/d10/d13/d19/d21/d88/d8b/f8c [1284222,115678] 0 2024-05-06T12:14:38.929 INFO:tasks.workunit.client.0.smithi092.stdout:8/645: dread d9/d21/d37/d46/d5c/d62/da5/fe0 [0,4194304] 0 2024-05-06T12:14:38.931 INFO:tasks.workunit.client.0.smithi092.stdout:8/646: stat d9/l50 0 2024-05-06T12:14:38.932 INFO:tasks.workunit.client.0.smithi092.stdout:0/814: truncate d7/dc/d28/d54/d1e/d46/df3/f108 402988 0 2024-05-06T12:14:38.932 INFO:tasks.workunit.client.0.smithi092.stdout:1/592: readlink d3/d42/d84/d4f/d58/d65/l80 0 2024-05-06T12:14:38.932 INFO:tasks.workunit.client.0.smithi092.stdout:4/683: mknod d9/d2b/d65/dbc/ce1 0 2024-05-06T12:14:38.933 INFO:tasks.workunit.client.0.smithi092.stdout:2/431: symlink df/d15/d50/d59/l7f 0 2024-05-06T12:14:38.937 INFO:tasks.workunit.client.0.smithi092.stdout:7/737: dwrite d9/de/d10/d17/d99/da6/fa7 [0,4194304] 0 2024-05-06T12:14:38.937 INFO:tasks.workunit.client.0.smithi092.stdout:2/432: mknod df/d15/c80 0 2024-05-06T12:14:38.939 INFO:tasks.workunit.client.0.smithi092.stdout:4/684: mknod d9/dd/da6/ce2 0 2024-05-06T12:14:38.940 INFO:tasks.workunit.client.0.smithi092.stdout:1/593: mknod d3/d10/db3/cb6 0 2024-05-06T12:14:38.945 INFO:tasks.workunit.client.0.smithi092.stdout:0/815: rmdir d7/dc/d28/d54/d12/d7f/db2/dbc 39 2024-05-06T12:14:38.947 INFO:tasks.workunit.client.0.smithi092.stdout:2/433: rename df/d17/f4a to df/d17/d1d/d33/f81 0 2024-05-06T12:14:38.953 INFO:tasks.workunit.client.0.smithi092.stdout:9/591: sync 2024-05-06T12:14:38.953 INFO:tasks.workunit.client.0.smithi092.stdout:3/510: sync 2024-05-06T12:14:38.956 INFO:tasks.workunit.client.0.smithi092.stdout:2/434: chown df/d15/c23 66 1 2024-05-06T12:14:38.958 INFO:tasks.workunit.client.0.smithi092.stdout:4/685: dwrite d9/d2b/f6a [0,4194304] 0 2024-05-06T12:14:38.960 INFO:tasks.workunit.client.0.smithi092.stdout:1/594: dwrite d3/d10/d13/d19/d21/f4d [0,4194304] 0 2024-05-06T12:14:38.961 INFO:tasks.workunit.client.0.smithi092.stdout:9/592: mknod d16/d2c/d2f/d3d/cc1 0 2024-05-06T12:14:38.964 INFO:tasks.workunit.client.0.smithi092.stdout:8/647: dread d9/d21/d54/f63 [0,4194304] 0 2024-05-06T12:14:38.965 INFO:tasks.workunit.client.0.smithi092.stdout:9/593: chown d16/d2c/d2f/d3d/f61 2157 1 2024-05-06T12:14:38.966 INFO:tasks.workunit.client.0.smithi092.stdout:4/686: mknod d9/dd/d90/ce3 0 2024-05-06T12:14:38.968 INFO:tasks.workunit.client.0.smithi092.stdout:0/816: rename d7/dc/d28/d54/d12/d7f/db2/dbc/ce8 to d7/dc/d28/d54/d1e/d39/da8/dc5/c10f 0 2024-05-06T12:14:38.968 INFO:tasks.workunit.client.0.smithi092.stdout:8/648: write d9/d5a/d9f/fbe [4008471,112979] 0 2024-05-06T12:14:38.969 INFO:tasks.workunit.client.0.smithi092.stdout:3/511: symlink d8/d24/d27/l9c 0 2024-05-06T12:14:38.972 INFO:tasks.workunit.client.0.smithi092.stdout:5/547: sync 2024-05-06T12:14:38.973 INFO:tasks.workunit.client.0.smithi092.stdout:9/594: chown d16/d2c/d2f/d3d/d33/d6b/d9e 86268 1 2024-05-06T12:14:38.976 INFO:tasks.workunit.client.0.smithi092.stdout:6/694: dread d4/d24/d26/dae/dc0/fcc [0,4194304] 0 2024-05-06T12:14:38.980 INFO:tasks.workunit.client.0.smithi092.stdout:2/435: rename df/d17/f62 to df/d15/d50/f82 0 2024-05-06T12:14:38.980 INFO:tasks.workunit.client.0.smithi092.stdout:7/738: dread d9/de/d10/d17/d99/db6/df1/f6b [0,4194304] 0 2024-05-06T12:14:38.981 INFO:tasks.workunit.client.0.smithi092.stdout:8/649: mknod d9/d21/d54/d79/d8a/ce8 0 2024-05-06T12:14:38.982 INFO:tasks.workunit.client.0.smithi092.stdout:0/817: creat d7/dc/d28/d54/d1e/d39/da8/dc5/f110 x:0 0 0 2024-05-06T12:14:38.984 INFO:tasks.workunit.client.0.smithi092.stdout:5/548: write d4/d5/d10/f17 [981530,45109] 0 2024-05-06T12:14:38.985 INFO:tasks.workunit.client.0.smithi092.stdout:2/436: chown df/d15/f4c 61 1 2024-05-06T12:14:38.986 INFO:tasks.workunit.client.0.smithi092.stdout:9/595: creat d16/d2c/d2f/d3d/fc2 x:0 0 0 2024-05-06T12:14:38.987 INFO:tasks.workunit.client.0.smithi092.stdout:3/512: dwrite d8/db/d14/d5a/f94 [0,4194304] 0 2024-05-06T12:14:38.990 INFO:tasks.workunit.client.0.smithi092.stdout:3/513: readlink d8/db/d4c/d3d/d49/l8b 0 2024-05-06T12:14:38.994 INFO:tasks.workunit.client.0.smithi092.stdout:8/650: creat d9/d21/d37/d46/fe9 x:0 0 0 2024-05-06T12:14:38.994 INFO:tasks.workunit.client.0.smithi092.stdout:9/596: dread - d16/d2c/d2f/d3d/d33/d57/f59 zero size 2024-05-06T12:14:38.995 INFO:tasks.workunit.client.0.smithi092.stdout:1/595: dread d3/d10/d13/d19/f40 [0,4194304] 0 2024-05-06T12:14:38.997 INFO:tasks.workunit.client.0.smithi092.stdout:4/687: creat d9/d27/fe4 x:0 0 0 2024-05-06T12:14:38.999 INFO:tasks.workunit.client.0.smithi092.stdout:0/818: dwrite d7/dc/dd4/f109 [8388608,4194304] 0 2024-05-06T12:14:39.010 INFO:tasks.workunit.client.0.smithi092.stdout:0/819: chown d7/dc/d9e/f40 1358245 1 2024-05-06T12:14:39.010 INFO:tasks.workunit.client.0.smithi092.stdout:1/596: creat d3/d42/d84/d9f/fb7 x:0 0 0 2024-05-06T12:14:39.013 INFO:tasks.workunit.client.0.smithi092.stdout:7/739: rename d9/de/d10/d17/d99/db6/fda to d9/de/ff4 0 2024-05-06T12:14:39.014 INFO:tasks.workunit.client.0.smithi092.stdout:0/820: readlink d7/dc/d28/lc1 0 2024-05-06T12:14:39.017 INFO:tasks.workunit.client.0.smithi092.stdout:5/549: link d4/d34/dae/lb1 d4/d34/d29/d54/d73/lc7 0 2024-05-06T12:14:39.019 INFO:tasks.workunit.client.0.smithi092.stdout:6/695: dwrite d4/f19 [0,4194304] 0 2024-05-06T12:14:39.019 INFO:tasks.workunit.client.0.smithi092.stdout:9/597: rename d16/d2c/d2f/d3d/d33/d37/f93 to d16/d2c/d4b/d76/fc3 0 2024-05-06T12:14:39.022 INFO:tasks.workunit.client.0.smithi092.stdout:2/437: dwrite f5 [0,4194304] 0 2024-05-06T12:14:39.022 INFO:tasks.workunit.client.0.smithi092.stdout:1/597: symlink d3/d10/d13/d19/d34/d89/lb8 0 2024-05-06T12:14:39.026 INFO:tasks.workunit.client.0.smithi092.stdout:6/696: write d4/d24/d54/f77 [1266898,26657] 0 2024-05-06T12:14:39.028 INFO:tasks.workunit.client.0.smithi092.stdout:3/514: dread d8/db/d14/d5a/f7b [0,4194304] 0 2024-05-06T12:14:39.030 INFO:tasks.workunit.client.0.smithi092.stdout:9/598: truncate d16/d2c/d2f/d3d/d33/d7e/dad/fa5 307877 0 2024-05-06T12:14:39.038 INFO:tasks.workunit.client.0.smithi092.stdout:8/651: dread d9/d21/d37/fa3 [0,4194304] 0 2024-05-06T12:14:39.049 INFO:tasks.workunit.client.0.smithi092.stdout:6/697: dread d4/d24/d26/dae/dc0/f71 [0,4194304] 0 2024-05-06T12:14:39.066 INFO:tasks.workunit.client.0.smithi092.stdout:0/821: rmdir d7/dc/d6f/da5/dc6/d100 39 2024-05-06T12:14:39.068 INFO:tasks.workunit.client.0.smithi092.stdout:4/688: rename d9/df/d62/f68 to d9/d27/da8/fe5 0 2024-05-06T12:14:39.072 INFO:tasks.workunit.client.0.smithi092.stdout:2/438: mkdir df/d17/d1d/d33/d4d/d83 0 2024-05-06T12:14:39.072 INFO:tasks.workunit.client.0.smithi092.stdout:2/439: stat df/d15/c23 0 2024-05-06T12:14:39.075 INFO:tasks.workunit.client.0.smithi092.stdout:6/698: rmdir d4/d24/d38/d5d 39 2024-05-06T12:14:39.083 INFO:tasks.workunit.client.0.smithi092.stdout:6/699: chown d4/d24/d38/d8c/ld3 216644 1 2024-05-06T12:14:39.083 INFO:tasks.workunit.client.0.smithi092.stdout:0/822: chown d7/dc/d6f/da5/dc6/d100/d10c 69 1 2024-05-06T12:14:39.087 INFO:tasks.workunit.client.0.smithi092.stdout:6/700: truncate d4/d20/f31 4333652 0 2024-05-06T12:14:39.090 INFO:tasks.workunit.client.0.smithi092.stdout:1/598: dwrite d3/d10/d13/d19/f31 [0,4194304] 0 2024-05-06T12:14:39.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:38 smithi190 ceph-mon[33798]: pgmap v21: 65 pgs: 65 active+clean; 1.2 GiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 38 MiB/s rd, 167 MiB/s wr, 517 op/s 2024-05-06T12:14:39.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:38 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:39.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:38 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:39.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:38 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:14:39.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:38 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:14:39.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:38 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:14:39.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:38 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:14:39.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:38 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:14:39.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:38 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:14:39.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:38 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:39.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:38 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:14:39.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:38 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:14:39.101 INFO:tasks.workunit.client.0.smithi092.stdout:4/689: mkdir d9/de6 0 2024-05-06T12:14:39.101 INFO:tasks.workunit.client.0.smithi092.stdout:3/515: link d8/db/d4c/d3d/d48/c57 d8/db/d14/d5a/c9d 0 2024-05-06T12:14:39.102 INFO:tasks.workunit.client.0.smithi092.stdout:3/516: chown d8/db/l2d 492463 1 2024-05-06T12:14:39.102 INFO:tasks.workunit.client.0.smithi092.stdout:9/599: link d16/d1e/da1/d43/c90 d16/d1e/cc4 0 2024-05-06T12:14:39.104 INFO:tasks.workunit.client.0.smithi092.stdout:0/823: rename d7/dc/d9e/l34 to d7/dc/dd4/df8/l111 0 2024-05-06T12:14:39.105 INFO:tasks.workunit.client.0.smithi092.stdout:6/701: write d4/d24/d26/dae/dc0/fa6 [245386,64936] 0 2024-05-06T12:14:39.107 INFO:tasks.workunit.client.0.smithi092.stdout:5/550: getdents d4/d5/d10/da1 0 2024-05-06T12:14:39.109 INFO:tasks.workunit.client.0.smithi092.stdout:4/690: unlink d9/dd/le 0 2024-05-06T12:14:39.110 INFO:tasks.workunit.client.0.smithi092.stdout:8/652: symlink d9/d21/d37/d46/d5c/d62/da5/de6/lea 0 2024-05-06T12:14:39.110 INFO:tasks.workunit.client.0.smithi092.stdout:1/599: read d3/d10/d13/d19/f5d [4098657,117357] 0 2024-05-06T12:14:39.112 INFO:tasks.workunit.client.0.smithi092.stdout:2/440: link df/f55 df/d17/d1d/d33/d4d/d83/f84 0 2024-05-06T12:14:39.116 INFO:tasks.workunit.client.0.smithi092.stdout:6/702: chown d4/d24/d38/d5d/d6d/l74 1 1 2024-05-06T12:14:39.118 INFO:tasks.workunit.client.0.smithi092.stdout:6/703: read d4/f8e [3256334,27565] 0 2024-05-06T12:14:39.119 INFO:tasks.workunit.client.0.smithi092.stdout:8/653: chown d9/d21/l22 3567 1 2024-05-06T12:14:39.120 INFO:tasks.workunit.client.0.smithi092.stdout:0/824: dread d7/dc/d28/d54/d1e/d26/d96/fba [0,4194304] 0 2024-05-06T12:14:39.121 INFO:tasks.workunit.client.0.smithi092.stdout:1/600: unlink d3/d42/d84/f5f 0 2024-05-06T12:14:39.122 INFO:tasks.workunit.client.0.smithi092.stdout:3/517: creat d8/d24/d33/d4b/d74/f9e x:0 0 0 2024-05-06T12:14:39.128 INFO:tasks.workunit.client.0.smithi092.stdout:6/704: dread - d4/d20/d22/f9e zero size 2024-05-06T12:14:39.128 INFO:tasks.workunit.client.0.smithi092.stdout:0/825: dwrite d7/dc/d28/d54/d1e/d46/d63/f9d [0,4194304] 0 2024-05-06T12:14:39.130 INFO:tasks.workunit.client.0.smithi092.stdout:0/826: stat d7/dc/d28/d54/d1e/d26/d4e/fce 0 2024-05-06T12:14:39.133 INFO:tasks.workunit.client.0.smithi092.stdout:5/551: link d4/d5/d30/l9a d4/d34/d29/d6c/lc8 0 2024-05-06T12:14:39.135 INFO:tasks.workunit.client.0.smithi092.stdout:2/441: dread df/d17/f6b [0,4194304] 0 2024-05-06T12:14:39.136 INFO:tasks.workunit.client.0.smithi092.stdout:6/705: mkdir d4/d24/d66/de5 0 2024-05-06T12:14:39.140 INFO:tasks.workunit.client.0.smithi092.stdout:9/600: getdents d16/d1e/da1 0 2024-05-06T12:14:39.141 INFO:tasks.workunit.client.0.smithi092.stdout:2/442: dread f5 [0,4194304] 0 2024-05-06T12:14:39.142 INFO:tasks.workunit.client.0.smithi092.stdout:9/601: chown d16/d1e/f25 102737 1 2024-05-06T12:14:39.142 INFO:tasks.workunit.client.0.smithi092.stdout:5/552: rename d4/d34/d28/d4e/d5d/d82/d77/cbd to d4/d34/dae/cc9 0 2024-05-06T12:14:39.144 INFO:tasks.workunit.client.0.smithi092.stdout:2/443: dread df/d17/d1d/d33/f81 [0,4194304] 0 2024-05-06T12:14:39.145 INFO:tasks.workunit.client.0.smithi092.stdout:6/706: mknod d4/ce6 0 2024-05-06T12:14:39.147 INFO:tasks.workunit.client.0.smithi092.stdout:0/827: unlink d7/dc/d31/de6/lef 0 2024-05-06T12:14:39.149 INFO:tasks.workunit.client.0.smithi092.stdout:5/553: fdatasync d4/d34/d28/d4e/f71 0 2024-05-06T12:14:39.149 INFO:tasks.workunit.client.0.smithi092.stdout:5/554: readlink d4/d5/d10/l52 0 2024-05-06T12:14:39.152 INFO:tasks.workunit.client.0.smithi092.stdout:0/828: creat d7/dc/d28/d54/d1e/d26/d96/d90/f112 x:0 0 0 2024-05-06T12:14:39.154 INFO:tasks.workunit.client.0.smithi092.stdout:0/829: write d7/dc/d28/d54/d1e/d46/df3/f108 [1226509,50434] 0 2024-05-06T12:14:39.156 INFO:tasks.workunit.client.0.smithi092.stdout:2/444: mkdir df/d17/d53/d85 0 2024-05-06T12:14:39.157 INFO:tasks.workunit.client.0.smithi092.stdout:2/445: chown df/d17/d1d/d33/f81 13638772 1 2024-05-06T12:14:39.160 INFO:tasks.workunit.client.0.smithi092.stdout:9/602: dread d16/d1e/da1/d43/f89 [0,4194304] 0 2024-05-06T12:14:39.161 INFO:tasks.workunit.client.0.smithi092.stdout:9/603: stat d16/d2c/d2f/d3d/d33/d6b/f77 0 2024-05-06T12:14:39.161 INFO:tasks.workunit.client.0.smithi092.stdout:2/446: creat df/d17/d7d/f86 x:0 0 0 2024-05-06T12:14:39.163 INFO:tasks.workunit.client.0.smithi092.stdout:2/447: write df/d17/d1d/f49 [873658,73156] 0 2024-05-06T12:14:39.165 INFO:tasks.workunit.client.0.smithi092.stdout:5/555: getdents d4/d34/d28/d4e/d4b 0 2024-05-06T12:14:39.169 INFO:tasks.workunit.client.0.smithi092.stdout:0/830: creat d7/dc/d28/d54/d12/d8f/f113 x:0 0 0 2024-05-06T12:14:39.170 INFO:tasks.workunit.client.0.smithi092.stdout:4/691: write d9/d27/d4a/f76 [1410762,75507] 0 2024-05-06T12:14:39.172 INFO:tasks.workunit.client.0.smithi092.stdout:2/448: write fe [2268562,75791] 0 2024-05-06T12:14:39.177 INFO:tasks.workunit.client.0.smithi092.stdout:9/604: rename d16/d1e/da1/f5f to d16/d2c/d2f/d3d/d33/fc5 0 2024-05-06T12:14:39.184 INFO:tasks.workunit.client.0.smithi092.stdout:0/831: dwrite d7/dc/d97/fd9 [0,4194304] 0 2024-05-06T12:14:39.184 INFO:tasks.workunit.client.0.smithi092.stdout:5/556: dread d4/d5/f3b [0,4194304] 0 2024-05-06T12:14:39.186 INFO:tasks.workunit.client.0.smithi092.stdout:4/692: dwrite f5 [0,4194304] 0 2024-05-06T12:14:39.187 INFO:tasks.workunit.client.0.smithi092.stdout:1/601: dwrite d3/d10/d13/d19/f5a [0,4194304] 0 2024-05-06T12:14:39.190 INFO:tasks.workunit.client.0.smithi092.stdout:0/832: fdatasync f1 0 2024-05-06T12:14:39.192 INFO:tasks.workunit.client.0.smithi092.stdout:6/707: chown d4/d20/d22/la2 121 1 2024-05-06T12:14:39.195 INFO:tasks.workunit.client.0.smithi092.stdout:5/557: read d4/d34/f56 [6601426,113925] 0 2024-05-06T12:14:39.195 INFO:tasks.workunit.client.0.smithi092.stdout:4/693: fdatasync d9/d27/d4a/f84 0 2024-05-06T12:14:39.199 INFO:tasks.workunit.client.0.smithi092.stdout:0/833: creat d7/dc/d28/d54/d12/d8f/dad/f114 x:0 0 0 2024-05-06T12:14:39.201 INFO:tasks.workunit.client.0.smithi092.stdout:9/605: dwrite d16/d1c/f52 [0,4194304] 0 2024-05-06T12:14:39.203 INFO:tasks.workunit.client.0.smithi092.stdout:9/606: chown d16/d2c/d3f/c41 519831582 1 2024-05-06T12:14:39.207 INFO:tasks.workunit.client.0.smithi092.stdout:1/602: creat d3/d10/d13/d19/d21/d72/d9a/da4/fb9 x:0 0 0 2024-05-06T12:14:39.207 INFO:tasks.workunit.client.0.smithi092.stdout:9/607: dread - d16/d2c/d3f/f9d zero size 2024-05-06T12:14:39.207 INFO:tasks.workunit.client.0.smithi092.stdout:2/449: truncate fe 1895834 0 2024-05-06T12:14:39.209 INFO:tasks.workunit.client.0.smithi092.stdout:4/694: creat d9/d27/da8/db3/fe7 x:0 0 0 2024-05-06T12:14:39.212 INFO:tasks.workunit.client.0.smithi092.stdout:0/834: mkdir d7/dc/d28/d54/d12/d7f/d115 0 2024-05-06T12:14:39.213 INFO:tasks.workunit.client.0.smithi092.stdout:9/608: creat d16/d2c/d2f/d3d/d33/d6b/d9e/fc6 x:0 0 0 2024-05-06T12:14:39.214 INFO:tasks.workunit.client.0.smithi092.stdout:2/450: mkdir df/d17/d1d/d33/d4d/d87 0 2024-05-06T12:14:39.216 INFO:tasks.workunit.client.0.smithi092.stdout:0/835: stat d7/dc/d28/d54/d1e/d5d/f76 0 2024-05-06T12:14:39.217 INFO:tasks.workunit.client.0.smithi092.stdout:1/603: rename d3/fa to d3/d10/d13/d19/d21/d72/d9a/da4/fba 0 2024-05-06T12:14:39.218 INFO:tasks.workunit.client.0.smithi092.stdout:6/708: dwrite d4/d4c/da9/fd2 [0,4194304] 0 2024-05-06T12:14:39.219 INFO:tasks.workunit.client.0.smithi092.stdout:4/695: dread d9/df/f78 [0,4194304] 0 2024-05-06T12:14:39.221 INFO:tasks.workunit.client.0.smithi092.stdout:4/696: read d9/d2b/f77 [615750,40114] 0 2024-05-06T12:14:39.221 INFO:tasks.workunit.client.0.smithi092.stdout:2/451: write df/f12 [3806866,115905] 0 2024-05-06T12:14:39.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:38 smithi092 ceph-mon[26954]: pgmap v21: 65 pgs: 65 active+clean; 1.2 GiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 38 MiB/s rd, 167 MiB/s wr, 517 op/s 2024-05-06T12:14:39.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:38 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:39.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:38 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:39.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:38 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:14:39.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:38 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:14:39.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:38 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:14:39.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:38 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:14:39.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:38 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:14:39.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:38 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-06T12:14:39.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:38 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:39.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:38 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:14:39.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:38 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:14:39.224 INFO:tasks.workunit.client.0.smithi092.stdout:5/558: link d4/d5/l6e d4/d34/d28/d4e/d5d/d82/d1f/d78/lca 0 2024-05-06T12:14:39.226 INFO:tasks.workunit.client.0.smithi092.stdout:0/836: fsync d7/dc/d28/d54/d12/f82 0 2024-05-06T12:14:39.226 INFO:tasks.workunit.client.0.smithi092.stdout:6/709: read d4/d24/d26/dae/dc0/f71 [4334285,20066] 0 2024-05-06T12:14:39.228 INFO:tasks.workunit.client.0.smithi092.stdout:2/452: rename df/d15/d50 to df/d15/d88 0 2024-05-06T12:14:39.229 INFO:tasks.workunit.client.0.smithi092.stdout:4/697: mknod d9/d2b/d7c/dab/ce8 0 2024-05-06T12:14:39.230 INFO:tasks.workunit.client.0.smithi092.stdout:4/698: readlink d9/d2b/d65/d83/lb8 0 2024-05-06T12:14:39.231 INFO:tasks.workunit.client.0.smithi092.stdout:1/604: write d3/d42/d84/d4f/d58/d65/d81/f83 [2382104,71943] 0 2024-05-06T12:14:39.235 INFO:tasks.workunit.client.0.smithi092.stdout:2/453: chown df/d17/c34 3491688 1 2024-05-06T12:14:39.236 INFO:tasks.workunit.client.0.smithi092.stdout:4/699: rmdir d9/d2b/d65/d83 39 2024-05-06T12:14:39.237 INFO:tasks.workunit.client.0.smithi092.stdout:4/700: dread - d9/dd/fa1 zero size 2024-05-06T12:14:39.237 INFO:tasks.workunit.client.0.smithi092.stdout:0/837: symlink d7/dc/d6f/da5/dc6/d100/d10c/l116 0 2024-05-06T12:14:39.239 INFO:tasks.workunit.client.0.smithi092.stdout:2/454: write df/d17/d51/f70 [560127,4134] 0 2024-05-06T12:14:39.240 INFO:tasks.workunit.client.0.smithi092.stdout:1/605: creat d3/d10/d13/fbb x:0 0 0 2024-05-06T12:14:39.253 INFO:tasks.workunit.client.0.smithi092.stdout:2/455: dread df/f79 [0,4194304] 0 2024-05-06T12:14:39.254 INFO:tasks.workunit.client.0.smithi092.stdout:1/606: dread d3/f2c [0,4194304] 0 2024-05-06T12:14:39.266 INFO:tasks.workunit.client.0.smithi092.stdout:2/456: dread df/d17/d1d/f49 [0,4194304] 0 2024-05-06T12:14:39.313 INFO:tasks.workunit.client.0.smithi092.stdout:7/740: sync 2024-05-06T12:14:39.313 INFO:tasks.workunit.client.0.smithi092.stdout:7/741: readlink d9/da5/l24 0 2024-05-06T12:14:39.342 INFO:tasks.workunit.client.0.smithi092.stdout:1/607: creat d3/d42/d84/d4f/d58/d65/d81/fbc x:0 0 0 2024-05-06T12:14:39.343 INFO:tasks.workunit.client.0.smithi092.stdout:2/457: creat df/d15/d88/f89 x:0 0 0 2024-05-06T12:14:39.346 INFO:tasks.workunit.client.0.smithi092.stdout:0/838: mkdir d7/dc/d6f/da5/dc6/d117 0 2024-05-06T12:14:39.347 INFO:tasks.workunit.client.0.smithi092.stdout:6/710: link c2 d4/d20/d23/ce7 0 2024-05-06T12:14:39.348 INFO:tasks.workunit.client.0.smithi092.stdout:5/559: getdents d4/d34/d28/d4e/d5d/d82/d1f/d78 0 2024-05-06T12:14:39.351 INFO:tasks.workunit.client.0.smithi092.stdout:1/608: creat d3/d10/d13/d19/d21/d72/fbd x:0 0 0 2024-05-06T12:14:39.352 INFO:tasks.workunit.client.0.smithi092.stdout:7/742: mkdir d9/de/d10/df0/df5 0 2024-05-06T12:14:39.355 INFO:tasks.workunit.client.0.smithi092.stdout:0/839: stat d7/dc/d28/d64/fee 0 2024-05-06T12:14:39.356 INFO:tasks.workunit.client.0.smithi092.stdout:6/711: mkdir d4/d24/d26/dae/de8 0 2024-05-06T12:14:39.357 INFO:tasks.workunit.client.0.smithi092.stdout:6/712: chown d4/d20/f1b 1 1 2024-05-06T12:14:39.359 INFO:tasks.workunit.client.0.smithi092.stdout:7/743: creat d9/de/d1f/ff6 x:0 0 0 2024-05-06T12:14:39.361 INFO:tasks.workunit.client.0.smithi092.stdout:1/609: rename d3/d10/d13/d19/d21/d88/d8b/f8c to d3/d10/d13/d19/d34/fbe 0 2024-05-06T12:14:39.364 INFO:tasks.workunit.client.0.smithi092.stdout:4/701: getdents d9/d2b/d65/d83 0 2024-05-06T12:14:39.364 INFO:tasks.workunit.client.0.smithi092.stdout:6/713: dwrite d4/d4c/d52/f4a [0,4194304] 0 2024-05-06T12:14:39.367 INFO:tasks.workunit.client.1.smithi190.stdout:7/2: sync 2024-05-06T12:14:39.367 INFO:tasks.workunit.client.1.smithi190.stdout:1/4: sync 2024-05-06T12:14:39.368 INFO:tasks.workunit.client.1.smithi190.stdout:7/3: dwrite - no filename 2024-05-06T12:14:39.369 INFO:tasks.workunit.client.0.smithi092.stdout:7/744: mkdir d9/de/d10/d17/d99/dc3/dd2/df7 0 2024-05-06T12:14:39.370 INFO:tasks.workunit.client.1.smithi190.stdout:1/5: creat f1 x:0 0 0 2024-05-06T12:14:39.370 INFO:tasks.workunit.client.1.smithi190.stdout:1/6: read - f1 zero size 2024-05-06T12:14:39.370 INFO:tasks.workunit.client.0.smithi092.stdout:0/840: getdents d7/dc/d28/d54/d1e/d39/d71 0 2024-05-06T12:14:39.371 INFO:tasks.workunit.client.0.smithi092.stdout:1/610: dwrite d3/f98 [0,4194304] 0 2024-05-06T12:14:39.371 INFO:tasks.workunit.client.1.smithi190.stdout:7/4: mkdir d0/d1 0 2024-05-06T12:14:39.372 INFO:tasks.workunit.client.1.smithi190.stdout:1/7: symlink l2 0 2024-05-06T12:14:39.373 INFO:tasks.workunit.client.0.smithi092.stdout:0/841: stat d7/dc/d28/d54/d12/d7f/db2 0 2024-05-06T12:14:39.373 INFO:tasks.workunit.client.1.smithi190.stdout:1/8: fsync f0 0 2024-05-06T12:14:39.373 INFO:tasks.workunit.client.0.smithi092.stdout:5/560: dread f0 [0,4194304] 0 2024-05-06T12:14:39.374 INFO:tasks.workunit.client.1.smithi190.stdout:7/5: mkdir d0/d1/d2 0 2024-05-06T12:14:39.376 INFO:tasks.workunit.client.1.smithi190.stdout:1/9: unlink f1 0 2024-05-06T12:14:39.377 INFO:tasks.workunit.client.0.smithi092.stdout:2/458: write df/d17/d1d/f4e [1631833,113483] 0 2024-05-06T12:14:39.380 INFO:tasks.workunit.client.1.smithi190.stdout:7/6: symlink d0/d1/l3 0 2024-05-06T12:14:39.380 INFO:tasks.workunit.client.1.smithi190.stdout:1/10: creat f3 x:0 0 0 2024-05-06T12:14:39.380 INFO:tasks.workunit.client.1.smithi190.stdout:7/7: chown d0/d1 65765840 1 2024-05-06T12:14:39.380 INFO:tasks.workunit.client.1.smithi190.stdout:7/8: chown d0/d1 49823 1 2024-05-06T12:14:39.380 INFO:tasks.workunit.client.1.smithi190.stdout:7/9: dwrite - no filename 2024-05-06T12:14:39.380 INFO:tasks.workunit.client.1.smithi190.stdout:7/10: write - no filename 2024-05-06T12:14:39.380 INFO:tasks.workunit.client.1.smithi190.stdout:1/11: truncate f3 1047616 0 2024-05-06T12:14:39.381 INFO:tasks.workunit.client.1.smithi190.stdout:1/12: readlink l2 0 2024-05-06T12:14:39.381 INFO:tasks.workunit.client.0.smithi092.stdout:1/611: unlink d3/d42/d84/f8e 0 2024-05-06T12:14:39.381 INFO:tasks.workunit.client.0.smithi092.stdout:4/702: symlink d9/d2b/d7c/dab/dd8/le9 0 2024-05-06T12:14:39.382 INFO:tasks.workunit.client.0.smithi092.stdout:7/745: rmdir d9/de/d10/d17/d21/d2e/d87 39 2024-05-06T12:14:39.382 INFO:tasks.workunit.client.1.smithi190.stdout:1/13: write f3 [732345,96733] 0 2024-05-06T12:14:39.383 INFO:tasks.workunit.client.1.smithi190.stdout:7/11: mkdir d0/d4 0 2024-05-06T12:14:39.383 INFO:tasks.workunit.client.1.smithi190.stdout:7/12: dwrite - no filename 2024-05-06T12:14:39.383 INFO:tasks.workunit.client.1.smithi190.stdout:1/14: readlink l2 0 2024-05-06T12:14:39.384 INFO:tasks.workunit.client.1.smithi190.stdout:1/15: truncate f0 55265 0 2024-05-06T12:14:39.384 INFO:tasks.workunit.client.1.smithi190.stdout:1/16: truncate f0 661895 0 2024-05-06T12:14:39.384 INFO:tasks.workunit.client.0.smithi092.stdout:2/459: rename df/d17/c34 to df/d5c/c8a 0 2024-05-06T12:14:39.385 INFO:tasks.workunit.client.1.smithi190.stdout:7/13: unlink d0/d1/l3 0 2024-05-06T12:14:39.385 INFO:tasks.workunit.client.1.smithi190.stdout:7/14: dread - no filename 2024-05-06T12:14:39.385 INFO:tasks.workunit.client.0.smithi092.stdout:4/703: dread d9/dd/f9e [4194304,4194304] 0 2024-05-06T12:14:39.389 INFO:tasks.workunit.client.1.smithi190.stdout:7/15: creat d0/d1/d2/f5 x:0 0 0 2024-05-06T12:14:39.391 INFO:tasks.workunit.client.1.smithi190.stdout:7/16: creat d0/d1/f6 x:0 0 0 2024-05-06T12:14:39.392 INFO:tasks.workunit.client.1.smithi190.stdout:1/17: dwrite f3 [0,4194304] 0 2024-05-06T12:14:39.393 INFO:tasks.workunit.client.1.smithi190.stdout:1/18: fsync f0 0 2024-05-06T12:14:39.395 INFO:tasks.workunit.client.1.smithi190.stdout:1/19: truncate f0 1502140 0 2024-05-06T12:14:39.396 INFO:tasks.workunit.client.1.smithi190.stdout:1/20: truncate f0 1752245 0 2024-05-06T12:14:39.400 INFO:tasks.workunit.client.1.smithi190.stdout:7/17: dwrite d0/d1/d2/f5 [0,4194304] 0 2024-05-06T12:14:39.413 INFO:tasks.workunit.client.1.smithi190.stdout:9/112: sync 2024-05-06T12:14:39.414 INFO:tasks.workunit.client.1.smithi190.stdout:2/19: sync 2024-05-06T12:14:39.414 INFO:tasks.workunit.client.1.smithi190.stdout:2/20: dwrite - no filename 2024-05-06T12:14:39.414 INFO:tasks.workunit.client.1.smithi190.stdout:2/21: write - no filename 2024-05-06T12:14:39.414 INFO:tasks.workunit.client.1.smithi190.stdout:6/9: sync 2024-05-06T12:14:39.414 INFO:tasks.workunit.client.1.smithi190.stdout:5/19: sync 2024-05-06T12:14:39.414 INFO:tasks.workunit.client.1.smithi190.stdout:4/7: sync 2024-05-06T12:14:39.414 INFO:tasks.workunit.client.1.smithi190.stdout:8/128: sync 2024-05-06T12:14:39.414 INFO:tasks.workunit.client.1.smithi190.stdout:3/57: sync 2024-05-06T12:14:39.414 INFO:tasks.workunit.client.1.smithi190.stdout:0/149: sync 2024-05-06T12:14:39.414 INFO:tasks.workunit.client.1.smithi190.stdout:9/113: write f7 [1930012,88483] 0 2024-05-06T12:14:39.415 INFO:tasks.workunit.client.1.smithi190.stdout:2/22: chown d2 0 1 2024-05-06T12:14:39.415 INFO:tasks.workunit.client.1.smithi190.stdout:2/23: write - no filename 2024-05-06T12:14:39.415 INFO:tasks.workunit.client.1.smithi190.stdout:2/24: dread - no filename 2024-05-06T12:14:39.415 INFO:tasks.workunit.client.1.smithi190.stdout:2/25: read - no filename 2024-05-06T12:14:39.415 INFO:tasks.workunit.client.1.smithi190.stdout:2/26: truncate - no filename 2024-05-06T12:14:39.417 INFO:tasks.workunit.client.1.smithi190.stdout:8/129: creat d3/d8/f2c x:0 0 0 2024-05-06T12:14:39.419 INFO:tasks.workunit.client.1.smithi190.stdout:3/58: mkdir d5/d7/d15 0 2024-05-06T12:14:39.423 INFO:tasks.workunit.client.1.smithi190.stdout:8/130: creat d3/f2d x:0 0 0 2024-05-06T12:14:39.424 INFO:tasks.workunit.client.1.smithi190.stdout:6/10: read - f2 zero size 2024-05-06T12:14:39.424 INFO:tasks.workunit.client.1.smithi190.stdout:4/8: fsync f1 0 2024-05-06T12:14:39.425 INFO:tasks.workunit.client.1.smithi190.stdout:6/11: stat f2 0 2024-05-06T12:14:39.426 INFO:tasks.workunit.client.1.smithi190.stdout:2/27: chown d2/c4 9861865 1 2024-05-06T12:14:39.426 INFO:tasks.workunit.client.1.smithi190.stdout:4/9: dread - f1 zero size 2024-05-06T12:14:39.428 INFO:tasks.workunit.client.1.smithi190.stdout:3/59: dwrite d5/ff [0,4194304] 0 2024-05-06T12:14:39.428 INFO:tasks.workunit.client.1.smithi190.stdout:2/28: chown d2/c4 76957 1 2024-05-06T12:14:39.428 INFO:tasks.workunit.client.1.smithi190.stdout:2/29: dread - no filename 2024-05-06T12:14:39.430 INFO:tasks.workunit.client.1.smithi190.stdout:5/20: creat d2/f9 x:0 0 0 2024-05-06T12:14:39.431 INFO:tasks.workunit.client.1.smithi190.stdout:6/12: creat f4 x:0 0 0 2024-05-06T12:14:39.432 INFO:tasks.workunit.client.1.smithi190.stdout:3/60: write d5/ff [1266371,70336] 0 2024-05-06T12:14:39.432 INFO:tasks.workunit.client.1.smithi190.stdout:0/150: write d9/df/f10 [2552191,84075] 0 2024-05-06T12:14:39.433 INFO:tasks.workunit.client.1.smithi190.stdout:6/13: readlink l3 0 2024-05-06T12:14:39.433 INFO:tasks.workunit.client.1.smithi190.stdout:8/131: creat d3/f2e x:0 0 0 2024-05-06T12:14:39.434 INFO:tasks.workunit.client.1.smithi190.stdout:5/21: chown d2/f3 7770 1 2024-05-06T12:14:39.434 INFO:tasks.workunit.client.1.smithi190.stdout:0/151: fsync d9/fb 0 2024-05-06T12:14:39.436 INFO:tasks.workunit.client.1.smithi190.stdout:4/10: rename f1 to d2/f4 0 2024-05-06T12:14:39.436 INFO:tasks.workunit.client.1.smithi190.stdout:6/14: dread - f4 zero size 2024-05-06T12:14:39.438 INFO:tasks.workunit.client.1.smithi190.stdout:9/114: dwrite f5 [0,4194304] 0 2024-05-06T12:14:39.439 INFO:tasks.workunit.client.1.smithi190.stdout:8/132: read - d3/d13/f21 zero size 2024-05-06T12:14:39.439 INFO:tasks.workunit.client.1.smithi190.stdout:2/30: symlink d2/l5 0 2024-05-06T12:14:39.445 INFO:tasks.workunit.client.0.smithi092.stdout:2/460: write df/d17/d1d/d33/f47 [508359,119539] 0 2024-05-06T12:14:39.446 INFO:tasks.workunit.client.0.smithi092.stdout:7/746: chown d9/de/da4/cd7 220462 1 2024-05-06T12:14:39.447 INFO:tasks.workunit.client.1.smithi190.stdout:0/152: mkdir d9/df/d21/d31 0 2024-05-06T12:14:39.447 INFO:tasks.workunit.client.1.smithi190.stdout:2/31: mknod d2/c6 0 2024-05-06T12:14:39.447 INFO:tasks.workunit.client.1.smithi190.stdout:2/32: dread - no filename 2024-05-06T12:14:39.447 INFO:tasks.workunit.client.1.smithi190.stdout:2/33: write - no filename 2024-05-06T12:14:39.447 INFO:tasks.workunit.client.0.smithi092.stdout:6/714: link d4/d24/d26/fa4 d4/d20/d23/fe9 0 2024-05-06T12:14:39.448 INFO:tasks.workunit.client.1.smithi190.stdout:3/61: creat d5/d7/d15/f16 x:0 0 0 2024-05-06T12:14:39.449 INFO:tasks.workunit.client.0.smithi092.stdout:4/704: stat d9/dd/cd1 0 2024-05-06T12:14:39.449 INFO:tasks.workunit.client.0.smithi092.stdout:7/747: mkdir d9/de/da4/df8 0 2024-05-06T12:14:39.450 INFO:tasks.workunit.client.1.smithi190.stdout:6/15: dwrite f2 [0,4194304] 0 2024-05-06T12:14:39.451 INFO:tasks.workunit.client.0.smithi092.stdout:2/461: creat df/d17/d1d/d33/d3c/f8b x:0 0 0 2024-05-06T12:14:39.452 INFO:tasks.workunit.client.0.smithi092.stdout:4/705: mkdir d9/d2b/d5b/daf/dea 0 2024-05-06T12:14:39.452 INFO:tasks.workunit.client.1.smithi190.stdout:3/62: chown d5/d14/d11/f13 10 1 2024-05-06T12:14:39.455 INFO:tasks.workunit.client.1.smithi190.stdout:9/115: mknod db/d23/c2e 0 2024-05-06T12:14:39.455 INFO:tasks.workunit.client.0.smithi092.stdout:5/561: dwrite d4/d34/d28/d4e/f87 [0,4194304] 0 2024-05-06T12:14:39.456 INFO:tasks.workunit.client.1.smithi190.stdout:6/16: truncate f4 542540 0 2024-05-06T12:14:39.456 INFO:tasks.workunit.client.1.smithi190.stdout:6/17: fsync f1 0 2024-05-06T12:14:39.457 INFO:tasks.workunit.client.0.smithi092.stdout:5/562: write d4/d34/d28/fac [539545,78142] 0 2024-05-06T12:14:39.457 INFO:tasks.workunit.client.0.smithi092.stdout:7/748: unlink d9/de/d10/d17/fb5 0 2024-05-06T12:14:39.459 INFO:tasks.workunit.client.1.smithi190.stdout:3/63: write d5/d14/d11/f13 [956150,6803] 0 2024-05-06T12:14:39.459 INFO:tasks.workunit.client.1.smithi190.stdout:8/133: dwrite d3/d8/fd [4194304,4194304] 0 2024-05-06T12:14:39.460 INFO:tasks.workunit.client.0.smithi092.stdout:6/715: link d4/d20/d22/cb7 d4/d4c/d52/d6c/da0/cea 0 2024-05-06T12:14:39.460 INFO:tasks.workunit.client.1.smithi190.stdout:0/153: unlink d9/df/d21/f23 0 2024-05-06T12:14:39.461 INFO:tasks.workunit.client.0.smithi092.stdout:6/716: dread - d4/d20/d22/f9e zero size 2024-05-06T12:14:39.462 INFO:tasks.workunit.client.0.smithi092.stdout:5/563: mkdir d4/d5/d10/d31/d53/dcb 0 2024-05-06T12:14:39.462 INFO:tasks.workunit.client.1.smithi190.stdout:2/34: mknod d2/c7 0 2024-05-06T12:14:39.463 INFO:tasks.workunit.client.0.smithi092.stdout:4/706: symlink d9/df/d48/leb 0 2024-05-06T12:14:39.464 INFO:tasks.workunit.client.0.smithi092.stdout:5/564: fsync d4/d5/d10/d31/d53/d64/f8b 0 2024-05-06T12:14:39.464 INFO:tasks.workunit.client.0.smithi092.stdout:4/707: chown d9/df/d62/d7b/db6/fd4 3 1 2024-05-06T12:14:39.467 INFO:tasks.workunit.client.0.smithi092.stdout:7/749: rename d9/de/d10/d17/d6a/cdf to d9/de/d10/d17/d21/d2e/cf9 0 2024-05-06T12:14:39.468 INFO:tasks.workunit.client.0.smithi092.stdout:7/750: readlink d9/de/d10/l4a 0 2024-05-06T12:14:39.469 INFO:tasks.workunit.client.0.smithi092.stdout:4/708: mknod d9/df/d62/dbb/cec 0 2024-05-06T12:14:39.470 INFO:tasks.workunit.client.1.smithi190.stdout:8/134: dread d3/d8/f1a [0,4194304] 0 2024-05-06T12:14:39.470 INFO:tasks.workunit.client.1.smithi190.stdout:8/135: dread - d3/f2e zero size 2024-05-06T12:14:39.470 INFO:tasks.workunit.client.0.smithi092.stdout:6/717: getdents d4 0 2024-05-06T12:14:39.471 INFO:tasks.workunit.client.1.smithi190.stdout:2/35: mkdir d2/d8 0 2024-05-06T12:14:39.471 INFO:tasks.workunit.client.0.smithi092.stdout:6/718: chown d4/le3 245537 1 2024-05-06T12:14:39.474 INFO:tasks.workunit.client.0.smithi092.stdout:4/709: getdents d9/d2b/d5b/daf/dea 0 2024-05-06T12:14:39.475 INFO:tasks.workunit.client.0.smithi092.stdout:2/462: write f6 [925851,115097] 0 2024-05-06T12:14:39.476 INFO:tasks.workunit.client.0.smithi092.stdout:4/710: write d9/fc0 [67683,2304] 0 2024-05-06T12:14:39.477 INFO:tasks.workunit.client.0.smithi092.stdout:6/719: creat d4/d4c/feb x:0 0 0 2024-05-06T12:14:39.478 INFO:tasks.workunit.client.0.smithi092.stdout:6/720: dread - d4/d20/fb2 zero size 2024-05-06T12:14:39.478 INFO:tasks.workunit.client.0.smithi092.stdout:2/463: unlink df/d15/c80 0 2024-05-06T12:14:39.479 INFO:tasks.workunit.client.1.smithi190.stdout:2/36: symlink d2/l9 0 2024-05-06T12:14:39.479 INFO:tasks.workunit.client.1.smithi190.stdout:2/37: dread - no filename 2024-05-06T12:14:39.479 INFO:tasks.workunit.client.1.smithi190.stdout:2/38: dwrite - no filename 2024-05-06T12:14:39.479 INFO:tasks.workunit.client.1.smithi190.stdout:2/39: truncate - no filename 2024-05-06T12:14:39.479 INFO:tasks.workunit.client.1.smithi190.stdout:2/40: read - no filename 2024-05-06T12:14:39.480 INFO:tasks.workunit.client.0.smithi092.stdout:6/721: readlink d4/d24/d38/d5d/d6d/d7b/lb9 0 2024-05-06T12:14:39.480 INFO:tasks.workunit.client.0.smithi092.stdout:2/464: rmdir df/d17/d53 39 2024-05-06T12:14:39.481 INFO:tasks.workunit.client.0.smithi092.stdout:6/722: mknod d4/d24/d38/d5d/d6d/d7b/cec 0 2024-05-06T12:14:39.482 INFO:tasks.workunit.client.0.smithi092.stdout:2/465: readlink df/d17/l20 0 2024-05-06T12:14:39.483 INFO:tasks.workunit.client.1.smithi190.stdout:9/116: dwrite f7 [0,4194304] 0 2024-05-06T12:14:39.483 INFO:tasks.workunit.client.0.smithi092.stdout:6/723: write d4/d4c/d52/f56 [691888,13390] 0 2024-05-06T12:14:39.484 INFO:tasks.workunit.client.0.smithi092.stdout:6/724: chown d4/d4c/d52/d6c/da0/dc3 1672550 1 2024-05-06T12:14:39.485 INFO:tasks.workunit.client.1.smithi190.stdout:2/41: readlink l0 0 2024-05-06T12:14:39.487 INFO:tasks.workunit.client.0.smithi092.stdout:6/725: rename d4/c82 to d4/d24/d4e/ced 0 2024-05-06T12:14:39.488 INFO:tasks.workunit.client.1.smithi190.stdout:3/64: dwrite d5/d7/d15/f16 [0,4194304] 0 2024-05-06T12:14:39.491 INFO:tasks.workunit.client.1.smithi190.stdout:0/154: dwrite d9/f1b [4194304,4194304] 0 2024-05-06T12:14:39.491 INFO:tasks.workunit.client.0.smithi092.stdout:6/726: truncate d4/d20/d23/d30/f6f 85216 0 2024-05-06T12:14:39.492 INFO:tasks.workunit.client.1.smithi190.stdout:2/42: creat d2/fa x:0 0 0 2024-05-06T12:14:39.493 INFO:tasks.workunit.client.1.smithi190.stdout:2/43: dread - d2/fa zero size 2024-05-06T12:14:39.493 INFO:tasks.workunit.client.0.smithi092.stdout:6/727: dread - d4/d24/d38/d5d/fa3 zero size 2024-05-06T12:14:39.494 INFO:tasks.workunit.client.1.smithi190.stdout:3/65: creat d5/d14/f17 x:0 0 0 2024-05-06T12:14:39.496 INFO:tasks.workunit.client.1.smithi190.stdout:0/155: rename d9/df/l18 to d9/df/l32 0 2024-05-06T12:14:39.497 INFO:tasks.workunit.client.0.smithi092.stdout:6/728: unlink d4/d24/c97 0 2024-05-06T12:14:39.499 INFO:tasks.workunit.client.0.smithi092.stdout:8/654: sync 2024-05-06T12:14:39.499 INFO:tasks.workunit.client.1.smithi190.stdout:2/44: rename d2/l5 to d2/lb 0 2024-05-06T12:14:39.500 INFO:tasks.workunit.client.1.smithi190.stdout:3/66: write d5/fc [970599,120925] 0 2024-05-06T12:14:39.503 INFO:tasks.workunit.client.1.smithi190.stdout:0/156: creat d9/df/f33 x:0 0 0 2024-05-06T12:14:39.505 INFO:tasks.workunit.client.1.smithi190.stdout:9/117: rename db/dd/df/c1c to db/d23/c2f 0 2024-05-06T12:14:39.506 INFO:tasks.workunit.client.1.smithi190.stdout:3/67: symlink d5/l18 0 2024-05-06T12:14:39.506 INFO:tasks.workunit.client.1.smithi190.stdout:9/118: write db/d23/f2a [1642105,107490] 0 2024-05-06T12:14:39.507 INFO:tasks.workunit.client.1.smithi190.stdout:2/45: mkdir d2/d8/dc 0 2024-05-06T12:14:39.507 INFO:tasks.workunit.client.1.smithi190.stdout:3/68: stat d5/d7/db/fd 0 2024-05-06T12:14:39.511 INFO:tasks.workunit.client.0.smithi092.stdout:6/729: dwrite d4/f1a [4194304,4194304] 0 2024-05-06T12:14:39.513 INFO:tasks.workunit.client.0.smithi092.stdout:6/730: fdatasync d4/f8e 0 2024-05-06T12:14:39.518 INFO:tasks.workunit.client.0.smithi092.stdout:2/466: dread fd [0,4194304] 0 2024-05-06T12:14:39.521 INFO:tasks.workunit.client.0.smithi092.stdout:8/655: dread d9/f36 [0,4194304] 0 2024-05-06T12:14:39.522 INFO:tasks.workunit.client.0.smithi092.stdout:8/656: stat d9/d21/f3f 0 2024-05-06T12:14:39.522 INFO:tasks.workunit.client.0.smithi092.stdout:6/731: dwrite d4/d24/f55 [4194304,4194304] 0 2024-05-06T12:14:39.527 INFO:tasks.workunit.client.0.smithi092.stdout:6/732: creat d4/d24/d26/dc4/fee x:0 0 0 2024-05-06T12:14:39.529 INFO:tasks.workunit.client.0.smithi092.stdout:6/733: write d4/f17 [5582916,68194] 0 2024-05-06T12:14:39.533 INFO:tasks.workunit.client.0.smithi092.stdout:6/734: link d4/d24/d26/dc4/fee d4/d4c/d52/ddc/fef 0 2024-05-06T12:14:39.536 INFO:tasks.workunit.client.0.smithi092.stdout:6/735: link d4/d24/d26/dc4/fca d4/d24/d38/d5d/d6d/dd8/ff0 0 2024-05-06T12:14:39.538 INFO:tasks.workunit.client.0.smithi092.stdout:6/736: rmdir d4/d4c/d52/d6c/da0/dc3 39 2024-05-06T12:14:39.542 INFO:tasks.workunit.client.0.smithi092.stdout:6/737: link d4/d24/d38/d5d/d6d/f84 d4/d24/d4e/ff1 0 2024-05-06T12:14:39.544 INFO:tasks.workunit.client.0.smithi092.stdout:6/738: mkdir d4/d20/de2/df2 0 2024-05-06T12:14:39.545 INFO:tasks.workunit.client.0.smithi092.stdout:6/739: write d4/f76 [1420796,87733] 0 2024-05-06T12:14:39.548 INFO:tasks.workunit.client.0.smithi092.stdout:6/740: dread d4/f11 [0,4194304] 0 2024-05-06T12:14:39.554 INFO:tasks.workunit.client.0.smithi092.stdout:8/657: write f0 [3503664,108673] 0 2024-05-06T12:14:39.560 INFO:tasks.workunit.client.0.smithi092.stdout:8/658: rename d9/d21/d37/d46/d5c/d93/fc4 to d9/d21/d37/d46/feb 0 2024-05-06T12:14:39.564 INFO:tasks.workunit.client.0.smithi092.stdout:8/659: mkdir d9/d21/d37/d46/d5c/d62/da5/dec 0 2024-05-06T12:14:39.566 INFO:tasks.workunit.client.0.smithi092.stdout:8/660: rename d9/d5a/fa9 to d9/d21/d37/d46/d5c/d62/da5/dec/fed 0 2024-05-06T12:14:39.567 INFO:tasks.workunit.client.0.smithi092.stdout:8/661: dread - d9/dc/fc8 zero size 2024-05-06T12:14:39.568 INFO:tasks.workunit.client.0.smithi092.stdout:8/662: fdatasync d9/f42 0 2024-05-06T12:14:39.570 INFO:tasks.workunit.client.1.smithi190.stdout:2/46: creat d2/fd x:0 0 0 2024-05-06T12:14:39.571 INFO:tasks.workunit.client.1.smithi190.stdout:3/69: symlink d5/d14/l19 0 2024-05-06T12:14:39.572 INFO:tasks.workunit.client.1.smithi190.stdout:3/70: write d5/fc [598481,110075] 0 2024-05-06T12:14:39.580 INFO:tasks.workunit.client.0.smithi092.stdout:8/663: dread f0 [0,4194304] 0 2024-05-06T12:14:39.582 INFO:tasks.workunit.client.0.smithi092.stdout:8/664: mkdir d9/d21/d37/d46/dee 0 2024-05-06T12:14:39.583 INFO:tasks.workunit.client.0.smithi092.stdout:8/665: dread - d9/d21/dc0/fcc zero size 2024-05-06T12:14:39.585 INFO:tasks.workunit.client.0.smithi092.stdout:8/666: creat d9/d21/d37/d46/d5c/d71/d75/fef x:0 0 0 2024-05-06T12:14:39.586 INFO:tasks.workunit.client.0.smithi092.stdout:8/667: truncate d9/d5a/d9f/fc6 624795 0 2024-05-06T12:14:39.587 INFO:tasks.workunit.client.0.smithi092.stdout:8/668: dread - d9/d21/d37/d46/fd2 zero size 2024-05-06T12:14:39.590 INFO:tasks.workunit.client.0.smithi092.stdout:8/669: dread d9/d21/d37/d46/f5e [0,4194304] 0 2024-05-06T12:14:39.592 INFO:tasks.workunit.client.0.smithi092.stdout:8/670: creat d9/d21/d37/d74/ff0 x:0 0 0 2024-05-06T12:14:39.594 INFO:tasks.workunit.client.0.smithi092.stdout:8/671: read d9/dc/f4d [886843,84688] 0 2024-05-06T12:14:39.597 INFO:tasks.workunit.client.0.smithi092.stdout:8/672: creat d9/d21/d37/d46/ff1 x:0 0 0 2024-05-06T12:14:39.601 INFO:tasks.workunit.client.0.smithi092.stdout:8/673: write d9/dc/f4d [254513,80641] 0 2024-05-06T12:14:39.611 INFO:tasks.workunit.client.0.smithi092.stdout:8/674: dread d9/dc/f2d [0,4194304] 0 2024-05-06T12:14:39.613 INFO:tasks.workunit.client.0.smithi092.stdout:8/675: write d9/d21/d54/d79/fd8 [614370,113081] 0 2024-05-06T12:14:39.620 INFO:tasks.workunit.client.0.smithi092.stdout:8/676: dwrite d9/f42 [4194304,4194304] 0 2024-05-06T12:14:39.622 INFO:tasks.workunit.client.0.smithi092.stdout:8/677: mkdir d9/d21/dc0/df2 0 2024-05-06T12:14:39.623 INFO:tasks.workunit.client.0.smithi092.stdout:8/678: fsync d9/dc/ff 0 2024-05-06T12:14:39.626 INFO:tasks.workunit.client.0.smithi092.stdout:8/679: rmdir d9/d21/d37/d46 39 2024-05-06T12:14:39.627 INFO:tasks.workunit.client.1.smithi190.stdout:4/11: sync 2024-05-06T12:14:39.627 INFO:tasks.workunit.client.1.smithi190.stdout:5/22: sync 2024-05-06T12:14:39.629 INFO:tasks.workunit.client.0.smithi092.stdout:8/680: truncate d9/d21/d37/d46/d5c/d62/da5/dec/fed 972223 0 2024-05-06T12:14:39.631 INFO:tasks.workunit.client.0.smithi092.stdout:8/681: dread - d9/d21/d37/d46/d5c/d71/d75/f9c zero size 2024-05-06T12:14:39.637 INFO:tasks.workunit.client.0.smithi092.stdout:8/682: write d9/d21/d37/d46/f5e [3155684,79292] 0 2024-05-06T12:14:39.638 INFO:tasks.workunit.client.0.smithi092.stdout:8/683: truncate d9/d21/d37/d46/d5c/d71/fca 400520 0 2024-05-06T12:14:39.641 INFO:tasks.workunit.client.0.smithi092.stdout:8/684: link d9/d21/f6f d9/d21/d37/d46/d5c/d71/ff3 0 2024-05-06T12:14:39.643 INFO:tasks.workunit.client.0.smithi092.stdout:8/685: mknod d9/d21/d37/d46/d5c/d71/d75/cf4 0 2024-05-06T12:14:39.643 INFO:tasks.workunit.client.0.smithi092.stdout:8/686: chown d9/d21/d54/f63 2 1 2024-05-06T12:14:39.645 INFO:tasks.workunit.client.0.smithi092.stdout:8/687: creat d9/d9d/da6/ff5 x:0 0 0 2024-05-06T12:14:39.672 INFO:tasks.workunit.client.0.smithi092.stdout:3/518: sync 2024-05-06T12:14:39.674 INFO:tasks.workunit.client.0.smithi092.stdout:3/519: creat d8/db/f9f x:0 0 0 2024-05-06T12:14:39.675 INFO:tasks.workunit.client.0.smithi092.stdout:3/520: dread - d8/d24/d33/d91/f9b zero size 2024-05-06T12:14:39.675 INFO:tasks.workunit.client.0.smithi092.stdout:8/688: rename d9/d21/d72/f7e to d9/d21/d37/ff6 0 2024-05-06T12:14:39.679 INFO:tasks.workunit.client.0.smithi092.stdout:8/689: creat d9/dc/d4a/d5f/ff7 x:0 0 0 2024-05-06T12:14:39.682 INFO:tasks.workunit.client.0.smithi092.stdout:3/521: rename d8/db/d14/d5a/f7b to d8/d24/d27/fa0 0 2024-05-06T12:14:39.684 INFO:tasks.workunit.client.0.smithi092.stdout:3/522: symlink d8/d24/d33/d39/la1 0 2024-05-06T12:14:39.685 INFO:tasks.workunit.client.0.smithi092.stdout:3/523: getdents d8/d24/d33/d39/d5b/d98 0 2024-05-06T12:14:39.687 INFO:tasks.workunit.client.0.smithi092.stdout:3/524: symlink d8/d24/d33/la2 0 2024-05-06T12:14:39.688 INFO:tasks.workunit.client.0.smithi092.stdout:3/525: chown d8/d24/d33/d4b/f41 1773 1 2024-05-06T12:14:39.691 INFO:tasks.workunit.client.0.smithi092.stdout:3/526: rename d8/d24/d27/f42 to d8/db/d4c/d40/d7a/fa3 0 2024-05-06T12:14:39.694 INFO:tasks.workunit.client.0.smithi092.stdout:8/690: write d9/d21/d37/d74/f94 [933392,90687] 0 2024-05-06T12:14:39.695 INFO:tasks.workunit.client.1.smithi190.stdout:4/12: symlink d2/l5 0 2024-05-06T12:14:39.696 INFO:tasks.workunit.client.1.smithi190.stdout:5/23: write d2/f3 [1003259,60368] 0 2024-05-06T12:14:39.697 INFO:tasks.workunit.client.0.smithi092.stdout:8/691: mknod d9/d21/d54/d79/d8a/cf8 0 2024-05-06T12:14:39.698 INFO:tasks.workunit.client.1.smithi190.stdout:4/13: rmdir d2 39 2024-05-06T12:14:39.700 INFO:tasks.workunit.client.0.smithi092.stdout:3/527: dread d8/d24/d33/f6a [0,4194304] 0 2024-05-06T12:14:39.701 INFO:tasks.workunit.client.1.smithi190.stdout:4/14: dread - d2/f4 zero size 2024-05-06T12:14:39.702 INFO:tasks.workunit.client.0.smithi092.stdout:3/528: rmdir d8/db/d4c 39 2024-05-06T12:14:39.703 INFO:tasks.workunit.client.0.smithi092.stdout:8/692: write d9/f33 [801910,46469] 0 2024-05-06T12:14:39.704 INFO:tasks.workunit.client.1.smithi190.stdout:4/15: rmdir d2 39 2024-05-06T12:14:39.704 INFO:tasks.workunit.client.0.smithi092.stdout:3/529: symlink d8/d24/d27/la4 0 2024-05-06T12:14:39.706 INFO:tasks.workunit.client.0.smithi092.stdout:3/530: write d8/d24/d33/d4b/d74/f9e [791610,61232] 0 2024-05-06T12:14:39.706 INFO:tasks.workunit.client.0.smithi092.stdout:8/693: mkdir d9/d21/d37/d74/df9 0 2024-05-06T12:14:39.707 INFO:tasks.workunit.client.1.smithi190.stdout:4/16: rename d2/l5 to d2/l6 0 2024-05-06T12:14:39.709 INFO:tasks.workunit.client.1.smithi190.stdout:4/17: mknod d2/c7 0 2024-05-06T12:14:39.712 INFO:tasks.workunit.client.0.smithi092.stdout:8/694: dread d9/d5a/dd9/fda [0,4194304] 0 2024-05-06T12:14:39.713 INFO:tasks.workunit.client.0.smithi092.stdout:3/531: dwrite d8/d24/d27/fa0 [0,4194304] 0 2024-05-06T12:14:39.714 INFO:tasks.workunit.client.0.smithi092.stdout:8/695: creat d9/d21/d54/ffa x:0 0 0 2024-05-06T12:14:39.716 INFO:tasks.workunit.client.0.smithi092.stdout:3/532: creat d8/db/d14/d5a/fa5 x:0 0 0 2024-05-06T12:14:39.718 INFO:tasks.workunit.client.0.smithi092.stdout:8/696: creat d9/d9d/ffb x:0 0 0 2024-05-06T12:14:39.719 INFO:tasks.workunit.client.0.smithi092.stdout:8/697: chown d9/d21/f3f 202840389 1 2024-05-06T12:14:39.721 INFO:tasks.workunit.client.0.smithi092.stdout:8/698: read d9/d21/d37/d46/f5e [1969565,32017] 0 2024-05-06T12:14:39.725 INFO:tasks.workunit.client.0.smithi092.stdout:3/533: dwrite d8/d24/d33/d39/f70 [0,4194304] 0 2024-05-06T12:14:39.725 INFO:tasks.workunit.client.0.smithi092.stdout:8/699: mkdir d9/d21/d37/d46/d5c/dfc 0 2024-05-06T12:14:39.726 INFO:tasks.workunit.client.1.smithi190.stdout:0/157: sync 2024-05-06T12:14:39.727 INFO:tasks.workunit.client.1.smithi190.stdout:0/158: read d9/dc/f1f [879496,37705] 0 2024-05-06T12:14:39.729 INFO:tasks.workunit.client.0.smithi092.stdout:3/534: symlink d8/db/d4c/d40/d67/la6 0 2024-05-06T12:14:39.730 INFO:tasks.workunit.client.1.smithi190.stdout:0/159: creat d9/df/d21/d31/f34 x:0 0 0 2024-05-06T12:14:39.732 INFO:tasks.workunit.client.0.smithi092.stdout:3/535: mknod d8/db/d4c/d3d/d49/ca7 0 2024-05-06T12:14:39.734 INFO:tasks.workunit.client.0.smithi092.stdout:3/536: read d8/d24/d27/fa0 [1527478,105963] 0 2024-05-06T12:14:39.735 INFO:tasks.workunit.client.1.smithi190.stdout:0/160: rename d9/ca to d9/dc/c35 0 2024-05-06T12:14:39.737 INFO:tasks.workunit.client.1.smithi190.stdout:0/161: mknod d9/dc/d28/c36 0 2024-05-06T12:14:39.740 INFO:tasks.workunit.client.0.smithi092.stdout:3/537: dwrite d8/d24/d27/fa0 [4194304,4194304] 0 2024-05-06T12:14:39.740 INFO:tasks.workunit.client.1.smithi190.stdout:0/162: creat d9/f37 x:0 0 0 2024-05-06T12:14:39.742 INFO:tasks.workunit.client.1.smithi190.stdout:0/163: fsync d9/dc/f1f 0 2024-05-06T12:14:39.742 INFO:tasks.workunit.client.0.smithi092.stdout:3/538: dread d8/db/f6c [0,4194304] 0 2024-05-06T12:14:39.747 INFO:tasks.workunit.client.0.smithi092.stdout:3/539: mkdir d8/d24/d33/da8 0 2024-05-06T12:14:39.748 INFO:tasks.workunit.client.0.smithi092.stdout:8/700: dread d9/dc/f4d [0,4194304] 0 2024-05-06T12:14:39.750 INFO:tasks.workunit.client.0.smithi092.stdout:8/701: mkdir d9/d9d/dfd 0 2024-05-06T12:14:39.754 INFO:tasks.workunit.client.0.smithi092.stdout:3/540: dwrite d8/d24/d33/d39/f44 [4194304,4194304] 0 2024-05-06T12:14:39.756 INFO:tasks.workunit.client.0.smithi092.stdout:3/541: rmdir d8/db/d4c/d40/d7a 39 2024-05-06T12:14:39.761 INFO:tasks.workunit.client.0.smithi092.stdout:3/542: creat d8/db/d4c/d3d/fa9 x:0 0 0 2024-05-06T12:14:39.762 INFO:tasks.workunit.client.0.smithi092.stdout:8/702: truncate f0 2751377 0 2024-05-06T12:14:39.768 INFO:tasks.workunit.client.0.smithi092.stdout:8/703: rename d9/d21/d37/d46/f92 to d9/d21/d37/d46/d5c/d62/d7a/ffe 0 2024-05-06T12:14:39.771 INFO:tasks.workunit.client.0.smithi092.stdout:8/704: fdatasync d9/d21/d37/d46/d5c/d62/da5/dec/fed 0 2024-05-06T12:14:39.776 INFO:tasks.workunit.client.1.smithi190.stdout:2/47: sync 2024-05-06T12:14:39.777 INFO:tasks.workunit.client.1.smithi190.stdout:2/48: chown d2/fa 69049 1 2024-05-06T12:14:39.778 INFO:tasks.workunit.client.1.smithi190.stdout:2/49: write d2/fd [139943,73448] 0 2024-05-06T12:14:39.782 INFO:tasks.workunit.client.1.smithi190.stdout:2/50: dread d2/fd [0,4194304] 0 2024-05-06T12:14:39.782 INFO:tasks.workunit.client.0.smithi092.stdout:3/543: dread d8/db/d14/f50 [0,4194304] 0 2024-05-06T12:14:39.782 INFO:tasks.workunit.client.1.smithi190.stdout:2/51: read - d2/fa zero size 2024-05-06T12:14:39.788 INFO:tasks.workunit.client.0.smithi092.stdout:8/705: dwrite d9/d21/d54/f7b [0,4194304] 0 2024-05-06T12:14:39.788 INFO:tasks.workunit.client.0.smithi092.stdout:3/544: write d8/db/d14/d5a/f8f [899224,68718] 0 2024-05-06T12:14:39.789 INFO:tasks.workunit.client.0.smithi092.stdout:3/545: unlink d8/db/d4c/d40/d67/c84 0 2024-05-06T12:14:39.792 INFO:tasks.workunit.client.0.smithi092.stdout:8/706: rename d9/d21/d37/d46/d5c/d71/d75/ccf to d9/d21/d37/dbd/dc3/cff 0 2024-05-06T12:14:39.792 INFO:tasks.workunit.client.0.smithi092.stdout:3/546: creat d8/db/d4c/d3d/d48/faa x:0 0 0 2024-05-06T12:14:39.795 INFO:tasks.workunit.client.0.smithi092.stdout:3/547: mknod d8/db/d4c/d40/cab 0 2024-05-06T12:14:39.797 INFO:tasks.workunit.client.0.smithi092.stdout:3/548: mknod d8/db/d4c/d3d/cac 0 2024-05-06T12:14:39.799 INFO:tasks.workunit.client.0.smithi092.stdout:8/707: write d9/d21/d37/d46/d5c/d62/f68 [5322,42364] 0 2024-05-06T12:14:39.799 INFO:tasks.workunit.client.0.smithi092.stdout:0/842: sync 2024-05-06T12:14:39.799 INFO:tasks.workunit.client.0.smithi092.stdout:9/609: sync 2024-05-06T12:14:39.800 INFO:tasks.workunit.client.0.smithi092.stdout:1/612: sync 2024-05-06T12:14:39.801 INFO:tasks.workunit.client.0.smithi092.stdout:3/549: creat d8/db/fad x:0 0 0 2024-05-06T12:14:39.802 INFO:tasks.workunit.client.0.smithi092.stdout:9/610: dread - d16/d2c/d2f/d3d/fc2 zero size 2024-05-06T12:14:39.804 INFO:tasks.workunit.client.0.smithi092.stdout:9/611: write d16/d1e/d7b/d8d/da4/fb7 [392637,13577] 0 2024-05-06T12:14:39.805 INFO:tasks.workunit.client.0.smithi092.stdout:1/613: mknod d3/d10/d13/d19/cbf 0 2024-05-06T12:14:39.805 INFO:tasks.workunit.client.1.smithi190.stdout:7/18: getdents d0/d1/d2 0 2024-05-06T12:14:39.806 INFO:tasks.workunit.client.1.smithi190.stdout:7/19: dread - d0/d1/f6 zero size 2024-05-06T12:14:39.807 INFO:tasks.workunit.client.0.smithi092.stdout:8/708: getdents d9/d21/d72 0 2024-05-06T12:14:39.807 INFO:tasks.workunit.client.1.smithi190.stdout:1/21: truncate f0 282547 0 2024-05-06T12:14:39.807 INFO:tasks.workunit.client.1.smithi190.stdout:7/20: write d0/d1/d2/f5 [2728171,85206] 0 2024-05-06T12:14:39.808 INFO:tasks.workunit.client.1.smithi190.stdout:7/21: chown d0/d1/d2 25 1 2024-05-06T12:14:39.810 INFO:tasks.workunit.client.1.smithi190.stdout:1/22: unlink l2 0 2024-05-06T12:14:39.810 INFO:tasks.workunit.client.0.smithi092.stdout:9/612: chown d16/d2c/d2f/d75/l99 10 1 2024-05-06T12:14:39.810 INFO:tasks.workunit.client.0.smithi092.stdout:8/709: mknod d9/d21/dc0/c100 0 2024-05-06T12:14:39.811 INFO:tasks.workunit.client.0.smithi092.stdout:3/550: dread d8/db/d14/d5a/f94 [0,4194304] 0 2024-05-06T12:14:39.813 INFO:tasks.workunit.client.1.smithi190.stdout:1/23: dread f3 [0,4194304] 0 2024-05-06T12:14:39.814 INFO:tasks.workunit.client.1.smithi190.stdout:7/22: creat d0/d4/f7 x:0 0 0 2024-05-06T12:14:39.814 INFO:tasks.workunit.client.0.smithi092.stdout:0/843: dwrite d7/dc/d28/d54/d1e/d26/d96/d90/f112 [0,4194304] 0 2024-05-06T12:14:39.815 INFO:tasks.workunit.client.0.smithi092.stdout:3/551: write d8/d1b/f81 [734885,62333] 0 2024-05-06T12:14:39.817 INFO:tasks.workunit.client.0.smithi092.stdout:1/614: getdents d3/d10/d13/d19/d21/d72/d9a 0 2024-05-06T12:14:39.818 INFO:tasks.workunit.client.1.smithi190.stdout:7/23: dread d0/d1/d2/f5 [0,4194304] 0 2024-05-06T12:14:39.819 INFO:tasks.workunit.client.0.smithi092.stdout:1/615: chown d3/d42/d84/d4f/d58/d65/l80 322 1 2024-05-06T12:14:39.821 INFO:tasks.workunit.client.0.smithi092.stdout:8/710: link d9/d21/f6f d9/d21/d37/d46/d5c/d62/da5/de6/f101 0 2024-05-06T12:14:39.821 INFO:tasks.workunit.client.0.smithi092.stdout:0/844: write d7/dc/d28/d54/d1e/d26/d3a/f95 [607402,111241] 0 2024-05-06T12:14:39.822 INFO:tasks.workunit.client.1.smithi190.stdout:7/24: creat d0/d4/f8 x:0 0 0 2024-05-06T12:14:39.824 INFO:tasks.workunit.client.0.smithi092.stdout:8/711: chown d9/dc/fc8 979871957 1 2024-05-06T12:14:39.824 INFO:tasks.workunit.client.1.smithi190.stdout:3/71: getdents d5/d7/d15 0 2024-05-06T12:14:39.824 INFO:tasks.workunit.client.1.smithi190.stdout:3/72: chown d5/d7/db 5193 1 2024-05-06T12:14:39.825 INFO:tasks.workunit.client.0.smithi092.stdout:1/616: mknod d3/d42/d84/d4f/d58/d65/cc0 0 2024-05-06T12:14:39.825 INFO:tasks.workunit.client.0.smithi092.stdout:8/712: mknod d9/d9d/c102 0 2024-05-06T12:14:39.826 INFO:tasks.workunit.client.1.smithi190.stdout:6/18: write f2 [4758966,38593] 0 2024-05-06T12:14:39.826 INFO:tasks.workunit.client.0.smithi092.stdout:1/617: chown d3/d10/d13/d19/d21/fb4 392800 1 2024-05-06T12:14:39.827 INFO:tasks.workunit.client.1.smithi190.stdout:3/73: mknod d5/d14/c1a 0 2024-05-06T12:14:39.827 INFO:tasks.workunit.client.0.smithi092.stdout:1/618: stat d3/d10/d13/d19/d57 0 2024-05-06T12:14:39.830 INFO:tasks.workunit.client.1.smithi190.stdout:6/19: rename l3 to l5 0 2024-05-06T12:14:39.830 INFO:tasks.workunit.client.0.smithi092.stdout:8/713: creat d9/d21/f103 x:0 0 0 2024-05-06T12:14:39.830 INFO:tasks.workunit.client.0.smithi092.stdout:8/714: readlink d9/l5b 0 2024-05-06T12:14:39.831 INFO:tasks.workunit.client.1.smithi190.stdout:6/20: dread f4 [0,4194304] 0 2024-05-06T12:14:39.832 INFO:tasks.workunit.client.1.smithi190.stdout:7/25: dwrite d0/d1/d2/f5 [0,4194304] 0 2024-05-06T12:14:39.832 INFO:tasks.workunit.client.1.smithi190.stdout:3/74: creat d5/d14/d11/f1b x:0 0 0 2024-05-06T12:14:39.832 INFO:tasks.workunit.client.0.smithi092.stdout:1/619: link d3/d10/d13/d19/f87 d3/d10/d13/d19/d21/d88/d8b/fc1 0 2024-05-06T12:14:39.833 INFO:tasks.workunit.client.1.smithi190.stdout:3/75: readlink d5/l18 0 2024-05-06T12:14:39.833 INFO:tasks.workunit.client.1.smithi190.stdout:3/76: chown d5 5717 1 2024-05-06T12:14:39.833 INFO:tasks.workunit.client.1.smithi190.stdout:7/26: fsync d0/d1/f6 0 2024-05-06T12:14:39.834 INFO:tasks.workunit.client.1.smithi190.stdout:7/27: dread - d0/d4/f8 zero size 2024-05-06T12:14:39.834 INFO:tasks.workunit.client.0.smithi092.stdout:8/715: mknod d9/d21/dc0/df2/c104 0 2024-05-06T12:14:39.835 INFO:tasks.workunit.client.1.smithi190.stdout:7/28: chown d0/d4/f7 353 1 2024-05-06T12:14:39.836 INFO:tasks.workunit.client.1.smithi190.stdout:3/77: mknod d5/d7/c1c 0 2024-05-06T12:14:39.838 INFO:tasks.workunit.client.0.smithi092.stdout:8/716: creat d9/d21/dc0/df2/f105 x:0 0 0 2024-05-06T12:14:39.839 INFO:tasks.workunit.client.0.smithi092.stdout:8/717: write d9/d9d/ffb [580737,792] 0 2024-05-06T12:14:39.840 INFO:tasks.workunit.client.1.smithi190.stdout:7/29: link d0/d4/f8 d0/d4/f9 0 2024-05-06T12:14:39.841 INFO:tasks.workunit.client.0.smithi092.stdout:8/718: write d9/dc/d4a/f97 [4424748,109160] 0 2024-05-06T12:14:39.842 INFO:tasks.workunit.client.1.smithi190.stdout:3/78: link d5/ff d5/f1d 0 2024-05-06T12:14:39.843 INFO:tasks.workunit.client.0.smithi092.stdout:3/552: dread f3 [0,4194304] 0 2024-05-06T12:14:39.844 INFO:tasks.workunit.client.1.smithi190.stdout:7/30: unlink d0/d4/f9 0 2024-05-06T12:14:39.845 INFO:tasks.workunit.client.0.smithi092.stdout:3/553: chown d8/d24/d33/d39/la1 2065870215 1 2024-05-06T12:14:39.847 INFO:tasks.workunit.client.0.smithi092.stdout:1/620: truncate d3/d10/d13/d46/f68 2743060 0 2024-05-06T12:14:39.847 INFO:tasks.workunit.client.0.smithi092.stdout:8/719: truncate f0 2640356 0 2024-05-06T12:14:39.848 INFO:tasks.workunit.client.1.smithi190.stdout:7/31: mkdir d0/da 0 2024-05-06T12:14:39.849 INFO:tasks.workunit.client.1.smithi190.stdout:7/32: chown d0/da 187378097 1 2024-05-06T12:14:39.849 INFO:tasks.workunit.client.0.smithi092.stdout:5/565: sync 2024-05-06T12:14:39.849 INFO:tasks.workunit.client.0.smithi092.stdout:4/711: sync 2024-05-06T12:14:39.849 INFO:tasks.workunit.client.0.smithi092.stdout:7/751: sync 2024-05-06T12:14:39.849 INFO:tasks.workunit.client.0.smithi092.stdout:2/467: sync 2024-05-06T12:14:39.849 INFO:tasks.workunit.client.0.smithi092.stdout:6/741: sync 2024-05-06T12:14:39.850 INFO:tasks.workunit.client.0.smithi092.stdout:3/554: mknod d8/d1b/d73/cae 0 2024-05-06T12:14:39.850 INFO:tasks.workunit.client.0.smithi092.stdout:9/613: dread d16/d1c/f2b [0,4194304] 0 2024-05-06T12:14:39.850 INFO:tasks.workunit.client.0.smithi092.stdout:8/720: write d9/d9d/ffb [1623620,123671] 0 2024-05-06T12:14:39.851 INFO:tasks.workunit.client.1.smithi190.stdout:8/136: dwrite d3/d8/f1f [0,4194304] 0 2024-05-06T12:14:39.851 INFO:tasks.workunit.client.1.smithi190.stdout:3/79: dread d5/ff [0,4194304] 0 2024-05-06T12:14:39.851 INFO:tasks.workunit.client.1.smithi190.stdout:9/119: rmdir db/dd/df 39 2024-05-06T12:14:39.853 INFO:tasks.workunit.client.1.smithi190.stdout:7/33: creat d0/d1/fb x:0 0 0 2024-05-06T12:14:39.853 INFO:tasks.workunit.client.1.smithi190.stdout:2/52: getdents d2/d8 0 2024-05-06T12:14:39.853 INFO:tasks.workunit.client.0.smithi092.stdout:1/621: fsync d3/d10/d13/f60 0 2024-05-06T12:14:39.854 INFO:tasks.workunit.client.1.smithi190.stdout:2/53: chown d2/c4 631866 1 2024-05-06T12:14:39.854 INFO:tasks.workunit.client.0.smithi092.stdout:0/845: dread d7/dc/d28/d54/d12/f14 [0,4194304] 0 2024-05-06T12:14:39.854 INFO:tasks.workunit.client.0.smithi092.stdout:3/555: symlink d8/db/d72/laf 0 2024-05-06T12:14:39.857 INFO:tasks.workunit.client.1.smithi190.stdout:7/34: creat d0/d1/fc x:0 0 0 2024-05-06T12:14:39.863 INFO:tasks.workunit.client.0.smithi092.stdout:2/468: truncate df/d17/d1d/d33/d3c/f8b 498386 0 2024-05-06T12:14:39.864 INFO:tasks.workunit.client.1.smithi190.stdout:8/137: creat d3/f2f x:0 0 0 2024-05-06T12:14:39.864 INFO:tasks.workunit.client.0.smithi092.stdout:1/622: write d3/d42/d84/d4f/d58/d65/d81/fbc [249103,92734] 0 2024-05-06T12:14:39.865 INFO:tasks.workunit.client.1.smithi190.stdout:3/80: creat d5/d7/f1e x:0 0 0 2024-05-06T12:14:39.867 INFO:tasks.workunit.client.0.smithi092.stdout:3/556: mkdir d8/db/d14/d5a/db0 0 2024-05-06T12:14:39.868 INFO:tasks.workunit.client.0.smithi092.stdout:9/614: dwrite fc [8388608,4194304] 0 2024-05-06T12:14:39.868 INFO:tasks.workunit.client.1.smithi190.stdout:9/120: dwrite db/dd/df/f1b [4194304,4194304] 0 2024-05-06T12:14:39.869 INFO:tasks.workunit.client.0.smithi092.stdout:5/566: getdents d4/d5/d10/d31/d53/dcb 0 2024-05-06T12:14:39.869 INFO:tasks.workunit.client.1.smithi190.stdout:9/121: fsync db/f2b 0 2024-05-06T12:14:39.869 INFO:tasks.workunit.client.1.smithi190.stdout:3/81: dread - d5/d7/f1e zero size 2024-05-06T12:14:39.870 INFO:tasks.workunit.client.1.smithi190.stdout:7/35: write d0/d4/f8 [691379,102532] 0 2024-05-06T12:14:39.872 INFO:tasks.workunit.client.0.smithi092.stdout:2/469: creat df/d17/d51/f8c x:0 0 0 2024-05-06T12:14:39.872 INFO:tasks.workunit.client.0.smithi092.stdout:3/557: read - d8/db/d4c/d3d/f96 zero size 2024-05-06T12:14:39.872 INFO:tasks.workunit.client.1.smithi190.stdout:9/122: readlink l8 0 2024-05-06T12:14:39.874 INFO:tasks.workunit.client.0.smithi092.stdout:0/846: symlink d7/dc/d28/d54/d1e/d26/d3a/de3/l118 0 2024-05-06T12:14:39.875 INFO:tasks.workunit.client.1.smithi190.stdout:2/54: dwrite d2/fd [0,4194304] 0 2024-05-06T12:14:39.876 INFO:tasks.workunit.client.0.smithi092.stdout:8/721: dwrite d9/dc/f4d [0,4194304] 0 2024-05-06T12:14:39.879 INFO:tasks.workunit.client.1.smithi190.stdout:8/138: rename d3/d8/f1f to d3/d8/f30 0 2024-05-06T12:14:39.880 INFO:tasks.workunit.client.0.smithi092.stdout:3/558: chown d8/d1b/f81 0 1 2024-05-06T12:14:39.883 INFO:tasks.workunit.client.1.smithi190.stdout:8/139: fdatasync d3/d8/f27 0 2024-05-06T12:14:39.884 INFO:tasks.workunit.client.1.smithi190.stdout:7/36: rename d0/d1/f6 to d0/d4/fd 0 2024-05-06T12:14:39.885 INFO:tasks.workunit.client.0.smithi092.stdout:8/722: write d9/d21/dc0/df2/f105 [207419,71516] 0 2024-05-06T12:14:39.888 INFO:tasks.workunit.client.1.smithi190.stdout:2/55: creat d2/fe x:0 0 0 2024-05-06T12:14:39.888 INFO:tasks.workunit.client.0.smithi092.stdout:0/847: mkdir d7/dc/d6f/da5/dc6/d117/d119 0 2024-05-06T12:14:39.889 INFO:tasks.workunit.client.0.smithi092.stdout:9/615: dread d16/d2c/d2f/d3d/d33/f71 [0,4194304] 0 2024-05-06T12:14:39.889 INFO:tasks.workunit.client.0.smithi092.stdout:4/712: dwrite d9/dd/f80 [0,4194304] 0 2024-05-06T12:14:39.890 INFO:tasks.workunit.client.1.smithi190.stdout:2/56: stat l0 0 2024-05-06T12:14:39.890 INFO:tasks.workunit.client.0.smithi092.stdout:5/567: link d4/d34/d28/d4e/d4b/l80 d4/d34/d29/lcc 0 2024-05-06T12:14:39.890 INFO:tasks.workunit.client.1.smithi190.stdout:8/140: fdatasync d3/f26 0 2024-05-06T12:14:39.891 INFO:tasks.workunit.client.1.smithi190.stdout:3/82: dwrite d5/d7/f1e [0,4194304] 0 2024-05-06T12:14:39.892 INFO:tasks.workunit.client.1.smithi190.stdout:2/57: chown d2/fe 1 1 2024-05-06T12:14:39.892 INFO:tasks.workunit.client.1.smithi190.stdout:3/83: dread - d5/d14/d11/f1b zero size 2024-05-06T12:14:39.894 INFO:tasks.workunit.client.1.smithi190.stdout:8/141: fsync d3/f24 0 2024-05-06T12:14:39.895 INFO:tasks.workunit.client.1.smithi190.stdout:8/142: chown d3/d8/c23 449 1 2024-05-06T12:14:39.896 INFO:tasks.workunit.client.0.smithi092.stdout:2/470: dread fd [4194304,4194304] 0 2024-05-06T12:14:39.897 INFO:tasks.workunit.client.0.smithi092.stdout:4/713: stat d9/d2b/d7c/dab/fc6 0 2024-05-06T12:14:39.897 INFO:tasks.workunit.client.0.smithi092.stdout:3/559: getdents d8/d1b/d73 0 2024-05-06T12:14:39.899 INFO:tasks.workunit.client.1.smithi190.stdout:3/84: rename d5/d7/db to d5/d7/d15/d1f 0 2024-05-06T12:14:39.900 INFO:tasks.workunit.client.0.smithi092.stdout:1/623: write d3/d10/d13/d19/f5d [3276469,88008] 0 2024-05-06T12:14:39.900 INFO:tasks.workunit.client.1.smithi190.stdout:3/85: chown d5/f1d 58 1 2024-05-06T12:14:39.902 INFO:tasks.workunit.client.0.smithi092.stdout:8/723: dwrite d9/fc2 [0,4194304] 0 2024-05-06T12:14:39.905 INFO:tasks.workunit.client.0.smithi092.stdout:8/724: write d9/dc/f4d [104776,126652] 0 2024-05-06T12:14:39.906 INFO:tasks.workunit.client.0.smithi092.stdout:9/616: symlink d16/d2c/d2f/lc7 0 2024-05-06T12:14:39.906 INFO:tasks.workunit.client.1.smithi190.stdout:2/58: symlink d2/d8/dc/lf 0 2024-05-06T12:14:39.908 INFO:tasks.workunit.client.1.smithi190.stdout:9/123: link db/d23/c2f db/d23/c30 0 2024-05-06T12:14:39.909 INFO:tasks.workunit.client.0.smithi092.stdout:8/725: dread d9/d21/d37/d46/d5c/d71/fca [0,4194304] 0 2024-05-06T12:14:39.910 INFO:tasks.workunit.client.1.smithi190.stdout:9/124: fdatasync db/f26 0 2024-05-06T12:14:39.910 INFO:tasks.workunit.client.0.smithi092.stdout:0/848: truncate d7/dc/d28/d54/d1e/d26/f92 374652 0 2024-05-06T12:14:39.912 INFO:tasks.workunit.client.1.smithi190.stdout:2/59: mkdir d2/d10 0 2024-05-06T12:14:39.914 INFO:tasks.workunit.client.0.smithi092.stdout:2/471: rename df/d5c/f73 to df/d15/d88/d59/f8d 0 2024-05-06T12:14:39.914 INFO:tasks.workunit.client.1.smithi190.stdout:2/60: write d2/fe [690998,57578] 0 2024-05-06T12:14:39.915 INFO:tasks.workunit.client.1.smithi190.stdout:2/61: fdatasync d2/fa 0 2024-05-06T12:14:39.916 INFO:tasks.workunit.client.0.smithi092.stdout:3/560: write d8/db/d4c/d40/d7a/fa3 [781464,36088] 0 2024-05-06T12:14:39.919 INFO:tasks.workunit.client.1.smithi190.stdout:3/86: creat d5/d7/f20 x:0 0 0 2024-05-06T12:14:39.921 INFO:tasks.workunit.client.0.smithi092.stdout:9/617: dwrite d16/d2c/d3f/f9d [0,4194304] 0 2024-05-06T12:14:39.921 INFO:tasks.workunit.client.1.smithi190.stdout:9/125: mknod db/dd/c31 0 2024-05-06T12:14:39.922 INFO:tasks.workunit.client.0.smithi092.stdout:5/568: rename d4/d34/d28/d4e/d5d/d82/f18 to d4/d5/d10/da1/fcd 0 2024-05-06T12:14:39.923 INFO:tasks.workunit.client.1.smithi190.stdout:3/87: mkdir d5/d14/d11/d21 0 2024-05-06T12:14:39.924 INFO:tasks.workunit.client.1.smithi190.stdout:2/62: mkdir d2/d10/d11 0 2024-05-06T12:14:39.924 INFO:tasks.workunit.client.0.smithi092.stdout:9/618: dread - d16/d2c/d2f/d3d/d33/d57/f59 zero size 2024-05-06T12:14:39.925 INFO:tasks.workunit.client.1.smithi190.stdout:3/88: write d5/d7/f1e [2598229,23073] 0 2024-05-06T12:14:39.928 INFO:tasks.workunit.client.0.smithi092.stdout:4/714: write d9/d2b/d7c/f95 [75949,91353] 0 2024-05-06T12:14:39.930 INFO:tasks.workunit.client.0.smithi092.stdout:2/472: dwrite df/d17/d1d/d33/d3c/f8b [0,4194304] 0 2024-05-06T12:14:39.930 INFO:tasks.workunit.client.0.smithi092.stdout:4/715: chown d9/d2b/d7c/l97 87 1 2024-05-06T12:14:39.932 INFO:tasks.workunit.client.0.smithi092.stdout:4/716: chown d9/f47 103 1 2024-05-06T12:14:39.932 INFO:tasks.workunit.client.1.smithi190.stdout:9/126: dwrite db/f11 [0,4194304] 0 2024-05-06T12:14:39.935 INFO:tasks.workunit.client.1.smithi190.stdout:3/89: unlink d5/d14/c1a 0 2024-05-06T12:14:39.940 INFO:tasks.workunit.client.0.smithi092.stdout:3/561: unlink d8/d24/d33/d39/c6b 0 2024-05-06T12:14:39.941 INFO:tasks.workunit.client.0.smithi092.stdout:4/717: rename d9/d2b/d7c/f95 to d9/dd/da6/dca/fed 0 2024-05-06T12:14:39.941 INFO:tasks.workunit.client.0.smithi092.stdout:3/562: chown d8/d1b/d73/cae 329 1 2024-05-06T12:14:39.942 INFO:tasks.workunit.client.0.smithi092.stdout:4/718: fsync d9/d27/da8/db3/fd2 0 2024-05-06T12:14:39.942 INFO:tasks.workunit.client.0.smithi092.stdout:9/619: mknod d16/d2c/d2f/d3d/d33/d37/cc8 0 2024-05-06T12:14:39.944 INFO:tasks.workunit.client.0.smithi092.stdout:3/563: fsync d8/db/d4c/d3d/f96 0 2024-05-06T12:14:39.944 INFO:tasks.workunit.client.1.smithi190.stdout:3/90: dwrite d5/d7/f20 [0,4194304] 0 2024-05-06T12:14:39.945 INFO:tasks.workunit.client.0.smithi092.stdout:9/620: write d16/d2c/f91 [1006583,12917] 0 2024-05-06T12:14:39.946 INFO:tasks.workunit.client.0.smithi092.stdout:8/726: write d9/dc/ff [2209608,77131] 0 2024-05-06T12:14:39.949 INFO:tasks.workunit.client.0.smithi092.stdout:2/473: mkdir df/d1b/d8e 0 2024-05-06T12:14:39.952 INFO:tasks.workunit.client.0.smithi092.stdout:2/474: read - df/f55 zero size 2024-05-06T12:14:39.955 INFO:tasks.workunit.client.0.smithi092.stdout:0/849: truncate d7/dc/d28/d54/d1e/d46/d63/f72 380071 0 2024-05-06T12:14:39.957 INFO:tasks.workunit.client.1.smithi190.stdout:3/91: dread d5/d14/d11/f13 [0,4194304] 0 2024-05-06T12:14:39.960 INFO:tasks.workunit.client.1.smithi190.stdout:3/92: creat d5/d7/d15/f22 x:0 0 0 2024-05-06T12:14:39.964 INFO:tasks.workunit.client.1.smithi190.stdout:3/93: creat d5/d7/f23 x:0 0 0 2024-05-06T12:14:39.966 INFO:tasks.workunit.client.0.smithi092.stdout:2/475: write df/d17/f22 [1364579,16204] 0 2024-05-06T12:14:39.968 INFO:tasks.workunit.client.0.smithi092.stdout:0/850: symlink d7/dc/d28/d64/l11a 0 2024-05-06T12:14:39.969 INFO:tasks.workunit.client.0.smithi092.stdout:4/719: dwrite d9/d27/da8/db3/fd2 [0,4194304] 0 2024-05-06T12:14:39.970 INFO:tasks.workunit.client.0.smithi092.stdout:9/621: dwrite d16/d1e/f2e [0,4194304] 0 2024-05-06T12:14:39.971 INFO:tasks.workunit.client.0.smithi092.stdout:1/624: dwrite d3/d10/d13/d19/f5a [0,4194304] 0 2024-05-06T12:14:39.971 INFO:tasks.workunit.client.0.smithi092.stdout:8/727: dwrite d9/d21/d37/d46/fe9 [0,4194304] 0 2024-05-06T12:14:39.973 INFO:tasks.workunit.client.1.smithi190.stdout:3/94: dwrite d5/d7/f1e [0,4194304] 0 2024-05-06T12:14:39.974 INFO:tasks.workunit.client.0.smithi092.stdout:1/625: dread - d3/d42/d84/d4f/fa8 zero size 2024-05-06T12:14:39.975 INFO:tasks.workunit.client.1.smithi190.stdout:5/24: truncate f0 2719393 0 2024-05-06T12:14:39.978 INFO:tasks.workunit.client.1.smithi190.stdout:3/95: creat d5/d14/f24 x:0 0 0 2024-05-06T12:14:39.978 INFO:tasks.workunit.client.0.smithi092.stdout:4/720: chown d9/d2b/fc7 243 1 2024-05-06T12:14:39.978 INFO:tasks.workunit.client.0.smithi092.stdout:8/728: read d9/dc/f4d [3472524,91942] 0 2024-05-06T12:14:39.979 INFO:tasks.workunit.client.1.smithi190.stdout:3/96: chown d5/d7/f1e 890871 1 2024-05-06T12:14:39.980 INFO:tasks.workunit.client.1.smithi190.stdout:3/97: chown d5/d14/d11/f1b 76 1 2024-05-06T12:14:39.981 INFO:tasks.workunit.client.0.smithi092.stdout:3/564: write d8/d24/d33/f64 [3404884,33319] 0 2024-05-06T12:14:39.982 INFO:tasks.workunit.client.1.smithi190.stdout:3/98: fdatasync d5/d7/d15/f22 0 2024-05-06T12:14:39.984 INFO:tasks.workunit.client.1.smithi190.stdout:3/99: rename d5/d14/d11 to d5/d14/d11/d21/d25 22 2024-05-06T12:14:39.984 INFO:tasks.workunit.client.1.smithi190.stdout:4/18: link d2/l6 d2/l8 0 2024-05-06T12:14:39.984 INFO:tasks.workunit.client.1.smithi190.stdout:4/19: chown d2 23392339 1 2024-05-06T12:14:39.987 INFO:tasks.workunit.client.1.smithi190.stdout:0/164: write d9/f26 [4077417,44905] 0 2024-05-06T12:14:39.988 INFO:tasks.workunit.client.0.smithi092.stdout:0/851: creat d7/dc/d28/d54/f11b x:0 0 0 2024-05-06T12:14:39.989 INFO:tasks.workunit.client.0.smithi092.stdout:4/721: dwrite d9/dd/f94 [0,4194304] 0 2024-05-06T12:14:39.990 INFO:tasks.workunit.client.1.smithi190.stdout:3/100: creat d5/f26 x:0 0 0 2024-05-06T12:14:39.991 INFO:tasks.workunit.client.1.smithi190.stdout:3/101: readlink d5/d14/l19 0 2024-05-06T12:14:39.991 INFO:tasks.workunit.client.1.smithi190.stdout:3/102: readlink d5/d7/l9 0 2024-05-06T12:14:39.992 INFO:tasks.workunit.client.0.smithi092.stdout:1/626: mknod d3/d10/d13/cc2 0 2024-05-06T12:14:39.993 INFO:tasks.workunit.client.1.smithi190.stdout:1/24: rename f0 to f4 0 2024-05-06T12:14:39.997 INFO:tasks.workunit.client.1.smithi190.stdout:5/25: getdents d2 0 2024-05-06T12:14:39.997 INFO:tasks.workunit.client.0.smithi092.stdout:8/729: mknod d9/d21/d37/d46/d5c/d62/da5/de6/c106 0 2024-05-06T12:14:39.997 INFO:tasks.workunit.client.1.smithi190.stdout:5/26: write d2/f9 [117577,18587] 0 2024-05-06T12:14:39.998 INFO:tasks.workunit.client.1.smithi190.stdout:0/165: rmdir d9/dc 39 2024-05-06T12:14:40.000 INFO:tasks.workunit.client.0.smithi092.stdout:0/852: dread d7/dc/d28/d54/d1e/d46/df3/f108 [0,4194304] 0 2024-05-06T12:14:40.000 INFO:tasks.workunit.client.1.smithi190.stdout:3/103: creat d5/f27 x:0 0 0 2024-05-06T12:14:40.000 INFO:tasks.workunit.client.0.smithi092.stdout:3/565: fdatasync d8/d24/f65 0 2024-05-06T12:14:40.002 INFO:tasks.workunit.client.1.smithi190.stdout:1/25: write f3 [1404515,112315] 0 2024-05-06T12:14:40.002 INFO:tasks.workunit.client.0.smithi092.stdout:1/627: mknod d3/d10/d13/cc3 0 2024-05-06T12:14:40.002 INFO:tasks.workunit.client.1.smithi190.stdout:1/26: chown f3 0 1 2024-05-06T12:14:40.003 INFO:tasks.workunit.client.1.smithi190.stdout:1/27: stat f3 0 2024-05-06T12:14:40.003 INFO:tasks.workunit.client.1.smithi190.stdout:1/28: read f3 [3978674,74024] 0 2024-05-06T12:14:40.004 INFO:tasks.workunit.client.1.smithi190.stdout:1/29: write f3 [4453634,63410] 0 2024-05-06T12:14:40.007 INFO:tasks.workunit.client.0.smithi092.stdout:1/628: chown d3/lb 17469357 1 2024-05-06T12:14:40.007 INFO:tasks.workunit.client.0.smithi092.stdout:9/622: getdents d16/d2c/d4b/d76 0 2024-05-06T12:14:40.008 INFO:tasks.workunit.client.1.smithi190.stdout:5/27: mkdir d2/da 0 2024-05-06T12:14:40.010 INFO:tasks.workunit.client.1.smithi190.stdout:6/21: readlink l5 0 2024-05-06T12:14:40.011 INFO:tasks.workunit.client.1.smithi190.stdout:3/104: creat d5/d7/d15/f28 x:0 0 0 2024-05-06T12:14:40.012 INFO:tasks.workunit.client.1.smithi190.stdout:3/105: chown d5/d14/d11/f13 1122 1 2024-05-06T12:14:40.014 INFO:tasks.workunit.client.1.smithi190.stdout:1/30: mkdir d5 0 2024-05-06T12:14:40.015 INFO:tasks.workunit.client.1.smithi190.stdout:4/20: link d2/l8 d2/l9 0 2024-05-06T12:14:40.017 INFO:tasks.workunit.client.1.smithi190.stdout:0/166: write d9/dc/fe [2413472,2726] 0 2024-05-06T12:14:40.017 INFO:tasks.workunit.client.1.smithi190.stdout:5/28: mknod d2/cb 0 2024-05-06T12:14:40.018 INFO:tasks.workunit.client.1.smithi190.stdout:0/167: fsync d9/dc/f13 0 2024-05-06T12:14:40.019 INFO:tasks.workunit.client.1.smithi190.stdout:0/168: dread d9/dc/f1f [0,4194304] 0 2024-05-06T12:14:40.020 INFO:tasks.workunit.client.1.smithi190.stdout:6/22: rename l5 to l6 0 2024-05-06T12:14:40.026 INFO:tasks.workunit.client.1.smithi190.stdout:5/29: creat d2/fc x:0 0 0 2024-05-06T12:14:40.027 INFO:tasks.workunit.client.1.smithi190.stdout:5/30: rename d2 to d2/dd 22 2024-05-06T12:14:40.027 INFO:tasks.workunit.client.1.smithi190.stdout:5/31: stat d2 0 2024-05-06T12:14:40.028 INFO:tasks.workunit.client.1.smithi190.stdout:3/106: symlink d5/d7/l29 0 2024-05-06T12:14:40.029 INFO:tasks.workunit.client.1.smithi190.stdout:5/32: dread - d2/f5 zero size 2024-05-06T12:14:40.030 INFO:tasks.workunit.client.1.smithi190.stdout:6/23: rename f4 to f7 0 2024-05-06T12:14:40.032 INFO:tasks.workunit.client.1.smithi190.stdout:6/24: truncate f1 469765 0 2024-05-06T12:14:40.032 INFO:tasks.workunit.client.1.smithi190.stdout:1/31: creat d5/f6 x:0 0 0 2024-05-06T12:14:40.033 INFO:tasks.workunit.client.1.smithi190.stdout:4/21: dwrite d2/f4 [0,4194304] 0 2024-05-06T12:14:40.034 INFO:tasks.workunit.client.1.smithi190.stdout:6/25: dread f1 [0,4194304] 0 2024-05-06T12:14:40.036 INFO:tasks.workunit.client.1.smithi190.stdout:3/107: mkdir d5/d14/d11/d2a 0 2024-05-06T12:14:40.040 INFO:tasks.workunit.client.1.smithi190.stdout:8/143: getdents d3/d8 0 2024-05-06T12:14:40.043 INFO:tasks.workunit.client.1.smithi190.stdout:4/22: rmdir d2 39 2024-05-06T12:14:40.046 INFO:tasks.workunit.client.1.smithi190.stdout:1/32: rename f3 to d5/f7 0 2024-05-06T12:14:40.048 INFO:tasks.workunit.client.1.smithi190.stdout:6/26: symlink l8 0 2024-05-06T12:14:40.049 INFO:tasks.workunit.client.1.smithi190.stdout:3/108: mknod d5/c2b 0 2024-05-06T12:14:40.052 INFO:tasks.workunit.client.1.smithi190.stdout:5/33: mkdir d2/da/de 0 2024-05-06T12:14:40.058 INFO:tasks.workunit.client.1.smithi190.stdout:3/109: unlink d5/d14/f17 0 2024-05-06T12:14:40.060 INFO:tasks.workunit.client.1.smithi190.stdout:5/34: symlink d2/da/lf 0 2024-05-06T12:14:40.060 INFO:tasks.workunit.client.1.smithi190.stdout:5/35: chown d2 135156838 1 2024-05-06T12:14:40.060 INFO:tasks.workunit.client.1.smithi190.stdout:5/36: chown d2/l6 500235369 1 2024-05-06T12:14:40.061 INFO:tasks.workunit.client.1.smithi190.stdout:4/23: mknod d2/ca 0 2024-05-06T12:14:40.064 INFO:tasks.workunit.client.1.smithi190.stdout:3/110: write d5/d14/d11/f13 [429933,41316] 0 2024-05-06T12:14:40.067 INFO:tasks.workunit.client.1.smithi190.stdout:2/63: truncate d2/fd 2979720 0 2024-05-06T12:14:40.070 INFO:tasks.workunit.client.1.smithi190.stdout:4/24: mknod d2/cb 0 2024-05-06T12:14:40.070 INFO:tasks.workunit.client.1.smithi190.stdout:5/37: dwrite d2/f9 [0,4194304] 0 2024-05-06T12:14:40.070 INFO:tasks.workunit.client.1.smithi190.stdout:8/144: link d3/fc d3/d8/d25/f31 0 2024-05-06T12:14:40.071 INFO:tasks.workunit.client.1.smithi190.stdout:4/25: fdatasync d2/f4 0 2024-05-06T12:14:40.078 INFO:tasks.workunit.client.1.smithi190.stdout:2/64: dread d2/fe [0,4194304] 0 2024-05-06T12:14:40.080 INFO:tasks.workunit.client.0.smithi092.stdout:2/476: write fe [1622145,14318] 0 2024-05-06T12:14:40.080 INFO:tasks.workunit.client.0.smithi092.stdout:1/629: read d3/f70 [1040,73558] 0 2024-05-06T12:14:40.080 INFO:tasks.workunit.client.0.smithi092.stdout:4/722: dwrite d9/d2b/d7c/dab/fc6 [0,4194304] 0 2024-05-06T12:14:40.081 INFO:tasks.workunit.client.1.smithi190.stdout:3/111: dwrite d5/d14/d11/f1b [0,4194304] 0 2024-05-06T12:14:40.081 INFO:tasks.workunit.client.1.smithi190.stdout:2/65: read d2/fe [741812,33412] 0 2024-05-06T12:14:40.082 INFO:tasks.workunit.client.0.smithi092.stdout:8/730: truncate d9/d21/d37/d46/d5c/d62/d7a/ffe 1950625 0 2024-05-06T12:14:40.082 INFO:tasks.workunit.client.0.smithi092.stdout:2/477: fdatasync df/f7b 0 2024-05-06T12:14:40.085 INFO:tasks.workunit.client.1.smithi190.stdout:4/26: dwrite d2/f4 [0,4194304] 0 2024-05-06T12:14:40.091 INFO:tasks.workunit.client.0.smithi092.stdout:0/853: dwrite d7/dc/d28/d54/d12/d104/f105 [0,4194304] 0 2024-05-06T12:14:40.097 INFO:tasks.workunit.client.1.smithi190.stdout:3/112: rmdir d5/d7/d15 39 2024-05-06T12:14:40.099 INFO:tasks.workunit.client.1.smithi190.stdout:3/113: dread - d5/f26 zero size 2024-05-06T12:14:40.099 INFO:tasks.workunit.client.0.smithi092.stdout:9/623: mknod d16/d2c/d2f/d3d/d33/d6b/d9e/cc9 0 2024-05-06T12:14:40.101 INFO:tasks.workunit.client.1.smithi190.stdout:8/145: dwrite d3/f12 [0,4194304] 0 2024-05-06T12:14:40.101 INFO:tasks.workunit.client.0.smithi092.stdout:8/731: chown d9/dc/f9e 80 1 2024-05-06T12:14:40.102 INFO:tasks.workunit.client.0.smithi092.stdout:2/478: write df/d15/d88/f82 [2587402,59125] 0 2024-05-06T12:14:40.102 INFO:tasks.workunit.client.0.smithi092.stdout:0/854: creat d7/dc/d28/d54/d1e/d26/d3a/f11c x:0 0 0 2024-05-06T12:14:40.105 INFO:tasks.workunit.client.0.smithi092.stdout:8/732: chown d9/d21/d54/d79/cc9 21048 1 2024-05-06T12:14:40.106 INFO:tasks.workunit.client.0.smithi092.stdout:9/624: write d16/d2c/d2f/d3d/d33/d6b/f81 [455240,92419] 0 2024-05-06T12:14:40.107 INFO:tasks.workunit.client.0.smithi092.stdout:8/733: rmdir d9/d21/d37 39 2024-05-06T12:14:40.109 INFO:tasks.workunit.client.0.smithi092.stdout:0/855: mkdir d7/dc/d28/d54/d12/d7f/db2/d11d 0 2024-05-06T12:14:40.109 INFO:tasks.workunit.client.1.smithi190.stdout:8/146: chown d3/c9 13731694 1 2024-05-06T12:14:40.113 INFO:tasks.workunit.client.1.smithi190.stdout:0/169: truncate d9/df/f15 936110 0 2024-05-06T12:14:40.114 INFO:tasks.workunit.client.0.smithi092.stdout:2/479: getdents df/d5c 0 2024-05-06T12:14:40.115 INFO:tasks.workunit.client.0.smithi092.stdout:8/734: mkdir d9/d21/d37/dbd/dc3/d107 0 2024-05-06T12:14:40.116 INFO:tasks.workunit.client.0.smithi092.stdout:9/625: mknod d16/d1e/da1/d78/db4/cca 0 2024-05-06T12:14:40.119 INFO:tasks.workunit.client.1.smithi190.stdout:0/170: readlink d9/df/l32 0 2024-05-06T12:14:40.119 INFO:tasks.workunit.client.1.smithi190.stdout:3/114: dwrite d5/d7/d15/f16 [0,4194304] 0 2024-05-06T12:14:40.119 INFO:tasks.workunit.client.0.smithi092.stdout:0/856: dwrite d7/dc/d9e/f40 [4194304,4194304] 0 2024-05-06T12:14:40.121 INFO:tasks.workunit.client.0.smithi092.stdout:8/735: symlink d9/d21/l108 0 2024-05-06T12:14:40.123 INFO:tasks.workunit.client.0.smithi092.stdout:9/626: rename d16/d94/da0/lb2 to d16/d1e/da1/d43/lcb 0 2024-05-06T12:14:40.125 INFO:tasks.workunit.client.0.smithi092.stdout:0/857: dread d7/dc/d28/d54/d1e/d26/f92 [0,4194304] 0 2024-05-06T12:14:40.127 INFO:tasks.workunit.client.0.smithi092.stdout:8/736: creat d9/d21/d37/d74/f109 x:0 0 0 2024-05-06T12:14:40.127 INFO:tasks.workunit.client.1.smithi190.stdout:4/27: dwrite d2/f4 [4194304,4194304] 0 2024-05-06T12:14:40.128 INFO:tasks.workunit.client.1.smithi190.stdout:8/147: dwrite d3/f2e [0,4194304] 0 2024-05-06T12:14:40.128 INFO:tasks.workunit.client.0.smithi092.stdout:8/737: write d9/dc/d4a/f97 [4016379,1071] 0 2024-05-06T12:14:40.130 INFO:tasks.workunit.client.0.smithi092.stdout:9/627: mknod d16/d2c/d2f/d3d/d33/ccc 0 2024-05-06T12:14:40.135 INFO:tasks.workunit.client.1.smithi190.stdout:5/38: dread f0 [0,4194304] 0 2024-05-06T12:14:40.135 INFO:tasks.workunit.client.1.smithi190.stdout:1/33: chown f4 414 1 2024-05-06T12:14:40.138 INFO:tasks.workunit.client.1.smithi190.stdout:2/66: write d2/fd [357058,12028] 0 2024-05-06T12:14:40.138 INFO:tasks.workunit.client.1.smithi190.stdout:5/39: dread - d2/f5 zero size 2024-05-06T12:14:40.138 INFO:tasks.workunit.client.1.smithi190.stdout:1/34: dread - d5/f6 zero size 2024-05-06T12:14:40.140 INFO:tasks.workunit.client.1.smithi190.stdout:4/28: dread d2/f4 [0,4194304] 0 2024-05-06T12:14:40.140 INFO:tasks.workunit.client.0.smithi092.stdout:8/738: dread d9/d21/d37/d46/fe9 [0,4194304] 0 2024-05-06T12:14:40.141 INFO:tasks.workunit.client.1.smithi190.stdout:8/148: readlink d3/l20 0 2024-05-06T12:14:40.142 INFO:tasks.workunit.client.1.smithi190.stdout:4/29: chown d2/f4 988798 1 2024-05-06T12:14:40.142 INFO:tasks.workunit.client.1.smithi190.stdout:5/40: write d2/f9 [3732699,98185] 0 2024-05-06T12:14:40.145 INFO:tasks.workunit.client.1.smithi190.stdout:3/115: creat d5/d7/f2c x:0 0 0 2024-05-06T12:14:40.146 INFO:tasks.workunit.client.1.smithi190.stdout:2/67: creat d2/f12 x:0 0 0 2024-05-06T12:14:40.146 INFO:tasks.workunit.client.1.smithi190.stdout:1/35: rmdir d5 39 2024-05-06T12:14:40.148 INFO:tasks.workunit.client.1.smithi190.stdout:6/27: dwrite f7 [0,4194304] 0 2024-05-06T12:14:40.148 INFO:tasks.workunit.client.1.smithi190.stdout:0/171: link l8 d9/df/d21/l38 0 2024-05-06T12:14:40.151 INFO:tasks.workunit.client.0.smithi092.stdout:8/739: getdents d9/d21/d37/d46/d5c/d62 0 2024-05-06T12:14:40.151 INFO:tasks.workunit.client.1.smithi190.stdout:6/28: chown l8 1052464044 1 2024-05-06T12:14:40.152 INFO:tasks.workunit.client.0.smithi092.stdout:7/752: sync 2024-05-06T12:14:40.154 INFO:tasks.workunit.client.0.smithi092.stdout:2/480: write df/f79 [2176013,89213] 0 2024-05-06T12:14:40.156 INFO:tasks.workunit.client.0.smithi092.stdout:4/723: dwrite d9/d2b/f7d [0,4194304] 0 2024-05-06T12:14:40.157 INFO:tasks.workunit.client.1.smithi190.stdout:6/29: write f2 [3809853,8702] 0 2024-05-06T12:14:40.157 INFO:tasks.workunit.client.1.smithi190.stdout:0/172: unlink d9/f1d 0 2024-05-06T12:14:40.158 INFO:tasks.workunit.client.0.smithi092.stdout:8/740: symlink d9/d21/d37/d74/df9/l10a 0 2024-05-06T12:14:40.158 INFO:tasks.workunit.client.1.smithi190.stdout:5/41: rename d2/c4 to d2/da/de/c10 0 2024-05-06T12:14:40.159 INFO:tasks.workunit.client.0.smithi092.stdout:7/753: symlink d9/dd/lfa 0 2024-05-06T12:14:40.159 INFO:tasks.workunit.client.0.smithi092.stdout:1/630: dwrite d3/d10/d13/d19/d34/f36 [0,4194304] 0 2024-05-06T12:14:40.160 INFO:tasks.workunit.client.1.smithi190.stdout:8/149: link d3/l20 d3/d8/d2b/l32 0 2024-05-06T12:14:40.161 INFO:tasks.workunit.client.0.smithi092.stdout:4/724: read - d9/d2b/d65/fc1 zero size 2024-05-06T12:14:40.161 INFO:tasks.workunit.client.1.smithi190.stdout:5/42: fsync d2/fc 0 2024-05-06T12:14:40.162 INFO:tasks.workunit.client.0.smithi092.stdout:2/481: mknod df/d5c/d60/c8f 0 2024-05-06T12:14:40.162 INFO:tasks.workunit.client.0.smithi092.stdout:2/482: chown df 6 1 2024-05-06T12:14:40.165 INFO:tasks.workunit.client.0.smithi092.stdout:7/754: symlink d9/de/d1f/dcf/lfb 0 2024-05-06T12:14:40.165 INFO:tasks.workunit.client.0.smithi092.stdout:1/631: rmdir d3/d42/d84/d4f/d7f 39 2024-05-06T12:14:40.165 INFO:tasks.workunit.client.1.smithi190.stdout:4/30: dread d2/f4 [0,4194304] 0 2024-05-06T12:14:40.165 INFO:tasks.workunit.client.1.smithi190.stdout:3/116: mknod d5/d14/d11/d21/c2d 0 2024-05-06T12:14:40.167 INFO:tasks.workunit.client.0.smithi092.stdout:2/483: write df/d17/d1d/d33/f44 [2082830,118126] 0 2024-05-06T12:14:40.169 INFO:tasks.workunit.client.0.smithi092.stdout:2/484: read df/f79 [1357220,78479] 0 2024-05-06T12:14:40.169 INFO:tasks.workunit.client.0.smithi092.stdout:4/725: unlink d9/f47 0 2024-05-06T12:14:40.170 INFO:tasks.workunit.client.0.smithi092.stdout:8/741: mkdir d9/d5a/dd9/d10b 0 2024-05-06T12:14:40.170 INFO:tasks.workunit.client.1.smithi190.stdout:0/173: write d9/dc/f13 [4762612,116330] 0 2024-05-06T12:14:40.172 INFO:tasks.workunit.client.1.smithi190.stdout:1/36: dwrite d5/f6 [0,4194304] 0 2024-05-06T12:14:40.172 INFO:tasks.workunit.client.1.smithi190.stdout:8/150: stat d3/l1e 0 2024-05-06T12:14:40.175 INFO:tasks.workunit.client.1.smithi190.stdout:2/68: dwrite d2/f12 [0,4194304] 0 2024-05-06T12:14:40.175 INFO:tasks.workunit.client.1.smithi190.stdout:0/174: chown d9/df/d21/d31 15580 1 2024-05-06T12:14:40.175 INFO:tasks.workunit.client.1.smithi190.stdout:4/31: symlink d2/lc 0 2024-05-06T12:14:40.176 INFO:tasks.workunit.client.0.smithi092.stdout:0/858: write d7/dc/d28/d54/d1e/d26/d96/fba [610186,16071] 0 2024-05-06T12:14:40.176 INFO:tasks.workunit.client.1.smithi190.stdout:2/69: chown d2/l3 48878010 1 2024-05-06T12:14:40.177 INFO:tasks.workunit.client.1.smithi190.stdout:0/175: read d9/f1b [2080551,77325] 0 2024-05-06T12:14:40.177 INFO:tasks.workunit.client.0.smithi092.stdout:2/485: symlink df/d15/d88/d59/l90 0 2024-05-06T12:14:40.177 INFO:tasks.workunit.client.1.smithi190.stdout:5/43: creat d2/da/de/f11 x:0 0 0 2024-05-06T12:14:40.178 INFO:tasks.workunit.client.0.smithi092.stdout:8/742: truncate d9/f36 2460715 0 2024-05-06T12:14:40.178 INFO:tasks.workunit.client.0.smithi092.stdout:2/486: stat df/d17/d1d/d33/d3c 0 2024-05-06T12:14:40.181 INFO:tasks.workunit.client.1.smithi190.stdout:5/44: truncate d2/da/de/f11 592115 0 2024-05-06T12:14:40.184 INFO:tasks.workunit.client.1.smithi190.stdout:3/117: creat d5/d14/d11/d21/f2e x:0 0 0 2024-05-06T12:14:40.186 INFO:tasks.workunit.client.1.smithi190.stdout:0/176: dread d9/dc/fe [0,4194304] 0 2024-05-06T12:14:40.187 INFO:tasks.workunit.client.1.smithi190.stdout:4/32: chown d2/l8 382 1 2024-05-06T12:14:40.189 INFO:tasks.workunit.client.0.smithi092.stdout:6/742: sync 2024-05-06T12:14:40.189 INFO:tasks.workunit.client.0.smithi092.stdout:5/569: sync 2024-05-06T12:14:40.190 INFO:tasks.workunit.client.1.smithi190.stdout:6/30: write f1 [945133,8229] 0 2024-05-06T12:14:40.191 INFO:tasks.workunit.client.1.smithi190.stdout:6/31: fsync f7 0 2024-05-06T12:14:40.191 INFO:tasks.workunit.client.1.smithi190.stdout:5/45: mknod d2/c12 0 2024-05-06T12:14:40.192 INFO:tasks.workunit.client.1.smithi190.stdout:5/46: chown d2/l6 13201 1 2024-05-06T12:14:40.192 INFO:tasks.workunit.client.0.smithi092.stdout:9/628: truncate d16/d2c/f91 318197 0 2024-05-06T12:14:40.192 INFO:tasks.workunit.client.1.smithi190.stdout:5/47: chown d2/l6 1443637 1 2024-05-06T12:14:40.193 INFO:tasks.workunit.client.0.smithi092.stdout:9/629: dread - d16/d2c/d2f/d3d/d33/d6b/fa2 zero size 2024-05-06T12:14:40.194 INFO:tasks.workunit.client.1.smithi190.stdout:3/118: creat d5/d7/d15/f2f x:0 0 0 2024-05-06T12:14:40.195 INFO:tasks.workunit.client.0.smithi092.stdout:9/630: fdatasync d16/d2c/d2f/d3d/d33/d6b/f81 0 2024-05-06T12:14:40.195 INFO:tasks.workunit.client.0.smithi092.stdout:1/632: link d3/d10/d13/d46/f55 d3/d10/db3/fc4 0 2024-05-06T12:14:40.196 INFO:tasks.workunit.client.1.smithi190.stdout:4/33: write d2/f4 [333960,14176] 0 2024-05-06T12:14:40.197 INFO:tasks.workunit.client.1.smithi190.stdout:0/177: mknod d9/df/d21/d31/c39 0 2024-05-06T12:14:40.198 INFO:tasks.workunit.client.1.smithi190.stdout:1/37: write f4 [689766,74210] 0 2024-05-06T12:14:40.198 INFO:tasks.workunit.client.0.smithi092.stdout:0/859: creat d7/dc/d28/d54/d12/d104/f11e x:0 0 0 2024-05-06T12:14:40.198 INFO:tasks.workunit.client.0.smithi092.stdout:2/487: rmdir df/d17 39 2024-05-06T12:14:40.200 INFO:tasks.workunit.client.0.smithi092.stdout:0/860: chown d7/dc/d28/d54/d1e/d46/df0 861636 1 2024-05-06T12:14:40.201 INFO:tasks.workunit.client.0.smithi092.stdout:8/743: truncate d9/d21/d37/dbd/fd5 1135437 0 2024-05-06T12:14:40.203 INFO:tasks.workunit.client.1.smithi190.stdout:3/119: creat d5/d14/d11/d21/f30 x:0 0 0 2024-05-06T12:14:40.204 INFO:tasks.workunit.client.1.smithi190.stdout:4/34: mkdir d2/dd 0 2024-05-06T12:14:40.204 INFO:tasks.workunit.client.1.smithi190.stdout:3/120: dread - d5/d7/d15/f28 zero size 2024-05-06T12:14:40.207 INFO:tasks.workunit.client.0.smithi092.stdout:9/631: dwrite d16/d2c/d3f/f8b [0,4194304] 0 2024-05-06T12:14:40.207 INFO:tasks.workunit.client.1.smithi190.stdout:0/178: write d9/f2c [2976802,123884] 0 2024-05-06T12:14:40.208 INFO:tasks.workunit.client.1.smithi190.stdout:1/38: creat d5/f8 x:0 0 0 2024-05-06T12:14:40.209 INFO:tasks.workunit.client.1.smithi190.stdout:5/48: link d2/f8 d2/f13 0 2024-05-06T12:14:40.211 INFO:tasks.workunit.client.1.smithi190.stdout:8/151: write d3/fc [1200699,33679] 0 2024-05-06T12:14:40.211 INFO:tasks.workunit.client.1.smithi190.stdout:5/49: fsync d2/f5 0 2024-05-06T12:14:40.213 INFO:tasks.workunit.client.0.smithi092.stdout:2/488: unlink df/f6d 0 2024-05-06T12:14:40.213 INFO:tasks.workunit.client.0.smithi092.stdout:7/755: dread d9/de/d10/d17/d99/db6/df1/f73 [0,4194304] 0 2024-05-06T12:14:40.213 INFO:tasks.workunit.client.1.smithi190.stdout:2/70: truncate d2/fd 714311 0 2024-05-06T12:14:40.214 INFO:tasks.workunit.client.0.smithi092.stdout:9/632: write d16/d2c/d3f/f96 [2160121,68701] 0 2024-05-06T12:14:40.215 INFO:tasks.workunit.client.1.smithi190.stdout:5/50: dread - d2/f5 zero size 2024-05-06T12:14:40.216 INFO:tasks.workunit.client.0.smithi092.stdout:9/633: readlink l15 0 2024-05-06T12:14:40.216 INFO:tasks.workunit.client.0.smithi092.stdout:5/570: dwrite d4/d34/d28/f72 [0,4194304] 0 2024-05-06T12:14:40.217 INFO:tasks.workunit.client.0.smithi092.stdout:9/634: write fc [5673927,108715] 0 2024-05-06T12:14:40.217 INFO:tasks.workunit.client.0.smithi092.stdout:8/744: creat d9/d21/d37/d46/d5c/d71/d75/f10c x:0 0 0 2024-05-06T12:14:40.219 INFO:tasks.workunit.client.1.smithi190.stdout:5/51: readlink d2/l7 0 2024-05-06T12:14:40.220 INFO:tasks.workunit.client.1.smithi190.stdout:2/71: chown d2/d8/dc/lf 31 1 2024-05-06T12:14:40.221 INFO:tasks.workunit.client.0.smithi092.stdout:1/633: mknod d3/d10/d13/db0/cc5 0 2024-05-06T12:14:40.221 INFO:tasks.workunit.client.1.smithi190.stdout:7/37: sync 2024-05-06T12:14:40.221 INFO:tasks.workunit.client.1.smithi190.stdout:9/127: sync 2024-05-06T12:14:40.221 INFO:tasks.workunit.client.1.smithi190.stdout:4/35: dwrite d2/f4 [4194304,4194304] 0 2024-05-06T12:14:40.224 INFO:tasks.workunit.client.1.smithi190.stdout:3/121: dwrite d5/d7/d15/d1f/fe [0,4194304] 0 2024-05-06T12:14:40.227 INFO:tasks.workunit.client.0.smithi092.stdout:7/756: dwrite d9/de/d1f/d37/fa0 [0,4194304] 0 2024-05-06T12:14:40.227 INFO:tasks.workunit.client.1.smithi190.stdout:6/32: dread f1 [0,4194304] 0 2024-05-06T12:14:40.231 INFO:tasks.workunit.client.1.smithi190.stdout:5/52: readlink d2/da/lf 0 2024-05-06T12:14:40.231 INFO:tasks.workunit.client.1.smithi190.stdout:9/128: chown db/dd/df/l28 319125 1 2024-05-06T12:14:40.238 INFO:tasks.workunit.client.1.smithi190.stdout:6/33: symlink l9 0 2024-05-06T12:14:40.240 INFO:tasks.workunit.client.1.smithi190.stdout:0/179: dread d9/fb [0,4194304] 0 2024-05-06T12:14:40.241 INFO:tasks.workunit.client.0.smithi092.stdout:7/757: unlink d9/de/d10/d17/d21/d2e/d9c/fbe 0 2024-05-06T12:14:40.245 INFO:tasks.workunit.client.0.smithi092.stdout:6/743: dwrite d4/d24/d66/f7e [0,4194304] 0 2024-05-06T12:14:40.245 INFO:tasks.workunit.client.1.smithi190.stdout:2/72: creat d2/d8/dc/f13 x:0 0 0 2024-05-06T12:14:40.246 INFO:tasks.workunit.client.1.smithi190.stdout:9/129: chown db/d23/c2e 323902035 1 2024-05-06T12:14:40.247 INFO:tasks.workunit.client.0.smithi092.stdout:8/745: mknod d9/c10d 0 2024-05-06T12:14:40.247 INFO:tasks.workunit.client.1.smithi190.stdout:5/53: fsync f0 0 2024-05-06T12:14:40.248 INFO:tasks.workunit.client.0.smithi092.stdout:1/634: mknod d3/d10/d13/d19/d21/cc6 0 2024-05-06T12:14:40.248 INFO:tasks.workunit.client.0.smithi092.stdout:0/861: dread d7/dc/d28/d54/d12/f1b [0,4194304] 0 2024-05-06T12:14:40.251 INFO:tasks.workunit.client.1.smithi190.stdout:0/180: write d9/f2f [446280,15344] 0 2024-05-06T12:14:40.255 INFO:tasks.workunit.client.1.smithi190.stdout:3/122: dwrite d5/d7/d15/f28 [0,4194304] 0 2024-05-06T12:14:40.255 INFO:tasks.workunit.client.0.smithi092.stdout:6/744: chown d4/d20/d22/l5a 637248 1 2024-05-06T12:14:40.257 INFO:tasks.workunit.client.1.smithi190.stdout:1/39: dwrite d5/f7 [0,4194304] 0 2024-05-06T12:14:40.259 INFO:tasks.workunit.client.0.smithi092.stdout:0/862: dread - d7/dc/d9e/f8c zero size 2024-05-06T12:14:40.261 INFO:tasks.workunit.client.1.smithi190.stdout:8/152: dwrite d3/d8/f1b [0,4194304] 0 2024-05-06T12:14:40.261 INFO:tasks.workunit.client.0.smithi092.stdout:9/635: dread d16/d1e/d7b/fb0 [0,4194304] 0 2024-05-06T12:14:40.262 INFO:tasks.workunit.client.0.smithi092.stdout:7/758: symlink d9/de/d1f/d37/d48/d93/lfc 0 2024-05-06T12:14:40.264 INFO:tasks.workunit.client.0.smithi092.stdout:0/863: unlink f1 0 2024-05-06T12:14:40.265 INFO:tasks.workunit.client.1.smithi190.stdout:5/54: rmdir d2/da 39 2024-05-06T12:14:40.265 INFO:tasks.workunit.client.0.smithi092.stdout:7/759: write d9/de/d10/d17/d21/d38/d77/fe5 [226391,28948] 0 2024-05-06T12:14:40.266 INFO:tasks.workunit.client.1.smithi190.stdout:4/36: dread d2/f4 [4194304,4194304] 0 2024-05-06T12:14:40.267 INFO:tasks.workunit.client.0.smithi092.stdout:9/636: write d16/d1e/da1/d43/f9b [1114937,62369] 0 2024-05-06T12:14:40.268 INFO:tasks.workunit.client.1.smithi190.stdout:8/153: chown d3/d13/l19 0 1 2024-05-06T12:14:40.269 INFO:tasks.workunit.client.0.smithi092.stdout:0/864: creat d7/dc/d28/d54/d1e/d39/f11f x:0 0 0 2024-05-06T12:14:40.270 INFO:tasks.workunit.client.1.smithi190.stdout:7/38: dwrite d0/d1/fb [0,4194304] 0 2024-05-06T12:14:40.270 INFO:tasks.workunit.client.1.smithi190.stdout:0/181: symlink d9/l3a 0 2024-05-06T12:14:40.270 INFO:tasks.workunit.client.1.smithi190.stdout:9/130: dread db/dd/f17 [0,4194304] 0 2024-05-06T12:14:40.273 INFO:tasks.workunit.client.0.smithi092.stdout:9/637: mknod d16/d2c/d4b/ccd 0 2024-05-06T12:14:40.274 INFO:tasks.workunit.client.1.smithi190.stdout:8/154: chown d3/d13/c28 7 1 2024-05-06T12:14:40.275 INFO:tasks.workunit.client.1.smithi190.stdout:6/34: getdents . 0 2024-05-06T12:14:40.275 INFO:tasks.workunit.client.0.smithi092.stdout:0/865: unlink d7/l60 0 2024-05-06T12:14:40.276 INFO:tasks.workunit.client.1.smithi190.stdout:1/40: link d5/f6 d5/f9 0 2024-05-06T12:14:40.277 INFO:tasks.workunit.client.0.smithi092.stdout:7/760: creat d9/de/d10/df0/df5/ffd x:0 0 0 2024-05-06T12:14:40.277 INFO:tasks.workunit.client.1.smithi190.stdout:5/55: mknod d2/c14 0 2024-05-06T12:14:40.279 INFO:tasks.workunit.client.1.smithi190.stdout:4/37: rename d2/ca to d2/dd/ce 0 2024-05-06T12:14:40.281 INFO:tasks.workunit.client.1.smithi190.stdout:7/39: creat d0/d4/fe x:0 0 0 2024-05-06T12:14:40.281 INFO:tasks.workunit.client.0.smithi092.stdout:6/745: dread d4/d20/d22/f7c [0,4194304] 0 2024-05-06T12:14:40.282 INFO:tasks.workunit.client.0.smithi092.stdout:7/761: creat d9/de/d10/d17/d99/dc3/dd2/ffe x:0 0 0 2024-05-06T12:14:40.282 INFO:tasks.workunit.client.1.smithi190.stdout:3/123: dwrite d5/d7/d15/f22 [0,4194304] 0 2024-05-06T12:14:40.284 INFO:tasks.workunit.client.0.smithi092.stdout:0/866: truncate d7/dc/d97/dc9/f10a 871179 0 2024-05-06T12:14:40.284 INFO:tasks.workunit.client.1.smithi190.stdout:3/124: chown d5/d14 0 1 2024-05-06T12:14:40.285 INFO:tasks.workunit.client.0.smithi092.stdout:5/571: write d4/d5/d30/f3c [822724,95794] 0 2024-05-06T12:14:40.285 INFO:tasks.workunit.client.1.smithi190.stdout:8/155: unlink d3/d8/c23 0 2024-05-06T12:14:40.286 INFO:tasks.workunit.client.0.smithi092.stdout:6/746: creat d4/d24/d38/d5d/d6d/d7b/ff3 x:0 0 0 2024-05-06T12:14:40.286 INFO:tasks.workunit.client.1.smithi190.stdout:6/35: creat fa x:0 0 0 2024-05-06T12:14:40.290 INFO:tasks.workunit.client.0.smithi092.stdout:5/572: creat d4/d34/d28/d4e/d5d/d82/fce x:0 0 0 2024-05-06T12:14:40.291 INFO:tasks.workunit.client.0.smithi092.stdout:0/867: mknod d7/dc/d28/d54/d1e/d46/df0/c120 0 2024-05-06T12:14:40.291 INFO:tasks.workunit.client.1.smithi190.stdout:6/36: write f7 [3589239,64932] 0 2024-05-06T12:14:40.292 INFO:tasks.workunit.client.0.smithi092.stdout:5/573: chown d4/d34/d28/fc3 233 1 2024-05-06T12:14:40.292 INFO:tasks.workunit.client.1.smithi190.stdout:9/131: creat db/d23/d25/f32 x:0 0 0 2024-05-06T12:14:40.293 INFO:tasks.workunit.client.0.smithi092.stdout:6/747: symlink d4/d20/d22/lf4 0 2024-05-06T12:14:40.293 INFO:tasks.workunit.client.1.smithi190.stdout:4/38: creat d2/dd/ff x:0 0 0 2024-05-06T12:14:40.294 INFO:tasks.workunit.client.0.smithi092.stdout:1/635: dread d3/d10/d13/d19/f5d [0,4194304] 0 2024-05-06T12:14:40.295 INFO:tasks.workunit.client.1.smithi190.stdout:4/39: write d2/dd/ff [878949,29079] 0 2024-05-06T12:14:40.295 INFO:tasks.workunit.client.0.smithi092.stdout:6/748: truncate d4/d4c/fc8 813722 0 2024-05-06T12:14:40.296 INFO:tasks.workunit.client.1.smithi190.stdout:3/125: unlink d5/d7/f20 0 2024-05-06T12:14:40.296 INFO:tasks.workunit.client.0.smithi092.stdout:7/762: rename d9/da5/l7f to d9/de/d10/d17/d21/lff 0 2024-05-06T12:14:40.296 INFO:tasks.workunit.client.1.smithi190.stdout:3/126: truncate d5/d14/f24 587421 0 2024-05-06T12:14:40.297 INFO:tasks.workunit.client.0.smithi092.stdout:5/574: rmdir d4/d34/d28/d4e/d5d/d82/d77 39 2024-05-06T12:14:40.298 INFO:tasks.workunit.client.1.smithi190.stdout:0/182: link d9/f26 d9/df/d21/f3b 0 2024-05-06T12:14:40.298 INFO:tasks.workunit.client.0.smithi092.stdout:0/868: dread d7/dc/d9e/fc0 [0,4194304] 0 2024-05-06T12:14:40.301 INFO:tasks.workunit.client.1.smithi190.stdout:2/73: dread d2/fd [0,4194304] 0 2024-05-06T12:14:40.301 INFO:tasks.workunit.client.0.smithi092.stdout:7/763: rename d9/de/d1f/d37/fb0 to d9/dd/f100 0 2024-05-06T12:14:40.302 INFO:tasks.workunit.client.1.smithi190.stdout:6/37: creat fb x:0 0 0 2024-05-06T12:14:40.302 INFO:tasks.workunit.client.1.smithi190.stdout:5/56: mkdir d2/da/de/d15 0 2024-05-06T12:14:40.304 INFO:tasks.workunit.client.1.smithi190.stdout:9/132: creat db/d21/f33 x:0 0 0 2024-05-06T12:14:40.305 INFO:tasks.workunit.client.1.smithi190.stdout:7/40: link d0/d4/fe d0/ff 0 2024-05-06T12:14:40.306 INFO:tasks.workunit.client.0.smithi092.stdout:0/869: creat d7/dc/d28/d54/d1e/d46/f121 x:0 0 0 2024-05-06T12:14:40.308 INFO:tasks.workunit.client.1.smithi190.stdout:3/127: mknod d5/d14/c31 0 2024-05-06T12:14:40.309 INFO:tasks.workunit.client.1.smithi190.stdout:4/40: rename d2/dd to d2/d10 0 2024-05-06T12:14:40.310 INFO:tasks.workunit.client.0.smithi092.stdout:0/870: mkdir d7/dc/d28/d54/d1e/d26/d3a/d122 0 2024-05-06T12:14:40.310 INFO:tasks.workunit.client.1.smithi190.stdout:8/156: symlink d3/d8/d2b/l33 0 2024-05-06T12:14:40.311 INFO:tasks.workunit.client.0.smithi092.stdout:6/749: link d4/d24/d4e/l60 d4/d4c/d52/d6c/da0/lf5 0 2024-05-06T12:14:40.311 INFO:tasks.workunit.client.0.smithi092.stdout:0/871: write d7/dc/d28/d54/d1e/fec [941852,125790] 0 2024-05-06T12:14:40.312 INFO:tasks.workunit.client.1.smithi190.stdout:0/183: symlink d9/dc/d28/l3c 0 2024-05-06T12:14:40.312 INFO:tasks.workunit.client.1.smithi190.stdout:0/184: chown d9/dc/l12 594 1 2024-05-06T12:14:40.313 INFO:tasks.workunit.client.0.smithi092.stdout:0/872: write d7/dc/d28/d54/d1e/d26/d96/d90/f112 [1898059,16694] 0 2024-05-06T12:14:40.315 INFO:tasks.workunit.client.1.smithi190.stdout:1/41: truncate d5/f6 2420051 0 2024-05-06T12:14:40.315 INFO:tasks.workunit.client.1.smithi190.stdout:6/38: creat fc x:0 0 0 2024-05-06T12:14:40.315 INFO:tasks.workunit.client.1.smithi190.stdout:6/39: dread - fb zero size 2024-05-06T12:14:40.316 INFO:tasks.workunit.client.1.smithi190.stdout:0/185: dread d9/f2f [0,4194304] 0 2024-05-06T12:14:40.316 INFO:tasks.workunit.client.1.smithi190.stdout:5/57: symlink d2/da/l16 0 2024-05-06T12:14:40.317 INFO:tasks.workunit.client.1.smithi190.stdout:6/40: write f7 [1627340,12475] 0 2024-05-06T12:14:40.317 INFO:tasks.workunit.client.1.smithi190.stdout:5/58: readlink d2/da/l16 0 2024-05-06T12:14:40.319 INFO:tasks.workunit.client.1.smithi190.stdout:6/41: write f7 [4661532,92339] 0 2024-05-06T12:14:40.320 INFO:tasks.workunit.client.0.smithi092.stdout:0/873: mknod d7/dc/d28/d54/c123 0 2024-05-06T12:14:40.320 INFO:tasks.workunit.client.1.smithi190.stdout:7/41: mknod d0/c10 0 2024-05-06T12:14:40.321 INFO:tasks.workunit.client.1.smithi190.stdout:7/42: chown d0/d4/fd 2694 1 2024-05-06T12:14:40.322 INFO:tasks.workunit.client.1.smithi190.stdout:8/157: unlink d3/d8/f1a 0 2024-05-06T12:14:40.324 INFO:tasks.workunit.client.0.smithi092.stdout:0/874: creat d7/dc/d6f/da5/dc6/d100/f124 x:0 0 0 2024-05-06T12:14:40.327 INFO:tasks.workunit.client.1.smithi190.stdout:6/42: creat fd x:0 0 0 2024-05-06T12:14:40.329 INFO:tasks.workunit.client.0.smithi092.stdout:1/636: dwrite d3/f5 [0,4194304] 0 2024-05-06T12:14:40.331 INFO:tasks.workunit.client.1.smithi190.stdout:2/74: getdents d2/d8 0 2024-05-06T12:14:40.335 INFO:tasks.workunit.client.0.smithi092.stdout:7/764: truncate d9/de/d1f/d37/fa0 2219488 0 2024-05-06T12:14:40.335 INFO:tasks.workunit.client.1.smithi190.stdout:8/158: creat d3/d8/d2b/f34 x:0 0 0 2024-05-06T12:14:40.340 INFO:tasks.workunit.client.1.smithi190.stdout:5/59: mknod d2/c17 0 2024-05-06T12:14:40.340 INFO:tasks.workunit.client.0.smithi092.stdout:5/575: dwrite d4/d5/d10/d31/f4a [0,4194304] 0 2024-05-06T12:14:40.341 INFO:tasks.workunit.client.1.smithi190.stdout:5/60: read - d2/f5 zero size 2024-05-06T12:14:40.346 INFO:tasks.workunit.client.0.smithi092.stdout:1/637: dwrite d3/d10/d13/d19/d21/d72/d9a/da4/fba [0,4194304] 0 2024-05-06T12:14:40.347 INFO:tasks.workunit.client.1.smithi190.stdout:8/159: dwrite d3/d8/f2c [0,4194304] 0 2024-05-06T12:14:40.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:39 smithi190 ceph-mon[33798]: Adjusting osd_memory_target on smithi190 to 2581M 2024-05-06T12:14:40.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:39 smithi190 ceph-mon[33798]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-06T12:14:40.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:39 smithi190 ceph-mon[33798]: Updating smithi190:/etc/ceph/ceph.conf 2024-05-06T12:14:40.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:39 smithi190 ceph-mon[33798]: Updating smithi190:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-06T12:14:40.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:39 smithi190 ceph-mon[33798]: Updating smithi092:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-06T12:14:40.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:39 smithi190 ceph-mon[33798]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-05-06T12:14:40.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:39 smithi190 ceph-mon[33798]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-05-06T12:14:40.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:39 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:40.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:39 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:40.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:39 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:40.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:39 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:40.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:39 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:40.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:39 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:40.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:39 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:40.351 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:39 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:40.351 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:39 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:40.351 INFO:tasks.workunit.client.0.smithi092.stdout:5/576: read d4/d5/d30/f40 [1090386,109890] 0 2024-05-06T12:14:40.354 INFO:tasks.workunit.client.1.smithi190.stdout:1/42: rename d5/f9 to d5/fa 0 2024-05-06T12:14:40.355 INFO:tasks.workunit.client.1.smithi190.stdout:9/133: rename f4 to db/dd/f34 0 2024-05-06T12:14:40.356 INFO:tasks.workunit.client.1.smithi190.stdout:0/186: dwrite d9/f26 [0,4194304] 0 2024-05-06T12:14:40.357 INFO:tasks.workunit.client.1.smithi190.stdout:6/43: link f2 fe 0 2024-05-06T12:14:40.357 INFO:tasks.workunit.client.1.smithi190.stdout:1/43: write d5/f8 [850415,7839] 0 2024-05-06T12:14:40.358 INFO:tasks.workunit.client.1.smithi190.stdout:3/128: getdents d5/d7 0 2024-05-06T12:14:40.358 INFO:tasks.workunit.client.0.smithi092.stdout:6/750: dwrite d4/d20/d23/d30/f6f [0,4194304] 0 2024-05-06T12:14:40.359 INFO:tasks.workunit.client.0.smithi092.stdout:7/765: dwrite d9/de/d10/df0/df5/ffd [0,4194304] 0 2024-05-06T12:14:40.360 INFO:tasks.workunit.client.1.smithi190.stdout:7/43: link d0/ff d0/d1/f11 0 2024-05-06T12:14:40.360 INFO:tasks.workunit.client.0.smithi092.stdout:0/875: dwrite d7/dc/d28/d54/d1e/d5d/f7c [0,4194304] 0 2024-05-06T12:14:40.361 INFO:tasks.workunit.client.1.smithi190.stdout:6/44: write fc [560562,107920] 0 2024-05-06T12:14:40.362 INFO:tasks.workunit.client.0.smithi092.stdout:5/577: dread - d4/d34/d29/da3/fa9 zero size 2024-05-06T12:14:40.363 INFO:tasks.workunit.client.1.smithi190.stdout:0/187: unlink d9/dc/cd 0 2024-05-06T12:14:40.364 INFO:tasks.workunit.client.1.smithi190.stdout:6/45: read - fb zero size 2024-05-06T12:14:40.365 INFO:tasks.workunit.client.1.smithi190.stdout:6/46: truncate fb 284522 0 2024-05-06T12:14:40.365 INFO:tasks.workunit.client.1.smithi190.stdout:9/134: symlink db/dd/d16/l35 0 2024-05-06T12:14:40.366 INFO:tasks.workunit.client.1.smithi190.stdout:6/47: truncate fb 1141539 0 2024-05-06T12:14:40.367 INFO:tasks.workunit.client.0.smithi092.stdout:6/751: unlink d4/d4c/da9/cb3 0 2024-05-06T12:14:40.367 INFO:tasks.workunit.client.1.smithi190.stdout:4/41: truncate d2/f4 4214435 0 2024-05-06T12:14:40.371 INFO:tasks.workunit.client.1.smithi190.stdout:8/160: mkdir d3/d35 0 2024-05-06T12:14:40.372 INFO:tasks.workunit.client.0.smithi092.stdout:7/766: dread d9/de/d1f/d37/d48/f7e [0,4194304] 0 2024-05-06T12:14:40.373 INFO:tasks.workunit.client.1.smithi190.stdout:1/44: read d5/f8 [616132,71027] 0 2024-05-06T12:14:40.374 INFO:tasks.workunit.client.0.smithi092.stdout:1/638: rename d3/d10/d13/d46/c99 to d3/d10/d13/d19/d21/cc7 0 2024-05-06T12:14:40.375 INFO:tasks.workunit.client.1.smithi190.stdout:8/161: chown d3/d8/d2b/l33 26126048 1 2024-05-06T12:14:40.379 INFO:tasks.workunit.client.1.smithi190.stdout:0/188: rmdir d9/dc/d28 39 2024-05-06T12:14:40.380 INFO:tasks.workunit.client.0.smithi092.stdout:7/767: fdatasync d9/f80 0 2024-05-06T12:14:40.381 INFO:tasks.workunit.client.1.smithi190.stdout:3/129: symlink d5/d14/d11/d2a/l32 0 2024-05-06T12:14:40.382 INFO:tasks.workunit.client.0.smithi092.stdout:1/639: symlink d3/d10/d13/d19/lc8 0 2024-05-06T12:14:40.382 INFO:tasks.workunit.client.1.smithi190.stdout:4/42: dwrite d2/d10/ff [0,4194304] 0 2024-05-06T12:14:40.382 INFO:tasks.workunit.client.0.smithi092.stdout:5/578: creat d4/d34/d28/d4e/d5d/d82/d1f/fcf x:0 0 0 2024-05-06T12:14:40.384 INFO:tasks.workunit.client.0.smithi092.stdout:1/640: symlink d3/d8f/lc9 0 2024-05-06T12:14:40.385 INFO:tasks.workunit.client.1.smithi190.stdout:6/48: mkdir df 0 2024-05-06T12:14:40.387 INFO:tasks.workunit.client.1.smithi190.stdout:3/130: read - d5/d7/f23 zero size 2024-05-06T12:14:40.387 INFO:tasks.workunit.client.0.smithi092.stdout:1/641: write d3/d42/d84/f7a [3107091,126968] 0 2024-05-06T12:14:40.388 INFO:tasks.workunit.client.1.smithi190.stdout:6/49: read - fa zero size 2024-05-06T12:14:40.391 INFO:tasks.workunit.client.0.smithi092.stdout:7/768: dwrite d9/de/ff4 [0,4194304] 0 2024-05-06T12:14:40.392 INFO:tasks.workunit.client.0.smithi092.stdout:1/642: mknod d3/d42/d84/d9f/cca 0 2024-05-06T12:14:40.392 INFO:tasks.workunit.client.1.smithi190.stdout:5/61: write d2/f13 [17160,29684] 0 2024-05-06T12:14:40.392 INFO:tasks.workunit.client.1.smithi190.stdout:8/162: symlink d3/d8/d2b/l36 0 2024-05-06T12:14:40.392 INFO:tasks.workunit.client.1.smithi190.stdout:2/75: dwrite d2/fd [0,4194304] 0 2024-05-06T12:14:40.394 INFO:tasks.workunit.client.0.smithi092.stdout:7/769: chown d9/de/d1f/d37 134401 1 2024-05-06T12:14:40.394 INFO:tasks.workunit.client.1.smithi190.stdout:4/43: mkdir d2/d11 0 2024-05-06T12:14:40.394 INFO:tasks.workunit.client.1.smithi190.stdout:5/62: readlink d2/l6 0 2024-05-06T12:14:40.398 INFO:tasks.workunit.client.0.smithi092.stdout:3/566: sync 2024-05-06T12:14:40.398 INFO:tasks.workunit.client.0.smithi092.stdout:1/643: truncate d3/d10/d13/d46/f55 1519846 0 2024-05-06T12:14:40.399 INFO:tasks.workunit.client.1.smithi190.stdout:5/63: write d2/fc [328673,51493] 0 2024-05-06T12:14:40.399 INFO:tasks.workunit.client.1.smithi190.stdout:7/44: dwrite d0/d4/fd [0,4194304] 0 2024-05-06T12:14:40.400 INFO:tasks.workunit.client.0.smithi092.stdout:1/644: read - d3/d10/d13/d19/d34/d89/fa5 zero size 2024-05-06T12:14:40.401 INFO:tasks.workunit.client.1.smithi190.stdout:3/131: mkdir d5/d7/d15/d1f/d33 0 2024-05-06T12:14:40.401 INFO:tasks.workunit.client.1.smithi190.stdout:5/64: dread - d2/f5 zero size 2024-05-06T12:14:40.402 INFO:tasks.workunit.client.1.smithi190.stdout:7/45: chown d0/d1/fc 429627 1 2024-05-06T12:14:40.402 INFO:tasks.workunit.client.0.smithi092.stdout:1/645: write d3/d42/d84/d4f/d58/d65/d81/fbc [693339,43523] 0 2024-05-06T12:14:40.402 INFO:tasks.workunit.client.1.smithi190.stdout:8/163: symlink d3/d8/l37 0 2024-05-06T12:14:40.403 INFO:tasks.workunit.client.1.smithi190.stdout:3/132: stat d5/d7/l29 0 2024-05-06T12:14:40.404 INFO:tasks.workunit.client.1.smithi190.stdout:3/133: write d5/d7/f23 [301190,34586] 0 2024-05-06T12:14:40.405 INFO:tasks.workunit.client.1.smithi190.stdout:4/44: dread d2/d10/ff [0,4194304] 0 2024-05-06T12:14:40.407 INFO:tasks.workunit.client.0.smithi092.stdout:3/567: getdents d8/db/d4c/d40/d7a/d90 0 2024-05-06T12:14:40.407 INFO:tasks.workunit.client.0.smithi092.stdout:3/568: write d8/db/d4c/d3d/fa9 [258598,39261] 0 2024-05-06T12:14:40.407 INFO:tasks.workunit.client.1.smithi190.stdout:6/50: mkdir df/d10 0 2024-05-06T12:14:40.407 INFO:tasks.workunit.client.1.smithi190.stdout:5/65: unlink d2/f13 0 2024-05-06T12:14:40.410 INFO:tasks.workunit.client.1.smithi190.stdout:9/135: rename db/d23/c30 to db/d23/c36 0 2024-05-06T12:14:40.410 INFO:tasks.workunit.client.0.smithi092.stdout:1/646: creat d3/d42/fcb x:0 0 0 2024-05-06T12:14:40.410 INFO:tasks.workunit.client.1.smithi190.stdout:9/136: chown db/d23/f2a 627417 1 2024-05-06T12:14:40.411 INFO:tasks.workunit.client.0.smithi092.stdout:3/569: write d8/db/d14/f50 [392609,76963] 0 2024-05-06T12:14:40.412 INFO:tasks.workunit.client.1.smithi190.stdout:7/46: rename d0/c10 to d0/d1/d2/c12 0 2024-05-06T12:14:40.412 INFO:tasks.workunit.client.1.smithi190.stdout:4/45: rename d2 to d2/d10/d12 22 2024-05-06T12:14:40.413 INFO:tasks.workunit.client.1.smithi190.stdout:8/164: dread - d3/f1c zero size 2024-05-06T12:14:40.414 INFO:tasks.workunit.client.0.smithi092.stdout:3/570: symlink d8/db/d4c/d40/d67/lb1 0 2024-05-06T12:14:40.416 INFO:tasks.workunit.client.1.smithi190.stdout:2/76: rmdir d2/d10/d11 0 2024-05-06T12:14:40.417 INFO:tasks.workunit.client.1.smithi190.stdout:9/137: creat db/d23/d25/f37 x:0 0 0 2024-05-06T12:14:40.417 INFO:tasks.workunit.client.1.smithi190.stdout:2/77: dread - d2/d8/dc/f13 zero size 2024-05-06T12:14:40.417 INFO:tasks.workunit.client.1.smithi190.stdout:7/47: rmdir d0 39 2024-05-06T12:14:40.419 INFO:tasks.workunit.client.0.smithi092.stdout:6/752: dwrite d4/d20/d23/f46 [0,4194304] 0 2024-05-06T12:14:40.420 INFO:tasks.workunit.client.0.smithi092.stdout:3/571: creat d8/d24/d33/d39/d5b/d98/fb2 x:0 0 0 2024-05-06T12:14:40.420 INFO:tasks.workunit.client.1.smithi190.stdout:3/134: rename d5/d7/d15/f2f to d5/d14/d11/f34 0 2024-05-06T12:14:40.421 INFO:tasks.workunit.client.0.smithi092.stdout:3/572: write d8/db/d4c/d3d/d48/faa [1013723,75247] 0 2024-05-06T12:14:40.422 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:39 smithi092 ceph-mon[26954]: Adjusting osd_memory_target on smithi190 to 2581M 2024-05-06T12:14:40.422 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:39 smithi092 ceph-mon[26954]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-06T12:14:40.423 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:39 smithi092 ceph-mon[26954]: Updating smithi190:/etc/ceph/ceph.conf 2024-05-06T12:14:40.423 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:39 smithi092 ceph-mon[26954]: Updating smithi190:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-06T12:14:40.423 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:39 smithi092 ceph-mon[26954]: Updating smithi092:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-06T12:14:40.423 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:39 smithi092 ceph-mon[26954]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-05-06T12:14:40.423 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:39 smithi092 ceph-mon[26954]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-05-06T12:14:40.423 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:39 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:40.423 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:39 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:40.423 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:39 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:40.423 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:39 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:40.423 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:39 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:40.423 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:39 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:40.423 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:39 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:40.423 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:39 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:40.423 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:39 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:40.424 INFO:tasks.workunit.client.1.smithi190.stdout:8/165: unlink d3/d8/d25/f29 0 2024-05-06T12:14:40.425 INFO:tasks.workunit.client.0.smithi092.stdout:1/647: dread d3/d10/d13/d19/d34/fbe [0,4194304] 0 2024-05-06T12:14:40.430 INFO:tasks.workunit.client.1.smithi190.stdout:4/46: symlink d2/d11/l13 0 2024-05-06T12:14:40.432 INFO:tasks.workunit.client.1.smithi190.stdout:9/138: dwrite db/f2b [4194304,4194304] 0 2024-05-06T12:14:40.432 INFO:tasks.workunit.client.0.smithi092.stdout:3/573: unlink d8/db/d4c/l7c 0 2024-05-06T12:14:40.434 INFO:tasks.workunit.client.0.smithi092.stdout:5/579: dwrite d4/d34/d28/d4e/f4f [0,4194304] 0 2024-05-06T12:14:40.435 INFO:tasks.workunit.client.1.smithi190.stdout:2/78: dwrite d2/d8/dc/f13 [0,4194304] 0 2024-05-06T12:14:40.435 INFO:tasks.workunit.client.0.smithi092.stdout:7/770: dread d9/de/d1f/f61 [0,4194304] 0 2024-05-06T12:14:40.436 INFO:tasks.workunit.client.1.smithi190.stdout:2/79: readlink d2/l9 0 2024-05-06T12:14:40.436 INFO:tasks.workunit.client.0.smithi092.stdout:3/574: stat d8/d24/d33/d4b/c10 0 2024-05-06T12:14:40.437 INFO:tasks.workunit.client.1.smithi190.stdout:8/166: creat d3/d8/d2b/f38 x:0 0 0 2024-05-06T12:14:40.438 INFO:tasks.workunit.client.0.smithi092.stdout:6/753: dwrite d4/d20/f83 [0,4194304] 0 2024-05-06T12:14:40.438 INFO:tasks.workunit.client.0.smithi092.stdout:7/771: chown d9/de/d10/d17/d6a 7065213 1 2024-05-06T12:14:40.442 INFO:tasks.workunit.client.1.smithi190.stdout:4/47: creat d2/d11/f14 x:0 0 0 2024-05-06T12:14:40.442 INFO:tasks.workunit.client.0.smithi092.stdout:7/772: write d9/de/d10/d17/d21/d38/d77/fee [911659,97408] 0 2024-05-06T12:14:40.447 INFO:tasks.workunit.client.1.smithi190.stdout:7/48: mknod d0/da/c13 0 2024-05-06T12:14:40.447 INFO:tasks.workunit.client.1.smithi190.stdout:0/189: dwrite d9/df/d21/f2a [0,4194304] 0 2024-05-06T12:14:40.452 INFO:tasks.workunit.client.1.smithi190.stdout:0/190: readlink d9/l3a 0 2024-05-06T12:14:40.452 INFO:tasks.workunit.client.1.smithi190.stdout:8/167: creat d3/d8/d25/f39 x:0 0 0 2024-05-06T12:14:40.453 INFO:tasks.workunit.client.0.smithi092.stdout:5/580: creat d4/d34/fd0 x:0 0 0 2024-05-06T12:14:40.455 INFO:tasks.workunit.client.1.smithi190.stdout:9/139: symlink db/d23/d29/l38 0 2024-05-06T12:14:40.456 INFO:tasks.workunit.client.0.smithi092.stdout:5/581: unlink d4/l95 0 2024-05-06T12:14:40.457 INFO:tasks.workunit.client.1.smithi190.stdout:5/66: rmdir d2 39 2024-05-06T12:14:40.458 INFO:tasks.workunit.client.0.smithi092.stdout:3/575: mknod d8/db/d4c/d3d/cb3 0 2024-05-06T12:14:40.459 INFO:tasks.workunit.client.1.smithi190.stdout:7/49: mkdir d0/d14 0 2024-05-06T12:14:40.461 INFO:tasks.workunit.client.0.smithi092.stdout:5/582: creat d4/d59/fd1 x:0 0 0 2024-05-06T12:14:40.461 INFO:tasks.workunit.client.0.smithi092.stdout:1/648: truncate d3/d10/d13/d19/f4e 2616775 0 2024-05-06T12:14:40.462 INFO:tasks.workunit.client.0.smithi092.stdout:3/576: fsync d8/db/d14/d5a/f94 0 2024-05-06T12:14:40.465 INFO:tasks.workunit.client.1.smithi190.stdout:4/48: dwrite d2/d11/f14 [0,4194304] 0 2024-05-06T12:14:40.465 INFO:tasks.workunit.client.0.smithi092.stdout:5/583: creat d4/d34/d29/da3/fd2 x:0 0 0 2024-05-06T12:14:40.465 INFO:tasks.workunit.client.1.smithi190.stdout:5/67: dread - d2/f5 zero size 2024-05-06T12:14:40.466 INFO:tasks.workunit.client.1.smithi190.stdout:3/135: getdents d5/d14/d11 0 2024-05-06T12:14:40.467 INFO:tasks.workunit.client.0.smithi092.stdout:3/577: chown d8/db/d4c/d40/d67/f97 58939 1 2024-05-06T12:14:40.467 INFO:tasks.workunit.client.1.smithi190.stdout:0/191: dwrite d9/dc/f1f [0,4194304] 0 2024-05-06T12:14:40.470 INFO:tasks.workunit.client.0.smithi092.stdout:5/584: rename d4/d34/d29/d54/db0/dbb to d4/d34/d29/da3/dd3 0 2024-05-06T12:14:40.471 INFO:tasks.workunit.client.0.smithi092.stdout:5/585: chown d4/f4c 1704 1 2024-05-06T12:14:40.472 INFO:tasks.workunit.client.1.smithi190.stdout:5/68: chown d2/c12 10090719 1 2024-05-06T12:14:40.472 INFO:tasks.workunit.client.1.smithi190.stdout:8/168: getdents d3/d35 0 2024-05-06T12:14:40.475 INFO:tasks.workunit.client.1.smithi190.stdout:3/136: dread - d5/d14/d11/d21/f30 zero size 2024-05-06T12:14:40.475 INFO:tasks.workunit.client.1.smithi190.stdout:9/140: symlink db/d21/d22/l39 0 2024-05-06T12:14:40.477 INFO:tasks.workunit.client.1.smithi190.stdout:4/49: unlink d2/l6 0 2024-05-06T12:14:40.478 INFO:tasks.workunit.client.1.smithi190.stdout:3/137: chown d5/f27 73418 1 2024-05-06T12:14:40.479 INFO:tasks.workunit.client.1.smithi190.stdout:7/50: dread d0/d1/fb [0,4194304] 0 2024-05-06T12:14:40.480 INFO:tasks.workunit.client.1.smithi190.stdout:5/69: mknod d2/da/de/c18 0 2024-05-06T12:14:40.482 INFO:tasks.workunit.client.1.smithi190.stdout:8/169: mkdir d3/d35/d3a 0 2024-05-06T12:14:40.483 INFO:tasks.workunit.client.1.smithi190.stdout:5/70: write d2/fc [513825,62426] 0 2024-05-06T12:14:40.485 INFO:tasks.workunit.client.1.smithi190.stdout:4/50: write d2/d11/f14 [4124339,26846] 0 2024-05-06T12:14:40.487 INFO:tasks.workunit.client.1.smithi190.stdout:3/138: creat d5/d14/d11/d2a/f35 x:0 0 0 2024-05-06T12:14:40.488 INFO:tasks.workunit.client.1.smithi190.stdout:0/192: dwrite d9/df/f2e [0,4194304] 0 2024-05-06T12:14:40.489 INFO:tasks.workunit.client.0.smithi092.stdout:7/773: truncate d9/dd/f100 354695 0 2024-05-06T12:14:40.490 INFO:tasks.workunit.client.1.smithi190.stdout:5/71: chown d2/da/de/c10 272204497 1 2024-05-06T12:14:40.490 INFO:tasks.workunit.client.1.smithi190.stdout:8/170: truncate d3/d8/f27 378695 0 2024-05-06T12:14:40.491 INFO:tasks.workunit.client.1.smithi190.stdout:8/171: chown d3/c22 93202744 1 2024-05-06T12:14:40.492 INFO:tasks.workunit.client.0.smithi092.stdout:6/754: dwrite d4/d24/d26/dae/dc0/f71 [0,4194304] 0 2024-05-06T12:14:40.493 INFO:tasks.workunit.client.1.smithi190.stdout:5/72: write d2/fc [1438591,32080] 0 2024-05-06T12:14:40.493 INFO:tasks.workunit.client.1.smithi190.stdout:0/193: chown d9/df/d21/f2a 60707 1 2024-05-06T12:14:40.493 INFO:tasks.workunit.client.1.smithi190.stdout:9/141: dread db/dd/d16/f1e [0,4194304] 0 2024-05-06T12:14:40.493 INFO:tasks.workunit.client.0.smithi092.stdout:6/755: readlink d4/d24/d26/l2e 0 2024-05-06T12:14:40.495 INFO:tasks.workunit.client.1.smithi190.stdout:5/73: chown d2/cb 457262707 1 2024-05-06T12:14:40.496 INFO:tasks.workunit.client.0.smithi092.stdout:1/649: write d3/d42/d84/d4f/d7f/fad [1429786,113812] 0 2024-05-06T12:14:40.497 INFO:tasks.workunit.client.1.smithi190.stdout:0/194: write d9/f2c [2029465,94485] 0 2024-05-06T12:14:40.499 INFO:tasks.workunit.client.1.smithi190.stdout:5/74: write d2/da/de/f11 [722754,79430] 0 2024-05-06T12:14:40.500 INFO:tasks.workunit.client.1.smithi190.stdout:7/51: getdents d0/d1 0 2024-05-06T12:14:40.501 INFO:tasks.workunit.client.1.smithi190.stdout:8/172: fdatasync d3/f17 0 2024-05-06T12:14:40.501 INFO:tasks.workunit.client.0.smithi092.stdout:1/650: symlink d3/d10/d13/d19/d57/lcc 0 2024-05-06T12:14:40.503 INFO:tasks.workunit.client.0.smithi092.stdout:1/651: write d3/d10/d13/d19/f5a [1609472,72877] 0 2024-05-06T12:14:40.504 INFO:tasks.workunit.client.1.smithi190.stdout:8/173: chown d3/d8/c1d 1 1 2024-05-06T12:14:40.504 INFO:tasks.workunit.client.0.smithi092.stdout:5/586: dwrite d4/d5/d10/d31/d53/f9e [0,4194304] 0 2024-05-06T12:14:40.505 INFO:tasks.workunit.client.1.smithi190.stdout:3/139: symlink d5/d7/d15/d1f/d33/l36 0 2024-05-06T12:14:40.505 INFO:tasks.workunit.client.0.smithi092.stdout:1/652: dread - d3/d10/d13/faa zero size 2024-05-06T12:14:40.507 INFO:tasks.workunit.client.0.smithi092.stdout:7/774: truncate d9/d62/fe1 294753 0 2024-05-06T12:14:40.510 INFO:tasks.workunit.client.0.smithi092.stdout:5/587: symlink d4/d34/d29/d54/dc5/ld4 0 2024-05-06T12:14:40.510 INFO:tasks.workunit.client.1.smithi190.stdout:0/195: dwrite d9/f2c [0,4194304] 0 2024-05-06T12:14:40.511 INFO:tasks.workunit.client.0.smithi092.stdout:5/588: dread - d4/d34/d28/d4e/d5d/d82/fc0 zero size 2024-05-06T12:14:40.515 INFO:tasks.workunit.client.1.smithi190.stdout:5/75: mknod d2/da/c19 0 2024-05-06T12:14:40.517 INFO:tasks.workunit.client.0.smithi092.stdout:5/589: unlink d4/d34/l7e 0 2024-05-06T12:14:40.519 INFO:tasks.workunit.client.0.smithi092.stdout:1/653: link d3/d10/d13/d19/d21/c56 d3/d10/d13/d19/d21/ccd 0 2024-05-06T12:14:40.519 INFO:tasks.workunit.client.1.smithi190.stdout:8/174: dwrite d3/d8/f30 [4194304,4194304] 0 2024-05-06T12:14:40.519 INFO:tasks.workunit.client.1.smithi190.stdout:7/52: mkdir d0/d15 0 2024-05-06T12:14:40.520 INFO:tasks.workunit.client.1.smithi190.stdout:4/51: fsync d2/f4 0 2024-05-06T12:14:40.520 INFO:tasks.workunit.client.0.smithi092.stdout:1/654: write d3/f5 [3808534,50007] 0 2024-05-06T12:14:40.520 INFO:tasks.workunit.client.0.smithi092.stdout:7/775: getdents d9/de/d10/d17/d6a 0 2024-05-06T12:14:40.522 INFO:tasks.workunit.client.0.smithi092.stdout:5/590: creat d4/d59/fd5 x:0 0 0 2024-05-06T12:14:40.522 INFO:tasks.workunit.client.1.smithi190.stdout:3/140: symlink d5/d7/d15/d1f/d33/l37 0 2024-05-06T12:14:40.523 INFO:tasks.workunit.client.1.smithi190.stdout:0/196: unlink d9/dc/fe 0 2024-05-06T12:14:40.524 INFO:tasks.workunit.client.1.smithi190.stdout:5/76: fdatasync f0 0 2024-05-06T12:14:40.526 INFO:tasks.workunit.client.1.smithi190.stdout:8/175: mkdir d3/d8/d25/d3b 0 2024-05-06T12:14:40.528 INFO:tasks.workunit.client.1.smithi190.stdout:9/142: creat db/d23/f3a x:0 0 0 2024-05-06T12:14:40.529 INFO:tasks.workunit.client.1.smithi190.stdout:9/143: chown db/dd/d16 1093165 1 2024-05-06T12:14:40.530 INFO:tasks.workunit.client.0.smithi092.stdout:5/591: mkdir d4/d34/d28/d4e/d4b/dd6 0 2024-05-06T12:14:40.530 INFO:tasks.workunit.client.0.smithi092.stdout:1/655: dwrite d3/d10/d13/d19/d21/d88/fa0 [0,4194304] 0 2024-05-06T12:14:40.530 INFO:tasks.workunit.client.1.smithi190.stdout:5/77: dread d2/f9 [0,4194304] 0 2024-05-06T12:14:40.530 INFO:tasks.workunit.client.1.smithi190.stdout:9/144: dread - db/d21/f33 zero size 2024-05-06T12:14:40.531 INFO:tasks.workunit.client.0.smithi092.stdout:1/656: chown d3/fb2 391030 1 2024-05-06T12:14:40.531 INFO:tasks.workunit.client.1.smithi190.stdout:5/78: read - d2/f5 zero size 2024-05-06T12:14:40.531 INFO:tasks.workunit.client.1.smithi190.stdout:3/141: readlink d5/l8 0 2024-05-06T12:14:40.532 INFO:tasks.workunit.client.1.smithi190.stdout:3/142: write d5/d7/d15/f28 [550524,50076] 0 2024-05-06T12:14:40.534 INFO:tasks.workunit.client.0.smithi092.stdout:7/776: rmdir d9/de/d10/d17/d99/da6/df3 0 2024-05-06T12:14:40.539 INFO:tasks.workunit.client.1.smithi190.stdout:8/176: rename d3/d8/d25/f31 to d3/d35/f3c 0 2024-05-06T12:14:40.541 INFO:tasks.workunit.client.1.smithi190.stdout:5/79: creat d2/da/f1a x:0 0 0 2024-05-06T12:14:40.541 INFO:tasks.workunit.client.0.smithi092.stdout:7/777: dwrite d9/de/ff4 [0,4194304] 0 2024-05-06T12:14:40.541 INFO:tasks.workunit.client.0.smithi092.stdout:1/657: rename d3/d10/d13/d19/d21/c7d to d3/d10/d13/d19/d21/cce 0 2024-05-06T12:14:40.542 INFO:tasks.workunit.client.0.smithi092.stdout:1/658: write d3/d10/d13/fbb [614198,112789] 0 2024-05-06T12:14:40.543 INFO:tasks.workunit.client.1.smithi190.stdout:0/197: truncate d9/df/f10 3540852 0 2024-05-06T12:14:40.545 INFO:tasks.workunit.client.1.smithi190.stdout:9/145: mkdir db/d23/d3b 0 2024-05-06T12:14:40.546 INFO:tasks.workunit.client.1.smithi190.stdout:5/80: creat d2/da/f1b x:0 0 0 2024-05-06T12:14:40.547 INFO:tasks.workunit.client.0.smithi092.stdout:1/659: mkdir d3/d42/d84/dcf 0 2024-05-06T12:14:40.548 INFO:tasks.workunit.client.1.smithi190.stdout:8/177: creat d3/d35/d3a/f3d x:0 0 0 2024-05-06T12:14:40.549 INFO:tasks.workunit.client.1.smithi190.stdout:8/178: fsync d3/f2d 0 2024-05-06T12:14:40.549 INFO:tasks.workunit.client.1.smithi190.stdout:9/146: write f7 [402588,89104] 0 2024-05-06T12:14:40.550 INFO:tasks.workunit.client.0.smithi092.stdout:1/660: creat d3/d10/d13/d19/d21/d72/d9a/fd0 x:0 0 0 2024-05-06T12:14:40.553 INFO:tasks.workunit.client.1.smithi190.stdout:0/198: fdatasync d9/df/f15 0 2024-05-06T12:14:40.556 INFO:tasks.workunit.client.1.smithi190.stdout:9/147: unlink db/d21/f33 0 2024-05-06T12:14:40.557 INFO:tasks.workunit.client.1.smithi190.stdout:5/81: dwrite d2/fc [0,4194304] 0 2024-05-06T12:14:40.559 INFO:tasks.workunit.client.1.smithi190.stdout:5/82: chown d2/f9 9570740 1 2024-05-06T12:14:40.560 INFO:tasks.workunit.client.1.smithi190.stdout:0/199: mknod d9/c3d 0 2024-05-06T12:14:40.561 INFO:tasks.workunit.client.1.smithi190.stdout:9/148: creat db/d21/f3c x:0 0 0 2024-05-06T12:14:40.562 INFO:tasks.workunit.client.1.smithi190.stdout:8/179: dwrite d3/f2d [0,4194304] 0 2024-05-06T12:14:40.563 INFO:tasks.workunit.client.1.smithi190.stdout:7/53: truncate d0/d4/fd 3501261 0 2024-05-06T12:14:40.564 INFO:tasks.workunit.client.1.smithi190.stdout:8/180: dread - d3/d8/d25/f39 zero size 2024-05-06T12:14:40.565 INFO:tasks.workunit.client.1.smithi190.stdout:9/149: write db/dd/d16/f24 [269739,29458] 0 2024-05-06T12:14:40.566 INFO:tasks.workunit.client.1.smithi190.stdout:7/54: write d0/d4/f8 [329432,74451] 0 2024-05-06T12:14:40.567 INFO:tasks.workunit.client.1.smithi190.stdout:8/181: write d3/f2e [681693,125398] 0 2024-05-06T12:14:40.589 INFO:tasks.workunit.client.1.smithi190.stdout:5/83: dwrite d2/f9 [0,4194304] 0 2024-05-06T12:14:40.594 INFO:tasks.workunit.client.1.smithi190.stdout:5/84: dread d2/da/de/f11 [0,4194304] 0 2024-05-06T12:14:40.595 INFO:tasks.workunit.client.1.smithi190.stdout:5/85: write d2/da/de/f11 [1281552,89987] 0 2024-05-06T12:14:40.597 INFO:tasks.workunit.client.1.smithi190.stdout:5/86: write d2/da/de/f11 [2392113,115466] 0 2024-05-06T12:14:40.616 INFO:tasks.workunit.client.0.smithi092.stdout:1/661: rename d3/d10/d13/lb1 to d3/d10/d13/d19/d21/d88/ld1 0 2024-05-06T12:14:40.617 INFO:tasks.workunit.client.0.smithi092.stdout:1/662: fdatasync d3/d10/d13/d19/f31 0 2024-05-06T12:14:40.619 INFO:tasks.workunit.client.0.smithi092.stdout:1/663: mkdir d3/d10/d13/dd2 0 2024-05-06T12:14:40.622 INFO:tasks.workunit.client.0.smithi092.stdout:4/726: sync 2024-05-06T12:14:40.625 INFO:tasks.workunit.client.1.smithi190.stdout:7/55: symlink d0/l16 0 2024-05-06T12:14:40.626 INFO:tasks.workunit.client.1.smithi190.stdout:0/200: creat d9/dc/d28/f3e x:0 0 0 2024-05-06T12:14:40.629 INFO:tasks.workunit.client.1.smithi190.stdout:5/87: unlink d2/da/f1a 0 2024-05-06T12:14:40.631 INFO:tasks.workunit.client.1.smithi190.stdout:5/88: truncate d2/da/f1b 758824 0 2024-05-06T12:14:40.632 INFO:tasks.workunit.client.1.smithi190.stdout:0/201: symlink d9/dc/d28/l3f 0 2024-05-06T12:14:40.632 INFO:tasks.workunit.client.1.smithi190.stdout:9/150: creat db/f3d x:0 0 0 2024-05-06T12:14:40.636 INFO:tasks.workunit.client.1.smithi190.stdout:5/89: unlink d2/da/de/f11 0 2024-05-06T12:14:40.639 INFO:tasks.workunit.client.1.smithi190.stdout:9/151: mknod db/d23/d29/c3e 0 2024-05-06T12:14:40.640 INFO:tasks.workunit.client.1.smithi190.stdout:9/152: write db/f26 [1107405,104022] 0 2024-05-06T12:14:40.641 INFO:tasks.workunit.client.1.smithi190.stdout:8/182: getdents d3/d35/d3a 0 2024-05-06T12:14:40.641 INFO:tasks.workunit.client.1.smithi190.stdout:9/153: dread - db/f1f zero size 2024-05-06T12:14:40.643 INFO:tasks.workunit.client.0.smithi092.stdout:4/727: mkdir d9/d2b/d7c/dab/dee 0 2024-05-06T12:14:40.643 INFO:tasks.workunit.client.0.smithi092.stdout:5/592: write d4/f9b [599187,81684] 0 2024-05-06T12:14:40.643 INFO:tasks.workunit.client.1.smithi190.stdout:5/90: creat d2/da/de/d15/f1c x:0 0 0 2024-05-06T12:14:40.644 INFO:tasks.workunit.client.1.smithi190.stdout:8/183: mkdir d3/d8/d2b/d3e 0 2024-05-06T12:14:40.645 INFO:tasks.workunit.client.1.smithi190.stdout:8/184: truncate d3/f1c 829051 0 2024-05-06T12:14:40.647 INFO:tasks.workunit.client.0.smithi092.stdout:1/664: rmdir d3/d42/d84 39 2024-05-06T12:14:40.648 INFO:tasks.workunit.client.1.smithi190.stdout:8/185: rmdir d3/d35 39 2024-05-06T12:14:40.649 INFO:tasks.workunit.client.0.smithi092.stdout:5/593: link d4/d5/f96 d4/d34/d29/da3/dc2/fd7 0 2024-05-06T12:14:40.650 INFO:tasks.workunit.client.1.smithi190.stdout:5/91: rename d2/c17 to d2/c1d 0 2024-05-06T12:14:40.652 INFO:tasks.workunit.client.1.smithi190.stdout:9/154: dwrite db/dd/f17 [0,4194304] 0 2024-05-06T12:14:40.653 INFO:tasks.workunit.client.1.smithi190.stdout:5/92: write d2/f5 [288375,99229] 0 2024-05-06T12:14:40.653 INFO:tasks.workunit.client.0.smithi092.stdout:4/728: dwrite d9/d2b/d5b/f8d [4194304,4194304] 0 2024-05-06T12:14:40.654 INFO:tasks.workunit.client.0.smithi092.stdout:7/778: dwrite d9/de/d10/d17/f8f [4194304,4194304] 0 2024-05-06T12:14:40.658 INFO:tasks.workunit.client.1.smithi190.stdout:8/186: dwrite d3/d8/f2c [0,4194304] 0 2024-05-06T12:14:40.658 INFO:tasks.workunit.client.1.smithi190.stdout:9/155: mknod db/d23/d3b/c3f 0 2024-05-06T12:14:40.659 INFO:tasks.workunit.client.0.smithi092.stdout:1/665: symlink d3/d42/d84/d4f/d58/d65/ld3 0 2024-05-06T12:14:40.659 INFO:tasks.workunit.client.0.smithi092.stdout:1/666: chown d3/d10/d13/d19/d21 1080732 1 2024-05-06T12:14:40.660 INFO:tasks.workunit.client.0.smithi092.stdout:7/779: stat d9/de/d10/d17/d21/d2e/cf9 0 2024-05-06T12:14:40.661 INFO:tasks.workunit.client.0.smithi092.stdout:4/729: rename d9/d2b/c57 to d9/d2b/d5b/daf/cef 0 2024-05-06T12:14:40.663 INFO:tasks.workunit.client.1.smithi190.stdout:8/187: mknod d3/d8/d2b/c3f 0 2024-05-06T12:14:40.664 INFO:tasks.workunit.client.1.smithi190.stdout:5/93: dwrite d2/fc [0,4194304] 0 2024-05-06T12:14:40.666 INFO:tasks.workunit.client.1.smithi190.stdout:9/156: mknod db/d21/d22/c40 0 2024-05-06T12:14:40.671 INFO:tasks.workunit.client.0.smithi092.stdout:1/667: rmdir d3/d10/d13 39 2024-05-06T12:14:40.672 INFO:tasks.workunit.client.1.smithi190.stdout:5/94: symlink d2/da/de/l1e 0 2024-05-06T12:14:40.672 INFO:tasks.workunit.client.1.smithi190.stdout:8/188: dread d3/fc [0,4194304] 0 2024-05-06T12:14:40.673 INFO:tasks.workunit.client.1.smithi190.stdout:9/157: dread f6 [0,4194304] 0 2024-05-06T12:14:40.673 INFO:tasks.workunit.client.0.smithi092.stdout:7/780: mkdir d9/de/d10/d17/d99/db6/df1/d101 0 2024-05-06T12:14:40.673 INFO:tasks.workunit.client.1.smithi190.stdout:8/189: fdatasync d3/f2e 0 2024-05-06T12:14:40.674 INFO:tasks.workunit.client.1.smithi190.stdout:8/190: readlink d3/l1e 0 2024-05-06T12:14:40.675 INFO:tasks.workunit.client.1.smithi190.stdout:9/158: write db/d23/d25/f32 [372843,124426] 0 2024-05-06T12:14:40.676 INFO:tasks.workunit.client.0.smithi092.stdout:4/730: rmdir d9/d27 39 2024-05-06T12:14:40.677 INFO:tasks.workunit.client.1.smithi190.stdout:5/95: rename d2/da/lf to d2/da/de/d15/l1f 0 2024-05-06T12:14:40.678 INFO:tasks.workunit.client.0.smithi092.stdout:1/668: fsync d3/d10/d13/d19/d21/d72/d9a/da4/fba 0 2024-05-06T12:14:40.680 INFO:tasks.workunit.client.0.smithi092.stdout:1/669: fdatasync d3/d42/d84/d4f/d58/f62 0 2024-05-06T12:14:40.681 INFO:tasks.workunit.client.1.smithi190.stdout:8/191: mkdir d3/d35/d40 0 2024-05-06T12:14:40.683 INFO:tasks.workunit.client.1.smithi190.stdout:5/96: creat d2/da/de/d15/f20 x:0 0 0 2024-05-06T12:14:40.684 INFO:tasks.workunit.client.0.smithi092.stdout:4/731: link d9/d27/da8/db3/dbf/lcf d9/d27/dc5/lf0 0 2024-05-06T12:14:40.685 INFO:tasks.workunit.client.1.smithi190.stdout:9/159: rename f7 to db/dd/f41 0 2024-05-06T12:14:40.686 INFO:tasks.workunit.client.1.smithi190.stdout:8/192: dread d3/f2e [0,4194304] 0 2024-05-06T12:14:40.686 INFO:tasks.workunit.client.1.smithi190.stdout:5/97: dread d2/f5 [0,4194304] 0 2024-05-06T12:14:40.687 INFO:tasks.workunit.client.1.smithi190.stdout:9/160: write db/f26 [1672322,58443] 0 2024-05-06T12:14:40.688 INFO:tasks.workunit.client.1.smithi190.stdout:8/193: dread - d3/d35/d3a/f3d zero size 2024-05-06T12:14:40.691 INFO:tasks.workunit.client.1.smithi190.stdout:5/98: symlink d2/da/de/d15/l21 0 2024-05-06T12:14:40.692 INFO:tasks.workunit.client.1.smithi190.stdout:9/161: symlink db/dd/d16/l42 0 2024-05-06T12:14:40.692 INFO:tasks.workunit.client.1.smithi190.stdout:5/99: read - d2/da/de/d15/f20 zero size 2024-05-06T12:14:40.693 INFO:tasks.workunit.client.1.smithi190.stdout:9/162: chown f6 161244 1 2024-05-06T12:14:40.695 INFO:tasks.workunit.client.1.smithi190.stdout:5/100: read - d2/da/de/d15/f20 zero size 2024-05-06T12:14:40.695 INFO:tasks.workunit.client.1.smithi190.stdout:9/163: read - db/dd/df/f2c zero size 2024-05-06T12:14:40.695 INFO:tasks.workunit.client.1.smithi190.stdout:8/194: dwrite d3/d8/f30 [0,4194304] 0 2024-05-06T12:14:40.696 INFO:tasks.workunit.client.1.smithi190.stdout:5/101: read d2/f9 [3559467,126084] 0 2024-05-06T12:14:40.702 INFO:tasks.workunit.client.0.smithi092.stdout:5/594: truncate d4/d59/f83 3490780 0 2024-05-06T12:14:40.705 INFO:tasks.workunit.client.0.smithi092.stdout:1/670: write d3/d10/d13/d19/f87 [811011,118439] 0 2024-05-06T12:14:40.706 INFO:tasks.workunit.client.0.smithi092.stdout:5/595: fdatasync d4/d34/d29/d54/d73/f92 0 2024-05-06T12:14:40.709 INFO:tasks.workunit.client.0.smithi092.stdout:5/596: truncate d4/d59/fd5 280758 0 2024-05-06T12:14:40.709 INFO:tasks.workunit.client.1.smithi190.stdout:9/164: creat db/d23/d3b/f43 x:0 0 0 2024-05-06T12:14:40.710 INFO:tasks.workunit.client.0.smithi092.stdout:7/781: dwrite d9/de/d1f/f69 [0,4194304] 0 2024-05-06T12:14:40.711 INFO:tasks.workunit.client.0.smithi092.stdout:1/671: fsync d3/d10/d13/d46/f55 0 2024-05-06T12:14:40.711 INFO:tasks.workunit.client.0.smithi092.stdout:5/597: fdatasync d4/d34/d28/d4e/f87 0 2024-05-06T12:14:40.711 INFO:tasks.workunit.client.1.smithi190.stdout:8/195: dwrite d3/f2e [0,4194304] 0 2024-05-06T12:14:40.714 INFO:tasks.workunit.client.1.smithi190.stdout:9/165: mknod db/d21/d22/c44 0 2024-05-06T12:14:40.714 INFO:tasks.workunit.client.0.smithi092.stdout:1/672: write d3/d10/d13/d19/fa9 [1296208,119999] 0 2024-05-06T12:14:40.714 INFO:tasks.workunit.client.1.smithi190.stdout:9/166: chown db/f26 424 1 2024-05-06T12:14:40.716 INFO:tasks.workunit.client.1.smithi190.stdout:8/196: rename d3/d8/d2b/l36 to d3/d8/l41 0 2024-05-06T12:14:40.718 INFO:tasks.workunit.client.1.smithi190.stdout:8/197: write d3/f2e [3735645,79417] 0 2024-05-06T12:14:40.720 INFO:tasks.workunit.client.0.smithi092.stdout:1/673: dread d3/d10/d13/d19/d34/f36 [0,4194304] 0 2024-05-06T12:14:40.721 INFO:tasks.workunit.client.1.smithi190.stdout:7/56: write d0/d4/fe [752271,21942] 0 2024-05-06T12:14:40.725 INFO:tasks.workunit.client.1.smithi190.stdout:8/198: rmdir d3/d35/d40 0 2024-05-06T12:14:40.726 INFO:tasks.workunit.client.1.smithi190.stdout:7/57: creat d0/d1/f17 x:0 0 0 2024-05-06T12:14:40.726 INFO:tasks.workunit.client.0.smithi092.stdout:1/674: dread d3/d10/d13/d46/f68 [0,4194304] 0 2024-05-06T12:14:40.728 INFO:tasks.workunit.client.1.smithi190.stdout:7/58: fsync d0/d1/fc 0 2024-05-06T12:14:40.730 INFO:tasks.workunit.client.0.smithi092.stdout:1/675: dread d3/f5 [0,4194304] 0 2024-05-06T12:14:40.732 INFO:tasks.workunit.client.0.smithi092.stdout:1/676: read d3/d42/d84/d4f/d58/f69 [500232,114022] 0 2024-05-06T12:14:40.733 INFO:tasks.workunit.client.1.smithi190.stdout:0/202: dwrite d9/df/f15 [0,4194304] 0 2024-05-06T12:14:40.733 INFO:tasks.workunit.client.0.smithi092.stdout:7/782: link d9/fb d9/f102 0 2024-05-06T12:14:40.734 INFO:tasks.workunit.client.0.smithi092.stdout:7/783: chown d9/fe8 129 1 2024-05-06T12:14:40.735 INFO:tasks.workunit.client.1.smithi190.stdout:8/199: dwrite d3/f2e [0,4194304] 0 2024-05-06T12:14:40.737 INFO:tasks.workunit.client.1.smithi190.stdout:8/200: chown d3/d35/f3c 5 1 2024-05-06T12:14:40.737 INFO:tasks.workunit.client.0.smithi092.stdout:1/677: getdents d3/d10/db3 0 2024-05-06T12:14:40.737 INFO:tasks.workunit.client.1.smithi190.stdout:7/59: dwrite d0/d1/fc [0,4194304] 0 2024-05-06T12:14:40.741 INFO:tasks.workunit.client.0.smithi092.stdout:7/784: dwrite d9/de/d10/d17/d99/fb3 [0,4194304] 0 2024-05-06T12:14:40.741 INFO:tasks.workunit.client.0.smithi092.stdout:5/598: truncate d4/d34/d28/d4e/f4f 498733 0 2024-05-06T12:14:40.743 INFO:tasks.workunit.client.0.smithi092.stdout:5/599: fsync d4/d34/fd0 0 2024-05-06T12:14:40.748 INFO:tasks.workunit.client.0.smithi092.stdout:1/678: link d3/d10/c49 d3/d42/cd4 0 2024-05-06T12:14:40.753 INFO:tasks.workunit.client.0.smithi092.stdout:7/785: dwrite d9/de/d10/d17/faf [0,4194304] 0 2024-05-06T12:14:40.754 INFO:tasks.workunit.client.0.smithi092.stdout:5/600: getdents d4/d34/d29/da3 0 2024-05-06T12:14:40.760 INFO:tasks.workunit.client.0.smithi092.stdout:1/679: rename d3/d8f to d3/d42/d84/d4f/dd5 0 2024-05-06T12:14:40.761 INFO:tasks.workunit.client.0.smithi092.stdout:7/786: truncate d9/de/d10/d17/d21/d2e/fa1 435466 0 2024-05-06T12:14:40.763 INFO:tasks.workunit.client.0.smithi092.stdout:7/787: stat d9/de/d10/d17/d6a/f9d 0 2024-05-06T12:14:40.763 INFO:tasks.workunit.client.0.smithi092.stdout:1/680: mkdir d3/d10/d13/d19/d57/dd6 0 2024-05-06T12:14:40.764 INFO:tasks.workunit.client.0.smithi092.stdout:5/601: dwrite d4/d59/fd1 [0,4194304] 0 2024-05-06T12:14:40.767 INFO:tasks.workunit.client.0.smithi092.stdout:1/681: fdatasync d3/d10/d13/faa 0 2024-05-06T12:14:40.773 INFO:tasks.workunit.client.0.smithi092.stdout:5/602: link d4/d34/d28/d4e/d5d/d82/d1f/d20/lb2 d4/d34/d28/d6b/ld8 0 2024-05-06T12:14:40.776 INFO:tasks.workunit.client.0.smithi092.stdout:5/603: rename d4/d5/d10/fa6 to d4/d34/d29/fd9 0 2024-05-06T12:14:40.778 INFO:tasks.workunit.client.0.smithi092.stdout:5/604: mkdir d4/d34/dda 0 2024-05-06T12:14:40.784 INFO:tasks.workunit.client.0.smithi092.stdout:5/605: dread d4/d34/d29/fd9 [0,4194304] 0 2024-05-06T12:14:40.786 INFO:tasks.workunit.client.0.smithi092.stdout:5/606: mkdir d4/d5/d10/d31/d53/ddb 0 2024-05-06T12:14:40.786 INFO:tasks.workunit.client.0.smithi092.stdout:5/607: chown d4/d5/d10/d31/d53/dcb 831730 1 2024-05-06T12:14:40.787 INFO:tasks.workunit.client.0.smithi092.stdout:5/608: readlink d4/d5/d10/da1/lbc 0 2024-05-06T12:14:40.791 INFO:tasks.workunit.client.0.smithi092.stdout:5/609: rename d4/d5/d30/c91 to d4/d34/d28/d4e/d5d/d82/d1f/cdc 0 2024-05-06T12:14:40.792 INFO:tasks.workunit.client.0.smithi092.stdout:5/610: creat d4/d59/fdd x:0 0 0 2024-05-06T12:14:40.794 INFO:tasks.workunit.client.0.smithi092.stdout:5/611: truncate d4/d34/d28/d4e/fc6 36437 0 2024-05-06T12:14:40.795 INFO:tasks.workunit.client.0.smithi092.stdout:1/682: dread d3/f2c [0,4194304] 0 2024-05-06T12:14:40.796 INFO:tasks.workunit.client.0.smithi092.stdout:1/683: fdatasync d3/f2c 0 2024-05-06T12:14:40.797 INFO:tasks.workunit.client.0.smithi092.stdout:5/612: rename d4/d34/d28/d4e/d5d/d82/d1f/d20/l60 to d4/d34/d28/d4e/d4b/dd6/lde 0 2024-05-06T12:14:40.800 INFO:tasks.workunit.client.0.smithi092.stdout:1/684: creat d3/d10/d13/d19/d21/d88/fd7 x:0 0 0 2024-05-06T12:14:40.800 INFO:tasks.workunit.client.0.smithi092.stdout:1/685: fdatasync d3/d42/f43 0 2024-05-06T12:14:40.801 INFO:tasks.workunit.client.0.smithi092.stdout:5/613: unlink d4/d34/d28/d4e/d5d/d82/d1f/d20/ca5 0 2024-05-06T12:14:40.812 INFO:tasks.workunit.client.0.smithi092.stdout:1/686: dwrite d3/d10/d13/d19/d34/f36 [0,4194304] 0 2024-05-06T12:14:40.814 INFO:tasks.workunit.client.0.smithi092.stdout:1/687: mknod d3/d10/d13/d19/d34/d89/cd8 0 2024-05-06T12:14:40.816 INFO:tasks.workunit.client.0.smithi092.stdout:1/688: stat d3/d10/d13/d19/d21/l44 0 2024-05-06T12:14:40.820 INFO:tasks.workunit.client.0.smithi092.stdout:1/689: dread d3/d10/d13/fbb [0,4194304] 0 2024-05-06T12:14:40.826 INFO:tasks.workunit.client.0.smithi092.stdout:1/690: dwrite d3/d10/d13/d19/d34/fbe [0,4194304] 0 2024-05-06T12:14:40.832 INFO:tasks.workunit.client.1.smithi190.stdout:0/203: creat d9/df/d21/f40 x:0 0 0 2024-05-06T12:14:40.832 INFO:tasks.workunit.client.0.smithi092.stdout:1/691: dwrite d3/d42/f43 [0,4194304] 0 2024-05-06T12:14:40.833 INFO:tasks.workunit.client.0.smithi092.stdout:2/489: sync 2024-05-06T12:14:40.833 INFO:tasks.workunit.client.1.smithi190.stdout:7/60: mkdir d0/d1/d18 0 2024-05-06T12:14:40.834 INFO:tasks.workunit.client.1.smithi190.stdout:0/204: symlink d9/dc/d28/l41 0 2024-05-06T12:14:40.835 INFO:tasks.workunit.client.0.smithi092.stdout:1/692: mknod d3/d42/d84/d4f/d58/d65/d81/cd9 0 2024-05-06T12:14:40.837 INFO:tasks.workunit.client.1.smithi190.stdout:7/61: creat d0/f19 x:0 0 0 2024-05-06T12:14:40.837 INFO:tasks.workunit.client.0.smithi092.stdout:1/693: write d3/d10/d13/d19/d21/d72/d9a/fd0 [904600,82905] 0 2024-05-06T12:14:40.837 INFO:tasks.workunit.client.1.smithi190.stdout:7/62: fsync d0/d4/fe 0 2024-05-06T12:14:40.838 INFO:tasks.workunit.client.1.smithi190.stdout:7/63: stat d0/ff 0 2024-05-06T12:14:40.839 INFO:tasks.workunit.client.0.smithi092.stdout:1/694: write d3/d42/d84/d4f/d58/d65/d81/f83 [4080434,99544] 0 2024-05-06T12:14:40.840 INFO:tasks.workunit.client.0.smithi092.stdout:1/695: symlink d3/d10/d13/d46/lda 0 2024-05-06T12:14:40.842 INFO:tasks.workunit.client.1.smithi190.stdout:0/205: fdatasync d9/f2c 0 2024-05-06T12:14:40.843 INFO:tasks.workunit.client.1.smithi190.stdout:0/206: truncate d9/f1b 8856346 0 2024-05-06T12:14:40.843 INFO:tasks.workunit.client.1.smithi190.stdout:7/64: rmdir d0/da 39 2024-05-06T12:14:40.843 INFO:tasks.workunit.client.1.smithi190.stdout:7/65: fdatasync d0/f19 0 2024-05-06T12:14:40.844 INFO:tasks.workunit.client.0.smithi092.stdout:1/696: dread d3/d10/d13/d19/d34/fbe [0,4194304] 0 2024-05-06T12:14:40.845 INFO:tasks.workunit.client.1.smithi190.stdout:0/207: mknod d9/df/d21/c42 0 2024-05-06T12:14:40.846 INFO:tasks.workunit.client.1.smithi190.stdout:0/208: readlink d9/dc/l22 0 2024-05-06T12:14:40.849 INFO:tasks.workunit.client.1.smithi190.stdout:0/209: symlink d9/l43 0 2024-05-06T12:14:40.851 INFO:tasks.workunit.client.0.smithi092.stdout:2/490: truncate df/f1e 659105 0 2024-05-06T12:14:40.851 INFO:tasks.workunit.client.1.smithi190.stdout:7/66: dwrite d0/ff [0,4194304] 0 2024-05-06T12:14:40.851 INFO:tasks.workunit.client.1.smithi190.stdout:0/210: chown d9/dc/c17 28144687 1 2024-05-06T12:14:40.853 INFO:tasks.workunit.client.1.smithi190.stdout:5/102: write f0 [3017879,84163] 0 2024-05-06T12:14:40.853 INFO:tasks.workunit.client.1.smithi190.stdout:7/67: write d0/d1/f11 [2093183,105175] 0 2024-05-06T12:14:40.853 INFO:tasks.workunit.client.1.smithi190.stdout:5/103: chown d2/da/l16 432477 1 2024-05-06T12:14:40.854 INFO:tasks.workunit.client.0.smithi092.stdout:2/491: creat df/d17/d7e/f91 x:0 0 0 2024-05-06T12:14:40.855 INFO:tasks.workunit.client.1.smithi190.stdout:0/211: mkdir d9/df/d21/d31/d44 0 2024-05-06T12:14:40.855 INFO:tasks.workunit.client.1.smithi190.stdout:9/167: rmdir db/d21 39 2024-05-06T12:14:40.855 INFO:tasks.workunit.client.1.smithi190.stdout:9/168: dread - db/f1f zero size 2024-05-06T12:14:40.858 INFO:tasks.workunit.client.1.smithi190.stdout:7/68: symlink d0/d14/l1a 0 2024-05-06T12:14:40.860 INFO:tasks.workunit.client.1.smithi190.stdout:7/69: chown d0/d1/fc 830 1 2024-05-06T12:14:40.860 INFO:tasks.workunit.client.1.smithi190.stdout:9/169: dread db/dd/d16/f1e [0,4194304] 0 2024-05-06T12:14:40.862 INFO:tasks.workunit.client.0.smithi092.stdout:2/492: dwrite df/d17/d7d/f86 [0,4194304] 0 2024-05-06T12:14:40.862 INFO:tasks.workunit.client.1.smithi190.stdout:0/212: creat d9/dc/f45 x:0 0 0 2024-05-06T12:14:40.865 INFO:tasks.workunit.client.1.smithi190.stdout:7/70: dread d0/d1/d2/f5 [0,4194304] 0 2024-05-06T12:14:40.866 INFO:tasks.workunit.client.1.smithi190.stdout:7/71: chown d0/l16 2 1 2024-05-06T12:14:40.869 INFO:tasks.workunit.client.1.smithi190.stdout:9/170: creat db/d21/f45 x:0 0 0 2024-05-06T12:14:40.870 INFO:tasks.workunit.client.1.smithi190.stdout:9/171: stat db/f26 0 2024-05-06T12:14:40.872 INFO:tasks.workunit.client.1.smithi190.stdout:7/72: creat d0/d1/d2/f1b x:0 0 0 2024-05-06T12:14:40.875 INFO:tasks.workunit.client.0.smithi092.stdout:2/493: write df/d17/d1d/f49 [1666801,90502] 0 2024-05-06T12:14:40.877 INFO:tasks.workunit.client.0.smithi092.stdout:2/494: fsync f5 0 2024-05-06T12:14:40.880 INFO:tasks.workunit.client.1.smithi190.stdout:9/172: dwrite db/dd/d16/f1e [4194304,4194304] 0 2024-05-06T12:14:40.880 INFO:tasks.workunit.client.0.smithi092.stdout:2/495: mknod df/d17/d1d/d33/d4d/d83/c92 0 2024-05-06T12:14:40.886 INFO:tasks.workunit.client.0.smithi092.stdout:2/496: dread df/d17/d1d/d33/f44 [0,4194304] 0 2024-05-06T12:14:40.889 INFO:tasks.workunit.client.1.smithi190.stdout:9/173: dwrite db/d23/d3b/f43 [0,4194304] 0 2024-05-06T12:14:40.889 INFO:tasks.workunit.client.1.smithi190.stdout:0/213: write d9/f2f [65232,104520] 0 2024-05-06T12:14:40.889 INFO:tasks.workunit.client.0.smithi092.stdout:2/497: link df/f55 df/d1b/d8e/f93 0 2024-05-06T12:14:40.890 INFO:tasks.workunit.client.0.smithi092.stdout:2/498: chown f5 38256 1 2024-05-06T12:14:40.890 INFO:tasks.workunit.client.1.smithi190.stdout:7/73: truncate d0/d1/d2/f5 517604 0 2024-05-06T12:14:40.890 INFO:tasks.workunit.client.1.smithi190.stdout:9/174: chown db/d21/d22 36768102 1 2024-05-06T12:14:40.891 INFO:tasks.workunit.client.0.smithi092.stdout:2/499: truncate df/d17/d51/f8c 775679 0 2024-05-06T12:14:40.895 INFO:tasks.workunit.client.1.smithi190.stdout:0/214: unlink d9/df/f10 0 2024-05-06T12:14:40.895 INFO:tasks.workunit.client.1.smithi190.stdout:1/45: sync 2024-05-06T12:14:40.895 INFO:tasks.workunit.client.1.smithi190.stdout:7/74: mknod d0/c1c 0 2024-05-06T12:14:40.898 INFO:tasks.workunit.client.1.smithi190.stdout:9/175: link db/d21/d22/l39 db/dd/d16/l46 0 2024-05-06T12:14:40.899 INFO:tasks.workunit.client.0.smithi092.stdout:2/500: dwrite df/d17/d1d/f41 [0,4194304] 0 2024-05-06T12:14:40.902 INFO:tasks.workunit.client.1.smithi190.stdout:9/176: mkdir db/d23/d25/d47 0 2024-05-06T12:14:40.905 INFO:tasks.workunit.client.1.smithi190.stdout:7/75: dwrite d0/d1/d2/f1b [0,4194304] 0 2024-05-06T12:14:40.905 INFO:tasks.workunit.client.1.smithi190.stdout:0/215: link d9/df/l32 d9/dc/l46 0 2024-05-06T12:14:40.906 INFO:tasks.workunit.client.1.smithi190.stdout:1/46: link d5/f6 d5/fb 0 2024-05-06T12:14:40.907 INFO:tasks.workunit.client.1.smithi190.stdout:9/177: mknod db/d23/d25/c48 0 2024-05-06T12:14:40.910 INFO:tasks.workunit.client.1.smithi190.stdout:9/178: chown db/d23/d3b/f43 97888 1 2024-05-06T12:14:40.911 INFO:tasks.workunit.client.1.smithi190.stdout:1/47: mknod d5/cc 0 2024-05-06T12:14:40.913 INFO:tasks.workunit.client.1.smithi190.stdout:9/179: write db/f26 [1992866,56741] 0 2024-05-06T12:14:40.914 INFO:tasks.workunit.client.1.smithi190.stdout:7/76: dread d0/d1/f11 [0,4194304] 0 2024-05-06T12:14:40.917 INFO:tasks.workunit.client.1.smithi190.stdout:0/216: mknod d9/df/d21/d31/d44/c47 0 2024-05-06T12:14:40.918 INFO:tasks.workunit.client.1.smithi190.stdout:0/217: chown d9/df/d21/f2a 2171830 1 2024-05-06T12:14:40.920 INFO:tasks.workunit.client.1.smithi190.stdout:1/48: truncate d5/f8 227665 0 2024-05-06T12:14:40.921 INFO:tasks.workunit.client.1.smithi190.stdout:7/77: creat d0/d1/d2/f1d x:0 0 0 2024-05-06T12:14:40.923 INFO:tasks.workunit.client.1.smithi190.stdout:9/180: truncate db/dd/f34 1049801 0 2024-05-06T12:14:40.924 INFO:tasks.workunit.client.1.smithi190.stdout:1/49: dread d5/f7 [0,4194304] 0 2024-05-06T12:14:40.924 INFO:tasks.workunit.client.1.smithi190.stdout:1/50: readlink - no filename 2024-05-06T12:14:40.924 INFO:tasks.workunit.client.1.smithi190.stdout:9/181: dread - db/d23/f3a zero size 2024-05-06T12:14:40.925 INFO:tasks.workunit.client.1.smithi190.stdout:7/78: write d0/ff [4176729,75759] 0 2024-05-06T12:14:40.925 INFO:tasks.workunit.client.1.smithi190.stdout:0/218: mkdir d9/dc/d48 0 2024-05-06T12:14:40.928 INFO:tasks.workunit.client.1.smithi190.stdout:9/182: mkdir db/d21/d22/d49 0 2024-05-06T12:14:40.928 INFO:tasks.workunit.client.1.smithi190.stdout:1/51: chown d5/fa 60544728 1 2024-05-06T12:14:40.929 INFO:tasks.workunit.client.1.smithi190.stdout:0/219: creat d9/df/d21/f49 x:0 0 0 2024-05-06T12:14:40.930 INFO:tasks.workunit.client.1.smithi190.stdout:9/183: read - db/d21/f45 zero size 2024-05-06T12:14:40.931 INFO:tasks.workunit.client.1.smithi190.stdout:1/52: mkdir d5/dd 0 2024-05-06T12:14:40.932 INFO:tasks.workunit.client.1.smithi190.stdout:0/220: chown d9/l3a 0 1 2024-05-06T12:14:40.934 INFO:tasks.workunit.client.1.smithi190.stdout:7/79: dwrite d0/d1/d2/f1d [0,4194304] 0 2024-05-06T12:14:40.935 INFO:tasks.workunit.client.1.smithi190.stdout:9/184: creat db/d21/f4a x:0 0 0 2024-05-06T12:14:40.936 INFO:tasks.workunit.client.0.smithi092.stdout:2/501: dread f1 [4194304,4194304] 0 2024-05-06T12:14:40.937 INFO:tasks.workunit.client.1.smithi190.stdout:1/53: creat d5/fe x:0 0 0 2024-05-06T12:14:40.938 INFO:tasks.workunit.client.0.smithi092.stdout:2/502: mknod df/d5c/c94 0 2024-05-06T12:14:40.941 INFO:tasks.workunit.client.1.smithi190.stdout:0/221: link d9/dc/f45 d9/df/d21/f4a 0 2024-05-06T12:14:40.944 INFO:tasks.workunit.client.1.smithi190.stdout:7/80: mknod d0/d15/c1e 0 2024-05-06T12:14:40.944 INFO:tasks.workunit.client.1.smithi190.stdout:0/222: write d9/df/d21/d31/f34 [347840,88620] 0 2024-05-06T12:14:40.945 INFO:tasks.workunit.client.0.smithi092.stdout:2/503: write df/d17/d1d/f21 [5170559,33742] 0 2024-05-06T12:14:40.946 INFO:tasks.workunit.client.1.smithi190.stdout:9/185: dwrite db/d23/d25/f37 [0,4194304] 0 2024-05-06T12:14:40.947 INFO:tasks.workunit.client.1.smithi190.stdout:9/186: stat db/dd/d16/l42 0 2024-05-06T12:14:40.949 INFO:tasks.workunit.client.1.smithi190.stdout:7/81: symlink d0/d4/l1f 0 2024-05-06T12:14:40.950 INFO:tasks.workunit.client.0.smithi092.stdout:2/504: dwrite df/d17/d1d/f49 [0,4194304] 0 2024-05-06T12:14:40.950 INFO:tasks.workunit.client.0.smithi092.stdout:2/505: readlink df/d17/d1d/l52 0 2024-05-06T12:14:40.951 INFO:tasks.workunit.client.1.smithi190.stdout:1/54: rmdir d5/dd 0 2024-05-06T12:14:40.954 INFO:tasks.workunit.client.1.smithi190.stdout:0/223: creat d9/df/d21/d31/d44/f4b x:0 0 0 2024-05-06T12:14:40.957 INFO:tasks.workunit.client.1.smithi190.stdout:9/187: rename db/d23/d3b to db/d23/d25/d47/d4b 0 2024-05-06T12:14:40.958 INFO:tasks.workunit.client.0.smithi092.stdout:2/506: dwrite df/d17/f2a [0,4194304] 0 2024-05-06T12:14:40.958 INFO:tasks.workunit.client.1.smithi190.stdout:9/188: fsync db/d21/f4a 0 2024-05-06T12:14:40.959 INFO:tasks.workunit.client.1.smithi190.stdout:1/55: creat d5/ff x:0 0 0 2024-05-06T12:14:40.961 INFO:tasks.workunit.client.0.smithi092.stdout:2/507: creat df/d15/f95 x:0 0 0 2024-05-06T12:14:40.961 INFO:tasks.workunit.client.1.smithi190.stdout:9/189: read db/dd/df/f1b [3030841,9898] 0 2024-05-06T12:14:40.962 INFO:tasks.workunit.client.1.smithi190.stdout:9/190: fsync db/d23/d25/d47/d4b/f43 0 2024-05-06T12:14:40.964 INFO:tasks.workunit.client.1.smithi190.stdout:9/191: write f3 [877576,125077] 0 2024-05-06T12:14:40.966 INFO:tasks.workunit.client.0.smithi092.stdout:2/508: read f1 [3733011,73018] 0 2024-05-06T12:14:40.970 INFO:tasks.workunit.client.1.smithi190.stdout:7/82: creat d0/f20 x:0 0 0 2024-05-06T12:14:40.971 INFO:tasks.workunit.client.1.smithi190.stdout:9/192: mkdir db/dd/d16/d4c 0 2024-05-06T12:14:40.974 INFO:tasks.workunit.client.1.smithi190.stdout:0/224: rename d9/df/d21/d31/d44 to d9/dc/d28/d4c 0 2024-05-06T12:14:40.976 INFO:tasks.workunit.client.1.smithi190.stdout:9/193: creat db/d23/f4d x:0 0 0 2024-05-06T12:14:40.978 INFO:tasks.workunit.client.1.smithi190.stdout:7/83: rename d0/d4/f8 to d0/d1/f21 0 2024-05-06T12:14:40.981 INFO:tasks.workunit.client.1.smithi190.stdout:0/225: getdents d9/dc/d48 0 2024-05-06T12:14:40.981 INFO:tasks.workunit.client.1.smithi190.stdout:7/84: dread d0/d1/fc [0,4194304] 0 2024-05-06T12:14:40.982 INFO:tasks.workunit.client.1.smithi190.stdout:1/56: chown d5/f8 10 1 2024-05-06T12:14:40.982 INFO:tasks.workunit.client.1.smithi190.stdout:1/57: readlink - no filename 2024-05-06T12:14:40.983 INFO:tasks.workunit.client.1.smithi190.stdout:9/194: mkdir db/dd/d16/d4c/d4e 0 2024-05-06T12:14:40.985 INFO:tasks.workunit.client.1.smithi190.stdout:7/85: write d0/d1/fb [3504739,112962] 0 2024-05-06T12:14:40.985 INFO:tasks.workunit.client.1.smithi190.stdout:1/58: symlink d5/l10 0 2024-05-06T12:14:40.988 INFO:tasks.workunit.client.1.smithi190.stdout:9/195: creat db/dd/d16/f4f x:0 0 0 2024-05-06T12:14:40.989 INFO:tasks.workunit.client.1.smithi190.stdout:1/59: mknod d5/c11 0 2024-05-06T12:14:40.990 INFO:tasks.workunit.client.1.smithi190.stdout:1/60: stat d5/f8 0 2024-05-06T12:14:40.991 INFO:tasks.workunit.client.1.smithi190.stdout:1/61: write d5/fe [419527,20685] 0 2024-05-06T12:14:40.991 INFO:tasks.workunit.client.1.smithi190.stdout:0/226: rename d9/dc/f20 to d9/dc/f4d 0 2024-05-06T12:14:40.991 INFO:tasks.workunit.client.1.smithi190.stdout:1/62: readlink d5/l10 0 2024-05-06T12:14:40.995 INFO:tasks.workunit.client.1.smithi190.stdout:9/196: symlink db/d23/l50 0 2024-05-06T12:14:40.995 INFO:tasks.workunit.client.1.smithi190.stdout:7/86: creat d0/f22 x:0 0 0 2024-05-06T12:14:40.996 INFO:tasks.workunit.client.1.smithi190.stdout:0/227: creat d9/dc/d48/f4e x:0 0 0 2024-05-06T12:14:40.997 INFO:tasks.workunit.client.1.smithi190.stdout:9/197: creat db/d21/d22/f51 x:0 0 0 2024-05-06T12:14:41.000 INFO:tasks.workunit.client.1.smithi190.stdout:7/87: dread d0/d1/d2/f5 [0,4194304] 0 2024-05-06T12:14:41.000 INFO:tasks.workunit.client.1.smithi190.stdout:0/228: unlink d9/df/f15 0 2024-05-06T12:14:41.004 INFO:tasks.workunit.client.1.smithi190.stdout:0/229: read d9/df/d21/d31/f34 [312083,89198] 0 2024-05-06T12:14:41.004 INFO:tasks.workunit.client.1.smithi190.stdout:9/198: dwrite db/d21/f3c [0,4194304] 0 2024-05-06T12:14:41.006 INFO:tasks.workunit.client.1.smithi190.stdout:1/63: getdents d5 0 2024-05-06T12:14:41.006 INFO:tasks.workunit.client.1.smithi190.stdout:0/230: chown d9/dc/d48/f4e 62461680 1 2024-05-06T12:14:41.006 INFO:tasks.workunit.client.1.smithi190.stdout:9/199: chown db/dd/c1d 6361 1 2024-05-06T12:14:41.007 INFO:tasks.workunit.client.1.smithi190.stdout:0/231: dread - d9/dc/d28/d4c/f4b zero size 2024-05-06T12:14:41.008 INFO:tasks.workunit.client.1.smithi190.stdout:7/88: creat d0/da/f23 x:0 0 0 2024-05-06T12:14:41.008 INFO:tasks.workunit.client.1.smithi190.stdout:0/232: chown d9/l3a 1015807 1 2024-05-06T12:14:41.009 INFO:tasks.workunit.client.1.smithi190.stdout:1/64: rmdir d5 39 2024-05-06T12:14:41.013 INFO:tasks.workunit.client.0.smithi092.stdout:2/509: mkdir df/d17/d51/d96 0 2024-05-06T12:14:41.013 INFO:tasks.workunit.client.0.smithi092.stdout:8/746: sync 2024-05-06T12:14:41.013 INFO:tasks.workunit.client.0.smithi092.stdout:9/638: sync 2024-05-06T12:14:41.013 INFO:tasks.workunit.client.0.smithi092.stdout:0/876: sync 2024-05-06T12:14:41.014 INFO:tasks.workunit.client.1.smithi190.stdout:9/200: dwrite f5 [0,4194304] 0 2024-05-06T12:14:41.015 INFO:tasks.workunit.client.0.smithi092.stdout:2/510: chown df/d15/d88/d59/f65 0 1 2024-05-06T12:14:41.015 INFO:tasks.workunit.client.0.smithi092.stdout:9/639: write d16/d1e/f86 [402297,125909] 0 2024-05-06T12:14:41.015 INFO:tasks.workunit.client.1.smithi190.stdout:9/201: chown la 2698 1 2024-05-06T12:14:41.016 INFO:tasks.workunit.client.0.smithi092.stdout:2/511: chown df/d15/d88/d59/l7f 67 1 2024-05-06T12:14:41.016 INFO:tasks.workunit.client.1.smithi190.stdout:7/89: rename d0/da/f23 to d0/d1/f24 0 2024-05-06T12:14:41.017 INFO:tasks.workunit.client.0.smithi092.stdout:9/640: dread - d16/d2c/d2f/d3d/d33/d6b/d9e/fba zero size 2024-05-06T12:14:41.017 INFO:tasks.workunit.client.1.smithi190.stdout:1/65: truncate d5/f8 949563 0 2024-05-06T12:14:41.018 INFO:tasks.workunit.client.0.smithi092.stdout:8/747: symlink d9/d9d/da6/l10e 0 2024-05-06T12:14:41.019 INFO:tasks.workunit.client.0.smithi092.stdout:8/748: chown d9/dc/d4a/f97 44 1 2024-05-06T12:14:41.019 INFO:tasks.workunit.client.1.smithi190.stdout:1/66: chown d5/f7 3913 1 2024-05-06T12:14:41.019 INFO:tasks.workunit.client.1.smithi190.stdout:0/233: dwrite d9/fb [0,4194304] 0 2024-05-06T12:14:41.022 INFO:tasks.workunit.client.0.smithi092.stdout:9/641: rename d16/d1e/da1/d43 to d16/d1e/d7b/dce 0 2024-05-06T12:14:41.023 INFO:tasks.workunit.client.0.smithi092.stdout:9/642: chown d16/d2c/d4b/cac 9019 1 2024-05-06T12:14:41.024 INFO:tasks.workunit.client.1.smithi190.stdout:7/90: dread d0/d1/f11 [0,4194304] 0 2024-05-06T12:14:41.024 INFO:tasks.workunit.client.0.smithi092.stdout:8/749: mkdir d9/d9d/d10f 0 2024-05-06T12:14:41.025 INFO:tasks.workunit.client.1.smithi190.stdout:9/202: rename db/f11 to db/f52 0 2024-05-06T12:14:41.025 INFO:tasks.workunit.client.1.smithi190.stdout:7/91: write d0/d1/f17 [929558,57520] 0 2024-05-06T12:14:41.027 INFO:tasks.workunit.client.1.smithi190.stdout:7/92: fsync d0/f19 0 2024-05-06T12:14:41.028 INFO:tasks.workunit.client.0.smithi092.stdout:8/750: readlink d9/d21/d37/d46/l89 0 2024-05-06T12:14:41.030 INFO:tasks.workunit.client.0.smithi092.stdout:9/643: creat d16/d2c/d2f/db1/fcf x:0 0 0 2024-05-06T12:14:41.032 INFO:tasks.workunit.client.0.smithi092.stdout:2/512: truncate f1 6093646 0 2024-05-06T12:14:41.034 INFO:tasks.workunit.client.0.smithi092.stdout:0/877: dwrite d7/dc/d28/d64/fee [0,4194304] 0 2024-05-06T12:14:41.034 INFO:tasks.workunit.client.0.smithi092.stdout:8/751: chown d9/d21/l3a 150 1 2024-05-06T12:14:41.034 INFO:tasks.workunit.client.0.smithi092.stdout:2/513: chown df/d5c/f66 89392 1 2024-05-06T12:14:41.038 INFO:tasks.workunit.client.0.smithi092.stdout:9/644: creat d16/d2c/d2f/d3d/d33/d6b/dbe/fd0 x:0 0 0 2024-05-06T12:14:41.040 INFO:tasks.workunit.client.0.smithi092.stdout:2/514: unlink df/f11 0 2024-05-06T12:14:41.043 INFO:tasks.workunit.client.0.smithi092.stdout:8/752: dwrite d9/dc/fc8 [0,4194304] 0 2024-05-06T12:14:41.043 INFO:tasks.workunit.client.0.smithi092.stdout:9/645: symlink d16/d2c/d4b/ld1 0 2024-05-06T12:14:41.046 INFO:tasks.workunit.client.0.smithi092.stdout:0/878: link d7/dc/l8a d7/dc/d28/d54/d12/d8f/l125 0 2024-05-06T12:14:41.050 INFO:tasks.workunit.client.0.smithi092.stdout:9/646: write d16/d2c/d2f/d3d/d33/f56 [1226706,14440] 0 2024-05-06T12:14:41.050 INFO:tasks.workunit.client.0.smithi092.stdout:2/515: dwrite df/d17/d51/f57 [0,4194304] 0 2024-05-06T12:14:41.052 INFO:tasks.workunit.client.1.smithi190.stdout:6/51: sync 2024-05-06T12:14:41.053 INFO:tasks.workunit.client.1.smithi190.stdout:2/80: sync 2024-05-06T12:14:41.053 INFO:tasks.workunit.client.1.smithi190.stdout:2/81: readlink d2/l9 0 2024-05-06T12:14:41.054 INFO:tasks.workunit.client.1.smithi190.stdout:6/52: truncate fc 1634172 0 2024-05-06T12:14:41.054 INFO:tasks.workunit.client.0.smithi092.stdout:9/647: creat d16/d1e/d7b/d8d/da4/fd2 x:0 0 0 2024-05-06T12:14:41.058 INFO:tasks.workunit.client.0.smithi092.stdout:2/516: creat df/d17/d51/d96/f97 x:0 0 0 2024-05-06T12:14:41.058 INFO:tasks.workunit.client.0.smithi092.stdout:9/648: fdatasync d16/d2c/d4b/d76/fc3 0 2024-05-06T12:14:41.059 INFO:tasks.workunit.client.1.smithi190.stdout:0/234: link d9/df/d21/d31/f34 d9/df/d21/f4f 0 2024-05-06T12:14:41.060 INFO:tasks.workunit.client.0.smithi092.stdout:2/517: write df/f61 [482898,111881] 0 2024-05-06T12:14:41.060 INFO:tasks.workunit.client.0.smithi092.stdout:9/649: fsync d16/d1e/d7b/dce/f89 0 2024-05-06T12:14:41.063 INFO:tasks.workunit.client.1.smithi190.stdout:2/82: dwrite d2/fd [4194304,4194304] 0 2024-05-06T12:14:41.063 INFO:tasks.workunit.client.0.smithi092.stdout:9/650: symlink d16/d2c/d2f/d3d/d33/d57/ld3 0 2024-05-06T12:14:41.064 INFO:tasks.workunit.client.1.smithi190.stdout:2/83: chown d2/l9 2057 1 2024-05-06T12:14:41.065 INFO:tasks.workunit.client.1.smithi190.stdout:1/67: truncate d5/f8 579020 0 2024-05-06T12:14:41.065 INFO:tasks.workunit.client.1.smithi190.stdout:0/235: mkdir d9/dc/d48/d50 0 2024-05-06T12:14:41.066 INFO:tasks.workunit.client.0.smithi092.stdout:2/518: link df/d17/d1d/d33/f68 df/d17/d53/d85/f98 0 2024-05-06T12:14:41.066 INFO:tasks.workunit.client.1.smithi190.stdout:6/53: mknod df/d10/c11 0 2024-05-06T12:14:41.066 INFO:tasks.workunit.client.1.smithi190.stdout:0/236: truncate d9/f1a 523898 0 2024-05-06T12:14:41.067 INFO:tasks.workunit.client.1.smithi190.stdout:0/237: chown d9/df/d21/c42 21153 1 2024-05-06T12:14:41.068 INFO:tasks.workunit.client.1.smithi190.stdout:9/203: link db/d21/f4a db/d23/f53 0 2024-05-06T12:14:41.069 INFO:tasks.workunit.client.0.smithi092.stdout:2/519: symlink df/d5c/d60/l99 0 2024-05-06T12:14:41.069 INFO:tasks.workunit.client.1.smithi190.stdout:1/68: dread d5/fa [0,4194304] 0 2024-05-06T12:14:41.069 INFO:tasks.workunit.client.1.smithi190.stdout:9/204: readlink db/dd/df/l28 0 2024-05-06T12:14:41.070 INFO:tasks.workunit.client.1.smithi190.stdout:2/84: rmdir d2/d8 39 2024-05-06T12:14:41.070 INFO:tasks.workunit.client.0.smithi092.stdout:2/520: stat df/d17/d1d/d33/d3c/f8b 0 2024-05-06T12:14:41.070 INFO:tasks.workunit.client.0.smithi092.stdout:9/651: dwrite d16/d1e/f2e [4194304,4194304] 0 2024-05-06T12:14:41.071 INFO:tasks.workunit.client.1.smithi190.stdout:9/205: write db/d23/d25/f37 [1494511,79052] 0 2024-05-06T12:14:41.072 INFO:tasks.workunit.client.1.smithi190.stdout:6/54: mkdir df/d12 0 2024-05-06T12:14:41.073 INFO:tasks.workunit.client.1.smithi190.stdout:1/69: dread d5/fe [0,4194304] 0 2024-05-06T12:14:41.074 INFO:tasks.workunit.client.0.smithi092.stdout:2/521: symlink df/d17/d1d/d33/d3c/l9a 0 2024-05-06T12:14:41.076 INFO:tasks.workunit.client.0.smithi092.stdout:2/522: mkdir df/d17/d7d/d9b 0 2024-05-06T12:14:41.077 INFO:tasks.workunit.client.0.smithi092.stdout:2/523: chown df/d17/d1d/d33/f68 118 1 2024-05-06T12:14:41.078 INFO:tasks.workunit.client.1.smithi190.stdout:9/206: readlink db/dd/d16/l46 0 2024-05-06T12:14:41.078 INFO:tasks.workunit.client.1.smithi190.stdout:0/238: mkdir d9/d51 0 2024-05-06T12:14:41.079 INFO:tasks.workunit.client.1.smithi190.stdout:2/85: dread d2/d8/dc/f13 [0,4194304] 0 2024-05-06T12:14:41.080 INFO:tasks.workunit.client.0.smithi092.stdout:9/652: dread d16/d2c/f91 [0,4194304] 0 2024-05-06T12:14:41.081 INFO:tasks.workunit.client.1.smithi190.stdout:2/86: readlink l1 0 2024-05-06T12:14:41.081 INFO:tasks.workunit.client.1.smithi190.stdout:0/239: truncate d9/df/d21/f4a 790321 0 2024-05-06T12:14:41.083 INFO:tasks.workunit.client.1.smithi190.stdout:6/55: creat df/d12/f13 x:0 0 0 2024-05-06T12:14:41.083 INFO:tasks.workunit.client.0.smithi092.stdout:2/524: dwrite df/f61 [0,4194304] 0 2024-05-06T12:14:41.085 INFO:tasks.workunit.client.1.smithi190.stdout:9/207: creat db/d21/d22/f54 x:0 0 0 2024-05-06T12:14:41.087 INFO:tasks.workunit.client.1.smithi190.stdout:6/56: write fb [1693937,24691] 0 2024-05-06T12:14:41.087 INFO:tasks.workunit.client.1.smithi190.stdout:0/240: write d9/df/d21/f2a [9030403,39861] 0 2024-05-06T12:14:41.088 INFO:tasks.workunit.client.1.smithi190.stdout:0/241: chown d9/d51 497331592 1 2024-05-06T12:14:41.089 INFO:tasks.workunit.client.1.smithi190.stdout:6/57: write f7 [4947576,20858] 0 2024-05-06T12:14:41.089 INFO:tasks.workunit.client.1.smithi190.stdout:2/87: unlink d2/d8/dc/lf 0 2024-05-06T12:14:41.092 INFO:tasks.workunit.client.1.smithi190.stdout:9/208: write db/dd/d16/f24 [22445,41629] 0 2024-05-06T12:14:41.093 INFO:tasks.workunit.client.1.smithi190.stdout:1/70: dwrite d5/fe [0,4194304] 0 2024-05-06T12:14:41.094 INFO:tasks.workunit.client.0.smithi092.stdout:9/653: rmdir d16/d2c/d2f/d3d/d33/d7e/dad/db6 0 2024-05-06T12:14:41.094 INFO:tasks.workunit.client.1.smithi190.stdout:1/71: dread - d5/ff zero size 2024-05-06T12:14:41.095 INFO:tasks.workunit.client.1.smithi190.stdout:1/72: stat d5/fa 0 2024-05-06T12:14:41.096 INFO:tasks.workunit.client.0.smithi092.stdout:9/654: write d16/d1c/d58/d92/fa8 [773434,85650] 0 2024-05-06T12:14:41.099 INFO:tasks.workunit.client.0.smithi092.stdout:2/525: dwrite df/d17/d1d/f21 [0,4194304] 0 2024-05-06T12:14:41.101 INFO:tasks.workunit.client.0.smithi092.stdout:2/526: chown fd 7 1 2024-05-06T12:14:41.101 INFO:tasks.workunit.client.1.smithi190.stdout:9/209: creat db/d21/d22/f55 x:0 0 0 2024-05-06T12:14:41.105 INFO:tasks.workunit.client.0.smithi092.stdout:8/753: dwrite d9/d21/d37/d46/d5c/d62/da5/de6/f101 [0,4194304] 0 2024-05-06T12:14:41.108 INFO:tasks.workunit.client.1.smithi190.stdout:0/242: creat d9/f52 x:0 0 0 2024-05-06T12:14:41.110 INFO:tasks.workunit.client.1.smithi190.stdout:9/210: creat db/d21/d22/f56 x:0 0 0 2024-05-06T12:14:41.110 INFO:tasks.workunit.client.0.smithi092.stdout:0/879: write d7/dc/d97/dc9/f10a [600680,18728] 0 2024-05-06T12:14:41.112 INFO:tasks.workunit.client.1.smithi190.stdout:2/88: creat d2/f14 x:0 0 0 2024-05-06T12:14:41.113 INFO:tasks.workunit.client.1.smithi190.stdout:2/89: readlink d2/l9 0 2024-05-06T12:14:41.114 INFO:tasks.workunit.client.1.smithi190.stdout:0/243: creat d9/dc/d28/d4c/f53 x:0 0 0 2024-05-06T12:14:41.115 INFO:tasks.workunit.client.1.smithi190.stdout:9/211: creat db/dd/d16/d4c/f57 x:0 0 0 2024-05-06T12:14:41.118 INFO:tasks.workunit.client.1.smithi190.stdout:6/58: getdents df/d12 0 2024-05-06T12:14:41.118 INFO:tasks.workunit.client.1.smithi190.stdout:2/90: dread d2/fd [4194304,4194304] 0 2024-05-06T12:14:41.120 INFO:tasks.workunit.client.0.smithi092.stdout:0/880: dread d7/dc/d28/d54/d1e/d26/d3a/f10e [0,4194304] 0 2024-05-06T12:14:41.121 INFO:tasks.workunit.client.1.smithi190.stdout:9/212: unlink db/dd/df/l1a 0 2024-05-06T12:14:41.122 INFO:tasks.workunit.client.1.smithi190.stdout:2/91: write d2/fa [905375,122502] 0 2024-05-06T12:14:41.125 INFO:tasks.workunit.client.0.smithi092.stdout:9/655: dread d16/d1e/f2e [0,4194304] 0 2024-05-06T12:14:41.127 INFO:tasks.workunit.client.0.smithi092.stdout:8/754: rmdir d9/d21/d37/d46/d5c/d62/da5 39 2024-05-06T12:14:41.127 INFO:tasks.workunit.client.1.smithi190.stdout:0/244: dwrite d9/df/d21/f40 [0,4194304] 0 2024-05-06T12:14:41.128 INFO:tasks.workunit.client.1.smithi190.stdout:2/92: creat d2/d8/f15 x:0 0 0 2024-05-06T12:14:41.129 INFO:tasks.workunit.client.1.smithi190.stdout:0/245: write d9/f26 [4595727,13466] 0 2024-05-06T12:14:41.130 INFO:tasks.workunit.client.1.smithi190.stdout:9/213: mkdir db/dd/d16/d58 0 2024-05-06T12:14:41.132 INFO:tasks.workunit.client.0.smithi092.stdout:9/656: rmdir d16/d2c/d2f/d3d/d33/d6b 39 2024-05-06T12:14:41.134 INFO:tasks.workunit.client.1.smithi190.stdout:6/59: link fa df/f14 0 2024-05-06T12:14:41.136 INFO:tasks.workunit.client.0.smithi092.stdout:8/755: creat d9/d21/dc0/df2/f110 x:0 0 0 2024-05-06T12:14:41.139 INFO:tasks.workunit.client.1.smithi190.stdout:0/246: dwrite d9/fb [0,4194304] 0 2024-05-06T12:14:41.140 INFO:tasks.workunit.client.0.smithi092.stdout:9/657: mkdir d16/d1c/dd4 0 2024-05-06T12:14:41.143 INFO:tasks.workunit.client.1.smithi190.stdout:9/214: rename db/dd/d16/l46 to db/d23/d25/d47/l59 0 2024-05-06T12:14:41.146 INFO:tasks.workunit.client.1.smithi190.stdout:6/60: creat df/f15 x:0 0 0 2024-05-06T12:14:41.147 INFO:tasks.workunit.client.0.smithi092.stdout:8/756: creat d9/d21/d37/d46/d5c/d62/da5/de6/f111 x:0 0 0 2024-05-06T12:14:41.148 INFO:tasks.workunit.client.1.smithi190.stdout:0/247: symlink d9/dc/d28/d4c/l54 0 2024-05-06T12:14:41.149 INFO:tasks.workunit.client.1.smithi190.stdout:9/215: dread db/dd/f41 [0,4194304] 0 2024-05-06T12:14:41.150 INFO:tasks.workunit.client.0.smithi092.stdout:9/658: link d16/d1e/d7b/dce/f8c d16/d2c/d2f/d3d/d33/d7e/fd5 0 2024-05-06T12:14:41.150 INFO:tasks.workunit.client.0.smithi092.stdout:8/757: dread - d9/d21/d37/d74/ff0 zero size 2024-05-06T12:14:41.150 INFO:tasks.workunit.client.0.smithi092.stdout:2/527: dwrite f1 [4194304,4194304] 0 2024-05-06T12:14:41.150 INFO:tasks.workunit.client.0.smithi092.stdout:2/528: dread - df/d15/f36 zero size 2024-05-06T12:14:41.151 INFO:tasks.workunit.client.1.smithi190.stdout:9/216: write db/d23/d25/f32 [13781,45814] 0 2024-05-06T12:14:41.152 INFO:tasks.workunit.client.0.smithi092.stdout:9/659: creat d16/d2c/d3f/fd6 x:0 0 0 2024-05-06T12:14:41.154 INFO:tasks.workunit.client.0.smithi092.stdout:8/758: mknod d9/d21/d54/d79/d8a/c112 0 2024-05-06T12:14:41.156 INFO:tasks.workunit.client.0.smithi092.stdout:9/660: creat d16/d1c/d58/d92/dab/fd7 x:0 0 0 2024-05-06T12:14:41.157 INFO:tasks.workunit.client.0.smithi092.stdout:8/759: dread d9/fb6 [0,4194304] 0 2024-05-06T12:14:41.157 INFO:tasks.workunit.client.0.smithi092.stdout:0/881: write d7/dc/d28/d54/d1e/f3e [1829250,18884] 0 2024-05-06T12:14:41.159 INFO:tasks.workunit.client.0.smithi092.stdout:8/760: fsync d9/dc/fc8 0 2024-05-06T12:14:41.159 INFO:tasks.workunit.client.0.smithi092.stdout:2/529: dwrite f6 [0,4194304] 0 2024-05-06T12:14:41.163 INFO:tasks.workunit.client.0.smithi092.stdout:9/661: unlink d16/d2c/d2f/d3d/d33/d57/f59 0 2024-05-06T12:14:41.163 INFO:tasks.workunit.client.1.smithi190.stdout:9/217: dwrite db/dd/df/f2c [0,4194304] 0 2024-05-06T12:14:41.164 INFO:tasks.workunit.client.1.smithi190.stdout:0/248: rmdir d9/dc/d28/d4c 39 2024-05-06T12:14:41.166 INFO:tasks.workunit.client.1.smithi190.stdout:0/249: fdatasync d9/dc/d28/f3e 0 2024-05-06T12:14:41.166 INFO:tasks.workunit.client.0.smithi092.stdout:0/882: unlink d7/dc/d28/d54/d12/d7f/db2/dbc/fc8 0 2024-05-06T12:14:41.167 INFO:tasks.workunit.client.0.smithi092.stdout:9/662: mknod d16/cd8 0 2024-05-06T12:14:41.167 INFO:tasks.workunit.client.0.smithi092.stdout:0/883: stat d7/dc/d28/d54/d1e/d26/d96 0 2024-05-06T12:14:41.169 INFO:tasks.workunit.client.0.smithi092.stdout:2/530: creat df/d17/d51/f9c x:0 0 0 2024-05-06T12:14:41.172 INFO:tasks.workunit.client.0.smithi092.stdout:9/663: mkdir d16/d1c/dd4/dd9 0 2024-05-06T12:14:41.175 INFO:tasks.workunit.client.0.smithi092.stdout:2/531: fdatasync df/d1b/d8e/f93 0 2024-05-06T12:14:41.178 INFO:tasks.workunit.client.0.smithi092.stdout:9/664: mkdir d16/d1c/d58/d92/dda 0 2024-05-06T12:14:41.178 INFO:tasks.workunit.client.0.smithi092.stdout:8/761: dwrite d9/d21/d54/f63 [4194304,4194304] 0 2024-05-06T12:14:41.179 INFO:tasks.workunit.client.0.smithi092.stdout:0/884: getdents d7/dc/d28/d54/d1e/d26/d3a 0 2024-05-06T12:14:41.181 INFO:tasks.workunit.client.0.smithi092.stdout:8/762: chown d9/d21/d37/l73 1815 1 2024-05-06T12:14:41.182 INFO:tasks.workunit.client.0.smithi092.stdout:0/885: symlink d7/dc/d28/d54/d1e/d46/df3/l126 0 2024-05-06T12:14:41.182 INFO:tasks.workunit.client.0.smithi092.stdout:8/763: write d9/dc/d4a/d5f/ff7 [500402,38675] 0 2024-05-06T12:14:41.184 INFO:tasks.workunit.client.0.smithi092.stdout:9/665: truncate d16/d2c/d3f/f60 3823479 0 2024-05-06T12:14:41.185 INFO:tasks.workunit.client.0.smithi092.stdout:8/764: symlink d9/d21/d37/d46/d5c/dfc/l113 0 2024-05-06T12:14:41.185 INFO:tasks.workunit.client.0.smithi092.stdout:8/765: chown d9/d21/d72 112756 1 2024-05-06T12:14:41.187 INFO:tasks.workunit.client.0.smithi092.stdout:8/766: chown d9/d9d/dfd 436 1 2024-05-06T12:14:41.190 INFO:tasks.workunit.client.0.smithi092.stdout:0/886: link d7/dc/d28/d54/d12/d8f/dad/f114 d7/dc/d6f/da5/dc6/f127 0 2024-05-06T12:14:41.193 INFO:tasks.workunit.client.0.smithi092.stdout:0/887: creat d7/dc/d31/de6/f128 x:0 0 0 2024-05-06T12:14:41.195 INFO:tasks.workunit.client.0.smithi092.stdout:8/767: dwrite d9/d21/d37/d46/d5c/d62/da5/fe0 [0,4194304] 0 2024-05-06T12:14:41.196 INFO:tasks.workunit.client.1.smithi190.stdout:4/52: sync 2024-05-06T12:14:41.197 INFO:tasks.workunit.client.0.smithi092.stdout:0/888: creat d7/dc/d28/d54/d1e/d46/df0/f129 x:0 0 0 2024-05-06T12:14:41.197 INFO:tasks.workunit.client.0.smithi092.stdout:8/768: write d9/dc/f5d [1505872,1738] 0 2024-05-06T12:14:41.198 INFO:tasks.workunit.client.0.smithi092.stdout:0/889: readlink d7/dc/d28/d54/d1e/d39/l5b 0 2024-05-06T12:14:41.199 INFO:tasks.workunit.client.1.smithi190.stdout:6/61: dwrite df/f14 [0,4194304] 0 2024-05-06T12:14:41.201 INFO:tasks.workunit.client.1.smithi190.stdout:1/73: dread d5/f8 [0,4194304] 0 2024-05-06T12:14:41.201 INFO:tasks.workunit.client.0.smithi092.stdout:0/890: rename d7/f88 to d7/dc/d6f/da5/dc6/d117/f12a 0 2024-05-06T12:14:41.203 INFO:tasks.workunit.client.0.smithi092.stdout:0/891: read - d7/dc/d28/d54/d12/fd8 zero size 2024-05-06T12:14:41.206 INFO:tasks.workunit.client.1.smithi190.stdout:9/218: dwrite db/d21/d22/f56 [0,4194304] 0 2024-05-06T12:14:41.208 INFO:tasks.workunit.client.1.smithi190.stdout:0/250: dread d9/df/d21/f40 [0,4194304] 0 2024-05-06T12:14:41.210 INFO:tasks.workunit.client.0.smithi092.stdout:0/892: link d7/dc/d6f/l81 d7/l12b 0 2024-05-06T12:14:41.211 INFO:tasks.workunit.client.1.smithi190.stdout:9/219: write f5 [803939,69285] 0 2024-05-06T12:14:41.214 INFO:tasks.workunit.client.1.smithi190.stdout:1/74: rename d5/ff to d5/f12 0 2024-05-06T12:14:41.214 INFO:tasks.workunit.client.1.smithi190.stdout:4/53: stat d2/d10 0 2024-05-06T12:14:41.216 INFO:tasks.workunit.client.0.smithi092.stdout:2/532: dread fe [0,4194304] 0 2024-05-06T12:14:41.216 INFO:tasks.workunit.client.0.smithi092.stdout:9/666: rmdir d16/d2c/d3f 39 2024-05-06T12:14:41.216 INFO:tasks.workunit.client.1.smithi190.stdout:1/75: dread d5/f8 [0,4194304] 0 2024-05-06T12:14:41.217 INFO:tasks.workunit.client.0.smithi092.stdout:2/533: write f1 [8410339,10009] 0 2024-05-06T12:14:41.218 INFO:tasks.workunit.client.1.smithi190.stdout:9/220: read db/d23/d25/f37 [2347822,20583] 0 2024-05-06T12:14:41.218 INFO:tasks.workunit.client.1.smithi190.stdout:0/251: creat d9/d51/f55 x:0 0 0 2024-05-06T12:14:41.220 INFO:tasks.workunit.client.1.smithi190.stdout:0/252: chown d9/df/d21/f49 1953 1 2024-05-06T12:14:41.222 INFO:tasks.workunit.client.0.smithi092.stdout:2/534: write df/d17/f6b [2142314,48483] 0 2024-05-06T12:14:41.227 INFO:tasks.workunit.client.0.smithi092.stdout:9/667: dwrite d16/d1e/d7b/dce/f51 [0,4194304] 0 2024-05-06T12:14:41.228 INFO:tasks.workunit.client.1.smithi190.stdout:9/221: creat db/f5a x:0 0 0 2024-05-06T12:14:41.231 INFO:tasks.workunit.client.1.smithi190.stdout:1/76: dwrite d5/f7 [4194304,4194304] 0 2024-05-06T12:14:41.232 INFO:tasks.workunit.client.0.smithi092.stdout:9/668: creat d16/d1e/da1/d78/db4/fdb x:0 0 0 2024-05-06T12:14:41.232 INFO:tasks.workunit.client.0.smithi092.stdout:2/535: dwrite df/d17/d1d/d33/f81 [0,4194304] 0 2024-05-06T12:14:41.234 INFO:tasks.workunit.client.0.smithi092.stdout:9/669: write d16/d2c/d2f/d3d/d33/f56 [4919178,121284] 0 2024-05-06T12:14:41.235 INFO:tasks.workunit.client.1.smithi190.stdout:9/222: dread db/d21/f3c [0,4194304] 0 2024-05-06T12:14:41.236 INFO:tasks.workunit.client.1.smithi190.stdout:1/77: fsync d5/fa 0 2024-05-06T12:14:41.236 INFO:tasks.workunit.client.0.smithi092.stdout:2/536: creat df/d17/d1d/d33/d4d/d83/f9d x:0 0 0 2024-05-06T12:14:41.238 INFO:tasks.workunit.client.0.smithi092.stdout:9/670: mkdir d16/d94/ddc 0 2024-05-06T12:14:41.240 INFO:tasks.workunit.client.0.smithi092.stdout:2/537: creat df/d15/d88/d59/f9e x:0 0 0 2024-05-06T12:14:41.240 INFO:tasks.workunit.client.1.smithi190.stdout:1/78: dread d5/fe [0,4194304] 0 2024-05-06T12:14:41.241 INFO:tasks.workunit.client.1.smithi190.stdout:9/223: creat db/d23/f5b x:0 0 0 2024-05-06T12:14:41.242 INFO:tasks.workunit.client.1.smithi190.stdout:1/79: truncate d5/f8 1228235 0 2024-05-06T12:14:41.243 INFO:tasks.workunit.client.1.smithi190.stdout:9/224: write db/dd/d16/d4c/f57 [588458,73957] 0 2024-05-06T12:14:41.244 INFO:tasks.workunit.client.1.smithi190.stdout:1/80: chown d5/f8 167021978 1 2024-05-06T12:14:41.244 INFO:tasks.workunit.client.0.smithi092.stdout:2/538: write df/d17/d1d/f4e [4911067,32696] 0 2024-05-06T12:14:41.247 INFO:tasks.workunit.client.1.smithi190.stdout:9/225: mkdir db/dd/d5c 0 2024-05-06T12:14:41.248 INFO:tasks.workunit.client.0.smithi092.stdout:9/671: dwrite d16/d94/da0/da7/fb5 [0,4194304] 0 2024-05-06T12:14:41.255 INFO:tasks.workunit.client.1.smithi190.stdout:4/54: write d2/f4 [954311,51488] 0 2024-05-06T12:14:41.255 INFO:tasks.workunit.client.0.smithi092.stdout:9/672: read d16/d2c/d2f/d3d/d33/d6b/f77 [2815678,118452] 0 2024-05-06T12:14:41.256 INFO:tasks.workunit.client.0.smithi092.stdout:2/539: dwrite df/d17/d1d/d33/f44 [0,4194304] 0 2024-05-06T12:14:41.258 INFO:tasks.workunit.client.0.smithi092.stdout:9/673: chown d16/d2c/d2f/d3d/d33/fc5 21 1 2024-05-06T12:14:41.259 INFO:tasks.workunit.client.0.smithi092.stdout:2/540: write df/d17/d1d/d33/d3c/f8b [1468622,74372] 0 2024-05-06T12:14:41.259 INFO:tasks.workunit.client.1.smithi190.stdout:9/226: dwrite f6 [0,4194304] 0 2024-05-06T12:14:41.263 INFO:tasks.workunit.client.0.smithi092.stdout:2/541: mknod df/d15/d88/c9f 0 2024-05-06T12:14:41.263 INFO:tasks.workunit.client.0.smithi092.stdout:0/893: dwrite f6 [0,4194304] 0 2024-05-06T12:14:41.266 INFO:tasks.workunit.client.0.smithi092.stdout:2/542: write df/d17/d51/f8c [851890,113035] 0 2024-05-06T12:14:41.266 INFO:tasks.workunit.client.1.smithi190.stdout:6/62: dwrite fe [4194304,4194304] 0 2024-05-06T12:14:41.267 INFO:tasks.workunit.client.0.smithi092.stdout:9/674: creat d16/d2c/d2f/d3d/d33/d57/fdd x:0 0 0 2024-05-06T12:14:41.270 INFO:tasks.workunit.client.1.smithi190.stdout:6/63: chown l6 13 1 2024-05-06T12:14:41.271 INFO:tasks.workunit.client.1.smithi190.stdout:9/227: rename db/dd/d16/l35 to db/d23/d29/l5d 0 2024-05-06T12:14:41.271 INFO:tasks.workunit.client.1.smithi190.stdout:1/81: dwrite d5/f6 [0,4194304] 0 2024-05-06T12:14:41.272 INFO:tasks.workunit.client.1.smithi190.stdout:0/253: dread d9/dc/f13 [0,4194304] 0 2024-05-06T12:14:41.273 INFO:tasks.workunit.client.1.smithi190.stdout:6/64: truncate f1 1341001 0 2024-05-06T12:14:41.274 INFO:tasks.workunit.client.0.smithi092.stdout:0/894: getdents d7/dc/d6f/da5/dc6/d117/d119 0 2024-05-06T12:14:41.274 INFO:tasks.workunit.client.0.smithi092.stdout:2/543: mkdir df/d17/d53/d85/da0 0 2024-05-06T12:14:41.275 INFO:tasks.workunit.client.0.smithi092.stdout:9/675: mkdir d16/d2c/d2f/d3d/d33/dde 0 2024-05-06T12:14:41.276 INFO:tasks.workunit.client.0.smithi092.stdout:2/544: creat df/d17/d7d/fa1 x:0 0 0 2024-05-06T12:14:41.276 INFO:tasks.workunit.client.0.smithi092.stdout:9/676: stat d16/d2c/c63 0 2024-05-06T12:14:41.277 INFO:tasks.workunit.client.1.smithi190.stdout:4/55: dwrite d2/f4 [0,4194304] 0 2024-05-06T12:14:41.277 INFO:tasks.workunit.client.1.smithi190.stdout:6/65: chown f1 3481447 1 2024-05-06T12:14:41.278 INFO:tasks.workunit.client.0.smithi092.stdout:0/895: dread d7/dc/d6f/fc2 [0,4194304] 0 2024-05-06T12:14:41.279 INFO:tasks.workunit.client.1.smithi190.stdout:0/254: mkdir d9/df/d56 0 2024-05-06T12:14:41.279 INFO:tasks.workunit.client.0.smithi092.stdout:9/677: truncate d16/d2c/d2f/d3d/fc2 170271 0 2024-05-06T12:14:41.283 INFO:tasks.workunit.client.0.smithi092.stdout:0/896: mkdir d7/dc/d97/dc9/d12c 0 2024-05-06T12:14:41.284 INFO:tasks.workunit.client.1.smithi190.stdout:1/82: link d5/f12 d5/f13 0 2024-05-06T12:14:41.286 INFO:tasks.workunit.client.1.smithi190.stdout:9/228: mkdir db/dd/d16/d4c/d4e/d5e 0 2024-05-06T12:14:41.286 INFO:tasks.workunit.client.0.smithi092.stdout:9/678: rename d16/d2c/d2f/d3d/f7d to d16/d1e/d7b/d8d/fdf 0 2024-05-06T12:14:41.287 INFO:tasks.workunit.client.0.smithi092.stdout:9/679: truncate d16/d1e/f2e 9365463 0 2024-05-06T12:14:41.288 INFO:tasks.workunit.client.0.smithi092.stdout:9/680: stat d16/d1c/d58/d92/dab 0 2024-05-06T12:14:41.289 INFO:tasks.workunit.client.0.smithi092.stdout:9/681: write d16/d2c/d2f/d3d/f61 [1238060,69012] 0 2024-05-06T12:14:41.290 INFO:tasks.workunit.client.0.smithi092.stdout:0/897: creat d7/dc/d31/f12d x:0 0 0 2024-05-06T12:14:41.291 INFO:tasks.workunit.client.1.smithi190.stdout:6/66: dwrite fc [0,4194304] 0 2024-05-06T12:14:41.292 INFO:tasks.workunit.client.0.smithi092.stdout:0/898: stat d7/dc/d28/d54/d12/d7f/cb7 0 2024-05-06T12:14:41.293 INFO:tasks.workunit.client.1.smithi190.stdout:4/56: dread d2/f4 [0,4194304] 0 2024-05-06T12:14:41.293 INFO:tasks.workunit.client.1.smithi190.stdout:9/229: truncate db/dd/d16/f4f 895107 0 2024-05-06T12:14:41.293 INFO:tasks.workunit.client.1.smithi190.stdout:6/67: write fa [1937093,95647] 0 2024-05-06T12:14:41.294 INFO:tasks.workunit.client.0.smithi092.stdout:9/682: rename d16/d2c/d4b/d76/fc3 to d16/d2c/d2f/d3d/d33/dde/fe0 0 2024-05-06T12:14:41.297 INFO:tasks.workunit.client.1.smithi190.stdout:9/230: readlink db/dd/d16/l20 0 2024-05-06T12:14:41.298 INFO:tasks.workunit.client.1.smithi190.stdout:6/68: write f2 [6793634,109090] 0 2024-05-06T12:14:41.298 INFO:tasks.workunit.client.1.smithi190.stdout:0/255: link d9/dc/d28/d4c/f53 d9/df/d56/f57 0 2024-05-06T12:14:41.298 INFO:tasks.workunit.client.0.smithi092.stdout:0/899: mknod d7/dc/d31/de6/c12e 0 2024-05-06T12:14:41.298 INFO:tasks.workunit.client.1.smithi190.stdout:0/256: stat d9/dc/d28/d4c 0 2024-05-06T12:14:41.299 INFO:tasks.workunit.client.1.smithi190.stdout:4/57: mknod d2/d11/c15 0 2024-05-06T12:14:41.299 INFO:tasks.workunit.client.0.smithi092.stdout:9/683: rmdir d16/d1e/d7b 39 2024-05-06T12:14:41.300 INFO:tasks.workunit.client.1.smithi190.stdout:0/257: rename d9 to d9/df/d58 22 2024-05-06T12:14:41.301 INFO:tasks.workunit.client.1.smithi190.stdout:6/69: rename df/f14 to df/f16 0 2024-05-06T12:14:41.303 INFO:tasks.workunit.client.0.smithi092.stdout:9/684: symlink d16/d1e/d7b/dce/le1 0 2024-05-06T12:14:41.304 INFO:tasks.workunit.client.1.smithi190.stdout:4/58: mkdir d2/d11/d16 0 2024-05-06T12:14:41.304 INFO:tasks.workunit.client.1.smithi190.stdout:6/70: chown l8 7899 1 2024-05-06T12:14:41.305 INFO:tasks.workunit.client.0.smithi092.stdout:9/685: symlink d16/d1c/d58/le2 0 2024-05-06T12:14:41.305 INFO:tasks.workunit.client.1.smithi190.stdout:9/231: dwrite db/f52 [0,4194304] 0 2024-05-06T12:14:41.307 INFO:tasks.workunit.client.1.smithi190.stdout:4/59: mknod d2/d10/c17 0 2024-05-06T12:14:41.308 INFO:tasks.workunit.client.0.smithi092.stdout:9/686: dread d16/d1e/f86 [0,4194304] 0 2024-05-06T12:14:41.308 INFO:tasks.workunit.client.1.smithi190.stdout:4/60: stat d2/f4 0 2024-05-06T12:14:41.309 INFO:tasks.workunit.client.1.smithi190.stdout:6/71: dread fa [0,4194304] 0 2024-05-06T12:14:41.310 INFO:tasks.workunit.client.1.smithi190.stdout:9/232: unlink db/d23/d25/d47/l59 0 2024-05-06T12:14:41.313 INFO:tasks.workunit.client.1.smithi190.stdout:4/61: dread d2/d11/f14 [0,4194304] 0 2024-05-06T12:14:41.315 INFO:tasks.workunit.client.1.smithi190.stdout:6/72: dread fe [4194304,4194304] 0 2024-05-06T12:14:41.316 INFO:tasks.workunit.client.0.smithi092.stdout:9/687: dwrite d16/d2c/d2f/d3d/d33/d6b/d9e/fba [0,4194304] 0 2024-05-06T12:14:41.316 INFO:tasks.workunit.client.1.smithi190.stdout:9/233: mknod db/dd/d16/d58/c5f 0 2024-05-06T12:14:41.316 INFO:tasks.workunit.client.1.smithi190.stdout:9/234: readlink db/dd/df/l28 0 2024-05-06T12:14:41.319 INFO:tasks.workunit.client.1.smithi190.stdout:6/73: mkdir df/d17 0 2024-05-06T12:14:41.321 INFO:tasks.workunit.client.1.smithi190.stdout:1/83: fsync d5/f13 0 2024-05-06T12:14:41.323 INFO:tasks.workunit.client.1.smithi190.stdout:6/74: rmdir df/d12 39 2024-05-06T12:14:41.325 INFO:tasks.workunit.client.0.smithi092.stdout:2/545: write df/d17/d53/d85/f98 [870247,18522] 0 2024-05-06T12:14:41.326 INFO:tasks.workunit.client.1.smithi190.stdout:9/235: dread db/dd/f17 [0,4194304] 0 2024-05-06T12:14:41.327 INFO:tasks.workunit.client.0.smithi092.stdout:2/546: fdatasync df/d15/d88/d59/f9e 0 2024-05-06T12:14:41.328 INFO:tasks.workunit.client.1.smithi190.stdout:1/84: symlink d5/l14 0 2024-05-06T12:14:41.328 INFO:tasks.workunit.client.1.smithi190.stdout:4/62: dwrite d2/d11/f14 [0,4194304] 0 2024-05-06T12:14:41.328 INFO:tasks.workunit.client.0.smithi092.stdout:2/547: fsync df/d17/d1d/d33/d3c/f8b 0 2024-05-06T12:14:41.329 INFO:tasks.workunit.client.1.smithi190.stdout:1/85: read d5/f7 [2954206,35631] 0 2024-05-06T12:14:41.329 INFO:tasks.workunit.client.0.smithi092.stdout:2/548: chown df/d15/l3b 746097 1 2024-05-06T12:14:41.331 INFO:tasks.workunit.client.0.smithi092.stdout:0/900: write d7/dc/f9c [5597909,22976] 0 2024-05-06T12:14:41.333 INFO:tasks.workunit.client.0.smithi092.stdout:2/549: link df/d15/l28 df/d15/la2 0 2024-05-06T12:14:41.334 INFO:tasks.workunit.client.0.smithi092.stdout:9/688: getdents d16/d1c/d58 0 2024-05-06T12:14:41.336 INFO:tasks.workunit.client.1.smithi190.stdout:0/258: truncate d9/dc/f1f 3483018 0 2024-05-06T12:14:41.336 INFO:tasks.workunit.client.0.smithi092.stdout:0/901: mkdir d7/dc/d28/d54/d1e/d26/d4e/dd7/d12f 0 2024-05-06T12:14:41.337 INFO:tasks.workunit.client.1.smithi190.stdout:0/259: readlink d9/l43 0 2024-05-06T12:14:41.338 INFO:tasks.workunit.client.0.smithi092.stdout:2/550: mknod df/d15/ca3 0 2024-05-06T12:14:41.338 INFO:tasks.workunit.client.0.smithi092.stdout:2/551: fdatasync df/d17/d1d/d33/f6f 0 2024-05-06T12:14:41.339 INFO:tasks.workunit.client.0.smithi092.stdout:2/552: chown df/d1b 2952 1 2024-05-06T12:14:41.339 INFO:tasks.workunit.client.0.smithi092.stdout:9/689: rmdir d16/d1c/d58/d92 39 2024-05-06T12:14:41.340 INFO:tasks.workunit.client.1.smithi190.stdout:4/63: creat d2/d11/d16/f18 x:0 0 0 2024-05-06T12:14:41.341 INFO:tasks.workunit.client.1.smithi190.stdout:1/86: unlink d5/f7 0 2024-05-06T12:14:41.342 INFO:tasks.workunit.client.0.smithi092.stdout:2/553: rmdir df/d17/d7e 39 2024-05-06T12:14:41.342 INFO:tasks.workunit.client.1.smithi190.stdout:6/75: getdents df/d10 0 2024-05-06T12:14:41.343 INFO:tasks.workunit.client.0.smithi092.stdout:9/690: fsync d16/d2c/d2f/d3d/d33/dde/fe0 0 2024-05-06T12:14:41.343 INFO:tasks.workunit.client.1.smithi190.stdout:0/260: creat d9/df/d56/f59 x:0 0 0 2024-05-06T12:14:41.343 INFO:tasks.workunit.client.1.smithi190.stdout:4/64: creat d2/d11/d16/f19 x:0 0 0 2024-05-06T12:14:41.344 INFO:tasks.workunit.client.0.smithi092.stdout:0/902: dwrite d7/dc/d28/d54/d1e/d26/d4e/fbb [0,4194304] 0 2024-05-06T12:14:41.344 INFO:tasks.workunit.client.1.smithi190.stdout:1/87: write d5/fe [3026495,106922] 0 2024-05-06T12:14:41.345 INFO:tasks.workunit.client.0.smithi092.stdout:2/554: symlink df/d15/d88/d59/la4 0 2024-05-06T12:14:41.345 INFO:tasks.workunit.client.0.smithi092.stdout:9/691: dread d16/d2c/f91 [0,4194304] 0 2024-05-06T12:14:41.346 INFO:tasks.workunit.client.1.smithi190.stdout:0/261: chown c4 447153122 1 2024-05-06T12:14:41.346 INFO:tasks.workunit.client.1.smithi190.stdout:3/143: sync 2024-05-06T12:14:41.347 INFO:tasks.workunit.client.1.smithi190.stdout:1/88: mkdir d5/d15 0 2024-05-06T12:14:41.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:40 smithi190 ceph-mon[33798]: Updating smithi092:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-06T12:14:41.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:40 smithi190 ceph-mon[33798]: Updating smithi190:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-06T12:14:41.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:40 smithi190 ceph-mon[33798]: pgmap v22: 65 pgs: 65 active+clean; 1.5 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 49 MiB/s rd, 185 MiB/s wr, 501 op/s 2024-05-06T12:14:41.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:40 smithi190 ceph-mon[33798]: Reconfiguring node-exporter.smithi092 (dependencies changed)... 2024-05-06T12:14:41.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:40 smithi190 ceph-mon[33798]: Reconfiguring daemon node-exporter.smithi092 on smithi092 2024-05-06T12:14:41.348 INFO:tasks.workunit.client.0.smithi092.stdout:2/555: readlink df/d15/la2 0 2024-05-06T12:14:41.349 INFO:tasks.workunit.client.1.smithi190.stdout:4/65: rename d2/d11/d16/f19 to d2/d11/d16/f1a 0 2024-05-06T12:14:41.350 INFO:tasks.workunit.client.0.smithi092.stdout:9/692: mknod d16/d2c/d88/ce3 0 2024-05-06T12:14:41.350 INFO:tasks.workunit.client.0.smithi092.stdout:2/556: mkdir df/d1b/da5 0 2024-05-06T12:14:41.351 INFO:tasks.workunit.client.0.smithi092.stdout:2/557: fsync df/d17/d51/f70 0 2024-05-06T12:14:41.352 INFO:tasks.workunit.client.0.smithi092.stdout:3/578: sync 2024-05-06T12:14:41.352 INFO:tasks.workunit.client.0.smithi092.stdout:6/756: sync 2024-05-06T12:14:41.352 INFO:tasks.workunit.client.1.smithi190.stdout:6/76: rename fa to df/f18 0 2024-05-06T12:14:41.353 INFO:tasks.workunit.client.0.smithi092.stdout:3/579: rmdir d8/d24 39 2024-05-06T12:14:41.353 INFO:tasks.workunit.client.1.smithi190.stdout:3/144: rmdir d5/d7/d15/d1f/d33 39 2024-05-06T12:14:41.354 INFO:tasks.workunit.client.1.smithi190.stdout:3/145: readlink d5/d14/l12 0 2024-05-06T12:14:41.354 INFO:tasks.workunit.client.0.smithi092.stdout:2/558: unlink df/d17/c25 0 2024-05-06T12:14:41.355 INFO:tasks.workunit.client.0.smithi092.stdout:9/693: getdents d16/d1e/da1 0 2024-05-06T12:14:41.355 INFO:tasks.workunit.client.0.smithi092.stdout:0/903: dread d7/dc/d9e/f61 [0,4194304] 0 2024-05-06T12:14:41.355 INFO:tasks.workunit.client.1.smithi190.stdout:0/262: mknod d9/dc/d28/d4c/c5a 0 2024-05-06T12:14:41.356 INFO:tasks.workunit.client.1.smithi190.stdout:6/77: dread df/f16 [0,4194304] 0 2024-05-06T12:14:41.357 INFO:tasks.workunit.client.1.smithi190.stdout:6/78: write fc [1905803,20875] 0 2024-05-06T12:14:41.357 INFO:tasks.workunit.client.1.smithi190.stdout:1/89: creat d5/f16 x:0 0 0 2024-05-06T12:14:41.359 INFO:tasks.workunit.client.0.smithi092.stdout:2/559: read - df/d17/d7e/f91 zero size 2024-05-06T12:14:41.360 INFO:tasks.workunit.client.1.smithi190.stdout:3/146: creat d5/d14/d11/d21/f38 x:0 0 0 2024-05-06T12:14:41.360 INFO:tasks.workunit.client.0.smithi092.stdout:6/757: symlink d4/d20/d23/lf6 0 2024-05-06T12:14:41.360 INFO:tasks.workunit.client.0.smithi092.stdout:0/904: write d7/dc/d28/d54/d1e/d46/f121 [452115,56464] 0 2024-05-06T12:14:41.361 INFO:tasks.workunit.client.0.smithi092.stdout:9/694: mkdir d16/d1e/d7b/dce/de4 0 2024-05-06T12:14:41.362 INFO:tasks.workunit.client.0.smithi092.stdout:3/580: dwrite d8/db/d14/d5a/fa5 [0,4194304] 0 2024-05-06T12:14:41.363 INFO:tasks.workunit.client.1.smithi190.stdout:9/236: write db/dd/df/f1b [2757827,105903] 0 2024-05-06T12:14:41.363 INFO:tasks.workunit.client.0.smithi092.stdout:2/560: fdatasync df/f55 0 2024-05-06T12:14:41.364 INFO:tasks.workunit.client.1.smithi190.stdout:0/263: symlink d9/dc/d48/d50/l5b 0 2024-05-06T12:14:41.364 INFO:tasks.workunit.client.0.smithi092.stdout:9/695: mkdir d16/d1c/de5 0 2024-05-06T12:14:41.365 INFO:tasks.workunit.client.0.smithi092.stdout:2/561: chown f5 1107660895 1 2024-05-06T12:14:41.366 INFO:tasks.workunit.client.0.smithi092.stdout:6/758: mkdir d4/d20/d23/d30/df7 0 2024-05-06T12:14:41.366 INFO:tasks.workunit.client.1.smithi190.stdout:6/79: symlink df/d12/l19 0 2024-05-06T12:14:41.366 INFO:tasks.workunit.client.1.smithi190.stdout:4/66: dwrite d2/d10/ff [0,4194304] 0 2024-05-06T12:14:41.367 INFO:tasks.workunit.client.0.smithi092.stdout:0/905: write d7/dc/d28/d54/d12/d7f/db2/fde [5137039,127202] 0 2024-05-06T12:14:41.367 INFO:tasks.workunit.client.1.smithi190.stdout:0/264: read d9/df/f2e [578310,105046] 0 2024-05-06T12:14:41.367 INFO:tasks.workunit.client.1.smithi190.stdout:9/237: write db/f2b [6644756,20786] 0 2024-05-06T12:14:41.368 INFO:tasks.workunit.client.1.smithi190.stdout:9/238: chown db/d23/d25/d47 129461 1 2024-05-06T12:14:41.369 INFO:tasks.workunit.client.0.smithi092.stdout:2/562: truncate df/d15/d88/d59/f9e 730918 0 2024-05-06T12:14:41.370 INFO:tasks.workunit.client.1.smithi190.stdout:1/90: creat d5/d15/f17 x:0 0 0 2024-05-06T12:14:41.370 INFO:tasks.workunit.client.0.smithi092.stdout:2/563: dread - df/f7b zero size 2024-05-06T12:14:41.370 INFO:tasks.workunit.client.0.smithi092.stdout:9/696: rename d16/l18 to d16/d2c/d2f/d75/le6 0 2024-05-06T12:14:41.370 INFO:tasks.workunit.client.1.smithi190.stdout:0/265: dread d9/f2f [0,4194304] 0 2024-05-06T12:14:41.371 INFO:tasks.workunit.client.0.smithi092.stdout:9/697: dread d16/d2c/f91 [0,4194304] 0 2024-05-06T12:14:41.373 INFO:tasks.workunit.client.1.smithi190.stdout:9/239: write db/f3d [924756,1149] 0 2024-05-06T12:14:41.374 INFO:tasks.workunit.client.1.smithi190.stdout:6/80: read fc [3711770,50737] 0 2024-05-06T12:14:41.374 INFO:tasks.workunit.client.0.smithi092.stdout:0/906: mknod d7/dc/d28/d54/d1e/d39/da8/dc5/c130 0 2024-05-06T12:14:41.374 INFO:tasks.workunit.client.0.smithi092.stdout:0/907: fsync d7/f9 0 2024-05-06T12:14:41.375 INFO:tasks.workunit.client.0.smithi092.stdout:0/908: readlink d7/dc/d6f/da5/lb6 0 2024-05-06T12:14:41.377 INFO:tasks.workunit.client.1.smithi190.stdout:0/266: write d9/df/d21/f49 [519218,62175] 0 2024-05-06T12:14:41.377 INFO:tasks.workunit.client.0.smithi092.stdout:3/581: mknod d8/d24/d27/cb4 0 2024-05-06T12:14:41.380 INFO:tasks.workunit.client.0.smithi092.stdout:9/698: creat d16/d94/da0/da7/fe7 x:0 0 0 2024-05-06T12:14:41.380 INFO:tasks.workunit.client.1.smithi190.stdout:9/240: readlink db/d23/d25/l27 0 2024-05-06T12:14:41.381 INFO:tasks.workunit.client.1.smithi190.stdout:9/241: readlink db/dd/df/l28 0 2024-05-06T12:14:41.382 INFO:tasks.workunit.client.0.smithi092.stdout:3/582: symlink d8/d24/d33/d91/lb5 0 2024-05-06T12:14:41.383 INFO:tasks.workunit.client.1.smithi190.stdout:1/91: symlink d5/l18 0 2024-05-06T12:14:41.383 INFO:tasks.workunit.client.0.smithi092.stdout:9/699: creat d16/d1e/da1/d78/fe8 x:0 0 0 2024-05-06T12:14:41.386 INFO:tasks.workunit.client.1.smithi190.stdout:6/81: write df/f18 [4595187,27505] 0 2024-05-06T12:14:41.388 INFO:tasks.workunit.client.1.smithi190.stdout:9/242: dwrite db/f3d [0,4194304] 0 2024-05-06T12:14:41.389 INFO:tasks.workunit.client.1.smithi190.stdout:9/243: fsync db/f52 0 2024-05-06T12:14:41.389 INFO:tasks.workunit.client.0.smithi092.stdout:0/909: dwrite d7/dc/d28/d54/d1e/d26/d3a/f10e [0,4194304] 0 2024-05-06T12:14:41.389 INFO:tasks.workunit.client.0.smithi092.stdout:9/700: creat d16/d1c/d58/d92/fe9 x:0 0 0 2024-05-06T12:14:41.392 INFO:tasks.workunit.client.0.smithi092.stdout:3/583: dwrite d8/d1b/f81 [0,4194304] 0 2024-05-06T12:14:41.395 INFO:tasks.workunit.client.0.smithi092.stdout:4/732: sync 2024-05-06T12:14:41.395 INFO:tasks.workunit.client.0.smithi092.stdout:1/697: sync 2024-05-06T12:14:41.395 INFO:tasks.workunit.client.0.smithi092.stdout:5/614: sync 2024-05-06T12:14:41.395 INFO:tasks.workunit.client.0.smithi092.stdout:8/769: sync 2024-05-06T12:14:41.395 INFO:tasks.workunit.client.0.smithi092.stdout:7/788: sync 2024-05-06T12:14:41.396 INFO:tasks.workunit.client.1.smithi190.stdout:6/82: mknod df/d10/c1a 0 2024-05-06T12:14:41.397 INFO:tasks.workunit.client.1.smithi190.stdout:9/244: mknod db/dd/d16/d4c/c60 0 2024-05-06T12:14:41.397 INFO:tasks.workunit.client.0.smithi092.stdout:0/910: read d7/dc/d28/d54/d1e/d5d/f76 [2889907,70838] 0 2024-05-06T12:14:41.401 INFO:tasks.workunit.client.0.smithi092.stdout:5/615: chown d4/d34/d29/d6c 53576 1 2024-05-06T12:14:41.402 INFO:tasks.workunit.client.1.smithi190.stdout:6/83: write fe [4066459,15838] 0 2024-05-06T12:14:41.403 INFO:tasks.workunit.client.1.smithi190.stdout:9/245: creat db/d23/d29/f61 x:0 0 0 2024-05-06T12:14:41.403 INFO:tasks.workunit.client.1.smithi190.stdout:1/92: link d5/f13 d5/f19 0 2024-05-06T12:14:41.404 INFO:tasks.workunit.client.0.smithi092.stdout:6/759: dwrite d4/d24/d38/d5d/fa3 [0,4194304] 0 2024-05-06T12:14:41.406 INFO:tasks.workunit.client.1.smithi190.stdout:1/93: creat d5/d15/f1a x:0 0 0 2024-05-06T12:14:41.406 INFO:tasks.workunit.client.0.smithi092.stdout:6/760: chown d4/d24/d38/d5d/d6d/le0 32658008 1 2024-05-06T12:14:41.407 INFO:tasks.workunit.client.0.smithi092.stdout:7/789: rename d9/de/d10/d17/d99/dc3 to d9/de/d10/d17/d99/dc3/dd2/d103 22 2024-05-06T12:14:41.409 INFO:tasks.workunit.client.1.smithi190.stdout:9/246: link db/f2b db/dd/d16/d4c/d4e/f62 0 2024-05-06T12:14:41.410 INFO:tasks.workunit.client.0.smithi092.stdout:7/790: chown d9/de/l1c 146464219 1 2024-05-06T12:14:41.410 INFO:tasks.workunit.client.1.smithi190.stdout:6/84: dread fc [0,4194304] 0 2024-05-06T12:14:41.414 INFO:tasks.workunit.client.0.smithi092.stdout:5/616: dwrite d4/d59/fd1 [0,4194304] 0 2024-05-06T12:14:41.415 INFO:tasks.workunit.client.1.smithi190.stdout:9/247: creat db/d23/d25/f63 x:0 0 0 2024-05-06T12:14:41.415 INFO:tasks.workunit.client.0.smithi092.stdout:8/770: dwrite d9/d21/f88 [0,4194304] 0 2024-05-06T12:14:41.417 INFO:tasks.workunit.client.1.smithi190.stdout:1/94: link d5/l14 d5/d15/l1b 0 2024-05-06T12:14:41.419 INFO:tasks.workunit.client.1.smithi190.stdout:4/67: truncate d2/d11/f14 1867771 0 2024-05-06T12:14:41.419 INFO:tasks.workunit.client.1.smithi190.stdout:1/95: dread - d5/d15/f1a zero size 2024-05-06T12:14:41.424 INFO:tasks.workunit.client.1.smithi190.stdout:6/85: mkdir df/d17/d1b 0 2024-05-06T12:14:41.424 INFO:tasks.workunit.client.1.smithi190.stdout:9/248: creat db/dd/d16/f64 x:0 0 0 2024-05-06T12:14:41.426 INFO:tasks.workunit.client.1.smithi190.stdout:1/96: mknod d5/d15/c1c 0 2024-05-06T12:14:41.427 INFO:tasks.workunit.client.1.smithi190.stdout:6/86: write df/d12/f13 [120308,54000] 0 2024-05-06T12:14:41.431 INFO:tasks.workunit.client.1.smithi190.stdout:6/87: chown fd 1993142 1 2024-05-06T12:14:41.431 INFO:tasks.workunit.client.1.smithi190.stdout:1/97: mknod d5/c1d 0 2024-05-06T12:14:41.433 INFO:tasks.workunit.client.1.smithi190.stdout:4/68: creat d2/f1b x:0 0 0 2024-05-06T12:14:41.436 INFO:tasks.workunit.client.1.smithi190.stdout:6/88: read f1 [462525,121767] 0 2024-05-06T12:14:41.436 INFO:tasks.workunit.client.1.smithi190.stdout:9/249: dwrite db/d23/f2a [0,4194304] 0 2024-05-06T12:14:41.436 INFO:tasks.workunit.client.1.smithi190.stdout:1/98: mkdir d5/d15/d1e 0 2024-05-06T12:14:41.436 INFO:tasks.workunit.client.1.smithi190.stdout:4/69: read - d2/f1b zero size 2024-05-06T12:14:41.440 INFO:tasks.workunit.client.1.smithi190.stdout:0/267: dwrite d9/df/d21/f4a [0,4194304] 0 2024-05-06T12:14:41.448 INFO:tasks.workunit.client.0.smithi092.stdout:4/733: creat d9/d2b/d7c/dab/ff1 x:0 0 0 2024-05-06T12:14:41.453 INFO:tasks.workunit.client.0.smithi092.stdout:2/564: dwrite df/f79 [0,4194304] 0 2024-05-06T12:14:41.459 INFO:tasks.workunit.client.1.smithi190.stdout:9/250: dwrite db/f1f [0,4194304] 0 2024-05-06T12:14:41.459 INFO:tasks.workunit.client.1.smithi190.stdout:0/268: link d9/l1e d9/df/d21/d31/l5c 0 2024-05-06T12:14:41.463 INFO:tasks.workunit.client.1.smithi190.stdout:7/93: sync 2024-05-06T12:14:41.463 INFO:tasks.workunit.client.1.smithi190.stdout:5/104: sync 2024-05-06T12:14:41.463 INFO:tasks.workunit.client.1.smithi190.stdout:8/201: sync 2024-05-06T12:14:41.463 INFO:tasks.workunit.client.1.smithi190.stdout:2/93: sync 2024-05-06T12:14:41.463 INFO:tasks.workunit.client.1.smithi190.stdout:1/99: dwrite d5/f6 [0,4194304] 0 2024-05-06T12:14:41.464 INFO:tasks.workunit.client.0.smithi092.stdout:2/565: dwrite df/d17/d51/f67 [0,4194304] 0 2024-05-06T12:14:41.465 INFO:tasks.workunit.client.0.smithi092.stdout:5/617: rename d4/d34/d29 to d4/d5/d10/d31/ddf 0 2024-05-06T12:14:41.466 INFO:tasks.workunit.client.1.smithi190.stdout:4/70: dwrite d2/d11/d16/f18 [0,4194304] 0 2024-05-06T12:14:41.467 INFO:tasks.workunit.client.1.smithi190.stdout:6/89: truncate fb 1536167 0 2024-05-06T12:14:41.468 INFO:tasks.workunit.client.0.smithi092.stdout:8/771: truncate d9/d21/d37/d46/fe9 4107335 0 2024-05-06T12:14:41.470 INFO:tasks.workunit.client.1.smithi190.stdout:2/94: creat d2/d8/dc/f16 x:0 0 0 2024-05-06T12:14:41.471 INFO:tasks.workunit.client.1.smithi190.stdout:2/95: fdatasync d2/d8/dc/f16 0 2024-05-06T12:14:41.472 INFO:tasks.workunit.client.1.smithi190.stdout:9/251: mknod db/dd/d16/c65 0 2024-05-06T12:14:41.472 INFO:tasks.workunit.client.0.smithi092.stdout:9/701: dwrite d16/d1e/d7b/fb0 [0,4194304] 0 2024-05-06T12:14:41.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:40 smithi092 ceph-mon[26954]: Updating smithi092:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-06T12:14:41.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:40 smithi092 ceph-mon[26954]: Updating smithi190:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-06T12:14:41.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:40 smithi092 ceph-mon[26954]: pgmap v22: 65 pgs: 65 active+clean; 1.5 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 49 MiB/s rd, 185 MiB/s wr, 501 op/s 2024-05-06T12:14:41.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:40 smithi092 ceph-mon[26954]: Reconfiguring node-exporter.smithi092 (dependencies changed)... 2024-05-06T12:14:41.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:40 smithi092 ceph-mon[26954]: Reconfiguring daemon node-exporter.smithi092 on smithi092 2024-05-06T12:14:41.474 INFO:tasks.workunit.client.0.smithi092.stdout:3/584: truncate d8/d24/d27/f2f 3606713 0 2024-05-06T12:14:41.474 INFO:tasks.workunit.client.1.smithi190.stdout:6/90: chown df/d10/c1a 661038629 1 2024-05-06T12:14:41.474 INFO:tasks.workunit.client.1.smithi190.stdout:8/202: rename d3/d8/c1d to d3/d13/c42 0 2024-05-06T12:14:41.475 INFO:tasks.workunit.client.1.smithi190.stdout:6/91: chown l6 953 1 2024-05-06T12:14:41.475 INFO:tasks.workunit.client.0.smithi092.stdout:3/585: chown d8/db/d4c/d40/d67/l78 99 1 2024-05-06T12:14:41.476 INFO:tasks.workunit.client.1.smithi190.stdout:1/100: creat d5/d15/f1f x:0 0 0 2024-05-06T12:14:41.477 INFO:tasks.workunit.client.0.smithi092.stdout:0/911: getdents d7/dc/d6f/da5/dc6/d117/d119 0 2024-05-06T12:14:41.477 INFO:tasks.workunit.client.1.smithi190.stdout:4/71: creat d2/d10/f1c x:0 0 0 2024-05-06T12:14:41.477 INFO:tasks.workunit.client.1.smithi190.stdout:6/92: read fe [3031183,5806] 0 2024-05-06T12:14:41.478 INFO:tasks.workunit.client.1.smithi190.stdout:7/94: symlink d0/d1/d18/l25 0 2024-05-06T12:14:41.479 INFO:tasks.workunit.client.0.smithi092.stdout:5/618: dread d4/f7d [0,4194304] 0 2024-05-06T12:14:41.480 INFO:tasks.workunit.client.1.smithi190.stdout:2/96: write d2/fd [2856044,76999] 0 2024-05-06T12:14:41.481 INFO:tasks.workunit.client.1.smithi190.stdout:9/252: unlink db/d23/d25/f63 0 2024-05-06T12:14:41.482 INFO:tasks.workunit.client.0.smithi092.stdout:0/912: read - d7/dc/d28/d54/d1e/d46/df0/f129 zero size 2024-05-06T12:14:41.483 INFO:tasks.workunit.client.1.smithi190.stdout:9/253: dread - db/d21/d22/f55 zero size 2024-05-06T12:14:41.485 INFO:tasks.workunit.client.0.smithi092.stdout:6/761: mknod d4/d20/d23/cf8 0 2024-05-06T12:14:41.485 INFO:tasks.workunit.client.1.smithi190.stdout:0/269: dread d9/df/d21/f49 [0,4194304] 0 2024-05-06T12:14:41.486 INFO:tasks.workunit.client.1.smithi190.stdout:5/105: rename d2/f9 to d2/da/de/d15/f22 0 2024-05-06T12:14:41.487 INFO:tasks.workunit.client.1.smithi190.stdout:8/203: mknod d3/d8/d2b/c43 0 2024-05-06T12:14:41.487 INFO:tasks.workunit.client.0.smithi092.stdout:1/698: creat d3/d10/d13/d19/d21/fdb x:0 0 0 2024-05-06T12:14:41.489 INFO:tasks.workunit.client.0.smithi092.stdout:8/772: creat d9/d21/d37/d74/df9/f114 x:0 0 0 2024-05-06T12:14:41.490 INFO:tasks.workunit.client.1.smithi190.stdout:5/106: readlink d2/da/l16 0 2024-05-06T12:14:41.490 INFO:tasks.workunit.client.1.smithi190.stdout:1/101: dwrite d5/d15/f1a [0,4194304] 0 2024-05-06T12:14:41.492 INFO:tasks.workunit.client.0.smithi092.stdout:9/702: rmdir d16/d1e/d7b/dce 39 2024-05-06T12:14:41.493 INFO:tasks.workunit.client.1.smithi190.stdout:7/95: fdatasync d0/d1/fc 0 2024-05-06T12:14:41.495 INFO:tasks.workunit.client.0.smithi092.stdout:5/619: mkdir d4/d34/dae/de0 0 2024-05-06T12:14:41.496 INFO:tasks.workunit.client.0.smithi092.stdout:2/566: symlink df/d1b/da5/la6 0 2024-05-06T12:14:41.496 INFO:tasks.workunit.client.1.smithi190.stdout:8/204: readlink d3/l5 0 2024-05-06T12:14:41.498 INFO:tasks.workunit.client.1.smithi190.stdout:2/97: rename l1 to d2/d8/dc/l17 0 2024-05-06T12:14:41.502 INFO:tasks.workunit.client.0.smithi092.stdout:8/773: dwrite d9/fe4 [0,4194304] 0 2024-05-06T12:14:41.502 INFO:tasks.workunit.client.0.smithi092.stdout:2/567: write df/d15/d88/d59/f65 [1510309,101813] 0 2024-05-06T12:14:41.502 INFO:tasks.workunit.client.0.smithi092.stdout:2/568: stat df/d17/d7d 0 2024-05-06T12:14:41.502 INFO:tasks.workunit.client.1.smithi190.stdout:7/96: creat d0/d1/d2/f26 x:0 0 0 2024-05-06T12:14:41.503 INFO:tasks.workunit.client.0.smithi092.stdout:2/569: stat df/d17/d1d/f46 0 2024-05-06T12:14:41.503 INFO:tasks.workunit.client.0.smithi092.stdout:8/774: write d9/d21/d37/d74/df9/f114 [574758,106836] 0 2024-05-06T12:14:41.506 INFO:tasks.workunit.client.0.smithi092.stdout:6/762: rename d4/d20/c1d to d4/d24/d26/dae/dc0/cf9 0 2024-05-06T12:14:41.506 INFO:tasks.workunit.client.1.smithi190.stdout:0/270: creat d9/dc/d28/f5d x:0 0 0 2024-05-06T12:14:41.507 INFO:tasks.workunit.client.1.smithi190.stdout:3/147: sync 2024-05-06T12:14:41.508 INFO:tasks.workunit.client.0.smithi092.stdout:9/703: mknod d16/d2c/d2f/d3d/d33/d37/cea 0 2024-05-06T12:14:41.509 INFO:tasks.workunit.client.1.smithi190.stdout:6/93: getdents df/d12 0 2024-05-06T12:14:41.510 INFO:tasks.workunit.client.1.smithi190.stdout:6/94: write f7 [5218702,65941] 0 2024-05-06T12:14:41.510 INFO:tasks.workunit.client.0.smithi092.stdout:5/620: mknod d4/d34/ce1 0 2024-05-06T12:14:41.511 INFO:tasks.workunit.client.1.smithi190.stdout:2/98: mkdir d2/d18 0 2024-05-06T12:14:41.511 INFO:tasks.workunit.client.1.smithi190.stdout:8/205: dread d3/f24 [0,4194304] 0 2024-05-06T12:14:41.513 INFO:tasks.workunit.client.1.smithi190.stdout:6/95: read df/d12/f13 [123671,72781] 0 2024-05-06T12:14:41.513 INFO:tasks.workunit.client.1.smithi190.stdout:3/148: mkdir d5/d7/d15/d39 0 2024-05-06T12:14:41.513 INFO:tasks.workunit.client.1.smithi190.stdout:0/271: chown d9/l1e 143542175 1 2024-05-06T12:14:41.514 INFO:tasks.workunit.client.0.smithi092.stdout:6/763: dwrite d4/d4c/d52/f4a [4194304,4194304] 0 2024-05-06T12:14:41.514 INFO:tasks.workunit.client.1.smithi190.stdout:3/149: chown d5/d7 1143 1 2024-05-06T12:14:41.514 INFO:tasks.workunit.client.1.smithi190.stdout:0/272: chown d9/dc/d28 159 1 2024-05-06T12:14:41.515 INFO:tasks.workunit.client.1.smithi190.stdout:3/150: dread - d5/f26 zero size 2024-05-06T12:14:41.515 INFO:tasks.workunit.client.1.smithi190.stdout:0/273: fdatasync d9/f2c 0 2024-05-06T12:14:41.515 INFO:tasks.workunit.client.1.smithi190.stdout:3/151: chown d5/d14 188048 1 2024-05-06T12:14:41.516 INFO:tasks.workunit.client.0.smithi092.stdout:9/704: mkdir d16/d1c/d58/d92/dab/deb 0 2024-05-06T12:14:41.516 INFO:tasks.workunit.client.0.smithi092.stdout:9/705: read - d16/d2c/d2f/db1/fcf zero size 2024-05-06T12:14:41.519 INFO:tasks.workunit.client.0.smithi092.stdout:9/706: read d16/d1e/f28 [2201937,39907] 0 2024-05-06T12:14:41.521 INFO:tasks.workunit.client.0.smithi092.stdout:5/621: creat d4/d5/d10/da1/fe2 x:0 0 0 2024-05-06T12:14:41.522 INFO:tasks.workunit.client.1.smithi190.stdout:6/96: unlink df/f16 0 2024-05-06T12:14:41.522 INFO:tasks.workunit.client.1.smithi190.stdout:0/274: dwrite d9/df/d21/f2a [0,4194304] 0 2024-05-06T12:14:41.523 INFO:tasks.workunit.client.1.smithi190.stdout:0/275: truncate d9/f1a 1286739 0 2024-05-06T12:14:41.524 INFO:tasks.workunit.client.1.smithi190.stdout:0/276: chown d9/df/d21/f2a 55438268 1 2024-05-06T12:14:41.524 INFO:tasks.workunit.client.0.smithi092.stdout:6/764: creat d4/d4c/d52/d6c/ffa x:0 0 0 2024-05-06T12:14:41.526 INFO:tasks.workunit.client.1.smithi190.stdout:3/152: creat d5/d7/f3a x:0 0 0 2024-05-06T12:14:41.527 INFO:tasks.workunit.client.0.smithi092.stdout:2/570: rmdir df/d17/d7d/d9b 0 2024-05-06T12:14:41.528 INFO:tasks.workunit.client.1.smithi190.stdout:8/206: symlink d3/d8/d2b/d3e/l44 0 2024-05-06T12:14:41.530 INFO:tasks.workunit.client.1.smithi190.stdout:5/107: rename d2/da/de/d15 to d2/d23 0 2024-05-06T12:14:41.531 INFO:tasks.workunit.client.1.smithi190.stdout:5/108: fdatasync f0 0 2024-05-06T12:14:41.532 INFO:tasks.workunit.client.0.smithi092.stdout:2/571: write df/d17/d7e/f91 [462049,53912] 0 2024-05-06T12:14:41.534 INFO:tasks.workunit.client.1.smithi190.stdout:0/277: symlink d9/df/d21/d31/l5e 0 2024-05-06T12:14:41.536 INFO:tasks.workunit.client.0.smithi092.stdout:2/572: write fe [655418,564] 0 2024-05-06T12:14:41.536 INFO:tasks.workunit.client.0.smithi092.stdout:7/791: symlink d9/de/l104 0 2024-05-06T12:14:41.537 INFO:tasks.workunit.client.1.smithi190.stdout:8/207: dwrite d3/d35/d3a/f3d [0,4194304] 0 2024-05-06T12:14:41.538 INFO:tasks.workunit.client.0.smithi092.stdout:2/573: chown df/d17/l7a 126 1 2024-05-06T12:14:41.538 INFO:tasks.workunit.client.0.smithi092.stdout:9/707: dwrite d16/d2c/d2f/d3d/d33/fc5 [0,4194304] 0 2024-05-06T12:14:41.539 INFO:tasks.workunit.client.1.smithi190.stdout:8/208: write d3/d8/d2b/f38 [449536,130288] 0 2024-05-06T12:14:41.540 INFO:tasks.workunit.client.1.smithi190.stdout:1/102: rename d5/l18 to d5/d15/l20 0 2024-05-06T12:14:41.542 INFO:tasks.workunit.client.1.smithi190.stdout:9/254: write db/dd/f41 [1334020,72262] 0 2024-05-06T12:14:41.543 INFO:tasks.workunit.client.1.smithi190.stdout:5/109: rename d2 to d2/da/d24 22 2024-05-06T12:14:41.543 INFO:tasks.workunit.client.0.smithi092.stdout:8/775: dread d9/d21/d54/f7b [0,4194304] 0 2024-05-06T12:14:41.544 INFO:tasks.workunit.client.1.smithi190.stdout:6/97: mknod df/c1c 0 2024-05-06T12:14:41.544 INFO:tasks.workunit.client.1.smithi190.stdout:5/110: write f0 [4026040,124424] 0 2024-05-06T12:14:41.546 INFO:tasks.workunit.client.0.smithi092.stdout:2/574: creat df/d15/d88/fa7 x:0 0 0 2024-05-06T12:14:41.546 INFO:tasks.workunit.client.1.smithi190.stdout:5/111: stat d2/da/de/l1e 0 2024-05-06T12:14:41.548 INFO:tasks.workunit.client.0.smithi092.stdout:2/575: write df/f12 [1049609,99861] 0 2024-05-06T12:14:41.548 INFO:tasks.workunit.client.0.smithi092.stdout:7/792: dwrite d9/de/d10/d17/d99/fb3 [4194304,4194304] 0 2024-05-06T12:14:41.549 INFO:tasks.workunit.client.1.smithi190.stdout:7/97: rmdir d0/d1/d2 39 2024-05-06T12:14:41.549 INFO:tasks.workunit.client.1.smithi190.stdout:8/209: rmdir d3/d35/d3a 39 2024-05-06T12:14:41.549 INFO:tasks.workunit.client.0.smithi092.stdout:7/793: stat d9/da5/l3c 0 2024-05-06T12:14:41.552 INFO:tasks.workunit.client.0.smithi092.stdout:2/576: rmdir df/d15 39 2024-05-06T12:14:41.553 INFO:tasks.workunit.client.0.smithi092.stdout:2/577: write df/d1b/f1c [3070358,115040] 0 2024-05-06T12:14:41.553 INFO:tasks.workunit.client.1.smithi190.stdout:6/98: read fe [8146395,6174] 0 2024-05-06T12:14:41.554 INFO:tasks.workunit.client.0.smithi092.stdout:8/776: mkdir d9/dc/d4a/d5f/da8/ddf/d115 0 2024-05-06T12:14:41.555 INFO:tasks.workunit.client.1.smithi190.stdout:5/112: dwrite d2/f3 [0,4194304] 0 2024-05-06T12:14:41.560 INFO:tasks.workunit.client.0.smithi092.stdout:7/794: mknod d9/de/d10/d17/d99/c105 0 2024-05-06T12:14:41.560 INFO:tasks.workunit.client.1.smithi190.stdout:8/210: stat d3/f6 0 2024-05-06T12:14:41.562 INFO:tasks.workunit.client.0.smithi092.stdout:8/777: dwrite d9/d21/d37/d46/fd2 [0,4194304] 0 2024-05-06T12:14:41.564 INFO:tasks.workunit.client.0.smithi092.stdout:7/795: mknod d9/d62/c106 0 2024-05-06T12:14:41.565 INFO:tasks.workunit.client.1.smithi190.stdout:7/98: mknod d0/d14/c27 0 2024-05-06T12:14:41.566 INFO:tasks.workunit.client.0.smithi092.stdout:8/778: truncate d9/dc/f2d 2502891 0 2024-05-06T12:14:41.568 INFO:tasks.workunit.client.1.smithi190.stdout:5/113: dwrite d2/d23/f20 [0,4194304] 0 2024-05-06T12:14:41.569 INFO:tasks.workunit.client.0.smithi092.stdout:8/779: chown d9/dc/c2f 1124 1 2024-05-06T12:14:41.570 INFO:tasks.workunit.client.1.smithi190.stdout:5/114: chown d2/da/de 440489516 1 2024-05-06T12:14:41.573 INFO:tasks.workunit.client.0.smithi092.stdout:7/796: dwrite d9/de/d1f/dcf/fe9 [0,4194304] 0 2024-05-06T12:14:41.574 INFO:tasks.workunit.client.0.smithi092.stdout:9/708: symlink d16/d1e/da1/lec 0 2024-05-06T12:14:41.574 INFO:tasks.workunit.client.1.smithi190.stdout:0/278: creat d9/f5f x:0 0 0 2024-05-06T12:14:41.575 INFO:tasks.workunit.client.0.smithi092.stdout:8/780: chown d9/d21/dc0/df2/f110 4727698 1 2024-05-06T12:14:41.576 INFO:tasks.workunit.client.0.smithi092.stdout:7/797: chown d9/de/d10/d17/d21/f5d 162 1 2024-05-06T12:14:41.577 INFO:tasks.workunit.client.0.smithi092.stdout:9/709: rmdir d16/d1c 39 2024-05-06T12:14:41.577 INFO:tasks.workunit.client.0.smithi092.stdout:7/798: read d9/de/d10/d17/d6a/fb9 [537609,79210] 0 2024-05-06T12:14:41.578 INFO:tasks.workunit.client.1.smithi190.stdout:7/99: symlink d0/d15/l28 0 2024-05-06T12:14:41.578 INFO:tasks.workunit.client.1.smithi190.stdout:0/279: fsync d9/f37 0 2024-05-06T12:14:41.584 INFO:tasks.workunit.client.0.smithi092.stdout:7/799: mknod d9/de/d10/d17/d21/d38/d77/c107 0 2024-05-06T12:14:41.585 INFO:tasks.workunit.client.1.smithi190.stdout:8/211: symlink d3/d8/d25/d3b/l45 0 2024-05-06T12:14:41.589 INFO:tasks.workunit.client.0.smithi092.stdout:9/710: dwrite d16/d1e/d7b/d8d/fdf [4194304,4194304] 0 2024-05-06T12:14:41.590 INFO:tasks.workunit.client.1.smithi190.stdout:3/153: truncate d5/d7/f23 1000926 0 2024-05-06T12:14:41.592 INFO:tasks.workunit.client.0.smithi092.stdout:7/800: mkdir d9/d62/d108 0 2024-05-06T12:14:41.592 INFO:tasks.workunit.client.1.smithi190.stdout:4/72: dwrite d2/d11/f14 [0,4194304] 0 2024-05-06T12:14:41.593 INFO:tasks.workunit.client.1.smithi190.stdout:9/255: getdents db/dd/d16/d4c/d4e 0 2024-05-06T12:14:41.593 INFO:tasks.workunit.client.1.smithi190.stdout:3/154: stat d5/c2b 0 2024-05-06T12:14:41.593 INFO:tasks.workunit.client.1.smithi190.stdout:6/99: dwrite f1 [0,4194304] 0 2024-05-06T12:14:41.595 INFO:tasks.workunit.client.1.smithi190.stdout:5/115: rename d2/d23/l1f to d2/d23/l25 0 2024-05-06T12:14:41.595 INFO:tasks.workunit.client.1.smithi190.stdout:8/212: chown d3/d8/d2b/d3e/l44 6 1 2024-05-06T12:14:41.598 INFO:tasks.workunit.client.1.smithi190.stdout:1/103: truncate d5/fb 614151 0 2024-05-06T12:14:41.598 INFO:tasks.workunit.client.1.smithi190.stdout:7/100: creat d0/d1/d2/f29 x:0 0 0 2024-05-06T12:14:41.601 INFO:tasks.workunit.client.1.smithi190.stdout:4/73: symlink d2/d11/l1d 0 2024-05-06T12:14:41.601 INFO:tasks.workunit.client.0.smithi092.stdout:1/699: write d3/d10/d13/d19/d21/f2f [3029027,115077] 0 2024-05-06T12:14:41.602 INFO:tasks.workunit.client.1.smithi190.stdout:6/100: symlink df/d10/l1d 0 2024-05-06T12:14:41.603 INFO:tasks.workunit.client.0.smithi092.stdout:9/711: unlink d16/d2c/d2f/d3d/d33/fc5 0 2024-05-06T12:14:41.603 INFO:tasks.workunit.client.1.smithi190.stdout:8/213: symlink d3/d13/l46 0 2024-05-06T12:14:41.604 INFO:tasks.workunit.client.1.smithi190.stdout:9/256: read f3 [804952,128309] 0 2024-05-06T12:14:41.606 INFO:tasks.workunit.client.0.smithi092.stdout:7/801: dwrite d9/de/d1f/dcf/fe9 [0,4194304] 0 2024-05-06T12:14:41.607 INFO:tasks.workunit.client.1.smithi190.stdout:7/101: symlink d0/d4/l2a 0 2024-05-06T12:14:41.608 INFO:tasks.workunit.client.1.smithi190.stdout:3/155: dwrite d5/d7/d15/f28 [0,4194304] 0 2024-05-06T12:14:41.610 INFO:tasks.workunit.client.0.smithi092.stdout:7/802: write d9/de/d10/d17/d21/d2e/d9c/fdc [1002622,3099] 0 2024-05-06T12:14:41.610 INFO:tasks.workunit.client.0.smithi092.stdout:6/765: rename d4/f17 to d4/d24/d26/ffb 0 2024-05-06T12:14:41.613 INFO:tasks.workunit.client.0.smithi092.stdout:6/766: dread - d4/d24/d38/d5d/d6d/dd8/ff0 zero size 2024-05-06T12:14:41.614 INFO:tasks.workunit.client.0.smithi092.stdout:7/803: rename d9/de/d10/l45 to d9/de/d2d/l109 0 2024-05-06T12:14:41.616 INFO:tasks.workunit.client.0.smithi092.stdout:6/767: write d4/d24/d38/d5d/fd0 [892298,53833] 0 2024-05-06T12:14:41.617 INFO:tasks.workunit.client.0.smithi092.stdout:7/804: write d9/de/d10/d17/d99/db6/fc8 [252528,24350] 0 2024-05-06T12:14:41.619 INFO:tasks.workunit.client.1.smithi190.stdout:9/257: dwrite db/f3d [0,4194304] 0 2024-05-06T12:14:41.620 INFO:tasks.workunit.client.0.smithi092.stdout:1/700: getdents d3/d42/d84/d9f 0 2024-05-06T12:14:41.623 INFO:tasks.workunit.client.0.smithi092.stdout:6/768: unlink d4/d24/d26/l5f 0 2024-05-06T12:14:41.624 INFO:tasks.workunit.client.0.smithi092.stdout:7/805: creat d9/dd/f10a x:0 0 0 2024-05-06T12:14:41.624 INFO:tasks.workunit.client.1.smithi190.stdout:7/102: dwrite d0/d1/d2/f26 [0,4194304] 0 2024-05-06T12:14:41.624 INFO:tasks.workunit.client.1.smithi190.stdout:7/103: chown d0/d1/fb 132721 1 2024-05-06T12:14:41.626 INFO:tasks.workunit.client.0.smithi092.stdout:6/769: mknod d4/cfc 0 2024-05-06T12:14:41.627 INFO:tasks.workunit.client.0.smithi092.stdout:1/701: getdents d3/daf 0 2024-05-06T12:14:41.631 INFO:tasks.workunit.client.0.smithi092.stdout:7/806: rename d9/de/d1f/cc7 to d9/de/d2d/c10b 0 2024-05-06T12:14:41.634 INFO:tasks.workunit.client.0.smithi092.stdout:7/807: rmdir d9/de/d1f/d37/d48/d97 39 2024-05-06T12:14:41.635 INFO:tasks.workunit.client.0.smithi092.stdout:5/622: truncate d4/d5/f3b 3771143 0 2024-05-06T12:14:41.635 INFO:tasks.workunit.client.1.smithi190.stdout:1/104: rename d5/l10 to d5/d15/d1e/l21 0 2024-05-06T12:14:41.635 INFO:tasks.workunit.client.0.smithi092.stdout:5/623: fdatasync d4/f9b 0 2024-05-06T12:14:41.636 INFO:tasks.workunit.client.0.smithi092.stdout:7/808: mknod d9/de/d10/c10c 0 2024-05-06T12:14:41.640 INFO:tasks.workunit.client.0.smithi092.stdout:5/624: symlink d4/d5/d10/da1/le3 0 2024-05-06T12:14:41.643 INFO:tasks.workunit.client.0.smithi092.stdout:7/809: rmdir d9/de/d10/d17/d99/db6 39 2024-05-06T12:14:41.644 INFO:tasks.workunit.client.0.smithi092.stdout:7/810: write d9/de/d10/df0/df5/ffd [4401818,2457] 0 2024-05-06T12:14:41.645 INFO:tasks.workunit.client.0.smithi092.stdout:5/625: chown d4/d34/d28/c6d 6 1 2024-05-06T12:14:41.646 INFO:tasks.workunit.client.0.smithi092.stdout:5/626: chown d4/d5/d10/da1/fe2 244941946 1 2024-05-06T12:14:41.647 INFO:tasks.workunit.client.0.smithi092.stdout:6/770: creat d4/d4c/d91/ffd x:0 0 0 2024-05-06T12:14:41.651 INFO:tasks.workunit.client.0.smithi092.stdout:5/627: creat d4/d5/d10/d31/ddf/d6c/fe4 x:0 0 0 2024-05-06T12:14:41.653 INFO:tasks.workunit.client.0.smithi092.stdout:7/811: truncate d9/dd/f43 787105 0 2024-05-06T12:14:41.655 INFO:tasks.workunit.client.0.smithi092.stdout:5/628: truncate d4/d5/d10/d31/ddf/da3/dc2/fd7 222095 0 2024-05-06T12:14:41.658 INFO:tasks.workunit.client.1.smithi190.stdout:6/101: unlink df/d10/l1d 0 2024-05-06T12:14:41.660 INFO:tasks.workunit.client.1.smithi190.stdout:8/214: read d3/d35/d3a/f3d [3431454,4902] 0 2024-05-06T12:14:41.661 INFO:tasks.workunit.client.0.smithi092.stdout:7/812: mkdir d9/d62/d108/d10d 0 2024-05-06T12:14:41.661 INFO:tasks.workunit.client.0.smithi092.stdout:6/771: link d4/d24/d38/d5d/d6d/f84 d4/d24/db0/ffe 0 2024-05-06T12:14:41.662 INFO:tasks.workunit.client.0.smithi092.stdout:2/578: dwrite df/d15/f36 [0,4194304] 0 2024-05-06T12:14:41.669 INFO:tasks.workunit.client.0.smithi092.stdout:2/579: symlink df/d17/d53/d85/la8 0 2024-05-06T12:14:41.671 INFO:tasks.workunit.client.0.smithi092.stdout:2/580: fdatasync df/d17/d1d/f41 0 2024-05-06T12:14:41.671 INFO:tasks.workunit.client.0.smithi092.stdout:6/772: symlink d4/d20/de2/df2/lff 0 2024-05-06T12:14:41.672 INFO:tasks.workunit.client.0.smithi092.stdout:7/813: dread d9/da5/f30 [0,4194304] 0 2024-05-06T12:14:41.672 INFO:tasks.workunit.client.1.smithi190.stdout:8/215: dwrite d3/d8/d2b/f34 [0,4194304] 0 2024-05-06T12:14:41.672 INFO:tasks.workunit.client.1.smithi190.stdout:1/105: readlink d5/l14 0 2024-05-06T12:14:41.673 INFO:tasks.workunit.client.0.smithi092.stdout:6/773: fdatasync d4/d24/d54/f86 0 2024-05-06T12:14:41.673 INFO:tasks.workunit.client.1.smithi190.stdout:8/216: chown d3/d13/l46 1 1 2024-05-06T12:14:41.677 INFO:tasks.workunit.client.1.smithi190.stdout:5/116: creat d2/f26 x:0 0 0 2024-05-06T12:14:41.681 INFO:tasks.workunit.client.0.smithi092.stdout:2/581: dwrite df/d17/d51/f8c [0,4194304] 0 2024-05-06T12:14:41.681 INFO:tasks.workunit.client.0.smithi092.stdout:7/814: getdents d9/de/d10/d17/d99/da6 0 2024-05-06T12:14:41.682 INFO:tasks.workunit.client.1.smithi190.stdout:9/258: link db/f52 db/d23/d25/d47/d4b/f66 0 2024-05-06T12:14:41.683 INFO:tasks.workunit.client.0.smithi092.stdout:2/582: fsync f1 0 2024-05-06T12:14:41.684 INFO:tasks.workunit.client.0.smithi092.stdout:6/774: dwrite d4/d4c/da9/fd2 [4194304,4194304] 0 2024-05-06T12:14:41.685 INFO:tasks.workunit.client.1.smithi190.stdout:4/74: creat d2/f1e x:0 0 0 2024-05-06T12:14:41.690 INFO:tasks.workunit.client.0.smithi092.stdout:6/775: rmdir d4/d24/d38/d5d/d6d 39 2024-05-06T12:14:41.690 INFO:tasks.workunit.client.0.smithi092.stdout:2/583: dread df/d17/d51/f67 [0,4194304] 0 2024-05-06T12:14:41.691 INFO:tasks.workunit.client.0.smithi092.stdout:9/712: dwrite d16/d2c/d2f/d3d/d33/f9a [0,4194304] 0 2024-05-06T12:14:41.691 INFO:tasks.workunit.client.1.smithi190.stdout:1/106: stat d5/d15/d1e/l21 0 2024-05-06T12:14:41.693 INFO:tasks.workunit.client.0.smithi092.stdout:9/713: chown d16/d2c/d2f/d3d/f3c 0 1 2024-05-06T12:14:41.694 INFO:tasks.workunit.client.1.smithi190.stdout:3/156: creat d5/d7/d15/d1f/f3b x:0 0 0 2024-05-06T12:14:41.694 INFO:tasks.workunit.client.0.smithi092.stdout:7/815: rename d9/d62/c106 to d9/de/d1f/d37/d48/d97/c10e 0 2024-05-06T12:14:41.695 INFO:tasks.workunit.client.1.smithi190.stdout:3/157: write d5/d7/d15/f28 [2416793,63230] 0 2024-05-06T12:14:41.696 INFO:tasks.workunit.client.1.smithi190.stdout:9/259: symlink db/d23/d29/l67 0 2024-05-06T12:14:41.697 INFO:tasks.workunit.client.1.smithi190.stdout:3/158: write d5/d7/d15/d1f/f3b [684243,34048] 0 2024-05-06T12:14:41.698 INFO:tasks.workunit.client.1.smithi190.stdout:4/75: creat d2/d11/d16/f1f x:0 0 0 2024-05-06T12:14:41.699 INFO:tasks.workunit.client.1.smithi190.stdout:3/159: fdatasync d5/d14/d11/d2a/f35 0 2024-05-06T12:14:41.700 INFO:tasks.workunit.client.1.smithi190.stdout:3/160: stat d5/f27 0 2024-05-06T12:14:41.700 INFO:tasks.workunit.client.0.smithi092.stdout:2/584: unlink df/d15/c24 0 2024-05-06T12:14:41.701 INFO:tasks.workunit.client.1.smithi190.stdout:4/76: truncate d2/d11/d16/f1f 267991 0 2024-05-06T12:14:41.703 INFO:tasks.workunit.client.0.smithi092.stdout:7/816: unlink d9/da5/f14 0 2024-05-06T12:14:41.706 INFO:tasks.workunit.client.0.smithi092.stdout:1/702: write d3/d10/d13/d19/d21/d72/f86 [2945116,52536] 0 2024-05-06T12:14:41.709 INFO:tasks.workunit.client.0.smithi092.stdout:1/703: write d3/d42/d84/d4f/d7f/fad [340212,49183] 0 2024-05-06T12:14:41.710 INFO:tasks.workunit.client.0.smithi092.stdout:2/585: rename df/d17/d1d/d33/d3c to df/d17/d53/da9 0 2024-05-06T12:14:41.711 INFO:tasks.workunit.client.0.smithi092.stdout:2/586: fsync df/d17/d1d/f41 0 2024-05-06T12:14:41.712 INFO:tasks.workunit.client.1.smithi190.stdout:5/117: rename d2/d23 to d2/d27 0 2024-05-06T12:14:41.717 INFO:tasks.workunit.client.0.smithi092.stdout:7/817: mknod d9/de/da4/df8/c10f 0 2024-05-06T12:14:41.718 INFO:tasks.workunit.client.1.smithi190.stdout:8/217: getdents d3/d35 0 2024-05-06T12:14:41.718 INFO:tasks.workunit.client.0.smithi092.stdout:2/587: read df/f2c [4053977,106225] 0 2024-05-06T12:14:41.719 INFO:tasks.workunit.client.0.smithi092.stdout:5/629: write d4/d5/d10/da1/fcd [889670,49379] 0 2024-05-06T12:14:41.719 INFO:tasks.workunit.client.1.smithi190.stdout:4/77: unlink d2/c3 0 2024-05-06T12:14:41.721 INFO:tasks.workunit.client.1.smithi190.stdout:5/118: dwrite d2/f26 [0,4194304] 0 2024-05-06T12:14:41.722 INFO:tasks.workunit.client.1.smithi190.stdout:2/99: sync 2024-05-06T12:14:41.723 INFO:tasks.workunit.client.1.smithi190.stdout:9/260: creat db/d23/f68 x:0 0 0 2024-05-06T12:14:41.723 INFO:tasks.workunit.client.0.smithi092.stdout:1/704: getdents d3 0 2024-05-06T12:14:41.725 INFO:tasks.workunit.client.1.smithi190.stdout:5/119: write d2/d27/f1c [719393,60869] 0 2024-05-06T12:14:41.726 INFO:tasks.workunit.client.1.smithi190.stdout:9/261: write db/f3d [4181,118503] 0 2024-05-06T12:14:41.726 INFO:tasks.workunit.client.0.smithi092.stdout:5/630: creat d4/d5/d10/d31/ddf/d54/d9f/fe5 x:0 0 0 2024-05-06T12:14:41.727 INFO:tasks.workunit.client.1.smithi190.stdout:8/218: write d3/d8/f16 [1374590,92771] 0 2024-05-06T12:14:41.727 INFO:tasks.workunit.client.0.smithi092.stdout:5/631: stat d4/d34/d28/d4e/d5d/c65 0 2024-05-06T12:14:41.728 INFO:tasks.workunit.client.0.smithi092.stdout:7/818: dwrite d9/de/d10/f12 [4194304,4194304] 0 2024-05-06T12:14:41.729 INFO:tasks.workunit.client.1.smithi190.stdout:1/107: link d5/f8 d5/f22 0 2024-05-06T12:14:41.732 INFO:tasks.workunit.client.0.smithi092.stdout:2/588: link df/d5c/c8a df/d17/d53/d85/da0/caa 0 2024-05-06T12:14:41.732 INFO:tasks.workunit.client.1.smithi190.stdout:4/78: dwrite d2/d10/ff [0,4194304] 0 2024-05-06T12:14:41.733 INFO:tasks.workunit.client.1.smithi190.stdout:1/108: chown f4 29589 1 2024-05-06T12:14:41.733 INFO:tasks.workunit.client.1.smithi190.stdout:2/100: mkdir d2/d8/d19 0 2024-05-06T12:14:41.736 INFO:tasks.workunit.client.1.smithi190.stdout:9/262: truncate db/d21/d22/f51 846768 0 2024-05-06T12:14:41.737 INFO:tasks.workunit.client.0.smithi092.stdout:1/705: dwrite d3/d10/d13/d19/d21/f24 [0,4194304] 0 2024-05-06T12:14:41.739 INFO:tasks.workunit.client.0.smithi092.stdout:7/819: dwrite d9/de/d1f/f69 [0,4194304] 0 2024-05-06T12:14:41.739 INFO:tasks.workunit.client.0.smithi092.stdout:6/776: write d4/d24/d26/ffb [4331227,121044] 0 2024-05-06T12:14:41.741 INFO:tasks.workunit.client.1.smithi190.stdout:9/263: chown db/d23/d25/d47/d4b/f43 340410 1 2024-05-06T12:14:41.741 INFO:tasks.workunit.client.1.smithi190.stdout:5/120: dwrite f0 [4194304,4194304] 0 2024-05-06T12:14:41.743 INFO:tasks.workunit.client.0.smithi092.stdout:5/632: rmdir d4/d5/d10/d31/ddf/da3/dd3 0 2024-05-06T12:14:41.747 INFO:tasks.workunit.client.1.smithi190.stdout:6/102: rename df/d10 to df/d17/d1b/d1e 0 2024-05-06T12:14:41.747 INFO:tasks.workunit.client.1.smithi190.stdout:1/109: mkdir d5/d23 0 2024-05-06T12:14:41.748 INFO:tasks.workunit.client.0.smithi092.stdout:6/777: unlink d4/d24/d38/f69 0 2024-05-06T12:14:41.750 INFO:tasks.workunit.client.1.smithi190.stdout:2/101: creat d2/d8/dc/f1a x:0 0 0 2024-05-06T12:14:41.752 INFO:tasks.workunit.client.0.smithi092.stdout:9/714: dwrite d16/d1e/d7b/dce/f50 [4194304,4194304] 0 2024-05-06T12:14:41.755 INFO:tasks.workunit.client.1.smithi190.stdout:9/264: dwrite db/d21/d22/f54 [0,4194304] 0 2024-05-06T12:14:41.760 INFO:tasks.workunit.client.1.smithi190.stdout:9/265: write db/d23/f3a [1039072,79892] 0 2024-05-06T12:14:41.761 INFO:tasks.workunit.client.0.smithi092.stdout:1/706: creat d3/d10/d13/d19/fdc x:0 0 0 2024-05-06T12:14:41.761 INFO:tasks.workunit.client.1.smithi190.stdout:3/161: dread d5/ff [0,4194304] 0 2024-05-06T12:14:41.762 INFO:tasks.workunit.client.0.smithi092.stdout:1/707: write d3/d10/d13/d19/d34/d89/fa5 [5065,47501] 0 2024-05-06T12:14:41.764 INFO:tasks.workunit.client.0.smithi092.stdout:3/586: sync 2024-05-06T12:14:41.765 INFO:tasks.workunit.client.0.smithi092.stdout:8/781: sync 2024-05-06T12:14:41.765 INFO:tasks.workunit.client.0.smithi092.stdout:4/734: sync 2024-05-06T12:14:41.765 INFO:tasks.workunit.client.0.smithi092.stdout:0/913: sync 2024-05-06T12:14:41.765 INFO:tasks.workunit.client.1.smithi190.stdout:9/266: dread - db/d23/f53 zero size 2024-05-06T12:14:41.765 INFO:tasks.workunit.client.0.smithi092.stdout:9/715: symlink d16/d2c/d2f/led 0 2024-05-06T12:14:41.767 INFO:tasks.workunit.client.0.smithi092.stdout:3/587: chown f4 5492 1 2024-05-06T12:14:41.767 INFO:tasks.workunit.client.0.smithi092.stdout:1/708: rename d3/d42/d84/f7a to d3/fdd 0 2024-05-06T12:14:41.770 INFO:tasks.workunit.client.0.smithi092.stdout:4/735: chown d9/d27/d4a/f75 226174 1 2024-05-06T12:14:41.770 INFO:tasks.workunit.client.1.smithi190.stdout:7/104: dwrite d0/d1/d2/f5 [0,4194304] 0 2024-05-06T12:14:41.771 INFO:tasks.workunit.client.1.smithi190.stdout:9/267: truncate db/dd/d16/f64 667515 0 2024-05-06T12:14:41.771 INFO:tasks.workunit.client.1.smithi190.stdout:8/219: rename d3/f24 to d3/d8/d2b/f47 0 2024-05-06T12:14:41.775 INFO:tasks.workunit.client.1.smithi190.stdout:2/102: symlink d2/d10/l1b 0 2024-05-06T12:14:41.776 INFO:tasks.workunit.client.1.smithi190.stdout:2/103: write d2/fd [5919620,99477] 0 2024-05-06T12:14:41.777 INFO:tasks.workunit.client.0.smithi092.stdout:9/716: symlink d16/d2c/d2f/db1/lee 0 2024-05-06T12:14:41.778 INFO:tasks.workunit.client.1.smithi190.stdout:2/104: dread - d2/d8/dc/f1a zero size 2024-05-06T12:14:41.778 INFO:tasks.workunit.client.0.smithi092.stdout:0/914: truncate d7/dc/d28/d54/d12/f82 1018210 0 2024-05-06T12:14:41.779 INFO:tasks.workunit.client.1.smithi190.stdout:2/105: dread - d2/d8/dc/f1a zero size 2024-05-06T12:14:41.779 INFO:tasks.workunit.client.0.smithi092.stdout:4/736: fdatasync d9/f34 0 2024-05-06T12:14:41.780 INFO:tasks.workunit.client.1.smithi190.stdout:2/106: write d2/fd [7647574,26172] 0 2024-05-06T12:14:41.782 INFO:tasks.workunit.client.1.smithi190.stdout:6/103: getdents df/d12 0 2024-05-06T12:14:41.782 INFO:tasks.workunit.client.1.smithi190.stdout:6/104: read - df/f15 zero size 2024-05-06T12:14:41.783 INFO:tasks.workunit.client.0.smithi092.stdout:0/915: creat d7/dc/dd4/f131 x:0 0 0 2024-05-06T12:14:41.784 INFO:tasks.workunit.client.0.smithi092.stdout:3/588: creat d8/d24/d27/fb6 x:0 0 0 2024-05-06T12:14:41.787 INFO:tasks.workunit.client.0.smithi092.stdout:9/717: creat d16/d1e/d7b/fef x:0 0 0 2024-05-06T12:14:41.788 INFO:tasks.workunit.client.0.smithi092.stdout:2/589: truncate fe 1972298 0 2024-05-06T12:14:41.789 INFO:tasks.workunit.client.0.smithi092.stdout:0/916: rename d7/dc/d6f/da5/dc6/d100/d10c to d7/dc/d97/dc9/d12c/d132 0 2024-05-06T12:14:41.790 INFO:tasks.workunit.client.0.smithi092.stdout:2/590: chown df/d17/f3e 8858241 1 2024-05-06T12:14:41.790 INFO:tasks.workunit.client.0.smithi092.stdout:0/917: dread - d7/dc/d6f/da5/dc6/d100/f124 zero size 2024-05-06T12:14:41.791 INFO:tasks.workunit.client.0.smithi092.stdout:9/718: mknod d16/d1e/d7b/dce/de4/cf0 0 2024-05-06T12:14:41.793 INFO:tasks.workunit.client.0.smithi092.stdout:0/918: mknod d7/dc/c133 0 2024-05-06T12:14:41.794 INFO:tasks.workunit.client.0.smithi092.stdout:9/719: mknod d16/d2c/d2f/d3d/d33/d6b/d9e/cf1 0 2024-05-06T12:14:41.797 INFO:tasks.workunit.client.0.smithi092.stdout:2/591: link df/d15/d88/fa7 df/d15/d88/d59/fab 0 2024-05-06T12:14:41.800 INFO:tasks.workunit.client.0.smithi092.stdout:7/820: dread d9/f80 [0,4194304] 0 2024-05-06T12:14:41.800 INFO:tasks.workunit.client.0.smithi092.stdout:9/720: symlink d16/d1c/d58/d92/lf2 0 2024-05-06T12:14:41.801 INFO:tasks.workunit.client.0.smithi092.stdout:0/919: creat d7/dc/d6f/da5/dc6/f134 x:0 0 0 2024-05-06T12:14:41.801 INFO:tasks.workunit.client.1.smithi190.stdout:0/280: sync 2024-05-06T12:14:41.802 INFO:tasks.workunit.client.0.smithi092.stdout:2/592: rmdir df/d17/d53/d85 39 2024-05-06T12:14:41.802 INFO:tasks.workunit.client.0.smithi092.stdout:9/721: fdatasync d16/d1c/d58/d92/fe9 0 2024-05-06T12:14:41.802 INFO:tasks.workunit.client.1.smithi190.stdout:0/281: write d9/dc/d28/f3e [592633,61113] 0 2024-05-06T12:14:41.805 INFO:tasks.workunit.client.0.smithi092.stdout:9/722: read - d16/d2c/d2f/d3d/d33/d6b/fa2 zero size 2024-05-06T12:14:41.806 INFO:tasks.workunit.client.0.smithi092.stdout:2/593: creat df/d17/d51/d96/fac x:0 0 0 2024-05-06T12:14:41.807 INFO:tasks.workunit.client.0.smithi092.stdout:0/920: creat d7/dc/d28/d54/d12/d7f/db2/dbc/f135 x:0 0 0 2024-05-06T12:14:41.808 INFO:tasks.workunit.client.0.smithi092.stdout:8/782: dread d9/dc/ff [0,4194304] 0 2024-05-06T12:14:41.808 INFO:tasks.workunit.client.0.smithi092.stdout:2/594: fsync df/d1b/f1c 0 2024-05-06T12:14:41.810 INFO:tasks.workunit.client.0.smithi092.stdout:9/723: dread - d16/d2c/d2f/d3d/d33/d57/fbf zero size 2024-05-06T12:14:41.810 INFO:tasks.workunit.client.1.smithi190.stdout:0/282: dwrite d9/fb [4194304,4194304] 0 2024-05-06T12:14:41.811 INFO:tasks.workunit.client.0.smithi092.stdout:4/737: dread d9/df/d62/f9d [0,4194304] 0 2024-05-06T12:14:41.812 INFO:tasks.workunit.client.0.smithi092.stdout:8/783: mkdir d9/d21/d54/d116 0 2024-05-06T12:14:41.813 INFO:tasks.workunit.client.0.smithi092.stdout:5/633: write d4/d34/d28/d4e/d5d/d82/f63 [442962,8944] 0 2024-05-06T12:14:41.813 INFO:tasks.workunit.client.0.smithi092.stdout:2/595: symlink df/d17/d7e/lad 0 2024-05-06T12:14:41.816 INFO:tasks.workunit.client.0.smithi092.stdout:0/921: creat d7/dc/d28/d54/d1e/d26/d96/dfb/f136 x:0 0 0 2024-05-06T12:14:41.816 INFO:tasks.workunit.client.0.smithi092.stdout:8/784: chown d9/d21/d54/d79/f82 1502205 1 2024-05-06T12:14:41.819 INFO:tasks.workunit.client.1.smithi190.stdout:0/283: dwrite d9/dc/d28/f5d [0,4194304] 0 2024-05-06T12:14:41.819 INFO:tasks.workunit.client.0.smithi092.stdout:5/634: mkdir d4/d34/d28/d4e/d67/de6 0 2024-05-06T12:14:41.822 INFO:tasks.workunit.client.0.smithi092.stdout:4/738: dwrite d9/d27/d4a/f84 [0,4194304] 0 2024-05-06T12:14:41.822 INFO:tasks.workunit.client.0.smithi092.stdout:2/596: write df/d17/d51/f67 [196548,38011] 0 2024-05-06T12:14:41.823 INFO:tasks.workunit.client.0.smithi092.stdout:6/778: truncate d4/d24/d26/fa4 3912138 0 2024-05-06T12:14:41.825 INFO:tasks.workunit.client.0.smithi092.stdout:7/821: chown d9/de/d1f/d37/f8c 7208520 1 2024-05-06T12:14:41.833 INFO:tasks.workunit.client.0.smithi092.stdout:5/635: write d4/d5/d10/d31/d53/d64/f5e [3090612,82565] 0 2024-05-06T12:14:41.834 INFO:tasks.workunit.client.0.smithi092.stdout:2/597: read df/d17/d1d/f4f [650554,2503] 0 2024-05-06T12:14:41.835 INFO:tasks.workunit.client.0.smithi092.stdout:4/739: mkdir d9/dd/da6/dca/df2 0 2024-05-06T12:14:41.837 INFO:tasks.workunit.client.0.smithi092.stdout:6/779: rename d4/d24/d54/f86 to d4/d4c/d52/ddc/f100 0 2024-05-06T12:14:41.839 INFO:tasks.workunit.client.0.smithi092.stdout:7/822: fsync d9/de/d10/d17/d6a/fb9 0 2024-05-06T12:14:41.843 INFO:tasks.workunit.client.0.smithi092.stdout:0/922: creat d7/dc/d28/d54/d12/d7f/db2/d11d/f137 x:0 0 0 2024-05-06T12:14:41.846 INFO:tasks.workunit.client.0.smithi092.stdout:4/740: symlink d9/df/d62/d7b/db6/lf3 0 2024-05-06T12:14:41.847 INFO:tasks.workunit.client.0.smithi092.stdout:2/598: rename df/f2c to df/d17/d51/d96/fae 0 2024-05-06T12:14:41.847 INFO:tasks.workunit.client.0.smithi092.stdout:2/599: dread - df/d15/f95 zero size 2024-05-06T12:14:41.849 INFO:tasks.workunit.client.0.smithi092.stdout:7/823: rmdir d9/de/d10/d17/d99/dc3/dd2 39 2024-05-06T12:14:41.852 INFO:tasks.workunit.client.0.smithi092.stdout:0/923: unlink d7/dc/d28/d54/d1e/d39/l8b 0 2024-05-06T12:14:41.854 INFO:tasks.workunit.client.0.smithi092.stdout:6/780: mkdir d4/d20/d23/d30/df7/d101 0 2024-05-06T12:14:41.856 INFO:tasks.workunit.client.0.smithi092.stdout:2/600: write df/f37 [3769817,46696] 0 2024-05-06T12:14:41.857 INFO:tasks.workunit.client.0.smithi092.stdout:7/824: creat d9/de/d10/d17/d6a/f110 x:0 0 0 2024-05-06T12:14:41.860 INFO:tasks.workunit.client.0.smithi092.stdout:6/781: mkdir d4/d4c/d91/d102 0 2024-05-06T12:14:41.861 INFO:tasks.workunit.client.0.smithi092.stdout:0/924: dwrite d7/dc/d28/d54/d12/d7f/db5/ff1 [0,4194304] 0 2024-05-06T12:14:41.861 INFO:tasks.workunit.client.0.smithi092.stdout:9/724: truncate d16/d1e/f25 38148 0 2024-05-06T12:14:41.862 INFO:tasks.workunit.client.1.smithi190.stdout:3/162: creat d5/d14/d11/d21/f3c x:0 0 0 2024-05-06T12:14:41.862 INFO:tasks.workunit.client.0.smithi092.stdout:6/782: write d4/d4c/f75 [122464,48880] 0 2024-05-06T12:14:41.864 INFO:tasks.workunit.client.1.smithi190.stdout:1/110: mkdir d5/d23/d24 0 2024-05-06T12:14:41.866 INFO:tasks.workunit.client.1.smithi190.stdout:8/220: rmdir d3/d8/d25 39 2024-05-06T12:14:41.866 INFO:tasks.workunit.client.0.smithi092.stdout:0/925: truncate d7/dc/d28/d54/d1e/d26/d4e/dd7/fdb 362526 0 2024-05-06T12:14:41.868 INFO:tasks.workunit.client.0.smithi092.stdout:1/709: creat d3/d10/d13/d19/fde x:0 0 0 2024-05-06T12:14:41.868 INFO:tasks.workunit.client.0.smithi092.stdout:8/785: write d9/d21/d37/d46/d5c/d71/fca [709014,29170] 0 2024-05-06T12:14:41.868 INFO:tasks.workunit.client.0.smithi092.stdout:0/926: truncate d7/dc/d28/d54/d1e/d46/f121 551268 0 2024-05-06T12:14:41.870 INFO:tasks.workunit.client.0.smithi092.stdout:1/710: fsync d3/d10/d13/d46/fa1 0 2024-05-06T12:14:41.871 INFO:tasks.workunit.client.0.smithi092.stdout:4/741: creat d9/d2b/d65/ff4 x:0 0 0 2024-05-06T12:14:41.871 INFO:tasks.workunit.client.1.smithi190.stdout:6/105: symlink df/d17/d1b/d1e/l1f 0 2024-05-06T12:14:41.871 INFO:tasks.workunit.client.0.smithi092.stdout:4/742: fsync d9/dd/f23 0 2024-05-06T12:14:41.876 INFO:tasks.workunit.client.1.smithi190.stdout:3/163: mkdir d5/d14/d11/d2a/d3d 0 2024-05-06T12:14:41.876 INFO:tasks.workunit.client.1.smithi190.stdout:3/164: dread - d5/d7/f3a zero size 2024-05-06T12:14:41.877 INFO:tasks.workunit.client.1.smithi190.stdout:7/105: unlink d0/d1/d2/c12 0 2024-05-06T12:14:41.879 INFO:tasks.workunit.client.0.smithi092.stdout:5/636: write d4/d34/d28/d4e/f4f [381352,58161] 0 2024-05-06T12:14:41.881 INFO:tasks.workunit.client.0.smithi092.stdout:6/783: symlink d4/d24/l103 0 2024-05-06T12:14:41.881 INFO:tasks.workunit.client.1.smithi190.stdout:4/79: getdents d2 0 2024-05-06T12:14:41.882 INFO:tasks.workunit.client.0.smithi092.stdout:6/784: fdatasync d4/d20/fb2 0 2024-05-06T12:14:41.882 INFO:tasks.workunit.client.1.smithi190.stdout:2/107: symlink d2/d18/l1c 0 2024-05-06T12:14:41.883 INFO:tasks.workunit.client.1.smithi190.stdout:2/108: fdatasync d2/f12 0 2024-05-06T12:14:41.883 INFO:tasks.workunit.client.0.smithi092.stdout:0/927: mkdir d7/d138 0 2024-05-06T12:14:41.885 INFO:tasks.workunit.client.1.smithi190.stdout:2/109: write d2/d8/f15 [42167,9450] 0 2024-05-06T12:14:41.885 INFO:tasks.workunit.client.1.smithi190.stdout:0/284: symlink d9/dc/l60 0 2024-05-06T12:14:41.885 INFO:tasks.workunit.client.0.smithi092.stdout:0/928: stat d7/dc/d6f/da5 0 2024-05-06T12:14:41.885 INFO:tasks.workunit.client.0.smithi092.stdout:8/786: getdents d9/d9d/dfd 0 2024-05-06T12:14:41.887 INFO:tasks.workunit.client.1.smithi190.stdout:3/165: unlink d5/d7/d15/d1f/fd 0 2024-05-06T12:14:41.887 INFO:tasks.workunit.client.0.smithi092.stdout:9/725: creat d16/d1c/d58/ff3 x:0 0 0 2024-05-06T12:14:41.888 INFO:tasks.workunit.client.1.smithi190.stdout:7/106: mkdir d0/d1/d18/d2b 0 2024-05-06T12:14:41.889 INFO:tasks.workunit.client.0.smithi092.stdout:7/825: symlink d9/de/d10/d17/d99/db6/df1/d101/l111 0 2024-05-06T12:14:41.889 INFO:tasks.workunit.client.0.smithi092.stdout:9/726: dread d16/f1b [0,4194304] 0 2024-05-06T12:14:41.890 INFO:tasks.workunit.client.0.smithi092.stdout:5/637: unlink d4/d34/d28/d4e/d5d/d82/c5b 0 2024-05-06T12:14:41.891 INFO:tasks.workunit.client.1.smithi190.stdout:8/221: mknod d3/d8/c48 0 2024-05-06T12:14:41.891 INFO:tasks.workunit.client.0.smithi092.stdout:2/601: write df/d17/f6b [2144858,96144] 0 2024-05-06T12:14:41.891 INFO:tasks.workunit.client.1.smithi190.stdout:8/222: fsync d3/f17 0 2024-05-06T12:14:41.892 INFO:tasks.workunit.client.0.smithi092.stdout:6/785: mknod d4/d4c/d91/d93/c104 0 2024-05-06T12:14:41.893 INFO:tasks.workunit.client.0.smithi092.stdout:2/602: write df/d17/d1d/f41 [177747,18760] 0 2024-05-06T12:14:41.893 INFO:tasks.workunit.client.1.smithi190.stdout:8/223: readlink d3/d8/d2b/l33 0 2024-05-06T12:14:41.893 INFO:tasks.workunit.client.1.smithi190.stdout:4/80: creat d2/d11/f20 x:0 0 0 2024-05-06T12:14:41.894 INFO:tasks.workunit.client.1.smithi190.stdout:8/224: read - d3/f2f zero size 2024-05-06T12:14:41.894 INFO:tasks.workunit.client.0.smithi092.stdout:2/603: chown df/d17/d1d/f41 13 1 2024-05-06T12:14:41.894 INFO:tasks.workunit.client.0.smithi092.stdout:0/929: write d7/dc/d28/d54/d1e/d39/da8/dc5/fd1 [720360,54093] 0 2024-05-06T12:14:41.895 INFO:tasks.workunit.client.1.smithi190.stdout:6/106: link fd df/d17/d1b/d1e/f20 0 2024-05-06T12:14:41.897 INFO:tasks.workunit.client.0.smithi092.stdout:2/604: truncate df/d17/f6b 3191930 0 2024-05-06T12:14:41.897 INFO:tasks.workunit.client.0.smithi092.stdout:1/711: dread d3/d10/d13/d19/d21/d72/f86 [0,4194304] 0 2024-05-06T12:14:41.898 INFO:tasks.workunit.client.0.smithi092.stdout:0/930: chown d7/dc/d28/d54/d1e/d26/d4e/dd7/fdb 49 1 2024-05-06T12:14:41.899 INFO:tasks.workunit.client.1.smithi190.stdout:0/285: symlink d9/dc/d48/l61 0 2024-05-06T12:14:41.899 INFO:tasks.workunit.client.0.smithi092.stdout:4/743: dread d9/dd/f23 [0,4194304] 0 2024-05-06T12:14:41.900 INFO:tasks.workunit.client.1.smithi190.stdout:0/286: stat d9/dc/f13 0 2024-05-06T12:14:41.900 INFO:tasks.workunit.client.0.smithi092.stdout:6/786: mkdir d4/d24/d66/d105 0 2024-05-06T12:14:41.902 INFO:tasks.workunit.client.0.smithi092.stdout:8/787: rename d9/d21/c32 to d9/d5a/dd9/c117 0 2024-05-06T12:14:41.902 INFO:tasks.workunit.client.1.smithi190.stdout:2/110: dwrite d2/fe [0,4194304] 0 2024-05-06T12:14:41.905 INFO:tasks.workunit.client.0.smithi092.stdout:1/712: creat d3/d10/d13/d19/d21/d72/fdf x:0 0 0 2024-05-06T12:14:41.906 INFO:tasks.workunit.client.0.smithi092.stdout:4/744: rmdir d9/d2b/d5b 39 2024-05-06T12:14:41.906 INFO:tasks.workunit.client.0.smithi092.stdout:5/638: dwrite d4/d34/d28/f74 [0,4194304] 0 2024-05-06T12:14:41.908 INFO:tasks.workunit.client.1.smithi190.stdout:6/107: creat df/d12/f21 x:0 0 0 2024-05-06T12:14:41.910 INFO:tasks.workunit.client.0.smithi092.stdout:7/826: creat d9/de/d10/d17/d99/dc3/dd2/df7/f112 x:0 0 0 2024-05-06T12:14:41.910 INFO:tasks.workunit.client.0.smithi092.stdout:7/827: readlink d9/dd/lfa 0 2024-05-06T12:14:41.911 INFO:tasks.workunit.client.1.smithi190.stdout:7/107: mknod d0/d1/c2c 0 2024-05-06T12:14:41.912 INFO:tasks.workunit.client.0.smithi092.stdout:7/828: write d9/de/d10/d17/d99/fb3 [5564600,65586] 0 2024-05-06T12:14:41.912 INFO:tasks.workunit.client.0.smithi092.stdout:1/713: chown d3/d42 0 1 2024-05-06T12:14:41.913 INFO:tasks.workunit.client.1.smithi190.stdout:5/121: truncate f0 7872051 0 2024-05-06T12:14:41.913 INFO:tasks.workunit.client.1.smithi190.stdout:6/108: dread f1 [0,4194304] 0 2024-05-06T12:14:41.913 INFO:tasks.workunit.client.0.smithi092.stdout:4/745: creat d9/df/ff5 x:0 0 0 2024-05-06T12:14:41.914 INFO:tasks.workunit.client.0.smithi092.stdout:9/727: getdents d16/d94/da0 0 2024-05-06T12:14:41.916 INFO:tasks.workunit.client.1.smithi190.stdout:8/225: mkdir d3/d8/d25/d3b/d49 0 2024-05-06T12:14:41.917 INFO:tasks.workunit.client.1.smithi190.stdout:8/226: chown d3/c22 711 1 2024-05-06T12:14:41.919 INFO:tasks.workunit.client.0.smithi092.stdout:7/829: dread d9/de/d10/d17/d99/db6/fc8 [0,4194304] 0 2024-05-06T12:14:41.922 INFO:tasks.workunit.client.1.smithi190.stdout:7/108: rename d0/d1/f17 to d0/d1/d18/f2d 0 2024-05-06T12:14:41.923 INFO:tasks.workunit.client.1.smithi190.stdout:3/166: rmdir d5/d14/d11/d21 39 2024-05-06T12:14:41.924 INFO:tasks.workunit.client.0.smithi092.stdout:4/746: write d9/d27/da8/fe5 [1750864,70034] 0 2024-05-06T12:14:41.924 INFO:tasks.workunit.client.0.smithi092.stdout:9/728: rmdir d16/d2c/d2f/d3d/d33/d7e 39 2024-05-06T12:14:41.924 INFO:tasks.workunit.client.1.smithi190.stdout:3/167: write d5/d14/d11/f1b [3538983,61464] 0 2024-05-06T12:14:41.926 INFO:tasks.workunit.client.0.smithi092.stdout:8/788: link d9/d21/cd4 d9/d21/d72/c118 0 2024-05-06T12:14:41.927 INFO:tasks.workunit.client.1.smithi190.stdout:2/111: creat d2/d8/d19/f1d x:0 0 0 2024-05-06T12:14:41.928 INFO:tasks.workunit.client.0.smithi092.stdout:1/714: symlink d3/d42/d84/dcf/le0 0 2024-05-06T12:14:41.929 INFO:tasks.workunit.client.1.smithi190.stdout:2/112: write d2/fe [602324,121269] 0 2024-05-06T12:14:41.930 INFO:tasks.workunit.client.1.smithi190.stdout:8/227: mkdir d3/d13/d4a 0 2024-05-06T12:14:41.930 INFO:tasks.workunit.client.0.smithi092.stdout:7/830: fsync d9/f40 0 2024-05-06T12:14:41.931 INFO:tasks.workunit.client.0.smithi092.stdout:2/605: truncate df/d17/d51/f67 3100892 0 2024-05-06T12:14:41.933 INFO:tasks.workunit.client.0.smithi092.stdout:0/931: dread d7/dc/d28/d54/d1e/d26/d96/d90/f112 [0,4194304] 0 2024-05-06T12:14:41.934 INFO:tasks.workunit.client.1.smithi190.stdout:7/109: creat d0/da/f2e x:0 0 0 2024-05-06T12:14:41.934 INFO:tasks.workunit.client.0.smithi092.stdout:9/729: creat d16/d2c/d2f/d3d/d33/dde/ff4 x:0 0 0 2024-05-06T12:14:41.935 INFO:tasks.workunit.client.0.smithi092.stdout:6/787: write d4/d24/d38/d5d/d6d/dd8/ff0 [182655,120910] 0 2024-05-06T12:14:41.935 INFO:tasks.workunit.client.1.smithi190.stdout:5/122: fsync d2/f8 0 2024-05-06T12:14:41.935 INFO:tasks.workunit.client.0.smithi092.stdout:8/789: fdatasync d9/d21/d37/fa3 0 2024-05-06T12:14:41.937 INFO:tasks.workunit.client.1.smithi190.stdout:2/113: dwrite d2/d8/f15 [0,4194304] 0 2024-05-06T12:14:41.937 INFO:tasks.workunit.client.1.smithi190.stdout:7/110: write d0/f19 [843027,15192] 0 2024-05-06T12:14:41.938 INFO:tasks.workunit.client.0.smithi092.stdout:7/831: symlink d9/de/da4/l113 0 2024-05-06T12:14:41.938 INFO:tasks.workunit.client.0.smithi092.stdout:5/639: write d4/d5/d10/f33 [265332,101586] 0 2024-05-06T12:14:41.939 INFO:tasks.workunit.client.1.smithi190.stdout:2/114: chown d2/fe 8 1 2024-05-06T12:14:41.940 INFO:tasks.workunit.client.0.smithi092.stdout:1/715: rename d3/d10/d13/d19/d34/d89/f95 to d3/d10/d13/d19/d21/d72/d9a/da4/fe1 0 2024-05-06T12:14:41.941 INFO:tasks.workunit.client.1.smithi190.stdout:9/268: sync 2024-05-06T12:14:41.943 INFO:tasks.workunit.client.0.smithi092.stdout:2/606: creat df/d17/d1d/d33/faf x:0 0 0 2024-05-06T12:14:41.944 INFO:tasks.workunit.client.0.smithi092.stdout:0/932: rmdir d7/dc/d9e 39 2024-05-06T12:14:41.944 INFO:tasks.workunit.client.0.smithi092.stdout:9/730: rmdir d16/d2c/d2f/d3d/d33/d6b 39 2024-05-06T12:14:41.946 INFO:tasks.workunit.client.1.smithi190.stdout:1/111: write d5/f12 [947946,15210] 0 2024-05-06T12:14:41.947 INFO:tasks.workunit.client.0.smithi092.stdout:7/832: fdatasync d9/de/d10/d17/d99/db6/df1/f98 0 2024-05-06T12:14:41.948 INFO:tasks.workunit.client.1.smithi190.stdout:8/228: creat d3/d8/d25/f4b x:0 0 0 2024-05-06T12:14:41.949 INFO:tasks.workunit.client.0.smithi092.stdout:4/747: dread d9/d2b/f6a [0,4194304] 0 2024-05-06T12:14:41.949 INFO:tasks.workunit.client.0.smithi092.stdout:5/640: dwrite d4/d5/d10/d31/ddf/d6c/fe4 [0,4194304] 0 2024-05-06T12:14:41.950 INFO:tasks.workunit.client.0.smithi092.stdout:9/731: dread d16/d1e/f86 [0,4194304] 0 2024-05-06T12:14:41.951 INFO:tasks.workunit.client.0.smithi092.stdout:0/933: truncate d7/dc/d6f/fa1 1143336 0 2024-05-06T12:14:41.951 INFO:tasks.workunit.client.1.smithi190.stdout:2/115: dwrite d2/f14 [0,4194304] 0 2024-05-06T12:14:41.951 INFO:tasks.workunit.client.1.smithi190.stdout:5/123: dwrite d2/d27/f20 [0,4194304] 0 2024-05-06T12:14:41.955 INFO:tasks.workunit.client.1.smithi190.stdout:3/168: unlink d5/d7/f2c 0 2024-05-06T12:14:41.955 INFO:tasks.workunit.client.0.smithi092.stdout:8/790: getdents d9/dc 0 2024-05-06T12:14:41.956 INFO:tasks.workunit.client.0.smithi092.stdout:0/934: chown d7/dc/d28/d54/d1e/d46/c5e 73 1 2024-05-06T12:14:41.956 INFO:tasks.workunit.client.0.smithi092.stdout:7/833: fsync d9/de/d10/d17/d21/d2e/ff2 0 2024-05-06T12:14:41.958 INFO:tasks.workunit.client.0.smithi092.stdout:1/716: link d3/d10/d13/d19/fde d3/d42/fe2 0 2024-05-06T12:14:41.959 INFO:tasks.workunit.client.1.smithi190.stdout:6/109: creat df/f22 x:0 0 0 2024-05-06T12:14:41.959 INFO:tasks.workunit.client.1.smithi190.stdout:6/110: chown f7 2 1 2024-05-06T12:14:41.960 INFO:tasks.workunit.client.0.smithi092.stdout:4/748: creat d9/dd/da6/ff6 x:0 0 0 2024-05-06T12:14:41.961 INFO:tasks.workunit.client.1.smithi190.stdout:9/269: creat db/dd/d16/d4c/f69 x:0 0 0 2024-05-06T12:14:41.962 INFO:tasks.workunit.client.1.smithi190.stdout:5/124: read d2/fc [2601711,97152] 0 2024-05-06T12:14:41.962 INFO:tasks.workunit.client.1.smithi190.stdout:9/270: dread - db/f5a zero size 2024-05-06T12:14:41.963 INFO:tasks.workunit.client.1.smithi190.stdout:9/271: chown f6 84 1 2024-05-06T12:14:41.964 INFO:tasks.workunit.client.1.smithi190.stdout:7/111: dwrite d0/d1/f24 [0,4194304] 0 2024-05-06T12:14:41.965 INFO:tasks.workunit.client.1.smithi190.stdout:5/125: write d2/d27/f20 [809120,107930] 0 2024-05-06T12:14:41.967 INFO:tasks.workunit.client.1.smithi190.stdout:4/81: truncate d2/d11/d16/f18 3621593 0 2024-05-06T12:14:41.971 INFO:tasks.workunit.client.0.smithi092.stdout:9/732: rename d16/d1e/d7b/d8d/fdf to d16/d2c/d2f/d3d/d33/dde/ff5 0 2024-05-06T12:14:41.972 INFO:tasks.workunit.client.0.smithi092.stdout:8/791: truncate d9/d21/d54/d79/d8a/fb8 762352 0 2024-05-06T12:14:41.972 INFO:tasks.workunit.client.1.smithi190.stdout:0/287: truncate d9/df/d21/f40 2987559 0 2024-05-06T12:14:41.973 INFO:tasks.workunit.client.0.smithi092.stdout:8/792: stat d9/d21/d54/d79/caf 0 2024-05-06T12:14:41.974 INFO:tasks.workunit.client.0.smithi092.stdout:1/717: creat d3/d10/d13/d19/d21/d88/d8b/fe3 x:0 0 0 2024-05-06T12:14:41.975 INFO:tasks.workunit.client.1.smithi190.stdout:6/111: mknod df/d17/c23 0 2024-05-06T12:14:41.976 INFO:tasks.workunit.client.1.smithi190.stdout:8/229: mknod d3/d8/c4c 0 2024-05-06T12:14:41.977 INFO:tasks.workunit.client.0.smithi092.stdout:7/834: truncate d9/de/d10/d17/d21/f3e 493590 0 2024-05-06T12:14:41.977 INFO:tasks.workunit.client.0.smithi092.stdout:2/607: rmdir df/d17/d1d/d33 39 2024-05-06T12:14:41.977 INFO:tasks.workunit.client.1.smithi190.stdout:2/116: rmdir d2/d8 39 2024-05-06T12:14:41.978 INFO:tasks.workunit.client.0.smithi092.stdout:4/749: creat d9/d2b/d7c/dab/dd8/ff7 x:0 0 0 2024-05-06T12:14:41.978 INFO:tasks.workunit.client.1.smithi190.stdout:8/230: chown d3/d35/d3a 1374280487 1 2024-05-06T12:14:41.979 INFO:tasks.workunit.client.0.smithi092.stdout:0/935: getdents d7/dc/d28/d54/d1e/d46/d63/d80 0 2024-05-06T12:14:41.980 INFO:tasks.workunit.client.1.smithi190.stdout:1/112: creat d5/d23/d24/f25 x:0 0 0 2024-05-06T12:14:41.980 INFO:tasks.workunit.client.0.smithi092.stdout:6/788: truncate d4/d24/d54/f77 1030638 0 2024-05-06T12:14:41.981 INFO:tasks.workunit.client.1.smithi190.stdout:0/288: dwrite d9/dc/d28/d4c/f4b [0,4194304] 0 2024-05-06T12:14:41.983 INFO:tasks.workunit.client.1.smithi190.stdout:0/289: chown d9/df/d21/d31/l5c 89 1 2024-05-06T12:14:41.985 INFO:tasks.workunit.client.0.smithi092.stdout:2/608: creat df/d15/d88/d59/fb0 x:0 0 0 2024-05-06T12:14:41.985 INFO:tasks.workunit.client.1.smithi190.stdout:7/112: unlink d0/d14/l1a 0 2024-05-06T12:14:41.986 INFO:tasks.workunit.client.1.smithi190.stdout:4/82: mknod d2/d11/c21 0 2024-05-06T12:14:41.988 INFO:tasks.workunit.client.0.smithi092.stdout:9/733: dread d16/d2c/d2f/d3d/d33/d6b/f81 [0,4194304] 0 2024-05-06T12:14:41.989 INFO:tasks.workunit.client.1.smithi190.stdout:6/112: symlink df/d17/d1b/d1e/l24 0 2024-05-06T12:14:41.991 INFO:tasks.workunit.client.0.smithi092.stdout:8/793: unlink d9/d21/d37/d46/fe9 0 2024-05-06T12:14:41.991 INFO:tasks.workunit.client.1.smithi190.stdout:2/117: mknod d2/d10/c1e 0 2024-05-06T12:14:41.993 INFO:tasks.workunit.client.1.smithi190.stdout:8/231: mknod d3/d8/d2b/c4d 0 2024-05-06T12:14:41.995 INFO:tasks.workunit.client.0.smithi092.stdout:2/609: unlink df/d15/d88/f82 0 2024-05-06T12:14:41.996 INFO:tasks.workunit.client.1.smithi190.stdout:9/272: symlink db/d23/l6a 0 2024-05-06T12:14:41.996 INFO:tasks.workunit.client.0.smithi092.stdout:5/641: dwrite d4/d59/f83 [4194304,4194304] 0 2024-05-06T12:14:41.996 INFO:tasks.workunit.client.1.smithi190.stdout:9/273: readlink la 0 2024-05-06T12:14:41.997 INFO:tasks.workunit.client.1.smithi190.stdout:5/126: fsync d2/d27/f22 0 2024-05-06T12:14:41.998 INFO:tasks.workunit.client.0.smithi092.stdout:2/610: chown df/d17/d51/d96/fac 4550556 1 2024-05-06T12:14:41.998 INFO:tasks.workunit.client.1.smithi190.stdout:5/127: readlink d2/da/de/l1e 0 2024-05-06T12:14:42.000 INFO:tasks.workunit.client.0.smithi092.stdout:5/642: write d4/d34/d28/d4e/d5d/d82/d1f/d20/f76 [220550,122860] 0 2024-05-06T12:14:42.000 INFO:tasks.workunit.client.0.smithi092.stdout:2/611: readlink df/d17/d53/da9/l9a 0 2024-05-06T12:14:42.002 INFO:tasks.workunit.client.0.smithi092.stdout:2/612: fsync df/d17/d51/f8c 0 2024-05-06T12:14:42.002 INFO:tasks.workunit.client.0.smithi092.stdout:8/794: mknod d9/d5a/dd9/c119 0 2024-05-06T12:14:42.003 INFO:tasks.workunit.client.0.smithi092.stdout:8/795: stat d9/d21/d37/dbd/ddb 0 2024-05-06T12:14:42.004 INFO:tasks.workunit.client.1.smithi190.stdout:4/83: symlink d2/d10/l22 0 2024-05-06T12:14:42.005 INFO:tasks.workunit.client.1.smithi190.stdout:4/84: dread - d2/d10/f1c zero size 2024-05-06T12:14:42.006 INFO:tasks.workunit.client.1.smithi190.stdout:6/113: fsync df/d12/f13 0 2024-05-06T12:14:42.009 INFO:tasks.workunit.client.1.smithi190.stdout:8/232: readlink d3/d8/l41 0 2024-05-06T12:14:42.009 INFO:tasks.workunit.client.1.smithi190.stdout:2/118: dread - d2/d8/d19/f1d zero size 2024-05-06T12:14:42.012 INFO:tasks.workunit.client.0.smithi092.stdout:8/796: dwrite d9/d21/d37/d46/d5c/fde [0,4194304] 0 2024-05-06T12:14:42.014 INFO:tasks.workunit.client.0.smithi092.stdout:6/789: dread d4/f11 [0,4194304] 0 2024-05-06T12:14:42.016 INFO:tasks.workunit.client.0.smithi092.stdout:7/835: write d9/de/d1f/d37/d48/d97/fbf [882598,25809] 0 2024-05-06T12:14:42.016 INFO:tasks.workunit.client.0.smithi092.stdout:6/790: chown d4/d24/d26/dae/dc0/fad 32994293 1 2024-05-06T12:14:42.017 INFO:tasks.workunit.client.0.smithi092.stdout:2/613: mknod df/d1b/da5/cb1 0 2024-05-06T12:14:42.018 INFO:tasks.workunit.client.0.smithi092.stdout:2/614: dread - df/f7b zero size 2024-05-06T12:14:42.018 INFO:tasks.workunit.client.1.smithi190.stdout:1/113: link d5/d15/f1a d5/d15/d1e/f26 0 2024-05-06T12:14:42.019 INFO:tasks.workunit.client.0.smithi092.stdout:5/643: symlink d4/d5/d10/d31/ddf/d54/d73/le7 0 2024-05-06T12:14:42.020 INFO:tasks.workunit.client.1.smithi190.stdout:1/114: write d5/d23/d24/f25 [75530,44808] 0 2024-05-06T12:14:42.020 INFO:tasks.workunit.client.0.smithi092.stdout:0/936: getdents d7/dc/d28/d54/d12/d7f/db2/d11d 0 2024-05-06T12:14:42.022 INFO:tasks.workunit.client.0.smithi092.stdout:9/734: dread d16/d1c/f7c [0,4194304] 0 2024-05-06T12:14:42.022 INFO:tasks.workunit.client.0.smithi092.stdout:8/797: mknod d9/dc/d4a/d5f/da8/ddf/c11a 0 2024-05-06T12:14:42.023 INFO:tasks.workunit.client.1.smithi190.stdout:7/113: mkdir d0/d1/d2f 0 2024-05-06T12:14:42.024 INFO:tasks.workunit.client.1.smithi190.stdout:4/85: creat d2/f23 x:0 0 0 2024-05-06T12:14:42.025 INFO:tasks.workunit.client.0.smithi092.stdout:6/791: mkdir d4/d20/d23/d30/df7/d106 0 2024-05-06T12:14:42.026 INFO:tasks.workunit.client.0.smithi092.stdout:5/644: mknod d4/d5/d10/da1/ce8 0 2024-05-06T12:14:42.027 INFO:tasks.workunit.client.1.smithi190.stdout:6/114: creat df/d17/d1b/f25 x:0 0 0 2024-05-06T12:14:42.027 INFO:tasks.workunit.client.0.smithi092.stdout:6/792: write d4/d24/d26/f4b [3298261,78935] 0 2024-05-06T12:14:42.027 INFO:tasks.workunit.client.1.smithi190.stdout:6/115: stat fd 0 2024-05-06T12:14:42.029 INFO:tasks.workunit.client.0.smithi092.stdout:4/750: truncate d9/d2b/d5b/f85 540203 0 2024-05-06T12:14:42.029 INFO:tasks.workunit.client.1.smithi190.stdout:5/128: dread d2/d27/f22 [0,4194304] 0 2024-05-06T12:14:42.030 INFO:tasks.workunit.client.1.smithi190.stdout:5/129: fdatasync d2/f3 0 2024-05-06T12:14:42.030 INFO:tasks.workunit.client.0.smithi092.stdout:0/937: dwrite d7/dc/d28/d54/d1e/d26/d3a/f95 [0,4194304] 0 2024-05-06T12:14:42.031 INFO:tasks.workunit.client.0.smithi092.stdout:9/735: creat d16/d2c/d2f/d75/ff6 x:0 0 0 2024-05-06T12:14:42.034 INFO:tasks.workunit.client.1.smithi190.stdout:1/115: dread d5/fb [0,4194304] 0 2024-05-06T12:14:42.035 INFO:tasks.workunit.client.0.smithi092.stdout:0/938: write d7/dc/d28/d54/d1e/d5d/f7c [3772319,35467] 0 2024-05-06T12:14:42.035 INFO:tasks.workunit.client.1.smithi190.stdout:8/233: dwrite d3/fc [0,4194304] 0 2024-05-06T12:14:42.038 INFO:tasks.workunit.client.0.smithi092.stdout:6/793: dread d4/d24/d38/d5d/fa3 [0,4194304] 0 2024-05-06T12:14:42.039 INFO:tasks.workunit.client.0.smithi092.stdout:7/836: creat d9/de/d10/d17/d21/f114 x:0 0 0 2024-05-06T12:14:42.040 INFO:tasks.workunit.client.0.smithi092.stdout:4/751: symlink d9/d27/da8/db3/dbf/lf8 0 2024-05-06T12:14:42.043 INFO:tasks.workunit.client.1.smithi190.stdout:4/86: dwrite d2/f1e [0,4194304] 0 2024-05-06T12:14:42.046 INFO:tasks.workunit.client.0.smithi092.stdout:2/615: getdents df/d17 0 2024-05-06T12:14:42.047 INFO:tasks.workunit.client.0.smithi092.stdout:9/736: creat d16/d2c/d2f/d3d/d33/d7e/dad/ff7 x:0 0 0 2024-05-06T12:14:42.049 INFO:tasks.workunit.client.0.smithi092.stdout:8/798: dread d9/d21/d54/f63 [0,4194304] 0 2024-05-06T12:14:42.050 INFO:tasks.workunit.client.0.smithi092.stdout:5/645: creat d4/d34/d28/d4e/d5d/d82/fe9 x:0 0 0 2024-05-06T12:14:42.053 INFO:tasks.workunit.client.0.smithi092.stdout:7/837: mknod d9/d62/d108/d10d/c115 0 2024-05-06T12:14:42.057 INFO:tasks.workunit.client.0.smithi092.stdout:0/939: creat d7/dc/d31/f139 x:0 0 0 2024-05-06T12:14:42.060 INFO:tasks.workunit.client.0.smithi092.stdout:9/737: rename d16/d2c/d4b/d76/f83 to d16/d1e/d7b/dce/de4/ff8 0 2024-05-06T12:14:42.060 INFO:tasks.workunit.client.0.smithi092.stdout:8/799: mknod d9/d21/d54/c11b 0 2024-05-06T12:14:42.062 INFO:tasks.workunit.client.0.smithi092.stdout:8/800: chown d9/d21/d37/d46/d5c/d71/d75/fef 22 1 2024-05-06T12:14:42.064 INFO:tasks.workunit.client.0.smithi092.stdout:2/616: rmdir df/d17/d1d/d33/d4d/d83 39 2024-05-06T12:14:42.064 INFO:tasks.workunit.client.0.smithi092.stdout:8/801: read d9/f42 [125255,54230] 0 2024-05-06T12:14:42.065 INFO:tasks.workunit.client.0.smithi092.stdout:2/617: chown df/f7b 660655342 1 2024-05-06T12:14:42.065 INFO:tasks.workunit.client.0.smithi092.stdout:0/940: rmdir d7/dc/d28/d54/d1e/d46/d63/d80 39 2024-05-06T12:14:42.065 INFO:tasks.workunit.client.0.smithi092.stdout:8/802: write d9/d21/dc0/fcc [555058,14784] 0 2024-05-06T12:14:42.067 INFO:tasks.workunit.client.0.smithi092.stdout:2/618: read df/d17/d1d/f46 [1066436,5985] 0 2024-05-06T12:14:42.068 INFO:tasks.workunit.client.0.smithi092.stdout:6/794: write d4/d20/d23/d30/f6f [619304,35424] 0 2024-05-06T12:14:42.071 INFO:tasks.workunit.client.0.smithi092.stdout:7/838: mknod d9/c116 0 2024-05-06T12:14:42.074 INFO:tasks.workunit.client.0.smithi092.stdout:4/752: getdents d9 0 2024-05-06T12:14:42.079 INFO:tasks.workunit.client.0.smithi092.stdout:9/738: dwrite d16/f29 [0,4194304] 0 2024-05-06T12:14:42.080 INFO:tasks.workunit.client.0.smithi092.stdout:7/839: rmdir d9/de/d10/d17/d21/d2e/d9c 39 2024-05-06T12:14:42.086 INFO:tasks.workunit.client.0.smithi092.stdout:9/739: dread d16/f1b [0,4194304] 0 2024-05-06T12:14:42.087 INFO:tasks.workunit.client.1.smithi190.stdout:0/290: truncate d9/dc/d28/f5d 3642470 0 2024-05-06T12:14:42.090 INFO:tasks.workunit.client.1.smithi190.stdout:3/169: unlink d5/d14/d11/d21/f3c 0 2024-05-06T12:14:42.091 INFO:tasks.workunit.client.1.smithi190.stdout:9/274: link db/f52 db/d23/d29/f6b 0 2024-05-06T12:14:42.091 INFO:tasks.workunit.client.0.smithi092.stdout:8/803: dwrite d9/d21/d37/d46/d5c/d71/d75/fef [0,4194304] 0 2024-05-06T12:14:42.092 INFO:tasks.workunit.client.0.smithi092.stdout:0/941: dwrite d7/dc/d28/d54/d1e/d26/d96/dfb/f102 [0,4194304] 0 2024-05-06T12:14:42.092 INFO:tasks.workunit.client.1.smithi190.stdout:9/275: read db/d21/d22/f56 [2797747,7021] 0 2024-05-06T12:14:42.092 INFO:tasks.workunit.client.1.smithi190.stdout:9/276: chown db/d23/l50 14918 1 2024-05-06T12:14:42.094 INFO:tasks.workunit.client.1.smithi190.stdout:7/114: creat d0/d1/d18/d2b/f30 x:0 0 0 2024-05-06T12:14:42.094 INFO:tasks.workunit.client.1.smithi190.stdout:9/277: dread - db/dd/d16/d4c/f69 zero size 2024-05-06T12:14:42.094 INFO:tasks.workunit.client.1.smithi190.stdout:2/119: creat d2/f1f x:0 0 0 2024-05-06T12:14:42.096 INFO:tasks.workunit.client.1.smithi190.stdout:7/115: fdatasync d0/d1/d2/f1b 0 2024-05-06T12:14:42.096 INFO:tasks.workunit.client.1.smithi190.stdout:1/116: unlink d5/f19 0 2024-05-06T12:14:42.097 INFO:tasks.workunit.client.1.smithi190.stdout:8/234: creat d3/d8/d25/d3b/f4e x:0 0 0 2024-05-06T12:14:42.099 INFO:tasks.workunit.client.1.smithi190.stdout:0/291: rmdir d9/dc/d28/d4c 39 2024-05-06T12:14:42.102 INFO:tasks.workunit.client.0.smithi092.stdout:2/619: truncate df/d17/f3e 678839 0 2024-05-06T12:14:42.103 INFO:tasks.workunit.client.0.smithi092.stdout:9/740: write d16/d2c/d2f/d3d/f61 [309116,119951] 0 2024-05-06T12:14:42.106 INFO:tasks.workunit.client.1.smithi190.stdout:0/292: dwrite d9/f1a [0,4194304] 0 2024-05-06T12:14:42.108 INFO:tasks.workunit.client.1.smithi190.stdout:2/120: mkdir d2/d8/dc/d20 0 2024-05-06T12:14:42.109 INFO:tasks.workunit.client.1.smithi190.stdout:7/116: mknod d0/d14/c31 0 2024-05-06T12:14:42.109 INFO:tasks.workunit.client.0.smithi092.stdout:5/646: dwrite d4/d5/d10/fba [0,4194304] 0 2024-05-06T12:14:42.109 INFO:tasks.workunit.client.0.smithi092.stdout:4/753: dwrite d9/d27/d4a/f75 [4194304,4194304] 0 2024-05-06T12:14:42.112 INFO:tasks.workunit.client.1.smithi190.stdout:4/87: symlink d2/d11/l24 0 2024-05-06T12:14:42.116 INFO:tasks.workunit.client.1.smithi190.stdout:5/130: creat d2/f28 x:0 0 0 2024-05-06T12:14:42.117 INFO:tasks.workunit.client.0.smithi092.stdout:7/840: dwrite d9/de/d10/d17/faf [0,4194304] 0 2024-05-06T12:14:42.121 INFO:tasks.workunit.client.1.smithi190.stdout:7/117: dread d0/d1/f21 [0,4194304] 0 2024-05-06T12:14:42.122 INFO:tasks.workunit.client.1.smithi190.stdout:0/293: creat d9/df/f62 x:0 0 0 2024-05-06T12:14:42.124 INFO:tasks.workunit.client.0.smithi092.stdout:2/620: creat df/d17/d7d/fb2 x:0 0 0 2024-05-06T12:14:42.126 INFO:tasks.workunit.client.0.smithi092.stdout:2/621: truncate df/d17/d1d/f21 6016715 0 2024-05-06T12:14:42.126 INFO:tasks.workunit.client.0.smithi092.stdout:4/754: unlink d9/d27/fe4 0 2024-05-06T12:14:42.128 INFO:tasks.workunit.client.0.smithi092.stdout:0/942: symlink d7/dc/d6f/da5/dc6/l13a 0 2024-05-06T12:14:42.129 INFO:tasks.workunit.client.1.smithi190.stdout:5/131: readlink d2/d27/l25 0 2024-05-06T12:14:42.130 INFO:tasks.workunit.client.0.smithi092.stdout:9/741: rename d16/d2c/d3f/c41 to d16/d1e/d7b/d8d/cf9 0 2024-05-06T12:14:42.131 INFO:tasks.workunit.client.1.smithi190.stdout:3/170: symlink d5/d14/d11/d2a/d3d/l3e 0 2024-05-06T12:14:42.131 INFO:tasks.workunit.client.0.smithi092.stdout:2/622: mknod df/d17/d1d/d33/cb3 0 2024-05-06T12:14:42.134 INFO:tasks.workunit.client.1.smithi190.stdout:9/278: truncate db/d23/d29/f6b 15658 0 2024-05-06T12:14:42.134 INFO:tasks.workunit.client.1.smithi190.stdout:2/121: rmdir d2 39 2024-05-06T12:14:42.135 INFO:tasks.workunit.client.0.smithi092.stdout:9/742: symlink d16/d1c/d58/d92/lfa 0 2024-05-06T12:14:42.135 INFO:tasks.workunit.client.1.smithi190.stdout:0/294: readlink d9/df/d21/l30 0 2024-05-06T12:14:42.137 INFO:tasks.workunit.client.0.smithi092.stdout:2/623: getdents df/d17/d1d 0 2024-05-06T12:14:42.139 INFO:tasks.workunit.client.1.smithi190.stdout:1/117: rename d5/f13 to d5/d15/d1e/f27 0 2024-05-06T12:14:42.139 INFO:tasks.workunit.client.0.smithi092.stdout:9/743: link d16/d2c/d2f/d3d/d33/d37/f6f d16/d2c/d2f/d3d/d33/dde/ffb 0 2024-05-06T12:14:42.140 INFO:tasks.workunit.client.1.smithi190.stdout:7/118: mkdir d0/d1/d32 0 2024-05-06T12:14:42.141 INFO:tasks.workunit.client.1.smithi190.stdout:1/118: read d5/f6 [192174,129320] 0 2024-05-06T12:14:42.142 INFO:tasks.workunit.client.1.smithi190.stdout:4/88: creat d2/f25 x:0 0 0 2024-05-06T12:14:42.142 INFO:tasks.workunit.client.1.smithi190.stdout:9/279: dwrite db/dd/d16/d4c/f69 [0,4194304] 0 2024-05-06T12:14:42.144 INFO:tasks.workunit.client.1.smithi190.stdout:9/280: chown db/dd/l15 4059489 1 2024-05-06T12:14:42.144 INFO:tasks.workunit.client.1.smithi190.stdout:4/89: truncate d2/d11/f20 993191 0 2024-05-06T12:14:42.145 INFO:tasks.workunit.client.1.smithi190.stdout:9/281: readlink db/d23/l50 0 2024-05-06T12:14:42.146 INFO:tasks.workunit.client.1.smithi190.stdout:4/90: truncate d2/f25 514227 0 2024-05-06T12:14:42.148 INFO:tasks.workunit.client.1.smithi190.stdout:4/91: truncate d2/f1e 4437320 0 2024-05-06T12:14:42.150 INFO:tasks.workunit.client.1.smithi190.stdout:3/171: unlink d5/l18 0 2024-05-06T12:14:42.151 INFO:tasks.workunit.client.1.smithi190.stdout:0/295: dread d9/df/d21/d31/f34 [0,4194304] 0 2024-05-06T12:14:42.154 INFO:tasks.workunit.client.1.smithi190.stdout:1/119: creat d5/d23/d24/f28 x:0 0 0 2024-05-06T12:14:42.154 INFO:tasks.workunit.client.1.smithi190.stdout:8/235: truncate d3/d8/f1b 2196869 0 2024-05-06T12:14:42.155 INFO:tasks.workunit.client.1.smithi190.stdout:5/132: link d2/d27/l21 d2/da/de/l29 0 2024-05-06T12:14:42.156 INFO:tasks.workunit.client.1.smithi190.stdout:9/282: creat db/dd/d16/f6c x:0 0 0 2024-05-06T12:14:42.157 INFO:tasks.workunit.client.1.smithi190.stdout:5/133: write d2/da/f1b [1260052,68699] 0 2024-05-06T12:14:42.159 INFO:tasks.workunit.client.1.smithi190.stdout:4/92: chown d2/d10/c17 3902040 1 2024-05-06T12:14:42.160 INFO:tasks.workunit.client.1.smithi190.stdout:4/93: truncate d2/d10/f1c 704261 0 2024-05-06T12:14:42.161 INFO:tasks.workunit.client.1.smithi190.stdout:4/94: write d2/d11/f14 [2433941,67465] 0 2024-05-06T12:14:42.162 INFO:tasks.workunit.client.1.smithi190.stdout:0/296: dread - d9/dc/d28/d4c/f53 zero size 2024-05-06T12:14:42.164 INFO:tasks.workunit.client.1.smithi190.stdout:7/119: mkdir d0/d1/d2f/d33 0 2024-05-06T12:14:42.164 INFO:tasks.workunit.client.1.smithi190.stdout:1/120: mknod d5/c29 0 2024-05-06T12:14:42.165 INFO:tasks.workunit.client.1.smithi190.stdout:1/121: stat d5/d15/d1e/l21 0 2024-05-06T12:14:42.165 INFO:tasks.workunit.client.1.smithi190.stdout:1/122: read d5/fb [293987,24932] 0 2024-05-06T12:14:42.167 INFO:tasks.workunit.client.1.smithi190.stdout:8/236: rename d3/d8/f16 to d3/d13/d4a/f4f 0 2024-05-06T12:14:42.169 INFO:tasks.workunit.client.1.smithi190.stdout:3/172: symlink d5/d14/d11/l3f 0 2024-05-06T12:14:42.169 INFO:tasks.workunit.client.1.smithi190.stdout:3/173: stat d5/d14/l19 0 2024-05-06T12:14:42.170 INFO:tasks.workunit.client.1.smithi190.stdout:4/95: read - d2/f1b zero size 2024-05-06T12:14:42.171 INFO:tasks.workunit.client.1.smithi190.stdout:9/283: dread db/d23/d25/f37 [0,4194304] 0 2024-05-06T12:14:42.173 INFO:tasks.workunit.client.1.smithi190.stdout:2/122: rename d2/d18 to d2/d21 0 2024-05-06T12:14:42.173 INFO:tasks.workunit.client.1.smithi190.stdout:2/123: chown d2/fe 121505051 1 2024-05-06T12:14:42.175 INFO:tasks.workunit.client.1.smithi190.stdout:1/123: creat d5/d15/f2a x:0 0 0 2024-05-06T12:14:42.177 INFO:tasks.workunit.client.1.smithi190.stdout:4/96: chown d2/d11/d16/f18 17291 1 2024-05-06T12:14:42.179 INFO:tasks.workunit.client.1.smithi190.stdout:0/297: mkdir d9/dc/d63 0 2024-05-06T12:14:42.179 INFO:tasks.workunit.client.1.smithi190.stdout:0/298: chown d9/c2d 1066 1 2024-05-06T12:14:42.180 INFO:tasks.workunit.client.1.smithi190.stdout:2/124: rmdir d2/d10 39 2024-05-06T12:14:42.181 INFO:tasks.workunit.client.1.smithi190.stdout:1/124: mkdir d5/d23/d2b 0 2024-05-06T12:14:42.183 INFO:tasks.workunit.client.1.smithi190.stdout:9/284: link db/d21/d22/f54 db/d23/d29/f6d 0 2024-05-06T12:14:42.186 INFO:tasks.workunit.client.1.smithi190.stdout:2/125: creat d2/d21/f22 x:0 0 0 2024-05-06T12:14:42.188 INFO:tasks.workunit.client.1.smithi190.stdout:1/125: creat d5/d23/d24/f2c x:0 0 0 2024-05-06T12:14:42.188 INFO:tasks.workunit.client.1.smithi190.stdout:1/126: readlink d5/l14 0 2024-05-06T12:14:42.189 INFO:tasks.workunit.client.1.smithi190.stdout:1/127: dread - d5/d15/f2a zero size 2024-05-06T12:14:42.191 INFO:tasks.workunit.client.0.smithi092.stdout:9/744: rename d16/d2c/d2f/d3d/d33/d6b/l84 to d16/d1e/da1/d78/lfc 0 2024-05-06T12:14:42.192 INFO:tasks.workunit.client.1.smithi190.stdout:9/285: dread db/dd/f34 [0,4194304] 0 2024-05-06T12:14:42.192 INFO:tasks.workunit.client.1.smithi190.stdout:2/126: dread d2/fe [0,4194304] 0 2024-05-06T12:14:42.193 INFO:tasks.workunit.client.1.smithi190.stdout:4/97: write d2/f4 [1387413,106422] 0 2024-05-06T12:14:42.196 INFO:tasks.workunit.client.0.smithi092.stdout:5/647: dwrite d4/f68 [0,4194304] 0 2024-05-06T12:14:42.196 INFO:tasks.workunit.client.1.smithi190.stdout:2/127: write d2/f12 [2964371,60500] 0 2024-05-06T12:14:42.197 INFO:tasks.workunit.client.0.smithi092.stdout:9/745: write d16/d1e/d7b/dce/f51 [3026292,27660] 0 2024-05-06T12:14:42.198 INFO:tasks.workunit.client.1.smithi190.stdout:0/299: dread d9/f26 [0,4194304] 0 2024-05-06T12:14:42.199 INFO:tasks.workunit.client.0.smithi092.stdout:4/755: dwrite d9/d2b/d5b/f85 [0,4194304] 0 2024-05-06T12:14:42.199 INFO:tasks.workunit.client.0.smithi092.stdout:5/648: chown d4/d34/d28/d4e/d4b 52 1 2024-05-06T12:14:42.200 INFO:tasks.workunit.client.1.smithi190.stdout:1/128: creat d5/f2d x:0 0 0 2024-05-06T12:14:42.201 INFO:tasks.workunit.client.1.smithi190.stdout:3/174: mkdir d5/d7/d15/d1f/d33/d40 0 2024-05-06T12:14:42.202 INFO:tasks.workunit.client.0.smithi092.stdout:5/649: stat d4/d5/d10/d31/ddf/d6c/l86 0 2024-05-06T12:14:42.208 INFO:tasks.workunit.client.0.smithi092.stdout:9/746: creat d16/d2c/d2f/d3d/ffd x:0 0 0 2024-05-06T12:14:42.208 INFO:tasks.workunit.client.0.smithi092.stdout:5/650: write d4/d5/d10/d31/ddf/d54/d9f/fe5 [847101,80909] 0 2024-05-06T12:14:42.209 INFO:tasks.workunit.client.0.smithi092.stdout:4/756: dread d9/dd/f23 [0,4194304] 0 2024-05-06T12:14:42.210 INFO:tasks.workunit.client.0.smithi092.stdout:7/841: dread d9/f102 [0,4194304] 0 2024-05-06T12:14:42.213 INFO:tasks.workunit.client.1.smithi190.stdout:3/175: dwrite d5/d7/d15/d1f/f3b [0,4194304] 0 2024-05-06T12:14:42.214 INFO:tasks.workunit.client.0.smithi092.stdout:2/624: dread df/d17/d1d/d33/f81 [0,4194304] 0 2024-05-06T12:14:42.214 INFO:tasks.workunit.client.0.smithi092.stdout:7/842: unlink d9/de/d10/c2c 0 2024-05-06T12:14:42.215 INFO:tasks.workunit.client.0.smithi092.stdout:2/625: chown df/d17/f2a 8531 1 2024-05-06T12:14:42.215 INFO:tasks.workunit.client.0.smithi092.stdout:9/747: write d16/d2c/d2f/d3d/d33/d6b/f77 [726500,2971] 0 2024-05-06T12:14:42.216 INFO:tasks.workunit.client.0.smithi092.stdout:4/757: truncate d9/df/f46 3934138 0 2024-05-06T12:14:42.216 INFO:tasks.workunit.client.0.smithi092.stdout:7/843: stat d9/de/d10/d17/d99/f83 0 2024-05-06T12:14:42.216 INFO:tasks.workunit.client.1.smithi190.stdout:1/129: symlink d5/d23/l2e 0 2024-05-06T12:14:42.217 INFO:tasks.workunit.client.0.smithi092.stdout:5/651: symlink d4/d5/d10/d31/ddf/d54/d9f/lea 0 2024-05-06T12:14:42.217 INFO:tasks.workunit.client.1.smithi190.stdout:1/130: fdatasync d5/f2d 0 2024-05-06T12:14:42.218 INFO:tasks.workunit.client.1.smithi190.stdout:8/237: dread d3/f12 [0,4194304] 0 2024-05-06T12:14:42.218 INFO:tasks.workunit.client.1.smithi190.stdout:8/238: chown d3/f6 425685 1 2024-05-06T12:14:42.219 INFO:tasks.workunit.client.1.smithi190.stdout:8/239: chown d3/d8 59578 1 2024-05-06T12:14:42.220 INFO:tasks.workunit.client.0.smithi092.stdout:5/652: dread - d4/d34/d28/d4e/d5d/d82/d1f/fcf zero size 2024-05-06T12:14:42.223 INFO:tasks.workunit.client.1.smithi190.stdout:2/128: mknod d2/d10/c23 0 2024-05-06T12:14:42.223 INFO:tasks.workunit.client.0.smithi092.stdout:4/758: symlink d9/d2b/d5b/daf/lf9 0 2024-05-06T12:14:42.224 INFO:tasks.workunit.client.0.smithi092.stdout:7/844: symlink d9/de/d10/d17/d99/da6/l117 0 2024-05-06T12:14:42.226 INFO:tasks.workunit.client.0.smithi092.stdout:2/626: dwrite df/d17/d1d/f2e [0,4194304] 0 2024-05-06T12:14:42.226 INFO:tasks.workunit.client.0.smithi092.stdout:7/845: fsync d9/fe8 0 2024-05-06T12:14:42.226 INFO:tasks.workunit.client.1.smithi190.stdout:6/116: sync 2024-05-06T12:14:42.226 INFO:tasks.workunit.client.1.smithi190.stdout:2/129: readlink d2/l9 0 2024-05-06T12:14:42.229 INFO:tasks.workunit.client.1.smithi190.stdout:1/131: dwrite d5/d15/f2a [0,4194304] 0 2024-05-06T12:14:42.230 INFO:tasks.workunit.client.0.smithi092.stdout:0/943: dread d7/dc/f9b [4194304,4194304] 0 2024-05-06T12:14:42.232 INFO:tasks.workunit.client.1.smithi190.stdout:1/132: fsync d5/f16 0 2024-05-06T12:14:42.233 INFO:tasks.workunit.client.0.smithi092.stdout:2/627: mkdir df/d1b/db4 0 2024-05-06T12:14:42.233 INFO:tasks.workunit.client.1.smithi190.stdout:5/134: truncate d2/da/f1b 1225811 0 2024-05-06T12:14:42.233 INFO:tasks.workunit.client.1.smithi190.stdout:6/117: stat df/d17/d1b/d1e/c1a 0 2024-05-06T12:14:42.234 INFO:tasks.workunit.client.0.smithi092.stdout:7/846: mknod d9/de/da4/df8/c118 0 2024-05-06T12:14:42.235 INFO:tasks.workunit.client.0.smithi092.stdout:4/759: dwrite d9/d2b/f39 [0,4194304] 0 2024-05-06T12:14:42.237 INFO:tasks.workunit.client.1.smithi190.stdout:3/176: fsync d5/ff 0 2024-05-06T12:14:42.237 INFO:tasks.workunit.client.1.smithi190.stdout:7/120: truncate d0/ff 3055788 0 2024-05-06T12:14:42.238 INFO:tasks.workunit.client.1.smithi190.stdout:4/98: rename d2/d10/ce to d2/c26 0 2024-05-06T12:14:42.238 INFO:tasks.workunit.client.0.smithi092.stdout:0/944: mkdir d7/dc/d28/d54/d1e/d26/d96/dfb/d13b 0 2024-05-06T12:14:42.240 INFO:tasks.workunit.client.0.smithi092.stdout:2/628: write fd [5085551,86233] 0 2024-05-06T12:14:42.240 INFO:tasks.workunit.client.1.smithi190.stdout:1/133: unlink f4 0 2024-05-06T12:14:42.242 INFO:tasks.workunit.client.0.smithi092.stdout:9/748: getdents d16/d2c/d2f/d3d 0 2024-05-06T12:14:42.242 INFO:tasks.workunit.client.1.smithi190.stdout:3/177: write d5/d14/d11/d21/f38 [751403,53988] 0 2024-05-06T12:14:42.244 INFO:tasks.workunit.client.1.smithi190.stdout:5/135: unlink f0 0 2024-05-06T12:14:42.245 INFO:tasks.workunit.client.1.smithi190.stdout:7/121: dread d0/d1/f21 [0,4194304] 0 2024-05-06T12:14:42.246 INFO:tasks.workunit.client.1.smithi190.stdout:1/134: write d5/f2d [403129,22036] 0 2024-05-06T12:14:42.249 INFO:tasks.workunit.client.0.smithi092.stdout:2/629: symlink df/d17/d1d/d33/lb5 0 2024-05-06T12:14:42.249 INFO:tasks.workunit.client.0.smithi092.stdout:4/760: dwrite d9/d27/da8/db3/fd2 [4194304,4194304] 0 2024-05-06T12:14:42.252 INFO:tasks.workunit.client.0.smithi092.stdout:9/749: symlink d16/d1c/lfe 0 2024-05-06T12:14:42.252 INFO:tasks.workunit.client.1.smithi190.stdout:4/99: dread d2/d11/f14 [0,4194304] 0 2024-05-06T12:14:42.253 INFO:tasks.workunit.client.0.smithi092.stdout:7/847: creat d9/f119 x:0 0 0 2024-05-06T12:14:42.254 INFO:tasks.workunit.client.0.smithi092.stdout:0/945: symlink d7/dc/d28/d54/d12/d7f/db5/dcc/l13c 0 2024-05-06T12:14:42.258 INFO:tasks.workunit.client.1.smithi190.stdout:3/178: creat d5/d14/d11/d21/f41 x:0 0 0 2024-05-06T12:14:42.258 INFO:tasks.workunit.client.1.smithi190.stdout:1/135: rmdir d5/d23 39 2024-05-06T12:14:42.259 INFO:tasks.workunit.client.0.smithi092.stdout:4/761: rename d9/d2b/l4b to d9/d2b/d7c/dab/dee/lfa 0 2024-05-06T12:14:42.260 INFO:tasks.workunit.client.1.smithi190.stdout:3/179: fdatasync d5/d14/f24 0 2024-05-06T12:14:42.263 INFO:tasks.workunit.client.0.smithi092.stdout:7/848: unlink d9/de/d2d/c10b 0 2024-05-06T12:14:42.264 INFO:tasks.workunit.client.1.smithi190.stdout:0/300: rename d9/df/d21/f40 to d9/d51/f64 0 2024-05-06T12:14:42.264 INFO:tasks.workunit.client.0.smithi092.stdout:9/750: dwrite d16/d1e/d7b/d8d/da4/fb7 [0,4194304] 0 2024-05-06T12:14:42.265 INFO:tasks.workunit.client.1.smithi190.stdout:9/286: dwrite db/f13 [0,4194304] 0 2024-05-06T12:14:42.265 INFO:tasks.workunit.client.0.smithi092.stdout:4/762: symlink d9/df/d62/d7b/db6/lfb 0 2024-05-06T12:14:42.266 INFO:tasks.workunit.client.0.smithi092.stdout:9/751: chown d16/cd8 1778 1 2024-05-06T12:14:42.266 INFO:tasks.workunit.client.0.smithi092.stdout:2/630: getdents df/d17/d1d/d33/d4d/d87 0 2024-05-06T12:14:42.267 INFO:tasks.workunit.client.0.smithi092.stdout:4/763: chown d9/d2b/d65/dbc 166259 1 2024-05-06T12:14:42.267 INFO:tasks.workunit.client.0.smithi092.stdout:5/653: dwrite d4/f9 [4194304,4194304] 0 2024-05-06T12:14:42.267 INFO:tasks.workunit.client.1.smithi190.stdout:7/122: mknod d0/c34 0 2024-05-06T12:14:42.268 INFO:tasks.workunit.client.0.smithi092.stdout:9/752: write d16/d1e/f70 [440296,91429] 0 2024-05-06T12:14:42.270 INFO:tasks.workunit.client.0.smithi092.stdout:0/946: rename d7/dc/d28/d54/d1e/d26/d96/fed to d7/dc/d97/dc9/d12c/d132/f13d 0 2024-05-06T12:14:42.275 INFO:tasks.workunit.client.1.smithi190.stdout:9/287: dwrite db/d23/f5b [0,4194304] 0 2024-05-06T12:14:42.275 INFO:tasks.workunit.client.1.smithi190.stdout:3/180: write d5/d7/f23 [766707,93366] 0 2024-05-06T12:14:42.276 INFO:tasks.workunit.client.0.smithi092.stdout:7/849: mknod d9/de/d10/d17/d6a/c11a 0 2024-05-06T12:14:42.278 INFO:tasks.workunit.client.0.smithi092.stdout:5/654: fsync d4/d5/fc4 0 2024-05-06T12:14:42.278 INFO:tasks.workunit.client.0.smithi092.stdout:4/764: unlink d9/d2b/d5b/daf/lf9 0 2024-05-06T12:14:42.279 INFO:tasks.workunit.client.1.smithi190.stdout:2/130: rename d2/f14 to d2/d8/d19/f24 0 2024-05-06T12:14:42.280 INFO:tasks.workunit.client.0.smithi092.stdout:0/947: mkdir d7/dc/dd4/df8/d13e 0 2024-05-06T12:14:42.282 INFO:tasks.workunit.client.0.smithi092.stdout:9/753: mkdir d16/d2c/d2f/d3d/d33/d57/dff 0 2024-05-06T12:14:42.282 INFO:tasks.workunit.client.0.smithi092.stdout:7/850: mknod d9/de/d10/d17/d6a/c11b 0 2024-05-06T12:14:42.284 INFO:tasks.workunit.client.0.smithi092.stdout:9/754: chown d16/d2c/d3f/f96 106867831 1 2024-05-06T12:14:42.286 INFO:tasks.workunit.client.1.smithi190.stdout:7/123: mkdir d0/d1/d2f/d35 0 2024-05-06T12:14:42.286 INFO:tasks.workunit.client.1.smithi190.stdout:0/301: dread d9/dc/d28/f5d [0,4194304] 0 2024-05-06T12:14:42.287 INFO:tasks.workunit.client.0.smithi092.stdout:5/655: dwrite d4/f9b [0,4194304] 0 2024-05-06T12:14:42.287 INFO:tasks.workunit.client.0.smithi092.stdout:4/765: dread - d9/df/d48/f74 zero size 2024-05-06T12:14:42.288 INFO:tasks.workunit.client.1.smithi190.stdout:8/240: truncate d3/f2d 3863367 0 2024-05-06T12:14:42.288 INFO:tasks.workunit.client.1.smithi190.stdout:0/302: dread - d9/f37 zero size 2024-05-06T12:14:42.288 INFO:tasks.workunit.client.0.smithi092.stdout:4/766: chown d9/d2b/d5b/daf/ccb 7766735 1 2024-05-06T12:14:42.289 INFO:tasks.workunit.client.1.smithi190.stdout:3/181: read d5/d14/d11/f1b [3746747,14188] 0 2024-05-06T12:14:42.290 INFO:tasks.workunit.client.0.smithi092.stdout:2/631: getdents df 0 2024-05-06T12:14:42.290 INFO:tasks.workunit.client.0.smithi092.stdout:7/851: creat d9/de/d2d/f11c x:0 0 0 2024-05-06T12:14:42.291 INFO:tasks.workunit.client.0.smithi092.stdout:5/656: readlink d4/lb9 0 2024-05-06T12:14:42.292 INFO:tasks.workunit.client.0.smithi092.stdout:9/755: symlink d16/d2c/d2f/d3d/d33/d37/l100 0 2024-05-06T12:14:42.294 INFO:tasks.workunit.client.0.smithi092.stdout:0/948: link d7/dc/d28/d54/d1e/d26/d4e/dd7/fdb d7/dc/d28/d54/d12/d8f/f13f 0 2024-05-06T12:14:42.294 INFO:tasks.workunit.client.0.smithi092.stdout:9/756: truncate d16/d1c/d58/d92/fe9 411737 0 2024-05-06T12:14:42.296 INFO:tasks.workunit.client.0.smithi092.stdout:9/757: truncate d16/d1c/d58/ff3 406793 0 2024-05-06T12:14:42.297 INFO:tasks.workunit.client.0.smithi092.stdout:9/758: chown d16/d1c/d58/l8f 9350 1 2024-05-06T12:14:42.297 INFO:tasks.workunit.client.1.smithi190.stdout:2/131: rmdir d2 39 2024-05-06T12:14:42.298 INFO:tasks.workunit.client.0.smithi092.stdout:7/852: rename d9/da5/l24 to d9/de/d10/d17/d99/db6/df1/l11d 0 2024-05-06T12:14:42.298 INFO:tasks.workunit.client.1.smithi190.stdout:7/124: dwrite d0/d1/d2/f1d [0,4194304] 0 2024-05-06T12:14:42.300 INFO:tasks.workunit.client.1.smithi190.stdout:6/118: rename df/d12/f13 to df/d17/f26 0 2024-05-06T12:14:42.302 INFO:tasks.workunit.client.0.smithi092.stdout:5/657: link d4/f9 d4/d5/d30/feb 0 2024-05-06T12:14:42.302 INFO:tasks.workunit.client.1.smithi190.stdout:0/303: dwrite d9/f5f [0,4194304] 0 2024-05-06T12:14:42.303 INFO:tasks.workunit.client.1.smithi190.stdout:9/288: mkdir db/dd/d16/d4c/d4e/d5e/d6e 0 2024-05-06T12:14:42.303 INFO:tasks.workunit.client.1.smithi190.stdout:8/241: dread d3/d13/d4a/f4f [0,4194304] 0 2024-05-06T12:14:42.303 INFO:tasks.workunit.client.0.smithi092.stdout:9/759: creat d16/d2c/d2f/d3d/d33/d6b/f101 x:0 0 0 2024-05-06T12:14:42.304 INFO:tasks.workunit.client.0.smithi092.stdout:9/760: chown d16/d2c/d2f/d3d/ffd 8210 1 2024-05-06T12:14:42.304 INFO:tasks.workunit.client.0.smithi092.stdout:2/632: rmdir df/d17/d1d/d33/d4d/d87 0 2024-05-06T12:14:42.304 INFO:tasks.workunit.client.1.smithi190.stdout:8/242: fsync d3/f17 0 2024-05-06T12:14:42.305 INFO:tasks.workunit.client.0.smithi092.stdout:7/853: mknod d9/de/d10/c11e 0 2024-05-06T12:14:42.307 INFO:tasks.workunit.client.0.smithi092.stdout:9/761: creat d16/d1c/dd4/f102 x:0 0 0 2024-05-06T12:14:42.308 INFO:tasks.workunit.client.1.smithi190.stdout:5/136: dwrite d2/d27/f22 [0,4194304] 0 2024-05-06T12:14:42.309 INFO:tasks.workunit.client.0.smithi092.stdout:0/949: rmdir d7/dc/d6f/da2/df2 0 2024-05-06T12:14:42.313 INFO:tasks.workunit.client.0.smithi092.stdout:5/658: mknod d4/d5/d10/cec 0 2024-05-06T12:14:42.316 INFO:tasks.workunit.client.1.smithi190.stdout:2/132: mkdir d2/d8/d19/d25 0 2024-05-06T12:14:42.316 INFO:tasks.workunit.client.1.smithi190.stdout:6/119: mknod df/c27 0 2024-05-06T12:14:42.317 INFO:tasks.workunit.client.0.smithi092.stdout:9/762: dwrite d16/d2c/d2f/d3d/d33/d57/fdd [0,4194304] 0 2024-05-06T12:14:42.317 INFO:tasks.workunit.client.0.smithi092.stdout:9/763: dread - d16/d94/da0/da7/fe7 zero size 2024-05-06T12:14:42.318 INFO:tasks.workunit.client.0.smithi092.stdout:0/950: mkdir d7/dc/d28/d64/d140 0 2024-05-06T12:14:42.322 INFO:tasks.workunit.client.0.smithi092.stdout:7/854: mkdir d9/de/d10/d17/d21/d2e/d87/d11f 0 2024-05-06T12:14:42.322 INFO:tasks.workunit.client.1.smithi190.stdout:3/182: creat d5/d7/d15/d1f/f42 x:0 0 0 2024-05-06T12:14:42.322 INFO:tasks.workunit.client.1.smithi190.stdout:5/137: rmdir d2/da 39 2024-05-06T12:14:42.323 INFO:tasks.workunit.client.0.smithi092.stdout:5/659: creat d4/d59/fed x:0 0 0 2024-05-06T12:14:42.323 INFO:tasks.workunit.client.1.smithi190.stdout:7/125: dwrite d0/d1/f21 [0,4194304] 0 2024-05-06T12:14:42.324 INFO:tasks.workunit.client.0.smithi092.stdout:0/951: write d7/dc/f9b [358046,68244] 0 2024-05-06T12:14:42.324 INFO:tasks.workunit.client.1.smithi190.stdout:3/183: dread - d5/d7/f3a zero size 2024-05-06T12:14:42.325 INFO:tasks.workunit.client.0.smithi092.stdout:0/952: chown d7/dc/d28/d54/d12/d7f/db2/lf5 31145531 1 2024-05-06T12:14:42.325 INFO:tasks.workunit.client.1.smithi190.stdout:6/120: mkdir df/d17/d1b/d28 0 2024-05-06T12:14:42.326 INFO:tasks.workunit.client.0.smithi092.stdout:7/855: mkdir d9/de/d10/d17/d99/da6/d120 0 2024-05-06T12:14:42.326 INFO:tasks.workunit.client.1.smithi190.stdout:7/126: write d0/d4/f7 [559444,31224] 0 2024-05-06T12:14:42.327 INFO:tasks.workunit.client.0.smithi092.stdout:0/953: read d7/dc/d28/d54/d12/d7f/db2/fde [5025427,74491] 0 2024-05-06T12:14:42.329 INFO:tasks.workunit.client.1.smithi190.stdout:7/127: truncate d0/f19 1102607 0 2024-05-06T12:14:42.329 INFO:tasks.workunit.client.0.smithi092.stdout:5/660: mknod d4/d5/d10/d31/ddf/d6c/cee 0 2024-05-06T12:14:42.332 INFO:tasks.workunit.client.1.smithi190.stdout:0/304: rename d9/df/d21/l38 to d9/l65 0 2024-05-06T12:14:42.333 INFO:tasks.workunit.client.0.smithi092.stdout:7/856: mkdir d9/d62/d121 0 2024-05-06T12:14:42.333 INFO:tasks.workunit.client.1.smithi190.stdout:2/133: truncate d2/fe 4952887 0 2024-05-06T12:14:42.334 INFO:tasks.workunit.client.0.smithi092.stdout:0/954: mkdir d7/dc/d28/d54/d1e/d26/d96/d141 0 2024-05-06T12:14:42.334 INFO:tasks.workunit.client.1.smithi190.stdout:0/305: write d9/df/f33 [505893,27352] 0 2024-05-06T12:14:42.336 INFO:tasks.workunit.client.1.smithi190.stdout:6/121: creat df/d17/d1b/f29 x:0 0 0 2024-05-06T12:14:42.337 INFO:tasks.workunit.client.0.smithi092.stdout:5/661: mknod d4/d5/d10/da1/cef 0 2024-05-06T12:14:42.337 INFO:tasks.workunit.client.0.smithi092.stdout:1/718: sync 2024-05-06T12:14:42.337 INFO:tasks.workunit.client.0.smithi092.stdout:3/589: sync 2024-05-06T12:14:42.338 INFO:tasks.workunit.client.1.smithi190.stdout:0/306: stat d9/dc/d28/l3f 0 2024-05-06T12:14:42.340 INFO:tasks.workunit.client.0.smithi092.stdout:7/857: mknod d9/de/d10/d17/d99/db6/df1/c122 0 2024-05-06T12:14:42.341 INFO:tasks.workunit.client.1.smithi190.stdout:5/138: symlink d2/l2a 0 2024-05-06T12:14:42.342 INFO:tasks.workunit.client.1.smithi190.stdout:3/184: dwrite d5/d14/f24 [0,4194304] 0 2024-05-06T12:14:42.342 INFO:tasks.workunit.client.0.smithi092.stdout:9/764: getdents d16/d1c/d58/d92/dab 0 2024-05-06T12:14:42.345 INFO:tasks.workunit.client.0.smithi092.stdout:5/662: dwrite d4/d5/d30/f3c [0,4194304] 0 2024-05-06T12:14:42.346 INFO:tasks.workunit.client.0.smithi092.stdout:3/590: mkdir d8/d1b/d73/db7 0 2024-05-06T12:14:42.346 INFO:tasks.workunit.client.0.smithi092.stdout:5/663: dread d4/d59/fd5 [0,4194304] 0 2024-05-06T12:14:42.347 INFO:tasks.workunit.client.1.smithi190.stdout:3/185: fdatasync d5/d7/d15/f16 0 2024-05-06T12:14:42.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:41 smithi190 ceph-mon[33798]: pgmap v23: 65 pgs: 65 active+clean; 1.5 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 48 MiB/s rd, 170 MiB/s wr, 425 op/s 2024-05-06T12:14:42.348 INFO:tasks.workunit.client.0.smithi092.stdout:1/719: symlink d3/d10/db3/le4 0 2024-05-06T12:14:42.349 INFO:tasks.workunit.client.0.smithi092.stdout:4/767: truncate d9/d27/d4a/f84 3319445 0 2024-05-06T12:14:42.349 INFO:tasks.workunit.client.1.smithi190.stdout:9/289: rename db/d23/c2f to db/dd/d16/d58/c6f 0 2024-05-06T12:14:42.351 INFO:tasks.workunit.client.0.smithi092.stdout:9/765: rmdir d16/d2c/d88 39 2024-05-06T12:14:42.351 INFO:tasks.workunit.client.1.smithi190.stdout:0/307: creat d9/dc/d48/d50/f66 x:0 0 0 2024-05-06T12:14:42.352 INFO:tasks.workunit.client.1.smithi190.stdout:1/136: dwrite d5/d15/d1e/f26 [4194304,4194304] 0 2024-05-06T12:14:42.352 INFO:tasks.workunit.client.0.smithi092.stdout:3/591: dread d8/d24/f65 [0,4194304] 0 2024-05-06T12:14:42.353 INFO:tasks.workunit.client.0.smithi092.stdout:3/592: stat d8/d24/d33/da8 0 2024-05-06T12:14:42.354 INFO:tasks.workunit.client.1.smithi190.stdout:2/134: dwrite d2/d8/dc/f16 [0,4194304] 0 2024-05-06T12:14:42.354 INFO:tasks.workunit.client.0.smithi092.stdout:1/720: unlink d3/d42/f85 0 2024-05-06T12:14:42.354 INFO:tasks.workunit.client.1.smithi190.stdout:7/128: fdatasync d0/d4/f7 0 2024-05-06T12:14:42.355 INFO:tasks.workunit.client.0.smithi092.stdout:7/858: mkdir d9/d123 0 2024-05-06T12:14:42.357 INFO:tasks.workunit.client.1.smithi190.stdout:7/129: dread - d0/f20 zero size 2024-05-06T12:14:42.358 INFO:tasks.workunit.client.0.smithi092.stdout:0/955: creat d7/dc/d28/d54/d1e/d46/d63/f142 x:0 0 0 2024-05-06T12:14:42.358 INFO:tasks.workunit.client.1.smithi190.stdout:5/139: dread d2/d27/f20 [0,4194304] 0 2024-05-06T12:14:42.358 INFO:tasks.workunit.client.1.smithi190.stdout:2/135: write d2/d8/f15 [2020399,80546] 0 2024-05-06T12:14:42.362 INFO:tasks.workunit.client.1.smithi190.stdout:7/130: write d0/d4/f7 [1430381,3883] 0 2024-05-06T12:14:42.363 INFO:tasks.workunit.client.0.smithi092.stdout:5/664: symlink d4/d5/d10/lf0 0 2024-05-06T12:14:42.363 INFO:tasks.workunit.client.0.smithi092.stdout:6/795: sync 2024-05-06T12:14:42.365 INFO:tasks.workunit.client.0.smithi092.stdout:8/804: sync 2024-05-06T12:14:42.368 INFO:tasks.workunit.client.1.smithi190.stdout:8/243: fsync d3/f1c 0 2024-05-06T12:14:42.370 INFO:tasks.workunit.client.1.smithi190.stdout:0/308: creat d9/df/d21/d31/f67 x:0 0 0 2024-05-06T12:14:42.371 INFO:tasks.workunit.client.0.smithi092.stdout:0/956: unlink d7/dc/d28/d54/d1e/d39/da8/dc5/c130 0 2024-05-06T12:14:42.371 INFO:tasks.workunit.client.0.smithi092.stdout:0/957: readlink d7/dc/d28/l98 0 2024-05-06T12:14:42.371 INFO:tasks.workunit.client.0.smithi092.stdout:2/633: truncate df/d17/f6b 1266389 0 2024-05-06T12:14:42.373 INFO:tasks.workunit.client.0.smithi092.stdout:9/766: mknod d16/d2c/d2f/c103 0 2024-05-06T12:14:42.373 INFO:tasks.workunit.client.0.smithi092.stdout:9/767: stat d16/d2c/d2f 0 2024-05-06T12:14:42.374 INFO:tasks.workunit.client.1.smithi190.stdout:5/140: unlink d2/d27/f22 0 2024-05-06T12:14:42.375 INFO:tasks.workunit.client.0.smithi092.stdout:6/796: fsync d4/fb 0 2024-05-06T12:14:42.376 INFO:tasks.workunit.client.0.smithi092.stdout:6/797: chown d4/d24/d26/f32 1 1 2024-05-06T12:14:42.377 INFO:tasks.workunit.client.0.smithi092.stdout:5/665: creat d4/d34/dae/ff1 x:0 0 0 2024-05-06T12:14:42.378 INFO:tasks.workunit.client.1.smithi190.stdout:7/131: mkdir d0/da/d36 0 2024-05-06T12:14:42.378 INFO:tasks.workunit.client.0.smithi092.stdout:7/859: mknod d9/de/d1f/d37/c124 0 2024-05-06T12:14:42.380 INFO:tasks.workunit.client.0.smithi092.stdout:0/958: chown d7/l12b 14980928 1 2024-05-06T12:14:42.382 INFO:tasks.workunit.client.0.smithi092.stdout:9/768: dwrite d16/d2c/d2f/d3d/d33/d6b/d9e/fba [4194304,4194304] 0 2024-05-06T12:14:42.383 INFO:tasks.workunit.client.1.smithi190.stdout:0/309: rename d9/dc/f13 to d9/df/d56/f68 0 2024-05-06T12:14:42.383 INFO:tasks.workunit.client.0.smithi092.stdout:1/721: rename d3/d42/d84/d4f/d58/f69 to d3/d10/d13/fe5 0 2024-05-06T12:14:42.384 INFO:tasks.workunit.client.1.smithi190.stdout:2/136: symlink d2/l26 0 2024-05-06T12:14:42.386 INFO:tasks.workunit.client.1.smithi190.stdout:5/141: symlink d2/d27/l2b 0 2024-05-06T12:14:42.386 INFO:tasks.workunit.client.1.smithi190.stdout:5/142: fdatasync d2/f28 0 2024-05-06T12:14:42.387 INFO:tasks.workunit.client.1.smithi190.stdout:5/143: chown d2 0 1 2024-05-06T12:14:42.387 INFO:tasks.workunit.client.0.smithi092.stdout:0/959: readlink d7/dc/d9e/l32 0 2024-05-06T12:14:42.388 INFO:tasks.workunit.client.0.smithi092.stdout:9/769: dread d16/d1e/d7b/d8d/da4/fb7 [0,4194304] 0 2024-05-06T12:14:42.389 INFO:tasks.workunit.client.1.smithi190.stdout:3/186: creat d5/d7/f43 x:0 0 0 2024-05-06T12:14:42.390 INFO:tasks.workunit.client.1.smithi190.stdout:8/244: truncate d3/d8/d2b/f47 997322 0 2024-05-06T12:14:42.392 INFO:tasks.workunit.client.1.smithi190.stdout:8/245: write d3/d8/f30 [1863415,96616] 0 2024-05-06T12:14:42.392 INFO:tasks.workunit.client.0.smithi092.stdout:1/722: mknod d3/d10/d13/db0/ce6 0 2024-05-06T12:14:42.395 INFO:tasks.workunit.client.0.smithi092.stdout:8/805: getdents d9/d21/d37/d74/df9 0 2024-05-06T12:14:42.397 INFO:tasks.workunit.client.0.smithi092.stdout:0/960: rmdir d7/dc/d28/d54/d12/d7f/db2/dbc 39 2024-05-06T12:14:42.397 INFO:tasks.workunit.client.1.smithi190.stdout:5/144: dwrite d2/d27/f1c [0,4194304] 0 2024-05-06T12:14:42.397 INFO:tasks.workunit.client.1.smithi190.stdout:9/290: getdents db/dd/d16/d4c/d4e/d5e 0 2024-05-06T12:14:42.398 INFO:tasks.workunit.client.0.smithi092.stdout:0/961: chown d7/dc/d28/d54/d1e/d39/da8/dc5 31496 1 2024-05-06T12:14:42.398 INFO:tasks.workunit.client.1.smithi190.stdout:3/187: mkdir d5/d14/d11/d21/d44 0 2024-05-06T12:14:42.400 INFO:tasks.workunit.client.1.smithi190.stdout:2/137: symlink d2/d8/dc/d20/l27 0 2024-05-06T12:14:42.400 INFO:tasks.workunit.client.1.smithi190.stdout:3/188: write d5/d7/d15/f28 [4673138,85186] 0 2024-05-06T12:14:42.402 INFO:tasks.workunit.client.0.smithi092.stdout:5/666: rename d4/d5/d10/d31/ddf/d54/d73/ca0 to d4/d5/d10/d31/d53/dcb/cf2 0 2024-05-06T12:14:42.405 INFO:tasks.workunit.client.1.smithi190.stdout:6/122: write df/d17/f26 [348312,47098] 0 2024-05-06T12:14:42.406 INFO:tasks.workunit.client.0.smithi092.stdout:2/634: getdents df/d17/d53 0 2024-05-06T12:14:42.407 INFO:tasks.workunit.client.0.smithi092.stdout:1/723: fdatasync d3/d10/d13/d19/d21/fb4 0 2024-05-06T12:14:42.409 INFO:tasks.workunit.client.0.smithi092.stdout:1/724: dread - d3/d10/d13/d19/d21/d72/fbd zero size 2024-05-06T12:14:42.409 INFO:tasks.workunit.client.0.smithi092.stdout:8/806: unlink d9/f42 0 2024-05-06T12:14:42.409 INFO:tasks.workunit.client.1.smithi190.stdout:2/138: creat d2/d8/f28 x:0 0 0 2024-05-06T12:14:42.410 INFO:tasks.workunit.client.0.smithi092.stdout:1/725: stat d3/d10/d13/d46/f61 0 2024-05-06T12:14:42.410 INFO:tasks.workunit.client.1.smithi190.stdout:1/137: write d5/f22 [1960610,28346] 0 2024-05-06T12:14:42.411 INFO:tasks.workunit.client.1.smithi190.stdout:7/132: getdents d0 0 2024-05-06T12:14:42.412 INFO:tasks.workunit.client.1.smithi190.stdout:9/291: dwrite f3 [0,4194304] 0 2024-05-06T12:14:42.412 INFO:tasks.workunit.client.0.smithi092.stdout:9/770: truncate d16/d2c/d2f/d3d/d33/dde/ffb 1148619 0 2024-05-06T12:14:42.412 INFO:tasks.workunit.client.1.smithi190.stdout:5/145: mkdir d2/da/d2c 0 2024-05-06T12:14:42.413 INFO:tasks.workunit.client.1.smithi190.stdout:9/292: fdatasync db/dd/d16/f64 0 2024-05-06T12:14:42.414 INFO:tasks.workunit.client.0.smithi092.stdout:7/860: dread d9/de/d10/f12 [0,4194304] 0 2024-05-06T12:14:42.414 INFO:tasks.workunit.client.0.smithi092.stdout:0/962: rename d7/dc/d28/d54/d1e/d46/df0/f129 to d7/dc/d28/d54/d1e/d46/f143 0 2024-05-06T12:14:42.415 INFO:tasks.workunit.client.1.smithi190.stdout:0/310: getdents d9/df/d21/d31 0 2024-05-06T12:14:42.416 INFO:tasks.workunit.client.1.smithi190.stdout:3/189: creat d5/d7/d15/d1f/d33/d40/f45 x:0 0 0 2024-05-06T12:14:42.417 INFO:tasks.workunit.client.1.smithi190.stdout:7/133: mkdir d0/d1/d18/d37 0 2024-05-06T12:14:42.419 INFO:tasks.workunit.client.0.smithi092.stdout:2/635: creat df/d17/d1d/fb6 x:0 0 0 2024-05-06T12:14:42.421 INFO:tasks.workunit.client.0.smithi092.stdout:4/768: dwrite d9/d2b/d5b/fa9 [0,4194304] 0 2024-05-06T12:14:42.421 INFO:tasks.workunit.client.0.smithi092.stdout:8/807: write d9/d21/d37/d46/d5c/d62/da5/fd6 [74144,39148] 0 2024-05-06T12:14:42.421 INFO:tasks.workunit.client.1.smithi190.stdout:6/123: creat df/d17/d1b/d28/f2a x:0 0 0 2024-05-06T12:14:42.421 INFO:tasks.workunit.client.1.smithi190.stdout:9/293: write db/dd/d16/d4c/f69 [1703467,110207] 0 2024-05-06T12:14:42.422 INFO:tasks.workunit.client.0.smithi092.stdout:1/726: creat d3/d42/d84/d4f/d7f/fe7 x:0 0 0 2024-05-06T12:14:42.422 INFO:tasks.workunit.client.1.smithi190.stdout:2/139: dread d2/d8/d19/f24 [0,4194304] 0 2024-05-06T12:14:42.425 INFO:tasks.workunit.client.0.smithi092.stdout:2/636: write df/d17/d53/da9/f8b [4238531,89900] 0 2024-05-06T12:14:42.427 INFO:tasks.workunit.client.1.smithi190.stdout:6/124: read df/d17/f26 [268611,118664] 0 2024-05-06T12:14:42.429 INFO:tasks.workunit.client.0.smithi092.stdout:0/963: symlink d7/dc/d28/d54/d1e/d26/d96/d90/l144 0 2024-05-06T12:14:42.430 INFO:tasks.workunit.client.1.smithi190.stdout:1/138: dwrite d5/d15/f1f [0,4194304] 0 2024-05-06T12:14:42.430 INFO:tasks.workunit.client.0.smithi092.stdout:3/593: dwrite d8/db/d4c/d40/d67/f80 [0,4194304] 0 2024-05-06T12:14:42.432 INFO:tasks.workunit.client.0.smithi092.stdout:9/771: mkdir d16/d1e/d7b/d104 0 2024-05-06T12:14:42.432 INFO:tasks.workunit.client.0.smithi092.stdout:3/594: write d8/d1b/f2b [2837582,99610] 0 2024-05-06T12:14:42.433 INFO:tasks.workunit.client.1.smithi190.stdout:6/125: creat df/d12/f2b x:0 0 0 2024-05-06T12:14:42.436 INFO:tasks.workunit.client.1.smithi190.stdout:6/126: write df/d12/f2b [765796,44586] 0 2024-05-06T12:14:42.436 INFO:tasks.workunit.client.0.smithi092.stdout:1/727: symlink d3/d10/d13/d19/d21/d88/d8b/le8 0 2024-05-06T12:14:42.438 INFO:tasks.workunit.client.0.smithi092.stdout:9/772: truncate d16/d1e/f70 708834 0 2024-05-06T12:14:42.438 INFO:tasks.workunit.client.0.smithi092.stdout:8/808: rename d9/d9d/da6 to d9/d21/d37/dbd/dc5/d11c 0 2024-05-06T12:14:42.439 INFO:tasks.workunit.client.1.smithi190.stdout:1/139: read d5/d23/d24/f25 [98706,99253] 0 2024-05-06T12:14:42.440 INFO:tasks.workunit.client.0.smithi092.stdout:2/637: creat df/d15/d88/d59/fb7 x:0 0 0 2024-05-06T12:14:42.440 INFO:tasks.workunit.client.1.smithi190.stdout:7/134: rename d0/d4/fe to d0/d1/d32/f38 0 2024-05-06T12:14:42.441 INFO:tasks.workunit.client.1.smithi190.stdout:2/140: getdents d2/d8/d19/d25 0 2024-05-06T12:14:42.442 INFO:tasks.workunit.client.1.smithi190.stdout:3/190: rmdir d5/d7 39 2024-05-06T12:14:42.443 INFO:tasks.workunit.client.1.smithi190.stdout:9/294: link db/dd/f17 db/d21/f70 0 2024-05-06T12:14:42.444 INFO:tasks.workunit.client.1.smithi190.stdout:9/295: stat db/f3d 0 2024-05-06T12:14:42.446 INFO:tasks.workunit.client.1.smithi190.stdout:5/146: write d2/f5 [996404,20881] 0 2024-05-06T12:14:42.447 INFO:tasks.workunit.client.0.smithi092.stdout:1/728: fdatasync d3/d42/d84/d4f/f97 0 2024-05-06T12:14:42.448 INFO:tasks.workunit.client.1.smithi190.stdout:7/135: creat d0/d4/f39 x:0 0 0 2024-05-06T12:14:42.448 INFO:tasks.workunit.client.1.smithi190.stdout:0/311: truncate d9/f1a 2626344 0 2024-05-06T12:14:42.450 INFO:tasks.workunit.client.1.smithi190.stdout:3/191: chown d5/fc 508594008 1 2024-05-06T12:14:42.451 INFO:tasks.workunit.client.0.smithi092.stdout:8/809: mkdir d9/d21/d37/dbd/dc3/d11d 0 2024-05-06T12:14:42.451 INFO:tasks.workunit.client.0.smithi092.stdout:6/798: dwrite d4/d24/d26/dae/dc0/fad [0,4194304] 0 2024-05-06T12:14:42.451 INFO:tasks.workunit.client.1.smithi190.stdout:2/141: mknod d2/d8/dc/c29 0 2024-05-06T12:14:42.452 INFO:tasks.workunit.client.1.smithi190.stdout:5/147: dread d2/f3 [0,4194304] 0 2024-05-06T12:14:42.453 INFO:tasks.workunit.client.1.smithi190.stdout:2/142: dread - d2/d8/f28 zero size 2024-05-06T12:14:42.453 INFO:tasks.workunit.client.0.smithi092.stdout:1/729: chown d3/d42/d84/d4f/d7f/fe7 7 1 2024-05-06T12:14:42.456 INFO:tasks.workunit.client.1.smithi190.stdout:5/148: truncate d2/f28 459964 0 2024-05-06T12:14:42.456 INFO:tasks.workunit.client.0.smithi092.stdout:9/773: dwrite d16/d2c/d2f/d3d/d33/f71 [0,4194304] 0 2024-05-06T12:14:42.456 INFO:tasks.workunit.client.0.smithi092.stdout:5/667: dread d4/d5/d30/f46 [0,4194304] 0 2024-05-06T12:14:42.458 INFO:tasks.workunit.client.1.smithi190.stdout:5/149: stat d2/l2a 0 2024-05-06T12:14:42.459 INFO:tasks.workunit.client.1.smithi190.stdout:6/127: rename df/d17/d1b to df/d2c 0 2024-05-06T12:14:42.459 INFO:tasks.workunit.client.0.smithi092.stdout:0/964: creat d7/dc/d28/d54/d1e/d26/d96/d141/f145 x:0 0 0 2024-05-06T12:14:42.460 INFO:tasks.workunit.client.1.smithi190.stdout:8/246: dwrite d3/d8/d2b/f47 [0,4194304] 0 2024-05-06T12:14:42.460 INFO:tasks.workunit.client.1.smithi190.stdout:3/192: write d5/d7/d15/d1f/fe [2958209,64849] 0 2024-05-06T12:14:42.461 INFO:tasks.workunit.client.0.smithi092.stdout:0/965: stat d7/dc/d28/d54/d1e/d26/d4e/fa0 0 2024-05-06T12:14:42.462 INFO:tasks.workunit.client.1.smithi190.stdout:5/150: write d2/d27/f1c [3657237,100132] 0 2024-05-06T12:14:42.463 INFO:tasks.workunit.client.1.smithi190.stdout:7/136: symlink d0/d1/d2f/d35/l3a 0 2024-05-06T12:14:42.463 INFO:tasks.workunit.client.0.smithi092.stdout:0/966: write d7/dc/d28/d64/fee [3455447,90121] 0 2024-05-06T12:14:42.464 INFO:tasks.workunit.client.1.smithi190.stdout:0/312: creat d9/dc/d63/f69 x:0 0 0 2024-05-06T12:14:42.465 INFO:tasks.workunit.client.0.smithi092.stdout:0/967: fsync d7/dc/d28/d54/d1e/d26/d96/f87 0 2024-05-06T12:14:42.465 INFO:tasks.workunit.client.1.smithi190.stdout:4/100: sync 2024-05-06T12:14:42.465 INFO:tasks.workunit.client.1.smithi190.stdout:2/143: dwrite d2/fd [0,4194304] 0 2024-05-06T12:14:42.466 INFO:tasks.workunit.client.0.smithi092.stdout:1/730: rmdir d3/d42/d84/dcf 39 2024-05-06T12:14:42.466 INFO:tasks.workunit.client.1.smithi190.stdout:5/151: fdatasync d2/f3 0 2024-05-06T12:14:42.468 INFO:tasks.workunit.client.0.smithi092.stdout:5/668: symlink d4/d5/d10/d31/ddf/d54/db0/lf3 0 2024-05-06T12:14:42.469 INFO:tasks.workunit.client.1.smithi190.stdout:0/313: fsync d9/df/f33 0 2024-05-06T12:14:42.470 INFO:tasks.workunit.client.0.smithi092.stdout:2/638: truncate df/d17/d51/f67 1140455 0 2024-05-06T12:14:42.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:41 smithi092 ceph-mon[26954]: pgmap v23: 65 pgs: 65 active+clean; 1.5 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 48 MiB/s rd, 170 MiB/s wr, 425 op/s 2024-05-06T12:14:42.472 INFO:tasks.workunit.client.1.smithi190.stdout:1/140: rename d5/d15/d1e/f26 to d5/d23/d24/f2f 0 2024-05-06T12:14:42.474 INFO:tasks.workunit.client.1.smithi190.stdout:0/314: chown d9/df/d56/f59 4605 1 2024-05-06T12:14:42.474 INFO:tasks.workunit.client.0.smithi092.stdout:4/769: dwrite d9/df/d62/f63 [0,4194304] 0 2024-05-06T12:14:42.476 INFO:tasks.workunit.client.1.smithi190.stdout:7/137: mkdir d0/d1/d2f/d3b 0 2024-05-06T12:14:42.478 INFO:tasks.workunit.client.1.smithi190.stdout:5/152: readlink d2/da/de/l29 0 2024-05-06T12:14:42.479 INFO:tasks.workunit.client.1.smithi190.stdout:2/144: creat d2/d8/d19/f2a x:0 0 0 2024-05-06T12:14:42.481 INFO:tasks.workunit.client.1.smithi190.stdout:7/138: chown d0/l16 65847790 1 2024-05-06T12:14:42.481 INFO:tasks.workunit.client.0.smithi092.stdout:8/810: unlink d9/d21/d37/d46/d5c/d62/fac 0 2024-05-06T12:14:42.483 INFO:tasks.workunit.client.1.smithi190.stdout:4/101: dwrite d2/f1b [0,4194304] 0 2024-05-06T12:14:42.484 INFO:tasks.workunit.client.0.smithi092.stdout:5/669: symlink d4/d34/d28/d4e/d67/lf4 0 2024-05-06T12:14:42.485 INFO:tasks.workunit.client.0.smithi092.stdout:5/670: chown d4/d34/dae 30 1 2024-05-06T12:14:42.485 INFO:tasks.workunit.client.1.smithi190.stdout:0/315: creat d9/df/d21/f6a x:0 0 0 2024-05-06T12:14:42.486 INFO:tasks.workunit.client.1.smithi190.stdout:8/247: dwrite d3/f26 [0,4194304] 0 2024-05-06T12:14:42.486 INFO:tasks.workunit.client.0.smithi092.stdout:9/774: truncate d16/d2c/d2f/d3d/f67 2612443 0 2024-05-06T12:14:42.486 INFO:tasks.workunit.client.0.smithi092.stdout:5/671: chown d4/d5/d10/d31/f4a 955 1 2024-05-06T12:14:42.487 INFO:tasks.workunit.client.1.smithi190.stdout:7/139: readlink d0/d1/d2f/d35/l3a 0 2024-05-06T12:14:42.487 INFO:tasks.workunit.client.0.smithi092.stdout:5/672: fsync d4/d5/f1d 0 2024-05-06T12:14:42.488 INFO:tasks.workunit.client.1.smithi190.stdout:7/140: chown d0/l16 54669 1 2024-05-06T12:14:42.488 INFO:tasks.workunit.client.0.smithi092.stdout:3/595: link d8/d24/d33/d4b/l35 d8/lb8 0 2024-05-06T12:14:42.488 INFO:tasks.workunit.client.0.smithi092.stdout:5/673: dread - d4/d59/fed zero size 2024-05-06T12:14:42.489 INFO:tasks.workunit.client.1.smithi190.stdout:3/193: mknod d5/d7/d15/d1f/c46 0 2024-05-06T12:14:42.490 INFO:tasks.workunit.client.1.smithi190.stdout:0/316: chown d9/dc/d28/l2b 1378 1 2024-05-06T12:14:42.490 INFO:tasks.workunit.client.0.smithi092.stdout:4/770: creat d9/d2b/d65/d83/ffc x:0 0 0 2024-05-06T12:14:42.491 INFO:tasks.workunit.client.0.smithi092.stdout:6/799: creat d4/d4c/d52/f107 x:0 0 0 2024-05-06T12:14:42.493 INFO:tasks.workunit.client.1.smithi190.stdout:5/153: dread d2/d27/f20 [0,4194304] 0 2024-05-06T12:14:42.496 INFO:tasks.workunit.client.1.smithi190.stdout:9/296: rename db/dd/d16/c65 to db/d23/c71 0 2024-05-06T12:14:42.498 INFO:tasks.workunit.client.1.smithi190.stdout:8/248: read d3/d8/fd [3424294,48937] 0 2024-05-06T12:14:42.499 INFO:tasks.workunit.client.1.smithi190.stdout:4/102: rmdir d2 39 2024-05-06T12:14:42.500 INFO:tasks.workunit.client.1.smithi190.stdout:9/297: dread db/dd/d16/d4c/f69 [0,4194304] 0 2024-05-06T12:14:42.503 INFO:tasks.workunit.client.0.smithi092.stdout:3/596: fdatasync d8/db/d4c/d40/d67/f71 0 2024-05-06T12:14:42.503 INFO:tasks.workunit.client.1.smithi190.stdout:3/194: creat d5/d14/f47 x:0 0 0 2024-05-06T12:14:42.503 INFO:tasks.workunit.client.0.smithi092.stdout:5/674: rmdir d4/d5/d10/d31/d53 39 2024-05-06T12:14:42.504 INFO:tasks.workunit.client.1.smithi190.stdout:0/317: symlink d9/df/l6b 0 2024-05-06T12:14:42.506 INFO:tasks.workunit.client.0.smithi092.stdout:4/771: creat d9/dd/d90/ffd x:0 0 0 2024-05-06T12:14:42.507 INFO:tasks.workunit.client.1.smithi190.stdout:7/141: mkdir d0/d1/d2f/d33/d3c 0 2024-05-06T12:14:42.507 INFO:tasks.workunit.client.1.smithi190.stdout:7/142: readlink d0/d15/l28 0 2024-05-06T12:14:42.508 INFO:tasks.workunit.client.0.smithi092.stdout:6/800: rmdir d4/d20/d23/d30 39 2024-05-06T12:14:42.508 INFO:tasks.workunit.client.1.smithi190.stdout:5/154: mknod d2/d27/c2d 0 2024-05-06T12:14:42.508 INFO:tasks.workunit.client.1.smithi190.stdout:5/155: readlink d2/l6 0 2024-05-06T12:14:42.512 INFO:tasks.workunit.client.0.smithi092.stdout:3/597: truncate d8/d24/d33/d4b/f56 4010436 0 2024-05-06T12:14:42.513 INFO:tasks.workunit.client.0.smithi092.stdout:4/772: fdatasync d9/d2b/fc7 0 2024-05-06T12:14:42.515 INFO:tasks.workunit.client.0.smithi092.stdout:1/731: getdents d3/d10/d13 0 2024-05-06T12:14:42.516 INFO:tasks.workunit.client.0.smithi092.stdout:6/801: write d4/d20/d23/d30/f5e [3413315,82268] 0 2024-05-06T12:14:42.516 INFO:tasks.workunit.client.1.smithi190.stdout:5/156: dread d2/f5 [0,4194304] 0 2024-05-06T12:14:42.517 INFO:tasks.workunit.client.0.smithi092.stdout:1/732: write d3/d42/f43 [4580623,125463] 0 2024-05-06T12:14:42.518 INFO:tasks.workunit.client.1.smithi190.stdout:0/318: truncate d9/df/f2e 2655908 0 2024-05-06T12:14:42.519 INFO:tasks.workunit.client.0.smithi092.stdout:0/968: dread d7/dc/d28/d54/d12/d7f/db2/fde [0,4194304] 0 2024-05-06T12:14:42.519 INFO:tasks.workunit.client.0.smithi092.stdout:3/598: mknod d8/d24/d33/d91/cb9 0 2024-05-06T12:14:42.520 INFO:tasks.workunit.client.1.smithi190.stdout:7/143: mkdir d0/da/d3d 0 2024-05-06T12:14:42.521 INFO:tasks.workunit.client.0.smithi092.stdout:0/969: write d7/dc/d28/d54/d12/d104/f11e [572864,25073] 0 2024-05-06T12:14:42.522 INFO:tasks.workunit.client.0.smithi092.stdout:9/775: getdents d16/d2c/d2f 0 2024-05-06T12:14:42.522 INFO:tasks.workunit.client.0.smithi092.stdout:3/599: read d8/d24/d33/f64 [3255073,33122] 0 2024-05-06T12:14:42.525 INFO:tasks.workunit.client.0.smithi092.stdout:5/675: link d4/d5/d10/d31/ddf/da3/fd2 d4/d34/d28/ff5 0 2024-05-06T12:14:42.527 INFO:tasks.workunit.client.0.smithi092.stdout:6/802: mkdir d4/d24/d26/dc4/d108 0 2024-05-06T12:14:42.527 INFO:tasks.workunit.client.1.smithi190.stdout:9/298: unlink db/d21/d22/l39 0 2024-05-06T12:14:42.528 INFO:tasks.workunit.client.1.smithi190.stdout:3/195: dwrite d5/f26 [0,4194304] 0 2024-05-06T12:14:42.529 INFO:tasks.workunit.client.1.smithi190.stdout:6/128: dwrite fe [0,4194304] 0 2024-05-06T12:14:42.531 INFO:tasks.workunit.client.0.smithi092.stdout:0/970: dwrite d7/dc/d28/d54/d1e/d46/f121 [0,4194304] 0 2024-05-06T12:14:42.531 INFO:tasks.workunit.client.0.smithi092.stdout:8/811: dread d9/d21/d54/d79/d8a/fb2 [0,4194304] 0 2024-05-06T12:14:42.531 INFO:tasks.workunit.client.1.smithi190.stdout:2/145: getdents d2/d8 0 2024-05-06T12:14:42.532 INFO:tasks.workunit.client.0.smithi092.stdout:8/812: chown d9/d21/d54/d79 433282282 1 2024-05-06T12:14:42.532 INFO:tasks.workunit.client.1.smithi190.stdout:2/146: chown d2/d8/dc 2 1 2024-05-06T12:14:42.533 INFO:tasks.workunit.client.0.smithi092.stdout:9/776: fsync d16/d2c/f35 0 2024-05-06T12:14:42.533 INFO:tasks.workunit.client.1.smithi190.stdout:9/299: chown db/f3d 811083899 1 2024-05-06T12:14:42.534 INFO:tasks.workunit.client.1.smithi190.stdout:5/157: fsync d2/d27/f20 0 2024-05-06T12:14:42.536 INFO:tasks.workunit.client.1.smithi190.stdout:3/196: dread - d5/d14/d11/f34 zero size 2024-05-06T12:14:42.536 INFO:tasks.workunit.client.0.smithi092.stdout:0/971: write d7/dc/d28/d54/d1e/d39/f57 [4925035,45357] 0 2024-05-06T12:14:42.538 INFO:tasks.workunit.client.1.smithi190.stdout:7/144: dwrite d0/d1/d2/f1d [0,4194304] 0 2024-05-06T12:14:42.538 INFO:tasks.workunit.client.0.smithi092.stdout:9/777: dread - d16/d1e/d7b/d8d/da4/fd2 zero size 2024-05-06T12:14:42.540 INFO:tasks.workunit.client.1.smithi190.stdout:7/145: chown d0/d1/d2f/d33 81598861 1 2024-05-06T12:14:42.542 INFO:tasks.workunit.client.0.smithi092.stdout:4/773: rmdir d9/d2b/d5b/daf/dea 0 2024-05-06T12:14:42.543 INFO:tasks.workunit.client.1.smithi190.stdout:4/103: rename d2/c26 to d2/d10/c27 0 2024-05-06T12:14:42.544 INFO:tasks.workunit.client.0.smithi092.stdout:6/803: creat d4/d24/d26/f109 x:0 0 0 2024-05-06T12:14:42.545 INFO:tasks.workunit.client.0.smithi092.stdout:5/676: truncate d4/d5/d10/d31/ddf/d54/d73/f92 460877 0 2024-05-06T12:14:42.545 INFO:tasks.workunit.client.1.smithi190.stdout:4/104: write d2/f25 [725076,68107] 0 2024-05-06T12:14:42.545 INFO:tasks.workunit.client.1.smithi190.stdout:2/147: fsync d2/d8/d19/f24 0 2024-05-06T12:14:42.546 INFO:tasks.workunit.client.1.smithi190.stdout:2/148: chown d2/l26 31 1 2024-05-06T12:14:42.547 INFO:tasks.workunit.client.1.smithi190.stdout:6/129: creat df/d2c/d1e/f2d x:0 0 0 2024-05-06T12:14:42.549 INFO:tasks.workunit.client.0.smithi092.stdout:2/639: truncate df/d17/d1d/d33/f44 1567527 0 2024-05-06T12:14:42.549 INFO:tasks.workunit.client.0.smithi092.stdout:6/804: dread d4/d24/d26/dc4/fca [0,4194304] 0 2024-05-06T12:14:42.549 INFO:tasks.workunit.client.1.smithi190.stdout:9/300: mknod db/d21/c72 0 2024-05-06T12:14:42.551 INFO:tasks.workunit.client.1.smithi190.stdout:3/197: unlink d5/d14/l12 0 2024-05-06T12:14:42.552 INFO:tasks.workunit.client.0.smithi092.stdout:9/778: rmdir d16/d2c/d2f/d3d/d33/d37 39 2024-05-06T12:14:42.553 INFO:tasks.workunit.client.0.smithi092.stdout:3/600: dread d8/d24/f6f [0,4194304] 0 2024-05-06T12:14:42.554 INFO:tasks.workunit.client.0.smithi092.stdout:9/779: chown d16/d2c/d2f/d3d/cc1 29 1 2024-05-06T12:14:42.554 INFO:tasks.workunit.client.0.smithi092.stdout:1/733: dread d3/d10/d13/d19/d21/fb4 [0,4194304] 0 2024-05-06T12:14:42.555 INFO:tasks.workunit.client.1.smithi190.stdout:7/146: read d0/ff [2214270,317] 0 2024-05-06T12:14:42.555 INFO:tasks.workunit.client.1.smithi190.stdout:2/149: dwrite d2/d8/d19/f2a [0,4194304] 0 2024-05-06T12:14:42.556 INFO:tasks.workunit.client.1.smithi190.stdout:5/158: rename d2/da/de to d2/da/d2c/d2e 0 2024-05-06T12:14:42.558 INFO:tasks.workunit.client.1.smithi190.stdout:4/105: symlink d2/d10/l28 0 2024-05-06T12:14:42.558 INFO:tasks.workunit.client.0.smithi092.stdout:8/813: rename d9/c60 to d9/d21/d37/dbd/dc3/d107/c11e 0 2024-05-06T12:14:42.559 INFO:tasks.workunit.client.1.smithi190.stdout:4/106: fsync d2/d10/f1c 0 2024-05-06T12:14:42.559 INFO:tasks.workunit.client.0.smithi092.stdout:8/814: chown d9/d21/f103 0 1 2024-05-06T12:14:42.560 INFO:tasks.workunit.client.1.smithi190.stdout:0/319: creat d9/dc/f6c x:0 0 0 2024-05-06T12:14:42.562 INFO:tasks.workunit.client.0.smithi092.stdout:9/780: mknod d16/d94/da0/da7/c105 0 2024-05-06T12:14:42.564 INFO:tasks.workunit.client.1.smithi190.stdout:3/198: symlink d5/d14/d11/d21/l48 0 2024-05-06T12:14:42.565 INFO:tasks.workunit.client.0.smithi092.stdout:5/677: mkdir d4/d34/d28/df6 0 2024-05-06T12:14:42.567 INFO:tasks.workunit.client.1.smithi190.stdout:5/159: creat d2/da/f2f x:0 0 0 2024-05-06T12:14:42.568 INFO:tasks.workunit.client.0.smithi092.stdout:0/972: rmdir d7/dc/d28/d54/d1e/d26/d4e/dd7/d12f 0 2024-05-06T12:14:42.569 INFO:tasks.workunit.client.0.smithi092.stdout:3/601: mknod d8/d24/d27/cba 0 2024-05-06T12:14:42.571 INFO:tasks.workunit.client.1.smithi190.stdout:4/107: truncate d2/d11/d16/f18 103312 0 2024-05-06T12:14:42.573 INFO:tasks.workunit.client.1.smithi190.stdout:3/199: creat d5/d14/d11/d2a/d3d/f49 x:0 0 0 2024-05-06T12:14:42.575 INFO:tasks.workunit.client.0.smithi092.stdout:0/973: dread d7/dc/d28/d54/d1e/d46/df3/ff6 [0,4194304] 0 2024-05-06T12:14:42.578 INFO:tasks.workunit.client.0.smithi092.stdout:5/678: fdatasync d4/d5/d10/d31/d53/d64/f25 0 2024-05-06T12:14:42.579 INFO:tasks.workunit.client.1.smithi190.stdout:6/130: creat df/f2e x:0 0 0 2024-05-06T12:14:42.585 INFO:tasks.workunit.client.1.smithi190.stdout:0/320: truncate d9/f2c 3090802 0 2024-05-06T12:14:42.586 INFO:tasks.workunit.client.1.smithi190.stdout:7/147: rename d0/d1/d2/f29 to d0/d1/f3e 0 2024-05-06T12:14:42.592 INFO:tasks.workunit.client.1.smithi190.stdout:9/301: truncate db/d23/d25/f32 275199 0 2024-05-06T12:14:42.593 INFO:tasks.workunit.client.1.smithi190.stdout:5/160: creat d2/f30 x:0 0 0 2024-05-06T12:14:42.594 INFO:tasks.workunit.client.1.smithi190.stdout:5/161: read d2/f26 [3375486,14313] 0 2024-05-06T12:14:42.595 INFO:tasks.workunit.client.1.smithi190.stdout:0/321: mknod d9/dc/d28/d4c/c6d 0 2024-05-06T12:14:42.596 INFO:tasks.workunit.client.1.smithi190.stdout:0/322: stat d9/df/d21/c25 0 2024-05-06T12:14:42.599 INFO:tasks.workunit.client.1.smithi190.stdout:3/200: getdents d5/d14 0 2024-05-06T12:14:42.601 INFO:tasks.workunit.client.1.smithi190.stdout:3/201: readlink d5/d14/d11/d2a/l32 0 2024-05-06T12:14:42.603 INFO:tasks.workunit.client.1.smithi190.stdout:9/302: dread db/d21/d22/f56 [0,4194304] 0 2024-05-06T12:14:42.604 INFO:tasks.workunit.client.1.smithi190.stdout:0/323: dwrite d9/dc/d48/d50/f66 [0,4194304] 0 2024-05-06T12:14:42.605 INFO:tasks.workunit.client.1.smithi190.stdout:0/324: chown d9/df/d21/d31/c39 48145390 1 2024-05-06T12:14:42.605 INFO:tasks.workunit.client.1.smithi190.stdout:5/162: link d2/da/f1b d2/f31 0 2024-05-06T12:14:42.606 INFO:tasks.workunit.client.1.smithi190.stdout:5/163: stat d2/da/l16 0 2024-05-06T12:14:42.608 INFO:tasks.workunit.client.1.smithi190.stdout:5/164: chown d2/d27/l2b 257339104 1 2024-05-06T12:14:42.608 INFO:tasks.workunit.client.1.smithi190.stdout:3/202: creat d5/d7/d15/d1f/d33/d40/f4a x:0 0 0 2024-05-06T12:14:42.609 INFO:tasks.workunit.client.1.smithi190.stdout:5/165: readlink d2/da/l16 0 2024-05-06T12:14:42.609 INFO:tasks.workunit.client.1.smithi190.stdout:9/303: mknod db/dd/d16/d4c/d4e/d5e/c73 0 2024-05-06T12:14:42.610 INFO:tasks.workunit.client.1.smithi190.stdout:5/166: readlink d2/l2a 0 2024-05-06T12:14:42.611 INFO:tasks.workunit.client.1.smithi190.stdout:5/167: chown d2/d27 26563 1 2024-05-06T12:14:42.614 INFO:tasks.workunit.client.1.smithi190.stdout:9/304: truncate db/f2b 5381433 0 2024-05-06T12:14:42.614 INFO:tasks.workunit.client.1.smithi190.stdout:0/325: dwrite d9/df/d21/f6a [0,4194304] 0 2024-05-06T12:14:42.623 INFO:tasks.workunit.client.1.smithi190.stdout:0/326: link d9/df/d21/l30 d9/df/l6e 0 2024-05-06T12:14:42.652 INFO:tasks.workunit.client.0.smithi092.stdout:2/640: getdents df/d17/d51/d96 0 2024-05-06T12:14:42.653 INFO:tasks.workunit.client.0.smithi092.stdout:2/641: chown df/f55 0 1 2024-05-06T12:14:42.653 INFO:tasks.workunit.client.0.smithi092.stdout:1/734: rename d3/f70 to d3/fe9 0 2024-05-06T12:14:42.654 INFO:tasks.workunit.client.1.smithi190.stdout:6/131: read f2 [7134343,9856] 0 2024-05-06T12:14:42.654 INFO:tasks.workunit.client.1.smithi190.stdout:6/132: readlink l6 0 2024-05-06T12:14:42.655 INFO:tasks.workunit.client.1.smithi190.stdout:6/133: dread - df/d2c/f25 zero size 2024-05-06T12:14:42.656 INFO:tasks.workunit.client.1.smithi190.stdout:6/134: read f7 [4008738,7577] 0 2024-05-06T12:14:42.656 INFO:tasks.workunit.client.0.smithi092.stdout:6/805: write d4/d24/d26/dae/dc0/fc9 [1102689,51457] 0 2024-05-06T12:14:42.659 INFO:tasks.workunit.client.0.smithi092.stdout:2/642: read - df/d15/d88/d59/fab zero size 2024-05-06T12:14:42.660 INFO:tasks.workunit.client.1.smithi190.stdout:2/150: truncate d2/d8/dc/f16 196032 0 2024-05-06T12:14:42.662 INFO:tasks.workunit.client.0.smithi092.stdout:9/781: dread d16/d1e/d7b/fb0 [0,4194304] 0 2024-05-06T12:14:42.662 INFO:tasks.workunit.client.0.smithi092.stdout:8/815: dread d9/f18 [0,4194304] 0 2024-05-06T12:14:42.662 INFO:tasks.workunit.client.1.smithi190.stdout:2/151: rmdir d2/d10 39 2024-05-06T12:14:42.663 INFO:tasks.workunit.client.1.smithi190.stdout:6/135: dwrite df/f22 [0,4194304] 0 2024-05-06T12:14:42.663 INFO:tasks.workunit.client.0.smithi092.stdout:5/679: fsync d4/d5/d10/d31/ddf/d54/f9c 0 2024-05-06T12:14:42.665 INFO:tasks.workunit.client.1.smithi190.stdout:6/136: stat df/f15 0 2024-05-06T12:14:42.666 INFO:tasks.workunit.client.0.smithi092.stdout:6/806: fdatasync d4/d24/d26/dae/dc0/fcc 0 2024-05-06T12:14:42.666 INFO:tasks.workunit.client.1.smithi190.stdout:6/137: chown l8 278 1 2024-05-06T12:14:42.667 INFO:tasks.workunit.client.0.smithi092.stdout:3/602: truncate d8/d24/d33/d39/f44 2833197 0 2024-05-06T12:14:42.669 INFO:tasks.workunit.client.1.smithi190.stdout:2/152: creat d2/d8/d19/d25/f2b x:0 0 0 2024-05-06T12:14:42.669 INFO:tasks.workunit.client.1.smithi190.stdout:4/108: write d2/d11/d16/f18 [188489,73354] 0 2024-05-06T12:14:42.671 INFO:tasks.workunit.client.1.smithi190.stdout:2/153: write d2/d8/d19/f2a [3719379,101076] 0 2024-05-06T12:14:42.672 INFO:tasks.workunit.client.0.smithi092.stdout:9/782: symlink d16/d2c/d2f/d3d/d33/d6b/d9e/l106 0 2024-05-06T12:14:42.672 INFO:tasks.workunit.client.1.smithi190.stdout:2/154: chown d2/d8/dc/d20 100114092 1 2024-05-06T12:14:42.673 INFO:tasks.workunit.client.1.smithi190.stdout:2/155: dread - d2/f1f zero size 2024-05-06T12:14:42.675 INFO:tasks.workunit.client.0.smithi092.stdout:8/816: stat d9/d21/d37/d46/d5c/d62/d7a/c91 0 2024-05-06T12:14:42.676 INFO:tasks.workunit.client.0.smithi092.stdout:5/680: creat d4/d34/d28/d4e/d4b/dd6/ff7 x:0 0 0 2024-05-06T12:14:42.677 INFO:tasks.workunit.client.0.smithi092.stdout:6/807: mkdir d4/d24/d4e/d10a 0 2024-05-06T12:14:42.678 INFO:tasks.workunit.client.0.smithi092.stdout:5/681: write d4/d5/d10/d31/ddf/d54/d9f/fe5 [388506,76604] 0 2024-05-06T12:14:42.680 INFO:tasks.workunit.client.0.smithi092.stdout:0/974: link d7/dc/d9e/c8e d7/dc/d6f/da5/dc6/d100/c146 0 2024-05-06T12:14:42.680 INFO:tasks.workunit.client.1.smithi190.stdout:6/138: dwrite f2 [4194304,4194304] 0 2024-05-06T12:14:42.680 INFO:tasks.workunit.client.0.smithi092.stdout:1/735: rename d3/d10/d13/d19/d21 to d3/d10/d13/d19/d21/dea 22 2024-05-06T12:14:42.682 INFO:tasks.workunit.client.0.smithi092.stdout:0/975: read - d7/dc/d31/f139 zero size 2024-05-06T12:14:42.683 INFO:tasks.workunit.client.1.smithi190.stdout:1/141: sync 2024-05-06T12:14:42.685 INFO:tasks.workunit.client.1.smithi190.stdout:3/203: truncate d5/d7/d15/d1f/f3b 4093225 0 2024-05-06T12:14:42.686 INFO:tasks.workunit.client.0.smithi092.stdout:0/976: write d7/dc/d28/d54/d1e/d26/d96/fba [1942863,47133] 0 2024-05-06T12:14:42.686 INFO:tasks.workunit.client.1.smithi190.stdout:7/148: dwrite d0/ff [0,4194304] 0 2024-05-06T12:14:42.687 INFO:tasks.workunit.client.0.smithi092.stdout:1/736: mkdir d3/d10/d13/d19/d21/d72/d9a/da4/deb 0 2024-05-06T12:14:42.687 INFO:tasks.workunit.client.0.smithi092.stdout:5/682: dwrite d4/f68 [0,4194304] 0 2024-05-06T12:14:42.688 INFO:tasks.workunit.client.1.smithi190.stdout:6/139: truncate f7 2397996 0 2024-05-06T12:14:42.692 INFO:tasks.workunit.client.1.smithi190.stdout:2/156: dwrite d2/d8/d19/f1d [0,4194304] 0 2024-05-06T12:14:42.693 INFO:tasks.workunit.client.0.smithi092.stdout:0/977: dread d7/dc/d28/d54/d1e/d26/d3a/f95 [0,4194304] 0 2024-05-06T12:14:42.696 INFO:tasks.workunit.client.0.smithi092.stdout:8/817: getdents d9/d21 0 2024-05-06T12:14:42.696 INFO:tasks.workunit.client.0.smithi092.stdout:0/978: read d7/dc/d28/d54/d1e/d39/f47 [4010759,83537] 0 2024-05-06T12:14:42.706 INFO:tasks.workunit.client.1.smithi190.stdout:3/204: rename d5/d14/d11/f34 to d5/d7/d15/f4b 0 2024-05-06T12:14:42.707 INFO:tasks.workunit.client.0.smithi092.stdout:2/643: dwrite df/d17/d1d/f46 [0,4194304] 0 2024-05-06T12:14:42.708 INFO:tasks.workunit.client.1.smithi190.stdout:5/168: dwrite d2/da/f1b [0,4194304] 0 2024-05-06T12:14:42.710 INFO:tasks.workunit.client.0.smithi092.stdout:9/783: truncate d16/d1e/f2e 85533 0 2024-05-06T12:14:42.710 INFO:tasks.workunit.client.0.smithi092.stdout:2/644: chown df/d15/d88/d59/fb7 3011 1 2024-05-06T12:14:42.711 INFO:tasks.workunit.client.0.smithi092.stdout:9/784: stat d16/d1e/lbb 0 2024-05-06T12:14:42.712 INFO:tasks.workunit.client.0.smithi092.stdout:3/603: dwrite d8/db/d4c/d40/d67/f71 [0,4194304] 0 2024-05-06T12:14:42.716 INFO:tasks.workunit.client.0.smithi092.stdout:0/979: mkdir d7/dc/d28/d54/d1e/d147 0 2024-05-06T12:14:42.716 INFO:tasks.workunit.client.1.smithi190.stdout:9/305: dwrite db/dd/f34 [0,4194304] 0 2024-05-06T12:14:42.716 INFO:tasks.workunit.client.1.smithi190.stdout:0/327: dwrite d9/f2f [0,4194304] 0 2024-05-06T12:14:42.717 INFO:tasks.workunit.client.0.smithi092.stdout:9/785: chown d16/d2c/d2f/d3d/d33/d6b/d9e/l106 277991 1 2024-05-06T12:14:42.722 INFO:tasks.workunit.client.0.smithi092.stdout:1/737: dread d3/f2c [0,4194304] 0 2024-05-06T12:14:42.724 INFO:tasks.workunit.client.0.smithi092.stdout:3/604: write d8/d24/d27/f2e [416335,14912] 0 2024-05-06T12:14:42.725 INFO:tasks.workunit.client.0.smithi092.stdout:0/980: creat d7/dc/d28/d54/d12/d7f/db5/f148 x:0 0 0 2024-05-06T12:14:42.726 INFO:tasks.workunit.client.1.smithi190.stdout:2/157: creat d2/d10/f2c x:0 0 0 2024-05-06T12:14:42.728 INFO:tasks.workunit.client.1.smithi190.stdout:4/109: dwrite d2/d11/f14 [0,4194304] 0 2024-05-06T12:14:42.729 INFO:tasks.workunit.client.0.smithi092.stdout:0/981: write d7/dc/d28/d64/f78 [5134516,104468] 0 2024-05-06T12:14:42.731 INFO:tasks.workunit.client.1.smithi190.stdout:6/140: dwrite df/f2e [0,4194304] 0 2024-05-06T12:14:42.732 INFO:tasks.workunit.client.0.smithi092.stdout:3/605: creat d8/db/d4c/d40/fbb x:0 0 0 2024-05-06T12:14:42.734 INFO:tasks.workunit.client.0.smithi092.stdout:9/786: dwrite d16/d1c/f7c [4194304,4194304] 0 2024-05-06T12:14:42.735 INFO:tasks.workunit.client.0.smithi092.stdout:1/738: creat d3/d42/d84/d4f/fec x:0 0 0 2024-05-06T12:14:42.737 INFO:tasks.workunit.client.1.smithi190.stdout:5/169: rmdir d2/da/d2c/d2e 39 2024-05-06T12:14:42.738 INFO:tasks.workunit.client.1.smithi190.stdout:0/328: unlink d9/df/d21/d31/l5e 0 2024-05-06T12:14:42.739 INFO:tasks.workunit.client.1.smithi190.stdout:3/205: dread d5/d7/f23 [0,4194304] 0 2024-05-06T12:14:42.740 INFO:tasks.workunit.client.0.smithi092.stdout:3/606: fdatasync d8/db/d4c/d3d/f60 0 2024-05-06T12:14:42.742 INFO:tasks.workunit.client.1.smithi190.stdout:4/110: chown d2/c7 1498 1 2024-05-06T12:14:42.742 INFO:tasks.workunit.client.1.smithi190.stdout:6/141: rmdir df/d12 39 2024-05-06T12:14:42.743 INFO:tasks.workunit.client.0.smithi092.stdout:0/982: link d7/dc/d28/d64/fee d7/dc/d28/d54/d12/d7f/db5/f149 0 2024-05-06T12:14:42.745 INFO:tasks.workunit.client.1.smithi190.stdout:5/170: fdatasync d2/f26 0 2024-05-06T12:14:42.747 INFO:tasks.workunit.client.0.smithi092.stdout:3/607: getdents d8/d1b/d73/db7 0 2024-05-06T12:14:42.747 INFO:tasks.workunit.client.1.smithi190.stdout:5/171: truncate d2/d27/f20 4418378 0 2024-05-06T12:14:42.749 INFO:tasks.workunit.client.0.smithi092.stdout:0/983: read d7/dc/d28/d54/d1e/d26/d96/fc3 [6262521,108281] 0 2024-05-06T12:14:42.751 INFO:tasks.workunit.client.0.smithi092.stdout:3/608: unlink d8/db/d14/d5a/fa5 0 2024-05-06T12:14:42.752 INFO:tasks.workunit.client.0.smithi092.stdout:3/609: dread - d8/d24/d33/d39/d5b/f6d zero size 2024-05-06T12:14:42.753 INFO:tasks.workunit.client.1.smithi190.stdout:5/172: symlink d2/d27/l32 0 2024-05-06T12:14:42.754 INFO:tasks.workunit.client.1.smithi190.stdout:3/206: mknod d5/d7/c4c 0 2024-05-06T12:14:42.755 INFO:tasks.workunit.client.0.smithi092.stdout:3/610: write d8/db/d4c/d3d/d49/f8a [1188278,93242] 0 2024-05-06T12:14:42.755 INFO:tasks.workunit.client.0.smithi092.stdout:2/645: dread df/d17/d7d/f86 [0,4194304] 0 2024-05-06T12:14:42.756 INFO:tasks.workunit.client.1.smithi190.stdout:0/329: mkdir d9/d6f 0 2024-05-06T12:14:42.757 INFO:tasks.workunit.client.1.smithi190.stdout:4/111: symlink d2/l29 0 2024-05-06T12:14:42.758 INFO:tasks.workunit.client.1.smithi190.stdout:6/142: symlink df/l2f 0 2024-05-06T12:14:42.760 INFO:tasks.workunit.client.0.smithi092.stdout:0/984: dwrite d7/dc/d28/d54/fdd [0,4194304] 0 2024-05-06T12:14:42.762 INFO:tasks.workunit.client.1.smithi190.stdout:7/149: dread d0/d1/fc [0,4194304] 0 2024-05-06T12:14:42.762 INFO:tasks.workunit.client.0.smithi092.stdout:3/611: rename d8/db/l61 to d8/db/d14/d5e/lbc 0 2024-05-06T12:14:42.763 INFO:tasks.workunit.client.1.smithi190.stdout:6/143: creat df/d2c/f30 x:0 0 0 2024-05-06T12:14:42.764 INFO:tasks.workunit.client.0.smithi092.stdout:0/985: read d7/dc/d28/d54/d12/d104/f11e [477287,32458] 0 2024-05-06T12:14:42.765 INFO:tasks.workunit.client.1.smithi190.stdout:3/207: dwrite d5/d7/f3a [0,4194304] 0 2024-05-06T12:14:42.765 INFO:tasks.workunit.client.1.smithi190.stdout:4/112: symlink d2/l2a 0 2024-05-06T12:14:42.766 INFO:tasks.workunit.client.0.smithi092.stdout:2/646: dwrite df/d17/f22 [0,4194304] 0 2024-05-06T12:14:42.767 INFO:tasks.workunit.client.1.smithi190.stdout:4/113: write d2/d10/f1c [745597,55294] 0 2024-05-06T12:14:42.767 INFO:tasks.workunit.client.0.smithi092.stdout:2/647: chown df/d17/d53/d85 2561456 1 2024-05-06T12:14:42.767 INFO:tasks.workunit.client.0.smithi092.stdout:0/986: chown d7/dc/d28/d54/d1e/d26/d96/dfb 58 1 2024-05-06T12:14:42.768 INFO:tasks.workunit.client.1.smithi190.stdout:4/114: fdatasync d2/f1b 0 2024-05-06T12:14:42.768 INFO:tasks.workunit.client.0.smithi092.stdout:3/612: mkdir d8/db/d4c/d40/d7a/dbd 0 2024-05-06T12:14:42.770 INFO:tasks.workunit.client.1.smithi190.stdout:7/150: link d0/d4/l1f d0/d15/l3f 0 2024-05-06T12:14:42.771 INFO:tasks.workunit.client.0.smithi092.stdout:3/613: write d8/d24/d33/d39/d5b/f6d [482097,78726] 0 2024-05-06T12:14:42.771 INFO:tasks.workunit.client.1.smithi190.stdout:3/208: symlink d5/d14/d11/d21/d44/l4d 0 2024-05-06T12:14:42.773 INFO:tasks.workunit.client.1.smithi190.stdout:3/209: truncate d5/d14/d11/d21/f41 171592 0 2024-05-06T12:14:42.776 INFO:tasks.workunit.client.0.smithi092.stdout:0/987: mknod d7/dc/d28/d64/d140/c14a 0 2024-05-06T12:14:42.777 INFO:tasks.workunit.client.1.smithi190.stdout:3/210: mknod d5/d7/d15/d1f/d33/d40/c4e 0 2024-05-06T12:14:42.777 INFO:tasks.workunit.client.1.smithi190.stdout:4/115: link d2/l29 d2/d11/l2b 0 2024-05-06T12:14:42.779 INFO:tasks.workunit.client.0.smithi092.stdout:0/988: write d7/dc/d28/d54/d12/d7f/ff7 [305178,30146] 0 2024-05-06T12:14:42.780 INFO:tasks.workunit.client.0.smithi092.stdout:8/818: dwrite d9/d21/f3f [0,4194304] 0 2024-05-06T12:14:42.780 INFO:tasks.workunit.client.0.smithi092.stdout:0/989: stat d7/dc/d28/d54/d1e/d26/d4e/dd7 0 2024-05-06T12:14:42.781 INFO:tasks.workunit.client.1.smithi190.stdout:4/116: symlink d2/l2c 0 2024-05-06T12:14:42.782 INFO:tasks.workunit.client.0.smithi092.stdout:0/990: chown d7/dc/d9e/fc0 1748440 1 2024-05-06T12:14:42.783 INFO:tasks.workunit.client.1.smithi190.stdout:7/151: dwrite d0/d1/d2/f1d [0,4194304] 0 2024-05-06T12:14:42.783 INFO:tasks.workunit.client.0.smithi092.stdout:0/991: stat d7/dc/d28/d54/d1e/d26/d4e/laa 0 2024-05-06T12:14:42.786 INFO:tasks.workunit.client.0.smithi092.stdout:0/992: chown d7/dc/d28/d54/d1e/d26/faf 9828101 1 2024-05-06T12:14:42.786 INFO:tasks.workunit.client.1.smithi190.stdout:7/152: write d0/ff [1689848,70289] 0 2024-05-06T12:14:42.788 INFO:tasks.workunit.client.1.smithi190.stdout:7/153: fdatasync d0/d1/d2/f5 0 2024-05-06T12:14:42.790 INFO:tasks.workunit.client.0.smithi092.stdout:9/787: dread d16/d2c/d2f/f45 [0,4194304] 0 2024-05-06T12:14:42.792 INFO:tasks.workunit.client.0.smithi092.stdout:8/819: read d9/dc/f4d [4138133,40571] 0 2024-05-06T12:14:42.792 INFO:tasks.workunit.client.0.smithi092.stdout:9/788: write d16/d1e/d7b/fef [103119,53870] 0 2024-05-06T12:14:42.793 INFO:tasks.workunit.client.0.smithi092.stdout:0/993: creat d7/dc/d28/d54/d1e/d46/d63/f14b x:0 0 0 2024-05-06T12:14:42.793 INFO:tasks.workunit.client.1.smithi190.stdout:4/117: dwrite d2/d10/ff [0,4194304] 0 2024-05-06T12:14:42.794 INFO:tasks.workunit.client.1.smithi190.stdout:4/118: fsync d2/f25 0 2024-05-06T12:14:42.796 INFO:tasks.workunit.client.1.smithi190.stdout:7/154: dread d0/d1/d2/f1d [0,4194304] 0 2024-05-06T12:14:42.797 INFO:tasks.workunit.client.0.smithi092.stdout:5/683: dwrite d4/d5/d10/d31/ddf/d54/d73/f92 [0,4194304] 0 2024-05-06T12:14:42.797 INFO:tasks.workunit.client.1.smithi190.stdout:4/119: truncate d2/d11/f20 1678643 0 2024-05-06T12:14:42.800 INFO:tasks.workunit.client.0.smithi092.stdout:5/684: readlink d4/d34/d28/d4e/d5d/d82/d1f/d78/lb5 0 2024-05-06T12:14:42.801 INFO:tasks.workunit.client.1.smithi190.stdout:0/330: rmdir d9/df/d21/d31 39 2024-05-06T12:14:42.802 INFO:tasks.workunit.client.1.smithi190.stdout:0/331: write d9/fb [8045623,88929] 0 2024-05-06T12:14:42.803 INFO:tasks.workunit.client.0.smithi092.stdout:8/820: dwrite d9/d21/d37/fa3 [0,4194304] 0 2024-05-06T12:14:42.804 INFO:tasks.workunit.client.0.smithi092.stdout:0/994: rename d7/dc/d6f/l81 to d7/dc/d28/d54/d1e/d26/d4e/dd7/l14c 0 2024-05-06T12:14:42.805 INFO:tasks.workunit.client.1.smithi190.stdout:0/332: unlink d9/dc/l12 0 2024-05-06T12:14:42.806 INFO:tasks.workunit.client.1.smithi190.stdout:9/306: fsync db/dd/f34 0 2024-05-06T12:14:42.808 INFO:tasks.workunit.client.0.smithi092.stdout:5/685: chown d4/d34/d28/d4e/d67/f79 401583561 1 2024-05-06T12:14:42.809 INFO:tasks.workunit.client.1.smithi190.stdout:4/120: rename d2/l29 to d2/d11/l2d 0 2024-05-06T12:14:42.810 INFO:tasks.workunit.client.1.smithi190.stdout:7/155: getdents d0/d4 0 2024-05-06T12:14:42.811 INFO:tasks.workunit.client.1.smithi190.stdout:0/333: rmdir d9/df/d56 39 2024-05-06T12:14:42.812 INFO:tasks.workunit.client.1.smithi190.stdout:0/334: chown c2 111973 1 2024-05-06T12:14:42.812 INFO:tasks.workunit.client.0.smithi092.stdout:1/739: truncate d3/fdd 285637 0 2024-05-06T12:14:42.814 INFO:tasks.workunit.client.0.smithi092.stdout:8/821: rename d9/d21/d37/d46/d5c/d62/da5/de7 to d9/d21/d54/d116/d11f 0 2024-05-06T12:14:42.815 INFO:tasks.workunit.client.1.smithi190.stdout:4/121: rename d2/d11/d16/f18 to d2/d11/f2e 0 2024-05-06T12:14:42.815 INFO:tasks.workunit.client.0.smithi092.stdout:8/822: fdatasync d9/d21/d37/d46/fd2 0 2024-05-06T12:14:42.816 INFO:tasks.workunit.client.0.smithi092.stdout:8/823: chown d9/d21/c41 31 1 2024-05-06T12:14:42.817 INFO:tasks.workunit.client.1.smithi190.stdout:2/158: truncate d2/d8/d19/f2a 2738326 0 2024-05-06T12:14:42.818 INFO:tasks.workunit.client.1.smithi190.stdout:0/335: readlink d9/df/l32 0 2024-05-06T12:14:42.819 INFO:tasks.workunit.client.1.smithi190.stdout:4/122: creat d2/d11/d16/f2f x:0 0 0 2024-05-06T12:14:42.820 INFO:tasks.workunit.client.1.smithi190.stdout:5/173: rmdir d2 39 2024-05-06T12:14:42.820 INFO:tasks.workunit.client.0.smithi092.stdout:8/824: unlink d9/d21/d37/fa3 0 2024-05-06T12:14:42.821 INFO:tasks.workunit.client.1.smithi190.stdout:2/159: mknod d2/d8/dc/c2d 0 2024-05-06T12:14:42.822 INFO:tasks.workunit.client.1.smithi190.stdout:2/160: chown d2/fd 6242 1 2024-05-06T12:14:42.824 INFO:tasks.workunit.client.0.smithi092.stdout:5/686: rename d4/d34/d28/d4e/d5d/d82/d77 to d4/d34/d28/d4e/d5d/d82/d1f/d20/df8 0 2024-05-06T12:14:42.826 INFO:tasks.workunit.client.1.smithi190.stdout:2/161: mkdir d2/d8/dc/d2e 0 2024-05-06T12:14:42.827 INFO:tasks.workunit.client.0.smithi092.stdout:8/825: symlink d9/d21/d37/d46/d5c/d62/l120 0 2024-05-06T12:14:42.827 INFO:tasks.workunit.client.1.smithi190.stdout:2/162: dread - d2/d21/f22 zero size 2024-05-06T12:14:42.832 INFO:tasks.workunit.client.1.smithi190.stdout:0/336: creat d9/f70 x:0 0 0 2024-05-06T12:14:42.833 INFO:tasks.workunit.client.0.smithi092.stdout:2/648: write df/d17/d1d/d33/f44 [1339698,17108] 0 2024-05-06T12:14:42.834 INFO:tasks.workunit.client.1.smithi190.stdout:4/123: dwrite d2/d11/d16/f1a [0,4194304] 0 2024-05-06T12:14:42.835 INFO:tasks.workunit.client.0.smithi092.stdout:8/826: dread d9/d21/d54/f63 [4194304,4194304] 0 2024-05-06T12:14:42.837 INFO:tasks.workunit.client.1.smithi190.stdout:4/124: write d2/d11/d16/f1a [295289,91760] 0 2024-05-06T12:14:42.838 INFO:tasks.workunit.client.1.smithi190.stdout:2/163: dread d2/d8/d19/f1d [0,4194304] 0 2024-05-06T12:14:42.840 INFO:tasks.workunit.client.1.smithi190.stdout:4/125: write d2/f23 [45183,110021] 0 2024-05-06T12:14:42.844 INFO:tasks.workunit.client.1.smithi190.stdout:3/211: truncate d5/d7/f3a 1809181 0 2024-05-06T12:14:42.845 INFO:tasks.workunit.client.1.smithi190.stdout:4/126: mknod d2/d11/c30 0 2024-05-06T12:14:42.845 INFO:tasks.workunit.client.0.smithi092.stdout:2/649: mknod df/d17/d1d/d33/d4d/d83/cb8 0 2024-05-06T12:14:42.847 INFO:tasks.workunit.client.0.smithi092.stdout:3/614: dwrite d8/d1b/f81 [4194304,4194304] 0 2024-05-06T12:14:42.849 INFO:tasks.workunit.client.1.smithi190.stdout:2/164: rename d2/d10/c1e to d2/d8/dc/c2f 0 2024-05-06T12:14:42.849 INFO:tasks.workunit.client.0.smithi092.stdout:2/650: readlink df/d17/l29 0 2024-05-06T12:14:42.851 INFO:tasks.workunit.client.1.smithi190.stdout:3/212: rmdir d5/d14/d11/d2a/d3d 39 2024-05-06T12:14:42.852 INFO:tasks.workunit.client.0.smithi092.stdout:8/827: dwrite d9/f76 [4194304,4194304] 0 2024-05-06T12:14:42.853 INFO:tasks.workunit.client.0.smithi092.stdout:9/789: dwrite d16/d2c/d2f/d3d/d33/f3b [0,4194304] 0 2024-05-06T12:14:42.854 INFO:tasks.workunit.client.0.smithi092.stdout:2/651: chown df/d15/d88/d59/l90 6872542 1 2024-05-06T12:14:42.855 INFO:tasks.workunit.client.0.smithi092.stdout:0/995: dread d7/dc/d28/d54/d1e/d39/f57 [0,4194304] 0 2024-05-06T12:14:42.856 INFO:tasks.workunit.client.0.smithi092.stdout:9/790: rmdir d16/d2c/d4b 39 2024-05-06T12:14:42.857 INFO:tasks.workunit.client.1.smithi190.stdout:2/165: mknod d2/c30 0 2024-05-06T12:14:42.860 INFO:tasks.workunit.client.0.smithi092.stdout:8/828: creat d9/d21/d37/d74/df9/f121 x:0 0 0 2024-05-06T12:14:42.860 INFO:tasks.workunit.client.1.smithi190.stdout:3/213: dwrite d5/d14/f47 [0,4194304] 0 2024-05-06T12:14:42.862 INFO:tasks.workunit.client.1.smithi190.stdout:8/249: sync 2024-05-06T12:14:42.862 INFO:tasks.workunit.client.0.smithi092.stdout:2/652: creat df/d17/d1d/d33/d4d/d83/fb9 x:0 0 0 2024-05-06T12:14:42.863 INFO:tasks.workunit.client.1.smithi190.stdout:3/214: dread - d5/d7/d15/d1f/d33/d40/f4a zero size 2024-05-06T12:14:42.864 INFO:tasks.workunit.client.0.smithi092.stdout:5/687: dwrite d4/d5/d30/f46 [4194304,4194304] 0 2024-05-06T12:14:42.864 INFO:tasks.workunit.client.0.smithi092.stdout:8/829: write d9/d21/f3f [1613914,54825] 0 2024-05-06T12:14:42.865 INFO:tasks.workunit.client.0.smithi092.stdout:9/791: creat d16/d2c/d2f/d75/f107 x:0 0 0 2024-05-06T12:14:42.867 INFO:tasks.workunit.client.0.smithi092.stdout:2/653: write df/d17/d53/d85/f98 [1928402,44757] 0 2024-05-06T12:14:42.868 INFO:tasks.workunit.client.0.smithi092.stdout:0/996: symlink d7/dc/d28/d54/d12/d7f/db2/dbc/l14d 0 2024-05-06T12:14:42.868 INFO:tasks.workunit.client.1.smithi190.stdout:2/166: fsync d2/fe 0 2024-05-06T12:14:42.873 INFO:tasks.workunit.client.0.smithi092.stdout:5/688: symlink d4/d34/d28/d4e/lf9 0 2024-05-06T12:14:42.873 INFO:tasks.workunit.client.1.smithi190.stdout:8/250: rename d3/d8/l41 to d3/d35/l50 0 2024-05-06T12:14:42.876 INFO:tasks.workunit.client.1.smithi190.stdout:5/174: dwrite d2/f5 [0,4194304] 0 2024-05-06T12:14:42.878 INFO:tasks.workunit.client.0.smithi092.stdout:0/997: rename d7/dc/d28/d54/d1e/d26/l4b to d7/dc/d28/d64/d140/l14e 0 2024-05-06T12:14:42.879 INFO:tasks.workunit.client.0.smithi092.stdout:1/740: dread d3/d10/f53 [0,4194304] 0 2024-05-06T12:14:42.879 INFO:tasks.workunit.client.0.smithi092.stdout:0/998: dread - d7/dc/d28/d54/d12/fd8 zero size 2024-05-06T12:14:42.882 INFO:tasks.workunit.client.0.smithi092.stdout:9/792: link d16/d2c/d2f/d3d/f38 d16/d1c/d58/d92/dda/f108 0 2024-05-06T12:14:42.884 INFO:tasks.workunit.client.0.smithi092.stdout:9/793: dread - d16/d2c/d2f/d3d/d33/d6b/f101 zero size 2024-05-06T12:14:42.884 INFO:tasks.workunit.client.1.smithi190.stdout:9/307: dread f5 [0,4194304] 0 2024-05-06T12:14:42.885 INFO:tasks.workunit.client.0.smithi092.stdout:0/999: read d7/dc/d28/d54/d1e/d46/d63/fb9 [153111,21489] 0 2024-05-06T12:14:42.887 INFO:tasks.workunit.client.0.smithi092.stdout:9/794: creat d16/d94/ddc/f109 x:0 0 0 2024-05-06T12:14:42.890 INFO:tasks.workunit.client.0.smithi092.stdout:9/795: fsync d16/d2c/d2f/d75/fc0 0 2024-05-06T12:14:42.890 INFO:tasks.workunit.client.1.smithi190.stdout:8/251: symlink d3/l51 0 2024-05-06T12:14:42.890 INFO:tasks.workunit.client.1.smithi190.stdout:9/308: mknod db/d21/c74 0 2024-05-06T12:14:42.890 INFO:tasks.workunit.client.0.smithi092.stdout:1/741: dwrite d3/f98 [0,4194304] 0 2024-05-06T12:14:42.892 INFO:tasks.workunit.client.0.smithi092.stdout:1/742: chown d3/d10/d13/d19/d21/fb4 15613370 1 2024-05-06T12:14:42.893 INFO:tasks.workunit.client.1.smithi190.stdout:5/175: dread d2/f26 [0,4194304] 0 2024-05-06T12:14:42.895 INFO:tasks.workunit.client.1.smithi190.stdout:9/309: mknod db/d21/d22/c75 0 2024-05-06T12:14:42.895 INFO:tasks.workunit.client.0.smithi092.stdout:5/689: dread d4/d34/d28/d4e/d5d/d82/d1f/d20/f76 [0,4194304] 0 2024-05-06T12:14:42.896 INFO:tasks.workunit.client.1.smithi190.stdout:0/337: dwrite d9/df/d21/d31/f34 [0,4194304] 0 2024-05-06T12:14:42.897 INFO:tasks.workunit.client.0.smithi092.stdout:1/743: mkdir d3/d42/d84/d4f/dd5/ded 0 2024-05-06T12:14:42.900 INFO:tasks.workunit.client.1.smithi190.stdout:5/176: mknod d2/da/d2c/c33 0 2024-05-06T12:14:42.900 INFO:tasks.workunit.client.0.smithi092.stdout:1/744: mknod d3/d10/d13/d19/d34/cee 0 2024-05-06T12:14:42.901 INFO:tasks.workunit.client.0.smithi092.stdout:3/615: dwrite d8/d24/d33/d39/f44 [0,4194304] 0 2024-05-06T12:14:42.902 INFO:tasks.workunit.client.0.smithi092.stdout:5/690: getdents d4/d34/d28/d6b 0 2024-05-06T12:14:42.903 INFO:tasks.workunit.client.0.smithi092.stdout:1/745: chown d3/cf 3455 1 2024-05-06T12:14:42.904 INFO:tasks.workunit.client.1.smithi190.stdout:0/338: rename d9/f70 to d9/df/f71 0 2024-05-06T12:14:42.905 INFO:tasks.workunit.client.1.smithi190.stdout:9/310: mkdir db/dd/d16/d4c/d4e/d5e/d6e/d76 0 2024-05-06T12:14:42.905 INFO:tasks.workunit.client.1.smithi190.stdout:5/177: symlink d2/da/l34 0 2024-05-06T12:14:42.906 INFO:tasks.workunit.client.1.smithi190.stdout:9/311: fsync db/d21/d22/f56 0 2024-05-06T12:14:42.907 INFO:tasks.workunit.client.1.smithi190.stdout:0/339: mkdir d9/df/d21/d72 0 2024-05-06T12:14:42.909 INFO:tasks.workunit.client.0.smithi092.stdout:5/691: mkdir d4/d5/d10/da1/dfa 0 2024-05-06T12:14:42.909 INFO:tasks.workunit.client.0.smithi092.stdout:8/830: dread d9/f76 [0,4194304] 0 2024-05-06T12:14:42.910 INFO:tasks.workunit.client.1.smithi190.stdout:0/340: mknod d9/dc/d28/c73 0 2024-05-06T12:14:42.912 INFO:tasks.workunit.client.1.smithi190.stdout:9/312: creat db/f77 x:0 0 0 2024-05-06T12:14:42.914 INFO:tasks.workunit.client.0.smithi092.stdout:5/692: readlink d4/d34/d28/d4e/d4b/l80 0 2024-05-06T12:14:42.914 INFO:tasks.workunit.client.1.smithi190.stdout:5/178: dwrite d2/d27/f1c [4194304,4194304] 0 2024-05-06T12:14:42.915 INFO:tasks.workunit.client.1.smithi190.stdout:0/341: truncate d9/df/d56/f59 674698 0 2024-05-06T12:14:42.916 INFO:tasks.workunit.client.0.smithi092.stdout:1/746: mknod d3/d42/d84/cef 0 2024-05-06T12:14:42.916 INFO:tasks.workunit.client.1.smithi190.stdout:0/342: fsync d9/dc/f6c 0 2024-05-06T12:14:42.917 INFO:tasks.workunit.client.0.smithi092.stdout:9/796: dread d16/d2c/d2f/d3d/d33/d57/f82 [0,4194304] 0 2024-05-06T12:14:42.918 INFO:tasks.workunit.client.0.smithi092.stdout:1/747: stat d3/d10/d13/d19/d34/d89/l96 0 2024-05-06T12:14:42.918 INFO:tasks.workunit.client.0.smithi092.stdout:8/831: dwrite d9/dc/d4a/d5f/ff7 [0,4194304] 0 2024-05-06T12:14:42.919 INFO:tasks.workunit.client.0.smithi092.stdout:5/693: write d4/d34/d28/d4e/d5d/d82/d1f/d20/f76 [1772505,66505] 0 2024-05-06T12:14:42.922 INFO:tasks.workunit.client.0.smithi092.stdout:8/832: creat d9/d21/f122 x:0 0 0 2024-05-06T12:14:42.922 INFO:tasks.workunit.client.1.smithi190.stdout:5/179: readlink d2/da/d2c/d2e/l1e 0 2024-05-06T12:14:42.922 INFO:tasks.workunit.client.0.smithi092.stdout:9/797: mknod d16/d1c/dd4/dd9/c10a 0 2024-05-06T12:14:42.923 INFO:tasks.workunit.client.0.smithi092.stdout:9/798: stat d16/d2c/d2f/d75/la6 0 2024-05-06T12:14:42.925 INFO:tasks.workunit.client.0.smithi092.stdout:1/748: symlink d3/d10/d13/d19/d21/d72/d9a/da4/deb/lf0 0 2024-05-06T12:14:42.926 INFO:tasks.workunit.client.0.smithi092.stdout:5/694: symlink d4/d5/d10/da1/dfa/lfb 0 2024-05-06T12:14:42.926 INFO:tasks.workunit.client.1.smithi190.stdout:0/343: dwrite d9/df/d21/d31/f67 [0,4194304] 0 2024-05-06T12:14:42.927 INFO:tasks.workunit.client.0.smithi092.stdout:9/799: creat d16/d2c/d2f/d75/f10b x:0 0 0 2024-05-06T12:14:42.929 INFO:tasks.workunit.client.0.smithi092.stdout:9/800: fsync d16/d2c/d2f/d3d/fc2 0 2024-05-06T12:14:42.929 INFO:tasks.workunit.client.0.smithi092.stdout:5/695: creat d4/ffc x:0 0 0 2024-05-06T12:14:42.929 INFO:tasks.workunit.client.1.smithi190.stdout:3/215: dread d5/d7/d15/d1f/fe [0,4194304] 0 2024-05-06T12:14:42.929 INFO:tasks.workunit.client.1.smithi190.stdout:5/180: mknod d2/d27/c35 0 2024-05-06T12:14:42.930 INFO:tasks.workunit.client.1.smithi190.stdout:5/181: dread - d2/da/f2f zero size 2024-05-06T12:14:42.931 INFO:tasks.workunit.client.0.smithi092.stdout:1/749: creat d3/d10/d13/d19/ff1 x:0 0 0 2024-05-06T12:14:42.933 INFO:tasks.workunit.client.0.smithi092.stdout:1/750: chown d3/d10/d13/d19/d21/f4d 54479 1 2024-05-06T12:14:42.937 INFO:tasks.workunit.client.1.smithi190.stdout:3/216: unlink d5/d14/d11/d2a/f35 0 2024-05-06T12:14:42.937 INFO:tasks.workunit.client.0.smithi092.stdout:1/751: creat d3/d10/d13/d46/ff2 x:0 0 0 2024-05-06T12:14:42.938 INFO:tasks.workunit.client.0.smithi092.stdout:1/752: read - d3/d10/d13/d19/fdc zero size 2024-05-06T12:14:42.939 INFO:tasks.workunit.client.1.smithi190.stdout:0/344: creat d9/dc/f74 x:0 0 0 2024-05-06T12:14:42.939 INFO:tasks.workunit.client.1.smithi190.stdout:2/167: dwrite d2/fe [0,4194304] 0 2024-05-06T12:14:42.940 INFO:tasks.workunit.client.0.smithi092.stdout:2/654: truncate df/f61 2913274 0 2024-05-06T12:14:42.941 INFO:tasks.workunit.client.0.smithi092.stdout:1/753: dread d3/d10/f53 [0,4194304] 0 2024-05-06T12:14:42.943 INFO:tasks.workunit.client.0.smithi092.stdout:2/655: truncate df/d15/d88/d59/f8d 1711135 0 2024-05-06T12:14:42.945 INFO:tasks.workunit.client.0.smithi092.stdout:1/754: creat d3/d10/d13/ff3 x:0 0 0 2024-05-06T12:14:42.946 INFO:tasks.workunit.client.0.smithi092.stdout:2/656: creat df/d17/d53/d85/fba x:0 0 0 2024-05-06T12:14:42.948 INFO:tasks.workunit.client.1.smithi190.stdout:2/168: dwrite d2/f12 [0,4194304] 0 2024-05-06T12:14:42.950 INFO:tasks.workunit.client.1.smithi190.stdout:5/182: creat d2/da/d2c/f36 x:0 0 0 2024-05-06T12:14:42.953 INFO:tasks.workunit.client.1.smithi190.stdout:8/252: getdents d3/d35/d3a 0 2024-05-06T12:14:42.953 INFO:tasks.workunit.client.0.smithi092.stdout:1/755: getdents d3/d42/d84/dcf 0 2024-05-06T12:14:42.957 INFO:tasks.workunit.client.1.smithi190.stdout:2/169: chown d2/d8/dc/f16 15 1 2024-05-06T12:14:42.957 INFO:tasks.workunit.client.0.smithi092.stdout:3/616: dwrite d8/db/d4c/d3d/d49/f4f [0,4194304] 0 2024-05-06T12:14:42.958 INFO:tasks.workunit.client.0.smithi092.stdout:3/617: readlink d8/db/l20 0 2024-05-06T12:14:42.959 INFO:tasks.workunit.client.1.smithi190.stdout:5/183: dwrite d2/da/f1b [0,4194304] 0 2024-05-06T12:14:42.959 INFO:tasks.workunit.client.0.smithi092.stdout:3/618: fdatasync d8/db/d14/d5a/f94 0 2024-05-06T12:14:42.960 INFO:tasks.workunit.client.1.smithi190.stdout:2/170: chown d2/d10/f2c 6 1 2024-05-06T12:14:42.961 INFO:tasks.workunit.client.1.smithi190.stdout:3/217: link d5/fc d5/d7/d15/f4f 0 2024-05-06T12:14:42.961 INFO:tasks.workunit.client.1.smithi190.stdout:0/345: creat d9/df/f75 x:0 0 0 2024-05-06T12:14:42.961 INFO:tasks.workunit.client.1.smithi190.stdout:0/346: chown d9 122625 1 2024-05-06T12:14:42.962 INFO:tasks.workunit.client.0.smithi092.stdout:8/833: mknod d9/d21/d72/c123 0 2024-05-06T12:14:42.963 INFO:tasks.workunit.client.1.smithi190.stdout:9/313: write db/dd/f17 [3085716,73673] 0 2024-05-06T12:14:42.963 INFO:tasks.workunit.client.1.smithi190.stdout:2/171: mknod d2/d8/dc/c31 0 2024-05-06T12:14:42.964 INFO:tasks.workunit.client.0.smithi092.stdout:9/801: rmdir d16/d2c/d2f/d75 39 2024-05-06T12:14:42.964 INFO:tasks.workunit.client.1.smithi190.stdout:0/347: fdatasync d9/dc/d63/f69 0 2024-05-06T12:14:42.964 INFO:tasks.workunit.client.0.smithi092.stdout:9/802: write f7 [3079153,83292] 0 2024-05-06T12:14:42.967 INFO:tasks.workunit.client.1.smithi190.stdout:5/184: unlink d2/d27/l2b 0 2024-05-06T12:14:42.968 INFO:tasks.workunit.client.1.smithi190.stdout:5/185: readlink d2/da/d2c/d2e/l1e 0 2024-05-06T12:14:42.971 INFO:tasks.workunit.client.1.smithi190.stdout:0/348: dread d9/dc/d48/d50/f66 [0,4194304] 0 2024-05-06T12:14:42.972 INFO:tasks.workunit.client.0.smithi092.stdout:9/803: mkdir d16/d10c 0 2024-05-06T12:14:42.973 INFO:tasks.workunit.client.1.smithi190.stdout:2/172: mkdir d2/d8/d19/d25/d32 0 2024-05-06T12:14:42.974 INFO:tasks.workunit.client.0.smithi092.stdout:5/696: dread d4/d5/f2a [0,4194304] 0 2024-05-06T12:14:42.976 INFO:tasks.workunit.client.0.smithi092.stdout:1/756: creat d3/d10/d13/d19/d34/d89/ff4 x:0 0 0 2024-05-06T12:14:42.977 INFO:tasks.workunit.client.0.smithi092.stdout:5/697: write d4/d34/d28/d4e/d5d/d82/f63 [89118,62577] 0 2024-05-06T12:14:42.977 INFO:tasks.workunit.client.0.smithi092.stdout:1/757: write d3/d10/d13/ff3 [260395,40562] 0 2024-05-06T12:14:42.979 INFO:tasks.workunit.client.1.smithi190.stdout:8/253: link d3/f12 d3/f52 0 2024-05-06T12:14:42.979 INFO:tasks.workunit.client.1.smithi190.stdout:8/254: chown d3/fc 1307 1 2024-05-06T12:14:42.980 INFO:tasks.workunit.client.1.smithi190.stdout:9/314: symlink db/d23/l78 0 2024-05-06T12:14:42.982 INFO:tasks.workunit.client.0.smithi092.stdout:9/804: creat d16/d2c/d2f/d3d/d33/d37/f10d x:0 0 0 2024-05-06T12:14:42.985 INFO:tasks.workunit.client.0.smithi092.stdout:1/758: unlink d3/d10/d13/d19/d21/d72/d9a/da4/fe1 0 2024-05-06T12:14:42.985 INFO:tasks.workunit.client.0.smithi092.stdout:2/657: dread df/d17/d53/da9/f8b [0,4194304] 0 2024-05-06T12:14:42.986 INFO:tasks.workunit.client.1.smithi190.stdout:3/218: creat d5/d14/d11/f50 x:0 0 0 2024-05-06T12:14:42.986 INFO:tasks.workunit.client.0.smithi092.stdout:5/698: dwrite d4/d5/f3b [0,4194304] 0 2024-05-06T12:14:42.987 INFO:tasks.workunit.client.1.smithi190.stdout:5/186: mknod d2/c37 0 2024-05-06T12:14:42.988 INFO:tasks.workunit.client.0.smithi092.stdout:8/834: write d9/f2a [3371569,4477] 0 2024-05-06T12:14:42.988 INFO:tasks.workunit.client.0.smithi092.stdout:5/699: chown d4/d5/d10/l52 1 1 2024-05-06T12:14:42.989 INFO:tasks.workunit.client.1.smithi190.stdout:9/315: dwrite db/d21/f45 [0,4194304] 0 2024-05-06T12:14:42.989 INFO:tasks.workunit.client.1.smithi190.stdout:3/219: dread d5/d14/d11/d21/f41 [0,4194304] 0 2024-05-06T12:14:42.991 INFO:tasks.workunit.client.1.smithi190.stdout:2/173: rename d2/d8/d19/d25 to d2/d8/d33 0 2024-05-06T12:14:42.991 INFO:tasks.workunit.client.0.smithi092.stdout:2/658: mkdir df/d1b/d8e/dbb 0 2024-05-06T12:14:42.992 INFO:tasks.workunit.client.0.smithi092.stdout:2/659: fdatasync fd 0 2024-05-06T12:14:42.993 INFO:tasks.workunit.client.0.smithi092.stdout:8/835: fsync d9/f2a 0 2024-05-06T12:14:42.993 INFO:tasks.workunit.client.0.smithi092.stdout:5/700: mkdir d4/d5/d30/dfd 0 2024-05-06T12:14:42.995 INFO:tasks.workunit.client.0.smithi092.stdout:8/836: fsync d9/d21/f122 0 2024-05-06T12:14:42.996 INFO:tasks.workunit.client.0.smithi092.stdout:1/759: dread d3/d10/d13/f1f [0,4194304] 0 2024-05-06T12:14:42.996 INFO:tasks.workunit.client.1.smithi190.stdout:8/255: symlink d3/l53 0 2024-05-06T12:14:42.997 INFO:tasks.workunit.client.0.smithi092.stdout:5/701: readlink d4/d5/d30/l9a 0 2024-05-06T12:14:42.999 INFO:tasks.workunit.client.0.smithi092.stdout:8/837: truncate d9/dc/d4a/f6b 4831441 0 2024-05-06T12:14:42.999 INFO:tasks.workunit.client.1.smithi190.stdout:0/349: creat d9/dc/d28/f76 x:0 0 0 2024-05-06T12:14:42.999 INFO:tasks.workunit.client.0.smithi092.stdout:5/702: write d4/d5/f3b [1570861,105293] 0 2024-05-06T12:14:43.000 INFO:tasks.workunit.client.1.smithi190.stdout:2/174: unlink d2/d10/c23 0 2024-05-06T12:14:43.001 INFO:tasks.workunit.client.1.smithi190.stdout:0/350: chown d9/df/c29 362024 1 2024-05-06T12:14:43.001 INFO:tasks.workunit.client.1.smithi190.stdout:5/187: dread d2/da/f1b [0,4194304] 0 2024-05-06T12:14:43.001 INFO:tasks.workunit.client.0.smithi092.stdout:2/660: symlink df/d1b/db4/lbc 0 2024-05-06T12:14:43.002 INFO:tasks.workunit.client.1.smithi190.stdout:2/175: dread - d2/f1f zero size 2024-05-06T12:14:43.002 INFO:tasks.workunit.client.0.smithi092.stdout:1/760: creat d3/d10/d13/d19/d34/ff5 x:0 0 0 2024-05-06T12:14:43.003 INFO:tasks.workunit.client.1.smithi190.stdout:9/316: rename f5 to db/d21/d22/d49/f79 0 2024-05-06T12:14:43.004 INFO:tasks.workunit.client.1.smithi190.stdout:3/220: dwrite d5/d14/f47 [0,4194304] 0 2024-05-06T12:14:43.004 INFO:tasks.workunit.client.0.smithi092.stdout:1/761: write d3/d10/d13/d19/d21/d72/fdf [406849,63118] 0 2024-05-06T12:14:43.005 INFO:tasks.workunit.client.0.smithi092.stdout:3/619: dread d8/d24/d33/f64 [0,4194304] 0 2024-05-06T12:14:43.006 INFO:tasks.workunit.client.1.smithi190.stdout:8/256: creat d3/d8/d2b/d3e/f54 x:0 0 0 2024-05-06T12:14:43.006 INFO:tasks.workunit.client.1.smithi190.stdout:5/188: fsync d2/fc 0 2024-05-06T12:14:43.007 INFO:tasks.workunit.client.0.smithi092.stdout:5/703: mknod d4/d34/d28/d6b/cfe 0 2024-05-06T12:14:43.008 INFO:tasks.workunit.client.1.smithi190.stdout:5/189: stat d2/d27/c2d 0 2024-05-06T12:14:43.009 INFO:tasks.workunit.client.1.smithi190.stdout:5/190: chown d2/l7 432777305 1 2024-05-06T12:14:43.010 INFO:tasks.workunit.client.1.smithi190.stdout:2/176: mknod d2/d8/c34 0 2024-05-06T12:14:43.010 INFO:tasks.workunit.client.0.smithi092.stdout:8/838: readlink d9/d21/d37/l4f 0 2024-05-06T12:14:43.011 INFO:tasks.workunit.client.1.smithi190.stdout:9/317: rename db/d23/d29/f6d to db/dd/d16/d4c/d4e/d5e/d6e/f7a 0 2024-05-06T12:14:43.011 INFO:tasks.workunit.client.1.smithi190.stdout:9/318: readlink l8 0 2024-05-06T12:14:43.012 INFO:tasks.workunit.client.0.smithi092.stdout:2/661: link df/d15/d88/d59/fb0 df/d1b/fbd 0 2024-05-06T12:14:43.013 INFO:tasks.workunit.client.1.smithi190.stdout:0/351: mkdir d9/dc/d28/d77 0 2024-05-06T12:14:43.015 INFO:tasks.workunit.client.0.smithi092.stdout:5/704: creat d4/d5/d10/d31/d53/dcb/fff x:0 0 0 2024-05-06T12:14:43.015 INFO:tasks.workunit.client.0.smithi092.stdout:8/839: unlink d9/d21/d37/d46/l89 0 2024-05-06T12:14:43.017 INFO:tasks.workunit.client.0.smithi092.stdout:1/762: dwrite d3/d10/d13/d19/d21/d88/fd7 [0,4194304] 0 2024-05-06T12:14:43.018 INFO:tasks.workunit.client.0.smithi092.stdout:9/805: dwrite d16/f17 [4194304,4194304] 0 2024-05-06T12:14:43.018 INFO:tasks.workunit.client.1.smithi190.stdout:5/191: mkdir d2/da/d2c/d2e/d38 0 2024-05-06T12:14:43.018 INFO:tasks.workunit.client.1.smithi190.stdout:0/352: read d9/dc/d28/d4c/f4b [322631,112826] 0 2024-05-06T12:14:43.019 INFO:tasks.workunit.client.1.smithi190.stdout:5/192: chown d2/c37 209 1 2024-05-06T12:14:43.021 INFO:tasks.workunit.client.1.smithi190.stdout:5/193: dread d2/f28 [0,4194304] 0 2024-05-06T12:14:43.022 INFO:tasks.workunit.client.0.smithi092.stdout:7/861: sync 2024-05-06T12:14:43.022 INFO:tasks.workunit.client.0.smithi092.stdout:4/774: sync 2024-05-06T12:14:43.023 INFO:tasks.workunit.client.0.smithi092.stdout:8/840: creat d9/d21/d72/f124 x:0 0 0 2024-05-06T12:14:43.024 INFO:tasks.workunit.client.0.smithi092.stdout:3/620: dwrite d8/d24/d27/f2e [0,4194304] 0 2024-05-06T12:14:43.028 INFO:tasks.workunit.client.1.smithi190.stdout:3/221: creat d5/f51 x:0 0 0 2024-05-06T12:14:43.028 INFO:tasks.workunit.client.0.smithi092.stdout:3/621: write d8/d24/d33/d39/d5b/d98/fb2 [388350,86556] 0 2024-05-06T12:14:43.029 INFO:tasks.workunit.client.0.smithi092.stdout:1/763: symlink d3/d10/d13/d46/lf6 0 2024-05-06T12:14:43.032 INFO:tasks.workunit.client.1.smithi190.stdout:1/142: sync 2024-05-06T12:14:43.033 INFO:tasks.workunit.client.1.smithi190.stdout:5/194: write d2/fc [883184,115599] 0 2024-05-06T12:14:43.034 INFO:tasks.workunit.client.0.smithi092.stdout:7/862: creat d9/de/d10/d17/d21/d2e/d9c/f125 x:0 0 0 2024-05-06T12:14:43.035 INFO:tasks.workunit.client.0.smithi092.stdout:9/806: mknod d16/d2c/d2f/d3d/d33/c10e 0 2024-05-06T12:14:43.036 INFO:tasks.workunit.client.1.smithi190.stdout:3/222: mkdir d5/d14/d52 0 2024-05-06T12:14:43.036 INFO:tasks.workunit.client.1.smithi190.stdout:5/195: fsync d2/da/f2f 0 2024-05-06T12:14:43.036 INFO:tasks.workunit.client.0.smithi092.stdout:8/841: getdents d9/dc/d4a/d5f/da8/ddf/d115 0 2024-05-06T12:14:43.037 INFO:tasks.workunit.client.1.smithi190.stdout:2/177: rmdir d2/d8/dc/d2e 0 2024-05-06T12:14:43.039 INFO:tasks.workunit.client.1.smithi190.stdout:0/353: mknod d9/dc/d28/d77/c78 0 2024-05-06T12:14:43.040 INFO:tasks.workunit.client.0.smithi092.stdout:5/705: rename d4/d5/d10/d31/ddf/d54/f62 to d4/d34/d28/d4e/d5d/d82/d1f/d20/f100 0 2024-05-06T12:14:43.040 INFO:tasks.workunit.client.0.smithi092.stdout:9/807: truncate d16/d2c/d2f/d3d/d33/d7e/fd5 911992 0 2024-05-06T12:14:43.040 INFO:tasks.workunit.client.0.smithi092.stdout:7/863: truncate d9/de/d10/d17/d21/f49 468832 0 2024-05-06T12:14:43.041 INFO:tasks.workunit.client.1.smithi190.stdout:1/143: creat d5/d15/f30 x:0 0 0 2024-05-06T12:14:43.042 INFO:tasks.workunit.client.1.smithi190.stdout:1/144: truncate d5/d15/f17 921121 0 2024-05-06T12:14:43.043 INFO:tasks.workunit.client.1.smithi190.stdout:1/145: fdatasync d5/d23/d24/f28 0 2024-05-06T12:14:43.043 INFO:tasks.workunit.client.0.smithi092.stdout:7/864: unlink d9/de/d10/c11e 0 2024-05-06T12:14:43.044 INFO:tasks.workunit.client.1.smithi190.stdout:3/223: creat d5/d7/d15/d1f/d33/f53 x:0 0 0 2024-05-06T12:14:43.045 INFO:tasks.workunit.client.1.smithi190.stdout:9/319: getdents db/d23/d29 0 2024-05-06T12:14:43.046 INFO:tasks.workunit.client.0.smithi092.stdout:8/842: link d9/d21/d72/c123 d9/dc/d4a/d5f/da8/ddf/d115/c125 0 2024-05-06T12:14:43.047 INFO:tasks.workunit.client.0.smithi092.stdout:5/706: dwrite d4/d34/d28/d4e/d5d/d82/d1f/fcf [0,4194304] 0 2024-05-06T12:14:43.047 INFO:tasks.workunit.client.1.smithi190.stdout:2/178: unlink d2/d8/f28 0 2024-05-06T12:14:43.048 INFO:tasks.workunit.client.0.smithi092.stdout:7/865: symlink d9/de/d10/d17/d21/d2e/d9c/l126 0 2024-05-06T12:14:43.048 INFO:tasks.workunit.client.1.smithi190.stdout:8/257: truncate d3/d8/d2b/f34 3060472 0 2024-05-06T12:14:43.049 INFO:tasks.workunit.client.0.smithi092.stdout:5/707: write d4/d34/d28/d4e/fc6 [927499,75428] 0 2024-05-06T12:14:43.049 INFO:tasks.workunit.client.1.smithi190.stdout:0/354: mknod d9/dc/d28/d4c/c79 0 2024-05-06T12:14:43.050 INFO:tasks.workunit.client.1.smithi190.stdout:5/196: mknod d2/c39 0 2024-05-06T12:14:43.050 INFO:tasks.workunit.client.1.smithi190.stdout:0/355: dread - d9/dc/f6c zero size 2024-05-06T12:14:43.050 INFO:tasks.workunit.client.1.smithi190.stdout:1/146: fdatasync d5/d23/d24/f2f 0 2024-05-06T12:14:43.053 INFO:tasks.workunit.client.1.smithi190.stdout:5/197: fsync d2/d27/f20 0 2024-05-06T12:14:43.053 INFO:tasks.workunit.client.0.smithi092.stdout:7/866: mknod d9/de/d10/d17/d99/db6/df1/d101/c127 0 2024-05-06T12:14:43.054 INFO:tasks.workunit.client.0.smithi092.stdout:7/867: chown d9/de/d10/d17/d99/db6/df1/f66 7836 1 2024-05-06T12:14:43.055 INFO:tasks.workunit.client.1.smithi190.stdout:0/356: write d9/df/f62 [1048121,66125] 0 2024-05-06T12:14:43.059 INFO:tasks.workunit.client.1.smithi190.stdout:2/179: dwrite d2/d8/d33/f2b [0,4194304] 0 2024-05-06T12:14:43.059 INFO:tasks.workunit.client.1.smithi190.stdout:1/147: symlink d5/d15/d1e/l31 0 2024-05-06T12:14:43.060 INFO:tasks.workunit.client.0.smithi092.stdout:5/708: creat d4/d34/d28/d4e/d5d/d82/d1f/d20/f101 x:0 0 0 2024-05-06T12:14:43.060 INFO:tasks.workunit.client.0.smithi092.stdout:1/764: write d3/d10/d13/d46/f61 [1243109,42538] 0 2024-05-06T12:14:43.061 INFO:tasks.workunit.client.1.smithi190.stdout:8/258: mkdir d3/d35/d55 0 2024-05-06T12:14:43.062 INFO:tasks.workunit.client.1.smithi190.stdout:8/259: dread - d3/d13/f21 zero size 2024-05-06T12:14:43.065 INFO:tasks.workunit.client.1.smithi190.stdout:3/224: rename d5/f26 to d5/d14/d11/d2a/d3d/f54 0 2024-05-06T12:14:43.069 INFO:tasks.workunit.client.0.smithi092.stdout:1/765: rename d3/d42/l63 to d3/d10/d13/d19/d21/d88/d8b/lf7 0 2024-05-06T12:14:43.070 INFO:tasks.workunit.client.0.smithi092.stdout:4/775: dread d9/d2b/fac [0,4194304] 0 2024-05-06T12:14:43.071 INFO:tasks.workunit.client.0.smithi092.stdout:1/766: chown d3/d10/d13/d19/d21/f24 7540957 1 2024-05-06T12:14:43.072 INFO:tasks.workunit.client.0.smithi092.stdout:1/767: chown d3/d10/d13/d46/lda 15775096 1 2024-05-06T12:14:43.073 INFO:tasks.workunit.client.0.smithi092.stdout:4/776: creat d9/d2b/d5b/daf/ffe x:0 0 0 2024-05-06T12:14:43.074 INFO:tasks.workunit.client.0.smithi092.stdout:8/843: truncate d9/d21/d54/f63 2052604 0 2024-05-06T12:14:43.075 INFO:tasks.workunit.client.1.smithi190.stdout:3/225: write d5/d7/d15/f4b [458427,13631] 0 2024-05-06T12:14:43.077 INFO:tasks.workunit.client.0.smithi092.stdout:1/768: rename d3/d10/d13/d46/lda to d3/d42/d84/d4f/d58/d65/d81/d8d/lf8 0 2024-05-06T12:14:43.077 INFO:tasks.workunit.client.1.smithi190.stdout:3/226: dread - d5/d14/d11/d21/f30 zero size 2024-05-06T12:14:43.078 INFO:tasks.workunit.client.0.smithi092.stdout:5/709: write d4/d5/d30/feb [5427751,48786] 0 2024-05-06T12:14:43.079 INFO:tasks.workunit.client.1.smithi190.stdout:8/260: dwrite d3/d35/f3c [0,4194304] 0 2024-05-06T12:14:43.079 INFO:tasks.workunit.client.1.smithi190.stdout:9/320: write db/dd/d16/d4c/f69 [2744208,31098] 0 2024-05-06T12:14:43.080 INFO:tasks.workunit.client.0.smithi092.stdout:7/868: dread d9/de/d10/d17/d99/f7d [0,4194304] 0 2024-05-06T12:14:43.081 INFO:tasks.workunit.client.1.smithi190.stdout:8/261: read - d3/f2f zero size 2024-05-06T12:14:43.081 INFO:tasks.workunit.client.1.smithi190.stdout:2/180: creat d2/f35 x:0 0 0 2024-05-06T12:14:43.081 INFO:tasks.workunit.client.1.smithi190.stdout:3/227: chown d5/d7/d15 3 1 2024-05-06T12:14:43.083 INFO:tasks.workunit.client.1.smithi190.stdout:6/144: sync 2024-05-06T12:14:43.083 INFO:tasks.workunit.client.1.smithi190.stdout:3/228: chown d5/d14/d11/d21 74914945 1 2024-05-06T12:14:43.083 INFO:tasks.workunit.client.0.smithi092.stdout:1/769: creat d3/d10/d13/d19/d21/d72/d9a/ff9 x:0 0 0 2024-05-06T12:14:43.083 INFO:tasks.workunit.client.1.smithi190.stdout:5/198: write d2/f26 [4891314,125513] 0 2024-05-06T12:14:43.084 INFO:tasks.workunit.client.1.smithi190.stdout:8/262: dread - d3/d8/d25/f39 zero size 2024-05-06T12:14:43.087 INFO:tasks.workunit.client.1.smithi190.stdout:0/357: truncate d9/f1b 2902610 0 2024-05-06T12:14:43.087 INFO:tasks.workunit.client.1.smithi190.stdout:1/148: getdents d5/d15/d1e 0 2024-05-06T12:14:43.088 INFO:tasks.workunit.client.1.smithi190.stdout:1/149: dread - d5/d15/f30 zero size 2024-05-06T12:14:43.089 INFO:tasks.workunit.client.0.smithi092.stdout:5/710: dwrite d4/d34/fd0 [0,4194304] 0 2024-05-06T12:14:43.089 INFO:tasks.workunit.client.1.smithi190.stdout:0/358: readlink d9/df/d21/d31/l5c 0 2024-05-06T12:14:43.090 INFO:tasks.workunit.client.0.smithi092.stdout:7/869: symlink d9/de/d1f/d37/d39/l128 0 2024-05-06T12:14:43.092 INFO:tasks.workunit.client.0.smithi092.stdout:1/770: write d3/d10/d13/fbb [1085237,88544] 0 2024-05-06T12:14:43.095 INFO:tasks.workunit.client.0.smithi092.stdout:8/844: dread d9/d21/d37/f3b [0,4194304] 0 2024-05-06T12:14:43.095 INFO:tasks.workunit.client.1.smithi190.stdout:5/199: rename d2/cb to d2/d27/c3a 0 2024-05-06T12:14:43.098 INFO:tasks.workunit.client.0.smithi092.stdout:5/711: rename d4/d34/d28/d4e/d5d/d82/d1f to d4/d5/d10/d31/ddf/d54/db0/d102 0 2024-05-06T12:14:43.098 INFO:tasks.workunit.client.1.smithi190.stdout:1/150: mkdir d5/d23/d32 0 2024-05-06T12:14:43.100 INFO:tasks.workunit.client.0.smithi092.stdout:7/870: truncate d9/de/d10/d17/d6a/f9d 1785156 0 2024-05-06T12:14:43.105 INFO:tasks.workunit.client.1.smithi190.stdout:8/263: unlink d3/d8/f1b 0 2024-05-06T12:14:43.107 INFO:tasks.workunit.client.1.smithi190.stdout:5/200: chown d2/c1d 44818 1 2024-05-06T12:14:43.109 INFO:tasks.workunit.client.1.smithi190.stdout:3/229: getdents d5/d7 0 2024-05-06T12:14:43.109 INFO:tasks.workunit.client.0.smithi092.stdout:1/771: creat d3/d42/d84/d4f/dd5/ded/ffa x:0 0 0 2024-05-06T12:14:43.110 INFO:tasks.workunit.client.1.smithi190.stdout:5/201: write d2/da/d2c/f36 [334408,41507] 0 2024-05-06T12:14:43.110 INFO:tasks.workunit.client.1.smithi190.stdout:1/151: dwrite d5/d23/d24/f28 [0,4194304] 0 2024-05-06T12:14:43.110 INFO:tasks.workunit.client.0.smithi092.stdout:5/712: chown d4/d5/d10/d31/ddf/d54/l90 1585 1 2024-05-06T12:14:43.111 INFO:tasks.workunit.client.1.smithi190.stdout:9/321: dread db/dd/f17 [0,4194304] 0 2024-05-06T12:14:43.112 INFO:tasks.workunit.client.0.smithi092.stdout:5/713: fsync d4/d59/fd1 0 2024-05-06T12:14:43.113 INFO:tasks.workunit.client.0.smithi092.stdout:4/777: dwrite d9/df/fb0 [0,4194304] 0 2024-05-06T12:14:43.114 INFO:tasks.workunit.client.0.smithi092.stdout:1/772: write d3/d10/d13/d19/d34/f36 [3275866,54523] 0 2024-05-06T12:14:43.114 INFO:tasks.workunit.client.0.smithi092.stdout:5/714: write d4/d5/f3b [3588450,122823] 0 2024-05-06T12:14:43.116 INFO:tasks.workunit.client.0.smithi092.stdout:4/778: fsync d9/dd/da6/ff6 0 2024-05-06T12:14:43.117 INFO:tasks.workunit.client.0.smithi092.stdout:1/773: write d3/d10/d13/d19/fa9 [306563,113338] 0 2024-05-06T12:14:43.118 INFO:tasks.workunit.client.0.smithi092.stdout:5/715: stat d4/d34/d28/d4e/d67/l7c 0 2024-05-06T12:14:43.118 INFO:tasks.workunit.client.1.smithi190.stdout:8/264: mknod d3/d35/d3a/c56 0 2024-05-06T12:14:43.124 INFO:tasks.workunit.client.1.smithi190.stdout:8/265: chown d3 870200 1 2024-05-06T12:14:43.124 INFO:tasks.workunit.client.1.smithi190.stdout:9/322: fsync db/dd/d16/d4c/f69 0 2024-05-06T12:14:43.126 INFO:tasks.workunit.client.1.smithi190.stdout:3/230: creat d5/d7/d15/d39/f55 x:0 0 0 2024-05-06T12:14:43.129 INFO:tasks.workunit.client.1.smithi190.stdout:5/202: dwrite d2/f8 [0,4194304] 0 2024-05-06T12:14:43.131 INFO:tasks.workunit.client.1.smithi190.stdout:9/323: write db/dd/f17 [1085253,84707] 0 2024-05-06T12:14:43.131 INFO:tasks.workunit.client.0.smithi092.stdout:8/845: mkdir d9/d21/d37/dbd/dc3/d11d/d126 0 2024-05-06T12:14:43.132 INFO:tasks.workunit.client.1.smithi190.stdout:6/145: dwrite df/f18 [0,4194304] 0 2024-05-06T12:14:43.132 INFO:tasks.workunit.client.0.smithi092.stdout:8/846: chown d9/d21/d72/lbf 1612 1 2024-05-06T12:14:43.134 INFO:tasks.workunit.client.0.smithi092.stdout:7/871: mkdir d9/de/d10/d17/d99/dc3/dd2/df7/d129 0 2024-05-06T12:14:43.134 INFO:tasks.workunit.client.1.smithi190.stdout:5/203: chown d2/f5 2042094579 1 2024-05-06T12:14:43.135 INFO:tasks.workunit.client.1.smithi190.stdout:7/156: sync 2024-05-06T12:14:43.136 INFO:tasks.workunit.client.0.smithi092.stdout:7/872: truncate d9/de/d10/d17/d21/fd4 83270 0 2024-05-06T12:14:43.136 INFO:tasks.workunit.client.1.smithi190.stdout:2/181: dwrite d2/d8/d19/f2a [0,4194304] 0 2024-05-06T12:14:43.142 INFO:tasks.workunit.client.0.smithi092.stdout:5/716: creat d4/d34/d28/d4e/d4b/f103 x:0 0 0 2024-05-06T12:14:43.142 INFO:tasks.workunit.client.0.smithi092.stdout:5/717: stat d4/d5/d10/d31/d53/ddb 0 2024-05-06T12:14:43.142 INFO:tasks.workunit.client.0.smithi092.stdout:8/847: mknod d9/d21/d37/d46/d5c/d71/c127 0 2024-05-06T12:14:43.142 INFO:tasks.workunit.client.0.smithi092.stdout:7/873: symlink d9/de/d1f/l12a 0 2024-05-06T12:14:43.143 INFO:tasks.workunit.client.1.smithi190.stdout:4/127: sync 2024-05-06T12:14:43.143 INFO:tasks.workunit.client.1.smithi190.stdout:8/266: getdents d3/d8 0 2024-05-06T12:14:43.143 INFO:tasks.workunit.client.1.smithi190.stdout:7/157: write d0/d1/d2/f5 [621792,115659] 0 2024-05-06T12:14:43.143 INFO:tasks.workunit.client.1.smithi190.stdout:6/146: truncate fc 477156 0 2024-05-06T12:14:43.143 INFO:tasks.workunit.client.1.smithi190.stdout:0/359: write d9/f1b [2695785,65291] 0 2024-05-06T12:14:43.143 INFO:tasks.workunit.client.0.smithi092.stdout:8/848: write d9/d21/d54/d79/f82 [1029471,107579] 0 2024-05-06T12:14:43.144 INFO:tasks.workunit.client.0.smithi092.stdout:5/718: mkdir d4/d5/d10/d31/d53/ddb/d104 0 2024-05-06T12:14:43.145 INFO:tasks.workunit.client.0.smithi092.stdout:8/849: fsync d9/d21/d37/d46/fd2 0 2024-05-06T12:14:43.146 INFO:tasks.workunit.client.1.smithi190.stdout:9/324: mknod db/dd/d16/d4c/d4e/d5e/d6e/c7b 0 2024-05-06T12:14:43.146 INFO:tasks.workunit.client.1.smithi190.stdout:2/182: mkdir d2/d36 0 2024-05-06T12:14:43.147 INFO:tasks.workunit.client.0.smithi092.stdout:4/779: dread d9/dd/f87 [0,4194304] 0 2024-05-06T12:14:43.147 INFO:tasks.workunit.client.1.smithi190.stdout:2/183: chown d2/l26 124484 1 2024-05-06T12:14:43.149 INFO:tasks.workunit.client.0.smithi092.stdout:5/719: creat d4/d34/d28/d6b/f105 x:0 0 0 2024-05-06T12:14:43.150 INFO:tasks.workunit.client.1.smithi190.stdout:5/204: truncate d2/d27/f20 3030778 0 2024-05-06T12:14:43.152 INFO:tasks.workunit.client.0.smithi092.stdout:7/874: read d9/de/d10/d17/d6a/f7b [1380018,23060] 0 2024-05-06T12:14:43.152 INFO:tasks.workunit.client.0.smithi092.stdout:1/774: dread d3/d10/db3/fc4 [0,4194304] 0 2024-05-06T12:14:43.153 INFO:tasks.workunit.client.1.smithi190.stdout:7/158: creat d0/f40 x:0 0 0 2024-05-06T12:14:43.155 INFO:tasks.workunit.client.1.smithi190.stdout:1/152: dread d5/d15/f2a [0,4194304] 0 2024-05-06T12:14:43.156 INFO:tasks.workunit.client.0.smithi092.stdout:1/775: symlink d3/daf/lfb 0 2024-05-06T12:14:43.156 INFO:tasks.workunit.client.1.smithi190.stdout:6/147: mknod df/c31 0 2024-05-06T12:14:43.157 INFO:tasks.workunit.client.0.smithi092.stdout:5/720: dread d4/d34/d28/d4e/d5d/d82/f63 [0,4194304] 0 2024-05-06T12:14:43.157 INFO:tasks.workunit.client.0.smithi092.stdout:1/776: read - d3/d10/d13/d46/fa1 zero size 2024-05-06T12:14:43.159 INFO:tasks.workunit.client.0.smithi092.stdout:5/721: chown d4/d34/f1b 126583 1 2024-05-06T12:14:43.159 INFO:tasks.workunit.client.1.smithi190.stdout:4/128: dwrite d2/d11/f2e [0,4194304] 0 2024-05-06T12:14:43.160 INFO:tasks.workunit.client.1.smithi190.stdout:5/205: unlink d2/d27/f1c 0 2024-05-06T12:14:43.161 INFO:tasks.workunit.client.0.smithi092.stdout:5/722: rmdir d4/d5/d10/d31/d53/d64 39 2024-05-06T12:14:43.161 INFO:tasks.workunit.client.1.smithi190.stdout:6/148: write df/d2c/d1e/f2d [116062,45670] 0 2024-05-06T12:14:43.162 INFO:tasks.workunit.client.1.smithi190.stdout:0/360: creat d9/d6f/f7a x:0 0 0 2024-05-06T12:14:43.162 INFO:tasks.workunit.client.1.smithi190.stdout:7/159: read d0/d1/f24 [825223,96781] 0 2024-05-06T12:14:43.163 INFO:tasks.workunit.client.1.smithi190.stdout:6/149: truncate fd 1037793 0 2024-05-06T12:14:43.164 INFO:tasks.workunit.client.0.smithi092.stdout:5/723: symlink d4/d34/d28/d4e/d5d/d82/l106 0 2024-05-06T12:14:43.166 INFO:tasks.workunit.client.0.smithi092.stdout:5/724: write d4/d34/d28/f74 [549532,26060] 0 2024-05-06T12:14:43.168 INFO:tasks.workunit.client.1.smithi190.stdout:1/153: unlink d5/f22 0 2024-05-06T12:14:43.173 INFO:tasks.workunit.client.0.smithi092.stdout:5/725: truncate d4/d5/d10/d31/ddf/d54/db0/d102/d20/df8/f89 199028 0 2024-05-06T12:14:43.173 INFO:tasks.workunit.client.1.smithi190.stdout:4/129: mkdir d2/d11/d16/d31 0 2024-05-06T12:14:43.175 INFO:tasks.workunit.client.0.smithi092.stdout:5/726: rmdir d4/d5/d10/d31/ddf/da3/dc2 39 2024-05-06T12:14:43.175 INFO:tasks.workunit.client.1.smithi190.stdout:4/130: write d2/f1b [3725553,34943] 0 2024-05-06T12:14:43.176 INFO:tasks.workunit.client.0.smithi092.stdout:5/727: write d4/d5/d10/f33 [784023,73084] 0 2024-05-06T12:14:43.177 INFO:tasks.workunit.client.1.smithi190.stdout:5/206: mkdir d2/da/d2c/d2e/d3b 0 2024-05-06T12:14:43.178 INFO:tasks.workunit.client.0.smithi092.stdout:4/780: dread d9/dd/f80 [0,4194304] 0 2024-05-06T12:14:43.178 INFO:tasks.workunit.client.0.smithi092.stdout:5/728: mkdir d4/d34/d28/d4e/d67/d107 0 2024-05-06T12:14:43.179 INFO:tasks.workunit.client.0.smithi092.stdout:4/781: stat d9/d2b/d65/dbc 0 2024-05-06T12:14:43.181 INFO:tasks.workunit.client.0.smithi092.stdout:5/729: truncate d4/d5/d10/d31/ddf/da3/fd2 553276 0 2024-05-06T12:14:43.181 INFO:tasks.workunit.client.0.smithi092.stdout:4/782: mknod d9/df/cff 0 2024-05-06T12:14:43.182 INFO:tasks.workunit.client.1.smithi190.stdout:6/150: chown df/d12/f21 5 1 2024-05-06T12:14:43.185 INFO:tasks.workunit.client.1.smithi190.stdout:2/184: dwrite d2/d8/dc/f16 [0,4194304] 0 2024-05-06T12:14:43.185 INFO:tasks.workunit.client.0.smithi092.stdout:5/730: mkdir d4/d5/d30/dfd/d108 0 2024-05-06T12:14:43.185 INFO:tasks.workunit.client.1.smithi190.stdout:1/154: unlink d5/c1d 0 2024-05-06T12:14:43.186 INFO:tasks.workunit.client.0.smithi092.stdout:8/850: write d9/f36 [2723151,29787] 0 2024-05-06T12:14:43.187 INFO:tasks.workunit.client.1.smithi190.stdout:1/155: chown d5/d15/f2a 4219 1 2024-05-06T12:14:43.193 INFO:tasks.workunit.client.1.smithi190.stdout:4/131: chown d2/cb 0 1 2024-05-06T12:14:43.193 INFO:tasks.workunit.client.1.smithi190.stdout:5/207: dread d2/f26 [0,4194304] 0 2024-05-06T12:14:43.193 INFO:tasks.workunit.client.1.smithi190.stdout:8/267: link d3/d8/c4c d3/d35/c57 0 2024-05-06T12:14:43.193 INFO:tasks.workunit.client.1.smithi190.stdout:9/325: dread db/d21/d22/f54 [0,4194304] 0 2024-05-06T12:14:43.194 INFO:tasks.workunit.client.1.smithi190.stdout:0/361: link d9/fb d9/df/d21/d31/f7b 0 2024-05-06T12:14:43.195 INFO:tasks.workunit.client.1.smithi190.stdout:7/160: dread d0/d1/d2/f5 [0,4194304] 0 2024-05-06T12:14:43.195 INFO:tasks.workunit.client.1.smithi190.stdout:9/326: readlink db/dd/l15 0 2024-05-06T12:14:43.196 INFO:tasks.workunit.client.1.smithi190.stdout:5/208: write d2/f30 [33619,13514] 0 2024-05-06T12:14:43.196 INFO:tasks.workunit.client.1.smithi190.stdout:8/268: dread d3/f1c [0,4194304] 0 2024-05-06T12:14:43.197 INFO:tasks.workunit.client.1.smithi190.stdout:7/161: readlink d0/d1/d18/l25 0 2024-05-06T12:14:43.199 INFO:tasks.workunit.client.1.smithi190.stdout:5/209: write d2/f8 [1401130,58041] 0 2024-05-06T12:14:43.200 INFO:tasks.workunit.client.1.smithi190.stdout:4/132: unlink d2/d11/c30 0 2024-05-06T12:14:43.201 INFO:tasks.workunit.client.1.smithi190.stdout:6/151: mknod df/c32 0 2024-05-06T12:14:43.201 INFO:tasks.workunit.client.1.smithi190.stdout:6/152: read - df/f15 zero size 2024-05-06T12:14:43.202 INFO:tasks.workunit.client.1.smithi190.stdout:0/362: creat d9/dc/d28/d4c/f7c x:0 0 0 2024-05-06T12:14:43.202 INFO:tasks.workunit.client.1.smithi190.stdout:6/153: readlink df/d12/l19 0 2024-05-06T12:14:43.203 INFO:tasks.workunit.client.1.smithi190.stdout:6/154: chown df/d12 978 1 2024-05-06T12:14:43.204 INFO:tasks.workunit.client.1.smithi190.stdout:6/155: write df/d2c/d1e/f2d [1036431,113458] 0 2024-05-06T12:14:43.205 INFO:tasks.workunit.client.1.smithi190.stdout:2/185: truncate d2/d8/d19/f24 3324299 0 2024-05-06T12:14:43.209 INFO:tasks.workunit.client.1.smithi190.stdout:7/162: fdatasync d0/d1/d18/f2d 0 2024-05-06T12:14:43.211 INFO:tasks.workunit.client.1.smithi190.stdout:9/327: rename db/dd/f17 to db/d23/d29/f7c 0 2024-05-06T12:14:43.211 INFO:tasks.workunit.client.1.smithi190.stdout:2/186: dwrite d2/d8/dc/f16 [0,4194304] 0 2024-05-06T12:14:43.217 INFO:tasks.workunit.client.1.smithi190.stdout:6/156: creat df/d2c/f33 x:0 0 0 2024-05-06T12:14:43.221 INFO:tasks.workunit.client.1.smithi190.stdout:8/269: mknod d3/d35/d55/c58 0 2024-05-06T12:14:43.222 INFO:tasks.workunit.client.1.smithi190.stdout:1/156: fsync d5/f8 0 2024-05-06T12:14:43.224 INFO:tasks.workunit.client.1.smithi190.stdout:7/163: creat d0/d1/d2/f41 x:0 0 0 2024-05-06T12:14:43.225 INFO:tasks.workunit.client.1.smithi190.stdout:7/164: fsync d0/d1/d2/f1b 0 2024-05-06T12:14:43.226 INFO:tasks.workunit.client.1.smithi190.stdout:7/165: truncate d0/d1/f3e 873629 0 2024-05-06T12:14:43.227 INFO:tasks.workunit.client.1.smithi190.stdout:9/328: creat db/dd/d16/d58/f7d x:0 0 0 2024-05-06T12:14:43.228 INFO:tasks.workunit.client.1.smithi190.stdout:9/329: chown db/dd/d16/d4c/d4e/d5e 11799915 1 2024-05-06T12:14:43.229 INFO:tasks.workunit.client.1.smithi190.stdout:2/187: creat d2/d8/d33/f37 x:0 0 0 2024-05-06T12:14:43.235 INFO:tasks.workunit.client.1.smithi190.stdout:9/330: dwrite db/dd/d16/f24 [0,4194304] 0 2024-05-06T12:14:43.237 INFO:tasks.workunit.client.1.smithi190.stdout:7/166: chown d0/d4/l1f 468 1 2024-05-06T12:14:43.238 INFO:tasks.workunit.client.1.smithi190.stdout:7/167: write d0/f22 [953662,111562] 0 2024-05-06T12:14:43.240 INFO:tasks.workunit.client.1.smithi190.stdout:5/210: creat d2/da/d2c/f3c x:0 0 0 2024-05-06T12:14:43.241 INFO:tasks.workunit.client.1.smithi190.stdout:4/133: creat d2/f32 x:0 0 0 2024-05-06T12:14:43.244 INFO:tasks.workunit.client.0.smithi092.stdout:6/808: sync 2024-05-06T12:14:43.245 INFO:tasks.workunit.client.0.smithi092.stdout:5/731: creat d4/d34/d28/d4e/d5d/d94/f109 x:0 0 0 2024-05-06T12:14:43.246 INFO:tasks.workunit.client.1.smithi190.stdout:0/363: rename c1 to d9/df/c7d 0 2024-05-06T12:14:43.247 INFO:tasks.workunit.client.1.smithi190.stdout:5/211: dwrite d2/f3 [0,4194304] 0 2024-05-06T12:14:43.247 INFO:tasks.workunit.client.1.smithi190.stdout:6/157: rmdir df 39 2024-05-06T12:14:43.249 INFO:tasks.workunit.client.1.smithi190.stdout:8/270: dread d3/d8/f2c [0,4194304] 0 2024-05-06T12:14:43.249 INFO:tasks.workunit.client.1.smithi190.stdout:9/331: mkdir db/d23/d7e 0 2024-05-06T12:14:43.251 INFO:tasks.workunit.client.1.smithi190.stdout:9/332: write db/d23/f5b [3845377,63383] 0 2024-05-06T12:14:43.252 INFO:tasks.workunit.client.1.smithi190.stdout:7/168: mkdir d0/da/d36/d42 0 2024-05-06T12:14:43.253 INFO:tasks.workunit.client.0.smithi092.stdout:6/809: dwrite d4/f1a [0,4194304] 0 2024-05-06T12:14:43.254 INFO:tasks.workunit.client.1.smithi190.stdout:9/333: readlink db/d23/l50 0 2024-05-06T12:14:43.255 INFO:tasks.workunit.client.0.smithi092.stdout:7/875: dwrite d9/dd/f85 [0,4194304] 0 2024-05-06T12:14:43.258 INFO:tasks.workunit.client.0.smithi092.stdout:5/732: read d4/d5/d10/d31/d53/d64/f25 [3987183,7614] 0 2024-05-06T12:14:43.259 INFO:tasks.workunit.client.1.smithi190.stdout:8/271: mknod d3/d13/d4a/c59 0 2024-05-06T12:14:43.260 INFO:tasks.workunit.client.0.smithi092.stdout:6/810: fsync d4/d24/d66/f88 0 2024-05-06T12:14:43.260 INFO:tasks.workunit.client.1.smithi190.stdout:4/134: symlink d2/d11/d16/d31/l33 0 2024-05-06T12:14:43.262 INFO:tasks.workunit.client.1.smithi190.stdout:4/135: chown d2/c7 6792778 1 2024-05-06T12:14:43.262 INFO:tasks.workunit.client.1.smithi190.stdout:7/169: creat d0/da/d36/f43 x:0 0 0 2024-05-06T12:14:43.263 INFO:tasks.workunit.client.0.smithi092.stdout:4/783: write d9/df/d62/dbb/fc9 [1122982,93207] 0 2024-05-06T12:14:43.264 INFO:tasks.workunit.client.1.smithi190.stdout:4/136: read d2/f23 [50904,30396] 0 2024-05-06T12:14:43.266 INFO:tasks.workunit.client.0.smithi092.stdout:5/733: write d4/d5/d10/d31/ddf/d54/db0/d102/d20/f8d [6940379,1670] 0 2024-05-06T12:14:43.268 INFO:tasks.workunit.client.1.smithi190.stdout:2/188: rename d2/d21/f22 to d2/f38 0 2024-05-06T12:14:43.268 INFO:tasks.workunit.client.0.smithi092.stdout:7/876: mknod d9/d62/d121/c12b 0 2024-05-06T12:14:43.269 INFO:tasks.workunit.client.1.smithi190.stdout:0/364: creat d9/df/d21/f7e x:0 0 0 2024-05-06T12:14:43.269 INFO:tasks.workunit.client.1.smithi190.stdout:4/137: dread d2/f23 [0,4194304] 0 2024-05-06T12:14:43.269 INFO:tasks.workunit.client.0.smithi092.stdout:4/784: mkdir d9/dd/d90/d100 0 2024-05-06T12:14:43.270 INFO:tasks.workunit.client.1.smithi190.stdout:4/138: chown d2/d11 6959586 1 2024-05-06T12:14:43.270 INFO:tasks.workunit.client.1.smithi190.stdout:0/365: fdatasync d9/df/d21/f7e 0 2024-05-06T12:14:43.271 INFO:tasks.workunit.client.1.smithi190.stdout:8/272: read d3/d8/d2b/f34 [119780,113435] 0 2024-05-06T12:14:43.271 INFO:tasks.workunit.client.1.smithi190.stdout:4/139: chown d2/f1e 2 1 2024-05-06T12:14:43.271 INFO:tasks.workunit.client.0.smithi092.stdout:6/811: fdatasync d4/d20/d23/f28 0 2024-05-06T12:14:43.271 INFO:tasks.workunit.client.1.smithi190.stdout:4/140: chown d2/d10 28 1 2024-05-06T12:14:43.272 INFO:tasks.workunit.client.1.smithi190.stdout:8/273: dread - d3/d8/d25/f4b zero size 2024-05-06T12:14:43.273 INFO:tasks.workunit.client.0.smithi092.stdout:6/812: write d4/d24/d38/d5d/fcf [338985,81940] 0 2024-05-06T12:14:43.273 INFO:tasks.workunit.client.0.smithi092.stdout:5/734: rmdir d4/d5/d10/d31/d53/ddb 39 2024-05-06T12:14:43.273 INFO:tasks.workunit.client.1.smithi190.stdout:5/212: creat d2/f3d x:0 0 0 2024-05-06T12:14:43.276 INFO:tasks.workunit.client.1.smithi190.stdout:2/189: symlink d2/d10/l39 0 2024-05-06T12:14:43.279 INFO:tasks.workunit.client.1.smithi190.stdout:7/170: getdents d0/d1/d18/d37 0 2024-05-06T12:14:43.281 INFO:tasks.workunit.client.0.smithi092.stdout:8/851: truncate d9/d21/f3f 289533 0 2024-05-06T12:14:43.281 INFO:tasks.workunit.client.1.smithi190.stdout:4/141: dwrite d2/d10/f1c [0,4194304] 0 2024-05-06T12:14:43.285 INFO:tasks.workunit.client.1.smithi190.stdout:9/334: getdents db/dd/df 0 2024-05-06T12:14:43.285 INFO:tasks.workunit.client.0.smithi092.stdout:6/813: creat d4/d24/d26/dc4/d108/f10b x:0 0 0 2024-05-06T12:14:43.286 INFO:tasks.workunit.client.1.smithi190.stdout:6/158: getdents df/d2c 0 2024-05-06T12:14:43.286 INFO:tasks.workunit.client.0.smithi092.stdout:8/852: dread d9/d21/d37/d74/df9/f114 [0,4194304] 0 2024-05-06T12:14:43.287 INFO:tasks.workunit.client.1.smithi190.stdout:6/159: stat fe 0 2024-05-06T12:14:43.287 INFO:tasks.workunit.client.1.smithi190.stdout:7/171: creat d0/d1/d32/f44 x:0 0 0 2024-05-06T12:14:43.291 INFO:tasks.workunit.client.1.smithi190.stdout:4/142: dwrite d2/d11/f2e [0,4194304] 0 2024-05-06T12:14:43.292 INFO:tasks.workunit.client.0.smithi092.stdout:8/853: mkdir d9/d21/d54/d128 0 2024-05-06T12:14:43.295 INFO:tasks.workunit.client.1.smithi190.stdout:7/172: write d0/da/f2e [1006215,111824] 0 2024-05-06T12:14:43.296 INFO:tasks.workunit.client.0.smithi092.stdout:7/877: dwrite d9/dd/f9b [0,4194304] 0 2024-05-06T12:14:43.297 INFO:tasks.workunit.client.1.smithi190.stdout:7/173: chown d0/d1/d2/f5 4486 1 2024-05-06T12:14:43.297 INFO:tasks.workunit.client.0.smithi092.stdout:8/854: mknod d9/d21/d37/d74/c129 0 2024-05-06T12:14:43.298 INFO:tasks.workunit.client.1.smithi190.stdout:5/213: rmdir d2/da/d2c/d2e/d38 0 2024-05-06T12:14:43.300 INFO:tasks.workunit.client.0.smithi092.stdout:4/785: dwrite d9/fd5 [4194304,4194304] 0 2024-05-06T12:14:43.302 INFO:tasks.workunit.client.1.smithi190.stdout:2/190: rmdir d2/d8/d33/d32 0 2024-05-06T12:14:43.302 INFO:tasks.workunit.client.1.smithi190.stdout:9/335: mknod db/dd/d5c/c7f 0 2024-05-06T12:14:43.304 INFO:tasks.workunit.client.1.smithi190.stdout:4/143: symlink d2/d11/d16/l34 0 2024-05-06T12:14:43.304 INFO:tasks.workunit.client.1.smithi190.stdout:9/336: dread - db/d23/f68 zero size 2024-05-06T12:14:43.306 INFO:tasks.workunit.client.1.smithi190.stdout:7/174: mkdir d0/d1/d32/d45 0 2024-05-06T12:14:43.306 INFO:tasks.workunit.client.0.smithi092.stdout:5/735: dread d4/d5/d10/f33 [0,4194304] 0 2024-05-06T12:14:43.307 INFO:tasks.workunit.client.0.smithi092.stdout:6/814: write d4/d24/d26/dc4/fee [880465,87232] 0 2024-05-06T12:14:43.307 INFO:tasks.workunit.client.1.smithi190.stdout:4/144: truncate d2/f23 601599 0 2024-05-06T12:14:43.309 INFO:tasks.workunit.client.0.smithi092.stdout:4/786: read d9/fd5 [3894491,57119] 0 2024-05-06T12:14:43.309 INFO:tasks.workunit.client.0.smithi092.stdout:5/736: mkdir d4/d5/d30/d10a 0 2024-05-06T12:14:43.309 INFO:tasks.workunit.client.0.smithi092.stdout:6/815: read d4/d24/d66/f88 [553452,103587] 0 2024-05-06T12:14:43.311 INFO:tasks.workunit.client.0.smithi092.stdout:5/737: chown d4/d34/d28/d4e/f4f 1 1 2024-05-06T12:14:43.311 INFO:tasks.workunit.client.1.smithi190.stdout:2/191: unlink d2/d8/dc/d20/l27 0 2024-05-06T12:14:43.312 INFO:tasks.workunit.client.1.smithi190.stdout:2/192: truncate d2/f38 251053 0 2024-05-06T12:14:43.313 INFO:tasks.workunit.client.0.smithi092.stdout:4/787: creat d9/d2b/d65/f101 x:0 0 0 2024-05-06T12:14:43.314 INFO:tasks.workunit.client.0.smithi092.stdout:5/738: write d4/d34/d28/d4e/d5d/d82/f63 [174822,48536] 0 2024-05-06T12:14:43.315 INFO:tasks.workunit.client.0.smithi092.stdout:7/878: write d9/fb [3153278,10636] 0 2024-05-06T12:14:43.316 INFO:tasks.workunit.client.1.smithi190.stdout:9/337: symlink db/dd/d16/d4c/d4e/l80 0 2024-05-06T12:14:43.319 INFO:tasks.workunit.client.1.smithi190.stdout:6/160: dwrite f1 [0,4194304] 0 2024-05-06T12:14:43.320 INFO:tasks.workunit.client.0.smithi092.stdout:7/879: mknod d9/de/d10/d17/d99/dc3/dd2/df7/d129/c12c 0 2024-05-06T12:14:43.322 INFO:tasks.workunit.client.1.smithi190.stdout:7/175: rename d0/d1/f24 to d0/da/d36/f46 0 2024-05-06T12:14:43.324 INFO:tasks.workunit.client.0.smithi092.stdout:5/739: dwrite d4/d5/d10/d31/ddf/fd9 [0,4194304] 0 2024-05-06T12:14:43.324 INFO:tasks.workunit.client.1.smithi190.stdout:5/214: dread d2/da/d2c/f36 [0,4194304] 0 2024-05-06T12:14:43.326 INFO:tasks.workunit.client.1.smithi190.stdout:8/274: dread d3/f6 [0,4194304] 0 2024-05-06T12:14:43.327 INFO:tasks.workunit.client.0.smithi092.stdout:5/740: rmdir d4/d5/d10/d31/d53/d64 39 2024-05-06T12:14:43.329 INFO:tasks.workunit.client.1.smithi190.stdout:9/338: rmdir db/dd/d16/d4c/d4e/d5e 39 2024-05-06T12:14:43.330 INFO:tasks.workunit.client.1.smithi190.stdout:6/161: symlink df/d2c/l34 0 2024-05-06T12:14:43.331 INFO:tasks.workunit.client.0.smithi092.stdout:5/741: unlink d4/d5/d10/d31/ddf/d54/f9c 0 2024-05-06T12:14:43.331 INFO:tasks.workunit.client.1.smithi190.stdout:6/162: chown c0 400043159 1 2024-05-06T12:14:43.332 INFO:tasks.workunit.client.1.smithi190.stdout:5/215: chown d2/d27/c2d 578 1 2024-05-06T12:14:43.335 INFO:tasks.workunit.client.0.smithi092.stdout:5/742: rename d4/d5/d10/da1/le3 to d4/d34/dda/l10b 0 2024-05-06T12:14:43.336 INFO:tasks.workunit.client.1.smithi190.stdout:8/275: chown d3/d8/l37 529834 1 2024-05-06T12:14:43.336 INFO:tasks.workunit.client.1.smithi190.stdout:9/339: truncate db/dd/df/f1b 9115015 0 2024-05-06T12:14:43.336 INFO:tasks.workunit.client.0.smithi092.stdout:5/743: chown d4/d5/d10/d31 154063627 1 2024-05-06T12:14:43.337 INFO:tasks.workunit.client.0.smithi092.stdout:5/744: truncate d4/d5/fc4 69933 0 2024-05-06T12:14:43.337 INFO:tasks.workunit.client.0.smithi092.stdout:5/745: fsync d4/d34/dae/ff1 0 2024-05-06T12:14:43.339 INFO:tasks.workunit.client.1.smithi190.stdout:7/176: dwrite d0/d1/d18/d2b/f30 [0,4194304] 0 2024-05-06T12:14:43.340 INFO:tasks.workunit.client.1.smithi190.stdout:5/216: chown d2/da/d2c/d2e/c18 5841 1 2024-05-06T12:14:43.342 INFO:tasks.workunit.client.1.smithi190.stdout:7/177: read d0/da/f2e [1082144,42674] 0 2024-05-06T12:14:43.343 INFO:tasks.workunit.client.1.smithi190.stdout:6/163: chown df/d2c/d1e/f2d 595754801 1 2024-05-06T12:14:43.345 INFO:tasks.workunit.client.1.smithi190.stdout:8/276: symlink d3/d8/d2b/l5a 0 2024-05-06T12:14:43.346 INFO:tasks.workunit.client.0.smithi092.stdout:6/816: dread d4/d4c/f5c [0,4194304] 0 2024-05-06T12:14:43.347 INFO:tasks.workunit.client.1.smithi190.stdout:6/164: creat df/d2c/f35 x:0 0 0 2024-05-06T12:14:43.352 INFO:tasks.workunit.client.0.smithi092.stdout:7/880: truncate d9/de/d10/d17/faf 2998987 0 2024-05-06T12:14:43.353 INFO:tasks.workunit.client.1.smithi190.stdout:8/277: truncate d3/f52 8323828 0 2024-05-06T12:14:43.355 INFO:tasks.workunit.client.1.smithi190.stdout:4/145: getdents d2/d11/d16 0 2024-05-06T12:14:43.355 INFO:tasks.workunit.client.1.smithi190.stdout:4/146: stat d2/d11/l24 0 2024-05-06T12:14:43.356 INFO:tasks.workunit.client.1.smithi190.stdout:6/165: unlink df/d2c/f29 0 2024-05-06T12:14:43.356 INFO:tasks.workunit.client.1.smithi190.stdout:5/217: dwrite d2/da/f1b [0,4194304] 0 2024-05-06T12:14:43.357 INFO:tasks.workunit.client.1.smithi190.stdout:4/147: dread - d2/d11/d16/f2f zero size 2024-05-06T12:14:43.358 INFO:tasks.workunit.client.0.smithi092.stdout:6/817: dwrite d4/d4c/d52/f107 [0,4194304] 0 2024-05-06T12:14:43.358 INFO:tasks.workunit.client.1.smithi190.stdout:9/340: getdents db/dd 0 2024-05-06T12:14:43.359 INFO:tasks.workunit.client.0.smithi092.stdout:4/788: dwrite d9/d27/f41 [0,4194304] 0 2024-05-06T12:14:43.359 INFO:tasks.workunit.client.1.smithi190.stdout:4/148: fsync d2/d11/d16/f2f 0 2024-05-06T12:14:43.360 INFO:tasks.workunit.client.0.smithi092.stdout:6/818: chown d4/d4c/d91/d93/cce 10746 1 2024-05-06T12:14:43.361 INFO:tasks.workunit.client.1.smithi190.stdout:9/341: dread - db/d21/f4a zero size 2024-05-06T12:14:43.361 INFO:tasks.workunit.client.1.smithi190.stdout:4/149: read d2/d11/d16/f1a [370971,27115] 0 2024-05-06T12:14:43.362 INFO:tasks.workunit.client.1.smithi190.stdout:7/178: truncate d0/d1/d2/f1b 3836299 0 2024-05-06T12:14:43.362 INFO:tasks.workunit.client.1.smithi190.stdout:2/193: rmdir d2/d8 39 2024-05-06T12:14:43.363 INFO:tasks.workunit.client.1.smithi190.stdout:7/179: write d0/d1/d2/f26 [5015466,83544] 0 2024-05-06T12:14:43.364 INFO:tasks.workunit.client.1.smithi190.stdout:7/180: chown d0/da/d3d 101382950 1 2024-05-06T12:14:43.364 INFO:tasks.workunit.client.0.smithi092.stdout:4/789: mkdir d9/d102 0 2024-05-06T12:14:43.368 INFO:tasks.workunit.client.1.smithi190.stdout:8/278: read d3/d35/d3a/f3d [177209,74269] 0 2024-05-06T12:14:43.369 INFO:tasks.workunit.client.1.smithi190.stdout:5/218: unlink d2/da/d2c/c33 0 2024-05-06T12:14:43.370 INFO:tasks.workunit.client.1.smithi190.stdout:8/279: write d3/f2f [746367,21847] 0 2024-05-06T12:14:43.371 INFO:tasks.workunit.client.0.smithi092.stdout:6/819: rename d4/d24/l73 to d4/d4c/d91/d93/d9d/l10c 0 2024-05-06T12:14:43.371 INFO:tasks.workunit.client.1.smithi190.stdout:5/219: write d2/da/f1b [1864092,122862] 0 2024-05-06T12:14:43.372 INFO:tasks.workunit.client.1.smithi190.stdout:9/342: fsync db/d23/f53 0 2024-05-06T12:14:43.372 INFO:tasks.workunit.client.1.smithi190.stdout:5/220: chown d2/c12 187100329 1 2024-05-06T12:14:43.372 INFO:tasks.workunit.client.0.smithi092.stdout:5/746: dread d4/d5/d10/d31/ddf/d6c/fe4 [0,4194304] 0 2024-05-06T12:14:43.373 INFO:tasks.workunit.client.1.smithi190.stdout:4/150: creat d2/d10/f35 x:0 0 0 2024-05-06T12:14:43.374 INFO:tasks.workunit.client.0.smithi092.stdout:6/820: truncate d4/d24/d26/dae/fbd 349468 0 2024-05-06T12:14:43.374 INFO:tasks.workunit.client.0.smithi092.stdout:6/821: chown d4/d24/d38/d5d/fcf 38 1 2024-05-06T12:14:43.376 INFO:tasks.workunit.client.0.smithi092.stdout:6/822: dread - d4/d4c/d52/d6c/da0/dc3/fde zero size 2024-05-06T12:14:43.377 INFO:tasks.workunit.client.1.smithi190.stdout:7/181: mknod d0/da/c47 0 2024-05-06T12:14:43.378 INFO:tasks.workunit.client.0.smithi092.stdout:6/823: symlink d4/d20/d23/d30/df7/d101/l10d 0 2024-05-06T12:14:43.379 INFO:tasks.workunit.client.0.smithi092.stdout:6/824: write d4/d24/d38/d5d/fd0 [1906694,9093] 0 2024-05-06T12:14:43.382 INFO:tasks.workunit.client.1.smithi190.stdout:9/343: mknod db/d21/d22/c81 0 2024-05-06T12:14:43.382 INFO:tasks.workunit.client.1.smithi190.stdout:4/151: fsync d2/d11/d16/f1a 0 2024-05-06T12:14:43.386 INFO:tasks.workunit.client.1.smithi190.stdout:7/182: dread d0/d1/d18/f2d [0,4194304] 0 2024-05-06T12:14:43.388 INFO:tasks.workunit.client.1.smithi190.stdout:6/166: getdents df/d17 0 2024-05-06T12:14:43.390 INFO:tasks.workunit.client.1.smithi190.stdout:9/344: dread db/d23/d25/f32 [0,4194304] 0 2024-05-06T12:14:43.391 INFO:tasks.workunit.client.0.smithi092.stdout:7/881: dwrite d9/de/d10/d17/d21/f49 [0,4194304] 0 2024-05-06T12:14:43.392 INFO:tasks.workunit.client.1.smithi190.stdout:9/345: write f3 [1940801,104019] 0 2024-05-06T12:14:43.393 INFO:tasks.workunit.client.0.smithi092.stdout:7/882: chown d9/fe8 34 1 2024-05-06T12:14:43.394 INFO:tasks.workunit.client.1.smithi190.stdout:6/167: read f7 [1573510,97171] 0 2024-05-06T12:14:43.394 INFO:tasks.workunit.client.0.smithi092.stdout:4/790: rename d9/dd/f7e to d9/dd/d90/d100/f103 0 2024-05-06T12:14:43.396 INFO:tasks.workunit.client.1.smithi190.stdout:2/194: getdents d2/d8/dc/d20 0 2024-05-06T12:14:43.397 INFO:tasks.workunit.client.0.smithi092.stdout:4/791: dread - d9/d2b/d65/f101 zero size 2024-05-06T12:14:43.398 INFO:tasks.workunit.client.0.smithi092.stdout:4/792: write d9/d2b/d65/fc1 [782391,8858] 0 2024-05-06T12:14:43.399 INFO:tasks.workunit.client.0.smithi092.stdout:6/825: rmdir d4/d4c 39 2024-05-06T12:14:43.399 INFO:tasks.workunit.client.0.smithi092.stdout:4/793: chown d9/d2b/d65 1422 1 2024-05-06T12:14:43.401 INFO:tasks.workunit.client.1.smithi190.stdout:9/346: write f6 [1842205,73183] 0 2024-05-06T12:14:43.402 INFO:tasks.workunit.client.1.smithi190.stdout:7/183: link d0/da/d36/f43 d0/d1/d2f/d3b/f48 0 2024-05-06T12:14:43.403 INFO:tasks.workunit.client.1.smithi190.stdout:7/184: write d0/f20 [524034,29916] 0 2024-05-06T12:14:43.403 INFO:tasks.workunit.client.1.smithi190.stdout:6/168: creat df/d2c/f36 x:0 0 0 2024-05-06T12:14:43.405 INFO:tasks.workunit.client.1.smithi190.stdout:2/195: symlink d2/d21/l3a 0 2024-05-06T12:14:43.406 INFO:tasks.workunit.client.0.smithi092.stdout:4/794: fdatasync d9/d2b/f77 0 2024-05-06T12:14:43.406 INFO:tasks.workunit.client.1.smithi190.stdout:2/196: chown d2/c7 329251687 1 2024-05-06T12:14:43.408 INFO:tasks.workunit.client.0.smithi092.stdout:5/747: dwrite d4/d34/d28/d4e/f71 [0,4194304] 0 2024-05-06T12:14:43.409 INFO:tasks.workunit.client.1.smithi190.stdout:7/185: mknod d0/d15/c49 0 2024-05-06T12:14:43.410 INFO:tasks.workunit.client.1.smithi190.stdout:6/169: mkdir df/d17/d37 0 2024-05-06T12:14:43.411 INFO:tasks.workunit.client.0.smithi092.stdout:5/748: mknod d4/d34/dae/c10c 0 2024-05-06T12:14:43.412 INFO:tasks.workunit.client.0.smithi092.stdout:4/795: mkdir d9/df/d62/d7b/db6/d104 0 2024-05-06T12:14:43.415 INFO:tasks.workunit.client.0.smithi092.stdout:5/749: mknod d4/d34/d28/d4e/d5d/c10d 0 2024-05-06T12:14:43.415 INFO:tasks.workunit.client.0.smithi092.stdout:4/796: creat d9/d2b/d5b/daf/f105 x:0 0 0 2024-05-06T12:14:43.415 INFO:tasks.workunit.client.1.smithi190.stdout:4/152: dread d2/f1e [0,4194304] 0 2024-05-06T12:14:43.416 INFO:tasks.workunit.client.0.smithi092.stdout:4/797: fsync d9/df/fb0 0 2024-05-06T12:14:43.416 INFO:tasks.workunit.client.0.smithi092.stdout:7/883: truncate d9/de/d10/d17/d21/d38/d77/fee 526569 0 2024-05-06T12:14:43.419 INFO:tasks.workunit.client.1.smithi190.stdout:7/186: dread d0/f22 [0,4194304] 0 2024-05-06T12:14:43.419 INFO:tasks.workunit.client.1.smithi190.stdout:4/153: read - d2/f32 zero size 2024-05-06T12:14:43.419 INFO:tasks.workunit.client.0.smithi092.stdout:6/826: dread d4/d24/d26/fa4 [0,4194304] 0 2024-05-06T12:14:43.421 INFO:tasks.workunit.client.0.smithi092.stdout:7/884: fdatasync d9/f6f 0 2024-05-06T12:14:43.422 INFO:tasks.workunit.client.0.smithi092.stdout:5/750: dwrite d4/d5/d30/f40 [0,4194304] 0 2024-05-06T12:14:43.422 INFO:tasks.workunit.client.0.smithi092.stdout:4/798: fdatasync d9/dd/d90/d100/f103 0 2024-05-06T12:14:43.423 INFO:tasks.workunit.client.1.smithi190.stdout:2/197: creat d2/f3b x:0 0 0 2024-05-06T12:14:43.424 INFO:tasks.workunit.client.0.smithi092.stdout:4/799: write d9/d2b/f39 [5172624,21759] 0 2024-05-06T12:14:43.425 INFO:tasks.workunit.client.1.smithi190.stdout:6/170: getdents df 0 2024-05-06T12:14:43.429 INFO:tasks.workunit.client.1.smithi190.stdout:4/154: dread d2/f1e [0,4194304] 0 2024-05-06T12:14:43.431 INFO:tasks.workunit.client.0.smithi092.stdout:7/885: rename d9/de/d1f/dcf/fd6 to d9/de/d10/d17/f12d 0 2024-05-06T12:14:43.432 INFO:tasks.workunit.client.1.smithi190.stdout:8/280: dread d3/f12 [4194304,4194304] 0 2024-05-06T12:14:43.433 INFO:tasks.workunit.client.1.smithi190.stdout:6/171: chown df/d2c/d1e/c1a 310250 1 2024-05-06T12:14:43.433 INFO:tasks.workunit.client.1.smithi190.stdout:5/221: dwrite d2/d27/f20 [0,4194304] 0 2024-05-06T12:14:43.436 INFO:tasks.workunit.client.1.smithi190.stdout:4/155: symlink d2/d10/l36 0 2024-05-06T12:14:43.437 INFO:tasks.workunit.client.1.smithi190.stdout:9/347: dwrite db/d23/d25/f32 [0,4194304] 0 2024-05-06T12:14:43.437 INFO:tasks.workunit.client.1.smithi190.stdout:8/281: chown d3/f17 11670214 1 2024-05-06T12:14:43.440 INFO:tasks.workunit.client.1.smithi190.stdout:2/198: dread d2/d8/d19/f1d [0,4194304] 0 2024-05-06T12:14:43.440 INFO:tasks.workunit.client.1.smithi190.stdout:8/282: fdatasync d3/d8/d2b/d3e/f54 0 2024-05-06T12:14:43.444 INFO:tasks.workunit.client.1.smithi190.stdout:8/283: truncate d3/d8/d2b/d3e/f54 372736 0 2024-05-06T12:14:43.445 INFO:tasks.workunit.client.1.smithi190.stdout:6/172: rename df/d2c/f33 to df/d17/f38 0 2024-05-06T12:14:43.449 INFO:tasks.workunit.client.1.smithi190.stdout:4/156: creat d2/f37 x:0 0 0 2024-05-06T12:14:43.450 INFO:tasks.workunit.client.1.smithi190.stdout:2/199: rename d2/d8/d19/f1d to d2/d8/dc/f3c 0 2024-05-06T12:14:43.452 INFO:tasks.workunit.client.1.smithi190.stdout:8/284: dwrite d3/d35/f3c [4194304,4194304] 0 2024-05-06T12:14:43.453 INFO:tasks.workunit.client.1.smithi190.stdout:9/348: creat db/dd/d16/d4c/d4e/f82 x:0 0 0 2024-05-06T12:14:43.455 INFO:tasks.workunit.client.1.smithi190.stdout:4/157: rename d2/f1b to d2/d11/d16/f38 0 2024-05-06T12:14:43.458 INFO:tasks.workunit.client.1.smithi190.stdout:8/285: dread d3/fc [0,4194304] 0 2024-05-06T12:14:43.465 INFO:tasks.workunit.client.1.smithi190.stdout:2/200: rmdir d2/d36 0 2024-05-06T12:14:43.465 INFO:tasks.workunit.client.1.smithi190.stdout:2/201: write d2/f35 [229173,81851] 0 2024-05-06T12:14:43.466 INFO:tasks.workunit.client.1.smithi190.stdout:9/349: rename db/d21/f3c to db/d23/f83 0 2024-05-06T12:14:43.468 INFO:tasks.workunit.client.1.smithi190.stdout:5/222: truncate d2/f5 3056439 0 2024-05-06T12:14:43.469 INFO:tasks.workunit.client.1.smithi190.stdout:9/350: write db/dd/f34 [4678789,129874] 0 2024-05-06T12:14:43.471 INFO:tasks.workunit.client.1.smithi190.stdout:5/223: truncate d2/f3d 788240 0 2024-05-06T12:14:43.471 INFO:tasks.workunit.client.1.smithi190.stdout:2/202: dread d2/d8/d33/f2b [0,4194304] 0 2024-05-06T12:14:43.471 INFO:tasks.workunit.client.1.smithi190.stdout:8/286: rename d3/d8/f10 to d3/d8/d25/f5b 0 2024-05-06T12:14:43.472 INFO:tasks.workunit.client.1.smithi190.stdout:2/203: read - d2/d10/f2c zero size 2024-05-06T12:14:43.472 INFO:tasks.workunit.client.1.smithi190.stdout:8/287: chown d3/d13/d4a/c59 3469 1 2024-05-06T12:14:43.473 INFO:tasks.workunit.client.1.smithi190.stdout:2/204: chown d2/d8/d19/f2a 76 1 2024-05-06T12:14:43.474 INFO:tasks.workunit.client.0.smithi092.stdout:2/662: sync 2024-05-06T12:14:43.475 INFO:tasks.workunit.client.1.smithi190.stdout:5/224: symlink d2/da/l3e 0 2024-05-06T12:14:43.476 INFO:tasks.workunit.client.0.smithi092.stdout:5/751: dwrite d4/d5/d10/d31/ddf/d54/db0/d102/d20/f100 [0,4194304] 0 2024-05-06T12:14:43.478 INFO:tasks.workunit.client.0.smithi092.stdout:6/827: write d4/d20/d23/f28 [4986617,18660] 0 2024-05-06T12:14:43.479 INFO:tasks.workunit.client.0.smithi092.stdout:4/800: dwrite d9/d2b/d5b/fb5 [0,4194304] 0 2024-05-06T12:14:43.480 INFO:tasks.workunit.client.1.smithi190.stdout:6/173: dwrite df/d2c/d1e/f20 [0,4194304] 0 2024-05-06T12:14:43.481 INFO:tasks.workunit.client.1.smithi190.stdout:5/225: mknod d2/da/d2c/d2e/c3f 0 2024-05-06T12:14:43.482 INFO:tasks.workunit.client.1.smithi190.stdout:6/174: stat df/d17/d37 0 2024-05-06T12:14:43.483 INFO:tasks.workunit.client.0.smithi092.stdout:5/752: truncate d4/d5/d10/d31/ddf/d54/db0/d102/fcf 5096083 0 2024-05-06T12:14:43.485 INFO:tasks.workunit.client.1.smithi190.stdout:6/175: chown df/f2e 699620378 1 2024-05-06T12:14:43.487 INFO:tasks.workunit.client.0.smithi092.stdout:7/886: link d9/de/da4/la9 d9/de/d10/d17/d99/da6/d120/l12e 0 2024-05-06T12:14:43.489 INFO:tasks.workunit.client.1.smithi190.stdout:6/176: creat df/d2c/f39 x:0 0 0 2024-05-06T12:14:43.490 INFO:tasks.workunit.client.0.smithi092.stdout:7/887: creat d9/de/d1f/dcf/f12f x:0 0 0 2024-05-06T12:14:43.491 INFO:tasks.workunit.client.1.smithi190.stdout:8/288: truncate d3/d8/d25/f4b 359400 0 2024-05-06T12:14:43.491 INFO:tasks.workunit.client.0.smithi092.stdout:4/801: link d9/dd/cd1 d9/df/d62/d7b/c106 0 2024-05-06T12:14:43.493 INFO:tasks.workunit.client.1.smithi190.stdout:8/289: chown d3/d8/f2c 102062 1 2024-05-06T12:14:43.493 INFO:tasks.workunit.client.0.smithi092.stdout:4/802: mkdir d9/df/d62/d7b/d107 0 2024-05-06T12:14:43.495 INFO:tasks.workunit.client.1.smithi190.stdout:9/351: dread db/d23/f2a [0,4194304] 0 2024-05-06T12:14:43.495 INFO:tasks.workunit.client.0.smithi092.stdout:7/888: mkdir d9/de/d10/d17/d21/d130 0 2024-05-06T12:14:43.496 INFO:tasks.workunit.client.1.smithi190.stdout:6/177: getdents df/d2c/d1e 0 2024-05-06T12:14:43.496 INFO:tasks.workunit.client.1.smithi190.stdout:9/352: chown db/dd 496 1 2024-05-06T12:14:43.496 INFO:tasks.workunit.client.1.smithi190.stdout:6/178: dread - df/d2c/f39 zero size 2024-05-06T12:14:43.497 INFO:tasks.workunit.client.1.smithi190.stdout:8/290: mknod d3/c5c 0 2024-05-06T12:14:43.497 INFO:tasks.workunit.client.0.smithi092.stdout:4/803: mknod d9/d2b/d65/c108 0 2024-05-06T12:14:43.499 INFO:tasks.workunit.client.0.smithi092.stdout:7/889: mknod d9/de/d10/d17/d21/c131 0 2024-05-06T12:14:43.499 INFO:tasks.workunit.client.1.smithi190.stdout:9/353: dread db/dd/d16/f4f [0,4194304] 0 2024-05-06T12:14:43.500 INFO:tasks.workunit.client.0.smithi092.stdout:4/804: rmdir d9/d27/da8 39 2024-05-06T12:14:43.501 INFO:tasks.workunit.client.1.smithi190.stdout:8/291: fdatasync d3/f2d 0 2024-05-06T12:14:43.502 INFO:tasks.workunit.client.0.smithi092.stdout:7/890: rename d9/da5/l3c to d9/de/d10/d17/l132 0 2024-05-06T12:14:43.503 INFO:tasks.workunit.client.1.smithi190.stdout:6/179: dread df/f18 [0,4194304] 0 2024-05-06T12:14:43.503 INFO:tasks.workunit.client.0.smithi092.stdout:5/753: write f0 [5131957,20848] 0 2024-05-06T12:14:43.505 INFO:tasks.workunit.client.1.smithi190.stdout:8/292: chown d3/d8/c4c 91658817 1 2024-05-06T12:14:43.505 INFO:tasks.workunit.client.0.smithi092.stdout:2/663: dwrite df/d15/d88/d59/f8d [0,4194304] 0 2024-05-06T12:14:43.506 INFO:tasks.workunit.client.1.smithi190.stdout:6/180: symlink df/d17/l3a 0 2024-05-06T12:14:43.509 INFO:tasks.workunit.client.1.smithi190.stdout:9/354: link db/d23/l6a db/d23/l84 0 2024-05-06T12:14:43.509 INFO:tasks.workunit.client.1.smithi190.stdout:9/355: fdatasync db/dd/f41 0 2024-05-06T12:14:43.512 INFO:tasks.workunit.client.1.smithi190.stdout:8/293: unlink d3/d13/c42 0 2024-05-06T12:14:43.513 INFO:tasks.workunit.client.1.smithi190.stdout:8/294: chown d3/d35/d3a 2129 1 2024-05-06T12:14:43.564 INFO:tasks.workunit.client.1.smithi190.stdout:3/231: sync 2024-05-06T12:14:43.564 INFO:tasks.workunit.client.1.smithi190.stdout:3/232: fsync d5/f51 0 2024-05-06T12:14:43.565 INFO:tasks.workunit.client.1.smithi190.stdout:3/233: write d5/d7/f43 [431405,120046] 0 2024-05-06T12:14:43.566 INFO:tasks.workunit.client.1.smithi190.stdout:3/234: stat d5/d14/d11/d2a/d3d/l3e 0 2024-05-06T12:14:43.571 INFO:tasks.workunit.client.1.smithi190.stdout:3/235: dwrite d5/d7/d15/f22 [4194304,4194304] 0 2024-05-06T12:14:43.586 INFO:tasks.workunit.client.0.smithi092.stdout:2/664: creat df/d15/fbe x:0 0 0 2024-05-06T12:14:43.588 INFO:tasks.workunit.client.0.smithi092.stdout:2/665: write df/d17/d53/d85/f98 [2858260,79552] 0 2024-05-06T12:14:43.588 INFO:tasks.workunit.client.0.smithi092.stdout:5/754: link d4/d34/f1b d4/d5/d10/d31/ddf/d54/d73/f10e 0 2024-05-06T12:14:43.590 INFO:tasks.workunit.client.1.smithi190.stdout:6/181: link df/l2f df/d17/l3b 0 2024-05-06T12:14:43.590 INFO:tasks.workunit.client.1.smithi190.stdout:9/356: creat db/d21/f85 x:0 0 0 2024-05-06T12:14:43.591 INFO:tasks.workunit.client.0.smithi092.stdout:2/666: rmdir df/d17/d1d/d33/d4d/d83 39 2024-05-06T12:14:43.591 INFO:tasks.workunit.client.0.smithi092.stdout:6/828: dread d4/d24/d38/d5d/fa3 [0,4194304] 0 2024-05-06T12:14:43.592 INFO:tasks.workunit.client.1.smithi190.stdout:9/357: write db/f26 [1582832,31251] 0 2024-05-06T12:14:43.592 INFO:tasks.workunit.client.0.smithi092.stdout:2/667: chown df/d15/d88/fa7 16054875 1 2024-05-06T12:14:43.592 INFO:tasks.workunit.client.0.smithi092.stdout:5/755: mkdir d4/d5/d10/d31/ddf/d54/db0/d10f 0 2024-05-06T12:14:43.594 INFO:tasks.workunit.client.1.smithi190.stdout:6/182: creat df/d17/d37/f3c x:0 0 0 2024-05-06T12:14:43.594 INFO:tasks.workunit.client.1.smithi190.stdout:6/183: read - df/d2c/f39 zero size 2024-05-06T12:14:43.595 INFO:tasks.workunit.client.0.smithi092.stdout:6/829: chown d4/d4c/d52/d6c/da0 11385 1 2024-05-06T12:14:43.597 INFO:tasks.workunit.client.0.smithi092.stdout:5/756: rename d4/f9b to d4/d34/d28/d4e/f110 0 2024-05-06T12:14:43.599 INFO:tasks.workunit.client.0.smithi092.stdout:7/891: dread d9/de/d10/d17/d21/f5d [0,4194304] 0 2024-05-06T12:14:43.599 INFO:tasks.workunit.client.1.smithi190.stdout:9/358: creat db/dd/d16/d4c/d4e/d5e/d6e/f86 x:0 0 0 2024-05-06T12:14:43.600 INFO:tasks.workunit.client.0.smithi092.stdout:6/830: read d4/d20/d22/f7a [574147,6538] 0 2024-05-06T12:14:43.601 INFO:tasks.workunit.client.1.smithi190.stdout:6/184: rename df/d2c/d1e/l1f to df/d2c/d1e/l3d 0 2024-05-06T12:14:43.601 INFO:tasks.workunit.client.0.smithi092.stdout:2/668: getdents df/d17/d53/d85 0 2024-05-06T12:14:43.601 INFO:tasks.workunit.client.0.smithi092.stdout:6/831: chown d4/d24/d66/f88 3 1 2024-05-06T12:14:43.602 INFO:tasks.workunit.client.1.smithi190.stdout:3/236: link d5/f1d d5/d14/f56 0 2024-05-06T12:14:43.606 INFO:tasks.workunit.client.1.smithi190.stdout:9/359: rename db/d21 to db/dd/d16/d58/d87 0 2024-05-06T12:14:43.607 INFO:tasks.workunit.client.1.smithi190.stdout:6/185: mknod df/d2c/c3e 0 2024-05-06T12:14:43.609 INFO:tasks.workunit.client.1.smithi190.stdout:6/186: creat df/f3f x:0 0 0 2024-05-06T12:14:43.609 INFO:tasks.workunit.client.1.smithi190.stdout:6/187: readlink l6 0 2024-05-06T12:14:43.610 INFO:tasks.workunit.client.0.smithi092.stdout:7/892: dwrite d9/de/d10/d17/d99/db6/df1/f98 [0,4194304] 0 2024-05-06T12:14:43.614 INFO:tasks.workunit.client.1.smithi190.stdout:3/237: link d5/d7/l9 d5/d7/d15/d39/l57 0 2024-05-06T12:14:43.615 INFO:tasks.workunit.client.0.smithi092.stdout:6/832: dread d4/f8e [0,4194304] 0 2024-05-06T12:14:43.616 INFO:tasks.workunit.client.0.smithi092.stdout:7/893: write d9/da5/f54 [830544,56081] 0 2024-05-06T12:14:43.617 INFO:tasks.workunit.client.1.smithi190.stdout:9/360: rename db/dd/d16/d4c/d4e/d5e/d6e to db/dd/d16/d58/d87/d22/d49/d88 0 2024-05-06T12:14:43.623 INFO:tasks.workunit.client.0.smithi092.stdout:6/833: dwrite d4/d24/d26/ffb [0,4194304] 0 2024-05-06T12:14:43.624 INFO:tasks.workunit.client.1.smithi190.stdout:9/361: link db/dd/d16/f6c db/dd/d16/d58/d87/d22/d49/f89 0 2024-05-06T12:14:43.625 INFO:tasks.workunit.client.0.smithi092.stdout:2/669: dread df/d17/d51/d96/fae [0,4194304] 0 2024-05-06T12:14:43.626 INFO:tasks.workunit.client.0.smithi092.stdout:5/757: dread d4/d5/d10/d31/ddf/da3/fbf [0,4194304] 0 2024-05-06T12:14:43.626 INFO:tasks.workunit.client.0.smithi092.stdout:6/834: creat d4/d4c/d91/d93/d9d/f10e x:0 0 0 2024-05-06T12:14:43.628 INFO:tasks.workunit.client.0.smithi092.stdout:6/835: write d4/d24/d26/f109 [509579,93510] 0 2024-05-06T12:14:43.628 INFO:tasks.workunit.client.1.smithi190.stdout:3/238: dwrite d5/f27 [0,4194304] 0 2024-05-06T12:14:43.628 INFO:tasks.workunit.client.1.smithi190.stdout:3/239: chown d5/d14/d11/d21/d44 24 1 2024-05-06T12:14:43.629 INFO:tasks.workunit.client.0.smithi092.stdout:5/758: chown d4/d5/l6e 138720107 1 2024-05-06T12:14:43.630 INFO:tasks.workunit.client.1.smithi190.stdout:3/240: chown d5/d7/d15/d39/f55 52746826 1 2024-05-06T12:14:43.635 INFO:tasks.workunit.client.1.smithi190.stdout:3/241: rename d5/d14/d11/l3f to d5/d14/d11/d21/l58 0 2024-05-06T12:14:43.639 INFO:tasks.workunit.client.0.smithi092.stdout:2/670: write df/d15/d88/d59/fab [604261,128951] 0 2024-05-06T12:14:43.640 INFO:tasks.workunit.client.0.smithi092.stdout:5/759: truncate d4/d34/d28/d4e/d5d/d82/f63 442503 0 2024-05-06T12:14:43.640 INFO:tasks.workunit.client.1.smithi190.stdout:6/188: getdents df/d2c 0 2024-05-06T12:14:43.641 INFO:tasks.workunit.client.0.smithi092.stdout:5/760: dread - d4/ffc zero size 2024-05-06T12:14:43.642 INFO:tasks.workunit.client.0.smithi092.stdout:7/894: dwrite d9/de/f34 [0,4194304] 0 2024-05-06T12:14:43.643 INFO:tasks.workunit.client.1.smithi190.stdout:6/189: creat df/d2c/d28/f40 x:0 0 0 2024-05-06T12:14:43.643 INFO:tasks.workunit.client.0.smithi092.stdout:2/671: write df/d17/d1d/d33/f68 [1486958,80486] 0 2024-05-06T12:14:43.645 INFO:tasks.workunit.client.1.smithi190.stdout:6/190: read - df/d2c/f35 zero size 2024-05-06T12:14:43.646 INFO:tasks.workunit.client.0.smithi092.stdout:5/761: dread - d4/d5/d10/d31/ddf/d54/fa2 zero size 2024-05-06T12:14:43.647 INFO:tasks.workunit.client.1.smithi190.stdout:3/242: dwrite d5/d7/f3a [0,4194304] 0 2024-05-06T12:14:43.647 INFO:tasks.workunit.client.0.smithi092.stdout:6/836: dwrite d4/d24/d38/fcb [0,4194304] 0 2024-05-06T12:14:43.648 INFO:tasks.workunit.client.0.smithi092.stdout:7/895: write d9/de/d10/d17/d21/d2e/d9c/fdc [1026863,69098] 0 2024-05-06T12:14:43.648 INFO:tasks.workunit.client.1.smithi190.stdout:6/191: symlink df/d2c/d28/l41 0 2024-05-06T12:14:43.649 INFO:tasks.workunit.client.0.smithi092.stdout:5/762: fsync d4/d34/d28/d4e/d67/f79 0 2024-05-06T12:14:43.650 INFO:tasks.workunit.client.1.smithi190.stdout:6/192: chown df/d2c/f35 7943 1 2024-05-06T12:14:43.653 INFO:tasks.workunit.client.0.smithi092.stdout:6/837: rmdir d4/d4c/d52/d6c/da0/dc3 39 2024-05-06T12:14:43.654 INFO:tasks.workunit.client.0.smithi092.stdout:7/896: creat d9/de/d10/d17/d99/db6/df1/d101/f133 x:0 0 0 2024-05-06T12:14:43.657 INFO:tasks.workunit.client.1.smithi190.stdout:9/362: dread db/f3d [0,4194304] 0 2024-05-06T12:14:43.658 INFO:tasks.workunit.client.1.smithi190.stdout:9/363: read - db/dd/d16/d58/d87/f4a zero size 2024-05-06T12:14:43.658 INFO:tasks.workunit.client.0.smithi092.stdout:2/672: getdents df/d17/d53/d85/da0 0 2024-05-06T12:14:43.661 INFO:tasks.workunit.client.0.smithi092.stdout:6/838: unlink d4/f8e 0 2024-05-06T12:14:43.662 INFO:tasks.workunit.client.0.smithi092.stdout:5/763: dread d4/d34/d28/d4e/f4f [0,4194304] 0 2024-05-06T12:14:43.663 INFO:tasks.workunit.client.0.smithi092.stdout:7/897: creat d9/d62/d121/f134 x:0 0 0 2024-05-06T12:14:43.667 INFO:tasks.workunit.client.0.smithi092.stdout:5/764: rmdir d4/d34/d28/d4e/d5d 39 2024-05-06T12:14:43.669 INFO:tasks.workunit.client.0.smithi092.stdout:7/898: dwrite d9/f102 [0,4194304] 0 2024-05-06T12:14:43.671 INFO:tasks.workunit.client.0.smithi092.stdout:7/899: chown d9/de/d10/d17/f8f 2425 1 2024-05-06T12:14:43.671 INFO:tasks.workunit.client.0.smithi092.stdout:2/673: rename df/d17/d53/d85/da0/caa to df/d17/d53/d85/da0/cbf 0 2024-05-06T12:14:43.671 INFO:tasks.workunit.client.0.smithi092.stdout:6/839: mkdir d4/d24/d38/d5d/d6d/dd8/d10f 0 2024-05-06T12:14:43.673 INFO:tasks.workunit.client.0.smithi092.stdout:2/674: write df/d17/d51/f70 [1569416,80707] 0 2024-05-06T12:14:43.678 INFO:tasks.workunit.client.0.smithi092.stdout:7/900: rename d9/de/d1f/d37/fa0 to d9/d62/d108/d10d/f135 0 2024-05-06T12:14:43.682 INFO:tasks.workunit.client.0.smithi092.stdout:6/840: dwrite d4/d24/d26/dae/dc0/fcc [0,4194304] 0 2024-05-06T12:14:43.682 INFO:tasks.workunit.client.0.smithi092.stdout:5/765: dwrite d4/d34/d28/d4e/d5d/d82/fc0 [0,4194304] 0 2024-05-06T12:14:43.683 INFO:tasks.workunit.client.1.smithi190.stdout:3/243: dread d5/d14/f56 [0,4194304] 0 2024-05-06T12:14:43.685 INFO:tasks.workunit.client.1.smithi190.stdout:3/244: creat d5/f59 x:0 0 0 2024-05-06T12:14:43.686 INFO:tasks.workunit.client.1.smithi190.stdout:3/245: stat d5/c2b 0 2024-05-06T12:14:43.691 INFO:tasks.workunit.client.0.smithi092.stdout:5/766: creat d4/d5/d30/d10a/f111 x:0 0 0 2024-05-06T12:14:43.692 INFO:tasks.workunit.client.0.smithi092.stdout:7/901: dwrite d9/de/d10/d17/d21/f49 [4194304,4194304] 0 2024-05-06T12:14:43.693 INFO:tasks.workunit.client.1.smithi190.stdout:9/364: dread db/dd/d16/d4c/f69 [0,4194304] 0 2024-05-06T12:14:43.694 INFO:tasks.workunit.client.1.smithi190.stdout:9/365: write db/dd/d16/d4c/d4e/f82 [911285,101620] 0 2024-05-06T12:14:43.695 INFO:tasks.workunit.client.0.smithi092.stdout:3/622: sync 2024-05-06T12:14:43.699 INFO:tasks.workunit.client.0.smithi092.stdout:7/902: rename d9/de/d10/d17/d6a/f7b to d9/de/d10/d17/d21/d38/f136 0 2024-05-06T12:14:43.705 INFO:tasks.workunit.client.0.smithi092.stdout:6/841: fsync d4/d24/d26/dae/dc0/fcc 0 2024-05-06T12:14:43.706 INFO:tasks.workunit.client.1.smithi190.stdout:0/366: sync 2024-05-06T12:14:43.706 INFO:tasks.workunit.client.1.smithi190.stdout:1/157: sync 2024-05-06T12:14:43.706 INFO:tasks.workunit.client.1.smithi190.stdout:7/187: sync 2024-05-06T12:14:43.706 INFO:tasks.workunit.client.1.smithi190.stdout:7/188: chown d0/d1 1049984711 1 2024-05-06T12:14:43.706 INFO:tasks.workunit.client.0.smithi092.stdout:7/903: dread d9/de/d1f/d37/fdd [0,4194304] 0 2024-05-06T12:14:43.706 INFO:tasks.workunit.client.0.smithi092.stdout:3/623: rename d8/db/d14/d5e to d8/d1b/dbe 0 2024-05-06T12:14:43.708 INFO:tasks.workunit.client.0.smithi092.stdout:2/675: truncate f6 1890082 0 2024-05-06T12:14:43.708 INFO:tasks.workunit.client.0.smithi092.stdout:6/842: readlink d4/ld6 0 2024-05-06T12:14:43.710 INFO:tasks.workunit.client.1.smithi190.stdout:1/158: unlink d5/d15/l20 0 2024-05-06T12:14:43.710 INFO:tasks.workunit.client.0.smithi092.stdout:2/676: write df/d15/f95 [424690,115989] 0 2024-05-06T12:14:43.711 INFO:tasks.workunit.client.1.smithi190.stdout:1/159: dread - d5/d23/d24/f2c zero size 2024-05-06T12:14:43.711 INFO:tasks.workunit.client.1.smithi190.stdout:7/189: link d0/d4/f39 d0/da/f4a 0 2024-05-06T12:14:43.712 INFO:tasks.workunit.client.0.smithi092.stdout:5/767: truncate d4/d5/d10/f17 3230165 0 2024-05-06T12:14:43.714 INFO:tasks.workunit.client.1.smithi190.stdout:0/367: dwrite d9/dc/f4d [0,4194304] 0 2024-05-06T12:14:43.715 INFO:tasks.workunit.client.0.smithi092.stdout:2/677: rename df/d17/d7d/f86 to df/d17/d7e/fc0 0 2024-05-06T12:14:43.715 INFO:tasks.workunit.client.0.smithi092.stdout:6/843: truncate d4/d24/d38/d5d/d6d/dd8/ff0 816169 0 2024-05-06T12:14:43.716 INFO:tasks.workunit.client.1.smithi190.stdout:1/160: mknod d5/d23/c33 0 2024-05-06T12:14:43.716 INFO:tasks.workunit.client.1.smithi190.stdout:1/161: stat d5/cc 0 2024-05-06T12:14:43.717 INFO:tasks.workunit.client.1.smithi190.stdout:7/190: symlink d0/d1/d2/l4b 0 2024-05-06T12:14:43.717 INFO:tasks.workunit.client.0.smithi092.stdout:2/678: read - df/d17/d1d/d33/d4d/d83/f9d zero size 2024-05-06T12:14:43.717 INFO:tasks.workunit.client.1.smithi190.stdout:3/246: dread d5/d7/f1e [0,4194304] 0 2024-05-06T12:14:43.718 INFO:tasks.workunit.client.0.smithi092.stdout:5/768: mknod d4/d5/d10/d31/ddf/d54/c112 0 2024-05-06T12:14:43.718 INFO:tasks.workunit.client.0.smithi092.stdout:6/844: truncate d4/d24/d38/faa 2359389 0 2024-05-06T12:14:43.718 INFO:tasks.workunit.client.1.smithi190.stdout:7/191: truncate d0/ff 4301560 0 2024-05-06T12:14:43.719 INFO:tasks.workunit.client.0.smithi092.stdout:6/845: write d4/d24/d26/dae/dc0/fad [8058415,118848] 0 2024-05-06T12:14:43.720 INFO:tasks.workunit.client.1.smithi190.stdout:1/162: creat d5/f34 x:0 0 0 2024-05-06T12:14:43.720 INFO:tasks.workunit.client.0.smithi092.stdout:2/679: creat df/d17/d53/d85/da0/fc1 x:0 0 0 2024-05-06T12:14:43.721 INFO:tasks.workunit.client.1.smithi190.stdout:1/163: chown d5/d23/c33 508618 1 2024-05-06T12:14:43.721 INFO:tasks.workunit.client.1.smithi190.stdout:7/192: mkdir d0/d4c 0 2024-05-06T12:14:43.721 INFO:tasks.workunit.client.1.smithi190.stdout:0/368: dread d9/df/f62 [0,4194304] 0 2024-05-06T12:14:43.722 INFO:tasks.workunit.client.1.smithi190.stdout:3/247: truncate d5/d7/d15/d1f/d33/d40/f45 557452 0 2024-05-06T12:14:43.723 INFO:tasks.workunit.client.1.smithi190.stdout:0/369: fsync d9/dc/d28/d4c/f7c 0 2024-05-06T12:14:43.723 INFO:tasks.workunit.client.1.smithi190.stdout:7/193: symlink d0/d1/d18/l4d 0 2024-05-06T12:14:43.725 INFO:tasks.workunit.client.1.smithi190.stdout:3/248: chown d5/l8 11347367 1 2024-05-06T12:14:43.725 INFO:tasks.workunit.client.0.smithi092.stdout:2/680: mkdir df/d17/d1d/d33/d4d/d83/dc2 0 2024-05-06T12:14:43.726 INFO:tasks.workunit.client.1.smithi190.stdout:3/249: write d5/d14/d11/f50 [967340,120019] 0 2024-05-06T12:14:43.727 INFO:tasks.workunit.client.1.smithi190.stdout:0/370: creat d9/df/d21/d72/f7f x:0 0 0 2024-05-06T12:14:43.729 INFO:tasks.workunit.client.1.smithi190.stdout:7/194: truncate d0/d1/fc 4994554 0 2024-05-06T12:14:43.729 INFO:tasks.workunit.client.1.smithi190.stdout:9/366: dread db/dd/df/f1b [0,4194304] 0 2024-05-06T12:14:43.730 INFO:tasks.workunit.client.0.smithi092.stdout:2/681: creat df/d17/d7e/fc3 x:0 0 0 2024-05-06T12:14:43.730 INFO:tasks.workunit.client.1.smithi190.stdout:3/250: chown d5/d7/l9 63087 1 2024-05-06T12:14:43.731 INFO:tasks.workunit.client.1.smithi190.stdout:3/251: truncate d5/f51 570209 0 2024-05-06T12:14:43.732 INFO:tasks.workunit.client.1.smithi190.stdout:0/371: mkdir d9/dc/d28/d4c/d80 0 2024-05-06T12:14:43.733 INFO:tasks.workunit.client.1.smithi190.stdout:3/252: fdatasync d5/d14/d11/d21/f30 0 2024-05-06T12:14:43.733 INFO:tasks.workunit.client.0.smithi092.stdout:3/624: truncate d8/d24/d33/d4b/d74/f9e 821457 0 2024-05-06T12:14:43.734 INFO:tasks.workunit.client.1.smithi190.stdout:9/367: chown db/d23/c36 27 1 2024-05-06T12:14:43.734 INFO:tasks.workunit.client.0.smithi092.stdout:2/682: mknod df/d17/d1d/d33/cc4 0 2024-05-06T12:14:43.735 INFO:tasks.workunit.client.1.smithi190.stdout:9/368: write db/f13 [3518605,124278] 0 2024-05-06T12:14:43.737 INFO:tasks.workunit.client.0.smithi092.stdout:5/769: link d4/d5/d10/d31/ddf/d54/d73/lc7 d4/d5/d10/d31/ddf/da3/l113 0 2024-05-06T12:14:43.738 INFO:tasks.workunit.client.1.smithi190.stdout:0/372: mknod d9/dc/d63/c81 0 2024-05-06T12:14:43.739 INFO:tasks.workunit.client.0.smithi092.stdout:2/683: unlink df/d5c/c8a 0 2024-05-06T12:14:43.740 INFO:tasks.workunit.client.1.smithi190.stdout:0/373: write d9/df/d21/f4f [1109003,44011] 0 2024-05-06T12:14:43.740 INFO:tasks.workunit.client.1.smithi190.stdout:1/164: dread d5/fa [0,4194304] 0 2024-05-06T12:14:43.741 INFO:tasks.workunit.client.0.smithi092.stdout:5/770: dread d4/f7d [0,4194304] 0 2024-05-06T12:14:43.741 INFO:tasks.workunit.client.1.smithi190.stdout:3/253: creat d5/d14/d11/d2a/d3d/f5a x:0 0 0 2024-05-06T12:14:43.742 INFO:tasks.workunit.client.1.smithi190.stdout:0/374: write d9/dc/d48/f4e [446403,92859] 0 2024-05-06T12:14:43.743 INFO:tasks.workunit.client.1.smithi190.stdout:3/254: chown d5/d7/f3a 96546 1 2024-05-06T12:14:43.743 INFO:tasks.workunit.client.0.smithi092.stdout:2/684: rename df/d17/d7e/f91 to df/d5c/fc5 0 2024-05-06T12:14:43.744 INFO:tasks.workunit.client.1.smithi190.stdout:3/255: dread - d5/d7/d15/d1f/d33/f53 zero size 2024-05-06T12:14:43.747 INFO:tasks.workunit.client.0.smithi092.stdout:2/685: symlink df/d17/d53/d85/lc6 0 2024-05-06T12:14:43.747 INFO:tasks.workunit.client.1.smithi190.stdout:1/165: unlink d5/d15/f1f 0 2024-05-06T12:14:43.748 INFO:tasks.workunit.client.0.smithi092.stdout:6/846: truncate d4/d20/d23/fe9 3538049 0 2024-05-06T12:14:43.748 INFO:tasks.workunit.client.0.smithi092.stdout:2/686: readlink df/d5c/d60/l99 0 2024-05-06T12:14:43.749 INFO:tasks.workunit.client.0.smithi092.stdout:6/847: dread - d4/d4c/feb zero size 2024-05-06T12:14:43.749 INFO:tasks.workunit.client.1.smithi190.stdout:3/256: creat d5/d14/d11/d21/d44/f5b x:0 0 0 2024-05-06T12:14:43.750 INFO:tasks.workunit.client.0.smithi092.stdout:6/848: chown d4/f6 25 1 2024-05-06T12:14:43.750 INFO:tasks.workunit.client.0.smithi092.stdout:2/687: mknod df/d17/d51/d96/cc7 0 2024-05-06T12:14:43.751 INFO:tasks.workunit.client.1.smithi190.stdout:7/195: write d0/d1/d2f/d3b/f48 [409138,82171] 0 2024-05-06T12:14:43.751 INFO:tasks.workunit.client.0.smithi092.stdout:6/849: dread d4/d20/d22/f7a [0,4194304] 0 2024-05-06T12:14:43.752 INFO:tasks.workunit.client.1.smithi190.stdout:9/369: creat db/d23/f8a x:0 0 0 2024-05-06T12:14:43.755 INFO:tasks.workunit.client.0.smithi092.stdout:3/625: dread d8/db/d4c/d3d/f5d [0,4194304] 0 2024-05-06T12:14:43.755 INFO:tasks.workunit.client.1.smithi190.stdout:3/257: mkdir d5/d7/d15/d1f/d33/d5c 0 2024-05-06T12:14:43.756 INFO:tasks.workunit.client.1.smithi190.stdout:1/166: creat d5/d23/d2b/f35 x:0 0 0 2024-05-06T12:14:43.756 INFO:tasks.workunit.client.0.smithi092.stdout:3/626: write d8/d1b/f81 [7802906,98930] 0 2024-05-06T12:14:43.757 INFO:tasks.workunit.client.1.smithi190.stdout:1/167: readlink d5/l14 0 2024-05-06T12:14:43.757 INFO:tasks.workunit.client.1.smithi190.stdout:9/370: creat db/d23/d25/f8b x:0 0 0 2024-05-06T12:14:43.757 INFO:tasks.workunit.client.1.smithi190.stdout:1/168: stat d5/d15/f1a 0 2024-05-06T12:14:43.758 INFO:tasks.workunit.client.1.smithi190.stdout:7/196: truncate d0/d4/fd 1655917 0 2024-05-06T12:14:43.759 INFO:tasks.workunit.client.1.smithi190.stdout:7/197: write d0/d1/d18/d2b/f30 [2204006,94752] 0 2024-05-06T12:14:43.759 INFO:tasks.workunit.client.1.smithi190.stdout:1/169: truncate d5/fa 265228 0 2024-05-06T12:14:43.760 INFO:tasks.workunit.client.1.smithi190.stdout:1/170: chown d5/d23/d24/f2f 0 1 2024-05-06T12:14:43.761 INFO:tasks.workunit.client.1.smithi190.stdout:9/371: mkdir db/dd/d16/d58/d87/d22/d49/d8c 0 2024-05-06T12:14:43.762 INFO:tasks.workunit.client.1.smithi190.stdout:7/198: mkdir d0/d1/d18/d4e 0 2024-05-06T12:14:43.763 INFO:tasks.workunit.client.0.smithi092.stdout:3/627: fdatasync d8/db/d4c/d3d/f5d 0 2024-05-06T12:14:43.764 INFO:tasks.workunit.client.0.smithi092.stdout:3/628: write d8/db/d14/f50 [1184760,105488] 0 2024-05-06T12:14:43.766 INFO:tasks.workunit.client.1.smithi190.stdout:9/372: dread db/dd/d16/d4c/f57 [0,4194304] 0 2024-05-06T12:14:43.767 INFO:tasks.workunit.client.1.smithi190.stdout:7/199: creat d0/d1/d18/f4f x:0 0 0 2024-05-06T12:14:43.768 INFO:tasks.workunit.client.1.smithi190.stdout:7/200: stat d0/d4/f39 0 2024-05-06T12:14:43.769 INFO:tasks.workunit.client.1.smithi190.stdout:7/201: write d0/d4/f7 [953934,14551] 0 2024-05-06T12:14:43.770 INFO:tasks.workunit.client.1.smithi190.stdout:7/202: dread - d0/d4/f39 zero size 2024-05-06T12:14:43.771 INFO:tasks.workunit.client.1.smithi190.stdout:9/373: creat db/dd/f8d x:0 0 0 2024-05-06T12:14:43.775 INFO:tasks.workunit.client.0.smithi092.stdout:2/688: getdents df/d17/d51 0 2024-05-06T12:14:43.777 INFO:tasks.workunit.client.0.smithi092.stdout:6/850: dwrite d4/d4c/d52/f6a [0,4194304] 0 2024-05-06T12:14:43.778 INFO:tasks.workunit.client.0.smithi092.stdout:2/689: read - df/d15/d88/f89 zero size 2024-05-06T12:14:43.778 INFO:tasks.workunit.client.0.smithi092.stdout:3/629: mknod d8/db/d4c/cbf 0 2024-05-06T12:14:43.780 INFO:tasks.workunit.client.0.smithi092.stdout:6/851: write d4/d4c/da9/fd2 [7854061,122381] 0 2024-05-06T12:14:43.781 INFO:tasks.workunit.client.1.smithi190.stdout:7/203: rename d0/d4/l2a to d0/d1/d18/d2b/l50 0 2024-05-06T12:14:43.785 INFO:tasks.workunit.client.0.smithi092.stdout:3/630: fdatasync d8/db/d4c/d3d/f5d 0 2024-05-06T12:14:43.785 INFO:tasks.workunit.client.1.smithi190.stdout:7/204: symlink d0/d1/d2f/d35/l51 0 2024-05-06T12:14:43.786 INFO:tasks.workunit.client.1.smithi190.stdout:7/205: dread - d0/da/f4a zero size 2024-05-06T12:14:43.787 INFO:tasks.workunit.client.0.smithi092.stdout:2/690: mkdir df/d17/d1d/dc8 0 2024-05-06T12:14:43.788 INFO:tasks.workunit.client.1.smithi190.stdout:7/206: write d0/d1/d2/f26 [5179600,16074] 0 2024-05-06T12:14:43.788 INFO:tasks.workunit.client.1.smithi190.stdout:1/171: truncate d5/fa 331042 0 2024-05-06T12:14:43.789 INFO:tasks.workunit.client.1.smithi190.stdout:7/207: readlink d0/d1/d18/l4d 0 2024-05-06T12:14:43.789 INFO:tasks.workunit.client.1.smithi190.stdout:1/172: write d5/f16 [294195,1787] 0 2024-05-06T12:14:43.792 INFO:tasks.workunit.client.0.smithi092.stdout:5/771: dread d4/d5/d30/feb [0,4194304] 0 2024-05-06T12:14:43.793 INFO:tasks.workunit.client.0.smithi092.stdout:2/691: mkdir df/d15/d88/dc9 0 2024-05-06T12:14:43.793 INFO:tasks.workunit.client.1.smithi190.stdout:9/374: dread db/dd/d16/f4f [0,4194304] 0 2024-05-06T12:14:43.795 INFO:tasks.workunit.client.1.smithi190.stdout:1/173: mknod d5/d15/d1e/c36 0 2024-05-06T12:14:43.796 INFO:tasks.workunit.client.0.smithi092.stdout:6/852: getdents d4/d20/d23 0 2024-05-06T12:14:43.796 INFO:tasks.workunit.client.1.smithi190.stdout:9/375: symlink db/dd/d16/d4c/l8e 0 2024-05-06T12:14:43.798 INFO:tasks.workunit.client.1.smithi190.stdout:7/208: dwrite d0/f19 [0,4194304] 0 2024-05-06T12:14:43.799 INFO:tasks.workunit.client.0.smithi092.stdout:6/853: mkdir d4/d4c/d91/d110 0 2024-05-06T12:14:43.799 INFO:tasks.workunit.client.0.smithi092.stdout:2/692: fsync df/d17/d1d/d33/faf 0 2024-05-06T12:14:43.799 INFO:tasks.workunit.client.1.smithi190.stdout:9/376: fsync db/dd/d16/d58/d87/d22/f51 0 2024-05-06T12:14:43.801 INFO:tasks.workunit.client.0.smithi092.stdout:3/631: dwrite d8/db/d4c/d40/d67/f80 [0,4194304] 0 2024-05-06T12:14:43.802 INFO:tasks.workunit.client.1.smithi190.stdout:9/377: readlink db/dd/df/l28 0 2024-05-06T12:14:43.803 INFO:tasks.workunit.client.0.smithi092.stdout:2/693: chown df/d17/d1d/f46 186 1 2024-05-06T12:14:43.803 INFO:tasks.workunit.client.1.smithi190.stdout:1/174: dread d5/d23/d24/f25 [0,4194304] 0 2024-05-06T12:14:43.804 INFO:tasks.workunit.client.1.smithi190.stdout:9/378: rmdir db/d23/d25/d47 39 2024-05-06T12:14:43.806 INFO:tasks.workunit.client.0.smithi092.stdout:3/632: symlink d8/d1b/dbe/lc0 0 2024-05-06T12:14:43.808 INFO:tasks.workunit.client.0.smithi092.stdout:5/772: dwrite d4/d34/d28/d4e/f87 [4194304,4194304] 0 2024-05-06T12:14:43.808 INFO:tasks.workunit.client.1.smithi190.stdout:1/175: link d5/fe d5/d23/d32/f37 0 2024-05-06T12:14:43.812 INFO:tasks.workunit.client.1.smithi190.stdout:1/176: creat d5/d23/d24/f38 x:0 0 0 2024-05-06T12:14:43.813 INFO:tasks.workunit.client.1.smithi190.stdout:1/177: readlink d5/d23/l2e 0 2024-05-06T12:14:43.814 INFO:tasks.workunit.client.1.smithi190.stdout:9/379: dwrite db/dd/d16/d58/d87/f85 [0,4194304] 0 2024-05-06T12:14:43.814 INFO:tasks.workunit.client.0.smithi092.stdout:5/773: unlink d4/d5/d10/d31/ddf/d54/d73/f92 0 2024-05-06T12:14:43.815 INFO:tasks.workunit.client.1.smithi190.stdout:9/380: rename db/dd to db/dd/d16/d4c/d4e/d5e/d8f 22 2024-05-06T12:14:43.817 INFO:tasks.workunit.client.0.smithi092.stdout:5/774: creat d4/d59/f114 x:0 0 0 2024-05-06T12:14:43.817 INFO:tasks.workunit.client.1.smithi190.stdout:9/381: truncate db/d23/f83 4923781 0 2024-05-06T12:14:43.818 INFO:tasks.workunit.client.1.smithi190.stdout:9/382: chown db/d23 56 1 2024-05-06T12:14:43.823 INFO:tasks.workunit.client.1.smithi190.stdout:9/383: mknod db/dd/d16/d58/d87/d22/d49/d88/d76/c90 0 2024-05-06T12:14:43.823 INFO:tasks.workunit.client.1.smithi190.stdout:9/384: fsync db/f1f 0 2024-05-06T12:14:43.827 INFO:tasks.workunit.client.0.smithi092.stdout:3/633: rename d8/d1b/dbe to d8/db/d4c/dc1 0 2024-05-06T12:14:43.827 INFO:tasks.workunit.client.0.smithi092.stdout:2/694: truncate df/f12 3154983 0 2024-05-06T12:14:43.829 INFO:tasks.workunit.client.1.smithi190.stdout:1/178: read d5/fb [201574,6599] 0 2024-05-06T12:14:43.829 INFO:tasks.workunit.client.0.smithi092.stdout:6/854: dwrite d4/d20/d23/d30/f33 [0,4194304] 0 2024-05-06T12:14:43.829 INFO:tasks.workunit.client.0.smithi092.stdout:2/695: mknod df/d5c/d60/cca 0 2024-05-06T12:14:43.830 INFO:tasks.workunit.client.1.smithi190.stdout:1/179: mknod d5/d23/c39 0 2024-05-06T12:14:43.831 INFO:tasks.workunit.client.0.smithi092.stdout:3/634: creat d8/db/d4c/d40/d7a/d90/fc2 x:0 0 0 2024-05-06T12:14:43.833 INFO:tasks.workunit.client.1.smithi190.stdout:1/180: mkdir d5/d23/d24/d3a 0 2024-05-06T12:14:43.833 INFO:tasks.workunit.client.0.smithi092.stdout:2/696: creat df/d17/d1d/d33/d4d/d83/dc2/fcb x:0 0 0 2024-05-06T12:14:43.835 INFO:tasks.workunit.client.0.smithi092.stdout:2/697: mknod df/d15/ccc 0 2024-05-06T12:14:43.835 INFO:tasks.workunit.client.1.smithi190.stdout:1/181: rmdir d5/d23/d32 39 2024-05-06T12:14:43.837 INFO:tasks.workunit.client.0.smithi092.stdout:2/698: rename df/d1b/l31 to df/d17/d53/da9/lcd 0 2024-05-06T12:14:43.838 INFO:tasks.workunit.client.1.smithi190.stdout:1/182: chown d5/f12 5 1 2024-05-06T12:14:43.838 INFO:tasks.workunit.client.0.smithi092.stdout:2/699: chown df/d15/fbe 16 1 2024-05-06T12:14:43.839 INFO:tasks.workunit.client.0.smithi092.stdout:5/775: fdatasync d4/d34/d28/d4e/f87 0 2024-05-06T12:14:43.840 INFO:tasks.workunit.client.0.smithi092.stdout:2/700: chown df/d17/d53/l77 89217283 1 2024-05-06T12:14:43.841 INFO:tasks.workunit.client.0.smithi092.stdout:6/855: dread d4/d20/d22/f7c [0,4194304] 0 2024-05-06T12:14:43.842 INFO:tasks.workunit.client.0.smithi092.stdout:2/701: mkdir df/d1b/db4/dce 0 2024-05-06T12:14:43.843 INFO:tasks.workunit.client.1.smithi190.stdout:1/183: dread d5/f12 [0,4194304] 0 2024-05-06T12:14:43.844 INFO:tasks.workunit.client.1.smithi190.stdout:9/385: dwrite db/dd/d16/f4f [0,4194304] 0 2024-05-06T12:14:43.846 INFO:tasks.workunit.client.1.smithi190.stdout:9/386: fdatasync db/dd/df/f1b 0 2024-05-06T12:14:43.847 INFO:tasks.workunit.client.1.smithi190.stdout:1/184: truncate d5/fe 2230913 0 2024-05-06T12:14:43.849 INFO:tasks.workunit.client.1.smithi190.stdout:9/387: truncate db/d23/d29/f6b 242113 0 2024-05-06T12:14:43.850 INFO:tasks.workunit.client.1.smithi190.stdout:1/185: creat d5/d23/d24/d3a/f3b x:0 0 0 2024-05-06T12:14:43.852 INFO:tasks.workunit.client.1.smithi190.stdout:9/388: write db/dd/d16/d4c/f57 [1622459,53860] 0 2024-05-06T12:14:43.857 INFO:tasks.workunit.client.0.smithi092.stdout:2/702: rename df/d17/d1d/d33 to df/d1b/dcf 0 2024-05-06T12:14:43.858 INFO:tasks.workunit.client.1.smithi190.stdout:1/186: dwrite d5/d23/d24/f38 [0,4194304] 0 2024-05-06T12:14:43.859 INFO:tasks.workunit.client.0.smithi092.stdout:2/703: mkdir df/dd0 0 2024-05-06T12:14:43.860 INFO:tasks.workunit.client.0.smithi092.stdout:4/805: sync 2024-05-06T12:14:43.860 INFO:tasks.workunit.client.0.smithi092.stdout:9/808: sync 2024-05-06T12:14:43.861 INFO:tasks.workunit.client.0.smithi092.stdout:1/777: sync 2024-05-06T12:14:43.861 INFO:tasks.workunit.client.0.smithi092.stdout:8/855: sync 2024-05-06T12:14:43.861 INFO:tasks.workunit.client.1.smithi190.stdout:1/187: creat d5/d23/f3c x:0 0 0 2024-05-06T12:14:43.862 INFO:tasks.workunit.client.0.smithi092.stdout:8/856: dread - d9/d21/d37/dbd/fdc zero size 2024-05-06T12:14:43.864 INFO:tasks.workunit.client.0.smithi092.stdout:1/778: fsync d3/d10/d13/d19/d21/d88/d8b/fe3 0 2024-05-06T12:14:43.865 INFO:tasks.workunit.client.0.smithi092.stdout:4/806: read d9/df/f14 [2873240,51842] 0 2024-05-06T12:14:43.866 INFO:tasks.workunit.client.0.smithi092.stdout:1/779: chown d3/d10/d13/d19 2060 1 2024-05-06T12:14:43.867 INFO:tasks.workunit.client.1.smithi190.stdout:1/188: dwrite d5/d15/f1a [0,4194304] 0 2024-05-06T12:14:43.868 INFO:tasks.workunit.client.1.smithi190.stdout:1/189: fdatasync d5/f2d 0 2024-05-06T12:14:43.868 INFO:tasks.workunit.client.1.smithi190.stdout:1/190: chown d5/fa 1 1 2024-05-06T12:14:43.870 INFO:tasks.workunit.client.0.smithi092.stdout:6/856: dread d4/fb [4194304,4194304] 0 2024-05-06T12:14:43.871 INFO:tasks.workunit.client.0.smithi092.stdout:4/807: symlink d9/dd/da6/dca/df2/l109 0 2024-05-06T12:14:43.873 INFO:tasks.workunit.client.1.smithi190.stdout:4/158: sync 2024-05-06T12:14:43.873 INFO:tasks.workunit.client.0.smithi092.stdout:8/857: link d9/dc/ca1 d9/dc/d4a/d5f/c12a 0 2024-05-06T12:14:43.875 INFO:tasks.workunit.client.0.smithi092.stdout:1/780: dwrite d3/d10/d13/d19/d21/d88/fd7 [0,4194304] 0 2024-05-06T12:14:43.875 INFO:tasks.workunit.client.0.smithi092.stdout:4/808: creat d9/df/d62/d7b/db6/d104/f10a x:0 0 0 2024-05-06T12:14:43.875 INFO:tasks.workunit.client.1.smithi190.stdout:4/159: readlink d2/d11/l2b 0 2024-05-06T12:14:43.877 INFO:tasks.workunit.client.0.smithi092.stdout:6/857: creat d4/f111 x:0 0 0 2024-05-06T12:14:43.879 INFO:tasks.workunit.client.0.smithi092.stdout:5/776: dread d4/d5/f2a [0,4194304] 0 2024-05-06T12:14:43.880 INFO:tasks.workunit.client.0.smithi092.stdout:8/858: creat d9/dc/d4a/d5f/da8/ddf/d115/f12b x:0 0 0 2024-05-06T12:14:43.881 INFO:tasks.workunit.client.1.smithi190.stdout:9/389: write db/dd/d16/f6c [594725,103266] 0 2024-05-06T12:14:43.882 INFO:tasks.workunit.client.0.smithi092.stdout:5/777: dread - d4/d59/f114 zero size 2024-05-06T12:14:43.882 INFO:tasks.workunit.client.0.smithi092.stdout:6/858: mknod d4/d24/d4e/c112 0 2024-05-06T12:14:43.883 INFO:tasks.workunit.client.1.smithi190.stdout:4/160: getdents d2 0 2024-05-06T12:14:43.885 INFO:tasks.workunit.client.0.smithi092.stdout:8/859: write d9/d21/d37/d46/fd2 [1119960,14959] 0 2024-05-06T12:14:43.885 INFO:tasks.workunit.client.0.smithi092.stdout:4/809: dread d9/d2b/d5b/fb5 [0,4194304] 0 2024-05-06T12:14:43.885 INFO:tasks.workunit.client.0.smithi092.stdout:5/778: write d4/d5/d10/da1/fcd [595660,123930] 0 2024-05-06T12:14:43.886 INFO:tasks.workunit.client.0.smithi092.stdout:1/781: getdents d3/d42/d84/d4f/d7f 0 2024-05-06T12:14:43.887 INFO:tasks.workunit.client.1.smithi190.stdout:4/161: symlink d2/l39 0 2024-05-06T12:14:43.888 INFO:tasks.workunit.client.0.smithi092.stdout:3/635: dread d8/db/d4c/d3d/f60 [0,4194304] 0 2024-05-06T12:14:43.888 INFO:tasks.workunit.client.0.smithi092.stdout:4/810: truncate d9/fe0 547436 0 2024-05-06T12:14:43.889 INFO:tasks.workunit.client.1.smithi190.stdout:1/191: write d5/d23/d24/f25 [849743,60290] 0 2024-05-06T12:14:43.889 INFO:tasks.workunit.client.1.smithi190.stdout:9/390: dwrite db/dd/df/f2c [0,4194304] 0 2024-05-06T12:14:43.890 INFO:tasks.workunit.client.0.smithi092.stdout:6/859: truncate d4/d24/d38/d5d/d6d/d7b/ff3 937834 0 2024-05-06T12:14:43.890 INFO:tasks.workunit.client.0.smithi092.stdout:5/779: truncate d4/d5/d10/d31/ddf/da3/fd2 1357181 0 2024-05-06T12:14:43.891 INFO:tasks.workunit.client.0.smithi092.stdout:3/636: chown d8/d24/d27/l9c 436904165 1 2024-05-06T12:14:43.891 INFO:tasks.workunit.client.0.smithi092.stdout:8/860: link d9/dc/fc8 d9/d21/d37/dbd/dc3/f12c 0 2024-05-06T12:14:43.891 INFO:tasks.workunit.client.0.smithi092.stdout:4/811: rmdir d9 39 2024-05-06T12:14:43.892 INFO:tasks.workunit.client.1.smithi190.stdout:9/391: write db/d23/d29/f61 [40938,35092] 0 2024-05-06T12:14:43.892 INFO:tasks.workunit.client.0.smithi092.stdout:1/782: mkdir d3/d10/d13/d19/d21/d88/d8b/dfc 0 2024-05-06T12:14:43.893 INFO:tasks.workunit.client.0.smithi092.stdout:7/904: sync 2024-05-06T12:14:43.893 INFO:tasks.workunit.client.1.smithi190.stdout:1/192: symlink d5/d23/d24/d3a/l3d 0 2024-05-06T12:14:43.896 INFO:tasks.workunit.client.0.smithi092.stdout:8/861: creat d9/dc/f12d x:0 0 0 2024-05-06T12:14:43.898 INFO:tasks.workunit.client.1.smithi190.stdout:9/392: truncate db/dd/f41 6801234 0 2024-05-06T12:14:43.900 INFO:tasks.workunit.client.1.smithi190.stdout:1/193: getdents d5 0 2024-05-06T12:14:43.900 INFO:tasks.workunit.client.0.smithi092.stdout:3/637: mknod d8/db/d14/d5a/db0/cc3 0 2024-05-06T12:14:43.901 INFO:tasks.workunit.client.0.smithi092.stdout:1/783: write d3/d10/d13/f60 [1034726,129590] 0 2024-05-06T12:14:43.901 INFO:tasks.workunit.client.0.smithi092.stdout:4/812: dread d9/df/fb0 [0,4194304] 0 2024-05-06T12:14:43.903 INFO:tasks.workunit.client.0.smithi092.stdout:1/784: readlink d3/d42/d84/d4f/l5c 0 2024-05-06T12:14:43.903 INFO:tasks.workunit.client.1.smithi190.stdout:9/393: truncate db/dd/d16/d58/d87/f4a 1025693 0 2024-05-06T12:14:43.904 INFO:tasks.workunit.client.1.smithi190.stdout:9/394: write db/dd/d16/f6c [1097015,12632] 0 2024-05-06T12:14:43.905 INFO:tasks.workunit.client.1.smithi190.stdout:1/194: rename d5/d23/d24/f38 to d5/d23/d24/d3a/f3e 0 2024-05-06T12:14:43.905 INFO:tasks.workunit.client.0.smithi092.stdout:4/813: creat d9/dd/da6/f10b x:0 0 0 2024-05-06T12:14:43.907 INFO:tasks.workunit.client.0.smithi092.stdout:9/809: write d16/d2c/d2f/d3d/d33/d57/f82 [1390927,87354] 0 2024-05-06T12:14:43.909 INFO:tasks.workunit.client.0.smithi092.stdout:4/814: read d9/fd5 [6870804,106632] 0 2024-05-06T12:14:43.910 INFO:tasks.workunit.client.1.smithi190.stdout:1/195: rmdir d5 39 2024-05-06T12:14:43.911 INFO:tasks.workunit.client.1.smithi190.stdout:9/395: creat db/dd/d16/d58/f91 x:0 0 0 2024-05-06T12:14:43.912 INFO:tasks.workunit.client.0.smithi092.stdout:4/815: chown d9/dd/da6/ce2 22178569 1 2024-05-06T12:14:43.913 INFO:tasks.workunit.client.0.smithi092.stdout:1/785: dwrite d3/d10/d13/d19/d21/d88/d8b/fc1 [0,4194304] 0 2024-05-06T12:14:43.916 INFO:tasks.workunit.client.1.smithi190.stdout:4/162: dread d2/f25 [0,4194304] 0 2024-05-06T12:14:43.918 INFO:tasks.workunit.client.0.smithi092.stdout:5/780: write d4/d5/d10/d31/ddf/da3/fa9 [114108,4371] 0 2024-05-06T12:14:43.918 INFO:tasks.workunit.client.1.smithi190.stdout:1/196: creat d5/d15/f3f x:0 0 0 2024-05-06T12:14:43.920 INFO:tasks.workunit.client.0.smithi092.stdout:6/860: write d4/d24/d26/fa4 [1641721,125186] 0 2024-05-06T12:14:43.921 INFO:tasks.workunit.client.1.smithi190.stdout:4/163: stat d2/d11/d16/l34 0 2024-05-06T12:14:43.922 INFO:tasks.workunit.client.0.smithi092.stdout:1/786: mknod d3/d10/d13/d19/d21/d72/d9a/cfd 0 2024-05-06T12:14:43.923 INFO:tasks.workunit.client.1.smithi190.stdout:9/396: dwrite db/dd/d16/d58/d87/d22/d49/d88/f7a [0,4194304] 0 2024-05-06T12:14:43.924 INFO:tasks.workunit.client.0.smithi092.stdout:4/816: symlink d9/dd/da6/dca/df2/l10c 0 2024-05-06T12:14:43.925 INFO:tasks.workunit.client.1.smithi190.stdout:4/164: mknod d2/d10/c3a 0 2024-05-06T12:14:43.927 INFO:tasks.workunit.client.1.smithi190.stdout:9/397: creat db/dd/d16/d4c/d4e/d5e/f92 x:0 0 0 2024-05-06T12:14:43.929 INFO:tasks.workunit.client.0.smithi092.stdout:2/704: dwrite df/d17/d1d/fb6 [0,4194304] 0 2024-05-06T12:14:43.929 INFO:tasks.workunit.client.0.smithi092.stdout:9/810: dwrite d16/d2c/d2f/d75/ff6 [0,4194304] 0 2024-05-06T12:14:43.932 INFO:tasks.workunit.client.0.smithi092.stdout:3/638: dread f3 [0,4194304] 0 2024-05-06T12:14:43.933 INFO:tasks.workunit.client.1.smithi190.stdout:4/165: dwrite d2/f37 [0,4194304] 0 2024-05-06T12:14:43.934 INFO:tasks.workunit.client.0.smithi092.stdout:5/781: mknod d4/d34/d28/d4e/d5d/d82/c115 0 2024-05-06T12:14:43.935 INFO:tasks.workunit.client.1.smithi190.stdout:1/197: stat d5/fe 0 2024-05-06T12:14:43.936 INFO:tasks.workunit.client.0.smithi092.stdout:8/862: dwrite d9/d21/d37/ff6 [0,4194304] 0 2024-05-06T12:14:43.937 INFO:tasks.workunit.client.0.smithi092.stdout:4/817: mkdir d9/dd/da6/dca/df2/d10d 0 2024-05-06T12:14:43.937 INFO:tasks.workunit.client.1.smithi190.stdout:1/198: chown d5/d15/f2a 39 1 2024-05-06T12:14:43.938 INFO:tasks.workunit.client.0.smithi092.stdout:6/861: dread d4/d24/d38/faa [0,4194304] 0 2024-05-06T12:14:43.939 INFO:tasks.workunit.client.0.smithi092.stdout:4/818: chown d9/d2b/f7d 21377900 1 2024-05-06T12:14:43.939 INFO:tasks.workunit.client.1.smithi190.stdout:4/166: rename d2/d11/l2b to d2/d11/l3b 0 2024-05-06T12:14:43.941 INFO:tasks.workunit.client.1.smithi190.stdout:1/199: fdatasync d5/d23/d24/d3a/f3e 0 2024-05-06T12:14:43.943 INFO:tasks.workunit.client.0.smithi092.stdout:8/863: unlink d9/d21/d54/d79/d8a/c112 0 2024-05-06T12:14:43.944 INFO:tasks.workunit.client.1.smithi190.stdout:1/200: chown d5/d15/f3f 172235110 1 2024-05-06T12:14:43.948 INFO:tasks.workunit.client.0.smithi092.stdout:4/819: creat d9/dd/d90/f10e x:0 0 0 2024-05-06T12:14:43.951 INFO:tasks.workunit.client.0.smithi092.stdout:3/639: rename d8/d24/d27/f4a to d8/d24/d33/d39/d5b/fc4 0 2024-05-06T12:14:43.955 INFO:tasks.workunit.client.1.smithi190.stdout:9/398: dread db/d23/d25/f37 [0,4194304] 0 2024-05-06T12:14:43.955 INFO:tasks.workunit.client.1.smithi190.stdout:1/201: dread d5/d23/d24/f25 [0,4194304] 0 2024-05-06T12:14:43.956 INFO:tasks.workunit.client.1.smithi190.stdout:9/399: chown db/dd/d16/d58/d87/d22/c40 0 1 2024-05-06T12:14:43.957 INFO:tasks.workunit.client.0.smithi092.stdout:3/640: read d8/d24/d33/d39/d5b/f6d [293735,32497] 0 2024-05-06T12:14:43.957 INFO:tasks.workunit.client.1.smithi190.stdout:1/202: mknod d5/d15/d1e/c40 0 2024-05-06T12:14:43.957 INFO:tasks.workunit.client.0.smithi092.stdout:6/862: dwrite d4/d24/d66/f7e [0,4194304] 0 2024-05-06T12:14:43.958 INFO:tasks.workunit.client.0.smithi092.stdout:9/811: dwrite d16/d2c/d2f/d3d/d33/d6b/d9e/fba [4194304,4194304] 0 2024-05-06T12:14:43.959 INFO:tasks.workunit.client.1.smithi190.stdout:1/203: symlink d5/d15/d1e/l41 0 2024-05-06T12:14:43.961 INFO:tasks.workunit.client.1.smithi190.stdout:1/204: dread d5/d23/d24/f25 [0,4194304] 0 2024-05-06T12:14:43.966 INFO:tasks.workunit.client.1.smithi190.stdout:1/205: link d5/d15/c1c d5/d23/c42 0 2024-05-06T12:14:43.967 INFO:tasks.workunit.client.0.smithi092.stdout:8/864: dwrite d9/d21/d37/d74/df9/f114 [0,4194304] 0 2024-05-06T12:14:43.970 INFO:tasks.workunit.client.0.smithi092.stdout:9/812: truncate d16/d2c/d3f/f46 438051 0 2024-05-06T12:14:43.974 INFO:tasks.workunit.client.0.smithi092.stdout:3/641: truncate d8/db/d4c/d40/d7a/fa3 4784498 0 2024-05-06T12:14:43.977 INFO:tasks.workunit.client.0.smithi092.stdout:9/813: write d16/d1c/d58/d92/fa8 [1454321,127174] 0 2024-05-06T12:14:43.980 INFO:tasks.workunit.client.0.smithi092.stdout:5/782: dwrite d4/d5/d30/f61 [0,4194304] 0 2024-05-06T12:14:43.981 INFO:tasks.workunit.client.0.smithi092.stdout:1/787: dwrite d3/d10/d13/d19/d21/d88/f91 [0,4194304] 0 2024-05-06T12:14:43.982 INFO:tasks.workunit.client.0.smithi092.stdout:3/642: creat d8/d24/d33/d4b/fc5 x:0 0 0 2024-05-06T12:14:43.986 INFO:tasks.workunit.client.0.smithi092.stdout:9/814: mkdir d16/d94/da0/da7/d10f 0 2024-05-06T12:14:43.986 INFO:tasks.workunit.client.0.smithi092.stdout:6/863: dwrite d4/d24/d26/f6e [0,4194304] 0 2024-05-06T12:14:43.988 INFO:tasks.workunit.client.0.smithi092.stdout:4/820: dread d9/d27/d4a/f4c [0,4194304] 0 2024-05-06T12:14:43.989 INFO:tasks.workunit.client.1.smithi190.stdout:9/400: dread db/f1f [0,4194304] 0 2024-05-06T12:14:43.989 INFO:tasks.workunit.client.0.smithi092.stdout:4/821: dread - d9/d2b/d7c/dab/dd8/ff7 zero size 2024-05-06T12:14:43.991 INFO:tasks.workunit.client.0.smithi092.stdout:5/783: fsync d4/d5/d10/d31/ddf/da3/fbf 0 2024-05-06T12:14:43.992 INFO:tasks.workunit.client.0.smithi092.stdout:5/784: fsync d4/d34/d28/d4e/d67/f79 0 2024-05-06T12:14:43.993 INFO:tasks.workunit.client.1.smithi190.stdout:9/401: symlink db/d23/d25/l93 0 2024-05-06T12:14:43.997 INFO:tasks.workunit.client.1.smithi190.stdout:9/402: symlink db/dd/d16/d4c/d4e/l94 0 2024-05-06T12:14:43.998 INFO:tasks.workunit.client.0.smithi092.stdout:9/815: rename c4 to d16/d94/da0/da7/d10f/c110 0 2024-05-06T12:14:44.000 INFO:tasks.workunit.client.1.smithi190.stdout:9/403: creat db/d23/d25/f95 x:0 0 0 2024-05-06T12:14:44.000 INFO:tasks.workunit.client.0.smithi092.stdout:5/785: dread d4/d5/f96 [0,4194304] 0 2024-05-06T12:14:44.001 INFO:tasks.workunit.client.0.smithi092.stdout:4/822: dread d9/fc0 [0,4194304] 0 2024-05-06T12:14:44.003 INFO:tasks.workunit.client.1.smithi190.stdout:9/404: dread db/f52 [0,4194304] 0 2024-05-06T12:14:44.005 INFO:tasks.workunit.client.1.smithi190.stdout:9/405: symlink db/dd/d16/d58/d87/d22/d49/d88/d76/l96 0 2024-05-06T12:14:44.007 INFO:tasks.workunit.client.1.smithi190.stdout:9/406: rename db/dd/df/c12 to db/dd/d16/d58/d87/c97 0 2024-05-06T12:14:44.008 INFO:tasks.workunit.client.1.smithi190.stdout:9/407: dread - db/d23/d25/f8b zero size 2024-05-06T12:14:44.010 INFO:tasks.workunit.client.0.smithi092.stdout:6/864: link d4/d20/l2f d4/d24/d26/dae/l113 0 2024-05-06T12:14:44.010 INFO:tasks.workunit.client.0.smithi092.stdout:5/786: chown d4/d34/d28/d4e/d5d/d82/f63 1404 1 2024-05-06T12:14:44.014 INFO:tasks.workunit.client.0.smithi092.stdout:9/816: dread d16/d2c/d2f/d3d/d33/d57/f82 [0,4194304] 0 2024-05-06T12:14:44.016 INFO:tasks.workunit.client.1.smithi190.stdout:9/408: write db/d23/f53 [742513,52444] 0 2024-05-06T12:14:44.017 INFO:tasks.workunit.client.0.smithi092.stdout:1/788: dwrite d3/d10/d13/d19/f4e [0,4194304] 0 2024-05-06T12:14:44.018 INFO:tasks.workunit.client.1.smithi190.stdout:9/409: write db/dd/d16/d58/d87/d22/d49/d88/f86 [235380,5877] 0 2024-05-06T12:14:44.019 INFO:tasks.workunit.client.0.smithi092.stdout:4/823: dwrite d9/f1c [4194304,4194304] 0 2024-05-06T12:14:44.020 INFO:tasks.workunit.client.1.smithi190.stdout:9/410: creat db/dd/d16/d58/d87/d22/d49/f98 x:0 0 0 2024-05-06T12:14:44.022 INFO:tasks.workunit.client.1.smithi190.stdout:9/411: mkdir db/dd/d16/d58/d87/d99 0 2024-05-06T12:14:44.023 INFO:tasks.workunit.client.0.smithi092.stdout:6/865: dwrite d4/d4c/d52/f70 [4194304,4194304] 0 2024-05-06T12:14:44.027 INFO:tasks.workunit.client.0.smithi092.stdout:5/787: rename d4/d34/d28/d4e/d5d/d82/fe9 to d4/d5/d30/dfd/d108/f116 0 2024-05-06T12:14:44.027 INFO:tasks.workunit.client.0.smithi092.stdout:4/824: mkdir d9/df/d62/d10f 0 2024-05-06T12:14:44.028 INFO:tasks.workunit.client.1.smithi190.stdout:9/412: fsync db/dd/d16/d58/d87/d22/d49/f98 0 2024-05-06T12:14:44.029 INFO:tasks.workunit.client.0.smithi092.stdout:1/789: symlink d3/d10/d13/d19/d21/d72/d9a/lfe 0 2024-05-06T12:14:44.029 INFO:tasks.workunit.client.0.smithi092.stdout:6/866: mknod d4/d4c/d91/c114 0 2024-05-06T12:14:44.032 INFO:tasks.workunit.client.1.smithi190.stdout:9/413: symlink db/dd/l9a 0 2024-05-06T12:14:44.036 INFO:tasks.workunit.client.1.smithi190.stdout:9/414: creat db/dd/d16/d4c/f9b x:0 0 0 2024-05-06T12:14:44.037 INFO:tasks.workunit.client.1.smithi190.stdout:9/415: fsync db/d23/f53 0 2024-05-06T12:14:44.039 INFO:tasks.workunit.client.1.smithi190.stdout:9/416: write db/f3d [4585364,30347] 0 2024-05-06T12:14:44.040 INFO:tasks.workunit.client.1.smithi190.stdout:9/417: truncate db/f5a 1048464 0 2024-05-06T12:14:44.041 INFO:tasks.workunit.client.1.smithi190.stdout:9/418: stat c9 0 2024-05-06T12:14:44.046 INFO:tasks.workunit.client.1.smithi190.stdout:5/226: sync 2024-05-06T12:14:44.046 INFO:tasks.workunit.client.1.smithi190.stdout:8/295: sync 2024-05-06T12:14:44.046 INFO:tasks.workunit.client.1.smithi190.stdout:2/205: sync 2024-05-06T12:14:44.046 INFO:tasks.workunit.client.1.smithi190.stdout:6/193: sync 2024-05-06T12:14:44.047 INFO:tasks.workunit.client.1.smithi190.stdout:6/194: chown l8 10697633 1 2024-05-06T12:14:44.049 INFO:tasks.workunit.client.1.smithi190.stdout:0/375: sync 2024-05-06T12:14:44.049 INFO:tasks.workunit.client.1.smithi190.stdout:9/419: dwrite db/dd/d16/d58/d87/d22/d49/f98 [0,4194304] 0 2024-05-06T12:14:44.050 INFO:tasks.workunit.client.1.smithi190.stdout:2/206: creat d2/d21/f3d x:0 0 0 2024-05-06T12:14:44.052 INFO:tasks.workunit.client.1.smithi190.stdout:2/207: chown d2/d8/d33/f2b 369558 1 2024-05-06T12:14:44.053 INFO:tasks.workunit.client.1.smithi190.stdout:8/296: fdatasync d3/d8/d25/f5b 0 2024-05-06T12:14:44.054 INFO:tasks.workunit.client.1.smithi190.stdout:5/227: creat d2/da/d2c/d2e/d3b/f40 x:0 0 0 2024-05-06T12:14:44.056 INFO:tasks.workunit.client.1.smithi190.stdout:6/195: creat df/d17/d37/f42 x:0 0 0 2024-05-06T12:14:44.056 INFO:tasks.workunit.client.1.smithi190.stdout:6/196: chown f2 1 1 2024-05-06T12:14:44.058 INFO:tasks.workunit.client.1.smithi190.stdout:9/420: creat db/dd/d16/d58/d87/d22/d49/d88/f9c x:0 0 0 2024-05-06T12:14:44.060 INFO:tasks.workunit.client.1.smithi190.stdout:5/228: fdatasync d2/f26 0 2024-05-06T12:14:44.061 INFO:tasks.workunit.client.1.smithi190.stdout:0/376: mknod d9/c82 0 2024-05-06T12:14:44.063 INFO:tasks.workunit.client.1.smithi190.stdout:9/421: write db/dd/f34 [1186750,61336] 0 2024-05-06T12:14:44.064 INFO:tasks.workunit.client.1.smithi190.stdout:8/297: mknod d3/d8/c5d 0 2024-05-06T12:14:44.066 INFO:tasks.workunit.client.1.smithi190.stdout:0/377: truncate d9/df/f62 1759280 0 2024-05-06T12:14:44.067 INFO:tasks.workunit.client.1.smithi190.stdout:2/208: dwrite d2/f38 [0,4194304] 0 2024-05-06T12:14:44.067 INFO:tasks.workunit.client.1.smithi190.stdout:9/422: dread - db/dd/d16/d58/d87/d22/f55 zero size 2024-05-06T12:14:44.068 INFO:tasks.workunit.client.1.smithi190.stdout:0/378: fsync d9/f2f 0 2024-05-06T12:14:44.069 INFO:tasks.workunit.client.1.smithi190.stdout:8/298: chown d3/d8/d2b/d3e 111 1 2024-05-06T12:14:44.070 INFO:tasks.workunit.client.1.smithi190.stdout:2/209: read - d2/f1f zero size 2024-05-06T12:14:44.070 INFO:tasks.workunit.client.1.smithi190.stdout:5/229: rename d2/da/l3e to d2/da/d2c/d2e/d3b/l41 0 2024-05-06T12:14:44.071 INFO:tasks.workunit.client.1.smithi190.stdout:0/379: chown d9/dc/d28/d4c/c5a 1 1 2024-05-06T12:14:44.071 INFO:tasks.workunit.client.1.smithi190.stdout:2/210: chown d2/d10/f2c 25347698 1 2024-05-06T12:14:44.071 INFO:tasks.workunit.client.1.smithi190.stdout:5/230: stat d2/da/d2c/f3c 0 2024-05-06T12:14:44.073 INFO:tasks.workunit.client.1.smithi190.stdout:5/231: chown d2/da/d2c/f3c 52405695 1 2024-05-06T12:14:44.074 INFO:tasks.workunit.client.1.smithi190.stdout:8/299: rename d3/c9 to d3/d35/d55/c5e 0 2024-05-06T12:14:44.076 INFO:tasks.workunit.client.1.smithi190.stdout:2/211: unlink d2/d21/l3a 0 2024-05-06T12:14:44.076 INFO:tasks.workunit.client.1.smithi190.stdout:2/212: dread - d2/d10/f2c zero size 2024-05-06T12:14:44.077 INFO:tasks.workunit.client.1.smithi190.stdout:2/213: write d2/f1f [332580,60237] 0 2024-05-06T12:14:44.077 INFO:tasks.workunit.client.1.smithi190.stdout:0/380: mkdir d9/dc/d48/d50/d83 0 2024-05-06T12:14:44.080 INFO:tasks.workunit.client.1.smithi190.stdout:5/232: creat d2/da/d2c/d2e/d3b/f42 x:0 0 0 2024-05-06T12:14:44.085 INFO:tasks.workunit.client.1.smithi190.stdout:9/423: truncate db/d23/d25/d47/d4b/f43 2997957 0 2024-05-06T12:14:44.086 INFO:tasks.workunit.client.0.smithi092.stdout:1/790: write d3/d42/d84/d4f/fa8 [979209,99157] 0 2024-05-06T12:14:44.086 INFO:tasks.workunit.client.0.smithi092.stdout:1/791: readlink d3/d10/d13/d19/d21/l28 0 2024-05-06T12:14:44.087 INFO:tasks.workunit.client.0.smithi092.stdout:9/817: rename d16/d2c/d2f/f45 to d16/d1e/d7b/d8d/f111 0 2024-05-06T12:14:44.087 INFO:tasks.workunit.client.1.smithi190.stdout:0/381: rename l8 to d9/d6f/l84 0 2024-05-06T12:14:44.088 INFO:tasks.workunit.client.0.smithi092.stdout:1/792: fdatasync d3/d10/d13/d19/d34/ff5 0 2024-05-06T12:14:44.088 INFO:tasks.workunit.client.0.smithi092.stdout:4/825: mknod d9/df/d48/c110 0 2024-05-06T12:14:44.089 INFO:tasks.workunit.client.1.smithi190.stdout:0/382: write d9/df/d21/f7e [410041,97741] 0 2024-05-06T12:14:44.089 INFO:tasks.workunit.client.0.smithi092.stdout:6/867: symlink d4/d20/d22/l115 0 2024-05-06T12:14:44.089 INFO:tasks.workunit.client.1.smithi190.stdout:2/214: rmdir d2 39 2024-05-06T12:14:44.091 INFO:tasks.workunit.client.0.smithi092.stdout:9/818: chown d16/d2c/d2f/d3d/d33/d37/f6f 4210413 1 2024-05-06T12:14:44.091 INFO:tasks.workunit.client.0.smithi092.stdout:5/788: creat d4/d5/d10/d31/d53/ddb/d104/f117 x:0 0 0 2024-05-06T12:14:44.092 INFO:tasks.workunit.client.1.smithi190.stdout:8/300: rename d3/d8/d25/f4b to d3/d13/d4a/f5f 0 2024-05-06T12:14:44.093 INFO:tasks.workunit.client.0.smithi092.stdout:1/793: creat d3/d42/d84/d4f/dd5/ded/fff x:0 0 0 2024-05-06T12:14:44.093 INFO:tasks.workunit.client.1.smithi190.stdout:5/233: creat d2/da/d2c/f43 x:0 0 0 2024-05-06T12:14:44.094 INFO:tasks.workunit.client.0.smithi092.stdout:6/868: rename d4/lf to d4/d4c/da9/l116 0 2024-05-06T12:14:44.095 INFO:tasks.workunit.client.0.smithi092.stdout:4/826: mkdir d9/d27/dc5/d111 0 2024-05-06T12:14:44.098 INFO:tasks.workunit.client.0.smithi092.stdout:1/794: rmdir d3/d10/d13/d19/d21 39 2024-05-06T12:14:44.100 INFO:tasks.workunit.client.0.smithi092.stdout:9/819: rename d16/d2c/d2f/d3d/d33/f66 to d16/d2c/d2f/d3d/d33/d6b/d9e/f112 0 2024-05-06T12:14:44.100 INFO:tasks.workunit.client.0.smithi092.stdout:5/789: symlink d4/d34/d28/df6/l118 0 2024-05-06T12:14:44.101 INFO:tasks.workunit.client.1.smithi190.stdout:9/424: rename db/dd/d16/d58/d87/f4a to db/dd/d16/d4c/d4e/f9d 0 2024-05-06T12:14:44.102 INFO:tasks.workunit.client.1.smithi190.stdout:2/215: dwrite d2/f38 [0,4194304] 0 2024-05-06T12:14:44.103 INFO:tasks.workunit.client.1.smithi190.stdout:5/234: creat d2/f44 x:0 0 0 2024-05-06T12:14:44.105 INFO:tasks.workunit.client.1.smithi190.stdout:9/425: dread db/d23/d25/d47/d4b/f66 [0,4194304] 0 2024-05-06T12:14:44.105 INFO:tasks.workunit.client.1.smithi190.stdout:5/235: write d2/f3 [1678920,120084] 0 2024-05-06T12:14:44.110 INFO:tasks.workunit.client.1.smithi190.stdout:2/216: chown d2/d8/dc/c2f 108759445 1 2024-05-06T12:14:44.111 INFO:tasks.workunit.client.1.smithi190.stdout:0/383: dread d9/df/d21/d31/f7b [0,4194304] 0 2024-05-06T12:14:44.112 INFO:tasks.workunit.client.1.smithi190.stdout:2/217: fsync d2/f3b 0 2024-05-06T12:14:44.113 INFO:tasks.workunit.client.0.smithi092.stdout:9/820: rename d16/d1c/d58/d92/dda to d16/d1c/d58/d92/dab/deb/d113 0 2024-05-06T12:14:44.114 INFO:tasks.workunit.client.0.smithi092.stdout:4/827: link d9/df/d62/dbb/fc9 d9/df/d62/d7b/f112 0 2024-05-06T12:14:44.114 INFO:tasks.workunit.client.0.smithi092.stdout:1/795: mknod d3/d10/d13/c100 0 2024-05-06T12:14:44.114 INFO:tasks.workunit.client.1.smithi190.stdout:8/301: dread d3/d35/d3a/f3d [0,4194304] 0 2024-05-06T12:14:44.115 INFO:tasks.workunit.client.1.smithi190.stdout:9/426: rename f3 to db/d23/d25/d47/d4b/f9e 0 2024-05-06T12:14:44.117 INFO:tasks.workunit.client.1.smithi190.stdout:5/236: dwrite d2/da/f1b [0,4194304] 0 2024-05-06T12:14:44.118 INFO:tasks.workunit.client.0.smithi092.stdout:6/869: truncate d4/d4c/d52/f70 749403 0 2024-05-06T12:14:44.118 INFO:tasks.workunit.client.0.smithi092.stdout:5/790: dwrite d4/d5/f2a [4194304,4194304] 0 2024-05-06T12:14:44.119 INFO:tasks.workunit.client.1.smithi190.stdout:9/427: fdatasync db/dd/d16/d58/d87/d22/d49/f89 0 2024-05-06T12:14:44.119 INFO:tasks.workunit.client.1.smithi190.stdout:5/237: write d2/f3d [332961,108869] 0 2024-05-06T12:14:44.119 INFO:tasks.workunit.client.1.smithi190.stdout:0/384: rename d9/dc/f4d to d9/dc/d48/d50/f85 0 2024-05-06T12:14:44.123 INFO:tasks.workunit.client.1.smithi190.stdout:0/385: dread d9/fb [4194304,4194304] 0 2024-05-06T12:14:44.124 INFO:tasks.workunit.client.1.smithi190.stdout:5/238: symlink d2/da/l45 0 2024-05-06T12:14:44.125 INFO:tasks.workunit.client.1.smithi190.stdout:2/218: getdents d2/d10 0 2024-05-06T12:14:44.125 INFO:tasks.workunit.client.1.smithi190.stdout:5/239: write d2/f8 [1413485,8272] 0 2024-05-06T12:14:44.127 INFO:tasks.workunit.client.1.smithi190.stdout:9/428: mkdir db/dd/d16/d58/d87/d22/d49/d88/d9f 0 2024-05-06T12:14:44.128 INFO:tasks.workunit.client.0.smithi092.stdout:5/791: dwrite d4/d5/d30/f3c [0,4194304] 0 2024-05-06T12:14:44.129 INFO:tasks.workunit.client.1.smithi190.stdout:8/302: mknod d3/d8/d25/d3b/d49/c60 0 2024-05-06T12:14:44.130 INFO:tasks.workunit.client.0.smithi092.stdout:5/792: write d4/d34/d28/d4e/fc6 [301408,121441] 0 2024-05-06T12:14:44.131 INFO:tasks.workunit.client.0.smithi092.stdout:5/793: chown d4/d5/d10/d31/ddf/d6c/l50 154474 1 2024-05-06T12:14:44.132 INFO:tasks.workunit.client.0.smithi092.stdout:5/794: chown d4 842730159 1 2024-05-06T12:14:44.135 INFO:tasks.workunit.client.1.smithi190.stdout:5/240: dwrite d2/f3 [0,4194304] 0 2024-05-06T12:14:44.138 INFO:tasks.workunit.client.1.smithi190.stdout:2/219: symlink d2/d10/l3e 0 2024-05-06T12:14:44.138 INFO:tasks.workunit.client.0.smithi092.stdout:5/795: dwrite d4/d59/fd1 [4194304,4194304] 0 2024-05-06T12:14:44.139 INFO:tasks.workunit.client.1.smithi190.stdout:5/241: stat d2/da/d2c/d2e/c3f 0 2024-05-06T12:14:44.139 INFO:tasks.workunit.client.1.smithi190.stdout:5/242: fdatasync d2/da/d2c/d2e/d3b/f42 0 2024-05-06T12:14:44.142 INFO:tasks.workunit.client.1.smithi190.stdout:8/303: mkdir d3/d35/d55/d61 0 2024-05-06T12:14:44.145 INFO:tasks.workunit.client.1.smithi190.stdout:8/304: write d3/f26 [638427,105952] 0 2024-05-06T12:14:44.147 INFO:tasks.workunit.client.1.smithi190.stdout:2/220: symlink d2/d10/l3f 0 2024-05-06T12:14:44.148 INFO:tasks.workunit.client.1.smithi190.stdout:5/243: unlink d2/l6 0 2024-05-06T12:14:44.151 INFO:tasks.workunit.client.1.smithi190.stdout:8/305: mknod d3/d35/d55/d61/c62 0 2024-05-06T12:14:44.151 INFO:tasks.workunit.client.1.smithi190.stdout:5/244: symlink d2/da/d2c/d2e/l46 0 2024-05-06T12:14:44.153 INFO:tasks.workunit.client.1.smithi190.stdout:2/221: read d2/d8/dc/f13 [4014395,118287] 0 2024-05-06T12:14:44.156 INFO:tasks.workunit.client.1.smithi190.stdout:8/306: creat d3/d35/d55/d61/f63 x:0 0 0 2024-05-06T12:14:44.157 INFO:tasks.workunit.client.0.smithi092.stdout:9/821: mkdir d16/d1c/dd4/d114 0 2024-05-06T12:14:44.157 INFO:tasks.workunit.client.1.smithi190.stdout:2/222: creat d2/d8/dc/d20/f40 x:0 0 0 2024-05-06T12:14:44.158 INFO:tasks.workunit.client.1.smithi190.stdout:2/223: readlink d2/d10/l3e 0 2024-05-06T12:14:44.159 INFO:tasks.workunit.client.1.smithi190.stdout:2/224: readlink d2/d10/l3e 0 2024-05-06T12:14:44.159 INFO:tasks.workunit.client.1.smithi190.stdout:5/245: dwrite d2/d27/f20 [4194304,4194304] 0 2024-05-06T12:14:44.160 INFO:tasks.workunit.client.1.smithi190.stdout:2/225: truncate d2/f38 5060579 0 2024-05-06T12:14:44.161 INFO:tasks.workunit.client.1.smithi190.stdout:5/246: truncate d2/da/f1b 4224163 0 2024-05-06T12:14:44.162 INFO:tasks.workunit.client.1.smithi190.stdout:2/226: dread - d2/d8/dc/f1a zero size 2024-05-06T12:14:44.162 INFO:tasks.workunit.client.1.smithi190.stdout:5/247: truncate d2/f44 933334 0 2024-05-06T12:14:44.166 INFO:tasks.workunit.client.0.smithi092.stdout:6/870: rename d4/d24/db0/ffe to d4/d4c/d91/f117 0 2024-05-06T12:14:44.166 INFO:tasks.workunit.client.1.smithi190.stdout:8/307: link d3/l5 d3/d8/d2b/l64 0 2024-05-06T12:14:44.166 INFO:tasks.workunit.client.1.smithi190.stdout:0/386: dread d9/df/d21/d31/f67 [0,4194304] 0 2024-05-06T12:14:44.167 INFO:tasks.workunit.client.1.smithi190.stdout:8/308: mkdir d3/d8/d25/d65 0 2024-05-06T12:14:44.167 INFO:tasks.workunit.client.1.smithi190.stdout:0/387: fsync d9/df/d21/f4f 0 2024-05-06T12:14:44.167 INFO:tasks.workunit.client.0.smithi092.stdout:9/822: unlink d16/d2c/d2f/db1/lee 0 2024-05-06T12:14:44.167 INFO:tasks.workunit.client.1.smithi190.stdout:2/227: read d2/fa [354187,99456] 0 2024-05-06T12:14:44.168 INFO:tasks.workunit.client.0.smithi092.stdout:1/796: dwrite d3/d10/d13/d19/d34/fbe [0,4194304] 0 2024-05-06T12:14:44.168 INFO:tasks.workunit.client.1.smithi190.stdout:8/309: read d3/d13/d4a/f4f [563012,90077] 0 2024-05-06T12:14:44.168 INFO:tasks.workunit.client.1.smithi190.stdout:3/258: sync 2024-05-06T12:14:44.169 INFO:tasks.workunit.client.1.smithi190.stdout:5/248: link d2/f31 d2/da/d2c/d2e/d3b/f47 0 2024-05-06T12:14:44.170 INFO:tasks.workunit.client.0.smithi092.stdout:4/828: link d9/d2b/d5b/daf/cef d9/df/d62/d7b/c113 0 2024-05-06T12:14:44.171 INFO:tasks.workunit.client.1.smithi190.stdout:3/259: chown d5/d14/d11/d2a/d3d/f5a 372433464 1 2024-05-06T12:14:44.172 INFO:tasks.workunit.client.1.smithi190.stdout:9/429: write db/dd/d16/d58/d87/d22/f56 [4564313,49414] 0 2024-05-06T12:14:44.173 INFO:tasks.workunit.client.1.smithi190.stdout:8/310: creat d3/d8/d25/f66 x:0 0 0 2024-05-06T12:14:44.173 INFO:tasks.workunit.client.1.smithi190.stdout:9/430: write db/f77 [1026384,29225] 0 2024-05-06T12:14:44.175 INFO:tasks.workunit.client.1.smithi190.stdout:9/431: dread - db/d23/f68 zero size 2024-05-06T12:14:44.175 INFO:tasks.workunit.client.1.smithi190.stdout:5/249: creat d2/d27/f48 x:0 0 0 2024-05-06T12:14:44.176 INFO:tasks.workunit.client.0.smithi092.stdout:5/796: rename d4/d34/d28/d4e/d5d/c10d to d4/d34/d28/d4e/d5d/d82/c119 0 2024-05-06T12:14:44.178 INFO:tasks.workunit.client.0.smithi092.stdout:6/871: dread d4/d24/d26/f32 [0,4194304] 0 2024-05-06T12:14:44.178 INFO:tasks.workunit.client.1.smithi190.stdout:9/432: fdatasync db/dd/d16/d58/d87/d22/d49/f89 0 2024-05-06T12:14:44.179 INFO:tasks.workunit.client.1.smithi190.stdout:2/228: link d2/f35 d2/d10/f41 0 2024-05-06T12:14:44.179 INFO:tasks.workunit.client.0.smithi092.stdout:9/823: dwrite d16/d1e/d7b/dce/f9b [0,4194304] 0 2024-05-06T12:14:44.180 INFO:tasks.workunit.client.0.smithi092.stdout:6/872: chown d4/d24/d66/f7e 1169 1 2024-05-06T12:14:44.181 INFO:tasks.workunit.client.0.smithi092.stdout:5/797: stat d4/d34/d28/d4e/d5d/d82/l106 0 2024-05-06T12:14:44.182 INFO:tasks.workunit.client.1.smithi190.stdout:3/260: mknod d5/d14/d11/c5d 0 2024-05-06T12:14:44.183 INFO:tasks.workunit.client.0.smithi092.stdout:9/824: write d16/d1e/d7b/fb0 [903095,50420] 0 2024-05-06T12:14:44.183 INFO:tasks.workunit.client.0.smithi092.stdout:4/829: dwrite d9/d2b/d5b/daf/ffe [0,4194304] 0 2024-05-06T12:14:44.184 INFO:tasks.workunit.client.1.smithi190.stdout:2/229: dread d2/d10/f41 [0,4194304] 0 2024-05-06T12:14:44.185 INFO:tasks.workunit.client.1.smithi190.stdout:8/311: mknod d3/d8/d25/d65/c67 0 2024-05-06T12:14:44.188 INFO:tasks.workunit.client.1.smithi190.stdout:9/433: dwrite db/d23/f68 [0,4194304] 0 2024-05-06T12:14:44.188 INFO:tasks.workunit.client.0.smithi092.stdout:6/873: rename d4/d20/d23/c95 to d4/d24/d26/dae/c118 0 2024-05-06T12:14:44.190 INFO:tasks.workunit.client.1.smithi190.stdout:5/250: mkdir d2/da/d2c/d49 0 2024-05-06T12:14:44.192 INFO:tasks.workunit.client.1.smithi190.stdout:2/230: creat d2/d21/f42 x:0 0 0 2024-05-06T12:14:44.193 INFO:tasks.workunit.client.0.smithi092.stdout:9/825: dwrite fc [4194304,4194304] 0 2024-05-06T12:14:44.194 INFO:tasks.workunit.client.0.smithi092.stdout:6/874: creat d4/d24/d26/dc4/f119 x:0 0 0 2024-05-06T12:14:44.195 INFO:tasks.workunit.client.0.smithi092.stdout:9/826: chown d16/d2c/c63 16333 1 2024-05-06T12:14:44.200 INFO:tasks.workunit.client.1.smithi190.stdout:2/231: mkdir d2/d8/d19/d43 0 2024-05-06T12:14:44.201 INFO:tasks.workunit.client.0.smithi092.stdout:5/798: rename d4/d5/d10/d31/ddf/lcc to d4/d5/d10/d31/d53/d64/l11a 0 2024-05-06T12:14:44.202 INFO:tasks.workunit.client.0.smithi092.stdout:6/875: read d4/fb [491612,29106] 0 2024-05-06T12:14:44.203 INFO:tasks.workunit.client.0.smithi092.stdout:5/799: write d4/d34/d28/d4e/d4b/f103 [224901,67528] 0 2024-05-06T12:14:44.204 INFO:tasks.workunit.client.1.smithi190.stdout:0/388: dread d9/df/f2e [0,4194304] 0 2024-05-06T12:14:44.204 INFO:tasks.workunit.client.1.smithi190.stdout:3/261: dwrite d5/d7/d15/d1f/f42 [0,4194304] 0 2024-05-06T12:14:44.204 INFO:tasks.workunit.client.0.smithi092.stdout:9/827: mkdir d16/d1c/d58/d92/d115 0 2024-05-06T12:14:44.206 INFO:tasks.workunit.client.0.smithi092.stdout:9/828: chown d16/d94/da0/da7/fb5 2 1 2024-05-06T12:14:44.207 INFO:tasks.workunit.client.0.smithi092.stdout:5/800: symlink d4/d5/d10/d31/ddf/d54/d73/l11b 0 2024-05-06T12:14:44.207 INFO:tasks.workunit.client.0.smithi092.stdout:9/829: write d16/d1e/d7b/dce/f51 [183730,32134] 0 2024-05-06T12:14:44.208 INFO:tasks.workunit.client.0.smithi092.stdout:5/801: chown d4/d34/dae 105913 1 2024-05-06T12:14:44.208 INFO:tasks.workunit.client.1.smithi190.stdout:9/434: dwrite db/dd/d16/d58/f7d [0,4194304] 0 2024-05-06T12:14:44.208 INFO:tasks.workunit.client.1.smithi190.stdout:3/262: readlink d5/d7/d15/d1f/d33/l37 0 2024-05-06T12:14:44.209 INFO:tasks.workunit.client.1.smithi190.stdout:8/312: dread d3/f52 [4194304,4194304] 0 2024-05-06T12:14:44.212 INFO:tasks.workunit.client.0.smithi092.stdout:5/802: mkdir d4/d5/d10/d31/ddf/d54/db0/d102/d78/d11c 0 2024-05-06T12:14:44.212 INFO:tasks.workunit.client.1.smithi190.stdout:8/313: write d3/d8/d2b/f38 [1357607,97692] 0 2024-05-06T12:14:44.215 INFO:tasks.workunit.client.1.smithi190.stdout:2/232: mknod d2/c44 0 2024-05-06T12:14:44.217 INFO:tasks.workunit.client.1.smithi190.stdout:2/233: write d2/f3b [731905,55671] 0 2024-05-06T12:14:44.218 INFO:tasks.workunit.client.1.smithi190.stdout:8/314: write d3/d13/f21 [541876,21754] 0 2024-05-06T12:14:44.222 INFO:tasks.workunit.client.1.smithi190.stdout:0/389: dread d9/df/d21/d31/f67 [0,4194304] 0 2024-05-06T12:14:44.222 INFO:tasks.workunit.client.1.smithi190.stdout:2/234: creat d2/d8/d19/f45 x:0 0 0 2024-05-06T12:14:44.223 INFO:tasks.workunit.client.1.smithi190.stdout:9/435: creat db/dd/d16/d58/d87/d22/d49/d8c/fa0 x:0 0 0 2024-05-06T12:14:44.224 INFO:tasks.workunit.client.1.smithi190.stdout:3/263: creat d5/f5e x:0 0 0 2024-05-06T12:14:44.225 INFO:tasks.workunit.client.1.smithi190.stdout:8/315: mkdir d3/d8/d68 0 2024-05-06T12:14:44.226 INFO:tasks.workunit.client.1.smithi190.stdout:0/390: mknod d9/dc/d63/c86 0 2024-05-06T12:14:44.228 INFO:tasks.workunit.client.1.smithi190.stdout:0/391: read - d9/dc/d28/d4c/f53 zero size 2024-05-06T12:14:44.229 INFO:tasks.workunit.client.1.smithi190.stdout:3/264: read d5/d14/f47 [48505,2583] 0 2024-05-06T12:14:44.229 INFO:tasks.workunit.client.1.smithi190.stdout:8/316: mknod d3/d35/c69 0 2024-05-06T12:14:44.230 INFO:tasks.workunit.client.1.smithi190.stdout:0/392: stat d9/dc/d28/d77/c78 0 2024-05-06T12:14:44.232 INFO:tasks.workunit.client.0.smithi092.stdout:4/830: truncate d9/dd/f9e 1501462 0 2024-05-06T12:14:44.232 INFO:tasks.workunit.client.1.smithi190.stdout:2/235: dwrite d2/d8/d19/f2a [4194304,4194304] 0 2024-05-06T12:14:44.234 INFO:tasks.workunit.client.0.smithi092.stdout:4/831: symlink d9/d2b/d5b/l114 0 2024-05-06T12:14:44.235 INFO:tasks.workunit.client.1.smithi190.stdout:5/251: truncate d2/f3 3601709 0 2024-05-06T12:14:44.235 INFO:tasks.workunit.client.1.smithi190.stdout:5/252: chown d2/c37 76222336 1 2024-05-06T12:14:44.236 INFO:tasks.workunit.client.1.smithi190.stdout:0/393: mkdir d9/dc/d48/d50/d87 0 2024-05-06T12:14:44.237 INFO:tasks.workunit.client.0.smithi092.stdout:4/832: rename d9/fd5 to d9/df/f115 0 2024-05-06T12:14:44.238 INFO:tasks.workunit.client.1.smithi190.stdout:8/317: rename d3/d13/l19 to d3/d8/d2b/d3e/l6a 0 2024-05-06T12:14:44.239 INFO:tasks.workunit.client.0.smithi092.stdout:4/833: rename d9/dd/da6/dca/df2/d10d to d9/dd/da6/dca/df2/d10d/d116 22 2024-05-06T12:14:44.241 INFO:tasks.workunit.client.1.smithi190.stdout:8/318: dread d3/f1c [0,4194304] 0 2024-05-06T12:14:44.245 INFO:tasks.workunit.client.1.smithi190.stdout:0/394: write d9/df/d56/f68 [2110111,17490] 0 2024-05-06T12:14:44.246 INFO:tasks.workunit.client.0.smithi092.stdout:4/834: dwrite d9/d2b/d65/ff4 [0,4194304] 0 2024-05-06T12:14:44.246 INFO:tasks.workunit.client.0.smithi092.stdout:9/830: dread d16/d1e/f28 [0,4194304] 0 2024-05-06T12:14:44.247 INFO:tasks.workunit.client.1.smithi190.stdout:8/319: mkdir d3/d8/d25/d3b/d6b 0 2024-05-06T12:14:44.250 INFO:tasks.workunit.client.1.smithi190.stdout:8/320: fdatasync d3/f12 0 2024-05-06T12:14:44.251 INFO:tasks.workunit.client.1.smithi190.stdout:5/253: rmdir d2/da/d2c/d49 0 2024-05-06T12:14:44.252 INFO:tasks.workunit.client.1.smithi190.stdout:5/254: fsync d2/d27/f48 0 2024-05-06T12:14:44.254 INFO:tasks.workunit.client.1.smithi190.stdout:2/236: getdents d2/d10 0 2024-05-06T12:14:44.256 INFO:tasks.workunit.client.1.smithi190.stdout:8/321: mknod d3/d8/d2b/d3e/c6c 0 2024-05-06T12:14:44.258 INFO:tasks.workunit.client.1.smithi190.stdout:5/255: rename d2/f31 to d2/da/f4a 0 2024-05-06T12:14:44.259 INFO:tasks.workunit.client.0.smithi092.stdout:5/803: dread d4/d5/d10/f33 [0,4194304] 0 2024-05-06T12:14:44.262 INFO:tasks.workunit.client.1.smithi190.stdout:8/322: creat d3/d8/d68/f6d x:0 0 0 2024-05-06T12:14:44.265 INFO:tasks.workunit.client.1.smithi190.stdout:8/323: mkdir d3/d13/d4a/d6e 0 2024-05-06T12:14:44.268 INFO:tasks.workunit.client.1.smithi190.stdout:3/265: dwrite d5/d14/f56 [0,4194304] 0 2024-05-06T12:14:44.270 INFO:tasks.workunit.client.0.smithi092.stdout:6/876: dread d4/d20/d23/d30/f5e [0,4194304] 0 2024-05-06T12:14:44.270 INFO:tasks.workunit.client.0.smithi092.stdout:5/804: rmdir d4 39 2024-05-06T12:14:44.271 INFO:tasks.workunit.client.1.smithi190.stdout:3/266: symlink d5/d14/d11/l5f 0 2024-05-06T12:14:44.272 INFO:tasks.workunit.client.0.smithi092.stdout:5/805: readlink d4/d34/l6f 0 2024-05-06T12:14:44.272 INFO:tasks.workunit.client.0.smithi092.stdout:9/831: dread d16/f17 [0,4194304] 0 2024-05-06T12:14:44.272 INFO:tasks.workunit.client.0.smithi092.stdout:6/877: chown d4/d24/d38/d5d/d6d/cbc 4091231 1 2024-05-06T12:14:44.274 INFO:tasks.workunit.client.0.smithi092.stdout:6/878: stat d4/d20/d23/d30/f5e 0 2024-05-06T12:14:44.274 INFO:tasks.workunit.client.0.smithi092.stdout:5/806: rmdir d4 39 2024-05-06T12:14:44.275 INFO:tasks.workunit.client.1.smithi190.stdout:3/267: mknod d5/c60 0 2024-05-06T12:14:44.278 INFO:tasks.workunit.client.1.smithi190.stdout:5/256: dread d2/da/f4a [0,4194304] 0 2024-05-06T12:14:44.278 INFO:tasks.workunit.client.1.smithi190.stdout:3/268: rmdir d5/d14/d11/d2a/d3d 39 2024-05-06T12:14:44.279 INFO:tasks.workunit.client.1.smithi190.stdout:5/257: read - d2/da/d2c/f3c zero size 2024-05-06T12:14:44.281 INFO:tasks.workunit.client.0.smithi092.stdout:6/879: dwrite d4/d4c/d52/f6a [0,4194304] 0 2024-05-06T12:14:44.281 INFO:tasks.workunit.client.0.smithi092.stdout:5/807: truncate d4/d5/fc4 1063781 0 2024-05-06T12:14:44.282 INFO:tasks.workunit.client.1.smithi190.stdout:3/269: creat d5/d14/d11/d21/d44/f61 x:0 0 0 2024-05-06T12:14:44.282 INFO:tasks.workunit.client.0.smithi092.stdout:6/880: readlink d4/d4c/le4 0 2024-05-06T12:14:44.283 INFO:tasks.workunit.client.1.smithi190.stdout:5/258: mknod d2/da/c4b 0 2024-05-06T12:14:44.284 INFO:tasks.workunit.client.0.smithi092.stdout:4/835: write d9/d2b/d65/f64 [899234,121703] 0 2024-05-06T12:14:44.284 INFO:tasks.workunit.client.0.smithi092.stdout:5/808: symlink d4/d5/d10/da1/l11d 0 2024-05-06T12:14:44.286 INFO:tasks.workunit.client.1.smithi190.stdout:5/259: mknod d2/da/d2c/d2e/d3b/c4c 0 2024-05-06T12:14:44.287 INFO:tasks.workunit.client.1.smithi190.stdout:5/260: chown d2/da/f1b 346 1 2024-05-06T12:14:44.289 INFO:tasks.workunit.client.1.smithi190.stdout:3/270: rename d5/ff to d5/f62 0 2024-05-06T12:14:44.291 INFO:tasks.workunit.client.1.smithi190.stdout:3/271: write d5/d7/d15/f28 [3922172,106631] 0 2024-05-06T12:14:44.294 INFO:tasks.workunit.client.1.smithi190.stdout:3/272: unlink d5/d7/d15/d39/l57 0 2024-05-06T12:14:44.295 INFO:tasks.workunit.client.1.smithi190.stdout:5/261: link d2/c14 d2/c4d 0 2024-05-06T12:14:44.295 INFO:tasks.workunit.client.1.smithi190.stdout:3/273: write d5/d7/d15/f22 [4667678,100585] 0 2024-05-06T12:14:44.297 INFO:tasks.workunit.client.1.smithi190.stdout:5/262: symlink d2/da/d2c/l4e 0 2024-05-06T12:14:44.301 INFO:tasks.workunit.client.1.smithi190.stdout:5/263: getdents d2/da/d2c/d2e 0 2024-05-06T12:14:44.301 INFO:tasks.workunit.client.0.smithi092.stdout:6/881: mkdir d4/d20/d23/d30/df7/d101/d11a 0 2024-05-06T12:14:44.302 INFO:tasks.workunit.client.0.smithi092.stdout:4/836: readlink d9/df/d62/d7b/db6/lf3 0 2024-05-06T12:14:44.303 INFO:tasks.workunit.client.1.smithi190.stdout:5/264: creat d2/d27/f4f x:0 0 0 2024-05-06T12:14:44.303 INFO:tasks.workunit.client.0.smithi092.stdout:4/837: chown d9/df/d62/f63 13929338 1 2024-05-06T12:14:44.306 INFO:tasks.workunit.client.0.smithi092.stdout:5/809: creat d4/d5/d30/dfd/d108/f11e x:0 0 0 2024-05-06T12:14:44.307 INFO:tasks.workunit.client.0.smithi092.stdout:5/810: write d4/d5/d10/da1/fcd [446171,84383] 0 2024-05-06T12:14:44.310 INFO:tasks.workunit.client.0.smithi092.stdout:5/811: readlink d4/d5/d10/d31/ddf/da3/l113 0 2024-05-06T12:14:44.312 INFO:tasks.workunit.client.1.smithi190.stdout:3/274: rmdir d5/d14/d11/d21/d44 39 2024-05-06T12:14:44.312 INFO:tasks.workunit.client.1.smithi190.stdout:2/237: dread d2/d8/d19/f24 [0,4194304] 0 2024-05-06T12:14:44.312 INFO:tasks.workunit.client.0.smithi092.stdout:9/832: write d16/d2c/d2f/d3d/d33/d6b/d9e/f112 [7921086,45849] 0 2024-05-06T12:14:44.313 INFO:tasks.workunit.client.0.smithi092.stdout:5/812: chown d4/d5/d10/d31/ddf/d54/db0/d102/d20/l2d 7 1 2024-05-06T12:14:44.314 INFO:tasks.workunit.client.0.smithi092.stdout:5/813: readlink d4/d34/l14 0 2024-05-06T12:14:44.315 INFO:tasks.workunit.client.1.smithi190.stdout:3/275: unlink d5/d14/d11/d21/f41 0 2024-05-06T12:14:44.316 INFO:tasks.workunit.client.1.smithi190.stdout:3/276: write d5/d7/d15/f22 [3321608,121292] 0 2024-05-06T12:14:44.317 INFO:tasks.workunit.client.1.smithi190.stdout:2/238: mkdir d2/d8/d33/d46 0 2024-05-06T12:14:44.317 INFO:tasks.workunit.client.0.smithi092.stdout:9/833: dread fc [4194304,4194304] 0 2024-05-06T12:14:44.321 INFO:tasks.workunit.client.0.smithi092.stdout:5/814: link d4/d34/d28/c6a d4/d5/d10/d31/ddf/d54/db0/d102/d78/c11f 0 2024-05-06T12:14:44.322 INFO:tasks.workunit.client.1.smithi190.stdout:3/277: creat d5/d7/d15/d1f/d33/d5c/f63 x:0 0 0 2024-05-06T12:14:44.324 INFO:tasks.workunit.client.0.smithi092.stdout:6/882: dwrite d4/d24/d54/f79 [0,4194304] 0 2024-05-06T12:14:44.325 INFO:tasks.workunit.client.1.smithi190.stdout:2/239: dwrite d2/f1f [0,4194304] 0 2024-05-06T12:14:44.326 INFO:tasks.workunit.client.0.smithi092.stdout:4/838: dwrite d9/d2b/f32 [4194304,4194304] 0 2024-05-06T12:14:44.326 INFO:tasks.workunit.client.0.smithi092.stdout:5/815: unlink d4/d5/c38 0 2024-05-06T12:14:44.327 INFO:tasks.workunit.client.1.smithi190.stdout:3/278: write d5/d14/d11/d21/d44/f61 [791746,93559] 0 2024-05-06T12:14:44.327 INFO:tasks.workunit.client.1.smithi190.stdout:2/240: chown d2/d8/dc/d20/f40 29 1 2024-05-06T12:14:44.327 INFO:tasks.workunit.client.1.smithi190.stdout:3/279: chown d5/d14/d11/d21/d44/f5b 658 1 2024-05-06T12:14:44.334 INFO:tasks.workunit.client.1.smithi190.stdout:3/280: mkdir d5/d7/d15/d1f/d33/d64 0 2024-05-06T12:14:44.335 INFO:tasks.workunit.client.0.smithi092.stdout:5/816: truncate d4/d34/d28/fc3 963102 0 2024-05-06T12:14:44.336 INFO:tasks.workunit.client.1.smithi190.stdout:2/241: dwrite d2/d10/f2c [0,4194304] 0 2024-05-06T12:14:44.339 INFO:tasks.workunit.client.1.smithi190.stdout:6/197: sync 2024-05-06T12:14:44.340 INFO:tasks.workunit.client.1.smithi190.stdout:4/167: sync 2024-05-06T12:14:44.340 INFO:tasks.workunit.client.1.smithi190.stdout:7/209: sync 2024-05-06T12:14:44.340 INFO:tasks.workunit.client.1.smithi190.stdout:1/206: sync 2024-05-06T12:14:44.340 INFO:tasks.workunit.client.1.smithi190.stdout:3/281: symlink d5/d7/d15/l65 0 2024-05-06T12:14:44.340 INFO:tasks.workunit.client.0.smithi092.stdout:4/839: dwrite d9/d2b/d65/f101 [0,4194304] 0 2024-05-06T12:14:44.340 INFO:tasks.workunit.client.0.smithi092.stdout:5/817: truncate d4/d5/f96 854220 0 2024-05-06T12:14:44.341 INFO:tasks.workunit.client.0.smithi092.stdout:4/840: chown d9/d2b/d7c/dab/dd8/le9 231937 1 2024-05-06T12:14:44.342 INFO:tasks.workunit.client.0.smithi092.stdout:5/818: creat d4/d5/d10/da1/dfa/f120 x:0 0 0 2024-05-06T12:14:44.343 INFO:tasks.workunit.client.1.smithi190.stdout:3/282: stat d5/d14/d11/d21/f30 0 2024-05-06T12:14:44.343 INFO:tasks.workunit.client.1.smithi190.stdout:2/242: mknod d2/d8/d19/d43/c47 0 2024-05-06T12:14:44.343 INFO:tasks.workunit.client.0.smithi092.stdout:4/841: symlink d9/d2b/d7c/l117 0 2024-05-06T12:14:44.345 INFO:tasks.workunit.client.1.smithi190.stdout:1/207: mkdir d5/d23/d24/d43 0 2024-05-06T12:14:44.345 INFO:tasks.workunit.client.0.smithi092.stdout:4/842: write d9/d2b/f39 [5647335,35866] 0 2024-05-06T12:14:44.346 INFO:tasks.workunit.client.0.smithi092.stdout:5/819: mknod d4/d5/d10/c121 0 2024-05-06T12:14:44.349 INFO:tasks.workunit.client.1.smithi190.stdout:6/198: unlink df/d2c/d1e/l24 0 2024-05-06T12:14:44.351 INFO:tasks.workunit.client.0.smithi092.stdout:5/820: fsync d4/d5/d30/feb 0 2024-05-06T12:14:44.352 INFO:tasks.workunit.client.1.smithi190.stdout:3/283: creat d5/d7/d15/d1f/d33/f66 x:0 0 0 2024-05-06T12:14:44.353 INFO:tasks.workunit.client.0.smithi092.stdout:5/821: write d4/d5/d30/f3c [210516,47458] 0 2024-05-06T12:14:44.353 INFO:tasks.workunit.client.1.smithi190.stdout:5/265: dwrite d2/f3 [0,4194304] 0 2024-05-06T12:14:44.355 INFO:tasks.workunit.client.0.smithi092.stdout:5/822: chown d4/d5/d10/d31/d53/d64/f25 0 1 2024-05-06T12:14:44.356 INFO:tasks.workunit.client.1.smithi190.stdout:2/243: dread d2/d8/d33/f2b [0,4194304] 0 2024-05-06T12:14:44.357 INFO:tasks.workunit.client.1.smithi190.stdout:7/210: getdents d0/d1/d32/d45 0 2024-05-06T12:14:44.360 INFO:tasks.workunit.client.1.smithi190.stdout:6/199: symlink df/d2c/l43 0 2024-05-06T12:14:44.363 INFO:tasks.workunit.client.1.smithi190.stdout:7/211: chown d0/d15/c49 26542005 1 2024-05-06T12:14:44.363 INFO:tasks.workunit.client.0.smithi092.stdout:9/834: truncate d16/d2c/d2f/d3d/d33/d57/f82 105584 0 2024-05-06T12:14:44.364 INFO:tasks.workunit.client.0.smithi092.stdout:5/823: dwrite d4/d5/d10/da1/fe2 [0,4194304] 0 2024-05-06T12:14:44.365 INFO:tasks.workunit.client.1.smithi190.stdout:2/244: creat d2/d8/f48 x:0 0 0 2024-05-06T12:14:44.365 INFO:tasks.workunit.client.1.smithi190.stdout:3/284: write d5/d14/d11/d2a/d3d/f49 [913400,127503] 0 2024-05-06T12:14:44.366 INFO:tasks.workunit.client.1.smithi190.stdout:7/212: chown d0/d1/d32/d45 10871 1 2024-05-06T12:14:44.368 INFO:tasks.workunit.client.1.smithi190.stdout:4/168: link d2/c7 d2/d11/c3c 0 2024-05-06T12:14:44.368 INFO:tasks.workunit.client.0.smithi092.stdout:9/835: write d16/d2c/d2f/d3d/d33/f9a [2740486,97284] 0 2024-05-06T12:14:44.369 INFO:tasks.workunit.client.1.smithi190.stdout:7/213: fdatasync d0/d1/d18/f4f 0 2024-05-06T12:14:44.369 INFO:tasks.workunit.client.0.smithi092.stdout:6/883: dwrite d4/f19 [4194304,4194304] 0 2024-05-06T12:14:44.370 INFO:tasks.workunit.client.1.smithi190.stdout:3/285: mknod d5/d14/c67 0 2024-05-06T12:14:44.374 INFO:tasks.workunit.client.0.smithi092.stdout:6/884: chown d4/fb 3507 1 2024-05-06T12:14:44.374 INFO:tasks.workunit.client.1.smithi190.stdout:2/245: unlink d2/d8/f48 0 2024-05-06T12:14:44.374 INFO:tasks.workunit.client.0.smithi092.stdout:5/824: mknod d4/d5/d30/dfd/d108/c122 0 2024-05-06T12:14:44.375 INFO:tasks.workunit.client.0.smithi092.stdout:5/825: readlink d4/d5/l6e 0 2024-05-06T12:14:44.377 INFO:tasks.workunit.client.1.smithi190.stdout:1/208: dwrite d5/fa [0,4194304] 0 2024-05-06T12:14:44.377 INFO:tasks.workunit.client.0.smithi092.stdout:9/836: symlink d16/d2c/d2f/d3d/d33/d57/l116 0 2024-05-06T12:14:44.378 INFO:tasks.workunit.client.1.smithi190.stdout:5/266: dwrite d2/da/d2c/f3c [0,4194304] 0 2024-05-06T12:14:44.378 INFO:tasks.workunit.client.1.smithi190.stdout:7/214: write d0/d1/d2f/d3b/f48 [893973,65033] 0 2024-05-06T12:14:44.378 INFO:tasks.workunit.client.0.smithi092.stdout:6/885: mkdir d4/d24/d38/d5d/daf/d11b 0 2024-05-06T12:14:44.383 INFO:tasks.workunit.client.0.smithi092.stdout:6/886: chown d4/d24/d38/d5d/d6d/d7b/ff3 193282047 1 2024-05-06T12:14:44.383 INFO:tasks.workunit.client.1.smithi190.stdout:5/267: write d2/da/d2c/d2e/d3b/f42 [762958,5098] 0 2024-05-06T12:14:44.383 INFO:tasks.workunit.client.1.smithi190.stdout:2/246: mknod d2/d21/c49 0 2024-05-06T12:14:44.384 INFO:tasks.workunit.client.0.smithi092.stdout:4/843: dwrite d9/d27/f50 [4194304,4194304] 0 2024-05-06T12:14:44.384 INFO:tasks.workunit.client.1.smithi190.stdout:5/268: stat d2/da/d2c/d2e/l29 0 2024-05-06T12:14:44.384 INFO:tasks.workunit.client.1.smithi190.stdout:3/286: symlink d5/d14/d52/l68 0 2024-05-06T12:14:44.385 INFO:tasks.workunit.client.1.smithi190.stdout:3/287: dread - d5/f5e zero size 2024-05-06T12:14:44.385 INFO:tasks.workunit.client.0.smithi092.stdout:6/887: chown d4/d24/d26/dc4/fca 6428792 1 2024-05-06T12:14:44.385 INFO:tasks.workunit.client.1.smithi190.stdout:6/200: dread df/d2c/d1e/f2d [0,4194304] 0 2024-05-06T12:14:44.386 INFO:tasks.workunit.client.1.smithi190.stdout:3/288: readlink d5/d14/d11/d21/d44/l4d 0 2024-05-06T12:14:44.387 INFO:tasks.workunit.client.1.smithi190.stdout:1/209: mkdir d5/d23/d24/d44 0 2024-05-06T12:14:44.387 INFO:tasks.workunit.client.0.smithi092.stdout:6/888: fdatasync d4/d24/d38/d5d/fcf 0 2024-05-06T12:14:44.388 INFO:tasks.workunit.client.1.smithi190.stdout:3/289: dread - d5/f5e zero size 2024-05-06T12:14:44.388 INFO:tasks.workunit.client.1.smithi190.stdout:7/215: creat d0/d4c/f52 x:0 0 0 2024-05-06T12:14:44.389 INFO:tasks.workunit.client.1.smithi190.stdout:2/247: mkdir d2/d10/d4a 0 2024-05-06T12:14:44.391 INFO:tasks.workunit.client.0.smithi092.stdout:4/844: rename d9/dd/da6/f10b to d9/d27/da8/f118 0 2024-05-06T12:14:44.392 INFO:tasks.workunit.client.1.smithi190.stdout:6/201: unlink df/d2c/c3e 0 2024-05-06T12:14:44.393 INFO:tasks.workunit.client.1.smithi190.stdout:1/210: creat d5/d23/d24/f45 x:0 0 0 2024-05-06T12:14:44.393 INFO:tasks.workunit.client.1.smithi190.stdout:2/248: read d2/f38 [4106276,129623] 0 2024-05-06T12:14:44.394 INFO:tasks.workunit.client.0.smithi092.stdout:4/845: mkdir d9/d119 0 2024-05-06T12:14:44.394 INFO:tasks.workunit.client.1.smithi190.stdout:2/249: chown d2/d21/f42 1467659 1 2024-05-06T12:14:44.396 INFO:tasks.workunit.client.0.smithi092.stdout:4/846: dread - d9/d2b/d5b/daf/f105 zero size 2024-05-06T12:14:44.396 INFO:tasks.workunit.client.0.smithi092.stdout:6/889: getdents d4/d24/d26 0 2024-05-06T12:14:44.396 INFO:tasks.workunit.client.1.smithi190.stdout:7/216: mkdir d0/da/d36/d53 0 2024-05-06T12:14:44.396 INFO:tasks.workunit.client.1.smithi190.stdout:1/211: dread d5/f12 [0,4194304] 0 2024-05-06T12:14:44.397 INFO:tasks.workunit.client.1.smithi190.stdout:3/290: mknod d5/d7/d15/c69 0 2024-05-06T12:14:44.399 INFO:tasks.workunit.client.1.smithi190.stdout:7/217: readlink d0/d1/d2f/d35/l51 0 2024-05-06T12:14:44.400 INFO:tasks.workunit.client.0.smithi092.stdout:4/847: read d9/f22 [2589766,7998] 0 2024-05-06T12:14:44.401 INFO:tasks.workunit.client.1.smithi190.stdout:6/202: write df/f2e [4323377,38311] 0 2024-05-06T12:14:44.401 INFO:tasks.workunit.client.1.smithi190.stdout:7/218: write d0/d1/fb [471581,127317] 0 2024-05-06T12:14:44.402 INFO:tasks.workunit.client.1.smithi190.stdout:2/250: mknod d2/d8/c4b 0 2024-05-06T12:14:44.403 INFO:tasks.workunit.client.1.smithi190.stdout:2/251: chown d2/c4 22746292 1 2024-05-06T12:14:44.405 INFO:tasks.workunit.client.1.smithi190.stdout:6/203: dread - df/d2c/f30 zero size 2024-05-06T12:14:44.405 INFO:tasks.workunit.client.0.smithi092.stdout:5/826: getdents d4/d5/d10/d31/ddf/d54/db0/d102/d78/d11c 0 2024-05-06T12:14:44.405 INFO:tasks.workunit.client.1.smithi190.stdout:3/291: rename d5/d7/d15/d1f/d33/d40/f4a to d5/d14/f6a 0 2024-05-06T12:14:44.408 INFO:tasks.workunit.client.0.smithi092.stdout:4/848: rename d9/d27/d4a/f75 to d9/f11a 0 2024-05-06T12:14:44.408 INFO:tasks.workunit.client.0.smithi092.stdout:6/890: dwrite d4/d20/d23/d30/f5e [0,4194304] 0 2024-05-06T12:14:44.410 INFO:tasks.workunit.client.1.smithi190.stdout:7/219: rename d0/d1/d2/f26 to d0/da/d36/d53/f54 0 2024-05-06T12:14:44.412 INFO:tasks.workunit.client.0.smithi092.stdout:5/827: dread - d4/d5/d10/d31/ddf/d54/db0/d102/d20/f101 zero size 2024-05-06T12:14:44.412 INFO:tasks.workunit.client.0.smithi092.stdout:6/891: unlink d4/ca8 0 2024-05-06T12:14:44.412 INFO:tasks.workunit.client.1.smithi190.stdout:6/204: mkdir df/d17/d44 0 2024-05-06T12:14:44.413 INFO:tasks.workunit.client.0.smithi092.stdout:6/892: dread - d4/d24/d26/dc4/d108/f10b zero size 2024-05-06T12:14:44.415 INFO:tasks.workunit.client.1.smithi190.stdout:4/169: dread d2/d10/ff [0,4194304] 0 2024-05-06T12:14:44.416 INFO:tasks.workunit.client.1.smithi190.stdout:6/205: chown df/d17/c23 20005596 1 2024-05-06T12:14:44.417 INFO:tasks.workunit.client.1.smithi190.stdout:3/292: dwrite d5/d7/d15/d1f/d33/d40/f45 [0,4194304] 0 2024-05-06T12:14:44.417 INFO:tasks.workunit.client.0.smithi092.stdout:5/828: symlink d4/d34/d28/d4e/l123 0 2024-05-06T12:14:44.418 INFO:tasks.workunit.client.1.smithi190.stdout:2/252: mkdir d2/d4c 0 2024-05-06T12:14:44.418 INFO:tasks.workunit.client.0.smithi092.stdout:9/837: truncate d16/d1e/d7b/d8d/f111 5412366 0 2024-05-06T12:14:44.419 INFO:tasks.workunit.client.0.smithi092.stdout:9/838: readlink d16/d2c/d2f/d75/la6 0 2024-05-06T12:14:44.419 INFO:tasks.workunit.client.0.smithi092.stdout:6/893: truncate d4/d24/d38/d5d/d6d/dd8/ff0 112594 0 2024-05-06T12:14:44.421 INFO:tasks.workunit.client.0.smithi092.stdout:5/829: read d4/d34/d28/d4e/d4b/f4d [130240,48319] 0 2024-05-06T12:14:44.422 INFO:tasks.workunit.client.1.smithi190.stdout:7/220: creat d0/da/f55 x:0 0 0 2024-05-06T12:14:44.423 INFO:tasks.workunit.client.1.smithi190.stdout:3/293: read d5/d14/d11/d21/f38 [716698,56009] 0 2024-05-06T12:14:44.424 INFO:tasks.workunit.client.1.smithi190.stdout:1/212: getdents d5/d15 0 2024-05-06T12:14:44.427 INFO:tasks.workunit.client.1.smithi190.stdout:1/213: write d5/d15/f30 [995174,25515] 0 2024-05-06T12:14:44.428 INFO:tasks.workunit.client.0.smithi092.stdout:5/830: getdents d4/d5/d10/d31/ddf/da3 0 2024-05-06T12:14:44.428 INFO:tasks.workunit.client.1.smithi190.stdout:6/206: creat df/d2c/f45 x:0 0 0 2024-05-06T12:14:44.429 INFO:tasks.workunit.client.0.smithi092.stdout:4/849: write d9/d27/d4a/fdc [19632,96908] 0 2024-05-06T12:14:44.429 INFO:tasks.workunit.client.1.smithi190.stdout:6/207: fdatasync f1 0 2024-05-06T12:14:44.430 INFO:tasks.workunit.client.0.smithi092.stdout:4/850: write d9/d2b/d5b/f98 [1105894,9512] 0 2024-05-06T12:14:44.431 INFO:tasks.workunit.client.1.smithi190.stdout:7/221: dwrite d0/d4/f7 [0,4194304] 0 2024-05-06T12:14:44.431 INFO:tasks.workunit.client.1.smithi190.stdout:6/208: chown df/d2c/d28 1216283 1 2024-05-06T12:14:44.433 INFO:tasks.workunit.client.0.smithi092.stdout:4/851: symlink d9/df/d62/dbb/l11b 0 2024-05-06T12:14:44.433 INFO:tasks.workunit.client.0.smithi092.stdout:5/831: dread d4/d5/d10/d31/ddf/da3/dc2/fd7 [0,4194304] 0 2024-05-06T12:14:44.433 INFO:tasks.workunit.client.1.smithi190.stdout:6/209: chown df/d17/d37/f42 773545826 1 2024-05-06T12:14:44.433 INFO:tasks.workunit.client.0.smithi092.stdout:4/852: fsync d9/d2b/d65/d83/ffc 0 2024-05-06T12:14:44.436 INFO:tasks.workunit.client.1.smithi190.stdout:3/294: mknod d5/d7/d15/d1f/d33/c6b 0 2024-05-06T12:14:44.437 INFO:tasks.workunit.client.1.smithi190.stdout:4/170: mknod d2/c3d 0 2024-05-06T12:14:44.437 INFO:tasks.workunit.client.0.smithi092.stdout:4/853: creat d9/d2b/d7c/f11c x:0 0 0 2024-05-06T12:14:44.438 INFO:tasks.workunit.client.0.smithi092.stdout:4/854: chown d9/d27/l72 8230355 1 2024-05-06T12:14:44.439 INFO:tasks.workunit.client.0.smithi092.stdout:5/832: rename d4/d5/d10/d31/d53/ddb/d104/f117 to d4/d5/d10/d31/ddf/d54/db0/d102/d78/f124 0 2024-05-06T12:14:44.442 INFO:tasks.workunit.client.0.smithi092.stdout:5/833: chown d4/d5/d10/d31/ddf/d54/c7f 0 1 2024-05-06T12:14:44.443 INFO:tasks.workunit.client.0.smithi092.stdout:2/705: sync 2024-05-06T12:14:44.443 INFO:tasks.workunit.client.0.smithi092.stdout:7/905: sync 2024-05-06T12:14:44.443 INFO:tasks.workunit.client.1.smithi190.stdout:7/222: dwrite d0/d1/d2/f41 [0,4194304] 0 2024-05-06T12:14:44.445 INFO:tasks.workunit.client.0.smithi092.stdout:4/855: dwrite d9/df/d62/d7b/db6/d104/f10a [0,4194304] 0 2024-05-06T12:14:44.445 INFO:tasks.workunit.client.1.smithi190.stdout:4/171: dread - d2/f32 zero size 2024-05-06T12:14:44.446 INFO:tasks.workunit.client.0.smithi092.stdout:7/906: stat d9/de/d10/d17/d6a/c11a 0 2024-05-06T12:14:44.448 INFO:tasks.workunit.client.0.smithi092.stdout:7/907: chown d9/de/d1f/d37/f74 353 1 2024-05-06T12:14:44.449 INFO:tasks.workunit.client.1.smithi190.stdout:3/295: dread d5/f62 [0,4194304] 0 2024-05-06T12:14:44.449 INFO:tasks.workunit.client.1.smithi190.stdout:5/269: dwrite d2/f28 [0,4194304] 0 2024-05-06T12:14:44.449 INFO:tasks.workunit.client.0.smithi092.stdout:9/839: dread d16/d2c/d3f/f96 [0,4194304] 0 2024-05-06T12:14:44.450 INFO:tasks.workunit.client.0.smithi092.stdout:4/856: symlink d9/dd/da6/dca/df2/l11d 0 2024-05-06T12:14:44.452 INFO:tasks.workunit.client.0.smithi092.stdout:2/706: creat df/fd1 x:0 0 0 2024-05-06T12:14:44.453 INFO:tasks.workunit.client.0.smithi092.stdout:4/857: mkdir d9/d27/d4a/d11e 0 2024-05-06T12:14:44.455 INFO:tasks.workunit.client.1.smithi190.stdout:1/214: rename d5/fb to d5/d23/d24/d43/f46 0 2024-05-06T12:14:44.455 INFO:tasks.workunit.client.0.smithi092.stdout:7/908: symlink d9/de/d10/d17/d21/l137 0 2024-05-06T12:14:44.455 INFO:tasks.workunit.client.0.smithi092.stdout:4/858: chown d9/dd/d90/cb7 5 1 2024-05-06T12:14:44.457 INFO:tasks.workunit.client.1.smithi190.stdout:1/215: truncate d5/d23/d24/f45 765036 0 2024-05-06T12:14:44.458 INFO:tasks.workunit.client.0.smithi092.stdout:2/707: fdatasync df/d15/f95 0 2024-05-06T12:14:44.459 INFO:tasks.workunit.client.0.smithi092.stdout:4/859: chown d9/d2b/d65/f64 2483 1 2024-05-06T12:14:44.461 INFO:tasks.workunit.client.1.smithi190.stdout:4/172: readlink d2/d10/l22 0 2024-05-06T12:14:44.463 INFO:tasks.workunit.client.0.smithi092.stdout:9/840: dwrite d16/d2c/d2f/d3d/d33/f56 [4194304,4194304] 0 2024-05-06T12:14:44.464 INFO:tasks.workunit.client.0.smithi092.stdout:7/909: symlink d9/d62/d108/d10d/l138 0 2024-05-06T12:14:44.464 INFO:tasks.workunit.client.1.smithi190.stdout:5/270: unlink d2/l7 0 2024-05-06T12:14:44.465 INFO:tasks.workunit.client.1.smithi190.stdout:1/216: dwrite d5/d23/d24/f28 [0,4194304] 0 2024-05-06T12:14:44.465 INFO:tasks.workunit.client.1.smithi190.stdout:5/271: chown d2/da/d2c/f36 524 1 2024-05-06T12:14:44.466 INFO:tasks.workunit.client.0.smithi092.stdout:4/860: creat d9/df/d62/d7b/f11f x:0 0 0 2024-05-06T12:14:44.468 INFO:tasks.workunit.client.1.smithi190.stdout:7/223: symlink d0/d1/l56 0 2024-05-06T12:14:44.470 INFO:tasks.workunit.client.0.smithi092.stdout:2/708: symlink df/ld2 0 2024-05-06T12:14:44.472 INFO:tasks.workunit.client.1.smithi190.stdout:7/224: write d0/da/f55 [122241,102771] 0 2024-05-06T12:14:44.472 INFO:tasks.workunit.client.0.smithi092.stdout:4/861: stat d9/l29 0 2024-05-06T12:14:44.473 INFO:tasks.workunit.client.0.smithi092.stdout:7/910: truncate d9/de/d1f/f35 1874758 0 2024-05-06T12:14:44.474 INFO:tasks.workunit.client.1.smithi190.stdout:1/217: rmdir d5 39 2024-05-06T12:14:44.479 INFO:tasks.workunit.client.1.smithi190.stdout:4/173: creat d2/d11/d16/f3e x:0 0 0 2024-05-06T12:14:44.479 INFO:tasks.workunit.client.0.smithi092.stdout:7/911: fdatasync d9/de/d10/d17/d99/db6/df1/f6b 0 2024-05-06T12:14:44.479 INFO:tasks.workunit.client.1.smithi190.stdout:2/253: truncate d2/fe 1155062 0 2024-05-06T12:14:44.480 INFO:tasks.workunit.client.0.smithi092.stdout:2/709: write df/d1b/dcf/f81 [3370528,120320] 0 2024-05-06T12:14:44.481 INFO:tasks.workunit.client.0.smithi092.stdout:4/862: mknod d9/dd/da6/dca/df2/d10d/c120 0 2024-05-06T12:14:44.483 INFO:tasks.workunit.client.1.smithi190.stdout:2/254: mknod d2/d10/c4d 0 2024-05-06T12:14:44.484 INFO:tasks.workunit.client.1.smithi190.stdout:1/218: mkdir d5/d23/d32/d47 0 2024-05-06T12:14:44.485 INFO:tasks.workunit.client.1.smithi190.stdout:1/219: truncate d5/f16 550047 0 2024-05-06T12:14:44.486 INFO:tasks.workunit.client.1.smithi190.stdout:4/174: mknod d2/c3f 0 2024-05-06T12:14:44.486 INFO:tasks.workunit.client.1.smithi190.stdout:2/255: rmdir d2 39 2024-05-06T12:14:44.486 INFO:tasks.workunit.client.0.smithi092.stdout:4/863: symlink d9/d2b/d7c/dab/dee/l121 0 2024-05-06T12:14:44.486 INFO:tasks.workunit.client.1.smithi190.stdout:1/220: readlink d5/d23/l2e 0 2024-05-06T12:14:44.487 INFO:tasks.workunit.client.1.smithi190.stdout:1/221: stat d5/d23/d24/d3a/f3e 0 2024-05-06T12:14:44.487 INFO:tasks.workunit.client.0.smithi092.stdout:2/710: symlink df/d17/d1d/dc8/ld3 0 2024-05-06T12:14:44.488 INFO:tasks.workunit.client.1.smithi190.stdout:2/256: dread - d2/d8/d19/f45 zero size 2024-05-06T12:14:44.488 INFO:tasks.workunit.client.0.smithi092.stdout:2/711: readlink df/ld2 0 2024-05-06T12:14:44.488 INFO:tasks.workunit.client.0.smithi092.stdout:4/864: mknod d9/dd/da6/dca/df2/d10d/c122 0 2024-05-06T12:14:44.489 INFO:tasks.workunit.client.0.smithi092.stdout:4/865: chown d9/dd/fa1 8541163 1 2024-05-06T12:14:44.489 INFO:tasks.workunit.client.1.smithi190.stdout:1/222: mknod d5/d23/d24/d43/c48 0 2024-05-06T12:14:44.490 INFO:tasks.workunit.client.0.smithi092.stdout:7/912: getdents d9/de/d10/d17/d21/d2e/d9c 0 2024-05-06T12:14:44.491 INFO:tasks.workunit.client.1.smithi190.stdout:4/175: rename d2/l9 to d2/d10/l40 0 2024-05-06T12:14:44.492 INFO:tasks.workunit.client.0.smithi092.stdout:2/712: creat df/d17/d53/da9/fd4 x:0 0 0 2024-05-06T12:14:44.493 INFO:tasks.workunit.client.0.smithi092.stdout:4/866: symlink d9/d27/dc5/d111/l123 0 2024-05-06T12:14:44.493 INFO:tasks.workunit.client.1.smithi190.stdout:4/176: write d2/d11/d16/f1a [477962,121519] 0 2024-05-06T12:14:44.494 INFO:tasks.workunit.client.1.smithi190.stdout:1/223: mknod d5/d23/d24/c49 0 2024-05-06T12:14:44.495 INFO:tasks.workunit.client.1.smithi190.stdout:1/224: fsync d5/d23/d24/d3a/f3e 0 2024-05-06T12:14:44.495 INFO:tasks.workunit.client.1.smithi190.stdout:2/257: creat d2/d8/d33/d46/f4e x:0 0 0 2024-05-06T12:14:44.495 INFO:tasks.workunit.client.1.smithi190.stdout:1/225: chown d5/d23/d32 7075128 1 2024-05-06T12:14:44.496 INFO:tasks.workunit.client.0.smithi092.stdout:2/713: unlink df/d1b/l32 0 2024-05-06T12:14:44.496 INFO:tasks.workunit.client.0.smithi092.stdout:5/834: dread d4/d34/fd0 [0,4194304] 0 2024-05-06T12:14:44.499 INFO:tasks.workunit.client.0.smithi092.stdout:4/867: mkdir d9/d27/da8/db3/d124 0 2024-05-06T12:14:44.500 INFO:tasks.workunit.client.1.smithi190.stdout:4/177: write d2/f23 [364624,96450] 0 2024-05-06T12:14:44.500 INFO:tasks.workunit.client.1.smithi190.stdout:3/296: truncate d5/d14/f24 1615330 0 2024-05-06T12:14:44.500 INFO:tasks.workunit.client.1.smithi190.stdout:3/297: chown d5 10846144 1 2024-05-06T12:14:44.501 INFO:tasks.workunit.client.0.smithi092.stdout:9/841: truncate d16/d2c/d2f/d75/ff6 635553 0 2024-05-06T12:14:44.501 INFO:tasks.workunit.client.1.smithi190.stdout:4/178: dread - d2/d11/d16/f3e zero size 2024-05-06T12:14:44.501 INFO:tasks.workunit.client.1.smithi190.stdout:4/179: dread - d2/f32 zero size 2024-05-06T12:14:44.502 INFO:tasks.workunit.client.0.smithi092.stdout:2/714: chown df/d17/d53/da9/c75 16504875 1 2024-05-06T12:14:44.502 INFO:tasks.workunit.client.0.smithi092.stdout:9/842: read d16/d1e/f86 [453811,99363] 0 2024-05-06T12:14:44.502 INFO:tasks.workunit.client.1.smithi190.stdout:1/226: rename d5/f6 to d5/d23/d32/f4a 0 2024-05-06T12:14:44.503 INFO:tasks.workunit.client.0.smithi092.stdout:2/715: chown df/d15/fbe 4 1 2024-05-06T12:14:44.504 INFO:tasks.workunit.client.1.smithi190.stdout:4/180: mkdir d2/d11/d16/d41 0 2024-05-06T12:14:44.507 INFO:tasks.workunit.client.0.smithi092.stdout:2/716: write df/d17/d1d/f49 [4389733,188] 0 2024-05-06T12:14:44.509 INFO:tasks.workunit.client.1.smithi190.stdout:3/298: rename d5/d14/d11/d21/d44/f61 to d5/d14/d11/d2a/d3d/f6c 0 2024-05-06T12:14:44.509 INFO:tasks.workunit.client.1.smithi190.stdout:5/272: dwrite d2/da/f1b [0,4194304] 0 2024-05-06T12:14:44.511 INFO:tasks.workunit.client.1.smithi190.stdout:4/181: creat d2/d11/d16/f42 x:0 0 0 2024-05-06T12:14:44.513 INFO:tasks.workunit.client.0.smithi092.stdout:2/717: mkdir df/d1b/dcf/d4d/dd5 0 2024-05-06T12:14:44.513 INFO:tasks.workunit.client.1.smithi190.stdout:5/273: fdatasync d2/da/d2c/d2e/d3b/f40 0 2024-05-06T12:14:44.516 INFO:tasks.workunit.client.0.smithi092.stdout:7/913: truncate d9/de/ff4 3630912 0 2024-05-06T12:14:44.517 INFO:tasks.workunit.client.1.smithi190.stdout:4/182: dread d2/f1e [4194304,4194304] 0 2024-05-06T12:14:44.520 INFO:tasks.workunit.client.0.smithi092.stdout:5/835: link d4/d5/d10/d31/ddf/d54/db0/d102/cdc d4/d5/d10/d31/ddf/d54/db0/d102/d20/c125 0 2024-05-06T12:14:44.521 INFO:tasks.workunit.client.1.smithi190.stdout:5/274: read d2/d27/f20 [912030,62310] 0 2024-05-06T12:14:44.522 INFO:tasks.workunit.client.0.smithi092.stdout:7/914: truncate d9/de/d1f/ff6 321135 0 2024-05-06T12:14:44.522 INFO:tasks.workunit.client.0.smithi092.stdout:2/718: creat df/d15/d88/dc9/fd6 x:0 0 0 2024-05-06T12:14:44.522 INFO:tasks.workunit.client.0.smithi092.stdout:4/868: write d9/f43 [145337,67991] 0 2024-05-06T12:14:44.524 INFO:tasks.workunit.client.1.smithi190.stdout:1/227: dwrite d5/d23/d24/d3a/f3b [0,4194304] 0 2024-05-06T12:14:44.526 INFO:tasks.workunit.client.1.smithi190.stdout:7/225: dwrite d0/d1/fc [0,4194304] 0 2024-05-06T12:14:44.526 INFO:tasks.workunit.client.0.smithi092.stdout:4/869: mknod d9/d27/dc5/d111/c125 0 2024-05-06T12:14:44.526 INFO:tasks.workunit.client.0.smithi092.stdout:5/836: getdents d4/d5/d30/dfd/d108 0 2024-05-06T12:14:44.527 INFO:tasks.workunit.client.0.smithi092.stdout:5/837: readlink d4/d34/d28/d4e/d67/lf4 0 2024-05-06T12:14:44.528 INFO:tasks.workunit.client.0.smithi092.stdout:2/719: read df/d17/d1d/f2e [2187947,863] 0 2024-05-06T12:14:44.529 INFO:tasks.workunit.client.1.smithi190.stdout:4/183: write d2/d11/d16/f2f [77406,10017] 0 2024-05-06T12:14:44.529 INFO:tasks.workunit.client.0.smithi092.stdout:4/870: truncate d9/d2b/d65/d83/ffc 555144 0 2024-05-06T12:14:44.530 INFO:tasks.workunit.client.0.smithi092.stdout:7/915: getdents d9/de/d10 0 2024-05-06T12:14:44.531 INFO:tasks.workunit.client.1.smithi190.stdout:3/299: dwrite d5/d7/d15/f22 [0,4194304] 0 2024-05-06T12:14:44.531 INFO:tasks.workunit.client.1.smithi190.stdout:5/275: readlink d2/da/d2c/d2e/l46 0 2024-05-06T12:14:44.531 INFO:tasks.workunit.client.0.smithi092.stdout:5/838: rmdir d4/d34/d28 39 2024-05-06T12:14:44.533 INFO:tasks.workunit.client.0.smithi092.stdout:2/720: write df/d17/d1d/f4f [3184089,37739] 0 2024-05-06T12:14:44.533 INFO:tasks.workunit.client.0.smithi092.stdout:4/871: mkdir d9/dd/da6/dca/d126 0 2024-05-06T12:14:44.534 INFO:tasks.workunit.client.0.smithi092.stdout:2/721: chown df/d1b/dcf/cc4 1583 1 2024-05-06T12:14:44.535 INFO:tasks.workunit.client.1.smithi190.stdout:8/324: sync 2024-05-06T12:14:44.535 INFO:tasks.workunit.client.1.smithi190.stdout:9/436: sync 2024-05-06T12:14:44.535 INFO:tasks.workunit.client.1.smithi190.stdout:0/395: sync 2024-05-06T12:14:44.536 INFO:tasks.workunit.client.0.smithi092.stdout:2/722: dread - df/d15/d88/d59/fb0 zero size 2024-05-06T12:14:44.537 INFO:tasks.workunit.client.1.smithi190.stdout:2/258: dwrite d2/d10/f41 [0,4194304] 0 2024-05-06T12:14:44.538 INFO:tasks.workunit.client.0.smithi092.stdout:9/843: dread d16/d2c/d2f/d3d/d33/f3b [0,4194304] 0 2024-05-06T12:14:44.540 INFO:tasks.workunit.client.1.smithi190.stdout:0/396: dread - d9/dc/f6c zero size 2024-05-06T12:14:44.540 INFO:tasks.workunit.client.1.smithi190.stdout:8/325: chown d3/d8/d25/f39 926640 1 2024-05-06T12:14:44.542 INFO:tasks.workunit.client.0.smithi092.stdout:9/844: mkdir d16/d94/ddc/d117 0 2024-05-06T12:14:44.544 INFO:tasks.workunit.client.1.smithi190.stdout:1/228: truncate d5/fe 1659170 0 2024-05-06T12:14:44.545 INFO:tasks.workunit.client.0.smithi092.stdout:5/839: getdents d4/d5/d30/d10a 0 2024-05-06T12:14:44.547 INFO:tasks.workunit.client.1.smithi190.stdout:4/184: read d2/d11/f2e [325010,124373] 0 2024-05-06T12:14:44.547 INFO:tasks.workunit.client.1.smithi190.stdout:7/226: mkdir d0/d1/d2f/d33/d57 0 2024-05-06T12:14:44.548 INFO:tasks.workunit.client.1.smithi190.stdout:4/185: stat d2/d11/d16/f2f 0 2024-05-06T12:14:44.549 INFO:tasks.workunit.client.1.smithi190.stdout:8/326: symlink d3/d8/d25/d3b/d49/l6f 0 2024-05-06T12:14:44.550 INFO:tasks.workunit.client.1.smithi190.stdout:2/259: mkdir d2/d8/d19/d4f 0 2024-05-06T12:14:44.551 INFO:tasks.workunit.client.0.smithi092.stdout:9/845: rmdir d16/d1c/dd4/d114 0 2024-05-06T12:14:44.552 INFO:tasks.workunit.client.1.smithi190.stdout:8/327: chown d3/d8/d25/f39 8119 1 2024-05-06T12:14:44.553 INFO:tasks.workunit.client.1.smithi190.stdout:5/276: mkdir d2/da/d2c/d50 0 2024-05-06T12:14:44.554 INFO:tasks.workunit.client.0.smithi092.stdout:5/840: symlink d4/d5/d10/d31/ddf/d54/db0/d102/d78/d11c/l126 0 2024-05-06T12:14:44.555 INFO:tasks.workunit.client.1.smithi190.stdout:3/300: read d5/d14/d11/f1b [1986150,127947] 0 2024-05-06T12:14:44.555 INFO:tasks.workunit.client.1.smithi190.stdout:0/397: dwrite d9/df/f33 [0,4194304] 0 2024-05-06T12:14:44.555 INFO:tasks.workunit.client.0.smithi092.stdout:9/846: creat d16/d2c/d2f/d3d/d33/d6b/f118 x:0 0 0 2024-05-06T12:14:44.558 INFO:tasks.workunit.client.1.smithi190.stdout:2/260: mkdir d2/d21/d50 0 2024-05-06T12:14:44.559 INFO:tasks.workunit.client.1.smithi190.stdout:8/328: symlink d3/d13/l70 0 2024-05-06T12:14:44.560 INFO:tasks.workunit.client.0.smithi092.stdout:5/841: symlink d4/d34/d28/d4e/d67/l127 0 2024-05-06T12:14:44.560 INFO:tasks.workunit.client.1.smithi190.stdout:2/261: chown d2/fa 16 1 2024-05-06T12:14:44.563 INFO:tasks.workunit.client.1.smithi190.stdout:7/227: creat d0/d1/d32/d45/f58 x:0 0 0 2024-05-06T12:14:44.564 INFO:tasks.workunit.client.1.smithi190.stdout:8/329: stat d3/d8/d25/d65/c67 0 2024-05-06T12:14:44.564 INFO:tasks.workunit.client.1.smithi190.stdout:3/301: unlink d5/d14/c67 0 2024-05-06T12:14:44.565 INFO:tasks.workunit.client.0.smithi092.stdout:5/842: chown d4/d5/d10/d31/ddf/d54/db0/d102/d20/l2d 365 1 2024-05-06T12:14:44.565 INFO:tasks.workunit.client.0.smithi092.stdout:9/847: getdents d16/d1e/d7b/dce 0 2024-05-06T12:14:44.567 INFO:tasks.workunit.client.1.smithi190.stdout:4/186: dwrite d2/d10/ff [0,4194304] 0 2024-05-06T12:14:44.569 INFO:tasks.workunit.client.1.smithi190.stdout:6/210: sync 2024-05-06T12:14:44.569 INFO:tasks.workunit.client.1.smithi190.stdout:8/330: chown d3/d13/d4a/f4f 642364 1 2024-05-06T12:14:44.572 INFO:tasks.workunit.client.1.smithi190.stdout:5/277: dwrite d2/f3 [0,4194304] 0 2024-05-06T12:14:44.572 INFO:tasks.workunit.client.1.smithi190.stdout:1/229: getdents d5/d23 0 2024-05-06T12:14:44.573 INFO:tasks.workunit.client.1.smithi190.stdout:2/262: rename d2/d10/l3e to d2/d21/d50/l51 0 2024-05-06T12:14:44.573 INFO:tasks.workunit.client.1.smithi190.stdout:5/278: chown d2 137090932 1 2024-05-06T12:14:44.574 INFO:tasks.workunit.client.0.smithi092.stdout:2/723: dwrite df/d15/d88/d59/fb0 [0,4194304] 0 2024-05-06T12:14:44.574 INFO:tasks.workunit.client.0.smithi092.stdout:7/916: dwrite d9/de/d10/d17/d6a/f9d [0,4194304] 0 2024-05-06T12:14:44.575 INFO:tasks.workunit.client.1.smithi190.stdout:0/398: dwrite d9/f5f [0,4194304] 0 2024-05-06T12:14:44.575 INFO:tasks.workunit.client.1.smithi190.stdout:7/228: stat d0/d4/fd 0 2024-05-06T12:14:44.581 INFO:tasks.workunit.client.0.smithi092.stdout:9/848: dwrite d16/d1c/d58/d92/fe9 [0,4194304] 0 2024-05-06T12:14:44.583 INFO:tasks.workunit.client.1.smithi190.stdout:0/399: write d9/df/f33 [1168563,53996] 0 2024-05-06T12:14:44.583 INFO:tasks.workunit.client.1.smithi190.stdout:7/229: write d0/d1/d32/d45/f58 [436687,7989] 0 2024-05-06T12:14:44.585 INFO:tasks.workunit.client.1.smithi190.stdout:0/400: truncate d9/f2f 4848471 0 2024-05-06T12:14:44.585 INFO:tasks.workunit.client.1.smithi190.stdout:4/187: creat d2/d11/d16/d31/f43 x:0 0 0 2024-05-06T12:14:44.586 INFO:tasks.workunit.client.0.smithi092.stdout:7/917: creat d9/d62/d108/d10d/f139 x:0 0 0 2024-05-06T12:14:44.587 INFO:tasks.workunit.client.0.smithi092.stdout:9/849: unlink d16/d1e/d7b/dce/f51 0 2024-05-06T12:14:44.588 INFO:tasks.workunit.client.1.smithi190.stdout:8/331: readlink d3/d8/d2b/l64 0 2024-05-06T12:14:44.589 INFO:tasks.workunit.client.1.smithi190.stdout:6/211: mkdir df/d17/d46 0 2024-05-06T12:14:44.590 INFO:tasks.workunit.client.0.smithi092.stdout:9/850: readlink d16/d1c/d58/d92/lf2 0 2024-05-06T12:14:44.594 INFO:tasks.workunit.client.1.smithi190.stdout:2/263: creat d2/d8/dc/d20/f52 x:0 0 0 2024-05-06T12:14:44.595 INFO:tasks.workunit.client.0.smithi092.stdout:5/843: dwrite d4/d5/d10/d31/ddf/d54/db0/d102/d78/f124 [0,4194304] 0 2024-05-06T12:14:44.595 INFO:tasks.workunit.client.0.smithi092.stdout:2/724: mknod df/d17/d53/d85/da0/cd7 0 2024-05-06T12:14:44.595 INFO:tasks.workunit.client.1.smithi190.stdout:5/279: read d2/f8 [1034481,101615] 0 2024-05-06T12:14:44.596 INFO:tasks.workunit.client.1.smithi190.stdout:1/230: symlink d5/l4b 0 2024-05-06T12:14:44.596 INFO:tasks.workunit.client.1.smithi190.stdout:7/230: truncate d0/d4/f39 305545 0 2024-05-06T12:14:44.603 INFO:tasks.workunit.client.0.smithi092.stdout:4/872: dread d9/d2b/f6a [0,4194304] 0 2024-05-06T12:14:44.603 INFO:tasks.workunit.client.0.smithi092.stdout:2/725: rename df/ld2 to df/d1b/db4/dce/ld8 0 2024-05-06T12:14:44.603 INFO:tasks.workunit.client.0.smithi092.stdout:7/918: dread d9/de/d1f/d37/d48/d97/fbf [0,4194304] 0 2024-05-06T12:14:44.604 INFO:tasks.workunit.client.0.smithi092.stdout:5/844: mknod d4/d5/d10/d31/c128 0 2024-05-06T12:14:44.604 INFO:tasks.workunit.client.1.smithi190.stdout:7/231: rename d0/da/c13 to d0/d15/c59 0 2024-05-06T12:14:44.605 INFO:tasks.workunit.client.1.smithi190.stdout:1/231: dwrite d5/f16 [0,4194304] 0 2024-05-06T12:14:44.606 INFO:tasks.workunit.client.1.smithi190.stdout:7/232: fsync d0/d4/f7 0 2024-05-06T12:14:44.606 INFO:tasks.workunit.client.0.smithi092.stdout:5/845: fsync d4/d5/d10/d31/d53/dcb/fff 0 2024-05-06T12:14:44.608 INFO:tasks.workunit.client.0.smithi092.stdout:4/873: mkdir d9/d2b/d65/d127 0 2024-05-06T12:14:44.609 INFO:tasks.workunit.client.1.smithi190.stdout:0/401: rename d9/df/d21/f6a to d9/df/d21/f88 0 2024-05-06T12:14:44.610 INFO:tasks.workunit.client.0.smithi092.stdout:2/726: rename df/d1b/dcf/d4d to df/d17/d53/da9/dd9 0 2024-05-06T12:14:44.610 INFO:tasks.workunit.client.0.smithi092.stdout:5/846: symlink d4/d5/d10/d31/d53/ddb/d104/l129 0 2024-05-06T12:14:44.612 INFO:tasks.workunit.client.0.smithi092.stdout:7/919: getdents d9/de/d10/d17/d21/d2e 0 2024-05-06T12:14:44.613 INFO:tasks.workunit.client.1.smithi190.stdout:2/264: link d2/c30 d2/d8/c53 0 2024-05-06T12:14:44.614 INFO:tasks.workunit.client.1.smithi190.stdout:2/265: dread - d2/d8/d33/d46/f4e zero size 2024-05-06T12:14:44.614 INFO:tasks.workunit.client.0.smithi092.stdout:7/920: dread - d9/de/fa2 zero size 2024-05-06T12:14:44.615 INFO:tasks.workunit.client.1.smithi190.stdout:7/233: rename d0/d1/d2f/d33/d3c to d0/d1/d18/d2b/d5a 0 2024-05-06T12:14:44.616 INFO:tasks.workunit.client.1.smithi190.stdout:7/234: chown d0/d1/d2f 15251 1 2024-05-06T12:14:44.616 INFO:tasks.workunit.client.0.smithi092.stdout:7/921: readlink d9/de/da4/la9 0 2024-05-06T12:14:44.617 INFO:tasks.workunit.client.1.smithi190.stdout:9/437: dread db/dd/d16/d58/d87/f70 [0,4194304] 0 2024-05-06T12:14:44.617 INFO:tasks.workunit.client.1.smithi190.stdout:6/212: creat df/f47 x:0 0 0 2024-05-06T12:14:44.617 INFO:tasks.workunit.client.0.smithi092.stdout:4/874: dwrite d9/dd/f80 [0,4194304] 0 2024-05-06T12:14:44.620 INFO:tasks.workunit.client.1.smithi190.stdout:2/266: mknod d2/d8/d19/d43/c54 0 2024-05-06T12:14:44.622 INFO:tasks.workunit.client.1.smithi190.stdout:3/302: dread d5/d7/d15/f28 [0,4194304] 0 2024-05-06T12:14:44.623 INFO:tasks.workunit.client.0.smithi092.stdout:8/865: sync 2024-05-06T12:14:44.623 INFO:tasks.workunit.client.0.smithi092.stdout:1/797: sync 2024-05-06T12:14:44.625 INFO:tasks.workunit.client.1.smithi190.stdout:1/232: rmdir d5/d23/d24/d44 0 2024-05-06T12:14:44.625 INFO:tasks.workunit.client.0.smithi092.stdout:4/875: readlink d9/l99 0 2024-05-06T12:14:44.630 INFO:tasks.workunit.client.1.smithi190.stdout:3/303: mkdir d5/d7/d6d 0 2024-05-06T12:14:44.631 INFO:tasks.workunit.client.0.smithi092.stdout:3/643: sync 2024-05-06T12:14:44.631 INFO:tasks.workunit.client.0.smithi092.stdout:6/894: sync 2024-05-06T12:14:44.631 INFO:tasks.workunit.client.1.smithi190.stdout:2/267: dwrite d2/f3b [0,4194304] 0 2024-05-06T12:14:44.632 INFO:tasks.workunit.client.1.smithi190.stdout:0/402: rename d9/dc/d28/d77/c78 to d9/dc/c89 0 2024-05-06T12:14:44.633 INFO:tasks.workunit.client.1.smithi190.stdout:2/268: fdatasync d2/d21/f42 0 2024-05-06T12:14:44.633 INFO:tasks.workunit.client.0.smithi092.stdout:7/922: dwrite d9/de/d1f/dcf/f12f [0,4194304] 0 2024-05-06T12:14:44.633 INFO:tasks.workunit.client.1.smithi190.stdout:2/269: stat d2 0 2024-05-06T12:14:44.634 INFO:tasks.workunit.client.1.smithi190.stdout:1/233: dread d5/d23/d24/d43/f46 [0,4194304] 0 2024-05-06T12:14:44.634 INFO:tasks.workunit.client.1.smithi190.stdout:0/403: chown d9/dc/f6c 16773 1 2024-05-06T12:14:44.634 INFO:tasks.workunit.client.1.smithi190.stdout:2/270: dread - d2/d8/dc/d20/f40 zero size 2024-05-06T12:14:44.635 INFO:tasks.workunit.client.1.smithi190.stdout:6/213: creat df/d17/d44/f48 x:0 0 0 2024-05-06T12:14:44.636 INFO:tasks.workunit.client.0.smithi092.stdout:7/923: fsync d9/f102 0 2024-05-06T12:14:44.636 INFO:tasks.workunit.client.1.smithi190.stdout:0/404: write d9/f5f [4424753,107769] 0 2024-05-06T12:14:44.637 INFO:tasks.workunit.client.0.smithi092.stdout:7/924: chown d9/de/l1c 14224661 1 2024-05-06T12:14:44.637 INFO:tasks.workunit.client.1.smithi190.stdout:9/438: creat db/d23/d25/fa1 x:0 0 0 2024-05-06T12:14:44.637 INFO:tasks.workunit.client.1.smithi190.stdout:1/234: stat d5/d15/d1e/l41 0 2024-05-06T12:14:44.640 INFO:tasks.workunit.client.0.smithi092.stdout:1/798: dread d3/d10/d13/d19/d21/d88/f91 [0,4194304] 0 2024-05-06T12:14:44.643 INFO:tasks.workunit.client.1.smithi190.stdout:1/235: read d5/d23/d24/f25 [178423,25976] 0 2024-05-06T12:14:44.643 INFO:tasks.workunit.client.0.smithi092.stdout:7/925: fsync d9/de/fa2 0 2024-05-06T12:14:44.644 INFO:tasks.workunit.client.1.smithi190.stdout:9/439: truncate db/d23/f4d 1035916 0 2024-05-06T12:14:44.644 INFO:tasks.workunit.client.1.smithi190.stdout:2/271: dread d2/d10/f2c [0,4194304] 0 2024-05-06T12:14:44.645 INFO:tasks.workunit.client.1.smithi190.stdout:6/214: chown fb 774 1 2024-05-06T12:14:44.647 INFO:tasks.workunit.client.0.smithi092.stdout:9/851: dread d16/d1e/d7b/d8d/f111 [0,4194304] 0 2024-05-06T12:14:44.647 INFO:tasks.workunit.client.0.smithi092.stdout:3/644: symlink d8/db/lc6 0 2024-05-06T12:14:44.648 INFO:tasks.workunit.client.0.smithi092.stdout:3/645: read - d8/d24/d27/fb6 zero size 2024-05-06T12:14:44.649 INFO:tasks.workunit.client.1.smithi190.stdout:0/405: symlink d9/dc/d63/l8a 0 2024-05-06T12:14:44.649 INFO:tasks.workunit.client.1.smithi190.stdout:5/280: write d2/f26 [4273404,124725] 0 2024-05-06T12:14:44.650 INFO:tasks.workunit.client.1.smithi190.stdout:5/281: dread - d2/d27/f48 zero size 2024-05-06T12:14:44.650 INFO:tasks.workunit.client.0.smithi092.stdout:7/926: symlink d9/de/d1f/d37/d48/d93/l13a 0 2024-05-06T12:14:44.651 INFO:tasks.workunit.client.1.smithi190.stdout:3/304: link d5/d7/d15/d39/f55 d5/d7/d15/d1f/d33/d5c/f6e 0 2024-05-06T12:14:44.652 INFO:tasks.workunit.client.0.smithi092.stdout:4/876: write d9/fc0 [794967,87743] 0 2024-05-06T12:14:44.653 INFO:tasks.workunit.client.1.smithi190.stdout:4/188: truncate d2/f1e 3868662 0 2024-05-06T12:14:44.654 INFO:tasks.workunit.client.1.smithi190.stdout:4/189: fsync d2/d11/d16/d31/f43 0 2024-05-06T12:14:44.654 INFO:tasks.workunit.client.0.smithi092.stdout:3/646: mkdir d8/db/d14/dc7 0 2024-05-06T12:14:44.655 INFO:tasks.workunit.client.1.smithi190.stdout:8/332: truncate d3/f2e 4054421 0 2024-05-06T12:14:44.655 INFO:tasks.workunit.client.0.smithi092.stdout:4/877: chown d9/d27/dc5/d111/l123 6 1 2024-05-06T12:14:44.656 INFO:tasks.workunit.client.1.smithi190.stdout:3/305: read d5/d7/f1e [2997313,68866] 0 2024-05-06T12:14:44.656 INFO:tasks.workunit.client.0.smithi092.stdout:1/799: creat d3/d10/d13/d19/d21/f101 x:0 0 0 2024-05-06T12:14:44.657 INFO:tasks.workunit.client.0.smithi092.stdout:6/895: write d4/f6 [3367775,81522] 0 2024-05-06T12:14:44.658 INFO:tasks.workunit.client.0.smithi092.stdout:7/927: mknod d9/de/d10/d17/d99/db6/df1/c13b 0 2024-05-06T12:14:44.659 INFO:tasks.workunit.client.0.smithi092.stdout:5/847: dwrite d4/d5/d10/f17 [0,4194304] 0 2024-05-06T12:14:44.659 INFO:tasks.workunit.client.1.smithi190.stdout:6/215: unlink df/d17/c23 0 2024-05-06T12:14:44.660 INFO:tasks.workunit.client.1.smithi190.stdout:0/406: creat d9/dc/d48/f8b x:0 0 0 2024-05-06T12:14:44.661 INFO:tasks.workunit.client.0.smithi092.stdout:3/647: unlink d8/db/d4c/d40/l53 0 2024-05-06T12:14:44.661 INFO:tasks.workunit.client.0.smithi092.stdout:7/928: stat d9/d62/d121/f134 0 2024-05-06T12:14:44.661 INFO:tasks.workunit.client.0.smithi092.stdout:4/878: creat d9/df/d62/d7b/f128 x:0 0 0 2024-05-06T12:14:44.662 INFO:tasks.workunit.client.1.smithi190.stdout:2/272: chown d2/d8/c53 7453 1 2024-05-06T12:14:44.663 INFO:tasks.workunit.client.0.smithi092.stdout:6/896: creat d4/d24/d9f/dc6/f11c x:0 0 0 2024-05-06T12:14:44.664 INFO:tasks.workunit.client.0.smithi092.stdout:3/648: mknod d8/d1b/d73/cc8 0 2024-05-06T12:14:44.664 INFO:tasks.workunit.client.0.smithi092.stdout:6/897: write d4/d20/d23/d30/f6f [2362701,105430] 0 2024-05-06T12:14:44.666 INFO:tasks.workunit.client.0.smithi092.stdout:3/649: fsync d8/d1b/f81 0 2024-05-06T12:14:44.666 INFO:tasks.workunit.client.0.smithi092.stdout:5/848: mknod d4/d5/d10/d31/ddf/d54/db0/c12a 0 2024-05-06T12:14:44.668 INFO:tasks.workunit.client.1.smithi190.stdout:3/306: mknod d5/d14/d11/d2a/d3d/c6f 0 2024-05-06T12:14:44.668 INFO:tasks.workunit.client.1.smithi190.stdout:3/307: stat d5/d14/d11/d2a 0 2024-05-06T12:14:44.671 INFO:tasks.workunit.client.1.smithi190.stdout:3/308: chown d5/d14/d11/d21/f30 0 1 2024-05-06T12:14:44.671 INFO:tasks.workunit.client.1.smithi190.stdout:7/235: truncate d0/ff 894826 0 2024-05-06T12:14:44.671 INFO:tasks.workunit.client.1.smithi190.stdout:6/216: creat df/d17/d37/f49 x:0 0 0 2024-05-06T12:14:44.672 INFO:tasks.workunit.client.1.smithi190.stdout:3/309: readlink d5/d14/d11/d2a/d3d/l3e 0 2024-05-06T12:14:44.672 INFO:tasks.workunit.client.1.smithi190.stdout:7/236: chown d0/d4c/f52 0 1 2024-05-06T12:14:44.672 INFO:tasks.workunit.client.1.smithi190.stdout:3/310: read - d5/d14/f6a zero size 2024-05-06T12:14:44.673 INFO:tasks.workunit.client.1.smithi190.stdout:7/237: read d0/f22 [189468,128266] 0 2024-05-06T12:14:44.675 INFO:tasks.workunit.client.1.smithi190.stdout:7/238: read d0/f19 [2964512,95836] 0 2024-05-06T12:14:44.678 INFO:tasks.workunit.client.0.smithi092.stdout:4/879: dwrite d9/f24 [0,4194304] 0 2024-05-06T12:14:44.680 INFO:tasks.workunit.client.0.smithi092.stdout:3/650: dwrite d8/d24/f77 [0,4194304] 0 2024-05-06T12:14:44.680 INFO:tasks.workunit.client.0.smithi092.stdout:5/849: dwrite d4/d5/d10/fba [0,4194304] 0 2024-05-06T12:14:44.681 INFO:tasks.workunit.client.1.smithi190.stdout:3/311: symlink d5/d7/d15/d39/l70 0 2024-05-06T12:14:44.681 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:44 smithi092 ceph-mon[26954]: pgmap v24: 65 pgs: 65 active+clean; 2.3 GiB data, 7.9 GiB used, 529 GiB / 536 GiB avail; 81 MiB/s rd, 289 MiB/s wr, 645 op/s 2024-05-06T12:14:44.684 INFO:tasks.workunit.client.0.smithi092.stdout:4/880: write d9/d2b/d5b/fb5 [4880027,9966] 0 2024-05-06T12:14:44.686 INFO:tasks.workunit.client.0.smithi092.stdout:5/850: fdatasync d4/d5/d10/f33 0 2024-05-06T12:14:44.687 INFO:tasks.workunit.client.1.smithi190.stdout:8/333: creat d3/f71 x:0 0 0 2024-05-06T12:14:44.688 INFO:tasks.workunit.client.1.smithi190.stdout:6/217: mknod df/c4a 0 2024-05-06T12:14:44.689 INFO:tasks.workunit.client.1.smithi190.stdout:0/407: creat d9/dc/d28/f8c x:0 0 0 2024-05-06T12:14:44.689 INFO:tasks.workunit.client.1.smithi190.stdout:5/282: getdents d2/da/d2c/d2e/d3b 0 2024-05-06T12:14:44.691 INFO:tasks.workunit.client.0.smithi092.stdout:9/852: dread d16/d2c/d2f/d3d/d33/d6b/d9e/f112 [4194304,4194304] 0 2024-05-06T12:14:44.691 INFO:tasks.workunit.client.0.smithi092.stdout:3/651: mknod d8/d24/cc9 0 2024-05-06T12:14:44.692 INFO:tasks.workunit.client.1.smithi190.stdout:3/312: creat d5/d7/f71 x:0 0 0 2024-05-06T12:14:44.693 INFO:tasks.workunit.client.1.smithi190.stdout:4/190: getdents d2/d11/d16/d31 0 2024-05-06T12:14:44.693 INFO:tasks.workunit.client.0.smithi092.stdout:9/853: fsync d16/d2c/d2f/d3d/d33/d37/f10d 0 2024-05-06T12:14:44.695 INFO:tasks.workunit.client.1.smithi190.stdout:6/218: mknod df/d17/c4b 0 2024-05-06T12:14:44.696 INFO:tasks.workunit.client.0.smithi092.stdout:9/854: chown d16/d2c/d2f/d3d/d33/d6b/d9e/l106 11348994 1 2024-05-06T12:14:44.697 INFO:tasks.workunit.client.0.smithi092.stdout:4/881: creat d9/df/d62/d7b/d107/f129 x:0 0 0 2024-05-06T12:14:44.698 INFO:tasks.workunit.client.1.smithi190.stdout:2/273: dread d2/d8/d19/f2a [0,4194304] 0 2024-05-06T12:14:44.699 INFO:tasks.workunit.client.1.smithi190.stdout:5/283: creat d2/da/d2c/d50/f51 x:0 0 0 2024-05-06T12:14:44.700 INFO:tasks.workunit.client.0.smithi092.stdout:9/855: truncate d16/d2c/d2f/d75/f107 446540 0 2024-05-06T12:14:44.700 INFO:tasks.workunit.client.1.smithi190.stdout:6/219: rename df/d12 to df/d12/d4c 22 2024-05-06T12:14:44.701 INFO:tasks.workunit.client.1.smithi190.stdout:6/220: fdatasync fe 0 2024-05-06T12:14:44.701 INFO:tasks.workunit.client.0.smithi092.stdout:6/898: write d4/d20/d23/d30/f36 [1716903,33570] 0 2024-05-06T12:14:44.702 INFO:tasks.workunit.client.0.smithi092.stdout:4/882: dread - d9/d27/fbe zero size 2024-05-06T12:14:44.702 INFO:tasks.workunit.client.0.smithi092.stdout:1/800: dread d3/d10/d13/d19/d21/f24 [0,4194304] 0 2024-05-06T12:14:44.704 INFO:tasks.workunit.client.0.smithi092.stdout:9/856: chown d16/d2c/d3f/fd6 29601732 1 2024-05-06T12:14:44.704 INFO:tasks.workunit.client.1.smithi190.stdout:5/284: rename d2/f3d to d2/da/d2c/d2e/d3b/f52 0 2024-05-06T12:14:44.705 INFO:tasks.workunit.client.0.smithi092.stdout:7/929: dread d9/de/d10/d17/d99/db6/df1/f98 [0,4194304] 0 2024-05-06T12:14:44.705 INFO:tasks.workunit.client.0.smithi092.stdout:6/899: fsync d4/d24/d26/dae/dc0/fad 0 2024-05-06T12:14:44.706 INFO:tasks.workunit.client.1.smithi190.stdout:6/221: symlink df/d12/l4d 0 2024-05-06T12:14:44.706 INFO:tasks.workunit.client.0.smithi092.stdout:5/851: write d4/d5/d10/d31/ddf/d54/db0/d102/d20/f101 [359484,96643] 0 2024-05-06T12:14:44.707 INFO:tasks.workunit.client.1.smithi190.stdout:2/274: dwrite d2/d8/dc/d20/f52 [0,4194304] 0 2024-05-06T12:14:44.709 INFO:tasks.workunit.client.0.smithi092.stdout:5/852: write d4/d59/f83 [7738478,7718] 0 2024-05-06T12:14:44.709 INFO:tasks.workunit.client.0.smithi092.stdout:9/857: chown d16/d2c/d2f/d3d/d33/d6b/d9e/l106 13098771 1 2024-05-06T12:14:44.710 INFO:tasks.workunit.client.1.smithi190.stdout:2/275: write d2/d8/d33/d46/f4e [944406,47269] 0 2024-05-06T12:14:44.710 INFO:tasks.workunit.client.0.smithi092.stdout:7/930: fsync d9/de/d10/f25 0 2024-05-06T12:14:44.710 INFO:tasks.workunit.client.0.smithi092.stdout:3/652: dwrite d8/db/d14/d5a/f94 [0,4194304] 0 2024-05-06T12:14:44.711 INFO:tasks.workunit.client.1.smithi190.stdout:2/276: stat d2/d8/dc 0 2024-05-06T12:14:44.712 INFO:tasks.workunit.client.0.smithi092.stdout:6/900: fsync d4/d24/d38/faa 0 2024-05-06T12:14:44.712 INFO:tasks.workunit.client.1.smithi190.stdout:7/239: dread d0/d1/d2/f1d [0,4194304] 0 2024-05-06T12:14:44.715 INFO:tasks.workunit.client.1.smithi190.stdout:2/277: mkdir d2/d8/d33/d46/d55 0 2024-05-06T12:14:44.717 INFO:tasks.workunit.client.1.smithi190.stdout:2/278: truncate d2/fa 1318323 0 2024-05-06T12:14:44.718 INFO:tasks.workunit.client.0.smithi092.stdout:6/901: chown d4/d20/d23/d30/c51 8 1 2024-05-06T12:14:44.718 INFO:tasks.workunit.client.1.smithi190.stdout:2/279: write d2/f1f [334597,1351] 0 2024-05-06T12:14:44.719 INFO:tasks.workunit.client.0.smithi092.stdout:3/653: creat d8/d1b/d73/fca x:0 0 0 2024-05-06T12:14:44.719 INFO:tasks.workunit.client.0.smithi092.stdout:7/931: rmdir d9/de/d10/d17/d21/d2e 39 2024-05-06T12:14:44.719 INFO:tasks.workunit.client.0.smithi092.stdout:4/883: dwrite d9/dd/da6/dca/fed [0,4194304] 0 2024-05-06T12:14:44.720 INFO:tasks.workunit.client.1.smithi190.stdout:7/240: creat d0/da/f5b x:0 0 0 2024-05-06T12:14:44.721 INFO:tasks.workunit.client.0.smithi092.stdout:4/884: chown d9/d27/d4a/f76 82 1 2024-05-06T12:14:44.721 INFO:tasks.workunit.client.0.smithi092.stdout:5/853: mkdir d4/d5/d30/dfd/d108/d12b 0 2024-05-06T12:14:44.722 INFO:tasks.workunit.client.1.smithi190.stdout:2/280: rename d2 to d2/d21/d50/d56 22 2024-05-06T12:14:44.722 INFO:tasks.workunit.client.0.smithi092.stdout:6/902: write d4/d4c/f75 [524061,79568] 0 2024-05-06T12:14:44.725 INFO:tasks.workunit.client.0.smithi092.stdout:4/885: write d9/d2b/d7c/dab/fc6 [1358575,102207] 0 2024-05-06T12:14:44.726 INFO:tasks.workunit.client.0.smithi092.stdout:4/886: stat d9/d27/f31 0 2024-05-06T12:14:44.729 INFO:tasks.workunit.client.0.smithi092.stdout:1/801: write d3/fdd [724048,126541] 0 2024-05-06T12:14:44.730 INFO:tasks.workunit.client.1.smithi190.stdout:8/334: dread d3/fc [4194304,4194304] 0 2024-05-06T12:14:44.731 INFO:tasks.workunit.client.1.smithi190.stdout:7/241: mkdir d0/da/d36/d53/d5c 0 2024-05-06T12:14:44.732 INFO:tasks.workunit.client.0.smithi092.stdout:1/802: readlink d3/d10/d13/d19/d21/d88/d8b/lab 0 2024-05-06T12:14:44.732 INFO:tasks.workunit.client.1.smithi190.stdout:7/242: dread - d0/f40 zero size 2024-05-06T12:14:44.733 INFO:tasks.workunit.client.0.smithi092.stdout:7/932: symlink d9/de/d10/d17/d99/dc3/dd2/l13c 0 2024-05-06T12:14:44.733 INFO:tasks.workunit.client.0.smithi092.stdout:9/858: write d16/d2c/d2f/d3d/f3c [2629520,7322] 0 2024-05-06T12:14:44.734 INFO:tasks.workunit.client.1.smithi190.stdout:7/243: chown d0/d15/c49 21503 1 2024-05-06T12:14:44.734 INFO:tasks.workunit.client.0.smithi092.stdout:6/903: dread d4/d24/d26/f6e [0,4194304] 0 2024-05-06T12:14:44.734 INFO:tasks.workunit.client.0.smithi092.stdout:3/654: dwrite d8/db/d14/d5a/f94 [0,4194304] 0 2024-05-06T12:14:44.735 INFO:tasks.workunit.client.1.smithi190.stdout:4/191: dread d2/d11/f14 [0,4194304] 0 2024-05-06T12:14:44.736 INFO:tasks.workunit.client.0.smithi092.stdout:4/887: unlink d9/d27/f4d 0 2024-05-06T12:14:44.736 INFO:tasks.workunit.client.1.smithi190.stdout:8/335: rename d3/d35/d3a/c56 to d3/d13/d4a/c72 0 2024-05-06T12:14:44.736 INFO:tasks.workunit.client.0.smithi092.stdout:9/859: dread d16/f1b [0,4194304] 0 2024-05-06T12:14:44.738 INFO:tasks.workunit.client.0.smithi092.stdout:5/854: getdents d4/d34/dae/de0 0 2024-05-06T12:14:44.739 INFO:tasks.workunit.client.1.smithi190.stdout:4/192: dread d2/d11/f20 [0,4194304] 0 2024-05-06T12:14:44.739 INFO:tasks.workunit.client.1.smithi190.stdout:2/281: truncate d2/fe 1490405 0 2024-05-06T12:14:44.739 INFO:tasks.workunit.client.0.smithi092.stdout:6/904: creat d4/d4c/d91/d93/d9d/f11d x:0 0 0 2024-05-06T12:14:44.740 INFO:tasks.workunit.client.1.smithi190.stdout:0/408: write d9/dc/d48/d50/f66 [4982855,1534] 0 2024-05-06T12:14:44.742 INFO:tasks.workunit.client.0.smithi092.stdout:3/655: rename d8/d24/d33/f6a to d8/d24/fcb 0 2024-05-06T12:14:44.743 INFO:tasks.workunit.client.0.smithi092.stdout:4/888: mknod d9/df/d62/d7b/d107/c12a 0 2024-05-06T12:14:44.743 INFO:tasks.workunit.client.1.smithi190.stdout:3/313: write d5/d7/d15/d1f/d33/d5c/f6e [879707,24348] 0 2024-05-06T12:14:44.744 INFO:tasks.workunit.client.0.smithi092.stdout:5/855: unlink d4/d5/d10/d31/d53/d64/l11a 0 2024-05-06T12:14:44.747 INFO:tasks.workunit.client.1.smithi190.stdout:4/193: read d2/d11/d16/f1a [736364,57060] 0 2024-05-06T12:14:44.748 INFO:tasks.workunit.client.1.smithi190.stdout:5/285: fsync d2/da/d2c/d50/f51 0 2024-05-06T12:14:44.749 INFO:tasks.workunit.client.1.smithi190.stdout:7/244: read d0/da/f2e [298664,59458] 0 2024-05-06T12:14:44.752 INFO:tasks.workunit.client.1.smithi190.stdout:2/282: truncate d2/d21/f3d 282664 0 2024-05-06T12:14:44.754 INFO:tasks.workunit.client.1.smithi190.stdout:5/286: mknod d2/da/c53 0 2024-05-06T12:14:44.754 INFO:tasks.workunit.client.1.smithi190.stdout:3/314: symlink d5/d14/d11/d2a/l72 0 2024-05-06T12:14:44.757 INFO:tasks.workunit.client.1.smithi190.stdout:0/409: dwrite d9/d6f/f7a [0,4194304] 0 2024-05-06T12:14:44.757 INFO:tasks.workunit.client.1.smithi190.stdout:2/283: read d2/f35 [1736158,87313] 0 2024-05-06T12:14:44.760 INFO:tasks.workunit.client.1.smithi190.stdout:0/410: stat c4 0 2024-05-06T12:14:44.761 INFO:tasks.workunit.client.1.smithi190.stdout:1/236: dwrite d5/fe [0,4194304] 0 2024-05-06T12:14:44.762 INFO:tasks.workunit.client.1.smithi190.stdout:3/315: truncate d5/d14/f6a 277237 0 2024-05-06T12:14:44.763 INFO:tasks.workunit.client.1.smithi190.stdout:7/245: truncate d0/d1/d18/f4f 838671 0 2024-05-06T12:14:44.769 INFO:tasks.workunit.client.1.smithi190.stdout:6/222: dwrite f7 [0,4194304] 0 2024-05-06T12:14:44.771 INFO:tasks.workunit.client.1.smithi190.stdout:7/246: chown d0/da/d36/d53/d5c 208 1 2024-05-06T12:14:44.771 INFO:tasks.workunit.client.1.smithi190.stdout:2/284: write d2/d8/f15 [4472192,87952] 0 2024-05-06T12:14:44.772 INFO:tasks.workunit.client.1.smithi190.stdout:8/336: dread d3/f52 [0,4194304] 0 2024-05-06T12:14:44.775 INFO:tasks.workunit.client.1.smithi190.stdout:4/194: getdents d2/d11/d16 0 2024-05-06T12:14:44.777 INFO:tasks.workunit.client.1.smithi190.stdout:3/316: fsync d5/d14/d11/d2a/d3d/f54 0 2024-05-06T12:14:44.779 INFO:tasks.workunit.client.1.smithi190.stdout:4/195: stat d2/d10/l40 0 2024-05-06T12:14:44.781 INFO:tasks.workunit.client.1.smithi190.stdout:7/247: rename d0/d15 to d0/d14/d5d 0 2024-05-06T12:14:44.782 INFO:tasks.workunit.client.1.smithi190.stdout:1/237: creat d5/d23/d32/d47/f4c x:0 0 0 2024-05-06T12:14:44.784 INFO:tasks.workunit.client.1.smithi190.stdout:8/337: fdatasync d3/d8/f2c 0 2024-05-06T12:14:44.784 INFO:tasks.workunit.client.1.smithi190.stdout:8/338: readlink d3/l51 0 2024-05-06T12:14:44.785 INFO:tasks.workunit.client.1.smithi190.stdout:3/317: creat d5/d7/f73 x:0 0 0 2024-05-06T12:14:44.786 INFO:tasks.workunit.client.1.smithi190.stdout:4/196: rmdir d2/d10 39 2024-05-06T12:14:44.789 INFO:tasks.workunit.client.1.smithi190.stdout:6/223: fdatasync f7 0 2024-05-06T12:14:44.789 INFO:tasks.workunit.client.1.smithi190.stdout:2/285: rename d2/d8/f15 to d2/d8/d19/d4f/f57 0 2024-05-06T12:14:44.789 INFO:tasks.workunit.client.1.smithi190.stdout:8/339: truncate d3/d8/d68/f6d 469751 0 2024-05-06T12:14:44.792 INFO:tasks.workunit.client.1.smithi190.stdout:1/238: dread d5/f8 [0,4194304] 0 2024-05-06T12:14:44.794 INFO:tasks.workunit.client.1.smithi190.stdout:3/318: mknod d5/d14/d11/d21/c74 0 2024-05-06T12:14:44.797 INFO:tasks.workunit.client.1.smithi190.stdout:7/248: rename d0/d1/d18/f2d to d0/d1/d18/d2b/d5a/f5e 0 2024-05-06T12:14:44.799 INFO:tasks.workunit.client.1.smithi190.stdout:5/287: truncate d2/da/d2c/f3c 2282173 0 2024-05-06T12:14:44.801 INFO:tasks.workunit.client.1.smithi190.stdout:8/340: creat d3/d35/d55/f73 x:0 0 0 2024-05-06T12:14:44.802 INFO:tasks.workunit.client.1.smithi190.stdout:8/341: stat d3/f71 0 2024-05-06T12:14:44.802 INFO:tasks.workunit.client.1.smithi190.stdout:0/411: dread d9/dc/d48/d50/f66 [0,4194304] 0 2024-05-06T12:14:44.803 INFO:tasks.workunit.client.1.smithi190.stdout:4/197: dwrite d2/d10/f35 [0,4194304] 0 2024-05-06T12:14:44.805 INFO:tasks.workunit.client.1.smithi190.stdout:8/342: read d3/f2d [3000325,78873] 0 2024-05-06T12:14:44.805 INFO:tasks.workunit.client.1.smithi190.stdout:6/224: mknod df/d17/d37/c4e 0 2024-05-06T12:14:44.812 INFO:tasks.workunit.client.1.smithi190.stdout:9/440: sync 2024-05-06T12:14:44.813 INFO:tasks.workunit.client.0.smithi092.stdout:3/656: rename d8/d24/l88 to d8/db/d14/lcc 0 2024-05-06T12:14:44.814 INFO:tasks.workunit.client.1.smithi190.stdout:4/198: creat d2/d11/f44 x:0 0 0 2024-05-06T12:14:44.814 INFO:tasks.workunit.client.0.smithi092.stdout:4/889: fsync d9/d27/da8/f118 0 2024-05-06T12:14:44.814 INFO:tasks.workunit.client.0.smithi092.stdout:6/905: creat d4/d24/d66/d105/f11e x:0 0 0 2024-05-06T12:14:44.814 INFO:tasks.workunit.client.1.smithi190.stdout:0/412: unlink d9/df/d21/f3b 0 2024-05-06T12:14:44.815 INFO:tasks.workunit.client.0.smithi092.stdout:1/803: write d3/f3d [248998,81021] 0 2024-05-06T12:14:44.815 INFO:tasks.workunit.client.0.smithi092.stdout:5/856: creat d4/d5/f12c x:0 0 0 2024-05-06T12:14:44.816 INFO:tasks.workunit.client.1.smithi190.stdout:4/199: write d2/d11/d16/d31/f43 [992467,29485] 0 2024-05-06T12:14:44.816 INFO:tasks.workunit.client.0.smithi092.stdout:6/906: stat d4/d24/d4e/ced 0 2024-05-06T12:14:44.817 INFO:tasks.workunit.client.0.smithi092.stdout:3/657: mknod d8/db/d4c/d40/d7a/dbd/ccd 0 2024-05-06T12:14:44.818 INFO:tasks.workunit.client.0.smithi092.stdout:1/804: dread - d3/d10/d13/d19/fdc zero size 2024-05-06T12:14:44.818 INFO:tasks.workunit.client.1.smithi190.stdout:2/286: link d2/l9 d2/d8/d19/d4f/l58 0 2024-05-06T12:14:44.819 INFO:tasks.workunit.client.1.smithi190.stdout:7/249: symlink d0/d1/d18/d37/l5f 0 2024-05-06T12:14:44.819 INFO:tasks.workunit.client.0.smithi092.stdout:5/857: creat d4/d5/d30/dfd/d108/f12d x:0 0 0 2024-05-06T12:14:44.820 INFO:tasks.workunit.client.1.smithi190.stdout:1/239: dwrite d5/f16 [4194304,4194304] 0 2024-05-06T12:14:44.820 INFO:tasks.workunit.client.1.smithi190.stdout:2/287: fsync d2/d8/d33/f37 0 2024-05-06T12:14:44.821 INFO:tasks.workunit.client.0.smithi092.stdout:3/658: creat d8/db/d14/d5a/db0/fce x:0 0 0 2024-05-06T12:14:44.821 INFO:tasks.workunit.client.1.smithi190.stdout:7/250: chown d0/d1/d2/f41 19280073 1 2024-05-06T12:14:44.821 INFO:tasks.workunit.client.1.smithi190.stdout:9/441: symlink db/d23/d25/d47/d4b/la2 0 2024-05-06T12:14:44.822 INFO:tasks.workunit.client.0.smithi092.stdout:6/907: dread d4/d4c/d52/ddc/f100 [0,4194304] 0 2024-05-06T12:14:44.822 INFO:tasks.workunit.client.0.smithi092.stdout:1/805: getdents d3/d42/d84 0 2024-05-06T12:14:44.823 INFO:tasks.workunit.client.1.smithi190.stdout:0/413: creat d9/dc/d63/f8d x:0 0 0 2024-05-06T12:14:44.827 INFO:tasks.workunit.client.0.smithi092.stdout:5/858: truncate d4/d5/d10/d31/ddf/d54/d73/f10e 970907 0 2024-05-06T12:14:44.828 INFO:tasks.workunit.client.1.smithi190.stdout:3/319: creat d5/d14/d11/d2a/f75 x:0 0 0 2024-05-06T12:14:44.828 INFO:tasks.workunit.client.0.smithi092.stdout:6/908: read d4/d24/d38/d5d/d6d/dd8/ff0 [28538,107629] 0 2024-05-06T12:14:44.828 INFO:tasks.workunit.client.0.smithi092.stdout:5/859: fsync d4/d5/f2a 0 2024-05-06T12:14:44.829 INFO:tasks.workunit.client.1.smithi190.stdout:1/240: rmdir d5/d23 39 2024-05-06T12:14:44.829 INFO:tasks.workunit.client.0.smithi092.stdout:1/806: mkdir d3/d102 0 2024-05-06T12:14:44.831 INFO:tasks.workunit.client.0.smithi092.stdout:7/933: dread d9/de/d10/d17/d99/db6/df1/f66 [0,4194304] 0 2024-05-06T12:14:44.831 INFO:tasks.workunit.client.1.smithi190.stdout:1/241: readlink d5/l4b 0 2024-05-06T12:14:44.832 INFO:tasks.workunit.client.1.smithi190.stdout:0/414: mknod d9/d6f/c8e 0 2024-05-06T12:14:44.834 INFO:tasks.workunit.client.1.smithi190.stdout:8/343: creat d3/d35/f74 x:0 0 0 2024-05-06T12:14:44.834 INFO:tasks.workunit.client.1.smithi190.stdout:9/442: dread db/dd/d16/d4c/d4e/f9d [0,4194304] 0 2024-05-06T12:14:44.835 INFO:tasks.workunit.client.0.smithi092.stdout:4/890: dread d9/df/d62/f9d [0,4194304] 0 2024-05-06T12:14:44.837 INFO:tasks.workunit.client.0.smithi092.stdout:4/891: stat d9/df/d62/d7b 0 2024-05-06T12:14:44.838 INFO:tasks.workunit.client.0.smithi092.stdout:1/807: readlink d3/d10/d13/d19/d21/l44 0 2024-05-06T12:14:44.839 INFO:tasks.workunit.client.1.smithi190.stdout:1/242: mknod d5/c4d 0 2024-05-06T12:14:44.839 INFO:tasks.workunit.client.1.smithi190.stdout:8/344: read - d3/d8/d25/f66 zero size 2024-05-06T12:14:44.839 INFO:tasks.workunit.client.1.smithi190.stdout:6/225: dwrite df/d2c/d1e/f2d [0,4194304] 0 2024-05-06T12:14:44.839 INFO:tasks.workunit.client.1.smithi190.stdout:5/288: dread d2/da/d2c/f3c [0,4194304] 0 2024-05-06T12:14:44.841 INFO:tasks.workunit.client.0.smithi092.stdout:7/934: creat d9/de/d10/d17/d21/d2e/d9c/f13d x:0 0 0 2024-05-06T12:14:44.841 INFO:tasks.workunit.client.0.smithi092.stdout:5/860: dwrite d4/d5/d10/da1/dfa/f120 [0,4194304] 0 2024-05-06T12:14:44.842 INFO:tasks.workunit.client.1.smithi190.stdout:4/200: dwrite d2/f25 [0,4194304] 0 2024-05-06T12:14:44.845 INFO:tasks.workunit.client.0.smithi092.stdout:1/808: creat d3/d10/d13/d19/d21/d72/d9a/da4/deb/f103 x:0 0 0 2024-05-06T12:14:44.847 INFO:tasks.workunit.client.0.smithi092.stdout:3/659: rmdir d8/db/d14/d5a 39 2024-05-06T12:14:44.847 INFO:tasks.workunit.client.1.smithi190.stdout:9/443: truncate db/d23/d25/f37 3347146 0 2024-05-06T12:14:44.849 INFO:tasks.workunit.client.1.smithi190.stdout:0/415: symlink d9/dc/d28/d4c/d80/l8f 0 2024-05-06T12:14:44.849 INFO:tasks.workunit.client.1.smithi190.stdout:1/243: dread - d5/d23/d32/d47/f4c zero size 2024-05-06T12:14:44.849 INFO:tasks.workunit.client.0.smithi092.stdout:6/909: dread d4/d24/d26/fa4 [0,4194304] 0 2024-05-06T12:14:44.850 INFO:tasks.workunit.client.1.smithi190.stdout:6/226: write df/d17/d37/f3c [96534,43507] 0 2024-05-06T12:14:44.850 INFO:tasks.workunit.client.0.smithi092.stdout:4/892: truncate d9/d27/f49 1428114 0 2024-05-06T12:14:44.851 INFO:tasks.workunit.client.0.smithi092.stdout:4/893: fsync d9/d27/d4a/f76 0 2024-05-06T12:14:44.851 INFO:tasks.workunit.client.0.smithi092.stdout:4/894: stat d9/dd/f23 0 2024-05-06T12:14:44.853 INFO:tasks.workunit.client.0.smithi092.stdout:4/895: chown d9/df/f14 5121 1 2024-05-06T12:14:44.854 INFO:tasks.workunit.client.1.smithi190.stdout:3/320: truncate d5/d14/f56 1663857 0 2024-05-06T12:14:44.855 INFO:tasks.workunit.client.0.smithi092.stdout:7/935: symlink d9/de/d1f/l13e 0 2024-05-06T12:14:44.857 INFO:tasks.workunit.client.0.smithi092.stdout:7/936: write d9/da5/f30 [895143,45400] 0 2024-05-06T12:14:44.857 INFO:tasks.workunit.client.1.smithi190.stdout:7/251: dwrite d0/f19 [4194304,4194304] 0 2024-05-06T12:14:44.858 INFO:tasks.workunit.client.1.smithi190.stdout:6/227: dread - df/d2c/f35 zero size 2024-05-06T12:14:44.858 INFO:tasks.workunit.client.1.smithi190.stdout:5/289: rename d2/d27/f4f to d2/d27/f54 0 2024-05-06T12:14:44.858 INFO:tasks.workunit.client.0.smithi092.stdout:6/910: symlink d4/d24/d26/dae/dc0/l11f 0 2024-05-06T12:14:44.859 INFO:tasks.workunit.client.0.smithi092.stdout:7/937: dread - d9/de/d10/d17/d99/db6/df1/d101/f133 zero size 2024-05-06T12:14:44.860 INFO:tasks.workunit.client.0.smithi092.stdout:1/809: creat d3/d10/d13/d46/f104 x:0 0 0 2024-05-06T12:14:44.860 INFO:tasks.workunit.client.1.smithi190.stdout:8/345: rmdir d3/d8/d25/d65 39 2024-05-06T12:14:44.862 INFO:tasks.workunit.client.1.smithi190.stdout:1/244: creat d5/d15/f4e x:0 0 0 2024-05-06T12:14:44.862 INFO:tasks.workunit.client.0.smithi092.stdout:4/896: rename d9/d27/d4a/c82 to d9/df/d62/d7b/db6/c12b 0 2024-05-06T12:14:44.863 INFO:tasks.workunit.client.1.smithi190.stdout:0/416: mknod d9/dc/d63/c90 0 2024-05-06T12:14:44.864 INFO:tasks.workunit.client.1.smithi190.stdout:5/290: symlink d2/da/l55 0 2024-05-06T12:14:44.864 INFO:tasks.workunit.client.0.smithi092.stdout:6/911: creat d4/d24/d26/dae/f120 x:0 0 0 2024-05-06T12:14:44.866 INFO:tasks.workunit.client.1.smithi190.stdout:5/291: readlink d2/da/l55 0 2024-05-06T12:14:44.866 INFO:tasks.workunit.client.1.smithi190.stdout:0/417: write d9/f2f [5880456,4068] 0 2024-05-06T12:14:44.866 INFO:tasks.workunit.client.0.smithi092.stdout:1/810: truncate d3/d10/d13/d19/f5a 1742407 0 2024-05-06T12:14:44.867 INFO:tasks.workunit.client.1.smithi190.stdout:9/444: mknod db/dd/d16/d4c/d4e/ca3 0 2024-05-06T12:14:44.867 INFO:tasks.workunit.client.0.smithi092.stdout:3/660: getdents d8/d1b 0 2024-05-06T12:14:44.868 INFO:tasks.workunit.client.1.smithi190.stdout:2/288: dread d2/d8/dc/f13 [0,4194304] 0 2024-05-06T12:14:44.868 INFO:tasks.workunit.client.1.smithi190.stdout:6/228: dread fb [0,4194304] 0 2024-05-06T12:14:44.869 INFO:tasks.workunit.client.1.smithi190.stdout:3/321: read d5/d7/d15/d1f/f3b [363709,40688] 0 2024-05-06T12:14:44.870 INFO:tasks.workunit.client.1.smithi190.stdout:8/346: mknod d3/d13/d4a/d6e/c75 0 2024-05-06T12:14:44.870 INFO:tasks.workunit.client.1.smithi190.stdout:6/229: chown df/d17 283 1 2024-05-06T12:14:44.871 INFO:tasks.workunit.client.0.smithi092.stdout:7/938: link d9/de/d1f/l12a d9/d62/d108/l13f 0 2024-05-06T12:14:44.871 INFO:tasks.workunit.client.1.smithi190.stdout:9/445: write db/dd/d16/d58/d87/d22/d49/d88/f86 [609373,34815] 0 2024-05-06T12:14:44.874 INFO:tasks.workunit.client.0.smithi092.stdout:3/661: unlink d8/db/d4c/d40/d67/lb1 0 2024-05-06T12:14:44.874 INFO:tasks.workunit.client.1.smithi190.stdout:1/245: mkdir d5/d4f 0 2024-05-06T12:14:44.875 INFO:tasks.workunit.client.0.smithi092.stdout:3/662: write d8/db/d14/f50 [578351,6348] 0 2024-05-06T12:14:44.877 INFO:tasks.workunit.client.0.smithi092.stdout:7/939: mkdir d9/d62/d108/d140 0 2024-05-06T12:14:44.877 INFO:tasks.workunit.client.1.smithi190.stdout:3/322: dread d5/d14/d11/d2a/d3d/f6c [0,4194304] 0 2024-05-06T12:14:44.880 INFO:tasks.workunit.client.0.smithi092.stdout:6/912: truncate d4/d24/d26/dae/dc0/fcc 961070 0 2024-05-06T12:14:44.881 INFO:tasks.workunit.client.1.smithi190.stdout:9/446: chown db/dd/c2d 28137 1 2024-05-06T12:14:44.882 INFO:tasks.workunit.client.1.smithi190.stdout:8/347: stat d3/f2e 0 2024-05-06T12:14:44.883 INFO:tasks.workunit.client.0.smithi092.stdout:5/861: dwrite d4/d5/d10/d31/d53/d64/f25 [4194304,4194304] 0 2024-05-06T12:14:44.884 INFO:tasks.workunit.client.0.smithi092.stdout:5/862: dread - d4/d5/d30/dfd/d108/f116 zero size 2024-05-06T12:14:44.885 INFO:tasks.workunit.client.1.smithi190.stdout:6/230: symlink df/d2c/d28/l4f 0 2024-05-06T12:14:44.888 INFO:tasks.workunit.client.0.smithi092.stdout:7/940: creat d9/de/d10/d17/d21/d38/d77/f141 x:0 0 0 2024-05-06T12:14:44.888 INFO:tasks.workunit.client.1.smithi190.stdout:0/418: mkdir d9/d91 0 2024-05-06T12:14:44.889 INFO:tasks.workunit.client.0.smithi092.stdout:6/913: chown d4/d20/d23/lf6 0 1 2024-05-06T12:14:44.890 INFO:tasks.workunit.client.0.smithi092.stdout:6/914: chown d4/d4c/da9 1 1 2024-05-06T12:14:44.890 INFO:tasks.workunit.client.1.smithi190.stdout:3/323: creat d5/d7/d15/d1f/d33/f76 x:0 0 0 2024-05-06T12:14:44.890 INFO:tasks.workunit.client.1.smithi190.stdout:7/252: truncate d0/d1/fc 2560264 0 2024-05-06T12:14:44.890 INFO:tasks.workunit.client.0.smithi092.stdout:3/663: mknod d8/db/ccf 0 2024-05-06T12:14:44.892 INFO:tasks.workunit.client.1.smithi190.stdout:4/201: dread d2/f37 [0,4194304] 0 2024-05-06T12:14:44.892 INFO:tasks.workunit.client.1.smithi190.stdout:5/292: link d2/f30 d2/d27/f56 0 2024-05-06T12:14:44.893 INFO:tasks.workunit.client.0.smithi092.stdout:1/811: write d3/d10/d13/d19/d21/d88/fa0 [5039487,76726] 0 2024-05-06T12:14:44.894 INFO:tasks.workunit.client.1.smithi190.stdout:5/293: write d2/da/f4a [4192803,57806] 0 2024-05-06T12:14:44.895 INFO:tasks.workunit.client.0.smithi092.stdout:5/863: rename d4/d5/d10/d31/ddf/d54/db0/d102/fcf to d4/d5/d10/d31/ddf/d6c/f12e 0 2024-05-06T12:14:44.897 INFO:tasks.workunit.client.1.smithi190.stdout:6/231: rename c0 to df/d2c/d28/c50 0 2024-05-06T12:14:44.898 INFO:tasks.workunit.client.1.smithi190.stdout:6/232: chown df/d17/f26 35082581 1 2024-05-06T12:14:44.898 INFO:tasks.workunit.client.1.smithi190.stdout:1/246: creat d5/d4f/f50 x:0 0 0 2024-05-06T12:14:44.899 INFO:tasks.workunit.client.1.smithi190.stdout:0/419: symlink d9/d51/l92 0 2024-05-06T12:14:44.899 INFO:tasks.workunit.client.1.smithi190.stdout:6/233: fsync df/d2c/d28/f2a 0 2024-05-06T12:14:44.899 INFO:tasks.workunit.client.0.smithi092.stdout:4/897: dwrite d9/df/d62/d7b/db6/fdd [0,4194304] 0 2024-05-06T12:14:44.900 INFO:tasks.workunit.client.1.smithi190.stdout:6/234: fdatasync fe 0 2024-05-06T12:14:44.900 INFO:tasks.workunit.client.1.smithi190.stdout:3/324: unlink d5/d14/d52/l68 0 2024-05-06T12:14:44.902 INFO:tasks.workunit.client.1.smithi190.stdout:4/202: symlink d2/d11/l45 0 2024-05-06T12:14:44.903 INFO:tasks.workunit.client.1.smithi190.stdout:4/203: dread - d2/d11/f44 zero size 2024-05-06T12:14:44.903 INFO:tasks.workunit.client.1.smithi190.stdout:8/348: creat d3/d8/d25/d3b/d6b/f76 x:0 0 0 2024-05-06T12:14:44.904 INFO:tasks.workunit.client.0.smithi092.stdout:6/915: dread d4/d4c/d52/ddc/fef [0,4194304] 0 2024-05-06T12:14:44.905 INFO:tasks.workunit.client.1.smithi190.stdout:4/204: write d2/d11/d16/f3e [753083,100916] 0 2024-05-06T12:14:44.905 INFO:tasks.workunit.client.1.smithi190.stdout:2/289: link d2/f35 d2/d8/d33/d46/d55/f59 0 2024-05-06T12:14:44.906 INFO:tasks.workunit.client.0.smithi092.stdout:1/812: mkdir d3/d10/d13/d19/d21/d88/d105 0 2024-05-06T12:14:44.906 INFO:tasks.workunit.client.1.smithi190.stdout:2/290: stat d2/c44 0 2024-05-06T12:14:44.908 INFO:tasks.workunit.client.0.smithi092.stdout:6/916: read d4/d4c/da9/fd2 [194677,83777] 0 2024-05-06T12:14:44.908 INFO:tasks.workunit.client.1.smithi190.stdout:1/247: fsync d5/fa 0 2024-05-06T12:14:44.908 INFO:tasks.workunit.client.0.smithi092.stdout:3/664: rename d8/f45 to d8/db/d4c/d40/d7a/d90/fd0 0 2024-05-06T12:14:44.910 INFO:tasks.workunit.client.1.smithi190.stdout:0/420: mkdir d9/d51/d93 0 2024-05-06T12:14:44.911 INFO:tasks.workunit.client.0.smithi092.stdout:4/898: rmdir d9/df/d48 39 2024-05-06T12:14:44.912 INFO:tasks.workunit.client.1.smithi190.stdout:2/291: dwrite d2/fd [0,4194304] 0 2024-05-06T12:14:44.913 INFO:tasks.workunit.client.1.smithi190.stdout:6/235: unlink l6 0 2024-05-06T12:14:44.915 INFO:tasks.workunit.client.1.smithi190.stdout:2/292: write d2/d8/dc/f1a [367205,67875] 0 2024-05-06T12:14:44.916 INFO:tasks.workunit.client.0.smithi092.stdout:7/941: write d9/de/d10/d17/d21/f3e [538342,34437] 0 2024-05-06T12:14:44.917 INFO:tasks.workunit.client.1.smithi190.stdout:8/349: creat d3/d8/d25/d3b/f77 x:0 0 0 2024-05-06T12:14:44.917 INFO:tasks.workunit.client.0.smithi092.stdout:6/917: rmdir d4/d20/d23/d30 39 2024-05-06T12:14:44.917 INFO:tasks.workunit.client.0.smithi092.stdout:7/942: fsync d9/fb 0 2024-05-06T12:14:44.918 INFO:tasks.workunit.client.0.smithi092.stdout:3/665: symlink d8/db/d4c/d3d/ld1 0 2024-05-06T12:14:44.923 INFO:tasks.workunit.client.1.smithi190.stdout:3/325: mknod d5/c77 0 2024-05-06T12:14:44.924 INFO:tasks.workunit.client.1.smithi190.stdout:6/236: mknod df/d2c/d28/c51 0 2024-05-06T12:14:44.925 INFO:tasks.workunit.client.1.smithi190.stdout:7/253: write d0/da/f2e [878002,125890] 0 2024-05-06T12:14:44.926 INFO:tasks.workunit.client.0.smithi092.stdout:7/943: chown d9/de/d10/d17/d6a/f78 664 1 2024-05-06T12:14:44.928 INFO:tasks.workunit.client.0.smithi092.stdout:3/666: write d8/db/d14/d5a/f8f [533704,67741] 0 2024-05-06T12:14:44.930 INFO:tasks.workunit.client.1.smithi190.stdout:1/248: dread d5/d15/f2a [0,4194304] 0 2024-05-06T12:14:44.931 INFO:tasks.workunit.client.0.smithi092.stdout:5/864: truncate d4/d34/f56 740994 0 2024-05-06T12:14:44.932 INFO:tasks.workunit.client.1.smithi190.stdout:3/326: symlink d5/d7/d15/d1f/d33/d40/l78 0 2024-05-06T12:14:44.932 INFO:tasks.workunit.client.1.smithi190.stdout:1/249: chown d5/d23/l2e 3475 1 2024-05-06T12:14:44.933 INFO:tasks.workunit.client.0.smithi092.stdout:7/944: read d9/dd/f100 [200406,5026] 0 2024-05-06T12:14:44.934 INFO:tasks.workunit.client.1.smithi190.stdout:2/293: creat d2/d4c/f5a x:0 0 0 2024-05-06T12:14:44.935 INFO:tasks.workunit.client.1.smithi190.stdout:7/254: dwrite d0/d1/d2f/d3b/f48 [0,4194304] 0 2024-05-06T12:14:44.935 INFO:tasks.workunit.client.1.smithi190.stdout:2/294: stat d2/f3b 0 2024-05-06T12:14:44.935 INFO:tasks.workunit.client.1.smithi190.stdout:1/250: dread - d5/d23/d2b/f35 zero size 2024-05-06T12:14:44.935 INFO:tasks.workunit.client.1.smithi190.stdout:2/295: chown d2 1 1 2024-05-06T12:14:44.935 INFO:tasks.workunit.client.1.smithi190.stdout:6/237: creat df/d2c/d1e/f52 x:0 0 0 2024-05-06T12:14:44.936 INFO:tasks.workunit.client.0.smithi092.stdout:5/865: mknod d4/d34/d28/d4e/d4b/c12f 0 2024-05-06T12:14:44.936 INFO:tasks.workunit.client.1.smithi190.stdout:7/255: fdatasync d0/f20 0 2024-05-06T12:14:44.937 INFO:tasks.workunit.client.0.smithi092.stdout:6/918: write d4/d20/d22/fdf [668487,84156] 0 2024-05-06T12:14:44.937 INFO:tasks.workunit.client.0.smithi092.stdout:6/919: readlink d4/d24/d38/d8c/ld3 0 2024-05-06T12:14:44.938 INFO:tasks.workunit.client.0.smithi092.stdout:4/899: truncate d9/d2b/d5b/daf/ffe 2556984 0 2024-05-06T12:14:44.939 INFO:tasks.workunit.client.1.smithi190.stdout:5/294: getdents d2/da 0 2024-05-06T12:14:44.940 INFO:tasks.workunit.client.1.smithi190.stdout:4/205: link d2/d11/l45 d2/d11/d16/d41/l46 0 2024-05-06T12:14:44.941 INFO:tasks.workunit.client.0.smithi092.stdout:1/813: link d3/l1d d3/d42/d84/d4f/dd5/ded/l106 0 2024-05-06T12:14:44.941 INFO:tasks.workunit.client.0.smithi092.stdout:7/945: truncate d9/de/d10/d17/d21/d2e/fa1 645930 0 2024-05-06T12:14:44.942 INFO:tasks.workunit.client.0.smithi092.stdout:1/814: fdatasync d3/d10/d13/d19/d21/d88/d8b/fc1 0 2024-05-06T12:14:44.942 INFO:tasks.workunit.client.1.smithi190.stdout:5/295: write d2/da/d2c/f43 [1046698,102515] 0 2024-05-06T12:14:44.943 INFO:tasks.workunit.client.0.smithi092.stdout:3/667: write d8/db/d4c/d40/d67/f97 [642100,114192] 0 2024-05-06T12:14:44.944 INFO:tasks.workunit.client.1.smithi190.stdout:1/251: dread d5/d23/d24/d3a/f3b [0,4194304] 0 2024-05-06T12:14:44.947 INFO:tasks.workunit.client.0.smithi092.stdout:5/866: dwrite d4/d34/d28/f72 [4194304,4194304] 0 2024-05-06T12:14:44.947 INFO:tasks.workunit.client.0.smithi092.stdout:4/900: unlink d9/fc0 0 2024-05-06T12:14:44.948 INFO:tasks.workunit.client.0.smithi092.stdout:6/920: mkdir d4/d24/d66/d105/d121 0 2024-05-06T12:14:44.950 INFO:tasks.workunit.client.1.smithi190.stdout:7/256: mkdir d0/d4c/d60 0 2024-05-06T12:14:44.951 INFO:tasks.workunit.client.1.smithi190.stdout:0/421: getdents d9/dc 0 2024-05-06T12:14:44.952 INFO:tasks.workunit.client.0.smithi092.stdout:7/946: write d9/de/d1f/f69 [3278914,89051] 0 2024-05-06T12:14:44.952 INFO:tasks.workunit.client.1.smithi190.stdout:7/257: chown d0/d14/d5d 884 1 2024-05-06T12:14:44.953 INFO:tasks.workunit.client.1.smithi190.stdout:8/350: dwrite d3/d8/d2b/f34 [0,4194304] 0 2024-05-06T12:14:44.955 INFO:tasks.workunit.client.0.smithi092.stdout:5/867: symlink d4/d5/d10/d31/ddf/d6c/l130 0 2024-05-06T12:14:44.955 INFO:tasks.workunit.client.1.smithi190.stdout:8/351: read d3/d8/d2b/d3e/f54 [246987,25642] 0 2024-05-06T12:14:44.958 INFO:tasks.workunit.client.1.smithi190.stdout:5/296: rename d2/da/d2c/l4e to d2/da/d2c/d2e/d3b/l57 0 2024-05-06T12:14:44.959 INFO:tasks.workunit.client.0.smithi092.stdout:7/947: creat d9/de/d1f/d37/d39/f142 x:0 0 0 2024-05-06T12:14:44.960 INFO:tasks.workunit.client.0.smithi092.stdout:7/948: chown d9/de/d10/d17/d6a/f7a 178 1 2024-05-06T12:14:44.961 INFO:tasks.workunit.client.0.smithi092.stdout:6/921: rename d4/d24/l41 to d4/d24/d26/dae/dc0/l122 0 2024-05-06T12:14:44.963 INFO:tasks.workunit.client.1.smithi190.stdout:6/238: mkdir df/d17/d46/d53 0 2024-05-06T12:14:44.963 INFO:tasks.workunit.client.1.smithi190.stdout:1/252: creat d5/d23/d32/d47/f51 x:0 0 0 2024-05-06T12:14:44.964 INFO:tasks.workunit.client.0.smithi092.stdout:5/868: unlink d4/d34/d28/d4e/f4f 0 2024-05-06T12:14:44.965 INFO:tasks.workunit.client.1.smithi190.stdout:3/327: write d5/d14/d11/f1b [3268689,100713] 0 2024-05-06T12:14:44.965 INFO:tasks.workunit.client.0.smithi092.stdout:5/869: fsync d4/d5/d30/f61 0 2024-05-06T12:14:44.965 INFO:tasks.workunit.client.0.smithi092.stdout:1/815: mknod d3/d10/d13/dd2/c107 0 2024-05-06T12:14:44.966 INFO:tasks.workunit.client.1.smithi190.stdout:3/328: chown d5/d14/d11/d2a/d3d 50077640 1 2024-05-06T12:14:44.966 INFO:tasks.workunit.client.1.smithi190.stdout:7/258: symlink d0/d14/l61 0 2024-05-06T12:14:44.966 INFO:tasks.workunit.client.0.smithi092.stdout:5/870: truncate d4/d5/d10/d31/d53/dcb/fff 387510 0 2024-05-06T12:14:44.967 INFO:tasks.workunit.client.1.smithi190.stdout:3/329: write d5/d14/d11/f1b [3953673,107314] 0 2024-05-06T12:14:44.968 INFO:tasks.workunit.client.1.smithi190.stdout:3/330: chown d5/d7/d15/d1f/d33 0 1 2024-05-06T12:14:44.969 INFO:tasks.workunit.client.0.smithi092.stdout:7/949: mknod d9/de/d10/c143 0 2024-05-06T12:14:44.969 INFO:tasks.workunit.client.1.smithi190.stdout:8/352: mknod d3/d8/d68/c78 0 2024-05-06T12:14:44.969 INFO:tasks.workunit.client.1.smithi190.stdout:5/297: write d2/da/d2c/f3c [3309145,71403] 0 2024-05-06T12:14:44.970 INFO:tasks.workunit.client.1.smithi190.stdout:6/239: creat df/d17/d44/f54 x:0 0 0 2024-05-06T12:14:44.971 INFO:tasks.workunit.client.0.smithi092.stdout:3/668: getdents d8/db/d4c/d3d/d49 0 2024-05-06T12:14:44.971 INFO:tasks.workunit.client.1.smithi190.stdout:4/206: link d2/d10/l36 d2/d11/d16/d41/l47 0 2024-05-06T12:14:44.972 INFO:tasks.workunit.client.1.smithi190.stdout:1/253: truncate d5/f8 322180 0 2024-05-06T12:14:44.972 INFO:tasks.workunit.client.0.smithi092.stdout:1/816: creat d3/d42/d84/d4f/d7f/f108 x:0 0 0 2024-05-06T12:14:44.973 INFO:tasks.workunit.client.1.smithi190.stdout:6/240: write df/d2c/f30 [854673,96223] 0 2024-05-06T12:14:44.973 INFO:tasks.workunit.client.0.smithi092.stdout:5/871: mkdir d4/d5/d30/dfd/d131 0 2024-05-06T12:14:44.973 INFO:tasks.workunit.client.0.smithi092.stdout:4/901: link d9/df/d62/c8e d9/d2b/c12c 0 2024-05-06T12:14:44.974 INFO:tasks.workunit.client.1.smithi190.stdout:2/296: dread d2/d8/d19/d4f/f57 [0,4194304] 0 2024-05-06T12:14:44.974 INFO:tasks.workunit.client.0.smithi092.stdout:4/902: chown d9/d27/d4a/fdc 7475841 1 2024-05-06T12:14:44.975 INFO:tasks.workunit.client.1.smithi190.stdout:7/259: rmdir d0/d1/d18 39 2024-05-06T12:14:44.975 INFO:tasks.workunit.client.0.smithi092.stdout:6/922: mknod d4/d24/d4e/d10a/c123 0 2024-05-06T12:14:44.977 INFO:tasks.workunit.client.0.smithi092.stdout:7/950: creat d9/de/d10/d17/d99/db6/df1/f144 x:0 0 0 2024-05-06T12:14:44.977 INFO:tasks.workunit.client.0.smithi092.stdout:3/669: mkdir d8/db/d4c/d3d/d49/dd2 0 2024-05-06T12:14:44.978 INFO:tasks.workunit.client.1.smithi190.stdout:8/353: creat d3/d8/d25/d3b/f79 x:0 0 0 2024-05-06T12:14:44.979 INFO:tasks.workunit.client.0.smithi092.stdout:5/872: symlink d4/d34/d28/d4e/d4b/l132 0 2024-05-06T12:14:44.979 INFO:tasks.workunit.client.1.smithi190.stdout:0/422: link d9/df/f2e d9/dc/d48/d50/d83/f94 0 2024-05-06T12:14:44.979 INFO:tasks.workunit.client.1.smithi190.stdout:5/298: dread d2/da/d2c/f43 [0,4194304] 0 2024-05-06T12:14:44.980 INFO:tasks.workunit.client.0.smithi092.stdout:5/873: fdatasync d4/d34/d28/d4e/d67/f79 0 2024-05-06T12:14:44.982 INFO:tasks.workunit.client.0.smithi092.stdout:7/951: unlink d9/de/d10/d17/d99/db6/df1/f98 0 2024-05-06T12:14:44.983 INFO:tasks.workunit.client.1.smithi190.stdout:8/354: write d3/d8/d25/d3b/f79 [730564,115455] 0 2024-05-06T12:14:44.984 INFO:tasks.workunit.client.0.smithi092.stdout:4/903: readlink d9/df/d48/l5a 0 2024-05-06T12:14:44.984 INFO:tasks.workunit.client.0.smithi092.stdout:5/874: fdatasync d4/d34/d28/d4e/d4b/f8c 0 2024-05-06T12:14:44.985 INFO:tasks.workunit.client.1.smithi190.stdout:3/331: mkdir d5/d79 0 2024-05-06T12:14:44.988 INFO:tasks.workunit.client.0.smithi092.stdout:5/875: unlink d4/d5/d10/d31/d53/d64/f5e 0 2024-05-06T12:14:44.988 INFO:tasks.workunit.client.1.smithi190.stdout:1/254: dwrite d5/d23/d32/f37 [0,4194304] 0 2024-05-06T12:14:44.989 INFO:tasks.workunit.client.0.smithi092.stdout:5/876: write d4/d5/d10/d31/ddf/d54/db0/d102/d78/f124 [990147,5419] 0 2024-05-06T12:14:44.990 INFO:tasks.workunit.client.1.smithi190.stdout:4/207: creat d2/f48 x:0 0 0 2024-05-06T12:14:44.991 INFO:tasks.workunit.client.1.smithi190.stdout:2/297: rmdir d2/d10/d4a 0 2024-05-06T12:14:44.991 INFO:tasks.workunit.client.1.smithi190.stdout:6/241: dwrite df/d17/d37/f3c [0,4194304] 0 2024-05-06T12:14:44.992 INFO:tasks.workunit.client.0.smithi092.stdout:4/904: creat d9/df/f12d x:0 0 0 2024-05-06T12:14:44.993 INFO:tasks.workunit.client.0.smithi092.stdout:7/952: dread d9/de/d10/d17/d99/f83 [0,4194304] 0 2024-05-06T12:14:44.993 INFO:tasks.workunit.client.1.smithi190.stdout:6/242: chown df/d17/d46/d53 29 1 2024-05-06T12:14:44.993 INFO:tasks.workunit.client.1.smithi190.stdout:4/208: fdatasync d2/d11/d16/f3e 0 2024-05-06T12:14:44.994 INFO:tasks.workunit.client.0.smithi092.stdout:5/877: creat d4/d5/d30/dfd/d108/f133 x:0 0 0 2024-05-06T12:14:44.995 INFO:tasks.workunit.client.1.smithi190.stdout:8/355: mkdir d3/d8/d25/d3b/d6b/d7a 0 2024-05-06T12:14:44.998 INFO:tasks.workunit.client.1.smithi190.stdout:3/332: fsync d5/d14/d11/d21/f38 0 2024-05-06T12:14:45.000 INFO:tasks.workunit.client.0.smithi092.stdout:7/953: rmdir d9/de/d10/df0/df5 39 2024-05-06T12:14:45.001 INFO:tasks.workunit.client.1.smithi190.stdout:7/260: link d0/d1/d18/d2b/f30 d0/da/d36/d53/f62 0 2024-05-06T12:14:45.002 INFO:tasks.workunit.client.1.smithi190.stdout:6/243: dwrite f1 [0,4194304] 0 2024-05-06T12:14:45.002 INFO:tasks.workunit.client.1.smithi190.stdout:1/255: creat d5/d15/f52 x:0 0 0 2024-05-06T12:14:45.004 INFO:tasks.workunit.client.1.smithi190.stdout:6/244: write df/d12/f2b [106970,115282] 0 2024-05-06T12:14:45.004 INFO:tasks.workunit.client.0.smithi092.stdout:5/878: dwrite d4/d5/d10/d31/d53/dcb/fff [0,4194304] 0 2024-05-06T12:14:45.004 INFO:tasks.workunit.client.1.smithi190.stdout:2/298: creat d2/d8/dc/f5b x:0 0 0 2024-05-06T12:14:45.008 INFO:tasks.workunit.client.0.smithi092.stdout:5/879: write d4/d34/d28/d4e/d5d/d94/f109 [431279,52188] 0 2024-05-06T12:14:45.008 INFO:tasks.workunit.client.0.smithi092.stdout:3/670: rmdir d8/db/d4c/d3d 39 2024-05-06T12:14:45.012 INFO:tasks.workunit.client.1.smithi190.stdout:5/299: creat d2/da/d2c/f58 x:0 0 0 2024-05-06T12:14:45.017 INFO:tasks.workunit.client.0.smithi092.stdout:1/817: dread d3/d10/d13/d19/d21/fb4 [0,4194304] 0 2024-05-06T12:14:45.020 INFO:tasks.workunit.client.1.smithi190.stdout:0/423: write d9/df/d56/f57 [715404,6220] 0 2024-05-06T12:14:45.021 INFO:tasks.workunit.client.0.smithi092.stdout:6/923: dwrite d4/d4c/d91/d93/d9d/fbf [0,4194304] 0 2024-05-06T12:14:45.022 INFO:tasks.workunit.client.1.smithi190.stdout:2/299: stat d2/l9 0 2024-05-06T12:14:45.022 INFO:tasks.workunit.client.1.smithi190.stdout:0/424: chown d9/df/d21/d72 0 1 2024-05-06T12:14:45.023 INFO:tasks.workunit.client.0.smithi092.stdout:1/818: getdents d3/d42/d84/dcf 0 2024-05-06T12:14:45.023 INFO:tasks.workunit.client.1.smithi190.stdout:0/425: chown d9/d91 846816516 1 2024-05-06T12:14:45.023 INFO:tasks.workunit.client.1.smithi190.stdout:2/300: readlink d2/d21/l1c 0 2024-05-06T12:14:45.023 INFO:tasks.workunit.client.1.smithi190.stdout:2/301: fdatasync d2/d21/f42 0 2024-05-06T12:14:45.024 INFO:tasks.workunit.client.0.smithi092.stdout:6/924: write d4/d4c/fc8 [91211,78003] 0 2024-05-06T12:14:45.025 INFO:tasks.workunit.client.1.smithi190.stdout:4/209: symlink d2/l49 0 2024-05-06T12:14:45.025 INFO:tasks.workunit.client.1.smithi190.stdout:2/302: write d2/fd [5365426,83534] 0 2024-05-06T12:14:45.026 INFO:tasks.workunit.client.1.smithi190.stdout:4/210: fdatasync d2/f25 0 2024-05-06T12:14:45.027 INFO:tasks.workunit.client.0.smithi092.stdout:1/819: creat d3/d10/d13/d19/d57/dd6/f109 x:0 0 0 2024-05-06T12:14:45.028 INFO:tasks.workunit.client.1.smithi190.stdout:5/300: mkdir d2/da/d59 0 2024-05-06T12:14:45.029 INFO:tasks.workunit.client.1.smithi190.stdout:5/301: write d2/da/f2f [594072,10086] 0 2024-05-06T12:14:45.029 INFO:tasks.workunit.client.0.smithi092.stdout:4/905: dread d9/d2b/f7d [0,4194304] 0 2024-05-06T12:14:45.030 INFO:tasks.workunit.client.0.smithi092.stdout:6/925: creat d4/d24/d38/f124 x:0 0 0 2024-05-06T12:14:45.031 INFO:tasks.workunit.client.0.smithi092.stdout:6/926: dread - d4/d4c/d52/d6c/ffa zero size 2024-05-06T12:14:45.032 INFO:tasks.workunit.client.1.smithi190.stdout:3/333: symlink d5/d14/d11/d21/l7a 0 2024-05-06T12:14:45.034 INFO:tasks.workunit.client.0.smithi092.stdout:6/927: stat d4/d24/d38/d5d/d6d/f84 0 2024-05-06T12:14:45.037 INFO:tasks.workunit.client.0.smithi092.stdout:3/671: write d8/db/d4c/d40/d7a/d90/fd0 [574243,66209] 0 2024-05-06T12:14:45.037 INFO:tasks.workunit.client.1.smithi190.stdout:6/245: dread f2 [0,4194304] 0 2024-05-06T12:14:45.039 INFO:tasks.workunit.client.0.smithi092.stdout:1/820: dwrite d3/d10/d13/d19/d21/fb4 [0,4194304] 0 2024-05-06T12:14:45.042 INFO:tasks.workunit.client.0.smithi092.stdout:4/906: dwrite d9/d2b/d7c/dab/dd8/ff7 [0,4194304] 0 2024-05-06T12:14:45.043 INFO:tasks.workunit.client.1.smithi190.stdout:5/302: mkdir d2/da/d2c/d2e/d3b/d5a 0 2024-05-06T12:14:45.044 INFO:tasks.workunit.client.0.smithi092.stdout:6/928: dread d4/d24/d26/dc4/fee [0,4194304] 0 2024-05-06T12:14:45.047 INFO:tasks.workunit.client.1.smithi190.stdout:0/426: unlink d9/f26 0 2024-05-06T12:14:45.048 INFO:tasks.workunit.client.0.smithi092.stdout:4/907: mkdir d9/d27/da8/db3/d124/d12e 0 2024-05-06T12:14:45.050 INFO:tasks.workunit.client.0.smithi092.stdout:4/908: read - d9/d2b/d7c/f11c zero size 2024-05-06T12:14:45.050 INFO:tasks.workunit.client.0.smithi092.stdout:3/672: dread d8/db/d4c/d40/f52 [4194304,4194304] 0 2024-05-06T12:14:45.051 INFO:tasks.workunit.client.0.smithi092.stdout:6/929: chown d4/d20/d23/d30/f36 261 1 2024-05-06T12:14:45.051 INFO:tasks.workunit.client.0.smithi092.stdout:5/880: dread d4/d34/d28/d4e/f71 [0,4194304] 0 2024-05-06T12:14:45.052 INFO:tasks.workunit.client.1.smithi190.stdout:0/427: write d9/f52 [647730,64280] 0 2024-05-06T12:14:45.054 INFO:tasks.workunit.client.0.smithi092.stdout:5/881: fdatasync d4/d34/d28/d4e/d4b/dd6/ff7 0 2024-05-06T12:14:45.054 INFO:tasks.workunit.client.1.smithi190.stdout:6/246: mknod df/d2c/d28/c55 0 2024-05-06T12:14:45.054 INFO:tasks.workunit.client.1.smithi190.stdout:4/211: mknod d2/c4a 0 2024-05-06T12:14:45.055 INFO:tasks.workunit.client.0.smithi092.stdout:6/930: creat d4/d4c/d52/ddc/f125 x:0 0 0 2024-05-06T12:14:45.056 INFO:tasks.workunit.client.0.smithi092.stdout:3/673: creat d8/d24/d33/da8/fd3 x:0 0 0 2024-05-06T12:14:45.058 INFO:tasks.workunit.client.1.smithi190.stdout:6/247: write df/d2c/f36 [512373,17889] 0 2024-05-06T12:14:45.060 INFO:tasks.workunit.client.1.smithi190.stdout:4/212: rename d2/d11 to d2/d11/d4b 22 2024-05-06T12:14:45.060 INFO:tasks.workunit.client.1.smithi190.stdout:5/303: dread d2/da/d2c/f3c [0,4194304] 0 2024-05-06T12:14:45.061 INFO:tasks.workunit.client.0.smithi092.stdout:1/821: write d3/d42/fe2 [1002826,122788] 0 2024-05-06T12:14:45.062 INFO:tasks.workunit.client.0.smithi092.stdout:6/931: creat d4/d24/d38/d5d/d6d/dd8/d10f/f126 x:0 0 0 2024-05-06T12:14:45.062 INFO:tasks.workunit.client.0.smithi092.stdout:5/882: dwrite d4/d34/d28/d4e/d67/f79 [0,4194304] 0 2024-05-06T12:14:45.062 INFO:tasks.workunit.client.1.smithi190.stdout:1/256: dwrite d5/d15/d1e/f27 [0,4194304] 0 2024-05-06T12:14:45.063 INFO:tasks.workunit.client.1.smithi190.stdout:7/261: dread d0/da/d36/f46 [0,4194304] 0 2024-05-06T12:14:45.064 INFO:tasks.workunit.client.0.smithi092.stdout:3/674: rmdir d8/db/d4c/d3d/d48 39 2024-05-06T12:14:45.064 INFO:tasks.workunit.client.0.smithi092.stdout:4/909: write d9/df/d62/d7b/f112 [1072432,96241] 0 2024-05-06T12:14:45.065 INFO:tasks.workunit.client.1.smithi190.stdout:5/304: write d2/f28 [395086,129] 0 2024-05-06T12:14:45.066 INFO:tasks.workunit.client.1.smithi190.stdout:9/447: sync 2024-05-06T12:14:45.066 INFO:tasks.workunit.client.0.smithi092.stdout:5/883: dread d4/f9 [0,4194304] 0 2024-05-06T12:14:45.067 INFO:tasks.workunit.client.0.smithi092.stdout:1/822: mkdir d3/d10/d13/d19/d34/d89/d10a 0 2024-05-06T12:14:45.068 INFO:tasks.workunit.client.0.smithi092.stdout:6/932: creat d4/d24/d66/d105/d121/f127 x:0 0 0 2024-05-06T12:14:45.069 INFO:tasks.workunit.client.1.smithi190.stdout:6/248: dread - df/f15 zero size 2024-05-06T12:14:45.070 INFO:tasks.workunit.client.0.smithi092.stdout:5/884: symlink d4/d5/l134 0 2024-05-06T12:14:45.072 INFO:tasks.workunit.client.1.smithi190.stdout:2/303: write d2/f35 [4163948,86270] 0 2024-05-06T12:14:45.073 INFO:tasks.workunit.client.1.smithi190.stdout:7/262: rmdir d0/d14/d5d 39 2024-05-06T12:14:45.074 INFO:tasks.workunit.client.1.smithi190.stdout:3/334: getdents d5/d7/d15/d1f/d33/d5c 0 2024-05-06T12:14:45.074 INFO:tasks.workunit.client.1.smithi190.stdout:8/356: dwrite d3/f18 [0,4194304] 0 2024-05-06T12:14:45.075 INFO:tasks.workunit.client.1.smithi190.stdout:1/257: mkdir d5/d23/d32/d47/d53 0 2024-05-06T12:14:45.075 INFO:tasks.workunit.client.1.smithi190.stdout:9/448: stat db/dd/d16/d58/d87/d22/d49/d8c/fa0 0 2024-05-06T12:14:45.076 INFO:tasks.workunit.client.0.smithi092.stdout:3/675: creat d8/db/d14/fd4 x:0 0 0 2024-05-06T12:14:45.077 INFO:tasks.workunit.client.1.smithi190.stdout:7/263: chown d0/d1/d2f/d3b 15842 1 2024-05-06T12:14:45.077 INFO:tasks.workunit.client.1.smithi190.stdout:8/357: chown d3/f12 5 1 2024-05-06T12:14:45.078 INFO:tasks.workunit.client.1.smithi190.stdout:1/258: read - d5/d23/f3c zero size 2024-05-06T12:14:45.079 INFO:tasks.workunit.client.1.smithi190.stdout:1/259: chown d5/d23/d24 133868 1 2024-05-06T12:14:45.079 INFO:tasks.workunit.client.1.smithi190.stdout:3/335: write d5/d7/d15/d1f/d33/f76 [25834,104222] 0 2024-05-06T12:14:45.080 INFO:tasks.workunit.client.1.smithi190.stdout:8/358: chown d3/d13/d4a/c59 80823780 1 2024-05-06T12:14:45.081 INFO:tasks.workunit.client.0.smithi092.stdout:4/910: dwrite d9/d2b/f6a [4194304,4194304] 0 2024-05-06T12:14:45.081 INFO:tasks.workunit.client.1.smithi190.stdout:4/213: link d2/f4 d2/d11/f4c 0 2024-05-06T12:14:45.081 INFO:tasks.workunit.client.1.smithi190.stdout:9/449: chown db/d23/d25/d47/d4b/f66 0 1 2024-05-06T12:14:45.082 INFO:tasks.workunit.client.1.smithi190.stdout:8/359: chown d3/d35/c69 99702 1 2024-05-06T12:14:45.082 INFO:tasks.workunit.client.0.smithi092.stdout:1/823: creat d3/d10/d13/d19/d21/f10b x:0 0 0 2024-05-06T12:14:45.085 INFO:tasks.workunit.client.0.smithi092.stdout:4/911: rmdir d9/dd/da6 39 2024-05-06T12:14:45.086 INFO:tasks.workunit.client.0.smithi092.stdout:4/912: fsync d9/d27/da8/db3/fe7 0 2024-05-06T12:14:45.087 INFO:tasks.workunit.client.0.smithi092.stdout:5/885: dwrite d4/d5/d10/d31/ddf/d6c/fe4 [0,4194304] 0 2024-05-06T12:14:45.088 INFO:tasks.workunit.client.1.smithi190.stdout:1/260: read d5/d23/d24/f2f [7706355,63327] 0 2024-05-06T12:14:45.089 INFO:tasks.workunit.client.1.smithi190.stdout:5/305: rename d2/d27/f56 to d2/da/d2c/f5b 0 2024-05-06T12:14:45.094 INFO:tasks.workunit.client.0.smithi092.stdout:1/824: dwrite d3/d42/d84/d4f/d7f/f108 [0,4194304] 0 2024-05-06T12:14:45.096 INFO:tasks.workunit.client.1.smithi190.stdout:9/450: dwrite db/dd/d16/d58/d87/d22/d49/f89 [0,4194304] 0 2024-05-06T12:14:45.097 INFO:tasks.workunit.client.1.smithi190.stdout:0/428: write d9/f1a [1791885,70218] 0 2024-05-06T12:14:45.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:44 smithi190 ceph-mon[33798]: pgmap v24: 65 pgs: 65 active+clean; 2.3 GiB data, 7.9 GiB used, 529 GiB / 536 GiB avail; 81 MiB/s rd, 289 MiB/s wr, 645 op/s 2024-05-06T12:14:45.102 INFO:tasks.workunit.client.0.smithi092.stdout:1/825: rmdir d3/d42/d84/d4f/dd5/ded 39 2024-05-06T12:14:45.102 INFO:tasks.workunit.client.0.smithi092.stdout:3/676: write f3 [3677278,94644] 0 2024-05-06T12:14:45.102 INFO:tasks.workunit.client.0.smithi092.stdout:6/933: write d4/d20/d22/f7a [517285,93817] 0 2024-05-06T12:14:45.103 INFO:tasks.workunit.client.0.smithi092.stdout:4/913: getdents d9/d27/da8/db3/d124 0 2024-05-06T12:14:45.104 INFO:tasks.workunit.client.1.smithi190.stdout:6/249: write df/d17/f26 [761405,52425] 0 2024-05-06T12:14:45.104 INFO:tasks.workunit.client.1.smithi190.stdout:5/306: truncate d2/da/d2c/d50/f51 737307 0 2024-05-06T12:14:45.104 INFO:tasks.workunit.client.1.smithi190.stdout:4/214: rename d2/d11/f44 to d2/f4d 0 2024-05-06T12:14:45.106 INFO:tasks.workunit.client.1.smithi190.stdout:1/261: dwrite d5/d23/d32/d47/f4c [0,4194304] 0 2024-05-06T12:14:45.106 INFO:tasks.workunit.client.0.smithi092.stdout:3/677: unlink d8/d24/d33/f92 0 2024-05-06T12:14:45.109 INFO:tasks.workunit.client.0.smithi092.stdout:6/934: mknod d4/d20/d23/d30/c128 0 2024-05-06T12:14:45.109 INFO:tasks.workunit.client.0.smithi092.stdout:4/914: read d9/d27/f7f [1569541,111410] 0 2024-05-06T12:14:45.111 INFO:tasks.workunit.client.0.smithi092.stdout:3/678: fsync d8/d24/d33/f62 0 2024-05-06T12:14:45.111 INFO:tasks.workunit.client.0.smithi092.stdout:3/679: chown d8/db/d4c 1397205272 1 2024-05-06T12:14:45.112 INFO:tasks.workunit.client.1.smithi190.stdout:2/304: truncate d2/f12 88395 0 2024-05-06T12:14:45.112 INFO:tasks.workunit.client.0.smithi092.stdout:6/935: symlink d4/d24/d38/l129 0 2024-05-06T12:14:45.113 INFO:tasks.workunit.client.1.smithi190.stdout:0/429: dread d9/dc/d48/d50/f66 [0,4194304] 0 2024-05-06T12:14:45.115 INFO:tasks.workunit.client.1.smithi190.stdout:5/307: symlink d2/da/d2c/d2e/l5c 0 2024-05-06T12:14:45.120 INFO:tasks.workunit.client.0.smithi092.stdout:1/826: read d3/d10/d13/d19/d34/f66 [222486,119552] 0 2024-05-06T12:14:45.120 INFO:tasks.workunit.client.1.smithi190.stdout:1/262: unlink d5/f2d 0 2024-05-06T12:14:45.120 INFO:tasks.workunit.client.0.smithi092.stdout:3/680: creat d8/d1b/fd5 x:0 0 0 2024-05-06T12:14:45.122 INFO:tasks.workunit.client.1.smithi190.stdout:5/308: read d2/da/d2c/d2e/d3b/f52 [665306,31662] 0 2024-05-06T12:14:45.122 INFO:tasks.workunit.client.1.smithi190.stdout:6/250: dwrite df/d2c/d1e/f52 [0,4194304] 0 2024-05-06T12:14:45.123 INFO:tasks.workunit.client.1.smithi190.stdout:7/264: write d0/d4/fd [2397816,25335] 0 2024-05-06T12:14:45.124 INFO:tasks.workunit.client.1.smithi190.stdout:9/451: dwrite db/dd/d16/d58/d87/f70 [0,4194304] 0 2024-05-06T12:14:45.126 INFO:tasks.workunit.client.1.smithi190.stdout:2/305: mknod d2/d8/d19/c5c 0 2024-05-06T12:14:45.128 INFO:tasks.workunit.client.1.smithi190.stdout:4/215: dread d2/d11/f20 [0,4194304] 0 2024-05-06T12:14:45.130 INFO:tasks.workunit.client.0.smithi092.stdout:5/886: dwrite d4/d5/d10/d31/ddf/da3/fbf [0,4194304] 0 2024-05-06T12:14:45.131 INFO:tasks.workunit.client.1.smithi190.stdout:0/430: truncate d9/dc/d28/d4c/f53 752343 0 2024-05-06T12:14:45.131 INFO:tasks.workunit.client.1.smithi190.stdout:2/306: chown d2/d8/dc/c29 605449 1 2024-05-06T12:14:45.132 INFO:tasks.workunit.client.1.smithi190.stdout:8/360: rename d3/d8/d25/d3b/f77 to d3/d8/d25/f7b 0 2024-05-06T12:14:45.134 INFO:tasks.workunit.client.1.smithi190.stdout:5/309: chown d2/c14 69 1 2024-05-06T12:14:45.138 INFO:tasks.workunit.client.1.smithi190.stdout:5/310: fdatasync d2/da/f4a 0 2024-05-06T12:14:45.138 INFO:tasks.workunit.client.1.smithi190.stdout:0/431: stat d9/df/f2e 0 2024-05-06T12:14:45.139 INFO:tasks.workunit.client.1.smithi190.stdout:4/216: mkdir d2/d11/d16/d4e 0 2024-05-06T12:14:45.143 INFO:tasks.workunit.client.1.smithi190.stdout:5/311: chown d2/da/l45 1671 1 2024-05-06T12:14:45.143 INFO:tasks.workunit.client.0.smithi092.stdout:6/936: truncate d4/d24/d38/d5d/d6d/fa5 4196301 0 2024-05-06T12:14:45.144 INFO:tasks.workunit.client.1.smithi190.stdout:3/336: dread d5/fc [0,4194304] 0 2024-05-06T12:14:45.144 INFO:tasks.workunit.client.0.smithi092.stdout:4/915: truncate d9/df/d48/f69 1207845 0 2024-05-06T12:14:45.145 INFO:tasks.workunit.client.0.smithi092.stdout:5/887: dread d4/d59/fd5 [0,4194304] 0 2024-05-06T12:14:45.147 INFO:tasks.workunit.client.0.smithi092.stdout:1/827: link d3/d10/d13/d19/d21/d88/c9e d3/d10/d13/d19/d21/d72/d9a/da4/deb/c10c 0 2024-05-06T12:14:45.147 INFO:tasks.workunit.client.1.smithi190.stdout:1/263: dwrite d5/d15/f17 [0,4194304] 0 2024-05-06T12:14:45.148 INFO:tasks.workunit.client.1.smithi190.stdout:7/265: creat d0/da/d3d/f63 x:0 0 0 2024-05-06T12:14:45.149 INFO:tasks.workunit.client.0.smithi092.stdout:1/828: read - d3/d42/d84/d4f/fec zero size 2024-05-06T12:14:45.149 INFO:tasks.workunit.client.0.smithi092.stdout:6/937: dread d4/d20/d23/fe9 [0,4194304] 0 2024-05-06T12:14:45.150 INFO:tasks.workunit.client.1.smithi190.stdout:8/361: write d3/d8/f27 [622421,92781] 0 2024-05-06T12:14:45.152 INFO:tasks.workunit.client.0.smithi092.stdout:1/829: write d3/f3d [1790545,44580] 0 2024-05-06T12:14:45.153 INFO:tasks.workunit.client.1.smithi190.stdout:7/266: readlink d0/d1/d2/l4b 0 2024-05-06T12:14:45.153 INFO:tasks.workunit.client.0.smithi092.stdout:4/916: link d9/d27/dc5/lf0 d9/d119/l12f 0 2024-05-06T12:14:45.154 INFO:tasks.workunit.client.1.smithi190.stdout:2/307: mkdir d2/d8/d33/d46/d5d 0 2024-05-06T12:14:45.154 INFO:tasks.workunit.client.1.smithi190.stdout:6/251: dwrite df/d2c/f45 [0,4194304] 0 2024-05-06T12:14:45.156 INFO:tasks.workunit.client.0.smithi092.stdout:4/917: mknod d9/d27/da8/db3/c130 0 2024-05-06T12:14:45.156 INFO:tasks.workunit.client.1.smithi190.stdout:9/452: dwrite db/dd/d16/d4c/d4e/f82 [0,4194304] 0 2024-05-06T12:14:45.157 INFO:tasks.workunit.client.0.smithi092.stdout:4/918: chown d9/f1c 126291 1 2024-05-06T12:14:45.157 INFO:tasks.workunit.client.0.smithi092.stdout:1/830: rmdir d3/d42/d84/d4f/d58/d65/d81/d8d 39 2024-05-06T12:14:45.157 INFO:tasks.workunit.client.1.smithi190.stdout:2/308: readlink d2/l26 0 2024-05-06T12:14:45.158 INFO:tasks.workunit.client.1.smithi190.stdout:1/264: read d5/d23/d24/d3a/f3b [247900,75289] 0 2024-05-06T12:14:45.159 INFO:tasks.workunit.client.1.smithi190.stdout:2/309: chown d2/f1f 602 1 2024-05-06T12:14:45.159 INFO:tasks.workunit.client.1.smithi190.stdout:5/312: unlink d2/f26 0 2024-05-06T12:14:45.160 INFO:tasks.workunit.client.1.smithi190.stdout:0/432: mkdir d9/d95 0 2024-05-06T12:14:45.162 INFO:tasks.workunit.client.1.smithi190.stdout:2/310: write d2/d8/dc/d20/f52 [577678,50420] 0 2024-05-06T12:14:45.162 INFO:tasks.workunit.client.1.smithi190.stdout:9/453: chown db/dd/d16/f64 147 1 2024-05-06T12:14:45.164 INFO:tasks.workunit.client.0.smithi092.stdout:4/919: dwrite d9/d2b/f6a [4194304,4194304] 0 2024-05-06T12:14:45.164 INFO:tasks.workunit.client.1.smithi190.stdout:3/337: mknod d5/d7/d6d/c7b 0 2024-05-06T12:14:45.165 INFO:tasks.workunit.client.1.smithi190.stdout:6/252: mkdir df/d17/d44/d56 0 2024-05-06T12:14:45.167 INFO:tasks.workunit.client.0.smithi092.stdout:4/920: chown d9/d27/d4a/f84 59920 1 2024-05-06T12:14:45.169 INFO:tasks.workunit.client.1.smithi190.stdout:1/265: mknod d5/d15/d1e/c54 0 2024-05-06T12:14:45.170 INFO:tasks.workunit.client.1.smithi190.stdout:5/313: dread d2/da/f2f [0,4194304] 0 2024-05-06T12:14:45.171 INFO:tasks.workunit.client.0.smithi092.stdout:4/921: link d9/d27/f4f d9/df/d62/d7b/f131 0 2024-05-06T12:14:45.172 INFO:tasks.workunit.client.1.smithi190.stdout:0/433: mkdir d9/dc/d28/d4c/d96 0 2024-05-06T12:14:45.173 INFO:tasks.workunit.client.0.smithi092.stdout:4/922: chown d9/df/l1b 78927893 1 2024-05-06T12:14:45.173 INFO:tasks.workunit.client.0.smithi092.stdout:4/923: chown d9/d27/l55 13585 1 2024-05-06T12:14:45.174 INFO:tasks.workunit.client.1.smithi190.stdout:0/434: chown d9/dc/d28 416 1 2024-05-06T12:14:45.175 INFO:tasks.workunit.client.0.smithi092.stdout:4/924: creat d9/df/d62/d10f/f132 x:0 0 0 2024-05-06T12:14:45.176 INFO:tasks.workunit.client.0.smithi092.stdout:4/925: stat d9/f24 0 2024-05-06T12:14:45.177 INFO:tasks.workunit.client.1.smithi190.stdout:2/311: creat d2/d21/f5e x:0 0 0 2024-05-06T12:14:45.178 INFO:tasks.workunit.client.0.smithi092.stdout:4/926: unlink d9/d27/d4a/f44 0 2024-05-06T12:14:45.180 INFO:tasks.workunit.client.1.smithi190.stdout:6/253: dread df/d17/f26 [0,4194304] 0 2024-05-06T12:14:45.180 INFO:tasks.workunit.client.1.smithi190.stdout:8/362: link d3/f2e d3/d13/d4a/f7c 0 2024-05-06T12:14:45.182 INFO:tasks.workunit.client.0.smithi092.stdout:6/938: dread d4/f6 [0,4194304] 0 2024-05-06T12:14:45.183 INFO:tasks.workunit.client.1.smithi190.stdout:1/266: mknod d5/d23/d32/c55 0 2024-05-06T12:14:45.185 INFO:tasks.workunit.client.1.smithi190.stdout:9/454: mkdir db/d23/da4 0 2024-05-06T12:14:45.185 INFO:tasks.workunit.client.0.smithi092.stdout:6/939: mkdir d4/d4c/d91/d93/d9d/d12a 0 2024-05-06T12:14:45.186 INFO:tasks.workunit.client.0.smithi092.stdout:6/940: chown d4/d24/d26/f6e 9667 1 2024-05-06T12:14:45.186 INFO:tasks.workunit.client.1.smithi190.stdout:9/455: chown db/d23/l78 1 1 2024-05-06T12:14:45.187 INFO:tasks.workunit.client.1.smithi190.stdout:2/312: creat d2/d8/d33/d46/f5f x:0 0 0 2024-05-06T12:14:45.187 INFO:tasks.workunit.client.1.smithi190.stdout:8/363: creat d3/d13/f7d x:0 0 0 2024-05-06T12:14:45.188 INFO:tasks.workunit.client.0.smithi092.stdout:6/941: dread d4/d24/d38/d5d/d6d/dd8/ff0 [0,4194304] 0 2024-05-06T12:14:45.189 INFO:tasks.workunit.client.1.smithi190.stdout:6/254: rename df/d17/d46 to df/d2c/d1e/d57 0 2024-05-06T12:14:45.190 INFO:tasks.workunit.client.1.smithi190.stdout:6/255: write df/d2c/d1e/f20 [4413939,128216] 0 2024-05-06T12:14:45.195 INFO:tasks.workunit.client.1.smithi190.stdout:3/338: dread d5/f62 [0,4194304] 0 2024-05-06T12:14:45.195 INFO:tasks.workunit.client.1.smithi190.stdout:6/256: truncate df/d12/f21 278235 0 2024-05-06T12:14:45.195 INFO:tasks.workunit.client.1.smithi190.stdout:1/267: mknod d5/d23/d2b/c56 0 2024-05-06T12:14:45.195 INFO:tasks.workunit.client.1.smithi190.stdout:1/268: dread - d5/f34 zero size 2024-05-06T12:14:45.196 INFO:tasks.workunit.client.1.smithi190.stdout:9/456: creat db/d23/d29/fa5 x:0 0 0 2024-05-06T12:14:45.198 INFO:tasks.workunit.client.1.smithi190.stdout:2/313: mknod d2/d10/c60 0 2024-05-06T12:14:45.198 INFO:tasks.workunit.client.1.smithi190.stdout:3/339: dread d5/d7/d15/f4b [0,4194304] 0 2024-05-06T12:14:45.199 INFO:tasks.workunit.client.1.smithi190.stdout:9/457: read db/f77 [549552,79965] 0 2024-05-06T12:14:45.199 INFO:tasks.workunit.client.0.smithi092.stdout:1/831: dread d3/d10/d13/d19/f5d [0,4194304] 0 2024-05-06T12:14:45.200 INFO:tasks.workunit.client.0.smithi092.stdout:5/888: write d4/d5/d10/d31/ddf/d54/db0/d102/d20/fa4 [22482,70142] 0 2024-05-06T12:14:45.200 INFO:tasks.workunit.client.1.smithi190.stdout:2/314: write d2/fd [131953,48865] 0 2024-05-06T12:14:45.201 INFO:tasks.workunit.client.0.smithi092.stdout:1/832: stat d3/d42/d84/d4f/d58/d65/d81/f8a 0 2024-05-06T12:14:45.201 INFO:tasks.workunit.client.0.smithi092.stdout:5/889: fdatasync d4/d5/d30/f40 0 2024-05-06T12:14:45.201 INFO:tasks.workunit.client.1.smithi190.stdout:1/269: dread d5/d15/f30 [0,4194304] 0 2024-05-06T12:14:45.203 INFO:tasks.workunit.client.1.smithi190.stdout:9/458: write db/dd/d16/d4c/d4e/d5e/f92 [735944,98221] 0 2024-05-06T12:14:45.204 INFO:tasks.workunit.client.1.smithi190.stdout:1/270: chown d5/d23/d32 815215 1 2024-05-06T12:14:45.205 INFO:tasks.workunit.client.1.smithi190.stdout:5/314: link d2/c14 d2/da/c5d 0 2024-05-06T12:14:45.207 INFO:tasks.workunit.client.1.smithi190.stdout:6/257: creat df/d17/d37/f58 x:0 0 0 2024-05-06T12:14:45.208 INFO:tasks.workunit.client.0.smithi092.stdout:1/833: symlink d3/daf/l10d 0 2024-05-06T12:14:45.208 INFO:tasks.workunit.client.1.smithi190.stdout:8/364: dwrite d3/d8/d25/f7b [0,4194304] 0 2024-05-06T12:14:45.209 INFO:tasks.workunit.client.0.smithi092.stdout:6/942: write d4/d4c/da9/fd2 [4713452,110215] 0 2024-05-06T12:14:45.209 INFO:tasks.workunit.client.1.smithi190.stdout:0/435: creat d9/df/d21/f97 x:0 0 0 2024-05-06T12:14:45.211 INFO:tasks.workunit.client.0.smithi092.stdout:5/890: dwrite d4/d5/d10/d31/ddf/d54/db0/d102/d20/fa4 [0,4194304] 0 2024-05-06T12:14:45.212 INFO:tasks.workunit.client.1.smithi190.stdout:8/365: write d3/d8/d25/d3b/d6b/f76 [519243,90281] 0 2024-05-06T12:14:45.212 INFO:tasks.workunit.client.0.smithi092.stdout:4/927: dwrite d9/d27/d4a/f4c [0,4194304] 0 2024-05-06T12:14:45.214 INFO:tasks.workunit.client.0.smithi092.stdout:6/943: symlink d4/d24/d4e/d10a/l12b 0 2024-05-06T12:14:45.216 INFO:tasks.workunit.client.0.smithi092.stdout:4/928: read d9/d2b/f39 [2825102,25682] 0 2024-05-06T12:14:45.219 INFO:tasks.workunit.client.0.smithi092.stdout:5/891: read d4/d5/d30/f40 [3722722,47057] 0 2024-05-06T12:14:45.219 INFO:tasks.workunit.client.1.smithi190.stdout:0/436: dwrite d9/dc/f45 [0,4194304] 0 2024-05-06T12:14:45.221 INFO:tasks.workunit.client.0.smithi092.stdout:4/929: creat d9/d2b/d7c/dab/dee/f133 x:0 0 0 2024-05-06T12:14:45.221 INFO:tasks.workunit.client.1.smithi190.stdout:3/340: mknod d5/d7/d15/d1f/d33/c7c 0 2024-05-06T12:14:45.221 INFO:tasks.workunit.client.0.smithi092.stdout:1/834: dwrite d3/d10/d13/d19/d21/d72/f86 [0,4194304] 0 2024-05-06T12:14:45.222 INFO:tasks.workunit.client.1.smithi190.stdout:3/341: read - d5/f5e zero size 2024-05-06T12:14:45.224 INFO:tasks.workunit.client.0.smithi092.stdout:6/944: mknod d4/d24/d38/d5d/c12c 0 2024-05-06T12:14:45.224 INFO:tasks.workunit.client.1.smithi190.stdout:2/315: rename d2/d8/dc/c29 to d2/d10/c61 0 2024-05-06T12:14:45.225 INFO:tasks.workunit.client.0.smithi092.stdout:5/892: mknod d4/d5/c135 0 2024-05-06T12:14:45.226 INFO:tasks.workunit.client.1.smithi190.stdout:2/316: fsync d2/d8/dc/f5b 0 2024-05-06T12:14:45.226 INFO:tasks.workunit.client.0.smithi092.stdout:4/930: chown d9/dd/d90/c93 385 1 2024-05-06T12:14:45.227 INFO:tasks.workunit.client.0.smithi092.stdout:1/835: chown d3/d10/d13/d19/d21/c2d 64031 1 2024-05-06T12:14:45.230 INFO:tasks.workunit.client.0.smithi092.stdout:5/893: fdatasync d4/d59/fd5 0 2024-05-06T12:14:45.230 INFO:tasks.workunit.client.1.smithi190.stdout:9/459: unlink db/dd/d16/d4c/d4e/l80 0 2024-05-06T12:14:45.232 INFO:tasks.workunit.client.0.smithi092.stdout:4/931: symlink d9/d27/da8/db3/d124/d12e/l134 0 2024-05-06T12:14:45.233 INFO:tasks.workunit.client.1.smithi190.stdout:3/342: dwrite d5/d14/d11/d2a/d3d/f49 [0,4194304] 0 2024-05-06T12:14:45.237 INFO:tasks.workunit.client.0.smithi092.stdout:4/932: unlink d9/df/d62/d7b/d107/c12a 0 2024-05-06T12:14:45.241 INFO:tasks.workunit.client.0.smithi092.stdout:5/894: dwrite d4/d34/d28/d4e/d4b/dd6/ff7 [0,4194304] 0 2024-05-06T12:14:45.241 INFO:tasks.workunit.client.1.smithi190.stdout:8/366: creat d3/d8/d2b/f7e x:0 0 0 2024-05-06T12:14:45.243 INFO:tasks.workunit.client.0.smithi092.stdout:5/895: write d4/f7d [261574,44878] 0 2024-05-06T12:14:45.244 INFO:tasks.workunit.client.0.smithi092.stdout:5/896: chown d4/d59/f83 11716 1 2024-05-06T12:14:45.245 INFO:tasks.workunit.client.1.smithi190.stdout:0/437: creat d9/df/d21/d72/f98 x:0 0 0 2024-05-06T12:14:45.250 INFO:tasks.workunit.client.0.smithi092.stdout:4/933: rmdir d9/de6 0 2024-05-06T12:14:45.251 INFO:tasks.workunit.client.1.smithi190.stdout:9/460: creat db/dd/d16/fa6 x:0 0 0 2024-05-06T12:14:45.253 INFO:tasks.workunit.client.1.smithi190.stdout:8/367: mkdir d3/d35/d55/d7f 0 2024-05-06T12:14:45.256 INFO:tasks.workunit.client.0.smithi092.stdout:4/934: dwrite d9/df/d62/d7b/db6/fdd [0,4194304] 0 2024-05-06T12:14:45.256 INFO:tasks.workunit.client.1.smithi190.stdout:6/258: rename df/c1c to df/d12/c59 0 2024-05-06T12:14:45.257 INFO:tasks.workunit.client.1.smithi190.stdout:0/438: dwrite d9/dc/d48/f4e [0,4194304] 0 2024-05-06T12:14:45.258 INFO:tasks.workunit.client.1.smithi190.stdout:9/461: dread - db/dd/d16/d58/d87/d22/d49/d88/f9c zero size 2024-05-06T12:14:45.260 INFO:tasks.workunit.client.1.smithi190.stdout:6/259: write df/d2c/f25 [842624,59796] 0 2024-05-06T12:14:45.260 INFO:tasks.workunit.client.0.smithi092.stdout:1/836: dwrite d3/d10/f53 [0,4194304] 0 2024-05-06T12:14:45.262 INFO:tasks.workunit.client.1.smithi190.stdout:5/315: creat d2/da/d2c/f5e x:0 0 0 2024-05-06T12:14:45.263 INFO:tasks.workunit.client.1.smithi190.stdout:8/368: creat d3/d8/d2b/f80 x:0 0 0 2024-05-06T12:14:45.263 INFO:tasks.workunit.client.0.smithi092.stdout:5/897: write d4/d34/d28/f3a [1605630,79062] 0 2024-05-06T12:14:45.266 INFO:tasks.workunit.client.1.smithi190.stdout:3/343: dwrite d5/d14/d11/d2a/d3d/f6c [0,4194304] 0 2024-05-06T12:14:45.267 INFO:tasks.workunit.client.1.smithi190.stdout:2/317: creat d2/f62 x:0 0 0 2024-05-06T12:14:45.268 INFO:tasks.workunit.client.0.smithi092.stdout:5/898: chown d4/d5/d10/d31/f4a 0 1 2024-05-06T12:14:45.271 INFO:tasks.workunit.client.1.smithi190.stdout:1/271: write d5/fa [2408660,14116] 0 2024-05-06T12:14:45.272 INFO:tasks.workunit.client.1.smithi190.stdout:8/369: mknod d3/d13/c81 0 2024-05-06T12:14:45.272 INFO:tasks.workunit.client.0.smithi092.stdout:5/899: rename d4/d34/d28/d4e/d67/de6 to d4/d5/d10/d31/ddf/d54/db0/d136 0 2024-05-06T12:14:45.273 INFO:tasks.workunit.client.1.smithi190.stdout:0/439: mknod d9/dc/d48/d50/d87/c99 0 2024-05-06T12:14:45.273 INFO:tasks.workunit.client.0.smithi092.stdout:5/900: chown d4/d5/d10/d31/ddf/da3/fbf 2 1 2024-05-06T12:14:45.274 INFO:tasks.workunit.client.0.smithi092.stdout:5/901: dread - d4/d59/f114 zero size 2024-05-06T12:14:45.276 INFO:tasks.workunit.client.1.smithi190.stdout:2/318: creat d2/d8/dc/d20/f63 x:0 0 0 2024-05-06T12:14:45.277 INFO:tasks.workunit.client.0.smithi092.stdout:5/902: creat d4/d34/d28/d4e/d5d/d94/f137 x:0 0 0 2024-05-06T12:14:45.278 INFO:tasks.workunit.client.0.smithi092.stdout:5/903: stat d4/d34/dda 0 2024-05-06T12:14:45.278 INFO:tasks.workunit.client.1.smithi190.stdout:5/316: dread d2/da/d2c/d2e/d3b/f52 [0,4194304] 0 2024-05-06T12:14:45.279 INFO:tasks.workunit.client.1.smithi190.stdout:6/260: dwrite df/d17/d44/f48 [0,4194304] 0 2024-05-06T12:14:45.280 INFO:tasks.workunit.client.0.smithi092.stdout:5/904: mkdir d4/d5/d30/dfd/d108/d138 0 2024-05-06T12:14:45.280 INFO:tasks.workunit.client.1.smithi190.stdout:1/272: truncate d5/d23/d2b/f35 152115 0 2024-05-06T12:14:45.281 INFO:tasks.workunit.client.0.smithi092.stdout:5/905: rmdir d4/d5/d10/d31/ddf/d54/db0/d102/d20/df8 39 2024-05-06T12:14:45.282 INFO:tasks.workunit.client.1.smithi190.stdout:8/370: rename d3/f26 to d3/d13/d4a/d6e/f82 0 2024-05-06T12:14:45.283 INFO:tasks.workunit.client.1.smithi190.stdout:0/440: symlink d9/dc/d28/l9a 0 2024-05-06T12:14:45.284 INFO:tasks.workunit.client.1.smithi190.stdout:0/441: stat d9/dc/d28/c73 0 2024-05-06T12:14:45.286 INFO:tasks.workunit.client.0.smithi092.stdout:5/906: link d4/d5/d30/c32 d4/d5/d30/dfd/d108/d138/c139 0 2024-05-06T12:14:45.286 INFO:tasks.workunit.client.1.smithi190.stdout:6/261: symlink df/d17/d37/l5a 0 2024-05-06T12:14:45.291 INFO:tasks.workunit.client.1.smithi190.stdout:0/442: unlink d9/dc/d28/d4c/f7c 0 2024-05-06T12:14:45.293 INFO:tasks.workunit.client.1.smithi190.stdout:8/371: mknod d3/d35/d55/d7f/c83 0 2024-05-06T12:14:45.294 INFO:tasks.workunit.client.1.smithi190.stdout:6/262: dwrite df/d2c/f39 [0,4194304] 0 2024-05-06T12:14:45.295 INFO:tasks.workunit.client.1.smithi190.stdout:8/372: read d3/f52 [5592890,15923] 0 2024-05-06T12:14:45.296 INFO:tasks.workunit.client.1.smithi190.stdout:6/263: write df/d12/f2b [1188396,106027] 0 2024-05-06T12:14:45.298 INFO:tasks.workunit.client.0.smithi092.stdout:4/935: dread d9/df/f14 [0,4194304] 0 2024-05-06T12:14:45.299 INFO:tasks.workunit.client.1.smithi190.stdout:0/443: creat d9/dc/d28/d4c/f9b x:0 0 0 2024-05-06T12:14:45.300 INFO:tasks.workunit.client.1.smithi190.stdout:1/273: read d5/d23/d32/f4a [1408891,19662] 0 2024-05-06T12:14:45.301 INFO:tasks.workunit.client.1.smithi190.stdout:6/264: unlink df/f3f 0 2024-05-06T12:14:45.302 INFO:tasks.workunit.client.0.smithi092.stdout:4/936: rename d9/d27/f49 to d9/d2b/d65/dbc/f135 0 2024-05-06T12:14:45.304 INFO:tasks.workunit.client.1.smithi190.stdout:1/274: creat d5/d23/d32/d47/f57 x:0 0 0 2024-05-06T12:14:45.306 INFO:tasks.workunit.client.1.smithi190.stdout:1/275: chown d5/d23/d32/d47/d53 13716764 1 2024-05-06T12:14:45.306 INFO:tasks.workunit.client.0.smithi092.stdout:1/837: dread d3/d10/d13/d19/d21/f4d [0,4194304] 0 2024-05-06T12:14:45.307 INFO:tasks.workunit.client.1.smithi190.stdout:0/444: creat d9/df/d21/f9c x:0 0 0 2024-05-06T12:14:45.309 INFO:tasks.workunit.client.1.smithi190.stdout:1/276: dread d5/d23/d24/f45 [0,4194304] 0 2024-05-06T12:14:45.310 INFO:tasks.workunit.client.0.smithi092.stdout:1/838: creat d3/d10/d13/d19/d57/dd6/f10e x:0 0 0 2024-05-06T12:14:45.310 INFO:tasks.workunit.client.1.smithi190.stdout:8/373: dwrite d3/d35/f3c [0,4194304] 0 2024-05-06T12:14:45.310 INFO:tasks.workunit.client.0.smithi092.stdout:5/907: dwrite d4/d5/d10/f17 [0,4194304] 0 2024-05-06T12:14:45.312 INFO:tasks.workunit.client.1.smithi190.stdout:8/374: write d3/d35/f3c [780133,59210] 0 2024-05-06T12:14:45.313 INFO:tasks.workunit.client.0.smithi092.stdout:5/908: symlink d4/d5/d10/d31/ddf/da3/l13a 0 2024-05-06T12:14:45.313 INFO:tasks.workunit.client.0.smithi092.stdout:5/909: chown d4/d34/f56 390774 1 2024-05-06T12:14:45.315 INFO:tasks.workunit.client.0.smithi092.stdout:1/839: creat d3/d42/d84/d4f/dd5/ded/f10f x:0 0 0 2024-05-06T12:14:45.318 INFO:tasks.workunit.client.0.smithi092.stdout:1/840: mknod d3/c110 0 2024-05-06T12:14:45.319 INFO:tasks.workunit.client.1.smithi190.stdout:1/277: mknod d5/d4f/c58 0 2024-05-06T12:14:45.320 INFO:tasks.workunit.client.0.smithi092.stdout:5/910: truncate d4/d34/d28/f74 308474 0 2024-05-06T12:14:45.321 INFO:tasks.workunit.client.1.smithi190.stdout:5/317: rename d2/da/d2c to d2/da/d59/d5f 0 2024-05-06T12:14:45.321 INFO:tasks.workunit.client.1.smithi190.stdout:8/375: creat d3/d8/d2b/d3e/f84 x:0 0 0 2024-05-06T12:14:45.322 INFO:tasks.workunit.client.1.smithi190.stdout:1/278: mkdir d5/d23/d32/d59 0 2024-05-06T12:14:45.323 INFO:tasks.workunit.client.0.smithi092.stdout:5/911: chown d4/d34/d28/d4e/d5d/d82/f63 164 1 2024-05-06T12:14:45.324 INFO:tasks.workunit.client.0.smithi092.stdout:1/841: creat d3/d10/d13/db0/f111 x:0 0 0 2024-05-06T12:14:45.324 INFO:tasks.workunit.client.1.smithi190.stdout:0/445: rename d9/f1a to d9/d51/f9d 0 2024-05-06T12:14:45.325 INFO:tasks.workunit.client.1.smithi190.stdout:0/446: chown d9/dc/d28/f8c 0 1 2024-05-06T12:14:45.326 INFO:tasks.workunit.client.0.smithi092.stdout:1/842: mkdir d3/d10/d13/d19/d21/d72/d112 0 2024-05-06T12:14:45.328 INFO:tasks.workunit.client.1.smithi190.stdout:1/279: creat d5/d15/d1e/f5a x:0 0 0 2024-05-06T12:14:45.329 INFO:tasks.workunit.client.1.smithi190.stdout:9/462: truncate db/dd/d16/d58/f7d 2749914 0 2024-05-06T12:14:45.330 INFO:tasks.workunit.client.0.smithi092.stdout:5/912: dwrite d4/d5/d10/d31/ddf/d54/db0/d102/d20/f8d [0,4194304] 0 2024-05-06T12:14:45.330 INFO:tasks.workunit.client.1.smithi190.stdout:0/447: creat d9/dc/d48/f9e x:0 0 0 2024-05-06T12:14:45.331 INFO:tasks.workunit.client.1.smithi190.stdout:5/318: dwrite d2/d27/f54 [0,4194304] 0 2024-05-06T12:14:45.332 INFO:tasks.workunit.client.0.smithi092.stdout:4/937: dread d9/d27/da8/db3/fd2 [4194304,4194304] 0 2024-05-06T12:14:45.332 INFO:tasks.workunit.client.0.smithi092.stdout:5/913: fdatasync d4/d5/d10/d31/d53/f9e 0 2024-05-06T12:14:45.333 INFO:tasks.workunit.client.0.smithi092.stdout:1/843: dread d3/f6a [0,4194304] 0 2024-05-06T12:14:45.334 INFO:tasks.workunit.client.1.smithi190.stdout:0/448: chown d9/df/d21/c24 95 1 2024-05-06T12:14:45.336 INFO:tasks.workunit.client.1.smithi190.stdout:1/280: rmdir d5/d23/d24/d3a 39 2024-05-06T12:14:45.337 INFO:tasks.workunit.client.0.smithi092.stdout:4/938: mknod d9/df/d48/c136 0 2024-05-06T12:14:45.337 INFO:tasks.workunit.client.1.smithi190.stdout:2/319: write d2/d8/d19/f24 [1926512,88902] 0 2024-05-06T12:14:45.338 INFO:tasks.workunit.client.1.smithi190.stdout:2/320: stat d2/d8/dc/f5b 0 2024-05-06T12:14:45.338 INFO:tasks.workunit.client.1.smithi190.stdout:0/449: mknod d9/dc/d48/c9f 0 2024-05-06T12:14:45.338 INFO:tasks.workunit.client.1.smithi190.stdout:2/321: fsync d2/f35 0 2024-05-06T12:14:45.341 INFO:tasks.workunit.client.1.smithi190.stdout:1/281: creat d5/d23/f5b x:0 0 0 2024-05-06T12:14:45.341 INFO:tasks.workunit.client.0.smithi092.stdout:1/844: creat d3/d10/d13/d19/d34/d89/d10a/f113 x:0 0 0 2024-05-06T12:14:45.342 INFO:tasks.workunit.client.1.smithi190.stdout:5/319: read d2/da/f1b [2379062,111927] 0 2024-05-06T12:14:45.343 INFO:tasks.workunit.client.1.smithi190.stdout:1/282: chown d5/d15/d1e/l31 28 1 2024-05-06T12:14:45.344 INFO:tasks.workunit.client.0.smithi092.stdout:5/914: getdents d4/d5/d10 0 2024-05-06T12:14:45.346 INFO:tasks.workunit.client.1.smithi190.stdout:5/320: write d2/da/d59/d5f/d50/f51 [1246043,99764] 0 2024-05-06T12:14:45.348 INFO:tasks.workunit.client.1.smithi190.stdout:5/321: stat d2/da/d59/d5f/d2e/d3b/f52 0 2024-05-06T12:14:45.349 INFO:tasks.workunit.client.0.smithi092.stdout:5/915: creat d4/d5/d10/d31/ddf/d54/db0/f13b x:0 0 0 2024-05-06T12:14:45.349 INFO:tasks.workunit.client.0.smithi092.stdout:5/916: write d4/d34/d28/d4e/fc6 [616568,65869] 0 2024-05-06T12:14:45.350 INFO:tasks.workunit.client.1.smithi190.stdout:0/450: dwrite d9/dc/d28/f8c [0,4194304] 0 2024-05-06T12:14:45.352 INFO:tasks.workunit.client.1.smithi190.stdout:2/322: link l0 d2/d8/d33/l64 0 2024-05-06T12:14:45.353 INFO:tasks.workunit.client.1.smithi190.stdout:0/451: dread - d9/dc/f74 zero size 2024-05-06T12:14:45.356 INFO:tasks.workunit.client.1.smithi190.stdout:6/265: truncate df/f22 510395 0 2024-05-06T12:14:45.360 INFO:tasks.workunit.client.1.smithi190.stdout:2/323: mknod d2/d8/d33/c65 0 2024-05-06T12:14:45.361 INFO:tasks.workunit.client.1.smithi190.stdout:5/322: dwrite d2/f28 [0,4194304] 0 2024-05-06T12:14:45.363 INFO:tasks.workunit.client.1.smithi190.stdout:6/266: rename df/d17/d37/l5a to df/d2c/d1e/l5b 0 2024-05-06T12:14:45.364 INFO:tasks.workunit.client.0.smithi092.stdout:1/845: dwrite d3/d10/d13/d46/f68 [0,4194304] 0 2024-05-06T12:14:45.366 INFO:tasks.workunit.client.1.smithi190.stdout:8/376: write d3/f2d [1618647,10176] 0 2024-05-06T12:14:45.366 INFO:tasks.workunit.client.0.smithi092.stdout:4/939: dread d9/d2b/d5b/f85 [0,4194304] 0 2024-05-06T12:14:45.369 INFO:tasks.workunit.client.1.smithi190.stdout:2/324: unlink d2/d8/d33/f2b 0 2024-05-06T12:14:45.369 INFO:tasks.workunit.client.0.smithi092.stdout:5/917: dread d4/d34/fd0 [0,4194304] 0 2024-05-06T12:14:45.370 INFO:tasks.workunit.client.0.smithi092.stdout:1/846: fsync d3/d10/d13/d19/d21/d88/d8b/fac 0 2024-05-06T12:14:45.371 INFO:tasks.workunit.client.1.smithi190.stdout:9/463: dread db/dd/d16/d58/d87/d22/d49/f79 [0,4194304] 0 2024-05-06T12:14:45.372 INFO:tasks.workunit.client.0.smithi092.stdout:5/918: fdatasync d4/d59/fd1 0 2024-05-06T12:14:45.372 INFO:tasks.workunit.client.0.smithi092.stdout:4/940: chown d9/d27/l72 103501320 1 2024-05-06T12:14:45.373 INFO:tasks.workunit.client.1.smithi190.stdout:2/325: chown d2/d8/dc/f1a 1471945444 1 2024-05-06T12:14:45.374 INFO:tasks.workunit.client.1.smithi190.stdout:9/464: write db/d23/d29/fa5 [227352,86193] 0 2024-05-06T12:14:45.375 INFO:tasks.workunit.client.1.smithi190.stdout:6/267: dwrite df/d2c/d1e/f20 [4194304,4194304] 0 2024-05-06T12:14:45.375 INFO:tasks.workunit.client.0.smithi092.stdout:1/847: creat d3/daf/f114 x:0 0 0 2024-05-06T12:14:45.376 INFO:tasks.workunit.client.1.smithi190.stdout:8/377: mknod d3/d35/d55/d61/c85 0 2024-05-06T12:14:45.376 INFO:tasks.workunit.client.1.smithi190.stdout:2/326: write d2/d4c/f5a [842766,31824] 0 2024-05-06T12:14:45.378 INFO:tasks.workunit.client.1.smithi190.stdout:8/378: dread - d3/f71 zero size 2024-05-06T12:14:45.378 INFO:tasks.workunit.client.1.smithi190.stdout:0/452: getdents d9/dc/d63 0 2024-05-06T12:14:45.379 INFO:tasks.workunit.client.1.smithi190.stdout:5/323: creat d2/f60 x:0 0 0 2024-05-06T12:14:45.379 INFO:tasks.workunit.client.0.smithi092.stdout:4/941: rename d9/d2b/d65/d83/lb8 to d9/d27/da8/l137 0 2024-05-06T12:14:45.381 INFO:tasks.workunit.client.0.smithi092.stdout:1/848: chown d3/d10/d13/d19/d57/c5e 352692 1 2024-05-06T12:14:45.382 INFO:tasks.workunit.client.0.smithi092.stdout:5/919: fdatasync d4/d5/d10/d31/ddf/d54/db0/d102/d20/df8/f89 0 2024-05-06T12:14:45.384 INFO:tasks.workunit.client.1.smithi190.stdout:9/465: mkdir db/dd/d16/d4c/da7 0 2024-05-06T12:14:45.385 INFO:tasks.workunit.client.0.smithi092.stdout:4/942: readlink d9/d2b/d65/ldb 0 2024-05-06T12:14:45.386 INFO:tasks.workunit.client.1.smithi190.stdout:2/327: rename d2/d10/l39 to d2/d8/dc/l66 0 2024-05-06T12:14:45.386 INFO:tasks.workunit.client.1.smithi190.stdout:9/466: write db/d23/d29/f7c [4000158,94069] 0 2024-05-06T12:14:45.387 INFO:tasks.workunit.client.1.smithi190.stdout:0/453: dwrite d9/df/d56/f68 [0,4194304] 0 2024-05-06T12:14:45.388 INFO:tasks.workunit.client.0.smithi092.stdout:2/727: sync 2024-05-06T12:14:45.388 INFO:tasks.workunit.client.1.smithi190.stdout:8/379: symlink d3/d8/d25/d3b/d49/l86 0 2024-05-06T12:14:45.388 INFO:tasks.workunit.client.1.smithi190.stdout:2/328: write d2/d8/d19/f24 [2655137,11612] 0 2024-05-06T12:14:45.390 INFO:tasks.workunit.client.0.smithi092.stdout:5/920: mkdir d4/d5/d10/d31/ddf/d54/db0/d102/d13c 0 2024-05-06T12:14:45.391 INFO:tasks.workunit.client.1.smithi190.stdout:5/324: truncate d2/da/d59/d5f/f3c 2517673 0 2024-05-06T12:14:45.391 INFO:tasks.workunit.client.1.smithi190.stdout:1/283: symlink d5/d23/l5c 0 2024-05-06T12:14:45.394 INFO:tasks.workunit.client.0.smithi092.stdout:5/921: getdents d4/d34/d28/d4e/d4b/dd6 0 2024-05-06T12:14:45.398 INFO:tasks.workunit.client.1.smithi190.stdout:5/325: dread d2/da/f2f [0,4194304] 0 2024-05-06T12:14:45.398 INFO:tasks.workunit.client.1.smithi190.stdout:5/326: dread - d2/da/d59/d5f/f58 zero size 2024-05-06T12:14:45.398 INFO:tasks.workunit.client.1.smithi190.stdout:2/329: truncate d2/d10/f2c 4048405 0 2024-05-06T12:14:45.398 INFO:tasks.workunit.client.1.smithi190.stdout:0/454: write d9/fb [7093331,42995] 0 2024-05-06T12:14:45.398 INFO:tasks.workunit.client.1.smithi190.stdout:9/467: creat db/d23/d7e/fa8 x:0 0 0 2024-05-06T12:14:45.398 INFO:tasks.workunit.client.1.smithi190.stdout:0/455: fdatasync d9/f1b 0 2024-05-06T12:14:45.399 INFO:tasks.workunit.client.0.smithi092.stdout:5/922: creat d4/d5/d10/d31/ddf/d54/dc5/f13d x:0 0 0 2024-05-06T12:14:45.399 INFO:tasks.workunit.client.0.smithi092.stdout:2/728: dread df/d15/f95 [0,4194304] 0 2024-05-06T12:14:45.399 INFO:tasks.workunit.client.1.smithi190.stdout:0/456: dread - d9/dc/d28/f76 zero size 2024-05-06T12:14:45.400 INFO:tasks.workunit.client.0.smithi092.stdout:2/729: stat df/d15/d88/f78 0 2024-05-06T12:14:45.401 INFO:tasks.workunit.client.0.smithi092.stdout:5/923: symlink d4/d5/d30/dfd/d108/d12b/l13e 0 2024-05-06T12:14:45.401 INFO:tasks.workunit.client.1.smithi190.stdout:8/380: creat d3/d35/f87 x:0 0 0 2024-05-06T12:14:45.401 INFO:tasks.workunit.client.0.smithi092.stdout:2/730: truncate df/d17/d7d/fb2 399588 0 2024-05-06T12:14:45.404 INFO:tasks.workunit.client.1.smithi190.stdout:0/457: creat d9/dc/d28/d4c/d96/fa0 x:0 0 0 2024-05-06T12:14:45.405 INFO:tasks.workunit.client.0.smithi092.stdout:2/731: truncate f1 2180798 0 2024-05-06T12:14:45.405 INFO:tasks.workunit.client.1.smithi190.stdout:8/381: truncate d3/d8/d2b/f80 736830 0 2024-05-06T12:14:45.408 INFO:tasks.workunit.client.1.smithi190.stdout:1/284: dwrite d5/d23/d24/d3a/f3e [4194304,4194304] 0 2024-05-06T12:14:45.408 INFO:tasks.workunit.client.1.smithi190.stdout:8/382: stat d3/d8/d25/d3b/d49/l86 0 2024-05-06T12:14:45.411 INFO:tasks.workunit.client.1.smithi190.stdout:1/285: readlink d5/d15/d1e/l41 0 2024-05-06T12:14:45.415 INFO:tasks.workunit.client.1.smithi190.stdout:2/330: dread d2/d8/d33/d46/f4e [0,4194304] 0 2024-05-06T12:14:45.418 INFO:tasks.workunit.client.1.smithi190.stdout:0/458: dwrite d9/df/d56/f59 [0,4194304] 0 2024-05-06T12:14:45.418 INFO:tasks.workunit.client.1.smithi190.stdout:2/331: read - d2/d8/dc/f5b zero size 2024-05-06T12:14:45.422 INFO:tasks.workunit.client.1.smithi190.stdout:8/383: link d3/d8/d2b/d3e/f84 d3/d8/d2b/d3e/f88 0 2024-05-06T12:14:45.422 INFO:tasks.workunit.client.1.smithi190.stdout:5/327: dread d2/d27/f20 [4194304,4194304] 0 2024-05-06T12:14:45.423 INFO:tasks.workunit.client.1.smithi190.stdout:8/384: fsync d3/d8/f2c 0 2024-05-06T12:14:45.424 INFO:tasks.workunit.client.0.smithi092.stdout:4/943: write d9/d2b/fc7 [536668,87936] 0 2024-05-06T12:14:45.426 INFO:tasks.workunit.client.0.smithi092.stdout:1/849: dread d3/d10/d13/d19/d34/f36 [0,4194304] 0 2024-05-06T12:14:45.427 INFO:tasks.workunit.client.1.smithi190.stdout:6/268: dread fd [0,4194304] 0 2024-05-06T12:14:45.427 INFO:tasks.workunit.client.0.smithi092.stdout:1/850: read - d3/d42/d84/d4f/d7f/fe7 zero size 2024-05-06T12:14:45.427 INFO:tasks.workunit.client.0.smithi092.stdout:4/944: write d9/d27/f41 [2151120,68644] 0 2024-05-06T12:14:45.428 INFO:tasks.workunit.client.1.smithi190.stdout:6/269: stat df/d2c/d28/f40 0 2024-05-06T12:14:45.428 INFO:tasks.workunit.client.1.smithi190.stdout:2/332: mknod d2/d4c/c67 0 2024-05-06T12:14:45.429 INFO:tasks.workunit.client.1.smithi190.stdout:1/286: rmdir d5/d23/d32/d59 0 2024-05-06T12:14:45.430 INFO:tasks.workunit.client.1.smithi190.stdout:6/270: read df/d12/f2b [84176,27032] 0 2024-05-06T12:14:45.431 INFO:tasks.workunit.client.0.smithi092.stdout:2/732: dread df/d17/d1d/fb6 [0,4194304] 0 2024-05-06T12:14:45.433 INFO:tasks.workunit.client.0.smithi092.stdout:4/945: symlink d9/d27/d4a/d11e/l138 0 2024-05-06T12:14:45.436 INFO:tasks.workunit.client.0.smithi092.stdout:1/851: link d3/d10/d13/d19/d21/d72/d9a/fd0 d3/d10/f115 0 2024-05-06T12:14:45.437 INFO:tasks.workunit.client.1.smithi190.stdout:6/271: dread f2 [4194304,4194304] 0 2024-05-06T12:14:45.438 INFO:tasks.workunit.client.0.smithi092.stdout:2/733: link df/d17/d7e/fc3 df/dd0/fda 0 2024-05-06T12:14:45.438 INFO:tasks.workunit.client.1.smithi190.stdout:1/287: dread d5/d23/d24/d3a/f3e [4194304,4194304] 0 2024-05-06T12:14:45.439 INFO:tasks.workunit.client.1.smithi190.stdout:0/459: dread d9/f5f [0,4194304] 0 2024-05-06T12:14:45.440 INFO:tasks.workunit.client.1.smithi190.stdout:0/460: stat d9/d91 0 2024-05-06T12:14:45.441 INFO:tasks.workunit.client.0.smithi092.stdout:4/946: dwrite d9/d27/d4a/f76 [0,4194304] 0 2024-05-06T12:14:45.441 INFO:tasks.workunit.client.0.smithi092.stdout:1/852: creat d3/d42/f116 x:0 0 0 2024-05-06T12:14:45.445 INFO:tasks.workunit.client.0.smithi092.stdout:7/954: sync 2024-05-06T12:14:45.445 INFO:tasks.workunit.client.0.smithi092.stdout:8/866: sync 2024-05-06T12:14:45.445 INFO:tasks.workunit.client.0.smithi092.stdout:6/945: sync 2024-05-06T12:14:45.445 INFO:tasks.workunit.client.0.smithi092.stdout:3/681: sync 2024-05-06T12:14:45.445 INFO:tasks.workunit.client.0.smithi092.stdout:9/860: sync 2024-05-06T12:14:45.446 INFO:tasks.workunit.client.0.smithi092.stdout:4/947: write f5 [3432461,99661] 0 2024-05-06T12:14:45.448 INFO:tasks.workunit.client.0.smithi092.stdout:8/867: chown d9/d21/d37/d46/feb 15607962 1 2024-05-06T12:14:45.449 INFO:tasks.workunit.client.0.smithi092.stdout:1/853: rename d3/d10/d13/d46/f55 to d3/d10/f117 0 2024-05-06T12:14:45.450 INFO:tasks.workunit.client.0.smithi092.stdout:3/682: mkdir d8/d24/d33/d39/dd6 0 2024-05-06T12:14:45.452 INFO:tasks.workunit.client.0.smithi092.stdout:6/946: mkdir d4/d4c/d52/d6c/d12d 0 2024-05-06T12:14:45.452 INFO:tasks.workunit.client.0.smithi092.stdout:3/683: write d8/d24/d27/f2e [1121165,34351] 0 2024-05-06T12:14:45.456 INFO:tasks.workunit.client.0.smithi092.stdout:9/861: dwrite d16/d1c/d58/ff3 [0,4194304] 0 2024-05-06T12:14:45.457 INFO:tasks.workunit.client.1.smithi190.stdout:4/217: sync 2024-05-06T12:14:45.459 INFO:tasks.workunit.client.0.smithi092.stdout:4/948: truncate d9/dd/f19 2178864 0 2024-05-06T12:14:45.463 INFO:tasks.workunit.client.0.smithi092.stdout:2/734: truncate df/d17/d1d/fb6 2599367 0 2024-05-06T12:14:45.465 INFO:tasks.workunit.client.0.smithi092.stdout:3/684: dread d8/db/f6c [0,4194304] 0 2024-05-06T12:14:45.467 INFO:tasks.workunit.client.0.smithi092.stdout:9/862: symlink d16/d1e/d7b/dce/l119 0 2024-05-06T12:14:45.470 INFO:tasks.workunit.client.0.smithi092.stdout:2/735: creat df/d17/d53/fdb x:0 0 0 2024-05-06T12:14:45.470 INFO:tasks.workunit.client.0.smithi092.stdout:2/736: chown df/d17/d53 131844 1 2024-05-06T12:14:45.472 INFO:tasks.workunit.client.0.smithi092.stdout:2/737: write df/d17/d7d/fb2 [160209,38131] 0 2024-05-06T12:14:45.474 INFO:tasks.workunit.client.0.smithi092.stdout:2/738: readlink df/d15/d88/d59/l7f 0 2024-05-06T12:14:45.475 INFO:tasks.workunit.client.0.smithi092.stdout:4/949: link d9/d2b/d65/dbc/lde d9/d2b/d7c/dab/l139 0 2024-05-06T12:14:45.477 INFO:tasks.workunit.client.0.smithi092.stdout:9/863: dwrite d16/d2c/d3f/f96 [0,4194304] 0 2024-05-06T12:14:45.479 INFO:tasks.workunit.client.0.smithi092.stdout:2/739: rename df/d1b/d8e/f93 to df/d15/fdc 0 2024-05-06T12:14:45.480 INFO:tasks.workunit.client.0.smithi092.stdout:8/868: write d9/dc/fbb [153673,40193] 0 2024-05-06T12:14:45.482 INFO:tasks.workunit.client.0.smithi092.stdout:1/854: write d3/d10/f115 [943647,26883] 0 2024-05-06T12:14:45.483 INFO:tasks.workunit.client.0.smithi092.stdout:8/869: chown d9/d21/d37/d46/d5c/d62/da5/fd6 183861680 1 2024-05-06T12:14:45.487 INFO:tasks.workunit.client.0.smithi092.stdout:7/955: write d9/de/d1f/d37/d39/feb [1297426,60156] 0 2024-05-06T12:14:45.488 INFO:tasks.workunit.client.0.smithi092.stdout:8/870: mkdir d9/d21/d54/d116/d11f/d12e 0 2024-05-06T12:14:45.489 INFO:tasks.workunit.client.0.smithi092.stdout:2/740: dwrite fd [0,4194304] 0 2024-05-06T12:14:45.490 INFO:tasks.workunit.client.0.smithi092.stdout:6/947: truncate d4/d24/d26/f6e 38844 0 2024-05-06T12:14:45.491 INFO:tasks.workunit.client.0.smithi092.stdout:1/855: read d3/d10/d13/f1f [1437836,58054] 0 2024-05-06T12:14:45.491 INFO:tasks.workunit.client.0.smithi092.stdout:6/948: chown d4/d24/c8b 106 1 2024-05-06T12:14:45.492 INFO:tasks.workunit.client.0.smithi092.stdout:8/871: unlink d9/d21/d37/ff6 0 2024-05-06T12:14:45.493 INFO:tasks.workunit.client.0.smithi092.stdout:6/949: write d4/d24/d66/d105/f11e [581608,43217] 0 2024-05-06T12:14:45.494 INFO:tasks.workunit.client.0.smithi092.stdout:2/741: rmdir df/d15/d88/dc9 39 2024-05-06T12:14:45.496 INFO:tasks.workunit.client.1.smithi190.stdout:8/385: mkdir d3/d8/d25/d3b/d6b/d7a/d89 0 2024-05-06T12:14:45.498 INFO:tasks.workunit.client.0.smithi092.stdout:7/956: truncate d9/de/d10/d17/d21/d2e/d9c/fcc 880467 0 2024-05-06T12:14:45.498 INFO:tasks.workunit.client.1.smithi190.stdout:5/328: symlink d2/da/d59/d5f/l61 0 2024-05-06T12:14:45.499 INFO:tasks.workunit.client.1.smithi190.stdout:5/329: readlink d2/da/l34 0 2024-05-06T12:14:45.499 INFO:tasks.workunit.client.0.smithi092.stdout:1/856: creat d3/d10/d13/d19/d57/f118 x:0 0 0 2024-05-06T12:14:45.500 INFO:tasks.workunit.client.0.smithi092.stdout:3/685: dwrite d8/db/d4c/d40/d7a/d90/f93 [0,4194304] 0 2024-05-06T12:14:45.501 INFO:tasks.workunit.client.1.smithi190.stdout:5/330: truncate d2/da/d59/d5f/f5b 231809 0 2024-05-06T12:14:45.504 INFO:tasks.workunit.client.0.smithi092.stdout:6/950: rename d4/d24/f55 to d4/d4c/d52/d6c/f12e 0 2024-05-06T12:14:45.506 INFO:tasks.workunit.client.1.smithi190.stdout:8/386: rmdir d3/d8/d25/d3b/d6b/d7a 39 2024-05-06T12:14:45.506 INFO:tasks.workunit.client.0.smithi092.stdout:2/742: unlink df/d17/d7e/fc0 0 2024-05-06T12:14:45.507 INFO:tasks.workunit.client.0.smithi092.stdout:4/950: truncate d9/df/d62/d7b/f112 1194119 0 2024-05-06T12:14:45.509 INFO:tasks.workunit.client.0.smithi092.stdout:9/864: dwrite d16/f1f [4194304,4194304] 0 2024-05-06T12:14:45.510 INFO:tasks.workunit.client.1.smithi190.stdout:5/331: write d2/da/d59/d5f/d2e/d3b/f52 [934112,125778] 0 2024-05-06T12:14:45.510 INFO:tasks.workunit.client.1.smithi190.stdout:1/288: mknod d5/d23/d32/d47/d53/c5d 0 2024-05-06T12:14:45.510 INFO:tasks.workunit.client.0.smithi092.stdout:4/951: chown d9/df/d62/d7b/db6/lfb 24177145 1 2024-05-06T12:14:45.511 INFO:tasks.workunit.client.1.smithi190.stdout:0/461: symlink d9/dc/d48/d50/d83/la1 0 2024-05-06T12:14:45.513 INFO:tasks.workunit.client.1.smithi190.stdout:9/468: dwrite db/d23/d29/f6b [0,4194304] 0 2024-05-06T12:14:45.514 INFO:tasks.workunit.client.1.smithi190.stdout:4/218: link d2/d11/f2e d2/d11/d16/f4f 0 2024-05-06T12:14:45.515 INFO:tasks.workunit.client.0.smithi092.stdout:9/865: truncate d16/d2c/d2f/d3d/d33/d6b/fa2 40199 0 2024-05-06T12:14:45.515 INFO:tasks.workunit.client.1.smithi190.stdout:8/387: rmdir d3/d8 39 2024-05-06T12:14:45.515 INFO:tasks.workunit.client.0.smithi092.stdout:1/857: mkdir d3/d10/d13/d19/d21/d72/d9a/da4/d119 0 2024-05-06T12:14:45.516 INFO:tasks.workunit.client.0.smithi092.stdout:9/866: chown d16/d2c/d2f/d3d/d33/f9a 55 1 2024-05-06T12:14:45.519 INFO:tasks.workunit.client.1.smithi190.stdout:1/289: write d5/d23/d24/d3a/f3e [4237304,22508] 0 2024-05-06T12:14:45.520 INFO:tasks.workunit.client.0.smithi092.stdout:9/867: write d16/d1e/f70 [1227391,104167] 0 2024-05-06T12:14:45.521 INFO:tasks.workunit.client.1.smithi190.stdout:0/462: creat d9/dc/d63/fa2 x:0 0 0 2024-05-06T12:14:45.522 INFO:tasks.workunit.client.1.smithi190.stdout:4/219: fdatasync d2/f4 0 2024-05-06T12:14:45.523 INFO:tasks.workunit.client.0.smithi092.stdout:6/951: rename d4/d24/d38/d5d/d6d/d7b/f7f to d4/d20/d23/d30/df7/f12f 0 2024-05-06T12:14:45.524 INFO:tasks.workunit.client.0.smithi092.stdout:1/858: write d3/d10/d13/d19/d21/d72/d9a/da4/deb/f103 [534872,105286] 0 2024-05-06T12:14:45.526 INFO:tasks.workunit.client.0.smithi092.stdout:2/743: fdatasync df/d1b/f72 0 2024-05-06T12:14:45.526 INFO:tasks.workunit.client.0.smithi092.stdout:4/952: dwrite d9/d2b/d7c/dab/dd8/ff7 [0,4194304] 0 2024-05-06T12:14:45.527 INFO:tasks.workunit.client.1.smithi190.stdout:4/220: read d2/d11/f2e [3036697,74274] 0 2024-05-06T12:14:45.527 INFO:tasks.workunit.client.0.smithi092.stdout:4/953: chown d9/d2b/d5b/daf/f105 18 1 2024-05-06T12:14:45.527 INFO:tasks.workunit.client.0.smithi092.stdout:7/957: chown d9/de/d10/d17/d21/d2e/d9c/fcc 1 1 2024-05-06T12:14:45.529 INFO:tasks.workunit.client.0.smithi092.stdout:3/686: mkdir d8/dd7 0 2024-05-06T12:14:45.531 INFO:tasks.workunit.client.1.smithi190.stdout:5/332: symlink d2/da/d59/d5f/d2e/d3b/d5a/l62 0 2024-05-06T12:14:45.537 INFO:tasks.workunit.client.0.smithi092.stdout:1/859: rmdir d3/d10/d13/dd2 39 2024-05-06T12:14:45.537 INFO:tasks.workunit.client.0.smithi092.stdout:1/860: chown d3/d10/d13/d19/d57/lcc 0 1 2024-05-06T12:14:45.538 INFO:tasks.workunit.client.0.smithi092.stdout:8/872: write f0 [1388785,77019] 0 2024-05-06T12:14:45.541 INFO:tasks.workunit.client.0.smithi092.stdout:4/954: creat d9/d27/da8/f13a x:0 0 0 2024-05-06T12:14:45.542 INFO:tasks.workunit.client.1.smithi190.stdout:6/272: dwrite df/d17/f26 [0,4194304] 0 2024-05-06T12:14:45.542 INFO:tasks.workunit.client.0.smithi092.stdout:7/958: mknod d9/d62/c145 0 2024-05-06T12:14:45.543 INFO:tasks.workunit.client.0.smithi092.stdout:9/868: getdents d16/d1c/de5 0 2024-05-06T12:14:45.544 INFO:tasks.workunit.client.1.smithi190.stdout:9/469: rename db/d23/d25/f37 to db/d23/fa9 0 2024-05-06T12:14:45.545 INFO:tasks.workunit.client.0.smithi092.stdout:2/744: rmdir df/d15/d88/dc9 39 2024-05-06T12:14:45.548 INFO:tasks.workunit.client.1.smithi190.stdout:4/221: creat d2/f50 x:0 0 0 2024-05-06T12:14:45.548 INFO:tasks.workunit.client.0.smithi092.stdout:1/861: mknod d3/d10/d13/d19/d21/d72/d9a/c11a 0 2024-05-06T12:14:45.549 INFO:tasks.workunit.client.1.smithi190.stdout:6/273: creat df/d17/f5c x:0 0 0 2024-05-06T12:14:45.552 INFO:tasks.workunit.client.0.smithi092.stdout:7/959: mkdir d9/de/d10/d17/d99/dc3/dd2/df7/d129/d146 0 2024-05-06T12:14:45.554 INFO:tasks.workunit.client.1.smithi190.stdout:1/290: rename d5/d23/c42 to d5/d23/d32/d47/c5e 0 2024-05-06T12:14:45.555 INFO:tasks.workunit.client.1.smithi190.stdout:9/470: mkdir db/daa 0 2024-05-06T12:14:45.555 INFO:tasks.workunit.client.0.smithi092.stdout:1/862: creat d3/d10/d13/d19/d34/d89/d10a/f11b x:0 0 0 2024-05-06T12:14:45.555 INFO:tasks.workunit.client.1.smithi190.stdout:4/222: mknod d2/d11/d16/d4e/c51 0 2024-05-06T12:14:45.556 INFO:tasks.workunit.client.0.smithi092.stdout:1/863: chown d3/d10/d13/d19/d34/d89 3368569 1 2024-05-06T12:14:45.556 INFO:tasks.workunit.client.1.smithi190.stdout:4/223: chown d2 3131 1 2024-05-06T12:14:45.556 INFO:tasks.workunit.client.1.smithi190.stdout:9/471: dread db/d23/f4d [0,4194304] 0 2024-05-06T12:14:45.557 INFO:tasks.workunit.client.1.smithi190.stdout:6/274: symlink df/d2c/d1e/d57/d53/l5d 0 2024-05-06T12:14:45.557 INFO:tasks.workunit.client.0.smithi092.stdout:4/955: creat d9/dd/da6/dca/df2/d10d/f13b x:0 0 0 2024-05-06T12:14:45.557 INFO:tasks.workunit.client.0.smithi092.stdout:3/687: link d8/db/d14/d5a/db0/cc3 d8/d24/d33/d91/cd8 0 2024-05-06T12:14:45.558 INFO:tasks.workunit.client.1.smithi190.stdout:4/224: truncate d2/f4d 1021094 0 2024-05-06T12:14:45.558 INFO:tasks.workunit.client.0.smithi092.stdout:7/960: unlink d9/de/l1c 0 2024-05-06T12:14:45.558 INFO:tasks.workunit.client.1.smithi190.stdout:6/275: write df/d17/f5c [667015,114435] 0 2024-05-06T12:14:45.559 INFO:tasks.workunit.client.1.smithi190.stdout:0/463: rename d9/df/d56/f59 to d9/d51/fa3 0 2024-05-06T12:14:45.560 INFO:tasks.workunit.client.0.smithi092.stdout:2/745: creat df/dd0/fdd x:0 0 0 2024-05-06T12:14:45.562 INFO:tasks.workunit.client.0.smithi092.stdout:3/688: unlink d8/db/d4c/d40/fbb 0 2024-05-06T12:14:45.565 INFO:tasks.workunit.client.1.smithi190.stdout:8/388: rename d3/d8/d25/d3b/d6b/d7a/d89 to d3/d8/d2b/d3e/d8a 0 2024-05-06T12:14:45.565 INFO:tasks.workunit.client.0.smithi092.stdout:7/961: mknod d9/de/d10/df0/df5/c147 0 2024-05-06T12:14:45.566 INFO:tasks.workunit.client.1.smithi190.stdout:6/276: dread df/d2c/f25 [0,4194304] 0 2024-05-06T12:14:45.566 INFO:tasks.workunit.client.0.smithi092.stdout:1/864: rename d3/daf/lfb to d3/d10/l11c 0 2024-05-06T12:14:45.567 INFO:tasks.workunit.client.1.smithi190.stdout:4/225: mknod d2/d11/d16/d4e/c52 0 2024-05-06T12:14:45.568 INFO:tasks.workunit.client.1.smithi190.stdout:5/333: dread d2/f5 [0,4194304] 0 2024-05-06T12:14:45.569 INFO:tasks.workunit.client.1.smithi190.stdout:0/464: dwrite d9/f52 [0,4194304] 0 2024-05-06T12:14:45.570 INFO:tasks.workunit.client.1.smithi190.stdout:5/334: chown d2/d27/c2d 4 1 2024-05-06T12:14:45.571 INFO:tasks.workunit.client.0.smithi092.stdout:1/865: chown d3/f98 87439 1 2024-05-06T12:14:45.571 INFO:tasks.workunit.client.0.smithi092.stdout:6/952: write d4/f1a [583464,67251] 0 2024-05-06T12:14:45.571 INFO:tasks.workunit.client.1.smithi190.stdout:1/291: rename d5/d15/c1c to d5/d15/c5f 0 2024-05-06T12:14:45.572 INFO:tasks.workunit.client.0.smithi092.stdout:8/873: write d9/d21/d37/dbd/dc5/d11c/ff5 [728585,68481] 0 2024-05-06T12:14:45.572 INFO:tasks.workunit.client.1.smithi190.stdout:5/335: write d2/da/f4a [98186,81230] 0 2024-05-06T12:14:45.572 INFO:tasks.workunit.client.0.smithi092.stdout:3/689: creat d8/db/d14/fd9 x:0 0 0 2024-05-06T12:14:45.577 INFO:tasks.workunit.client.1.smithi190.stdout:9/472: rename db/d23/d25/f8b to db/daa/fab 0 2024-05-06T12:14:45.580 INFO:tasks.workunit.client.0.smithi092.stdout:8/874: truncate d9/dc/d4a/d5f/da8/ddf/d115/f12b 985427 0 2024-05-06T12:14:45.581 INFO:tasks.workunit.client.0.smithi092.stdout:6/953: mknod d4/d24/d38/d5d/daf/c130 0 2024-05-06T12:14:45.582 INFO:tasks.workunit.client.1.smithi190.stdout:8/389: truncate d3/d8/d2b/d3e/f54 1378365 0 2024-05-06T12:14:45.583 INFO:tasks.workunit.client.0.smithi092.stdout:9/869: truncate d16/d1c/f7c 6196855 0 2024-05-06T12:14:45.583 INFO:tasks.workunit.client.1.smithi190.stdout:0/465: symlink d9/d51/la4 0 2024-05-06T12:14:45.584 INFO:tasks.workunit.client.0.smithi092.stdout:2/746: getdents df/d17 0 2024-05-06T12:14:45.584 INFO:tasks.workunit.client.0.smithi092.stdout:1/866: dwrite d3/d10/d13/fbb [0,4194304] 0 2024-05-06T12:14:45.585 INFO:tasks.workunit.client.1.smithi190.stdout:0/466: chown d9/df/d21/c24 0 1 2024-05-06T12:14:45.585 INFO:tasks.workunit.client.0.smithi092.stdout:4/956: dread d9/dd/f23 [0,4194304] 0 2024-05-06T12:14:45.585 INFO:tasks.workunit.client.0.smithi092.stdout:6/954: fdatasync d4/d4c/d91/d93/d9d/fbf 0 2024-05-06T12:14:45.586 INFO:tasks.workunit.client.1.smithi190.stdout:5/336: mknod d2/da/d59/d5f/d2e/d3b/c63 0 2024-05-06T12:14:45.587 INFO:tasks.workunit.client.1.smithi190.stdout:5/337: dread - d2/f60 zero size 2024-05-06T12:14:45.587 INFO:tasks.workunit.client.1.smithi190.stdout:4/226: dwrite d2/f37 [0,4194304] 0 2024-05-06T12:14:45.587 INFO:tasks.workunit.client.0.smithi092.stdout:9/870: write d16/d2c/d3f/f96 [1304475,26725] 0 2024-05-06T12:14:45.587 INFO:tasks.workunit.client.0.smithi092.stdout:8/875: creat d9/f12f x:0 0 0 2024-05-06T12:14:45.587 INFO:tasks.workunit.client.0.smithi092.stdout:4/957: dread - d9/df/ff5 zero size 2024-05-06T12:14:45.589 INFO:tasks.workunit.client.0.smithi092.stdout:2/747: write df/d15/d88/d59/f9e [116527,85519] 0 2024-05-06T12:14:45.589 INFO:tasks.workunit.client.0.smithi092.stdout:1/867: write d3/d10/d13/d19/f5a [3365300,37506] 0 2024-05-06T12:14:45.590 INFO:tasks.workunit.client.1.smithi190.stdout:8/390: mknod d3/d8/d25/d3b/d6b/d7a/c8b 0 2024-05-06T12:14:45.591 INFO:tasks.workunit.client.0.smithi092.stdout:6/955: write d4/d24/d26/dae/dc0/fc9 [1241210,118148] 0 2024-05-06T12:14:45.592 INFO:tasks.workunit.client.0.smithi092.stdout:2/748: readlink df/d15/d88/d59/l90 0 2024-05-06T12:14:45.592 INFO:tasks.workunit.client.1.smithi190.stdout:8/391: chown d3/d8/d2b/l5a 28 1 2024-05-06T12:14:45.593 INFO:tasks.workunit.client.1.smithi190.stdout:1/292: link d5/f16 d5/d23/d24/d43/f60 0 2024-05-06T12:14:45.593 INFO:tasks.workunit.client.0.smithi092.stdout:7/962: write d9/de/d10/d17/d21/d2e/d87/f9f [116884,101293] 0 2024-05-06T12:14:45.599 INFO:tasks.workunit.client.0.smithi092.stdout:9/871: symlink d16/d2c/d2f/d3d/d33/d7e/l11a 0 2024-05-06T12:14:45.599 INFO:tasks.workunit.client.1.smithi190.stdout:0/467: unlink d9/df/d21/d31/f67 0 2024-05-06T12:14:45.602 INFO:tasks.workunit.client.1.smithi190.stdout:8/392: mknod d3/d35/d55/d61/c8c 0 2024-05-06T12:14:45.606 INFO:tasks.workunit.client.1.smithi190.stdout:6/277: link df/c31 df/c5e 0 2024-05-06T12:14:45.606 INFO:tasks.workunit.client.1.smithi190.stdout:1/293: dread d5/f8 [0,4194304] 0 2024-05-06T12:14:45.606 INFO:tasks.workunit.client.1.smithi190.stdout:4/227: dread d2/d11/d16/d31/f43 [0,4194304] 0 2024-05-06T12:14:45.608 INFO:tasks.workunit.client.1.smithi190.stdout:5/338: dwrite d2/da/d59/d5f/d2e/d3b/f42 [0,4194304] 0 2024-05-06T12:14:45.609 INFO:tasks.workunit.client.1.smithi190.stdout:1/294: readlink d5/d23/l2e 0 2024-05-06T12:14:45.610 INFO:tasks.workunit.client.0.smithi092.stdout:3/690: write d8/d24/d27/f2f [561447,85674] 0 2024-05-06T12:14:45.610 INFO:tasks.workunit.client.1.smithi190.stdout:6/278: symlink df/d17/d44/l5f 0 2024-05-06T12:14:45.610 INFO:tasks.workunit.client.0.smithi092.stdout:6/956: fsync d4/d24/d66/f88 0 2024-05-06T12:14:45.611 INFO:tasks.workunit.client.1.smithi190.stdout:5/339: write d2/fc [2215390,33101] 0 2024-05-06T12:14:45.611 INFO:tasks.workunit.client.0.smithi092.stdout:2/749: mknod df/d17/cde 0 2024-05-06T12:14:45.611 INFO:tasks.workunit.client.0.smithi092.stdout:4/958: mknod d9/dd/c13c 0 2024-05-06T12:14:45.612 INFO:tasks.workunit.client.1.smithi190.stdout:4/228: unlink d2/d11/d16/f42 0 2024-05-06T12:14:45.612 INFO:tasks.workunit.client.0.smithi092.stdout:2/750: dread - df/dd0/fdd zero size 2024-05-06T12:14:45.613 INFO:tasks.workunit.client.1.smithi190.stdout:6/279: write f1 [274370,130320] 0 2024-05-06T12:14:45.614 INFO:tasks.workunit.client.0.smithi092.stdout:1/868: mknod d3/d10/d13/d19/c11d 0 2024-05-06T12:14:45.614 INFO:tasks.workunit.client.1.smithi190.stdout:4/229: fdatasync d2/f25 0 2024-05-06T12:14:45.615 INFO:tasks.workunit.client.0.smithi092.stdout:4/959: write d9/df/fb0 [1822633,11301] 0 2024-05-06T12:14:45.616 INFO:tasks.workunit.client.1.smithi190.stdout:1/295: rmdir d5/d23/d32/d47 39 2024-05-06T12:14:45.616 INFO:tasks.workunit.client.1.smithi190.stdout:8/393: dwrite d3/d35/d55/f73 [0,4194304] 0 2024-05-06T12:14:45.618 INFO:tasks.workunit.client.1.smithi190.stdout:5/340: creat d2/da/f64 x:0 0 0 2024-05-06T12:14:45.619 INFO:tasks.workunit.client.1.smithi190.stdout:6/280: creat df/d17/f60 x:0 0 0 2024-05-06T12:14:45.619 INFO:tasks.workunit.client.0.smithi092.stdout:3/691: getdents d8/d1b/d73/db7 0 2024-05-06T12:14:45.621 INFO:tasks.workunit.client.1.smithi190.stdout:4/230: unlink d2/f25 0 2024-05-06T12:14:45.621 INFO:tasks.workunit.client.0.smithi092.stdout:1/869: write d3/d10/d13/d19/f5d [4856726,26836] 0 2024-05-06T12:14:45.623 INFO:tasks.workunit.client.1.smithi190.stdout:8/394: creat d3/d8/d2b/d3e/f8d x:0 0 0 2024-05-06T12:14:45.625 INFO:tasks.workunit.client.0.smithi092.stdout:6/957: dwrite d4/d24/d26/ffb [0,4194304] 0 2024-05-06T12:14:45.625 INFO:tasks.workunit.client.0.smithi092.stdout:4/960: symlink d9/df/d62/d7b/db6/l13d 0 2024-05-06T12:14:45.625 INFO:tasks.workunit.client.1.smithi190.stdout:8/395: write d3/f52 [3138827,67614] 0 2024-05-06T12:14:45.626 INFO:tasks.workunit.client.0.smithi092.stdout:1/870: chown d3/d10/d13/d19/d21/d88/fa0 1 1 2024-05-06T12:14:45.627 INFO:tasks.workunit.client.1.smithi190.stdout:4/231: rmdir d2/d11/d16/d31 39 2024-05-06T12:14:45.628 INFO:tasks.workunit.client.0.smithi092.stdout:4/961: write d9/df/f78 [1714988,118378] 0 2024-05-06T12:14:45.629 INFO:tasks.workunit.client.1.smithi190.stdout:6/281: dwrite df/d17/d37/f49 [0,4194304] 0 2024-05-06T12:14:45.629 INFO:tasks.workunit.client.1.smithi190.stdout:5/341: mkdir d2/d65 0 2024-05-06T12:14:45.631 INFO:tasks.workunit.client.1.smithi190.stdout:4/232: chown d2/d11/c15 159028527 1 2024-05-06T12:14:45.634 INFO:tasks.workunit.client.0.smithi092.stdout:7/963: dread d9/de/d10/f2b [4194304,4194304] 0 2024-05-06T12:14:45.634 INFO:tasks.workunit.client.0.smithi092.stdout:4/962: rmdir d9/d2b/d7c/dab 39 2024-05-06T12:14:45.635 INFO:tasks.workunit.client.0.smithi092.stdout:9/872: write d16/d2c/d3f/f60 [4324832,6138] 0 2024-05-06T12:14:45.636 INFO:tasks.workunit.client.0.smithi092.stdout:6/958: mkdir d4/d4c/d91/d110/d131 0 2024-05-06T12:14:45.637 INFO:tasks.workunit.client.0.smithi092.stdout:4/963: truncate d9/df/d62/d7b/d107/f129 446272 0 2024-05-06T12:14:45.638 INFO:tasks.workunit.client.0.smithi092.stdout:6/959: read - d4/d4c/d52/ddc/f125 zero size 2024-05-06T12:14:45.638 INFO:tasks.workunit.client.1.smithi190.stdout:4/233: chown d2/d11/d16/d4e/c51 525 1 2024-05-06T12:14:45.640 INFO:tasks.workunit.client.1.smithi190.stdout:1/296: creat d5/d23/d32/f61 x:0 0 0 2024-05-06T12:14:45.641 INFO:tasks.workunit.client.0.smithi092.stdout:9/873: rename d16/d1e/d7b/fb0 to d16/d1c/d58/d92/dab/deb/d113/f11b 0 2024-05-06T12:14:45.642 INFO:tasks.workunit.client.1.smithi190.stdout:4/234: write d2/f32 [460699,97096] 0 2024-05-06T12:14:45.642 INFO:tasks.workunit.client.1.smithi190.stdout:8/396: dwrite d3/d8/d25/f66 [0,4194304] 0 2024-05-06T12:14:45.643 INFO:tasks.workunit.client.0.smithi092.stdout:7/964: creat d9/de/d10/d17/d21/f148 x:0 0 0 2024-05-06T12:14:45.647 INFO:tasks.workunit.client.1.smithi190.stdout:6/282: rename df/d17/d44/d56 to df/d2c/d61 0 2024-05-06T12:14:45.647 INFO:tasks.workunit.client.0.smithi092.stdout:6/960: write d4/d4c/d52/ddc/f100 [1335945,76136] 0 2024-05-06T12:14:45.649 INFO:tasks.workunit.client.0.smithi092.stdout:7/965: truncate d9/de/d10/d17/d99/db6/df1/f144 783994 0 2024-05-06T12:14:45.650 INFO:tasks.workunit.client.1.smithi190.stdout:8/397: creat d3/d8/d2b/d3e/f8e x:0 0 0 2024-05-06T12:14:45.652 INFO:tasks.workunit.client.1.smithi190.stdout:8/398: dread d3/d13/d4a/f5f [0,4194304] 0 2024-05-06T12:14:45.653 INFO:tasks.workunit.client.1.smithi190.stdout:8/399: write d3/d8/d25/f66 [3869759,66930] 0 2024-05-06T12:14:45.654 INFO:tasks.workunit.client.0.smithi092.stdout:9/874: rename d16/d2c/d2f/d3d/d33/dde/ff4 to d16/d2c/d88/f11c 0 2024-05-06T12:14:45.654 INFO:tasks.workunit.client.0.smithi092.stdout:6/961: stat d4/d4c/d91/d93/d9d/cbe 0 2024-05-06T12:14:45.655 INFO:tasks.workunit.client.0.smithi092.stdout:7/966: read d9/de/d1f/d37/d48/d97/fbf [557708,78944] 0 2024-05-06T12:14:45.656 INFO:tasks.workunit.client.1.smithi190.stdout:1/297: mkdir d5/d23/d62 0 2024-05-06T12:14:45.658 INFO:tasks.workunit.client.0.smithi092.stdout:7/967: write d9/d62/d121/f134 [830490,4144] 0 2024-05-06T12:14:45.659 INFO:tasks.workunit.client.1.smithi190.stdout:4/235: mkdir d2/d11/d16/d53 0 2024-05-06T12:14:45.660 INFO:tasks.workunit.client.0.smithi092.stdout:1/871: dwrite d3/d10/d13/d19/d21/d88/d8b/fac [0,4194304] 0 2024-05-06T12:14:45.662 INFO:tasks.workunit.client.0.smithi092.stdout:6/962: read d4/d20/d23/f28 [644836,124246] 0 2024-05-06T12:14:45.662 INFO:tasks.workunit.client.0.smithi092.stdout:7/968: readlink d9/d62/d108/d10d/l138 0 2024-05-06T12:14:45.663 INFO:tasks.workunit.client.1.smithi190.stdout:6/283: dwrite df/f2e [0,4194304] 0 2024-05-06T12:14:45.664 INFO:tasks.workunit.client.0.smithi092.stdout:3/692: dread d8/db/d4c/d40/d7a/d90/fd0 [4194304,4194304] 0 2024-05-06T12:14:45.664 INFO:tasks.workunit.client.1.smithi190.stdout:8/400: dwrite d3/d8/d2b/f47 [0,4194304] 0 2024-05-06T12:14:45.665 INFO:tasks.workunit.client.1.smithi190.stdout:4/236: rmdir d2/d10 39 2024-05-06T12:14:45.669 INFO:tasks.workunit.client.1.smithi190.stdout:1/298: mknod d5/d23/d32/c63 0 2024-05-06T12:14:45.669 INFO:tasks.workunit.client.1.smithi190.stdout:1/299: stat d5/c4d 0 2024-05-06T12:14:45.670 INFO:tasks.workunit.client.1.smithi190.stdout:6/284: chown df/d2c/d1e/l3d 0 1 2024-05-06T12:14:45.670 INFO:tasks.workunit.client.0.smithi092.stdout:4/964: dread d9/d2b/d5b/fb5 [0,4194304] 0 2024-05-06T12:14:45.674 INFO:tasks.workunit.client.1.smithi190.stdout:8/401: mknod d3/c8f 0 2024-05-06T12:14:45.676 INFO:tasks.workunit.client.1.smithi190.stdout:6/285: creat df/d17/d44/f62 x:0 0 0 2024-05-06T12:14:45.677 INFO:tasks.workunit.client.1.smithi190.stdout:6/286: write df/d2c/f39 [2233239,111927] 0 2024-05-06T12:14:45.678 INFO:tasks.workunit.client.1.smithi190.stdout:4/237: rename d2/l39 to d2/d11/d16/d31/l54 0 2024-05-06T12:14:45.680 INFO:tasks.workunit.client.0.smithi092.stdout:1/872: dread d3/d10/d13/d19/d21/d72/d9a/fd0 [0,4194304] 0 2024-05-06T12:14:45.680 INFO:tasks.workunit.client.0.smithi092.stdout:9/875: truncate fc 11292223 0 2024-05-06T12:14:45.681 INFO:tasks.workunit.client.1.smithi190.stdout:1/300: symlink d5/d23/d2b/l64 0 2024-05-06T12:14:45.681 INFO:tasks.workunit.client.0.smithi092.stdout:1/873: write d3/d10/d13/d19/f87 [1700318,100669] 0 2024-05-06T12:14:45.682 INFO:tasks.workunit.client.1.smithi190.stdout:6/287: dread df/d2c/f39 [0,4194304] 0 2024-05-06T12:14:45.683 INFO:tasks.workunit.client.0.smithi092.stdout:1/874: chown d3/d10/d13/d19/d21/d72/f86 247836 1 2024-05-06T12:14:45.686 INFO:tasks.workunit.client.0.smithi092.stdout:9/876: dread d16/d1e/f25 [0,4194304] 0 2024-05-06T12:14:45.691 INFO:tasks.workunit.client.1.smithi190.stdout:6/288: dwrite df/d17/d44/f62 [0,4194304] 0 2024-05-06T12:14:45.694 INFO:tasks.workunit.client.1.smithi190.stdout:4/238: creat d2/f55 x:0 0 0 2024-05-06T12:14:45.696 INFO:tasks.workunit.client.1.smithi190.stdout:5/342: dwrite d2/f30 [0,4194304] 0 2024-05-06T12:14:45.697 INFO:tasks.workunit.client.1.smithi190.stdout:6/289: rename df/d12/l19 to df/d17/l63 0 2024-05-06T12:14:45.699 INFO:tasks.workunit.client.1.smithi190.stdout:4/239: creat d2/d11/d16/d31/f56 x:0 0 0 2024-05-06T12:14:45.700 INFO:tasks.workunit.client.1.smithi190.stdout:6/290: symlink df/d17/l64 0 2024-05-06T12:14:45.700 INFO:tasks.workunit.client.1.smithi190.stdout:5/343: rmdir d2/da/d59/d5f/d2e 39 2024-05-06T12:14:45.702 INFO:tasks.workunit.client.1.smithi190.stdout:6/291: creat df/d12/f65 x:0 0 0 2024-05-06T12:14:45.703 INFO:tasks.workunit.client.1.smithi190.stdout:5/344: rename d2 to d2/d66 22 2024-05-06T12:14:45.704 INFO:tasks.workunit.client.1.smithi190.stdout:6/292: symlink df/d2c/d1e/d57/l66 0 2024-05-06T12:14:45.705 INFO:tasks.workunit.client.1.smithi190.stdout:6/293: read df/d17/d37/f49 [3733248,40643] 0 2024-05-06T12:14:45.708 INFO:tasks.workunit.client.1.smithi190.stdout:5/345: getdents d2/d27 0 2024-05-06T12:14:45.709 INFO:tasks.workunit.client.1.smithi190.stdout:5/346: stat d2/da/c4b 0 2024-05-06T12:14:45.710 INFO:tasks.workunit.client.1.smithi190.stdout:6/294: dwrite f1 [0,4194304] 0 2024-05-06T12:14:45.711 INFO:tasks.workunit.client.1.smithi190.stdout:5/347: symlink d2/da/d59/d5f/d50/l67 0 2024-05-06T12:14:45.713 INFO:tasks.workunit.client.1.smithi190.stdout:5/348: chown d2/da/d59/d5f/d2e/d3b 5408 1 2024-05-06T12:14:45.713 INFO:tasks.workunit.client.1.smithi190.stdout:6/295: dread df/d17/f5c [0,4194304] 0 2024-05-06T12:14:45.714 INFO:tasks.workunit.client.1.smithi190.stdout:7/267: sync 2024-05-06T12:14:45.714 INFO:tasks.workunit.client.1.smithi190.stdout:3/344: sync 2024-05-06T12:14:45.714 INFO:tasks.workunit.client.1.smithi190.stdout:7/268: readlink d0/d1/l56 0 2024-05-06T12:14:45.715 INFO:tasks.workunit.client.1.smithi190.stdout:3/345: chown d5/d14/f6a 0 1 2024-05-06T12:14:45.715 INFO:tasks.workunit.client.1.smithi190.stdout:6/296: write df/d17/f38 [454066,11046] 0 2024-05-06T12:14:45.717 INFO:tasks.workunit.client.1.smithi190.stdout:5/349: rename d2/f28 to d2/da/d59/d5f/d2e/d3b/f68 0 2024-05-06T12:14:45.719 INFO:tasks.workunit.client.1.smithi190.stdout:7/269: symlink d0/d1/d18/d2b/d5a/l64 0 2024-05-06T12:14:45.719 INFO:tasks.workunit.client.1.smithi190.stdout:5/350: write d2/da/f1b [3454143,125069] 0 2024-05-06T12:14:45.721 INFO:tasks.workunit.client.1.smithi190.stdout:7/270: chown d0/d1/d2/f5 2822051 1 2024-05-06T12:14:45.722 INFO:tasks.workunit.client.1.smithi190.stdout:6/297: dread df/d17/d44/f48 [0,4194304] 0 2024-05-06T12:14:45.722 INFO:tasks.workunit.client.1.smithi190.stdout:3/346: dwrite d5/d14/d11/f50 [0,4194304] 0 2024-05-06T12:14:45.731 INFO:tasks.workunit.client.1.smithi190.stdout:5/351: mknod d2/da/d59/d5f/d50/c69 0 2024-05-06T12:14:45.732 INFO:tasks.workunit.client.1.smithi190.stdout:5/352: stat d2/da/d59/d5f/l61 0 2024-05-06T12:14:45.733 INFO:tasks.workunit.client.1.smithi190.stdout:6/298: creat df/d2c/d1e/d57/d53/f67 x:0 0 0 2024-05-06T12:14:45.734 INFO:tasks.workunit.client.1.smithi190.stdout:6/299: chown df/d2c/d1e/f52 60421219 1 2024-05-06T12:14:45.735 INFO:tasks.workunit.client.1.smithi190.stdout:3/347: rename d5/d7/d15/d1f/d33/d5c/f6e to d5/d7/d15/d1f/d33/d40/f7d 0 2024-05-06T12:14:45.736 INFO:tasks.workunit.client.1.smithi190.stdout:5/353: creat d2/da/d59/f6a x:0 0 0 2024-05-06T12:14:45.738 INFO:tasks.workunit.client.1.smithi190.stdout:7/271: dwrite d0/d1/d18/d2b/d5a/f5e [0,4194304] 0 2024-05-06T12:14:45.739 INFO:tasks.workunit.client.0.smithi092.stdout:4/965: rename d9/l1e to d9/d2b/d7c/l13e 0 2024-05-06T12:14:45.739 INFO:tasks.workunit.client.0.smithi092.stdout:7/969: write d9/de/d10/d17/d99/db6/df1/f66 [2730172,80067] 0 2024-05-06T12:14:45.741 INFO:tasks.workunit.client.1.smithi190.stdout:6/300: link df/d2c/d28/l4f df/l68 0 2024-05-06T12:14:45.742 INFO:tasks.workunit.client.1.smithi190.stdout:6/301: write df/f2e [5223991,41173] 0 2024-05-06T12:14:45.744 INFO:tasks.workunit.client.1.smithi190.stdout:7/272: link d0/f20 d0/da/d36/d53/f65 0 2024-05-06T12:14:45.745 INFO:tasks.workunit.client.0.smithi092.stdout:9/877: rename d16/d1c/d58/d92/fe9 to d16/d1c/d58/d92/dab/deb/d113/f11d 0 2024-05-06T12:14:45.745 INFO:tasks.workunit.client.0.smithi092.stdout:7/970: write d9/da5/f54 [3456588,37460] 0 2024-05-06T12:14:45.745 INFO:tasks.workunit.client.1.smithi190.stdout:6/302: fdatasync df/d2c/f39 0 2024-05-06T12:14:45.747 INFO:tasks.workunit.client.0.smithi092.stdout:7/971: dread - d9/de/d10/d17/f12d zero size 2024-05-06T12:14:45.747 INFO:tasks.workunit.client.0.smithi092.stdout:9/878: readlink d16/d2c/d2f/d3d/d33/d57/l6c 0 2024-05-06T12:14:45.753 INFO:tasks.workunit.client.0.smithi092.stdout:7/972: mknod d9/de/d10/c149 0 2024-05-06T12:14:45.755 INFO:tasks.workunit.client.0.smithi092.stdout:7/973: write d9/de/d1f/f69 [2992115,39088] 0 2024-05-06T12:14:45.763 INFO:tasks.workunit.client.0.smithi092.stdout:6/963: dread d4/d24/d38/fcb [0,4194304] 0 2024-05-06T12:14:45.765 INFO:tasks.workunit.client.0.smithi092.stdout:7/974: symlink d9/de/d10/d17/d99/dc3/dd2/df7/l14a 0 2024-05-06T12:14:45.765 INFO:tasks.workunit.client.0.smithi092.stdout:3/693: dread d8/db/d4c/d3d/d49/f4f [0,4194304] 0 2024-05-06T12:14:45.766 INFO:tasks.workunit.client.0.smithi092.stdout:1/875: dwrite d3/d10/d13/d19/f1c [0,4194304] 0 2024-05-06T12:14:45.770 INFO:tasks.workunit.client.0.smithi092.stdout:4/966: rename d9/df/d62/dbb/fc9 to d9/f13f 0 2024-05-06T12:14:45.770 INFO:tasks.workunit.client.0.smithi092.stdout:9/879: link d16/d1e/da1/d78/db4/fdb d16/d1c/d58/d92/dab/deb/f11e 0 2024-05-06T12:14:45.772 INFO:tasks.workunit.client.0.smithi092.stdout:6/964: symlink d4/d4c/d91/d110/l132 0 2024-05-06T12:14:45.773 INFO:tasks.workunit.client.1.smithi190.stdout:8/402: truncate d3/f17 409793 0 2024-05-06T12:14:45.775 INFO:tasks.workunit.client.0.smithi092.stdout:3/694: fsync d8/d24/d33/f64 0 2024-05-06T12:14:45.777 INFO:tasks.workunit.client.0.smithi092.stdout:3/695: chown d8/d24/d33/d4b/f41 1 1 2024-05-06T12:14:45.778 INFO:tasks.workunit.client.0.smithi092.stdout:4/967: creat d9/d2b/d65/d83/f140 x:0 0 0 2024-05-06T12:14:45.778 INFO:tasks.workunit.client.1.smithi190.stdout:8/403: rename d3/d13/d4a to d3/d8/d25/d3b/d90 0 2024-05-06T12:14:45.779 INFO:tasks.workunit.client.1.smithi190.stdout:8/404: write d3/d8/d68/f6d [136164,16684] 0 2024-05-06T12:14:45.780 INFO:tasks.workunit.client.1.smithi190.stdout:8/405: chown d3/d8/d2b/l64 2 1 2024-05-06T12:14:45.781 INFO:tasks.workunit.client.0.smithi092.stdout:6/965: dread d4/d20/d22/fdf [0,4194304] 0 2024-05-06T12:14:45.783 INFO:tasks.workunit.client.1.smithi190.stdout:8/406: creat d3/d35/d3a/f91 x:0 0 0 2024-05-06T12:14:45.785 INFO:tasks.workunit.client.1.smithi190.stdout:4/240: truncate d2/f37 554094 0 2024-05-06T12:14:45.786 INFO:tasks.workunit.client.0.smithi092.stdout:7/975: creat d9/de/d10/d17/d99/dc3/f14b x:0 0 0 2024-05-06T12:14:45.786 INFO:tasks.workunit.client.1.smithi190.stdout:8/407: read d3/f6 [7999500,103572] 0 2024-05-06T12:14:45.787 INFO:tasks.workunit.client.1.smithi190.stdout:8/408: write d3/d8/d2b/f80 [1027929,7496] 0 2024-05-06T12:14:45.789 INFO:tasks.workunit.client.1.smithi190.stdout:8/409: dread - d3/d35/d3a/f91 zero size 2024-05-06T12:14:45.789 INFO:tasks.workunit.client.1.smithi190.stdout:4/241: creat d2/d11/d16/d53/f57 x:0 0 0 2024-05-06T12:14:45.791 INFO:tasks.workunit.client.0.smithi092.stdout:1/876: link d3/c110 d3/d42/d84/d4f/d58/d65/d81/c11e 0 2024-05-06T12:14:45.792 INFO:tasks.workunit.client.0.smithi092.stdout:3/696: getdents d8/d24/d33 0 2024-05-06T12:14:45.793 INFO:tasks.workunit.client.0.smithi092.stdout:3/697: readlink d8/d1b/d73/l95 0 2024-05-06T12:14:45.793 INFO:tasks.workunit.client.1.smithi190.stdout:8/410: creat d3/d8/d25/d3b/d90/f92 x:0 0 0 2024-05-06T12:14:45.794 INFO:tasks.workunit.client.0.smithi092.stdout:1/877: mknod d3/d10/d13/d19/d21/d88/d8b/c11f 0 2024-05-06T12:14:45.795 INFO:tasks.workunit.client.1.smithi190.stdout:8/411: dread - d3/d35/d55/d61/f63 zero size 2024-05-06T12:14:45.795 INFO:tasks.workunit.client.1.smithi190.stdout:4/242: fdatasync d2/f1e 0 2024-05-06T12:14:45.800 INFO:tasks.workunit.client.1.smithi190.stdout:7/273: dread d0/d1/d2/f5 [0,4194304] 0 2024-05-06T12:14:45.801 INFO:tasks.workunit.client.1.smithi190.stdout:8/412: unlink d3/d8/d25/f5b 0 2024-05-06T12:14:45.806 INFO:tasks.workunit.client.1.smithi190.stdout:4/243: creat d2/d10/f58 x:0 0 0 2024-05-06T12:14:45.806 INFO:tasks.workunit.client.1.smithi190.stdout:8/413: read d3/d8/d2b/f80 [123769,3372] 0 2024-05-06T12:14:45.806 INFO:tasks.workunit.client.1.smithi190.stdout:3/348: rmdir d5/d7/d15/d1f/d33 39 2024-05-06T12:14:45.806 INFO:tasks.workunit.client.1.smithi190.stdout:3/349: stat d5/d14/d11 0 2024-05-06T12:14:45.807 INFO:tasks.workunit.client.1.smithi190.stdout:8/414: dread d3/f18 [0,4194304] 0 2024-05-06T12:14:45.808 INFO:tasks.workunit.client.1.smithi190.stdout:8/415: write d3/d13/f21 [446355,32509] 0 2024-05-06T12:14:45.815 INFO:tasks.workunit.client.1.smithi190.stdout:8/416: readlink d3/d35/l50 0 2024-05-06T12:14:45.815 INFO:tasks.workunit.client.1.smithi190.stdout:6/303: write fb [485504,28935] 0 2024-05-06T12:14:45.815 INFO:tasks.workunit.client.1.smithi190.stdout:3/350: symlink d5/l7e 0 2024-05-06T12:14:45.817 INFO:tasks.workunit.client.1.smithi190.stdout:6/304: write df/d2c/d1e/f2d [2637542,113539] 0 2024-05-06T12:14:45.818 INFO:tasks.workunit.client.1.smithi190.stdout:7/274: dread d0/d1/f21 [0,4194304] 0 2024-05-06T12:14:45.822 INFO:tasks.workunit.client.1.smithi190.stdout:7/275: symlink d0/d1/d18/l66 0 2024-05-06T12:14:45.824 INFO:tasks.workunit.client.1.smithi190.stdout:8/417: rename d3/d8/d25/d3b/f4e to d3/d8/d25/d65/f93 0 2024-05-06T12:14:45.824 INFO:tasks.workunit.client.1.smithi190.stdout:4/244: rmdir d2/d11/d16/d53 39 2024-05-06T12:14:45.828 INFO:tasks.workunit.client.1.smithi190.stdout:7/276: dwrite d0/da/d36/f43 [0,4194304] 0 2024-05-06T12:14:45.830 INFO:tasks.workunit.client.1.smithi190.stdout:8/418: creat d3/d8/d25/d3b/d90/d6e/f94 x:0 0 0 2024-05-06T12:14:45.831 INFO:tasks.workunit.client.1.smithi190.stdout:7/277: readlink d0/d14/d5d/l28 0 2024-05-06T12:14:45.833 INFO:tasks.workunit.client.1.smithi190.stdout:8/419: rmdir d3/d8/d68 39 2024-05-06T12:14:45.833 INFO:tasks.workunit.client.1.smithi190.stdout:8/420: dread - d3/d8/d2b/f7e zero size 2024-05-06T12:14:45.834 INFO:tasks.workunit.client.1.smithi190.stdout:4/245: creat d2/f59 x:0 0 0 2024-05-06T12:14:45.836 INFO:tasks.workunit.client.1.smithi190.stdout:4/246: rename d2/d11/d16/f4f to d2/d11/d16/d4e/f5a 0 2024-05-06T12:14:45.838 INFO:tasks.workunit.client.1.smithi190.stdout:4/247: write d2/d11/d16/d53/f57 [500380,8672] 0 2024-05-06T12:14:45.848 INFO:tasks.workunit.client.0.smithi092.stdout:5/924: sync 2024-05-06T12:14:45.849 INFO:tasks.workunit.client.0.smithi092.stdout:3/698: symlink d8/db/d72/lda 0 2024-05-06T12:14:45.850 INFO:tasks.workunit.client.1.smithi190.stdout:4/248: dread d2/f1e [0,4194304] 0 2024-05-06T12:14:45.852 INFO:tasks.workunit.client.0.smithi092.stdout:5/925: chown d4/d5/d10/da1/lbc 3 1 2024-05-06T12:14:45.852 INFO:tasks.workunit.client.0.smithi092.stdout:3/699: read d8/db/d4c/d3d/d49/f4f [2466266,36313] 0 2024-05-06T12:14:45.852 INFO:tasks.workunit.client.0.smithi092.stdout:9/880: write d16/f2a [272197,43690] 0 2024-05-06T12:14:45.854 INFO:tasks.workunit.client.0.smithi092.stdout:1/878: mknod d3/d10/d13/d19/c120 0 2024-05-06T12:14:45.854 INFO:tasks.workunit.client.0.smithi092.stdout:4/968: write d9/df/d48/f56 [1734167,54473] 0 2024-05-06T12:14:45.855 INFO:tasks.workunit.client.0.smithi092.stdout:4/969: fdatasync d9/d27/f50 0 2024-05-06T12:14:45.862 INFO:tasks.workunit.client.1.smithi190.stdout:3/351: truncate d5/d14/d11/f1b 964074 0 2024-05-06T12:14:45.862 INFO:tasks.workunit.client.1.smithi190.stdout:3/352: read - d5/f5e zero size 2024-05-06T12:14:45.863 INFO:tasks.workunit.client.1.smithi190.stdout:7/278: truncate d0/d1/f3e 612633 0 2024-05-06T12:14:45.863 INFO:tasks.workunit.client.1.smithi190.stdout:7/279: readlink d0/d14/l61 0 2024-05-06T12:14:45.865 INFO:tasks.workunit.client.0.smithi092.stdout:1/879: truncate d3/d10/f37 3861147 0 2024-05-06T12:14:45.865 INFO:tasks.workunit.client.1.smithi190.stdout:3/353: unlink d5/d7/d15/f16 0 2024-05-06T12:14:45.867 INFO:tasks.workunit.client.0.smithi092.stdout:6/966: dread d4/d20/d23/d30/f5e [0,4194304] 0 2024-05-06T12:14:45.867 INFO:tasks.workunit.client.0.smithi092.stdout:3/700: dread f3 [4194304,4194304] 0 2024-05-06T12:14:45.867 INFO:tasks.workunit.client.0.smithi092.stdout:9/881: dread - d16/d2c/d4b/fae zero size 2024-05-06T12:14:45.868 INFO:tasks.workunit.client.1.smithi190.stdout:7/280: mknod d0/d4c/c67 0 2024-05-06T12:14:45.869 INFO:tasks.workunit.client.1.smithi190.stdout:2/333: sync 2024-05-06T12:14:45.869 INFO:tasks.workunit.client.1.smithi190.stdout:3/354: mkdir d5/d7/d7f 0 2024-05-06T12:14:45.871 INFO:tasks.workunit.client.0.smithi092.stdout:1/880: read d3/d10/d13/d19/d21/f24 [3119494,128516] 0 2024-05-06T12:14:45.871 INFO:tasks.workunit.client.0.smithi092.stdout:6/967: fsync d4/d24/d4e/ff1 0 2024-05-06T12:14:45.871 INFO:tasks.workunit.client.1.smithi190.stdout:7/281: creat d0/d1/d18/f68 x:0 0 0 2024-05-06T12:14:45.872 INFO:tasks.workunit.client.0.smithi092.stdout:3/701: mknod d8/db/d4c/d40/d7a/d90/cdb 0 2024-05-06T12:14:45.873 INFO:tasks.workunit.client.0.smithi092.stdout:4/970: dwrite d9/f43 [0,4194304] 0 2024-05-06T12:14:45.874 INFO:tasks.workunit.client.0.smithi092.stdout:5/926: dwrite d4/f4c [0,4194304] 0 2024-05-06T12:14:45.874 INFO:tasks.workunit.client.1.smithi190.stdout:7/282: creat d0/f69 x:0 0 0 2024-05-06T12:14:45.874 INFO:tasks.workunit.client.1.smithi190.stdout:3/355: write d5/d7/d15/d1f/d33/f53 [160626,27594] 0 2024-05-06T12:14:45.875 INFO:tasks.workunit.client.0.smithi092.stdout:6/968: readlink d4/d24/d38/d5d/d6d/l74 0 2024-05-06T12:14:45.875 INFO:tasks.workunit.client.1.smithi190.stdout:8/421: truncate d3/f17 126215 0 2024-05-06T12:14:45.875 INFO:tasks.workunit.client.1.smithi190.stdout:4/249: dwrite d2/d11/f20 [0,4194304] 0 2024-05-06T12:14:45.876 INFO:tasks.workunit.client.0.smithi092.stdout:6/969: stat d4/d4c/d52/f6a 0 2024-05-06T12:14:45.878 INFO:tasks.workunit.client.1.smithi190.stdout:3/356: write d5/d7/d15/d1f/d33/d5c/f63 [996613,43265] 0 2024-05-06T12:14:45.880 INFO:tasks.workunit.client.1.smithi190.stdout:8/422: symlink d3/d8/d25/d65/l95 0 2024-05-06T12:14:45.880 INFO:tasks.workunit.client.1.smithi190.stdout:3/357: dread d5/d14/f6a [0,4194304] 0 2024-05-06T12:14:45.884 INFO:tasks.workunit.client.0.smithi092.stdout:1/881: dwrite d3/d10/f53 [0,4194304] 0 2024-05-06T12:14:45.886 INFO:tasks.workunit.client.0.smithi092.stdout:6/970: unlink d4/d4c/d52/f70 0 2024-05-06T12:14:45.887 INFO:tasks.workunit.client.1.smithi190.stdout:8/423: creat d3/d35/d55/f96 x:0 0 0 2024-05-06T12:14:45.888 INFO:tasks.workunit.client.1.smithi190.stdout:8/424: truncate d3/d8/d25/d3b/d90/f4f 2255860 0 2024-05-06T12:14:45.889 INFO:tasks.workunit.client.0.smithi092.stdout:3/702: creat d8/d1b/d73/db7/fdc x:0 0 0 2024-05-06T12:14:45.890 INFO:tasks.workunit.client.0.smithi092.stdout:4/971: mknod d9/df/c141 0 2024-05-06T12:14:45.893 INFO:tasks.workunit.client.1.smithi190.stdout:7/283: dread d0/d1/f3e [0,4194304] 0 2024-05-06T12:14:45.894 INFO:tasks.workunit.client.1.smithi190.stdout:7/284: write d0/d1/fb [1070489,40238] 0 2024-05-06T12:14:45.896 INFO:tasks.workunit.client.0.smithi092.stdout:1/882: dread d3/d10/d13/d19/d21/d72/f86 [0,4194304] 0 2024-05-06T12:14:45.897 INFO:tasks.workunit.client.1.smithi190.stdout:4/250: dread d2/f23 [0,4194304] 0 2024-05-06T12:14:45.898 INFO:tasks.workunit.client.0.smithi092.stdout:5/927: link d4/d5/d10/d31/c69 d4/d34/d28/d4e/d67/c13f 0 2024-05-06T12:14:45.898 INFO:tasks.workunit.client.1.smithi190.stdout:7/285: mkdir d0/d14/d6a 0 2024-05-06T12:14:45.900 INFO:tasks.workunit.client.1.smithi190.stdout:4/251: mknod d2/d11/d16/d53/c5b 0 2024-05-06T12:14:45.903 INFO:tasks.workunit.client.0.smithi092.stdout:1/883: write d3/f98 [4480096,17228] 0 2024-05-06T12:14:45.903 INFO:tasks.workunit.client.0.smithi092.stdout:6/971: getdents d4/d4c/da9 0 2024-05-06T12:14:45.904 INFO:tasks.workunit.client.1.smithi190.stdout:4/252: creat d2/d11/d16/d31/f5c x:0 0 0 2024-05-06T12:14:45.905 INFO:tasks.workunit.client.0.smithi092.stdout:5/928: chown d4/d5/d10/d31/ddf/da3/fd2 2411 1 2024-05-06T12:14:45.906 INFO:tasks.workunit.client.0.smithi092.stdout:9/882: dread d16/d2c/d2f/d3d/d33/f3b [0,4194304] 0 2024-05-06T12:14:45.907 INFO:tasks.workunit.client.1.smithi190.stdout:7/286: unlink d0/d14/d5d/c59 0 2024-05-06T12:14:45.908 INFO:tasks.workunit.client.0.smithi092.stdout:5/929: write d4/d5/d30/d10a/f111 [97813,11824] 0 2024-05-06T12:14:45.908 INFO:tasks.workunit.client.1.smithi190.stdout:4/253: symlink d2/d11/l5d 0 2024-05-06T12:14:45.909 INFO:tasks.workunit.client.1.smithi190.stdout:7/287: creat d0/d4/f6b x:0 0 0 2024-05-06T12:14:45.912 INFO:tasks.workunit.client.1.smithi190.stdout:7/288: mkdir d0/d1/d18/d6c 0 2024-05-06T12:14:45.913 INFO:tasks.workunit.client.1.smithi190.stdout:7/289: chown d0/d4c/f52 100795 1 2024-05-06T12:14:45.913 INFO:tasks.workunit.client.0.smithi092.stdout:5/930: getdents d4/d34/d28 0 2024-05-06T12:14:45.914 INFO:tasks.workunit.client.1.smithi190.stdout:7/290: rename d0 to d0/d1/d32/d45/d6d 22 2024-05-06T12:14:45.914 INFO:tasks.workunit.client.1.smithi190.stdout:4/254: dwrite d2/f32 [0,4194304] 0 2024-05-06T12:14:45.915 INFO:tasks.workunit.client.0.smithi092.stdout:1/884: dread d3/d10/d13/d46/f61 [0,4194304] 0 2024-05-06T12:14:45.916 INFO:tasks.workunit.client.0.smithi092.stdout:9/883: dwrite d16/d2c/d2f/d3d/d33/f9a [0,4194304] 0 2024-05-06T12:14:45.918 INFO:tasks.workunit.client.0.smithi092.stdout:1/885: stat d3/d10/d13/d19/d21/d72/f86 0 2024-05-06T12:14:45.919 INFO:tasks.workunit.client.0.smithi092.stdout:5/931: mknod d4/d5/d10/d31/ddf/d54/db0/d136/c140 0 2024-05-06T12:14:45.923 INFO:tasks.workunit.client.1.smithi190.stdout:7/291: dread d0/da/f2e [0,4194304] 0 2024-05-06T12:14:45.923 INFO:tasks.workunit.client.1.smithi190.stdout:4/255: rename d2/f23 to d2/f5e 0 2024-05-06T12:14:45.925 INFO:tasks.workunit.client.0.smithi092.stdout:5/932: stat d4/d5/d10/cec 0 2024-05-06T12:14:45.926 INFO:tasks.workunit.client.0.smithi092.stdout:3/703: truncate d8/db/d4c/d40/d7a/fa3 3189980 0 2024-05-06T12:14:45.929 INFO:tasks.workunit.client.1.smithi190.stdout:3/358: dwrite d5/d7/d15/f4f [0,4194304] 0 2024-05-06T12:14:45.930 INFO:tasks.workunit.client.0.smithi092.stdout:5/933: dread - d4/d5/d30/dfd/d108/f12d zero size 2024-05-06T12:14:45.930 INFO:tasks.workunit.client.0.smithi092.stdout:1/886: dwrite d3/d10/d13/ff3 [0,4194304] 0 2024-05-06T12:14:45.930 INFO:tasks.workunit.client.1.smithi190.stdout:4/256: write d2/d11/f14 [1063992,58547] 0 2024-05-06T12:14:45.932 INFO:tasks.workunit.client.1.smithi190.stdout:7/292: creat d0/d14/f6e x:0 0 0 2024-05-06T12:14:45.933 INFO:tasks.workunit.client.1.smithi190.stdout:4/257: dread d2/d11/d16/f1f [0,4194304] 0 2024-05-06T12:14:45.933 INFO:tasks.workunit.client.1.smithi190.stdout:8/425: dread d3/f6 [4194304,4194304] 0 2024-05-06T12:14:45.934 INFO:tasks.workunit.client.0.smithi092.stdout:5/934: unlink d4/d34/c13 0 2024-05-06T12:14:45.936 INFO:tasks.workunit.client.0.smithi092.stdout:5/935: dread - d4/d34/d28/d4e/d5d/d82/fce zero size 2024-05-06T12:14:45.936 INFO:tasks.workunit.client.0.smithi092.stdout:1/887: unlink d3/d10/d13/d19/fdc 0 2024-05-06T12:14:45.937 INFO:tasks.workunit.client.1.smithi190.stdout:7/293: mkdir d0/da/d36/d53/d6f 0 2024-05-06T12:14:45.937 INFO:tasks.workunit.client.0.smithi092.stdout:3/704: dwrite d8/d1b/f3e [0,4194304] 0 2024-05-06T12:14:45.939 INFO:tasks.workunit.client.0.smithi092.stdout:5/936: chown d4/d34/dae/laa 9207883 1 2024-05-06T12:14:45.941 INFO:tasks.workunit.client.0.smithi092.stdout:9/884: write d16/d2c/d2f/d3d/f38 [5032243,32836] 0 2024-05-06T12:14:45.943 INFO:tasks.workunit.client.1.smithi190.stdout:7/294: rename d0/d1/d2f to d0/d1/d32/d45/d70 0 2024-05-06T12:14:45.946 INFO:tasks.workunit.client.0.smithi092.stdout:5/937: creat d4/d5/d10/d31/ddf/d54/db0/d102/d13c/f141 x:0 0 0 2024-05-06T12:14:45.948 INFO:tasks.workunit.client.0.smithi092.stdout:3/705: creat d8/db/d4c/d3d/fdd x:0 0 0 2024-05-06T12:14:45.949 INFO:tasks.workunit.client.0.smithi092.stdout:3/706: stat d8/db/f4d 0 2024-05-06T12:14:45.950 INFO:tasks.workunit.client.0.smithi092.stdout:9/885: link d16/d2c/d4b/cac d16/d1c/d58/c11f 0 2024-05-06T12:14:45.951 INFO:tasks.workunit.client.0.smithi092.stdout:3/707: dread - d8/db/f9f zero size 2024-05-06T12:14:45.951 INFO:tasks.workunit.client.1.smithi190.stdout:7/295: dwrite d0/da/d3d/f63 [0,4194304] 0 2024-05-06T12:14:45.954 INFO:tasks.workunit.client.1.smithi190.stdout:7/296: mknod d0/d1/d18/d2b/c71 0 2024-05-06T12:14:45.957 INFO:tasks.workunit.client.0.smithi092.stdout:5/938: dwrite d4/d5/d10/d31/ddf/d6c/f12e [0,4194304] 0 2024-05-06T12:14:45.958 INFO:tasks.workunit.client.0.smithi092.stdout:5/939: dread - d4/d5/f12c zero size 2024-05-06T12:14:45.959 INFO:tasks.workunit.client.0.smithi092.stdout:5/940: dread - d4/d5/d30/dfd/d108/f11e zero size 2024-05-06T12:14:45.964 INFO:tasks.workunit.client.0.smithi092.stdout:9/886: dread d16/d2c/d3f/f46 [0,4194304] 0 2024-05-06T12:14:45.964 INFO:tasks.workunit.client.0.smithi092.stdout:3/708: read d8/d24/d33/d4b/d74/f9e [326739,129616] 0 2024-05-06T12:14:45.965 INFO:tasks.workunit.client.0.smithi092.stdout:3/709: chown d8/db/d72/c75 92366776 1 2024-05-06T12:14:45.966 INFO:tasks.workunit.client.0.smithi092.stdout:5/941: creat d4/d34/d28/d4e/d4b/f142 x:0 0 0 2024-05-06T12:14:45.967 INFO:tasks.workunit.client.0.smithi092.stdout:6/972: dread d4/d20/f83 [0,4194304] 0 2024-05-06T12:14:45.970 INFO:tasks.workunit.client.0.smithi092.stdout:6/973: truncate d4/d4c/da9/fd2 8924399 0 2024-05-06T12:14:45.972 INFO:tasks.workunit.client.1.smithi190.stdout:3/359: write d5/d7/d15/f4b [781521,115263] 0 2024-05-06T12:14:45.973 INFO:tasks.workunit.client.1.smithi190.stdout:3/360: chown d5/d7/d15/d1f/d33/d40 41322 1 2024-05-06T12:14:45.974 INFO:tasks.workunit.client.0.smithi092.stdout:6/974: dread d4/d20/d23/fe9 [0,4194304] 0 2024-05-06T12:14:45.976 INFO:tasks.workunit.client.1.smithi190.stdout:3/361: fsync d5/d14/d11/d21/f2e 0 2024-05-06T12:14:45.977 INFO:tasks.workunit.client.1.smithi190.stdout:8/426: write d3/d8/d25/d3b/d90/f7c [2510968,29780] 0 2024-05-06T12:14:45.977 INFO:tasks.workunit.client.0.smithi092.stdout:6/975: symlink d4/d20/d23/d30/l133 0 2024-05-06T12:14:45.978 INFO:tasks.workunit.client.1.smithi190.stdout:8/427: dread - d3/d35/d3a/f91 zero size 2024-05-06T12:14:45.979 INFO:tasks.workunit.client.1.smithi190.stdout:8/428: chown d3/d8/d25/d3b 6036263 1 2024-05-06T12:14:45.979 INFO:tasks.workunit.client.1.smithi190.stdout:8/429: fdatasync d3/d35/d55/f96 0 2024-05-06T12:14:45.980 INFO:tasks.workunit.client.0.smithi092.stdout:6/976: stat d4/d24/d26/dae/l113 0 2024-05-06T12:14:45.980 INFO:tasks.workunit.client.1.smithi190.stdout:3/362: mkdir d5/d14/d11/d21/d44/d80 0 2024-05-06T12:14:45.981 INFO:tasks.workunit.client.0.smithi092.stdout:6/977: stat d4/d20/d22 0 2024-05-06T12:14:45.983 INFO:tasks.workunit.client.1.smithi190.stdout:3/363: mknod d5/d7/d7f/c81 0 2024-05-06T12:14:45.985 INFO:tasks.workunit.client.1.smithi190.stdout:8/430: rename d3/d8/d68/f6d to d3/d8/d25/d3b/d6b/d7a/f97 0 2024-05-06T12:14:45.988 INFO:tasks.workunit.client.1.smithi190.stdout:3/364: rename d5/d14/d11/d2a to d5/d14/d11/d21/d44/d80/d82 0 2024-05-06T12:14:45.988 INFO:tasks.workunit.client.0.smithi092.stdout:9/887: truncate d16/d2c/d2f/d3d/f61 224310 0 2024-05-06T12:14:45.989 INFO:tasks.workunit.client.0.smithi092.stdout:1/888: dread d3/d10/d13/d19/d21/d72/d9a/da4/fba [0,4194304] 0 2024-05-06T12:14:45.989 INFO:tasks.workunit.client.1.smithi190.stdout:8/431: creat d3/f98 x:0 0 0 2024-05-06T12:14:45.990 INFO:tasks.workunit.client.0.smithi092.stdout:9/888: chown d16/d1e/da1/d78/fe8 23342 1 2024-05-06T12:14:45.990 INFO:tasks.workunit.client.1.smithi190.stdout:3/365: dread d5/d7/d15/f4b [0,4194304] 0 2024-05-06T12:14:45.991 INFO:tasks.workunit.client.0.smithi092.stdout:6/978: truncate d4/d24/d54/f79 159244 0 2024-05-06T12:14:45.991 INFO:tasks.workunit.client.1.smithi190.stdout:8/432: write d3/d35/d55/d61/f63 [389754,56770] 0 2024-05-06T12:14:45.992 INFO:tasks.workunit.client.0.smithi092.stdout:3/710: dwrite d8/d24/d33/d39/d5b/fc4 [0,4194304] 0 2024-05-06T12:14:45.992 INFO:tasks.workunit.client.1.smithi190.stdout:8/433: truncate d3/d8/f27 896351 0 2024-05-06T12:14:45.993 INFO:tasks.workunit.client.1.smithi190.stdout:3/366: read d5/d7/d15/f4b [263939,102486] 0 2024-05-06T12:14:45.993 INFO:tasks.workunit.client.0.smithi092.stdout:9/889: chown d16/d2c/d2f/d3d/d33/d7e/l11a 6 1 2024-05-06T12:14:45.994 INFO:tasks.workunit.client.0.smithi092.stdout:5/942: dread d4/d5/d10/d31/ddf/d54/db0/d102/d78/f124 [0,4194304] 0 2024-05-06T12:14:45.994 INFO:tasks.workunit.client.1.smithi190.stdout:8/434: chown d3/d8/d2b/d3e/f8e 1 1 2024-05-06T12:14:45.994 INFO:tasks.workunit.client.0.smithi092.stdout:3/711: chown d8/db/d4c/d3d/cb3 769836 1 2024-05-06T12:14:45.995 INFO:tasks.workunit.client.1.smithi190.stdout:3/367: unlink d5/d7/d15/f4f 0 2024-05-06T12:14:45.996 INFO:tasks.workunit.client.0.smithi092.stdout:6/979: read d4/d4c/d52/f107 [3638165,91962] 0 2024-05-06T12:14:45.996 INFO:tasks.workunit.client.1.smithi190.stdout:3/368: readlink d5/d14/d11/d21/d44/d80/d82/l72 0 2024-05-06T12:14:45.997 INFO:tasks.workunit.client.0.smithi092.stdout:1/889: truncate d3/d42/d84/d9f/fb7 986900 0 2024-05-06T12:14:45.998 INFO:tasks.workunit.client.0.smithi092.stdout:9/890: fsync d16/d2c/d2f/d3d/d33/f87 0 2024-05-06T12:14:45.999 INFO:tasks.workunit.client.0.smithi092.stdout:3/712: symlink d8/db/d72/lde 0 2024-05-06T12:14:46.005 INFO:tasks.workunit.client.0.smithi092.stdout:5/943: dwrite d4/d5/d10/d31/ddf/d6c/f12e [4194304,4194304] 0 2024-05-06T12:14:46.005 INFO:tasks.workunit.client.0.smithi092.stdout:1/890: rmdir d3/d10/d13/d19/d21/d72/d9a 39 2024-05-06T12:14:46.009 INFO:tasks.workunit.client.0.smithi092.stdout:5/944: dread - d4/d5/d10/d31/ddf/d54/fa2 zero size 2024-05-06T12:14:46.010 INFO:tasks.workunit.client.1.smithi190.stdout:8/435: write d3/d8/d2b/d3e/f84 [231345,45149] 0 2024-05-06T12:14:46.011 INFO:tasks.workunit.client.0.smithi092.stdout:5/945: dread - d4/d5/d30/dfd/d108/f11e zero size 2024-05-06T12:14:46.013 INFO:tasks.workunit.client.0.smithi092.stdout:5/946: chown d4/d5/d10/d31/d53/ddb/d104 202170912 1 2024-05-06T12:14:46.016 INFO:tasks.workunit.client.1.smithi190.stdout:3/369: dwrite d5/d14/d11/d21/f38 [0,4194304] 0 2024-05-06T12:14:46.017 INFO:tasks.workunit.client.0.smithi092.stdout:5/947: rename d4/d34/d28/c6d to d4/d59/c143 0 2024-05-06T12:14:46.018 INFO:tasks.workunit.client.0.smithi092.stdout:5/948: write d4/d5/d10/d31/d53/d64/f25 [6629749,122484] 0 2024-05-06T12:14:46.019 INFO:tasks.workunit.client.0.smithi092.stdout:1/891: truncate d3/d10/db3/fc4 1110361 0 2024-05-06T12:14:46.019 INFO:tasks.workunit.client.1.smithi190.stdout:3/370: creat d5/d14/d11/d21/d44/d80/f83 x:0 0 0 2024-05-06T12:14:46.019 INFO:tasks.workunit.client.0.smithi092.stdout:6/980: write d4/d20/f31 [3973108,94262] 0 2024-05-06T12:14:46.022 INFO:tasks.workunit.client.0.smithi092.stdout:5/949: chown d4/d5/d10/d31/ddf/d54/db0/d102/d20/f100 2048376 1 2024-05-06T12:14:46.024 INFO:tasks.workunit.client.1.smithi190.stdout:3/371: rename d5/d14/d52 to d5/d7/d15/d1f/d84 0 2024-05-06T12:14:46.025 INFO:tasks.workunit.client.1.smithi190.stdout:3/372: write d5/d7/d15/d1f/fe [1106010,48273] 0 2024-05-06T12:14:46.027 INFO:tasks.workunit.client.1.smithi190.stdout:8/436: write d3/d8/d2b/f80 [1634189,97409] 0 2024-05-06T12:14:46.028 INFO:tasks.workunit.client.0.smithi092.stdout:1/892: dwrite d3/d10/d13/fbb [0,4194304] 0 2024-05-06T12:14:46.031 INFO:tasks.workunit.client.1.smithi190.stdout:3/373: dread d5/fc [0,4194304] 0 2024-05-06T12:14:46.031 INFO:tasks.workunit.client.1.smithi190.stdout:8/437: creat d3/d8/d25/d3b/d6b/f99 x:0 0 0 2024-05-06T12:14:46.041 INFO:tasks.workunit.client.1.smithi190.stdout:3/374: dwrite d5/f27 [4194304,4194304] 0 2024-05-06T12:14:46.042 INFO:tasks.workunit.client.1.smithi190.stdout:8/438: dwrite d3/f98 [0,4194304] 0 2024-05-06T12:14:46.046 INFO:tasks.workunit.client.1.smithi190.stdout:8/439: chown d3/d8/d25/d3b/d6b/d7a/f97 0 1 2024-05-06T12:14:46.048 INFO:tasks.workunit.client.1.smithi190.stdout:8/440: fdatasync d3/d8/d2b/f38 0 2024-05-06T12:14:46.052 INFO:tasks.workunit.client.1.smithi190.stdout:3/375: dwrite d5/f5e [0,4194304] 0 2024-05-06T12:14:46.053 INFO:tasks.workunit.client.1.smithi190.stdout:3/376: chown d5/d14/d11/d21/l7a 9474 1 2024-05-06T12:14:46.054 INFO:tasks.workunit.client.1.smithi190.stdout:8/441: rename d3/d8/d25/d3b/d49/c60 to d3/d35/d55/d61/c9a 0 2024-05-06T12:14:46.055 INFO:tasks.workunit.client.1.smithi190.stdout:3/377: creat d5/d14/d11/d21/d44/f85 x:0 0 0 2024-05-06T12:14:46.057 INFO:tasks.workunit.client.1.smithi190.stdout:9/473: sync 2024-05-06T12:14:46.058 INFO:tasks.workunit.client.1.smithi190.stdout:3/378: creat d5/d7/f86 x:0 0 0 2024-05-06T12:14:46.058 INFO:tasks.workunit.client.1.smithi190.stdout:8/442: creat d3/d8/d25/f9b x:0 0 0 2024-05-06T12:14:46.059 INFO:tasks.workunit.client.1.smithi190.stdout:3/379: write d5/d14/d11/f50 [1453826,102212] 0 2024-05-06T12:14:46.060 INFO:tasks.workunit.client.1.smithi190.stdout:8/443: chown d3/d8/d2b/d3e/f8d 35239969 1 2024-05-06T12:14:46.060 INFO:tasks.workunit.client.0.smithi092.stdout:6/981: dread d4/fb [0,4194304] 0 2024-05-06T12:14:46.066 INFO:tasks.workunit.client.0.smithi092.stdout:3/713: write d8/d24/d33/d91/f9b [200748,88224] 0 2024-05-06T12:14:46.067 INFO:tasks.workunit.client.0.smithi092.stdout:3/714: chown d8/db/d4c/cbf 503932 1 2024-05-06T12:14:46.075 INFO:tasks.workunit.client.0.smithi092.stdout:9/891: dwrite d16/d2c/d2f/d3d/d33/d7e/dad/fa5 [0,4194304] 0 2024-05-06T12:14:46.076 INFO:tasks.workunit.client.1.smithi190.stdout:3/380: truncate d5/d14/f47 3191448 0 2024-05-06T12:14:46.078 INFO:tasks.workunit.client.0.smithi092.stdout:3/715: dwrite d8/d24/f77 [4194304,4194304] 0 2024-05-06T12:14:46.080 INFO:tasks.workunit.client.0.smithi092.stdout:9/892: truncate d16/d1e/d7b/d8d/da4/fd2 916059 0 2024-05-06T12:14:46.084 INFO:tasks.workunit.client.0.smithi092.stdout:3/716: mkdir d8/d24/d33/d4b/ddf 0 2024-05-06T12:14:46.084 INFO:tasks.workunit.client.0.smithi092.stdout:9/893: rmdir d16/d2c/d2f/d75 39 2024-05-06T12:14:46.087 INFO:tasks.workunit.client.0.smithi092.stdout:6/982: dread d4/d20/f1c [4194304,4194304] 0 2024-05-06T12:14:46.088 INFO:tasks.workunit.client.0.smithi092.stdout:6/983: dread - d4/d24/d26/dae/f120 zero size 2024-05-06T12:14:46.089 INFO:tasks.workunit.client.1.smithi190.stdout:9/474: dread db/d23/d25/d47/d4b/f43 [0,4194304] 0 2024-05-06T12:14:46.090 INFO:tasks.workunit.client.0.smithi092.stdout:9/894: truncate d16/d2c/d2f/d3d/d33/d57/f64 657912 0 2024-05-06T12:14:46.090 INFO:tasks.workunit.client.1.smithi190.stdout:9/475: chown db/d23/d25/l27 719862 1 2024-05-06T12:14:46.091 INFO:tasks.workunit.client.0.smithi092.stdout:6/984: fsync d4/d24/d26/dc4/d108/f10b 0 2024-05-06T12:14:46.093 INFO:tasks.workunit.client.1.smithi190.stdout:9/476: mkdir db/dd/d5c/dac 0 2024-05-06T12:14:46.094 INFO:tasks.workunit.client.0.smithi092.stdout:9/895: creat d16/d2c/d88/f120 x:0 0 0 2024-05-06T12:14:46.095 INFO:tasks.workunit.client.0.smithi092.stdout:3/717: link d8/d24/d33/d4b/cf d8/db/d14/d5a/ce0 0 2024-05-06T12:14:46.095 INFO:tasks.workunit.client.0.smithi092.stdout:9/896: chown d16/d1c/f2b 3308 1 2024-05-06T12:14:46.097 INFO:tasks.workunit.client.0.smithi092.stdout:9/897: dread - d16/d2c/d2f/db1/fcf zero size 2024-05-06T12:14:46.098 INFO:tasks.workunit.client.1.smithi190.stdout:9/477: dwrite db/dd/d16/d58/d87/f70 [0,4194304] 0 2024-05-06T12:14:46.099 INFO:tasks.workunit.client.0.smithi092.stdout:6/985: dwrite d4/d4c/da9/fd2 [0,4194304] 0 2024-05-06T12:14:46.100 INFO:tasks.workunit.client.1.smithi190.stdout:9/478: stat db/d23/l6a 0 2024-05-06T12:14:46.101 INFO:tasks.workunit.client.0.smithi092.stdout:3/718: mknod d8/db/d4c/d40/d7a/dbd/ce1 0 2024-05-06T12:14:46.104 INFO:tasks.workunit.client.0.smithi092.stdout:6/986: symlink d4/d24/d38/d5d/daf/l134 0 2024-05-06T12:14:46.108 INFO:tasks.workunit.client.1.smithi190.stdout:9/479: dwrite db/f13 [4194304,4194304] 0 2024-05-06T12:14:46.110 INFO:tasks.workunit.client.1.smithi190.stdout:3/381: dread d5/d7/d15/d1f/d33/d40/f45 [0,4194304] 0 2024-05-06T12:14:46.110 INFO:tasks.workunit.client.0.smithi092.stdout:6/987: mkdir d4/d20/d23/d135 0 2024-05-06T12:14:46.113 INFO:tasks.workunit.client.0.smithi092.stdout:6/988: mknod d4/d20/d22/c136 0 2024-05-06T12:14:46.115 INFO:tasks.workunit.client.1.smithi190.stdout:9/480: link db/dd/d5c/c7f db/d23/d25/d47/cad 0 2024-05-06T12:14:46.117 INFO:tasks.workunit.client.1.smithi190.stdout:3/382: getdents d5/d7/d6d 0 2024-05-06T12:14:46.118 INFO:tasks.workunit.client.0.smithi092.stdout:9/898: rename d16/d2c/d2f/d3d/d33/d57/f64 to d16/d2c/d2f/d3d/d33/d57/dff/f121 0 2024-05-06T12:14:46.119 INFO:tasks.workunit.client.1.smithi190.stdout:9/481: unlink db/f2b 0 2024-05-06T12:14:46.121 INFO:tasks.workunit.client.1.smithi190.stdout:9/482: creat db/dd/d16/fae x:0 0 0 2024-05-06T12:14:46.123 INFO:tasks.workunit.client.1.smithi190.stdout:9/483: readlink db/dd/df/l28 0 2024-05-06T12:14:46.125 INFO:tasks.workunit.client.1.smithi190.stdout:9/484: getdents db/dd/d16/d58/d87/d22/d49/d88/d9f 0 2024-05-06T12:14:46.126 INFO:tasks.workunit.client.0.smithi092.stdout:6/989: rename d4/d4c/d91/d93/d9d/d12a to d4/d4c/da9/d137 0 2024-05-06T12:14:46.127 INFO:tasks.workunit.client.0.smithi092.stdout:3/719: write d8/db/d4c/d40/d7a/fa3 [2235767,53766] 0 2024-05-06T12:14:46.127 INFO:tasks.workunit.client.0.smithi092.stdout:5/950: dread d4/d34/d28/d4e/f110 [0,4194304] 0 2024-05-06T12:14:46.128 INFO:tasks.workunit.client.0.smithi092.stdout:9/899: write d16/d2c/d2f/d3d/d33/f71 [4562455,111430] 0 2024-05-06T12:14:46.129 INFO:tasks.workunit.client.1.smithi190.stdout:9/485: link db/d23/f8a db/dd/d16/d58/d87/d22/d49/d88/faf 0 2024-05-06T12:14:46.130 INFO:tasks.workunit.client.0.smithi092.stdout:3/720: fdatasync d8/d24/d27/fa0 0 2024-05-06T12:14:46.134 INFO:tasks.workunit.client.1.smithi190.stdout:3/383: truncate d5/d14/d11/d21/d44/d80/d82/d3d/f6c 4124245 0 2024-05-06T12:14:46.135 INFO:tasks.workunit.client.1.smithi190.stdout:3/384: write d5/f51 [105543,102761] 0 2024-05-06T12:14:46.136 INFO:tasks.workunit.client.0.smithi092.stdout:6/990: dwrite d4/d24/d66/d105/f11e [0,4194304] 0 2024-05-06T12:14:46.136 INFO:tasks.workunit.client.1.smithi190.stdout:3/385: stat d5/d14/d11/d21/f30 0 2024-05-06T12:14:46.140 INFO:tasks.workunit.client.1.smithi190.stdout:9/486: truncate db/dd/d16/d58/d87/f85 1894507 0 2024-05-06T12:14:46.140 INFO:tasks.workunit.client.1.smithi190.stdout:9/487: chown db/dd/d16/l20 22 1 2024-05-06T12:14:46.141 INFO:tasks.workunit.client.1.smithi190.stdout:3/386: mknod d5/d14/d11/d21/d44/d80/c87 0 2024-05-06T12:14:46.144 INFO:tasks.workunit.client.1.smithi190.stdout:3/387: unlink d5/d14/d11/f50 0 2024-05-06T12:14:46.144 INFO:tasks.workunit.client.1.smithi190.stdout:9/488: read db/dd/d16/f4f [1556408,53477] 0 2024-05-06T12:14:46.145 INFO:tasks.workunit.client.1.smithi190.stdout:9/489: stat db/d23/f2a 0 2024-05-06T12:14:46.146 INFO:tasks.workunit.client.1.smithi190.stdout:9/490: write db/dd/d16/d4c/f57 [2475955,52025] 0 2024-05-06T12:14:46.150 INFO:tasks.workunit.client.1.smithi190.stdout:9/491: creat db/dd/d5c/fb0 x:0 0 0 2024-05-06T12:14:46.151 INFO:tasks.workunit.client.1.smithi190.stdout:9/492: stat db/dd/d16/d58/d87/d22/d49/d88 0 2024-05-06T12:14:46.151 INFO:tasks.workunit.client.0.smithi092.stdout:3/721: creat d8/db/d4c/d3d/d49/fe2 x:0 0 0 2024-05-06T12:14:46.154 INFO:tasks.workunit.client.0.smithi092.stdout:9/900: symlink d16/d1e/l122 0 2024-05-06T12:14:46.154 INFO:tasks.workunit.client.1.smithi190.stdout:3/388: rename d5/d7/d15/f4b to d5/d7/d15/d1f/f88 0 2024-05-06T12:14:46.155 INFO:tasks.workunit.client.0.smithi092.stdout:3/722: symlink d8/d1b/d73/le3 0 2024-05-06T12:14:46.158 INFO:tasks.workunit.client.0.smithi092.stdout:3/723: mkdir d8/de4 0 2024-05-06T12:14:46.159 INFO:tasks.workunit.client.0.smithi092.stdout:6/991: dwrite d4/d20/f83 [0,4194304] 0 2024-05-06T12:14:46.164 INFO:tasks.workunit.client.0.smithi092.stdout:9/901: dwrite d16/d2c/d2f/d3d/d33/f3b [0,4194304] 0 2024-05-06T12:14:46.165 INFO:tasks.workunit.client.0.smithi092.stdout:9/902: fdatasync d16/d1e/da1/d78/fe8 0 2024-05-06T12:14:46.168 INFO:tasks.workunit.client.0.smithi092.stdout:5/951: truncate d4/d5/d10/d31/f4a 2054754 0 2024-05-06T12:14:46.168 INFO:tasks.workunit.client.1.smithi190.stdout:3/389: dread d5/d7/d15/f28 [0,4194304] 0 2024-05-06T12:14:46.170 INFO:tasks.workunit.client.0.smithi092.stdout:5/952: chown d4/d5/d10/d31/ddf/d54/db0/d102/d20/f100 123226405 1 2024-05-06T12:14:46.170 INFO:tasks.workunit.client.0.smithi092.stdout:9/903: rename d16/d1e/d7b/dce/c4e to d16/d1e/da1/d78/db4/c123 0 2024-05-06T12:14:46.180 INFO:tasks.workunit.client.0.smithi092.stdout:9/904: rename d16/d2c/d2f/d3d/d33/c47 to d16/d2c/c124 0 2024-05-06T12:14:46.184 INFO:tasks.workunit.client.0.smithi092.stdout:9/905: readlink d16/d2c/d2f/d75/le6 0 2024-05-06T12:14:46.186 INFO:tasks.workunit.client.0.smithi092.stdout:9/906: creat d16/d2c/d2f/f125 x:0 0 0 2024-05-06T12:14:46.189 INFO:tasks.workunit.client.0.smithi092.stdout:9/907: mkdir d16/d1c/de5/d126 0 2024-05-06T12:14:46.190 INFO:tasks.workunit.client.0.smithi092.stdout:5/953: truncate d4/d5/d10/d31/d53/dcb/fff 715545 0 2024-05-06T12:14:46.195 INFO:tasks.workunit.client.0.smithi092.stdout:5/954: symlink d4/d5/d10/d31/ddf/da3/dc2/l144 0 2024-05-06T12:14:46.197 INFO:tasks.workunit.client.0.smithi092.stdout:9/908: dwrite d16/d2c/d2f/d75/f107 [0,4194304] 0 2024-05-06T12:14:46.197 INFO:tasks.workunit.client.0.smithi092.stdout:6/992: dread d4/d20/d23/d30/f33 [0,4194304] 0 2024-05-06T12:14:46.198 INFO:tasks.workunit.client.0.smithi092.stdout:5/955: chown d4/d34/d28/d4e/d5d/d82/fc0 20 1 2024-05-06T12:14:46.199 INFO:tasks.workunit.client.0.smithi092.stdout:6/993: write d4/d4c/d52/f6a [1582783,19499] 0 2024-05-06T12:14:46.204 INFO:tasks.workunit.client.0.smithi092.stdout:5/956: mknod d4/d5/d10/d31/ddf/d54/db0/d102/d78/c145 0 2024-05-06T12:14:46.209 INFO:tasks.workunit.client.0.smithi092.stdout:5/957: symlink d4/d5/d30/dfd/d131/l146 0 2024-05-06T12:14:46.210 INFO:tasks.workunit.client.0.smithi092.stdout:5/958: fdatasync d4/f68 0 2024-05-06T12:14:46.211 INFO:tasks.workunit.client.0.smithi092.stdout:5/959: dread - d4/d5/f12c zero size 2024-05-06T12:14:46.211 INFO:tasks.workunit.client.0.smithi092.stdout:6/994: dwrite d4/d4c/d52/f107 [0,4194304] 0 2024-05-06T12:14:46.212 INFO:tasks.workunit.client.0.smithi092.stdout:5/960: stat d4/d5/d10/d31/d53 0 2024-05-06T12:14:46.212 INFO:tasks.workunit.client.0.smithi092.stdout:6/995: readlink d4/d20/l21 0 2024-05-06T12:14:46.213 INFO:tasks.workunit.client.1.smithi190.stdout:0/468: sync 2024-05-06T12:14:46.219 INFO:tasks.workunit.client.0.smithi092.stdout:6/996: creat d4/d4c/d52/d6c/da0/dc3/f138 x:0 0 0 2024-05-06T12:14:46.221 INFO:tasks.workunit.client.0.smithi092.stdout:5/961: creat d4/d5/d10/d31/ddf/d54/db0/d102/d20/f147 x:0 0 0 2024-05-06T12:14:46.222 INFO:tasks.workunit.client.0.smithi092.stdout:9/909: dwrite d16/d2c/d2f/f31 [0,4194304] 0 2024-05-06T12:14:46.226 INFO:tasks.workunit.client.1.smithi190.stdout:9/493: rename db/dd/c1d to db/d23/d7e/cb1 0 2024-05-06T12:14:46.226 INFO:tasks.workunit.client.0.smithi092.stdout:6/997: rename d4/d24/d66/d105 to d4/d24/d38/d5d/d6d/dd8/d10f/d139 0 2024-05-06T12:14:46.228 INFO:tasks.workunit.client.0.smithi092.stdout:5/962: rmdir d4/d5/d10/d31 39 2024-05-06T12:14:46.228 INFO:tasks.workunit.client.1.smithi190.stdout:9/494: creat db/dd/d16/fb2 x:0 0 0 2024-05-06T12:14:46.229 INFO:tasks.workunit.client.0.smithi092.stdout:9/910: read d16/d2c/d2f/d3d/d33/f71 [1363654,67247] 0 2024-05-06T12:14:46.230 INFO:tasks.workunit.client.1.smithi190.stdout:3/390: fsync d5/d7/d15/d1f/f88 0 2024-05-06T12:14:46.231 INFO:tasks.workunit.client.1.smithi190.stdout:9/495: dread db/d23/d29/fa5 [0,4194304] 0 2024-05-06T12:14:46.234 INFO:tasks.workunit.client.0.smithi092.stdout:6/998: dwrite d4/f19 [4194304,4194304] 0 2024-05-06T12:14:46.236 INFO:tasks.workunit.client.0.smithi092.stdout:9/911: symlink d16/d2c/d2f/l127 0 2024-05-06T12:14:46.237 INFO:tasks.workunit.client.0.smithi092.stdout:5/963: fdatasync d4/d5/d10/d31/ddf/d54/db0/d102/d78/f124 0 2024-05-06T12:14:46.239 INFO:tasks.workunit.client.0.smithi092.stdout:6/999: rmdir d4/d24/d38/d5d/daf 39 2024-05-06T12:14:46.240 INFO:tasks.workunit.client.0.smithi092.stdout:9/912: stat d16/d1e/d7b/d8d/laa 0 2024-05-06T12:14:46.241 INFO:tasks.workunit.client.0.smithi092.stdout:9/913: rmdir d16 39 2024-05-06T12:14:46.245 INFO:tasks.workunit.client.0.smithi092.stdout:5/964: getdents d4 0 2024-05-06T12:14:46.245 INFO:tasks.workunit.client.0.smithi092.stdout:9/914: mknod d16/d2c/d2f/d3d/d33/dde/c128 0 2024-05-06T12:14:46.247 INFO:tasks.workunit.client.0.smithi092.stdout:5/965: symlink d4/d5/d30/dfd/d108/l148 0 2024-05-06T12:14:46.250 INFO:tasks.workunit.client.0.smithi092.stdout:5/966: mkdir d4/d5/d10/d31/ddf/da3/dc2/d149 0 2024-05-06T12:14:46.251 INFO:tasks.workunit.client.1.smithi190.stdout:0/469: dread d9/dc/d48/d50/f85 [0,4194304] 0 2024-05-06T12:14:46.251 INFO:tasks.workunit.client.1.smithi190.stdout:0/470: chown d9/dc/d28/d4c/f53 4749154 1 2024-05-06T12:14:46.252 INFO:tasks.workunit.client.1.smithi190.stdout:0/471: read - d9/dc/f6c zero size 2024-05-06T12:14:46.253 INFO:tasks.workunit.client.0.smithi092.stdout:9/915: dwrite d16/d2c/d2f/d3d/d33/d37/f10d [0,4194304] 0 2024-05-06T12:14:46.255 INFO:tasks.workunit.client.0.smithi092.stdout:5/967: rmdir d4/d5/d10/d31/ddf/d54/db0/d102/d78/d11c 39 2024-05-06T12:14:46.258 INFO:tasks.workunit.client.0.smithi092.stdout:5/968: rename d4/d5/d30/dfd/d108/d12b to d4/d5/d30/dfd/d108/d12b/d14a 22 2024-05-06T12:14:46.260 INFO:tasks.workunit.client.0.smithi092.stdout:9/916: dwrite d16/d1e/f70 [0,4194304] 0 2024-05-06T12:14:46.265 INFO:tasks.workunit.client.0.smithi092.stdout:9/917: read d16/d1e/d7b/d8d/da4/fb7 [2767654,75923] 0 2024-05-06T12:14:46.268 INFO:tasks.workunit.client.0.smithi092.stdout:5/969: dwrite d4/d59/f83 [0,4194304] 0 2024-05-06T12:14:46.269 INFO:tasks.workunit.client.0.smithi092.stdout:9/918: write d16/d1c/d58/d92/dab/fd7 [433031,107147] 0 2024-05-06T12:14:46.271 INFO:tasks.workunit.client.0.smithi092.stdout:9/919: dread - d16/d1e/f6a zero size 2024-05-06T12:14:46.275 INFO:tasks.workunit.client.0.smithi092.stdout:5/970: dwrite d4/d5/d10/d31/ddf/d6c/fe4 [0,4194304] 0 2024-05-06T12:14:46.296 INFO:tasks.workunit.client.0.smithi092.stdout:5/971: dwrite d4/d5/d10/d31/ddf/d6c/f51 [0,4194304] 0 2024-05-06T12:14:46.298 INFO:tasks.workunit.client.0.smithi092.stdout:5/972: write d4/d34/fd0 [2390477,95107] 0 2024-05-06T12:14:46.315 INFO:tasks.workunit.client.1.smithi190.stdout:3/391: creat d5/d7/d15/d1f/d33/d5c/f89 x:0 0 0 2024-05-06T12:14:46.408 INFO:tasks.workunit.client.0.smithi092.stdout:5/973: symlink d4/d5/d10/d31/d53/dcb/l14b 0 2024-05-06T12:14:46.408 INFO:tasks.workunit.client.0.smithi092.stdout:5/974: stat d4/d34/d28 0 2024-05-06T12:14:46.408 INFO:tasks.workunit.client.0.smithi092.stdout:5/975: fdatasync d4/d5/d10/d31/ddf/da3/fa9 0 2024-05-06T12:14:46.408 INFO:tasks.workunit.client.0.smithi092.stdout:5/976: truncate d4/d5/d10/d31/ddf/d54/db0/f13b 192405 0 2024-05-06T12:14:46.408 INFO:tasks.workunit.client.0.smithi092.stdout:5/977: creat d4/d34/d28/d4e/d67/f14c x:0 0 0 2024-05-06T12:14:46.409 INFO:tasks.workunit.client.0.smithi092.stdout:5/978: link d4/d5/d30/dfd/d108/f116 d4/d5/d10/d31/ddf/da3/dc2/f14d 0 2024-05-06T12:14:46.409 INFO:tasks.workunit.client.0.smithi092.stdout:5/979: mknod d4/d5/d30/d10a/c14e 0 2024-05-06T12:14:46.409 INFO:tasks.workunit.client.0.smithi092.stdout:5/980: mknod d4/d5/d10/d31/d53/ddb/d104/c14f 0 2024-05-06T12:14:46.409 INFO:tasks.workunit.client.0.smithi092.stdout:5/981: creat d4/d5/d30/dfd/d131/f150 x:0 0 0 2024-05-06T12:14:46.409 INFO:tasks.workunit.client.0.smithi092.stdout:5/982: chown d4/d5/d10/d31/ddf/d54/db0/d102/d20 179880 1 2024-05-06T12:14:46.409 INFO:tasks.workunit.client.0.smithi092.stdout:5/983: write d4/d59/fd5 [1128439,75931] 0 2024-05-06T12:14:46.409 INFO:tasks.workunit.client.0.smithi092.stdout:5/984: read d4/d5/d10/d31/ddf/d54/db0/d102/d20/f100 [1999607,121597] 0 2024-05-06T12:14:46.409 INFO:tasks.workunit.client.0.smithi092.stdout:5/985: write d4/d34/d28/d4e/d4b/f103 [563752,109957] 0 2024-05-06T12:14:46.409 INFO:tasks.workunit.client.0.smithi092.stdout:5/986: mkdir d4/d5/d10/d31/ddf/d54/d151 0 2024-05-06T12:14:46.409 INFO:tasks.workunit.client.0.smithi092.stdout:5/987: stat d4/d5/d30/d10a/f111 0 2024-05-06T12:14:46.409 INFO:tasks.workunit.client.0.smithi092.stdout:5/988: dread d4/d5/d10/d31/d53/f9e [0,4194304] 0 2024-05-06T12:14:46.409 INFO:tasks.workunit.client.0.smithi092.stdout:5/989: dwrite d4/f4c [0,4194304] 0 2024-05-06T12:14:46.416 INFO:tasks.workunit.client.1.smithi190.stdout:3/392: fdatasync d5/d7/f3a 0 2024-05-06T12:14:46.416 INFO:tasks.workunit.client.1.smithi190.stdout:0/472: mknod d9/d51/ca5 0 2024-05-06T12:14:46.416 INFO:tasks.workunit.client.1.smithi190.stdout:9/496: mknod db/cb3 0 2024-05-06T12:14:46.416 INFO:tasks.workunit.client.1.smithi190.stdout:3/393: rmdir d5/d7/d7f 39 2024-05-06T12:14:46.416 INFO:tasks.workunit.client.1.smithi190.stdout:3/394: write d5/d7/d15/d1f/fe [3100011,48471] 0 2024-05-06T12:14:46.416 INFO:tasks.workunit.client.1.smithi190.stdout:0/473: symlink d9/dc/d48/la6 0 2024-05-06T12:14:46.416 INFO:tasks.workunit.client.1.smithi190.stdout:9/497: write db/d23/d25/d47/d4b/f9e [3353488,5036] 0 2024-05-06T12:14:46.416 INFO:tasks.workunit.client.1.smithi190.stdout:3/395: creat d5/d14/f8a x:0 0 0 2024-05-06T12:14:46.416 INFO:tasks.workunit.client.1.smithi190.stdout:9/498: rmdir db/dd/d16/d58/d87/d22/d49 39 2024-05-06T12:14:46.416 INFO:tasks.workunit.client.1.smithi190.stdout:9/499: mknod db/d23/d25/d47/cb4 0 2024-05-06T12:14:46.417 INFO:tasks.workunit.client.1.smithi190.stdout:3/396: dwrite d5/d7/d15/d1f/d33/d40/f7d [0,4194304] 0 2024-05-06T12:14:46.417 INFO:tasks.workunit.client.1.smithi190.stdout:9/500: mkdir db/dd/d16/d58/d87/d22/d49/d88/db5 0 2024-05-06T12:14:46.417 INFO:tasks.workunit.client.1.smithi190.stdout:3/397: write d5/d7/f23 [576614,84780] 0 2024-05-06T12:14:46.417 INFO:tasks.workunit.client.1.smithi190.stdout:3/398: dread - d5/d7/d15/d1f/d33/f66 zero size 2024-05-06T12:14:46.417 INFO:tasks.workunit.client.1.smithi190.stdout:0/474: write d9/f52 [5214076,52127] 0 2024-05-06T12:14:46.417 INFO:tasks.workunit.client.1.smithi190.stdout:9/501: dread db/dd/d16/d58/d87/f70 [0,4194304] 0 2024-05-06T12:14:46.417 INFO:tasks.workunit.client.1.smithi190.stdout:3/399: stat d5/d7/d15/d1f/d33/d40/f45 0 2024-05-06T12:14:46.417 INFO:tasks.workunit.client.1.smithi190.stdout:9/502: read - db/dd/d16/d58/d87/d22/d49/d88/faf zero size 2024-05-06T12:14:46.417 INFO:tasks.workunit.client.1.smithi190.stdout:0/475: mkdir d9/dc/d28/d4c/d80/da7 0 2024-05-06T12:14:46.417 INFO:tasks.workunit.client.1.smithi190.stdout:0/476: symlink d9/dc/d28/d77/la8 0 2024-05-06T12:14:46.417 INFO:tasks.workunit.client.1.smithi190.stdout:1/301: sync 2024-05-06T12:14:46.417 INFO:tasks.workunit.client.1.smithi190.stdout:3/400: rmdir d5/d7/d15/d1f/d33/d64 0 2024-05-06T12:14:46.418 INFO:tasks.workunit.client.1.smithi190.stdout:1/302: chown d5/d23/d24/d3a/f3b 960102550 1 2024-05-06T12:14:46.418 INFO:tasks.workunit.client.1.smithi190.stdout:9/503: symlink db/dd/d16/d58/d87/d99/lb6 0 2024-05-06T12:14:46.418 INFO:tasks.workunit.client.1.smithi190.stdout:0/477: link d9/dc/d48/c9f d9/ca9 0 2024-05-06T12:14:46.418 INFO:tasks.workunit.client.1.smithi190.stdout:0/478: dread - d9/df/d21/d72/f98 zero size 2024-05-06T12:14:46.418 INFO:tasks.workunit.client.1.smithi190.stdout:9/504: getdents db/d23/d25 0 2024-05-06T12:14:46.418 INFO:tasks.workunit.client.1.smithi190.stdout:9/505: write db/dd/d16/d58/d87/d22/d49/d8c/fa0 [901099,62133] 0 2024-05-06T12:14:46.418 INFO:tasks.workunit.client.1.smithi190.stdout:0/479: getdents d9/dc/d48/d50/d87 0 2024-05-06T12:14:46.418 INFO:tasks.workunit.client.1.smithi190.stdout:0/480: stat d9/d51/ca5 0 2024-05-06T12:14:46.418 INFO:tasks.workunit.client.1.smithi190.stdout:1/303: dread d5/d23/d32/f4a [0,4194304] 0 2024-05-06T12:14:46.418 INFO:tasks.workunit.client.1.smithi190.stdout:0/481: dread d9/df/d21/f49 [0,4194304] 0 2024-05-06T12:14:46.418 INFO:tasks.workunit.client.1.smithi190.stdout:3/401: dread d5/d7/d15/d1f/fe [0,4194304] 0 2024-05-06T12:14:46.418 INFO:tasks.workunit.client.1.smithi190.stdout:1/304: truncate d5/d23/d24/d3a/f3b 343053 0 2024-05-06T12:14:46.418 INFO:tasks.workunit.client.1.smithi190.stdout:0/482: link d9/df/l6b d9/df/d56/laa 0 2024-05-06T12:14:46.419 INFO:tasks.workunit.client.1.smithi190.stdout:1/305: write d5/d23/d32/d47/f4c [1026421,64235] 0 2024-05-06T12:14:46.419 INFO:tasks.workunit.client.1.smithi190.stdout:3/402: rename d5/d14/d11/d21/d44/d80/d82/f75 to d5/d14/d11/d21/d44/d80/d82/f8b 0 2024-05-06T12:14:46.419 INFO:tasks.workunit.client.1.smithi190.stdout:1/306: rename d5/d23 to d5/d4f/d65 0 2024-05-06T12:14:46.419 INFO:tasks.workunit.client.1.smithi190.stdout:3/403: mknod d5/d7/d15/d1f/d33/d40/c8c 0 2024-05-06T12:14:46.419 INFO:tasks.workunit.client.1.smithi190.stdout:0/483: getdents d9/dc 0 2024-05-06T12:14:46.419 INFO:tasks.workunit.client.1.smithi190.stdout:0/484: chown d9/dc/d28/d4c/c6d 155718452 1 2024-05-06T12:14:46.419 INFO:tasks.workunit.client.1.smithi190.stdout:3/404: creat d5/d7/d7f/f8d x:0 0 0 2024-05-06T12:14:46.419 INFO:tasks.workunit.client.1.smithi190.stdout:3/405: mknod d5/d7/d6d/c8e 0 2024-05-06T12:14:46.419 INFO:tasks.workunit.client.1.smithi190.stdout:0/485: write d9/dc/d48/d50/d83/f94 [3426420,77812] 0 2024-05-06T12:14:46.419 INFO:tasks.workunit.client.1.smithi190.stdout:3/406: write d5/d14/d11/d21/d44/d80/d82/d3d/f6c [1885047,103628] 0 2024-05-06T12:14:46.419 INFO:tasks.workunit.client.1.smithi190.stdout:0/486: mkdir d9/df/d56/dab 0 2024-05-06T12:14:46.419 INFO:tasks.workunit.client.1.smithi190.stdout:3/407: write d5/d7/f1e [3480264,114822] 0 2024-05-06T12:14:46.419 INFO:tasks.workunit.client.1.smithi190.stdout:0/487: symlink d9/lac 0 2024-05-06T12:14:46.420 INFO:tasks.workunit.client.1.smithi190.stdout:0/488: write d9/dc/d28/d4c/f53 [797379,15243] 0 2024-05-06T12:14:46.420 INFO:tasks.workunit.client.1.smithi190.stdout:3/408: dwrite d5/d7/f86 [0,4194304] 0 2024-05-06T12:14:46.420 INFO:tasks.workunit.client.1.smithi190.stdout:3/409: creat d5/d14/f8f x:0 0 0 2024-05-06T12:14:46.420 INFO:tasks.workunit.client.1.smithi190.stdout:3/410: write d5/d7/d15/d1f/d33/d5c/f89 [138785,55063] 0 2024-05-06T12:14:46.420 INFO:tasks.workunit.client.1.smithi190.stdout:3/411: unlink d5/d7/d15/d1f/c46 0 2024-05-06T12:14:46.420 INFO:tasks.workunit.client.1.smithi190.stdout:3/412: fdatasync d5/d7/d15/d1f/d33/f76 0 2024-05-06T12:14:46.420 INFO:tasks.workunit.client.1.smithi190.stdout:3/413: fdatasync d5/f59 0 2024-05-06T12:14:46.420 INFO:tasks.workunit.client.1.smithi190.stdout:0/489: dread d9/df/d21/f4f [0,4194304] 0 2024-05-06T12:14:46.420 INFO:tasks.workunit.client.1.smithi190.stdout:3/414: read d5/d14/d11/d21/d44/d80/d82/d3d/f54 [3685420,96750] 0 2024-05-06T12:14:46.420 INFO:tasks.workunit.client.1.smithi190.stdout:0/490: fsync d9/dc/d28/d4c/f4b 0 2024-05-06T12:14:46.421 INFO:tasks.workunit.client.1.smithi190.stdout:3/415: rename d5/d14/f47 to d5/d7/f90 0 2024-05-06T12:14:46.421 INFO:tasks.workunit.client.0.smithi092.stdout:2/751: sync 2024-05-06T12:14:46.421 INFO:tasks.workunit.client.0.smithi092.stdout:8/876: sync 2024-05-06T12:14:46.427 INFO:tasks.workunit.client.1.smithi190.stdout:3/416: mknod d5/d7/d15/d39/c91 0 2024-05-06T12:14:46.428 INFO:tasks.workunit.client.1.smithi190.stdout:3/417: chown d5/d7/d15/d1f/d33/d40/c4e 5675363 1 2024-05-06T12:14:46.429 INFO:tasks.workunit.client.1.smithi190.stdout:3/418: mkdir d5/d14/d11/d21/d44/d92 0 2024-05-06T12:14:46.435 INFO:tasks.workunit.client.1.smithi190.stdout:3/419: dwrite d5/d14/d11/d21/d44/d80/d82/d3d/f6c [0,4194304] 0 2024-05-06T12:14:46.437 INFO:tasks.workunit.client.1.smithi190.stdout:5/354: sync 2024-05-06T12:14:46.437 INFO:tasks.workunit.client.1.smithi190.stdout:6/305: sync 2024-05-06T12:14:46.438 INFO:tasks.workunit.client.1.smithi190.stdout:3/420: truncate d5/d7/d15/d1f/f3b 2869630 0 2024-05-06T12:14:46.440 INFO:tasks.workunit.client.1.smithi190.stdout:6/306: fsync fd 0 2024-05-06T12:14:46.440 INFO:tasks.workunit.client.1.smithi190.stdout:0/491: dread d9/df/d21/f2a [0,4194304] 0 2024-05-06T12:14:46.441 INFO:tasks.workunit.client.1.smithi190.stdout:3/421: fdatasync d5/d14/d11/f13 0 2024-05-06T12:14:46.444 INFO:tasks.workunit.client.1.smithi190.stdout:5/355: rename d2/c12 to d2/d65/c6b 0 2024-05-06T12:14:46.444 INFO:tasks.workunit.client.1.smithi190.stdout:5/356: write d2/f30 [787324,124427] 0 2024-05-06T12:14:46.447 INFO:tasks.workunit.client.1.smithi190.stdout:6/307: creat df/d2c/d1e/d57/f69 x:0 0 0 2024-05-06T12:14:46.449 INFO:tasks.workunit.client.1.smithi190.stdout:0/492: symlink d9/dc/d28/d4c/d80/lad 0 2024-05-06T12:14:46.455 INFO:tasks.workunit.client.1.smithi190.stdout:5/357: mkdir d2/d6c 0 2024-05-06T12:14:46.457 INFO:tasks.workunit.client.0.smithi092.stdout:7/976: sync 2024-05-06T12:14:46.457 INFO:tasks.workunit.client.0.smithi092.stdout:1/893: sync 2024-05-06T12:14:46.457 INFO:tasks.workunit.client.0.smithi092.stdout:4/972: sync 2024-05-06T12:14:46.457 INFO:tasks.workunit.client.0.smithi092.stdout:3/724: sync 2024-05-06T12:14:46.459 INFO:tasks.workunit.client.0.smithi092.stdout:4/973: chown d9/dd/da6/dca/df2/d10d 7722 1 2024-05-06T12:14:46.459 INFO:tasks.workunit.client.1.smithi190.stdout:6/308: link df/f2e df/d2c/d1e/d57/f6a 0 2024-05-06T12:14:46.460 INFO:tasks.workunit.client.1.smithi190.stdout:5/358: write d2/da/d59/d5f/f36 [222028,33808] 0 2024-05-06T12:14:46.460 INFO:tasks.workunit.client.1.smithi190.stdout:6/309: write df/d2c/f36 [1037689,31926] 0 2024-05-06T12:14:46.465 INFO:tasks.workunit.client.1.smithi190.stdout:6/310: dread df/d17/f26 [0,4194304] 0 2024-05-06T12:14:46.480 INFO:tasks.workunit.client.0.smithi092.stdout:8/877: symlink d9/d21/d54/l130 0 2024-05-06T12:14:46.480 INFO:tasks.workunit.client.0.smithi092.stdout:8/878: readlink d9/d21/d72/lbf 0 2024-05-06T12:14:46.483 INFO:tasks.workunit.client.0.smithi092.stdout:2/752: creat df/d17/d7e/fdf x:0 0 0 2024-05-06T12:14:46.484 INFO:tasks.workunit.client.0.smithi092.stdout:2/753: chown df/d17/d53/d85/la8 0 1 2024-05-06T12:14:46.487 INFO:tasks.workunit.client.0.smithi092.stdout:3/725: unlink d8/db/d4c/d40/cab 0 2024-05-06T12:14:46.489 INFO:tasks.workunit.client.0.smithi092.stdout:7/977: symlink d9/de/d10/d17/d99/dc3/dd2/l14c 0 2024-05-06T12:14:46.492 INFO:tasks.workunit.client.0.smithi092.stdout:5/990: mknod d4/d5/d10/da1/dfa/c152 0 2024-05-06T12:14:46.495 INFO:tasks.workunit.client.0.smithi092.stdout:5/991: read d4/d5/d10/d31/ddf/d54/db0/f13b [185321,64891] 0 2024-05-06T12:14:46.497 INFO:tasks.workunit.client.0.smithi092.stdout:7/978: dwrite d9/dd/f10a [0,4194304] 0 2024-05-06T12:14:46.498 INFO:tasks.workunit.client.1.smithi190.stdout:5/359: fsync d2/da/d59/d5f/f3c 0 2024-05-06T12:14:46.500 INFO:tasks.workunit.client.1.smithi190.stdout:3/422: dread d5/d7/d15/d1f/f42 [0,4194304] 0 2024-05-06T12:14:46.500 INFO:tasks.workunit.client.1.smithi190.stdout:5/360: write d2/f60 [7661,96322] 0 2024-05-06T12:14:46.501 INFO:tasks.workunit.client.1.smithi190.stdout:6/311: getdents df/d2c/d61 0 2024-05-06T12:14:46.502 INFO:tasks.workunit.client.1.smithi190.stdout:6/312: fdatasync df/d17/d37/f58 0 2024-05-06T12:14:46.505 INFO:tasks.workunit.client.1.smithi190.stdout:3/423: symlink d5/l93 0 2024-05-06T12:14:46.507 INFO:tasks.workunit.client.1.smithi190.stdout:6/313: rename fe to df/d2c/d1e/d57/f6b 0 2024-05-06T12:14:46.509 INFO:tasks.workunit.client.0.smithi092.stdout:8/879: dwrite d9/d21/d54/f7b [0,4194304] 0 2024-05-06T12:14:46.511 INFO:tasks.workunit.client.0.smithi092.stdout:5/992: dwrite d4/ffc [0,4194304] 0 2024-05-06T12:14:46.527 INFO:tasks.workunit.client.1.smithi190.stdout:3/424: write d5/d14/d11/d21/d44/d80/d82/d3d/f49 [4148874,47422] 0 2024-05-06T12:14:46.530 INFO:tasks.workunit.client.0.smithi092.stdout:2/754: mkdir df/d17/d53/d85/da0/de0 0 2024-05-06T12:14:46.531 INFO:tasks.workunit.client.0.smithi092.stdout:7/979: link d9/de/d10/f2b d9/de/da4/df8/f14d 0 2024-05-06T12:14:46.532 INFO:tasks.workunit.client.1.smithi190.stdout:3/425: write d5/d14/d11/d21/d44/d80/d82/f8b [924587,80375] 0 2024-05-06T12:14:46.533 INFO:tasks.workunit.client.0.smithi092.stdout:2/755: readlink df/d5c/l6e 0 2024-05-06T12:14:46.535 INFO:tasks.workunit.client.0.smithi092.stdout:8/880: read d9/d21/d37/d46/d5c/d71/f80 [345160,106130] 0 2024-05-06T12:14:46.535 INFO:tasks.workunit.client.0.smithi092.stdout:7/980: creat d9/de/d2d/f14e x:0 0 0 2024-05-06T12:14:46.535 INFO:tasks.workunit.client.1.smithi190.stdout:6/314: dwrite df/d2c/d1e/d57/f6a [0,4194304] 0 2024-05-06T12:14:46.537 INFO:tasks.workunit.client.0.smithi092.stdout:7/981: chown d9/de/d10/d17/d6a/f78 7337 1 2024-05-06T12:14:46.537 INFO:tasks.workunit.client.1.smithi190.stdout:3/426: mkdir d5/d94 0 2024-05-06T12:14:46.538 INFO:tasks.workunit.client.1.smithi190.stdout:5/361: dread d2/d27/f54 [0,4194304] 0 2024-05-06T12:14:46.538 INFO:tasks.workunit.client.0.smithi092.stdout:5/993: dread d4/d34/d28/d4e/d4b/f8c [0,4194304] 0 2024-05-06T12:14:46.538 INFO:tasks.workunit.client.0.smithi092.stdout:7/982: chown d9/de/d10/d17/d21/d2e/d87/d11f 0 1 2024-05-06T12:14:46.539 INFO:tasks.workunit.client.1.smithi190.stdout:6/315: symlink df/d2c/d28/l6c 0 2024-05-06T12:14:46.540 INFO:tasks.workunit.client.1.smithi190.stdout:3/427: creat d5/d79/f95 x:0 0 0 2024-05-06T12:14:46.541 INFO:tasks.workunit.client.1.smithi190.stdout:2/334: sync 2024-05-06T12:14:46.541 INFO:tasks.workunit.client.1.smithi190.stdout:4/258: sync 2024-05-06T12:14:46.541 INFO:tasks.workunit.client.1.smithi190.stdout:7/297: sync 2024-05-06T12:14:46.541 INFO:tasks.workunit.client.1.smithi190.stdout:8/444: sync 2024-05-06T12:14:46.541 INFO:tasks.workunit.client.1.smithi190.stdout:5/362: mkdir d2/da/d59/d5f/d50/d6d 0 2024-05-06T12:14:46.542 INFO:tasks.workunit.client.0.smithi092.stdout:2/756: creat df/d17/d53/da9/dd9/dd5/fe1 x:0 0 0 2024-05-06T12:14:46.543 INFO:tasks.workunit.client.1.smithi190.stdout:6/316: chown df/c5e 7316581 1 2024-05-06T12:14:46.547 INFO:tasks.workunit.client.1.smithi190.stdout:2/335: dread d2/d21/f3d [0,4194304] 0 2024-05-06T12:14:46.548 INFO:tasks.workunit.client.1.smithi190.stdout:8/445: read d3/f52 [58133,35398] 0 2024-05-06T12:14:46.550 INFO:tasks.workunit.client.0.smithi092.stdout:7/983: link d9/de/d10/d17/d99/db6/ld0 d9/d62/d9a/l14f 0 2024-05-06T12:14:46.551 INFO:tasks.workunit.client.0.smithi092.stdout:7/984: write d9/de/d10/d17/d21/d38/d77/f141 [254230,17526] 0 2024-05-06T12:14:46.552 INFO:tasks.workunit.client.1.smithi190.stdout:8/446: read d3/f52 [13043,76164] 0 2024-05-06T12:14:46.554 INFO:tasks.workunit.client.1.smithi190.stdout:6/317: rename fb to df/d12/f6d 0 2024-05-06T12:14:46.555 INFO:tasks.workunit.client.0.smithi092.stdout:7/985: mknod d9/de/d10/d17/d99/da6/c150 0 2024-05-06T12:14:46.556 INFO:tasks.workunit.client.1.smithi190.stdout:3/428: dread d5/d7/d15/d1f/d33/d40/f7d [0,4194304] 0 2024-05-06T12:14:46.557 INFO:tasks.workunit.client.0.smithi092.stdout:7/986: write d9/de/d10/d17/d21/fd4 [1125177,89318] 0 2024-05-06T12:14:46.559 INFO:tasks.workunit.client.1.smithi190.stdout:5/363: dwrite d2/d27/f48 [0,4194304] 0 2024-05-06T12:14:46.560 INFO:tasks.workunit.client.0.smithi092.stdout:7/987: truncate d9/de/d10/d17/d21/d2e/d9c/f13d 580928 0 2024-05-06T12:14:46.563 INFO:tasks.workunit.client.1.smithi190.stdout:8/447: creat d3/d8/d25/d3b/d6b/d7a/f9c x:0 0 0 2024-05-06T12:14:46.563 INFO:tasks.workunit.client.1.smithi190.stdout:5/364: read d2/da/d59/d5f/f36 [165638,113631] 0 2024-05-06T12:14:46.564 INFO:tasks.workunit.client.1.smithi190.stdout:5/365: dread - d2/da/f64 zero size 2024-05-06T12:14:46.564 INFO:tasks.workunit.client.0.smithi092.stdout:3/726: creat d8/d24/d33/d4b/fe5 x:0 0 0 2024-05-06T12:14:46.566 INFO:tasks.workunit.client.0.smithi092.stdout:3/727: dread - d8/db/d4c/d40/d7a/d90/fc2 zero size 2024-05-06T12:14:46.568 INFO:tasks.workunit.client.0.smithi092.stdout:3/728: mknod d8/db/d4c/d3d/d49/ce6 0 2024-05-06T12:14:46.569 INFO:tasks.workunit.client.1.smithi190.stdout:7/298: write d0/d1/fc [1034493,127487] 0 2024-05-06T12:14:46.569 INFO:tasks.workunit.client.1.smithi190.stdout:2/336: getdents d2/d8/d33/d46/d55 0 2024-05-06T12:14:46.570 INFO:tasks.workunit.client.0.smithi092.stdout:3/729: dread - d8/db/d4c/d3d/d49/fe2 zero size 2024-05-06T12:14:46.571 INFO:tasks.workunit.client.1.smithi190.stdout:2/337: chown d2/f35 76961 1 2024-05-06T12:14:46.571 INFO:tasks.workunit.client.1.smithi190.stdout:8/448: chown d3/d8/c4c 2 1 2024-05-06T12:14:46.574 INFO:tasks.workunit.client.0.smithi092.stdout:3/730: symlink d8/d24/d33/d4b/ddf/le7 0 2024-05-06T12:14:46.574 INFO:tasks.workunit.client.1.smithi190.stdout:4/259: dread d2/d10/f35 [0,4194304] 0 2024-05-06T12:14:46.574 INFO:tasks.workunit.client.1.smithi190.stdout:5/366: unlink d2/d27/c35 0 2024-05-06T12:14:46.575 INFO:tasks.workunit.client.1.smithi190.stdout:8/449: truncate d3/d35/d55/d61/f63 724870 0 2024-05-06T12:14:46.575 INFO:tasks.workunit.client.1.smithi190.stdout:4/260: stat d2/d10 0 2024-05-06T12:14:46.576 INFO:tasks.workunit.client.1.smithi190.stdout:3/429: creat d5/d7/d15/d1f/f96 x:0 0 0 2024-05-06T12:14:46.576 INFO:tasks.workunit.client.1.smithi190.stdout:8/450: chown d3/f2e 5867958 1 2024-05-06T12:14:46.579 INFO:tasks.workunit.client.1.smithi190.stdout:8/451: chown d3/d8/d25/d3b/d49/l6f 790 1 2024-05-06T12:14:46.580 INFO:tasks.workunit.client.1.smithi190.stdout:4/261: symlink d2/d10/l5f 0 2024-05-06T12:14:46.581 INFO:tasks.workunit.client.0.smithi092.stdout:3/731: dwrite d8/d1b/d73/db7/fdc [0,4194304] 0 2024-05-06T12:14:46.581 INFO:tasks.workunit.client.1.smithi190.stdout:4/262: fsync d2/d10/f58 0 2024-05-06T12:14:46.583 INFO:tasks.workunit.client.0.smithi092.stdout:8/881: creat d9/d21/d37/dbd/f131 x:0 0 0 2024-05-06T12:14:46.584 INFO:tasks.workunit.client.0.smithi092.stdout:3/732: unlink d8/db/d4c/d40/d67/la6 0 2024-05-06T12:14:46.585 INFO:tasks.workunit.client.0.smithi092.stdout:3/733: dread - d8/d1b/d73/fca zero size 2024-05-06T12:14:46.585 INFO:tasks.workunit.client.1.smithi190.stdout:3/430: fdatasync d5/d7/d15/d1f/fe 0 2024-05-06T12:14:46.586 INFO:tasks.workunit.client.0.smithi092.stdout:3/734: truncate d8/d24/d27/fb6 408692 0 2024-05-06T12:14:46.587 INFO:tasks.workunit.client.1.smithi190.stdout:8/452: unlink d3/d8/d2b/c3f 0 2024-05-06T12:14:46.588 INFO:tasks.workunit.client.1.smithi190.stdout:3/431: write d5/d14/f8a [412769,92414] 0 2024-05-06T12:14:46.594 INFO:tasks.workunit.client.0.smithi092.stdout:3/735: link d8/db/d14/c30 d8/db/d4c/d3d/d49/dd2/ce8 0 2024-05-06T12:14:46.596 INFO:tasks.workunit.client.0.smithi092.stdout:3/736: write d8/db/d4c/d40/d7a/d90/fc2 [594547,33888] 0 2024-05-06T12:14:46.597 INFO:tasks.workunit.client.1.smithi190.stdout:2/338: dread d2/d8/dc/d20/f52 [0,4194304] 0 2024-05-06T12:14:46.598 INFO:tasks.workunit.client.1.smithi190.stdout:8/453: creat d3/d8/d25/d3b/d6b/d7a/f9d x:0 0 0 2024-05-06T12:14:46.598 INFO:tasks.workunit.client.1.smithi190.stdout:5/367: truncate d2/f44 380658 0 2024-05-06T12:14:46.598 INFO:tasks.workunit.client.1.smithi190.stdout:7/299: dwrite d0/da/d36/f46 [0,4194304] 0 2024-05-06T12:14:46.603 INFO:tasks.workunit.client.1.smithi190.stdout:2/339: truncate d2/fa 1324257 0 2024-05-06T12:14:46.604 INFO:tasks.workunit.client.1.smithi190.stdout:5/368: chown d2/d27/f48 51 1 2024-05-06T12:14:46.605 INFO:tasks.workunit.client.1.smithi190.stdout:2/340: chown d2/f38 479811 1 2024-05-06T12:14:46.605 INFO:tasks.workunit.client.1.smithi190.stdout:7/300: readlink d0/d1/d18/d2b/d5a/l64 0 2024-05-06T12:14:46.607 INFO:tasks.workunit.client.1.smithi190.stdout:8/454: dread d3/d8/d2b/d3e/f54 [0,4194304] 0 2024-05-06T12:14:46.607 INFO:tasks.workunit.client.1.smithi190.stdout:7/301: write d0/d4c/f52 [750836,80487] 0 2024-05-06T12:14:46.609 INFO:tasks.workunit.client.1.smithi190.stdout:4/263: dread d2/d11/d16/f38 [0,4194304] 0 2024-05-06T12:14:46.616 INFO:tasks.workunit.client.1.smithi190.stdout:7/302: dwrite d0/da/f5b [0,4194304] 0 2024-05-06T12:14:46.617 INFO:tasks.workunit.client.0.smithi092.stdout:9/920: sync 2024-05-06T12:14:46.630 INFO:tasks.workunit.client.0.smithi092.stdout:5/994: rename d4/d34/d28/d4e/d4b/f8c to d4/d34/d28/f153 0 2024-05-06T12:14:46.638 INFO:tasks.workunit.client.0.smithi092.stdout:5/995: dwrite d4/d5/fc4 [0,4194304] 0 2024-05-06T12:14:46.639 INFO:tasks.workunit.client.1.smithi190.stdout:3/432: truncate d5/d7/d15/d1f/f42 3388162 0 2024-05-06T12:14:46.642 INFO:tasks.workunit.client.1.smithi190.stdout:7/303: symlink d0/d1/d32/d45/d70/d33/l72 0 2024-05-06T12:14:46.646 INFO:tasks.workunit.client.1.smithi190.stdout:2/341: fsync d2/f12 0 2024-05-06T12:14:46.649 INFO:tasks.workunit.client.1.smithi190.stdout:5/369: mkdir d2/da/d59/d5f/d6e 0 2024-05-06T12:14:46.651 INFO:tasks.workunit.client.1.smithi190.stdout:7/304: dwrite d0/d1/d18/f68 [0,4194304] 0 2024-05-06T12:14:46.651 INFO:tasks.workunit.client.1.smithi190.stdout:3/433: fdatasync d5/d14/d11/f13 0 2024-05-06T12:14:46.656 INFO:tasks.workunit.client.1.smithi190.stdout:2/342: mknod d2/d4c/c68 0 2024-05-06T12:14:46.657 INFO:tasks.workunit.client.1.smithi190.stdout:3/434: dread d5/d14/d11/d21/d44/d80/d82/d3d/f6c [0,4194304] 0 2024-05-06T12:14:46.660 INFO:tasks.workunit.client.0.smithi092.stdout:8/882: rename d9/d21/d37/d74/f94 to d9/d21/d37/d46/f132 0 2024-05-06T12:14:46.660 INFO:tasks.workunit.client.0.smithi092.stdout:9/921: write d16/f1b [297921,91366] 0 2024-05-06T12:14:46.661 INFO:tasks.workunit.client.1.smithi190.stdout:7/305: chown d0/d1/d2/f1b 32286 1 2024-05-06T12:14:46.661 INFO:tasks.workunit.client.1.smithi190.stdout:7/306: stat d0/da/d36/d53 0 2024-05-06T12:14:46.662 INFO:tasks.workunit.client.1.smithi190.stdout:7/307: chown d0/d1/d18/d2b/f30 353 1 2024-05-06T12:14:46.663 INFO:tasks.workunit.client.0.smithi092.stdout:8/883: rmdir d9/d21/d37/d46/d5c/d62/d7a 39 2024-05-06T12:14:46.665 INFO:tasks.workunit.client.1.smithi190.stdout:9/506: sync 2024-05-06T12:14:46.665 INFO:tasks.workunit.client.1.smithi190.stdout:1/307: sync 2024-05-06T12:14:46.665 INFO:tasks.workunit.client.0.smithi092.stdout:9/922: truncate d16/d2c/d2f/d3d/d33/dde/ffb 1604745 0 2024-05-06T12:14:46.666 INFO:tasks.workunit.client.1.smithi190.stdout:7/308: write d0/d1/f21 [4634036,88327] 0 2024-05-06T12:14:46.667 INFO:tasks.workunit.client.0.smithi092.stdout:9/923: fsync d16/d1c/d58/d92/dab/fd7 0 2024-05-06T12:14:46.669 INFO:tasks.workunit.client.1.smithi190.stdout:3/435: symlink d5/d14/d11/d21/d44/d80/d82/l97 0 2024-05-06T12:14:46.670 INFO:tasks.workunit.client.0.smithi092.stdout:5/996: link d4/d5/d10/d31/c128 d4/d5/d10/d31/ddf/d54/db0/d102/d13c/c154 0 2024-05-06T12:14:46.672 INFO:tasks.workunit.client.1.smithi190.stdout:1/308: creat d5/d4f/d65/d32/d47/f66 x:0 0 0 2024-05-06T12:14:46.673 INFO:tasks.workunit.client.1.smithi190.stdout:8/455: dread d3/d8/d25/d3b/d90/d6e/f82 [0,4194304] 0 2024-05-06T12:14:46.673 INFO:tasks.workunit.client.0.smithi092.stdout:9/924: rename d16/d1c/d58/d92/dab/deb/f11e to d16/d2c/d4b/d76/f129 0 2024-05-06T12:14:46.674 INFO:tasks.workunit.client.1.smithi190.stdout:8/456: chown d3/d8/d2b/l33 1200639235 1 2024-05-06T12:14:46.674 INFO:tasks.workunit.client.0.smithi092.stdout:5/997: write d4/d34/d28/f3a [4095620,92383] 0 2024-05-06T12:14:46.676 INFO:tasks.workunit.client.0.smithi092.stdout:5/998: dread - d4/d5/d30/dfd/d131/f150 zero size 2024-05-06T12:14:46.676 INFO:tasks.workunit.client.0.smithi092.stdout:9/925: creat d16/d2c/d88/f12a x:0 0 0 2024-05-06T12:14:46.677 INFO:tasks.workunit.client.1.smithi190.stdout:3/436: truncate d5/f1d 131406 0 2024-05-06T12:14:46.678 INFO:tasks.workunit.client.0.smithi092.stdout:9/926: write d16/d1e/d7b/dce/f89 [5739381,111744] 0 2024-05-06T12:14:46.679 INFO:tasks.workunit.client.0.smithi092.stdout:5/999: readlink d4/d5/d10/d31/ddf/d54/dc5/ld4 0 2024-05-06T12:14:46.680 INFO:tasks.workunit.client.1.smithi190.stdout:4/264: mknod d2/d11/c60 0 2024-05-06T12:14:46.681 INFO:tasks.workunit.client.0.smithi092.stdout:8/884: write d9/d21/f3f [745282,100021] 0 2024-05-06T12:14:46.682 INFO:tasks.workunit.client.1.smithi190.stdout:5/370: link d2/c4d d2/da/c6f 0 2024-05-06T12:14:46.683 INFO:tasks.workunit.client.1.smithi190.stdout:9/507: dread db/dd/d16/d4c/d4e/f9d [0,4194304] 0 2024-05-06T12:14:46.683 INFO:tasks.workunit.client.0.smithi092.stdout:9/927: fsync d16/d2c/d2f/d3d/f61 0 2024-05-06T12:14:46.684 INFO:tasks.workunit.client.0.smithi092.stdout:8/885: fdatasync d9/d21/d37/d46/d5c/d71/ff3 0 2024-05-06T12:14:46.685 INFO:tasks.workunit.client.1.smithi190.stdout:9/508: readlink db/d23/d29/l67 0 2024-05-06T12:14:46.685 INFO:tasks.workunit.client.1.smithi190.stdout:5/371: truncate d2/da/f2f 1510082 0 2024-05-06T12:14:46.686 INFO:tasks.workunit.client.1.smithi190.stdout:7/309: link d0/d14/c27 d0/da/d36/c73 0 2024-05-06T12:14:46.687 INFO:tasks.workunit.client.1.smithi190.stdout:0/493: sync 2024-05-06T12:14:46.687 INFO:tasks.workunit.client.1.smithi190.stdout:6/318: sync 2024-05-06T12:14:46.689 INFO:tasks.workunit.client.1.smithi190.stdout:3/437: creat d5/d14/d11/d21/d44/d92/f98 x:0 0 0 2024-05-06T12:14:46.690 INFO:tasks.workunit.client.1.smithi190.stdout:3/438: write d5/d7/f86 [1302071,83316] 0 2024-05-06T12:14:46.691 INFO:tasks.workunit.client.0.smithi092.stdout:9/928: dwrite d16/d2c/d2f/d3d/d33/d6b/f118 [0,4194304] 0 2024-05-06T12:14:46.691 INFO:tasks.workunit.client.1.smithi190.stdout:9/509: write db/daa/fab [193811,125907] 0 2024-05-06T12:14:46.694 INFO:tasks.workunit.client.1.smithi190.stdout:3/439: chown d5/d14/d11/d21/f38 9888334 1 2024-05-06T12:14:46.694 INFO:tasks.workunit.client.1.smithi190.stdout:2/343: dread d2/d8/d33/d46/d55/f59 [0,4194304] 0 2024-05-06T12:14:46.695 INFO:tasks.workunit.client.0.smithi092.stdout:8/886: getdents d9/d21/d54/d79 0 2024-05-06T12:14:46.696 INFO:tasks.workunit.client.1.smithi190.stdout:5/372: truncate d2/f5 2703999 0 2024-05-06T12:14:46.696 INFO:tasks.workunit.client.0.smithi092.stdout:8/887: chown d9/dc/d4a/d5f/da8/ddf 29609960 1 2024-05-06T12:14:46.697 INFO:tasks.workunit.client.1.smithi190.stdout:4/265: dread d2/d11/d16/f3e [0,4194304] 0 2024-05-06T12:14:46.697 INFO:tasks.workunit.client.1.smithi190.stdout:3/440: write d5/d14/f8a [964710,113817] 0 2024-05-06T12:14:46.697 INFO:tasks.workunit.client.1.smithi190.stdout:7/310: mknod d0/d1/d18/d2b/d5a/c74 0 2024-05-06T12:14:46.699 INFO:tasks.workunit.client.0.smithi092.stdout:9/929: creat d16/f12b x:0 0 0 2024-05-06T12:14:46.700 INFO:tasks.workunit.client.1.smithi190.stdout:6/319: chown df/d2c/d1e/l5b 596 1 2024-05-06T12:14:46.700 INFO:tasks.workunit.client.0.smithi092.stdout:8/888: read - d9/d21/d54/ffa zero size 2024-05-06T12:14:46.702 INFO:tasks.workunit.client.0.smithi092.stdout:9/930: read d16/d2c/d2f/d3d/d33/d57/dff/f121 [113397,30138] 0 2024-05-06T12:14:46.702 INFO:tasks.workunit.client.1.smithi190.stdout:1/309: getdents d5/d4f 0 2024-05-06T12:14:46.703 INFO:tasks.workunit.client.0.smithi092.stdout:8/889: symlink d9/dc/d4a/l133 0 2024-05-06T12:14:46.704 INFO:tasks.workunit.client.0.smithi092.stdout:9/931: rmdir d16/d2c/d2f/d3d/d33/d7e 39 2024-05-06T12:14:46.708 INFO:tasks.workunit.client.1.smithi190.stdout:8/457: getdents d3 0 2024-05-06T12:14:46.708 INFO:tasks.workunit.client.0.smithi092.stdout:8/890: rename d9/d21/d72/cab to d9/d21/c134 0 2024-05-06T12:14:46.708 INFO:tasks.workunit.client.1.smithi190.stdout:6/320: dwrite f1 [0,4194304] 0 2024-05-06T12:14:46.711 INFO:tasks.workunit.client.1.smithi190.stdout:2/344: mknod d2/d8/c69 0 2024-05-06T12:14:46.711 INFO:tasks.workunit.client.1.smithi190.stdout:6/321: truncate df/d17/f60 386609 0 2024-05-06T12:14:46.711 INFO:tasks.workunit.client.0.smithi092.stdout:8/891: creat d9/d21/d37/dbd/dc3/f135 x:0 0 0 2024-05-06T12:14:46.711 INFO:tasks.workunit.client.1.smithi190.stdout:5/373: rmdir d2 39 2024-05-06T12:14:46.714 INFO:tasks.workunit.client.1.smithi190.stdout:3/441: creat d5/d14/d11/d21/d44/d80/d82/d3d/f99 x:0 0 0 2024-05-06T12:14:46.715 INFO:tasks.workunit.client.0.smithi092.stdout:8/892: rename d9/f36 to d9/d9d/f136 0 2024-05-06T12:14:46.715 INFO:tasks.workunit.client.1.smithi190.stdout:0/494: getdents d9/d51/d93 0 2024-05-06T12:14:46.720 INFO:tasks.workunit.client.1.smithi190.stdout:7/311: rename d0/d1/d32/d45/d70/d33/l72 to d0/da/d36/l75 0 2024-05-06T12:14:46.720 INFO:tasks.workunit.client.1.smithi190.stdout:8/458: dwrite d3/d8/d25/d3b/d6b/d7a/f9c [0,4194304] 0 2024-05-06T12:14:46.721 INFO:tasks.workunit.client.1.smithi190.stdout:9/510: dread db/f3d [0,4194304] 0 2024-05-06T12:14:46.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:46 smithi092 ceph-mon[26954]: pgmap v25: 65 pgs: 65 active+clean; 2.3 GiB data, 7.9 GiB used, 529 GiB / 536 GiB avail; 64 MiB/s rd, 211 MiB/s wr, 442 op/s 2024-05-06T12:14:46.722 INFO:tasks.workunit.client.1.smithi190.stdout:9/511: chown db/dd/d16/d58/d87/d22/d49/f89 314 1 2024-05-06T12:14:46.725 INFO:tasks.workunit.client.1.smithi190.stdout:5/374: dread - d2/da/d59/d5f/d2e/d3b/f40 zero size 2024-05-06T12:14:46.726 INFO:tasks.workunit.client.1.smithi190.stdout:5/375: fsync d2/da/f4a 0 2024-05-06T12:14:46.726 INFO:tasks.workunit.client.1.smithi190.stdout:4/266: mkdir d2/d11/d16/d61 0 2024-05-06T12:14:46.727 INFO:tasks.workunit.client.1.smithi190.stdout:4/267: readlink d2/d10/l28 0 2024-05-06T12:14:46.728 INFO:tasks.workunit.client.1.smithi190.stdout:4/268: chown d2/f4 101537 1 2024-05-06T12:14:46.729 INFO:tasks.workunit.client.1.smithi190.stdout:3/442: mkdir d5/d7/d6d/d9a 0 2024-05-06T12:14:46.731 INFO:tasks.workunit.client.1.smithi190.stdout:0/495: write d9/dc/d48/d50/f66 [2724458,54494] 0 2024-05-06T12:14:46.732 INFO:tasks.workunit.client.1.smithi190.stdout:1/310: symlink d5/d4f/d65/d62/l67 0 2024-05-06T12:14:46.732 INFO:tasks.workunit.client.0.smithi092.stdout:8/893: dwrite d9/d21/f103 [0,4194304] 0 2024-05-06T12:14:46.733 INFO:tasks.workunit.client.1.smithi190.stdout:1/311: chown d5/d4f/d65/d32/d47/f4c 493187770 1 2024-05-06T12:14:46.736 INFO:tasks.workunit.client.0.smithi092.stdout:8/894: read d9/d21/f3f [119431,30111] 0 2024-05-06T12:14:46.739 INFO:tasks.workunit.client.0.smithi092.stdout:8/895: mknod d9/dc/d4a/c137 0 2024-05-06T12:14:46.740 INFO:tasks.workunit.client.1.smithi190.stdout:5/376: mknod d2/da/d59/d5f/d50/c70 0 2024-05-06T12:14:46.741 INFO:tasks.workunit.client.0.smithi092.stdout:8/896: symlink d9/d21/d37/d46/d5c/dfc/l138 0 2024-05-06T12:14:46.745 INFO:tasks.workunit.client.1.smithi190.stdout:0/496: mkdir d9/df/d56/dae 0 2024-05-06T12:14:46.745 INFO:tasks.workunit.client.0.smithi092.stdout:8/897: truncate d9/d21/f6f 3630279 0 2024-05-06T12:14:46.746 INFO:tasks.workunit.client.1.smithi190.stdout:7/312: creat d0/da/d36/d42/f76 x:0 0 0 2024-05-06T12:14:46.746 INFO:tasks.workunit.client.0.smithi092.stdout:8/898: fsync d9/d21/d37/dbd/dc5/d11c/ff5 0 2024-05-06T12:14:46.747 INFO:tasks.workunit.client.1.smithi190.stdout:1/312: mkdir d5/d4f/d68 0 2024-05-06T12:14:46.749 INFO:tasks.workunit.client.0.smithi092.stdout:8/899: rmdir d9/dc/d4a/d5f/da8/ddf/d115 39 2024-05-06T12:14:46.751 INFO:tasks.workunit.client.0.smithi092.stdout:8/900: mknod d9/d21/d37/dbd/dc5/c139 0 2024-05-06T12:14:46.751 INFO:tasks.workunit.client.1.smithi190.stdout:5/377: symlink d2/da/l71 0 2024-05-06T12:14:46.752 INFO:tasks.workunit.client.0.smithi092.stdout:8/901: chown d9/dc/d4a/d5f/da8/cd0 0 1 2024-05-06T12:14:46.752 INFO:tasks.workunit.client.0.smithi092.stdout:8/902: fsync d9/f12f 0 2024-05-06T12:14:46.753 INFO:tasks.workunit.client.1.smithi190.stdout:2/345: creat d2/f6a x:0 0 0 2024-05-06T12:14:46.755 INFO:tasks.workunit.client.1.smithi190.stdout:0/497: write d9/d51/fa3 [3444065,12340] 0 2024-05-06T12:14:46.758 INFO:tasks.workunit.client.1.smithi190.stdout:0/498: chown d9/df/d56/f57 20329 1 2024-05-06T12:14:46.759 INFO:tasks.workunit.client.0.smithi092.stdout:8/903: getdents d9/d21/d37/d46/d5c/d62/da5/de6 0 2024-05-06T12:14:46.760 INFO:tasks.workunit.client.1.smithi190.stdout:1/313: write d5/d4f/d65/d24/f25 [1749401,41670] 0 2024-05-06T12:14:46.760 INFO:tasks.workunit.client.1.smithi190.stdout:9/512: link db/dd/c2d db/dd/d16/d58/d87/d99/cb7 0 2024-05-06T12:14:46.760 INFO:tasks.workunit.client.0.smithi092.stdout:8/904: mkdir d9/d21/d37/dbd/dc3/d107/d13a 0 2024-05-06T12:14:46.762 INFO:tasks.workunit.client.1.smithi190.stdout:2/346: dwrite d2/d8/dc/d20/f40 [0,4194304] 0 2024-05-06T12:14:46.762 INFO:tasks.workunit.client.1.smithi190.stdout:5/378: mkdir d2/da/d59/d5f/d2e/d3b/d72 0 2024-05-06T12:14:46.763 INFO:tasks.workunit.client.1.smithi190.stdout:8/459: getdents d3/d8/d25/d3b/d49 0 2024-05-06T12:14:46.765 INFO:tasks.workunit.client.0.smithi092.stdout:8/905: dread d9/d21/dc0/fcc [0,4194304] 0 2024-05-06T12:14:46.767 INFO:tasks.workunit.client.1.smithi190.stdout:1/314: rmdir d5/d15 39 2024-05-06T12:14:46.768 INFO:tasks.workunit.client.1.smithi190.stdout:1/315: chown d5/d4f/d65 1040358 1 2024-05-06T12:14:46.768 INFO:tasks.workunit.client.1.smithi190.stdout:2/347: rmdir d2/d21/d50 39 2024-05-06T12:14:46.768 INFO:tasks.workunit.client.1.smithi190.stdout:1/316: readlink d5/l14 0 2024-05-06T12:14:46.769 INFO:tasks.workunit.client.0.smithi092.stdout:8/906: getdents d9/d5a/dd9 0 2024-05-06T12:14:46.769 INFO:tasks.workunit.client.1.smithi190.stdout:5/379: dwrite d2/da/d59/f6a [0,4194304] 0 2024-05-06T12:14:46.770 INFO:tasks.workunit.client.0.smithi092.stdout:8/907: truncate d9/dc/f12d 715835 0 2024-05-06T12:14:46.771 INFO:tasks.workunit.client.1.smithi190.stdout:8/460: creat d3/d8/d2b/f9e x:0 0 0 2024-05-06T12:14:46.772 INFO:tasks.workunit.client.1.smithi190.stdout:8/461: readlink d3/l7 0 2024-05-06T12:14:46.772 INFO:tasks.workunit.client.0.smithi092.stdout:8/908: mkdir d9/d9d/d13b 0 2024-05-06T12:14:46.773 INFO:tasks.workunit.client.0.smithi092.stdout:8/909: chown d9/d21/d37/dbd/dc5 12892727 1 2024-05-06T12:14:46.774 INFO:tasks.workunit.client.1.smithi190.stdout:8/462: dread - d3/d13/f7d zero size 2024-05-06T12:14:46.775 INFO:tasks.workunit.client.0.smithi092.stdout:8/910: fdatasync d9/d21/d37/d74/f83 0 2024-05-06T12:14:46.777 INFO:tasks.workunit.client.0.smithi092.stdout:8/911: write d9/d21/d37/d74/df9/f114 [4242678,27502] 0 2024-05-06T12:14:46.777 INFO:tasks.workunit.client.1.smithi190.stdout:6/322: truncate df/d17/d37/f3c 3921355 0 2024-05-06T12:14:46.777 INFO:tasks.workunit.client.1.smithi190.stdout:8/463: read d3/f12 [96112,88058] 0 2024-05-06T12:14:46.777 INFO:tasks.workunit.client.1.smithi190.stdout:3/443: write d5/d14/d11/d21/f2e [101750,77047] 0 2024-05-06T12:14:46.780 INFO:tasks.workunit.client.1.smithi190.stdout:2/348: creat d2/d8/d33/f6b x:0 0 0 2024-05-06T12:14:46.780 INFO:tasks.workunit.client.0.smithi092.stdout:8/912: fdatasync d9/d21/d37/d46/d5c/d62/da5/fc7 0 2024-05-06T12:14:46.781 INFO:tasks.workunit.client.1.smithi190.stdout:5/380: symlink d2/da/d59/l73 0 2024-05-06T12:14:46.781 INFO:tasks.workunit.client.1.smithi190.stdout:5/381: chown d2/f3 1 1 2024-05-06T12:14:46.781 INFO:tasks.workunit.client.0.smithi092.stdout:8/913: readlink d9/d21/d37/l56 0 2024-05-06T12:14:46.782 INFO:tasks.workunit.client.0.smithi092.stdout:8/914: chown d9/c31 3 1 2024-05-06T12:14:46.782 INFO:tasks.workunit.client.0.smithi092.stdout:8/915: stat d9/dc/f9e 0 2024-05-06T12:14:46.787 INFO:tasks.workunit.client.1.smithi190.stdout:8/464: mkdir d3/d8/d68/d9f 0 2024-05-06T12:14:46.789 INFO:tasks.workunit.client.0.smithi092.stdout:8/916: dwrite d9/dc/ff [0,4194304] 0 2024-05-06T12:14:46.791 INFO:tasks.workunit.client.1.smithi190.stdout:3/444: unlink d5/d7/f43 0 2024-05-06T12:14:46.793 INFO:tasks.workunit.client.1.smithi190.stdout:7/313: write d0/d1/d18/d2b/f30 [105481,96960] 0 2024-05-06T12:14:46.793 INFO:tasks.workunit.client.1.smithi190.stdout:0/499: link d9/f2c d9/dc/d48/d50/d83/faf 0 2024-05-06T12:14:46.794 INFO:tasks.workunit.client.0.smithi092.stdout:8/917: rename d9/dc/fc8 to d9/d21/d37/dbd/dc5/f13c 0 2024-05-06T12:14:46.796 INFO:tasks.workunit.client.1.smithi190.stdout:9/513: getdents db/dd/d16/d58/d87/d22/d49/d8c 0 2024-05-06T12:14:46.797 INFO:tasks.workunit.client.0.smithi092.stdout:8/918: creat d9/d5a/dd9/d10b/f13d x:0 0 0 2024-05-06T12:14:46.798 INFO:tasks.workunit.client.0.smithi092.stdout:8/919: dread - d9/d21/dc0/df2/f110 zero size 2024-05-06T12:14:46.800 INFO:tasks.workunit.client.1.smithi190.stdout:9/514: write db/dd/df/f2c [3019570,59557] 0 2024-05-06T12:14:46.800 INFO:tasks.workunit.client.1.smithi190.stdout:6/323: rename df/l2f to df/d17/l6e 0 2024-05-06T12:14:46.801 INFO:tasks.workunit.client.0.smithi092.stdout:8/920: fsync d9/d21/d37/d46/d5c/d62/f68 0 2024-05-06T12:14:46.803 INFO:tasks.workunit.client.1.smithi190.stdout:5/382: dwrite d2/da/d59/d5f/d2e/d3b/f68 [4194304,4194304] 0 2024-05-06T12:14:46.803 INFO:tasks.workunit.client.0.smithi092.stdout:8/921: dread - d9/d21/d37/d46/ff1 zero size 2024-05-06T12:14:46.806 INFO:tasks.workunit.client.1.smithi190.stdout:3/445: creat d5/d7/d15/d39/f9b x:0 0 0 2024-05-06T12:14:46.806 INFO:tasks.workunit.client.0.smithi092.stdout:8/922: truncate d9/fd3 4244465 0 2024-05-06T12:14:46.812 INFO:tasks.workunit.client.1.smithi190.stdout:1/317: write d5/d15/f1a [4992622,78123] 0 2024-05-06T12:14:46.813 INFO:tasks.workunit.client.1.smithi190.stdout:1/318: fdatasync d5/d15/f52 0 2024-05-06T12:14:46.813 INFO:tasks.workunit.client.1.smithi190.stdout:2/349: rename d2/d8/d33/c65 to d2/d8/d33/d46/d55/c6c 0 2024-05-06T12:14:46.816 INFO:tasks.workunit.client.1.smithi190.stdout:9/515: mknod db/dd/d16/d58/cb8 0 2024-05-06T12:14:46.818 INFO:tasks.workunit.client.0.smithi092.stdout:8/923: dwrite d9/dc/d4a/f6b [0,4194304] 0 2024-05-06T12:14:46.819 INFO:tasks.workunit.client.1.smithi190.stdout:5/383: rename d2/d27/c3a to d2/d27/c74 0 2024-05-06T12:14:46.819 INFO:tasks.workunit.client.1.smithi190.stdout:5/384: readlink d2/da/l34 0 2024-05-06T12:14:46.820 INFO:tasks.workunit.client.1.smithi190.stdout:8/465: write d3/f12 [234076,58815] 0 2024-05-06T12:14:46.823 INFO:tasks.workunit.client.1.smithi190.stdout:1/319: mknod d5/d4f/c69 0 2024-05-06T12:14:46.826 INFO:tasks.workunit.client.0.smithi092.stdout:8/924: link d9/d21/d37/f8f d9/d21/d54/d79/d8a/f13e 0 2024-05-06T12:14:46.827 INFO:tasks.workunit.client.0.smithi092.stdout:8/925: fsync d9/d21/d54/f7b 0 2024-05-06T12:14:46.828 INFO:tasks.workunit.client.1.smithi190.stdout:5/385: mknod d2/da/d59/d5f/d2e/d3b/c75 0 2024-05-06T12:14:46.829 INFO:tasks.workunit.client.1.smithi190.stdout:3/446: truncate d5/d7/d15/d1f/f42 3899375 0 2024-05-06T12:14:46.830 INFO:tasks.workunit.client.0.smithi092.stdout:8/926: getdents d9/d21/d37/dbd/dc5/d11c 0 2024-05-06T12:14:46.834 INFO:tasks.workunit.client.1.smithi190.stdout:6/324: write df/d17/d37/f3c [3963977,61502] 0 2024-05-06T12:14:46.834 INFO:tasks.workunit.client.1.smithi190.stdout:9/516: creat db/dd/d16/d58/d87/d22/d49/d88/db5/fb9 x:0 0 0 2024-05-06T12:14:46.834 INFO:tasks.workunit.client.1.smithi190.stdout:0/500: getdents d9/dc/d48 0 2024-05-06T12:14:46.836 INFO:tasks.workunit.client.1.smithi190.stdout:7/314: write d0/d1/f11 [902080,2426] 0 2024-05-06T12:14:46.836 INFO:tasks.workunit.client.1.smithi190.stdout:5/386: dread d2/da/d59/d5f/d2e/d3b/f52 [0,4194304] 0 2024-05-06T12:14:46.837 INFO:tasks.workunit.client.0.smithi092.stdout:8/927: dwrite d9/d21/d37/d46/d5c/d71/d75/fef [0,4194304] 0 2024-05-06T12:14:46.838 INFO:tasks.workunit.client.1.smithi190.stdout:2/350: write d2/f38 [180969,42553] 0 2024-05-06T12:14:46.839 INFO:tasks.workunit.client.1.smithi190.stdout:6/325: rmdir df/d17/d37 39 2024-05-06T12:14:46.840 INFO:tasks.workunit.client.1.smithi190.stdout:0/501: mknod d9/dc/d28/d4c/d96/cb0 0 2024-05-06T12:14:46.841 INFO:tasks.workunit.client.1.smithi190.stdout:9/517: dwrite db/dd/d5c/fb0 [0,4194304] 0 2024-05-06T12:14:46.842 INFO:tasks.workunit.client.1.smithi190.stdout:6/326: write df/d2c/f36 [1695070,63242] 0 2024-05-06T12:14:46.842 INFO:tasks.workunit.client.1.smithi190.stdout:0/502: chown d9/df/f33 268 1 2024-05-06T12:14:46.843 INFO:tasks.workunit.client.1.smithi190.stdout:8/466: rename d3/d8/d2b/f7e to d3/d35/fa0 0 2024-05-06T12:14:46.845 INFO:tasks.workunit.client.1.smithi190.stdout:3/447: mknod d5/d7/c9c 0 2024-05-06T12:14:46.845 INFO:tasks.workunit.client.0.smithi092.stdout:8/928: dwrite d9/d21/f122 [0,4194304] 0 2024-05-06T12:14:46.846 INFO:tasks.workunit.client.1.smithi190.stdout:9/518: fdatasync db/dd/d16/d58/d87/f45 0 2024-05-06T12:14:46.850 INFO:tasks.workunit.client.0.smithi092.stdout:8/929: link d9/d21/f103 d9/d5a/f13f 0 2024-05-06T12:14:46.850 INFO:tasks.workunit.client.1.smithi190.stdout:2/351: dread d2/d10/f41 [0,4194304] 0 2024-05-06T12:14:46.852 INFO:tasks.workunit.client.1.smithi190.stdout:2/352: write d2/f3b [1563136,20927] 0 2024-05-06T12:14:46.852 INFO:tasks.workunit.client.0.smithi092.stdout:8/930: fdatasync d9/dc/f87 0 2024-05-06T12:14:46.853 INFO:tasks.workunit.client.1.smithi190.stdout:7/315: mkdir d0/da/d36/d53/d77 0 2024-05-06T12:14:46.854 INFO:tasks.workunit.client.1.smithi190.stdout:5/387: mkdir d2/da/d59/d5f/d2e/d3b/d76 0 2024-05-06T12:14:46.854 INFO:tasks.workunit.client.1.smithi190.stdout:9/519: truncate db/dd/d16/d58/d87/d22/d49/d88/db5/fb9 468508 0 2024-05-06T12:14:46.854 INFO:tasks.workunit.client.0.smithi092.stdout:8/931: fdatasync d9/fc2 0 2024-05-06T12:14:46.856 INFO:tasks.workunit.client.1.smithi190.stdout:0/503: creat d9/dc/fb1 x:0 0 0 2024-05-06T12:14:46.856 INFO:tasks.workunit.client.1.smithi190.stdout:6/327: dread - df/d17/d37/f58 zero size 2024-05-06T12:14:46.857 INFO:tasks.workunit.client.1.smithi190.stdout:1/320: link d5/d4f/d65/d24/d3a/l3d d5/d4f/d65/l6a 0 2024-05-06T12:14:46.858 INFO:tasks.workunit.client.1.smithi190.stdout:0/504: write d9/dc/fb1 [860357,43271] 0 2024-05-06T12:14:46.858 INFO:tasks.workunit.client.0.smithi092.stdout:8/932: rename d9/d9d/d10f to d9/d21/d54/d116/d11f/d12e/d140 0 2024-05-06T12:14:46.859 INFO:tasks.workunit.client.0.smithi092.stdout:8/933: chown d9/d21/d37/lb7 5092 1 2024-05-06T12:14:46.859 INFO:tasks.workunit.client.1.smithi190.stdout:3/448: rename d5/d7/d15/f22 to d5/d7/d6d/f9d 0 2024-05-06T12:14:46.860 INFO:tasks.workunit.client.1.smithi190.stdout:3/449: fdatasync d5/d7/f73 0 2024-05-06T12:14:46.861 INFO:tasks.workunit.client.1.smithi190.stdout:7/316: symlink d0/da/d36/l78 0 2024-05-06T12:14:46.861 INFO:tasks.workunit.client.0.smithi092.stdout:8/934: creat d9/d9d/d13b/f141 x:0 0 0 2024-05-06T12:14:46.862 INFO:tasks.workunit.client.1.smithi190.stdout:5/388: creat d2/da/d59/f77 x:0 0 0 2024-05-06T12:14:46.862 INFO:tasks.workunit.client.1.smithi190.stdout:7/317: read d0/da/f55 [46543,5883] 0 2024-05-06T12:14:46.863 INFO:tasks.workunit.client.1.smithi190.stdout:8/467: mknod d3/d8/d25/d3b/d6b/ca1 0 2024-05-06T12:14:46.866 INFO:tasks.workunit.client.1.smithi190.stdout:6/328: creat df/d2c/d1e/d57/d53/f6f x:0 0 0 2024-05-06T12:14:46.867 INFO:tasks.workunit.client.1.smithi190.stdout:6/329: chown df/c4a 568210 1 2024-05-06T12:14:46.868 INFO:tasks.workunit.client.1.smithi190.stdout:2/353: rename d2/d8/dc/f5b to d2/d8/d33/d46/d55/f6d 0 2024-05-06T12:14:46.870 INFO:tasks.workunit.client.1.smithi190.stdout:2/354: fsync d2/d8/dc/f1a 0 2024-05-06T12:14:46.870 INFO:tasks.workunit.client.1.smithi190.stdout:5/389: dwrite d2/da/f4a [0,4194304] 0 2024-05-06T12:14:46.871 INFO:tasks.workunit.client.1.smithi190.stdout:5/390: fsync d2/fc 0 2024-05-06T12:14:46.874 INFO:tasks.workunit.client.1.smithi190.stdout:1/321: mkdir d5/d6b 0 2024-05-06T12:14:46.876 INFO:tasks.workunit.client.1.smithi190.stdout:7/318: mknod d0/d1/d32/d45/d70/d35/c79 0 2024-05-06T12:14:46.877 INFO:tasks.workunit.client.1.smithi190.stdout:7/319: write d0/d4/f6b [34241,337] 0 2024-05-06T12:14:46.879 INFO:tasks.workunit.client.1.smithi190.stdout:5/391: dwrite d2/d27/f48 [0,4194304] 0 2024-05-06T12:14:46.886 INFO:tasks.workunit.client.1.smithi190.stdout:6/330: creat df/d2c/d61/f70 x:0 0 0 2024-05-06T12:14:46.894 INFO:tasks.workunit.client.1.smithi190.stdout:1/322: mknod d5/d4f/d65/d24/d3a/c6c 0 2024-05-06T12:14:46.894 INFO:tasks.workunit.client.1.smithi190.stdout:2/355: dwrite d2/d8/dc/d20/f52 [0,4194304] 0 2024-05-06T12:14:46.894 INFO:tasks.workunit.client.1.smithi190.stdout:8/468: write d3/f6 [8737200,113398] 0 2024-05-06T12:14:46.896 INFO:tasks.workunit.client.1.smithi190.stdout:7/320: symlink d0/d1/d18/d37/l7a 0 2024-05-06T12:14:46.898 INFO:tasks.workunit.client.1.smithi190.stdout:5/392: rename d2/da/d59/d5f/d2e/l1e to d2/da/d59/d5f/d50/d6d/l78 0 2024-05-06T12:14:46.899 INFO:tasks.workunit.client.1.smithi190.stdout:6/331: symlink df/d2c/d1e/d57/d53/l71 0 2024-05-06T12:14:46.900 INFO:tasks.workunit.client.1.smithi190.stdout:0/505: truncate d9/d6f/f7a 3323216 0 2024-05-06T12:14:46.902 INFO:tasks.workunit.client.1.smithi190.stdout:9/520: getdents db/d23 0 2024-05-06T12:14:46.902 INFO:tasks.workunit.client.1.smithi190.stdout:1/323: fdatasync d5/fa 0 2024-05-06T12:14:46.904 INFO:tasks.workunit.client.1.smithi190.stdout:2/356: unlink d2/d8/dc/f3c 0 2024-05-06T12:14:46.907 INFO:tasks.workunit.client.1.smithi190.stdout:6/332: creat df/d12/f72 x:0 0 0 2024-05-06T12:14:46.907 INFO:tasks.workunit.client.1.smithi190.stdout:5/393: dwrite d2/da/f4a [4194304,4194304] 0 2024-05-06T12:14:46.909 INFO:tasks.workunit.client.1.smithi190.stdout:5/394: dread - d2/da/d59/d5f/d2e/d3b/f40 zero size 2024-05-06T12:14:46.912 INFO:tasks.workunit.client.1.smithi190.stdout:2/357: mknod d2/d8/d19/d4f/c6e 0 2024-05-06T12:14:46.912 INFO:tasks.workunit.client.1.smithi190.stdout:8/469: creat d3/d8/fa2 x:0 0 0 2024-05-06T12:14:46.913 INFO:tasks.workunit.client.1.smithi190.stdout:2/358: write d2/d10/f41 [4560032,113444] 0 2024-05-06T12:14:46.914 INFO:tasks.workunit.client.1.smithi190.stdout:4/269: sync 2024-05-06T12:14:46.914 INFO:tasks.workunit.client.1.smithi190.stdout:0/506: creat d9/df/d56/dae/fb2 x:0 0 0 2024-05-06T12:14:46.915 INFO:tasks.workunit.client.1.smithi190.stdout:8/470: write d3/d35/d55/d61/f63 [144238,18688] 0 2024-05-06T12:14:46.916 INFO:tasks.workunit.client.1.smithi190.stdout:6/333: rmdir df/d17/d37 39 2024-05-06T12:14:46.917 INFO:tasks.workunit.client.1.smithi190.stdout:7/321: creat d0/d14/f7b x:0 0 0 2024-05-06T12:14:46.917 INFO:tasks.workunit.client.1.smithi190.stdout:8/471: chown d3/d8/d25/d3b/l45 119477 1 2024-05-06T12:14:46.918 INFO:tasks.workunit.client.1.smithi190.stdout:5/395: mknod d2/da/d59/d5f/d2e/c79 0 2024-05-06T12:14:46.919 INFO:tasks.workunit.client.1.smithi190.stdout:3/450: truncate d5/d14/d11/d21/f38 2194640 0 2024-05-06T12:14:46.919 INFO:tasks.workunit.client.1.smithi190.stdout:7/322: chown d0/d1/d18/d2b/f30 1008574 1 2024-05-06T12:14:46.920 INFO:tasks.workunit.client.1.smithi190.stdout:3/451: truncate d5/d7/f71 679027 0 2024-05-06T12:14:46.922 INFO:tasks.workunit.client.1.smithi190.stdout:2/359: rename d2/d10/f41 to d2/d4c/f6f 0 2024-05-06T12:14:46.924 INFO:tasks.workunit.client.1.smithi190.stdout:4/270: unlink d2/f4 0 2024-05-06T12:14:46.925 INFO:tasks.workunit.client.1.smithi190.stdout:6/334: mkdir df/d2c/d61/d73 0 2024-05-06T12:14:46.927 INFO:tasks.workunit.client.1.smithi190.stdout:9/521: creat db/dd/d16/d58/d87/d22/fba x:0 0 0 2024-05-06T12:14:46.928 INFO:tasks.workunit.client.1.smithi190.stdout:8/472: creat d3/fa3 x:0 0 0 2024-05-06T12:14:46.933 INFO:tasks.workunit.client.1.smithi190.stdout:7/323: creat d0/d1/d2/f7c x:0 0 0 2024-05-06T12:14:46.935 INFO:tasks.workunit.client.1.smithi190.stdout:6/335: dread df/d2c/f36 [0,4194304] 0 2024-05-06T12:14:46.935 INFO:tasks.workunit.client.1.smithi190.stdout:3/452: chown d5/d14/f56 63978 1 2024-05-06T12:14:46.936 INFO:tasks.workunit.client.1.smithi190.stdout:2/360: dread d2/f3b [0,4194304] 0 2024-05-06T12:14:46.937 INFO:tasks.workunit.client.1.smithi190.stdout:1/324: write d5/d15/f30 [51696,92340] 0 2024-05-06T12:14:46.941 INFO:tasks.workunit.client.1.smithi190.stdout:5/396: creat d2/d6c/f7a x:0 0 0 2024-05-06T12:14:46.942 INFO:tasks.workunit.client.1.smithi190.stdout:7/324: creat d0/d1/d32/d45/f7d x:0 0 0 2024-05-06T12:14:46.946 INFO:tasks.workunit.client.1.smithi190.stdout:4/271: creat d2/d11/d16/d41/f62 x:0 0 0 2024-05-06T12:14:46.946 INFO:tasks.workunit.client.1.smithi190.stdout:7/325: dread d0/da/d36/f43 [0,4194304] 0 2024-05-06T12:14:46.946 INFO:tasks.workunit.client.1.smithi190.stdout:7/326: fsync d0/d14/f6e 0 2024-05-06T12:14:46.947 INFO:tasks.workunit.client.1.smithi190.stdout:3/453: mkdir d5/d14/d11/d21/d44/d92/d9e 0 2024-05-06T12:14:46.948 INFO:tasks.workunit.client.1.smithi190.stdout:3/454: fsync d5/d7/d15/d1f/fe 0 2024-05-06T12:14:46.951 INFO:tasks.workunit.client.1.smithi190.stdout:2/361: unlink d2/d8/d33/f6b 0 2024-05-06T12:14:46.955 INFO:tasks.workunit.client.1.smithi190.stdout:5/397: symlink d2/da/d59/d5f/d50/l7b 0 2024-05-06T12:14:46.956 INFO:tasks.workunit.client.1.smithi190.stdout:6/336: mknod df/c74 0 2024-05-06T12:14:46.969 INFO:tasks.workunit.client.1.smithi190.stdout:4/272: link d2/f32 d2/d10/f63 0 2024-05-06T12:14:46.971 INFO:tasks.workunit.client.1.smithi190.stdout:7/327: link d0/da/d36/f43 d0/da/f7e 0 2024-05-06T12:14:46.973 INFO:tasks.workunit.client.1.smithi190.stdout:1/325: creat d5/f6d x:0 0 0 2024-05-06T12:14:46.974 INFO:tasks.workunit.client.1.smithi190.stdout:1/326: write d5/d4f/d65/d24/f2c [328347,103263] 0 2024-05-06T12:14:46.976 INFO:tasks.workunit.client.1.smithi190.stdout:9/522: getdents db/dd/d16/d58/d87/d22/d49/d88/d76 0 2024-05-06T12:14:46.977 INFO:tasks.workunit.client.1.smithi190.stdout:9/523: chown db/daa 5223 1 2024-05-06T12:14:46.978 INFO:tasks.workunit.client.1.smithi190.stdout:8/473: creat d3/d8/d25/d3b/d6b/d7a/fa4 x:0 0 0 2024-05-06T12:14:46.979 INFO:tasks.workunit.client.1.smithi190.stdout:5/398: truncate d2/f44 1098868 0 2024-05-06T12:14:46.983 INFO:tasks.workunit.client.1.smithi190.stdout:3/455: write d5/d14/f24 [449963,97658] 0 2024-05-06T12:14:46.984 INFO:tasks.workunit.client.1.smithi190.stdout:4/273: unlink d2/f4d 0 2024-05-06T12:14:46.984 INFO:tasks.workunit.client.1.smithi190.stdout:9/524: dwrite db/dd/df/f1b [4194304,4194304] 0 2024-05-06T12:14:46.990 INFO:tasks.workunit.client.1.smithi190.stdout:5/399: dread d2/da/f1b [4194304,4194304] 0 2024-05-06T12:14:46.990 INFO:tasks.workunit.client.1.smithi190.stdout:2/362: creat d2/d21/f70 x:0 0 0 2024-05-06T12:14:46.991 INFO:tasks.workunit.client.1.smithi190.stdout:6/337: write df/f22 [1484824,15372] 0 2024-05-06T12:14:46.994 INFO:tasks.workunit.client.1.smithi190.stdout:1/327: dread d5/d4f/d65/d24/f2f [0,4194304] 0 2024-05-06T12:14:46.997 INFO:tasks.workunit.client.1.smithi190.stdout:9/525: dwrite db/dd/d16/d4c/f69 [0,4194304] 0 2024-05-06T12:14:46.997 INFO:tasks.workunit.client.1.smithi190.stdout:2/363: mkdir d2/d8/d71 0 2024-05-06T12:14:46.999 INFO:tasks.workunit.client.1.smithi190.stdout:2/364: write d2/d8/d33/f37 [111591,105914] 0 2024-05-06T12:14:47.001 INFO:tasks.workunit.client.1.smithi190.stdout:8/474: creat d3/d35/d55/fa5 x:0 0 0 2024-05-06T12:14:47.003 INFO:tasks.workunit.client.1.smithi190.stdout:7/328: rmdir d0/da/d36/d53/d6f 0 2024-05-06T12:14:47.004 INFO:tasks.workunit.client.1.smithi190.stdout:7/329: read - d0/da/d36/d42/f76 zero size 2024-05-06T12:14:47.007 INFO:tasks.workunit.client.1.smithi190.stdout:1/328: mkdir d5/d4f/d65/d32/d47/d53/d6e 0 2024-05-06T12:14:47.010 INFO:tasks.workunit.client.1.smithi190.stdout:7/330: dread d0/d1/d32/d45/d70/d3b/f48 [0,4194304] 0 2024-05-06T12:14:47.010 INFO:tasks.workunit.client.1.smithi190.stdout:3/456: creat d5/d14/d11/f9f x:0 0 0 2024-05-06T12:14:47.011 INFO:tasks.workunit.client.1.smithi190.stdout:8/475: mknod d3/d35/d3a/ca6 0 2024-05-06T12:14:47.013 INFO:tasks.workunit.client.1.smithi190.stdout:5/400: dwrite d2/da/d59/d5f/f36 [0,4194304] 0 2024-05-06T12:14:47.013 INFO:tasks.workunit.client.1.smithi190.stdout:4/274: creat d2/f64 x:0 0 0 2024-05-06T12:14:47.013 INFO:tasks.workunit.client.1.smithi190.stdout:8/476: dread - d3/d8/d25/d65/f93 zero size 2024-05-06T12:14:47.014 INFO:tasks.workunit.client.1.smithi190.stdout:2/365: mkdir d2/d72 0 2024-05-06T12:14:47.015 INFO:tasks.workunit.client.1.smithi190.stdout:9/526: symlink db/dd/d16/d4c/d4e/d5e/lbb 0 2024-05-06T12:14:47.015 INFO:tasks.workunit.client.1.smithi190.stdout:8/477: chown d3/d35/d55/d61/c85 394 1 2024-05-06T12:14:47.020 INFO:tasks.workunit.client.1.smithi190.stdout:7/331: mknod d0/d4c/c7f 0 2024-05-06T12:14:47.023 INFO:tasks.workunit.client.1.smithi190.stdout:5/401: mknod d2/d27/c7c 0 2024-05-06T12:14:47.025 INFO:tasks.workunit.client.1.smithi190.stdout:1/329: creat d5/d4f/d65/d32/d47/d53/d6e/f6f x:0 0 0 2024-05-06T12:14:47.026 INFO:tasks.workunit.client.1.smithi190.stdout:1/330: readlink d5/l4b 0 2024-05-06T12:14:47.028 INFO:tasks.workunit.client.1.smithi190.stdout:9/527: symlink db/d23/d29/lbc 0 2024-05-06T12:14:47.028 INFO:tasks.workunit.client.1.smithi190.stdout:1/331: fsync d5/d15/f30 0 2024-05-06T12:14:47.030 INFO:tasks.workunit.client.1.smithi190.stdout:6/338: getdents df/d2c/d1e/d57 0 2024-05-06T12:14:47.030 INFO:tasks.workunit.client.1.smithi190.stdout:3/457: symlink d5/d94/la0 0 2024-05-06T12:14:47.032 INFO:tasks.workunit.client.1.smithi190.stdout:7/332: unlink d0/d1/d18/d2b/d5a/c74 0 2024-05-06T12:14:47.034 INFO:tasks.workunit.client.1.smithi190.stdout:2/366: symlink d2/d8/d33/d46/d5d/l73 0 2024-05-06T12:14:47.035 INFO:tasks.workunit.client.1.smithi190.stdout:5/402: dwrite d2/da/d59/d5f/d2e/d3b/f40 [0,4194304] 0 2024-05-06T12:14:47.036 INFO:tasks.workunit.client.1.smithi190.stdout:7/333: chown d0/da/d36/f43 233446667 1 2024-05-06T12:14:47.041 INFO:tasks.workunit.client.1.smithi190.stdout:9/528: creat db/dd/d16/d58/d87/d22/d49/d88/db5/fbd x:0 0 0 2024-05-06T12:14:47.042 INFO:tasks.workunit.client.1.smithi190.stdout:2/367: chown d2/d8/d19/d4f/c6e 0 1 2024-05-06T12:14:47.042 INFO:tasks.workunit.client.1.smithi190.stdout:3/458: fsync d5/d7/d6d/f9d 0 2024-05-06T12:14:47.042 INFO:tasks.workunit.client.1.smithi190.stdout:4/275: creat d2/d11/d16/d41/f65 x:0 0 0 2024-05-06T12:14:47.044 INFO:tasks.workunit.client.1.smithi190.stdout:8/478: dwrite d3/d8/f30 [0,4194304] 0 2024-05-06T12:14:47.044 INFO:tasks.workunit.client.1.smithi190.stdout:7/334: creat d0/d1/d18/d2b/f80 x:0 0 0 2024-05-06T12:14:47.050 INFO:tasks.workunit.client.1.smithi190.stdout:3/459: mkdir d5/d7/d15/d1f/d33/d40/da1 0 2024-05-06T12:14:47.051 INFO:tasks.workunit.client.1.smithi190.stdout:4/276: fsync d2/f1e 0 2024-05-06T12:14:47.052 INFO:tasks.workunit.client.1.smithi190.stdout:8/479: truncate d3/d8/d25/d3b/d6b/f76 824052 0 2024-05-06T12:14:47.054 INFO:tasks.workunit.client.1.smithi190.stdout:1/332: link d5/d4f/d65/d24/f45 d5/d4f/d65/d32/f70 0 2024-05-06T12:14:47.056 INFO:tasks.workunit.client.1.smithi190.stdout:7/335: dread d0/da/d36/f46 [0,4194304] 0 2024-05-06T12:14:47.059 INFO:tasks.workunit.client.1.smithi190.stdout:3/460: rename d5/d14/d11/d21/d44/d80/d82/d3d/f5a to d5/d14/d11/d21/d44/d92/d9e/fa2 0 2024-05-06T12:14:47.062 INFO:tasks.workunit.client.1.smithi190.stdout:6/339: dread df/f18 [0,4194304] 0 2024-05-06T12:14:47.064 INFO:tasks.workunit.client.1.smithi190.stdout:7/336: mkdir d0/d1/d81 0 2024-05-06T12:14:47.066 INFO:tasks.workunit.client.1.smithi190.stdout:5/403: dread d2/da/d59/d5f/f3c [0,4194304] 0 2024-05-06T12:14:47.066 INFO:tasks.workunit.client.1.smithi190.stdout:7/337: write d0/d1/d32/f38 [1256562,39461] 0 2024-05-06T12:14:47.067 INFO:tasks.workunit.client.1.smithi190.stdout:1/333: truncate d5/d4f/d65/d24/d43/f60 1704038 0 2024-05-06T12:14:47.068 INFO:tasks.workunit.client.1.smithi190.stdout:8/480: dread d3/d8/d2b/f38 [0,4194304] 0 2024-05-06T12:14:47.069 INFO:tasks.workunit.client.1.smithi190.stdout:5/404: write d2/d27/f48 [2270945,69855] 0 2024-05-06T12:14:47.070 INFO:tasks.workunit.client.1.smithi190.stdout:6/340: rename f2 to df/d2c/f75 0 2024-05-06T12:14:47.070 INFO:tasks.workunit.client.1.smithi190.stdout:9/529: dread db/dd/f34 [0,4194304] 0 2024-05-06T12:14:47.070 INFO:tasks.workunit.client.1.smithi190.stdout:2/368: write d2/d10/f2c [1209231,10578] 0 2024-05-06T12:14:47.073 INFO:tasks.workunit.client.1.smithi190.stdout:5/405: write d2/da/d59/d5f/d2e/d3b/f68 [3325976,21357] 0 2024-05-06T12:14:47.073 INFO:tasks.workunit.client.1.smithi190.stdout:7/338: rmdir d0/da/d36/d53 39 2024-05-06T12:14:47.073 INFO:tasks.workunit.client.1.smithi190.stdout:4/277: write d2/d11/f4c [4791131,40721] 0 2024-05-06T12:14:47.075 INFO:tasks.workunit.client.1.smithi190.stdout:8/481: chown d3/d13/c81 3 1 2024-05-06T12:14:47.075 INFO:tasks.workunit.client.1.smithi190.stdout:7/339: write d0/d1/d2/f7c [7608,113807] 0 2024-05-06T12:14:47.076 INFO:tasks.workunit.client.1.smithi190.stdout:1/334: unlink d5/d4f/d65/d24/f2f 0 2024-05-06T12:14:47.078 INFO:tasks.workunit.client.1.smithi190.stdout:6/341: creat df/d12/f76 x:0 0 0 2024-05-06T12:14:47.081 INFO:tasks.workunit.client.1.smithi190.stdout:0/507: sync 2024-05-06T12:14:47.081 INFO:tasks.workunit.client.1.smithi190.stdout:6/342: dread - df/d12/f65 zero size 2024-05-06T12:14:47.081 INFO:tasks.workunit.client.1.smithi190.stdout:5/406: fsync d2/da/f4a 0 2024-05-06T12:14:47.084 INFO:tasks.workunit.client.1.smithi190.stdout:9/530: dwrite db/d23/f3a [0,4194304] 0 2024-05-06T12:14:47.084 INFO:tasks.workunit.client.1.smithi190.stdout:8/482: rename d3/d8/d2b/f80 to d3/d35/d3a/fa7 0 2024-05-06T12:14:47.090 INFO:tasks.workunit.client.1.smithi190.stdout:9/531: readlink db/dd/d16/d58/d87/d22/d49/d88/d76/l96 0 2024-05-06T12:14:47.090 INFO:tasks.workunit.client.1.smithi190.stdout:3/461: getdents d5/d14/d11/d21/d44/d92 0 2024-05-06T12:14:47.090 INFO:tasks.workunit.client.1.smithi190.stdout:2/369: dwrite d2/d4c/f6f [0,4194304] 0 2024-05-06T12:14:47.094 INFO:tasks.workunit.client.1.smithi190.stdout:7/340: symlink d0/da/d36/d53/l82 0 2024-05-06T12:14:47.094 INFO:tasks.workunit.client.1.smithi190.stdout:9/532: write db/dd/d16/d58/d87/d22/d49/d88/f9c [483470,113506] 0 2024-05-06T12:14:47.094 INFO:tasks.workunit.client.1.smithi190.stdout:7/341: stat d0/d1/d2/f1d 0 2024-05-06T12:14:47.095 INFO:tasks.workunit.client.1.smithi190.stdout:7/342: readlink d0/l16 0 2024-05-06T12:14:47.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:46 smithi190 ceph-mon[33798]: pgmap v25: 65 pgs: 65 active+clean; 2.3 GiB data, 7.9 GiB used, 529 GiB / 536 GiB avail; 64 MiB/s rd, 211 MiB/s wr, 442 op/s 2024-05-06T12:14:47.098 INFO:tasks.workunit.client.1.smithi190.stdout:6/343: mknod df/d2c/d1e/c77 0 2024-05-06T12:14:47.102 INFO:tasks.workunit.client.1.smithi190.stdout:2/370: creat d2/d8/d33/d46/d5d/f74 x:0 0 0 2024-05-06T12:14:47.103 INFO:tasks.workunit.client.1.smithi190.stdout:5/407: dwrite d2/d27/f54 [0,4194304] 0 2024-05-06T12:14:47.103 INFO:tasks.workunit.client.1.smithi190.stdout:3/462: mknod d5/d7/d15/d1f/d33/d5c/ca3 0 2024-05-06T12:14:47.105 INFO:tasks.workunit.client.1.smithi190.stdout:3/463: write d5/d7/d15/d1f/d33/f66 [860682,3381] 0 2024-05-06T12:14:47.106 INFO:tasks.workunit.client.1.smithi190.stdout:0/508: mkdir d9/d95/db3 0 2024-05-06T12:14:47.107 INFO:tasks.workunit.client.1.smithi190.stdout:9/533: creat db/dd/d16/d4c/d4e/d5e/fbe x:0 0 0 2024-05-06T12:14:47.108 INFO:tasks.workunit.client.0.smithi092.stdout:1/894: sync 2024-05-06T12:14:47.109 INFO:tasks.workunit.client.1.smithi190.stdout:1/335: rename d5/d4f/d65/d32/c63 to d5/d4f/d65/d32/c71 0 2024-05-06T12:14:47.111 INFO:tasks.workunit.client.1.smithi190.stdout:2/371: creat d2/d4c/f75 x:0 0 0 2024-05-06T12:14:47.111 INFO:tasks.workunit.client.0.smithi092.stdout:1/895: symlink d3/d42/d84/d4f/dd5/ded/l121 0 2024-05-06T12:14:47.112 INFO:tasks.workunit.client.1.smithi190.stdout:5/408: mkdir d2/d6c/d7d 0 2024-05-06T12:14:47.113 INFO:tasks.workunit.client.1.smithi190.stdout:3/464: mkdir d5/d14/d11/d21/d44/d92/d9e/da4 0 2024-05-06T12:14:47.114 INFO:tasks.workunit.client.0.smithi092.stdout:1/896: creat d3/d10/d13/d19/d21/d88/d105/f122 x:0 0 0 2024-05-06T12:14:47.115 INFO:tasks.workunit.client.1.smithi190.stdout:0/509: dwrite d9/df/d21/f2a [0,4194304] 0 2024-05-06T12:14:47.115 INFO:tasks.workunit.client.1.smithi190.stdout:9/534: truncate db/dd/d16/d58/d87/d22/d49/d88/db5/fbd 889506 0 2024-05-06T12:14:47.115 INFO:tasks.workunit.client.1.smithi190.stdout:5/409: rename d2/da/d59/d5f to d2/da/d59/d5f/d2e/d3b/d7e 22 2024-05-06T12:14:47.116 INFO:tasks.workunit.client.1.smithi190.stdout:8/483: link d3/d8/d25/d65/l95 d3/d8/d25/d3b/la8 0 2024-05-06T12:14:47.117 INFO:tasks.workunit.client.1.smithi190.stdout:9/535: chown la 1107 1 2024-05-06T12:14:47.118 INFO:tasks.workunit.client.1.smithi190.stdout:3/465: mknod d5/d94/ca5 0 2024-05-06T12:14:47.118 INFO:tasks.workunit.client.1.smithi190.stdout:5/410: write d2/d6c/f7a [698466,30245] 0 2024-05-06T12:14:47.119 INFO:tasks.workunit.client.1.smithi190.stdout:2/372: mknod d2/c76 0 2024-05-06T12:14:47.121 INFO:tasks.workunit.client.1.smithi190.stdout:5/411: chown d2/da/d59/d5f/d2e/c18 39 1 2024-05-06T12:14:47.121 INFO:tasks.workunit.client.0.smithi092.stdout:1/897: dwrite d3/d42/d84/d4f/dd5/ded/f10f [0,4194304] 0 2024-05-06T12:14:47.122 INFO:tasks.workunit.client.1.smithi190.stdout:0/510: mknod d9/df/d56/cb4 0 2024-05-06T12:14:47.123 INFO:tasks.workunit.client.1.smithi190.stdout:8/484: unlink d3/d35/d55/d7f/c83 0 2024-05-06T12:14:47.125 INFO:tasks.workunit.client.1.smithi190.stdout:9/536: rename db/d23/f4d to db/d23/d25/d47/fbf 0 2024-05-06T12:14:47.126 INFO:tasks.workunit.client.1.smithi190.stdout:2/373: dread d2/d8/dc/f1a [0,4194304] 0 2024-05-06T12:14:47.128 INFO:tasks.workunit.client.1.smithi190.stdout:8/485: mknod d3/d8/d25/d3b/d90/d6e/ca9 0 2024-05-06T12:14:47.128 INFO:tasks.workunit.client.1.smithi190.stdout:8/486: write d3/d35/f87 [907682,3015] 0 2024-05-06T12:14:47.129 INFO:tasks.workunit.client.1.smithi190.stdout:8/487: dread - d3/d35/d55/fa5 zero size 2024-05-06T12:14:47.133 INFO:tasks.workunit.client.0.smithi092.stdout:1/898: dread d3/d10/f53 [0,4194304] 0 2024-05-06T12:14:47.149 INFO:tasks.workunit.client.1.smithi190.stdout:3/466: mknod d5/d14/d11/d21/d44/d92/d9e/da4/ca6 0 2024-05-06T12:14:47.152 INFO:tasks.workunit.client.1.smithi190.stdout:8/488: mknod d3/d35/d55/caa 0 2024-05-06T12:14:47.155 INFO:tasks.workunit.client.1.smithi190.stdout:3/467: dwrite d5/d7/d15/d1f/d33/f66 [0,4194304] 0 2024-05-06T12:14:47.156 INFO:tasks.workunit.client.1.smithi190.stdout:3/468: fsync d5/f51 0 2024-05-06T12:14:47.156 INFO:tasks.workunit.client.1.smithi190.stdout:5/412: getdents d2/da/d59/d5f 0 2024-05-06T12:14:47.158 INFO:tasks.workunit.client.1.smithi190.stdout:5/413: write d2/fc [4408270,63714] 0 2024-05-06T12:14:47.158 INFO:tasks.workunit.client.1.smithi190.stdout:8/489: mknod d3/d8/d2b/d3e/cab 0 2024-05-06T12:14:47.161 INFO:tasks.workunit.client.1.smithi190.stdout:2/374: rmdir d2/d72 0 2024-05-06T12:14:47.162 INFO:tasks.workunit.client.1.smithi190.stdout:7/343: truncate d0/d1/d2/f5 4176094 0 2024-05-06T12:14:47.165 INFO:tasks.workunit.client.1.smithi190.stdout:1/336: truncate d5/d4f/d65/d32/d47/f4c 294019 0 2024-05-06T12:14:47.166 INFO:tasks.workunit.client.1.smithi190.stdout:1/337: fsync d5/d4f/d65/d2b/f35 0 2024-05-06T12:14:47.167 INFO:tasks.workunit.client.1.smithi190.stdout:5/414: unlink d2/da/d59/d5f/d2e/c18 0 2024-05-06T12:14:47.167 INFO:tasks.workunit.client.1.smithi190.stdout:1/338: stat d5/d4f/d65/d62/l67 0 2024-05-06T12:14:47.168 INFO:tasks.workunit.client.1.smithi190.stdout:2/375: symlink d2/d10/l77 0 2024-05-06T12:14:47.169 INFO:tasks.workunit.client.1.smithi190.stdout:8/490: dread d3/d13/f21 [0,4194304] 0 2024-05-06T12:14:47.173 INFO:tasks.workunit.client.1.smithi190.stdout:1/339: creat d5/d4f/d65/d24/d3a/f72 x:0 0 0 2024-05-06T12:14:47.174 INFO:tasks.workunit.client.1.smithi190.stdout:1/340: chown d5/d4f/d65/d24 1782785804 1 2024-05-06T12:14:47.174 INFO:tasks.workunit.client.1.smithi190.stdout:2/376: mknod d2/d8/dc/c78 0 2024-05-06T12:14:47.175 INFO:tasks.workunit.client.1.smithi190.stdout:3/469: rename d5/d7/d15/d39/c91 to d5/ca7 0 2024-05-06T12:14:47.175 INFO:tasks.workunit.client.1.smithi190.stdout:9/537: readlink db/d23/l6a 0 2024-05-06T12:14:47.176 INFO:tasks.workunit.client.1.smithi190.stdout:5/415: creat d2/da/d59/d5f/d6e/f7f x:0 0 0 2024-05-06T12:14:47.176 INFO:tasks.workunit.client.1.smithi190.stdout:2/377: write d2/d8/d19/f2a [5023622,26426] 0 2024-05-06T12:14:47.180 INFO:tasks.workunit.client.1.smithi190.stdout:2/378: mkdir d2/d8/d19/d43/d79 0 2024-05-06T12:14:47.182 INFO:tasks.workunit.client.1.smithi190.stdout:5/416: link d2/f3 d2/da/f80 0 2024-05-06T12:14:47.183 INFO:tasks.workunit.client.1.smithi190.stdout:3/470: dwrite d5/d14/f6a [0,4194304] 0 2024-05-06T12:14:47.185 INFO:tasks.workunit.client.1.smithi190.stdout:8/491: rename d3/f12 to d3/d8/d25/fac 0 2024-05-06T12:14:47.188 INFO:tasks.workunit.client.1.smithi190.stdout:5/417: mkdir d2/da/d59/d81 0 2024-05-06T12:14:47.191 INFO:tasks.workunit.client.0.smithi092.stdout:4/974: sync 2024-05-06T12:14:47.191 INFO:tasks.workunit.client.1.smithi190.stdout:3/471: dwrite d5/d14/d11/d21/d44/d80/d82/d3d/f49 [0,4194304] 0 2024-05-06T12:14:47.195 INFO:tasks.workunit.client.1.smithi190.stdout:5/418: mknod d2/d27/c82 0 2024-05-06T12:14:47.197 INFO:tasks.workunit.client.0.smithi092.stdout:4/975: rename d9/d27/dc5/lf0 to d9/df/d62/d7b/db6/l142 0 2024-05-06T12:14:47.198 INFO:tasks.workunit.client.1.smithi190.stdout:2/379: link d2/d8/dc/f1a d2/d8/d33/d46/f7a 0 2024-05-06T12:14:47.199 INFO:tasks.workunit.client.1.smithi190.stdout:3/472: mknod d5/d7/d15/d1f/d84/ca8 0 2024-05-06T12:14:47.200 INFO:tasks.workunit.client.1.smithi190.stdout:3/473: truncate d5/d14/f24 2123135 0 2024-05-06T12:14:47.201 INFO:tasks.workunit.client.1.smithi190.stdout:1/341: rename d5/d15/f1a to d5/d4f/d65/d24/d3a/f73 0 2024-05-06T12:14:47.204 INFO:tasks.workunit.client.1.smithi190.stdout:5/419: mkdir d2/d65/d83 0 2024-05-06T12:14:47.206 INFO:tasks.workunit.client.1.smithi190.stdout:5/420: write d2/da/d59/f77 [69325,120725] 0 2024-05-06T12:14:47.207 INFO:tasks.workunit.client.1.smithi190.stdout:3/474: dwrite d5/d14/f8f [0,4194304] 0 2024-05-06T12:14:47.209 INFO:tasks.workunit.client.1.smithi190.stdout:3/475: chown d5/d14/d11/d21/f30 6696 1 2024-05-06T12:14:47.209 INFO:tasks.workunit.client.1.smithi190.stdout:3/476: readlink d5/l7e 0 2024-05-06T12:14:47.213 INFO:tasks.workunit.client.1.smithi190.stdout:3/477: rmdir d5/d14/d11/d21/d44/d92/d9e/da4 39 2024-05-06T12:14:47.214 INFO:tasks.workunit.client.1.smithi190.stdout:3/478: write d5/d7/f71 [1700569,97603] 0 2024-05-06T12:14:47.221 INFO:tasks.workunit.client.1.smithi190.stdout:8/492: mkdir d3/d8/d2b/d3e/dad 0 2024-05-06T12:14:47.224 INFO:tasks.workunit.client.0.smithi092.stdout:4/976: dread d9/d2b/f32 [0,4194304] 0 2024-05-06T12:14:47.225 INFO:tasks.workunit.client.1.smithi190.stdout:2/380: dread d2/d8/d19/f24 [0,4194304] 0 2024-05-06T12:14:47.225 INFO:tasks.workunit.client.0.smithi092.stdout:4/977: chown d9/df/d62/d7b/d107/f129 4 1 2024-05-06T12:14:47.227 INFO:tasks.workunit.client.1.smithi190.stdout:3/479: dread d5/d14/f24 [0,4194304] 0 2024-05-06T12:14:47.227 INFO:tasks.workunit.client.1.smithi190.stdout:8/493: rename d3/d8/d2b/d3e/l44 to d3/lae 0 2024-05-06T12:14:47.230 INFO:tasks.workunit.client.1.smithi190.stdout:2/381: mkdir d2/d8/d19/d7b 0 2024-05-06T12:14:47.232 INFO:tasks.workunit.client.1.smithi190.stdout:2/382: write d2/d8/d33/d46/d5d/f74 [867558,22334] 0 2024-05-06T12:14:47.234 INFO:tasks.workunit.client.1.smithi190.stdout:3/480: rename d5/d14/d11/d21/f2e to d5/d7/d15/fa9 0 2024-05-06T12:14:47.236 INFO:tasks.workunit.client.1.smithi190.stdout:8/494: creat d3/d8/d68/faf x:0 0 0 2024-05-06T12:14:47.237 INFO:tasks.workunit.client.1.smithi190.stdout:5/421: truncate d2/d27/f54 1156701 0 2024-05-06T12:14:47.239 INFO:tasks.workunit.client.1.smithi190.stdout:2/383: creat d2/d8/d19/d43/f7c x:0 0 0 2024-05-06T12:14:47.239 INFO:tasks.workunit.client.1.smithi190.stdout:2/384: stat d2/d4c 0 2024-05-06T12:14:47.241 INFO:tasks.workunit.client.0.smithi092.stdout:9/932: sync 2024-05-06T12:14:47.241 INFO:tasks.workunit.client.0.smithi092.stdout:2/757: sync 2024-05-06T12:14:47.241 INFO:tasks.workunit.client.0.smithi092.stdout:7/988: sync 2024-05-06T12:14:47.241 INFO:tasks.workunit.client.0.smithi092.stdout:8/935: sync 2024-05-06T12:14:47.241 INFO:tasks.workunit.client.0.smithi092.stdout:3/737: sync 2024-05-06T12:14:47.245 INFO:tasks.workunit.client.0.smithi092.stdout:8/936: readlink d9/d21/d54/d79/lad 0 2024-05-06T12:14:47.246 INFO:tasks.workunit.client.0.smithi092.stdout:9/933: write d16/d2c/d2f/d75/f107 [4816762,7354] 0 2024-05-06T12:14:47.246 INFO:tasks.workunit.client.0.smithi092.stdout:3/738: write d8/d1b/f2b [7019286,36585] 0 2024-05-06T12:14:47.246 INFO:tasks.workunit.client.1.smithi190.stdout:8/495: mkdir d3/d8/db0 0 2024-05-06T12:14:47.248 INFO:tasks.workunit.client.1.smithi190.stdout:3/481: link d5/d7/d15/c69 d5/d79/caa 0 2024-05-06T12:14:47.249 INFO:tasks.workunit.client.0.smithi092.stdout:8/937: symlink d9/d21/dc0/df2/l142 0 2024-05-06T12:14:47.250 INFO:tasks.workunit.client.0.smithi092.stdout:3/739: creat d8/db/d4c/d40/d67/fe9 x:0 0 0 2024-05-06T12:14:47.251 INFO:tasks.workunit.client.1.smithi190.stdout:2/385: unlink d2/d21/d50/l51 0 2024-05-06T12:14:47.252 INFO:tasks.workunit.client.0.smithi092.stdout:7/989: creat d9/de/f151 x:0 0 0 2024-05-06T12:14:47.254 INFO:tasks.workunit.client.1.smithi190.stdout:3/482: dread d5/d14/d11/f13 [0,4194304] 0 2024-05-06T12:14:47.254 INFO:tasks.workunit.client.0.smithi092.stdout:8/938: creat d9/dc/d4a/d5f/da8/f143 x:0 0 0 2024-05-06T12:14:47.255 INFO:tasks.workunit.client.1.smithi190.stdout:3/483: write d5/d14/d11/d21/d44/d80/f83 [302234,30821] 0 2024-05-06T12:14:47.256 INFO:tasks.workunit.client.1.smithi190.stdout:8/496: rename d3/d8/d25/d3b/d6b/f76 to d3/fb1 0 2024-05-06T12:14:47.256 INFO:tasks.workunit.client.0.smithi092.stdout:7/990: creat d9/de/d10/d17/d99/db6/f152 x:0 0 0 2024-05-06T12:14:47.257 INFO:tasks.workunit.client.1.smithi190.stdout:3/484: write d5/d7/d15/d1f/fe [1926592,5611] 0 2024-05-06T12:14:47.259 INFO:tasks.workunit.client.1.smithi190.stdout:2/386: symlink d2/d8/d71/l7d 0 2024-05-06T12:14:47.260 INFO:tasks.workunit.client.0.smithi092.stdout:7/991: creat d9/de/d10/d17/d21/d2e/d87/f153 x:0 0 0 2024-05-06T12:14:47.261 INFO:tasks.workunit.client.0.smithi092.stdout:4/978: write d9/d27/f4f [1330949,126586] 0 2024-05-06T12:14:47.262 INFO:tasks.workunit.client.0.smithi092.stdout:7/992: readlink d9/de/d1f/d37/d39/l128 0 2024-05-06T12:14:47.264 INFO:tasks.workunit.client.0.smithi092.stdout:3/740: dread f3 [0,4194304] 0 2024-05-06T12:14:47.265 INFO:tasks.workunit.client.0.smithi092.stdout:4/979: rename d9/d27/fbe to d9/d102/f143 0 2024-05-06T12:14:47.265 INFO:tasks.workunit.client.0.smithi092.stdout:7/993: read d9/fb [2173933,84302] 0 2024-05-06T12:14:47.265 INFO:tasks.workunit.client.1.smithi190.stdout:2/387: rename d2/d8/dc/d20 to d2/d8/d19/d43/d79/d7e 0 2024-05-06T12:14:47.267 INFO:tasks.workunit.client.1.smithi190.stdout:2/388: dread - d2/d8/d33/d46/f5f zero size 2024-05-06T12:14:47.267 INFO:tasks.workunit.client.1.smithi190.stdout:3/485: truncate d5/d7/d15/d1f/f3b 745364 0 2024-05-06T12:14:47.267 INFO:tasks.workunit.client.1.smithi190.stdout:8/497: dwrite d3/d35/d55/fa5 [0,4194304] 0 2024-05-06T12:14:47.268 INFO:tasks.workunit.client.0.smithi092.stdout:4/980: unlink d9/df/d62/d7b/db6/fd4 0 2024-05-06T12:14:47.269 INFO:tasks.workunit.client.1.smithi190.stdout:5/422: write d2/da/d59/d5f/d2e/d3b/f47 [2879952,22208] 0 2024-05-06T12:14:47.269 INFO:tasks.workunit.client.0.smithi092.stdout:7/994: mknod d9/d62/d108/d140/c154 0 2024-05-06T12:14:47.270 INFO:tasks.workunit.client.1.smithi190.stdout:2/389: mknod d2/d8/d33/c7f 0 2024-05-06T12:14:47.270 INFO:tasks.workunit.client.1.smithi190.stdout:8/498: truncate d3/d8/d25/d3b/d90/f5f 719231 0 2024-05-06T12:14:47.271 INFO:tasks.workunit.client.0.smithi092.stdout:3/741: rmdir d8/d24/d33/d39/dd6 0 2024-05-06T12:14:47.272 INFO:tasks.workunit.client.0.smithi092.stdout:3/742: truncate d8/db/d4c/d40/d67/fe9 825895 0 2024-05-06T12:14:47.273 INFO:tasks.workunit.client.1.smithi190.stdout:3/486: creat d5/d14/d11/fab x:0 0 0 2024-05-06T12:14:47.273 INFO:tasks.workunit.client.0.smithi092.stdout:7/995: creat d9/de/d1f/d37/d48/d93/f155 x:0 0 0 2024-05-06T12:14:47.274 INFO:tasks.workunit.client.1.smithi190.stdout:8/499: creat d3/d8/d25/d3b/d6b/d7a/fb2 x:0 0 0 2024-05-06T12:14:47.274 INFO:tasks.workunit.client.1.smithi190.stdout:3/487: write d5/d14/d11/f9f [1029999,23789] 0 2024-05-06T12:14:47.274 INFO:tasks.workunit.client.1.smithi190.stdout:2/390: mknod d2/c80 0 2024-05-06T12:14:47.275 INFO:tasks.workunit.client.0.smithi092.stdout:7/996: readlink d9/laa 0 2024-05-06T12:14:47.276 INFO:tasks.workunit.client.0.smithi092.stdout:3/743: mkdir d8/db/d4c/d40/d7a/dbd/dea 0 2024-05-06T12:14:47.277 INFO:tasks.workunit.client.0.smithi092.stdout:3/744: write d8/d24/d33/d39/d5b/fc4 [470981,11331] 0 2024-05-06T12:14:47.280 INFO:tasks.workunit.client.1.smithi190.stdout:8/500: unlink d3/d8/d25/d3b/d90/f92 0 2024-05-06T12:14:47.282 INFO:tasks.workunit.client.1.smithi190.stdout:8/501: unlink d3/d8/d2b/d3e/f8e 0 2024-05-06T12:14:47.284 INFO:tasks.workunit.client.0.smithi092.stdout:3/745: dwrite d8/db/d4c/d40/d7a/d90/fd0 [4194304,4194304] 0 2024-05-06T12:14:47.285 INFO:tasks.workunit.client.1.smithi190.stdout:8/502: creat d3/d8/d25/d3b/d90/fb3 x:0 0 0 2024-05-06T12:14:47.286 INFO:tasks.workunit.client.1.smithi190.stdout:8/503: write d3/d8/d25/f7b [2052587,70268] 0 2024-05-06T12:14:47.288 INFO:tasks.workunit.client.0.smithi092.stdout:4/981: dread d9/df/d62/d7b/db6/d104/f10a [0,4194304] 0 2024-05-06T12:14:47.288 INFO:tasks.workunit.client.0.smithi092.stdout:3/746: rmdir d8/db/d4c/d3d/d48 39 2024-05-06T12:14:47.290 INFO:tasks.workunit.client.0.smithi092.stdout:4/982: fdatasync d9/d2b/f77 0 2024-05-06T12:14:47.292 INFO:tasks.workunit.client.0.smithi092.stdout:3/747: rename d8/db/f4d to d8/d24/d33/d4b/ddf/feb 0 2024-05-06T12:14:47.295 INFO:tasks.workunit.client.0.smithi092.stdout:4/983: fsync d9/dd/f5f 0 2024-05-06T12:14:47.298 INFO:tasks.workunit.client.0.smithi092.stdout:3/748: rename d8/db/d4c/d3d/d49/l8d to d8/db/d4c/d40/d7a/dbd/lec 0 2024-05-06T12:14:47.301 INFO:tasks.workunit.client.0.smithi092.stdout:3/749: write d8/db/d4c/d3d/fdd [819098,13137] 0 2024-05-06T12:14:47.305 INFO:tasks.workunit.client.0.smithi092.stdout:3/750: fsync d8/db/d4c/d40/d7a/d90/f93 0 2024-05-06T12:14:47.306 INFO:tasks.workunit.client.0.smithi092.stdout:3/751: dread - d8/db/d4c/d3d/d49/fe2 zero size 2024-05-06T12:14:47.307 INFO:tasks.workunit.client.0.smithi092.stdout:2/758: dwrite df/d17/d51/d96/f97 [0,4194304] 0 2024-05-06T12:14:47.310 INFO:tasks.workunit.client.0.smithi092.stdout:8/939: write d9/d21/f81 [479573,108430] 0 2024-05-06T12:14:47.311 INFO:tasks.workunit.client.1.smithi190.stdout:5/423: write d2/da/d59/d5f/f3c [2012359,64784] 0 2024-05-06T12:14:47.311 INFO:tasks.workunit.client.0.smithi092.stdout:4/984: rmdir d9/dd/da6/dca/d126 0 2024-05-06T12:14:47.312 INFO:tasks.workunit.client.1.smithi190.stdout:2/391: write d2/d8/dc/f13 [4446350,24940] 0 2024-05-06T12:14:47.314 INFO:tasks.workunit.client.0.smithi092.stdout:3/752: rename d8/db/d72 to d8/db/d4c/d3d/d49/ded 0 2024-05-06T12:14:47.315 INFO:tasks.workunit.client.1.smithi190.stdout:2/392: fdatasync d2/d8/d33/d46/f4e 0 2024-05-06T12:14:47.316 INFO:tasks.workunit.client.0.smithi092.stdout:2/759: mkdir df/d17/d51/d96/de2 0 2024-05-06T12:14:47.316 INFO:tasks.workunit.client.1.smithi190.stdout:2/393: dread - d2/f62 zero size 2024-05-06T12:14:47.317 INFO:tasks.workunit.client.1.smithi190.stdout:5/424: rename d2/da/d59/d5f/f5e to d2/da/d59/f84 0 2024-05-06T12:14:47.321 INFO:tasks.workunit.client.0.smithi092.stdout:2/760: fsync df/d1b/dcf/f47 0 2024-05-06T12:14:47.322 INFO:tasks.workunit.client.0.smithi092.stdout:8/940: mknod d9/d21/d54/d128/c144 0 2024-05-06T12:14:47.324 INFO:tasks.workunit.client.0.smithi092.stdout:4/985: truncate d9/df/d62/d7b/f112 432633 0 2024-05-06T12:14:47.326 INFO:tasks.workunit.client.1.smithi190.stdout:4/278: sync 2024-05-06T12:14:47.326 INFO:tasks.workunit.client.1.smithi190.stdout:0/511: sync 2024-05-06T12:14:47.326 INFO:tasks.workunit.client.1.smithi190.stdout:6/344: sync 2024-05-06T12:14:47.327 INFO:tasks.workunit.client.0.smithi092.stdout:2/761: rmdir df/d17/d53/da9/dd9/d83/dc2 39 2024-05-06T12:14:47.328 INFO:tasks.workunit.client.1.smithi190.stdout:2/394: creat d2/d8/dc/f81 x:0 0 0 2024-05-06T12:14:47.329 INFO:tasks.workunit.client.1.smithi190.stdout:8/504: write d3/d35/d3a/f3d [5056879,125231] 0 2024-05-06T12:14:47.331 INFO:tasks.workunit.client.0.smithi092.stdout:7/997: dwrite d9/de/d1f/d37/d39/f59 [4194304,4194304] 0 2024-05-06T12:14:47.331 INFO:tasks.workunit.client.1.smithi190.stdout:0/512: symlink d9/dc/d48/lb5 0 2024-05-06T12:14:47.332 INFO:tasks.workunit.client.0.smithi092.stdout:2/762: rename df/d1b to df/dd0/de3 0 2024-05-06T12:14:47.334 INFO:tasks.workunit.client.0.smithi092.stdout:2/763: write df/d15/f95 [713877,125793] 0 2024-05-06T12:14:47.334 INFO:tasks.workunit.client.1.smithi190.stdout:4/279: mkdir d2/d11/d16/d61/d66 0 2024-05-06T12:14:47.336 INFO:tasks.workunit.client.0.smithi092.stdout:3/753: getdents d8/d24/d33/d39 0 2024-05-06T12:14:47.339 INFO:tasks.workunit.client.1.smithi190.stdout:2/395: dwrite d2/f35 [0,4194304] 0 2024-05-06T12:14:47.339 INFO:tasks.workunit.client.1.smithi190.stdout:4/280: dread - d2/d10/f58 zero size 2024-05-06T12:14:47.340 INFO:tasks.workunit.client.1.smithi190.stdout:0/513: mkdir d9/dc/db6 0 2024-05-06T12:14:47.341 INFO:tasks.workunit.client.0.smithi092.stdout:1/899: sync 2024-05-06T12:14:47.341 INFO:tasks.workunit.client.0.smithi092.stdout:7/998: creat d9/d62/d108/d140/f156 x:0 0 0 2024-05-06T12:14:47.343 INFO:tasks.workunit.client.1.smithi190.stdout:4/281: dread - d2/f48 zero size 2024-05-06T12:14:47.344 INFO:tasks.workunit.client.0.smithi092.stdout:2/764: dwrite df/d17/d51/d96/f97 [4194304,4194304] 0 2024-05-06T12:14:47.344 INFO:tasks.workunit.client.1.smithi190.stdout:8/505: dwrite d3/d35/d55/fa5 [0,4194304] 0 2024-05-06T12:14:47.346 INFO:tasks.workunit.client.1.smithi190.stdout:3/488: dwrite d5/d7/d15/d1f/f3b [0,4194304] 0 2024-05-06T12:14:47.346 INFO:tasks.workunit.client.1.smithi190.stdout:4/282: dread - d2/d11/d16/d41/f62 zero size 2024-05-06T12:14:47.347 INFO:tasks.workunit.client.1.smithi190.stdout:2/396: creat d2/d8/d19/d43/d79/f82 x:0 0 0 2024-05-06T12:14:47.348 INFO:tasks.workunit.client.1.smithi190.stdout:0/514: creat d9/dc/db6/fb7 x:0 0 0 2024-05-06T12:14:47.351 INFO:tasks.workunit.client.0.smithi092.stdout:1/900: dwrite d3/f6a [0,4194304] 0 2024-05-06T12:14:47.352 INFO:tasks.workunit.client.1.smithi190.stdout:3/489: dread d5/d7/d15/d1f/f88 [0,4194304] 0 2024-05-06T12:14:47.353 INFO:tasks.workunit.client.0.smithi092.stdout:3/754: rmdir d8/db/d4c/d40/d7a/dbd 39 2024-05-06T12:14:47.353 INFO:tasks.workunit.client.0.smithi092.stdout:4/986: link d9/dd/da6/dca/df2/d10d/c122 d9/d27/da8/db3/d124/d12e/c144 0 2024-05-06T12:14:47.353 INFO:tasks.workunit.client.1.smithi190.stdout:2/397: chown d2/d8/dc/l17 3165665 1 2024-05-06T12:14:47.354 INFO:tasks.workunit.client.1.smithi190.stdout:3/490: rename d5/d7/d15/d1f to d5/d7/d15/d1f/d84/dac 22 2024-05-06T12:14:47.358 INFO:tasks.workunit.client.1.smithi190.stdout:5/425: creat d2/da/d59/d5f/d2e/d3b/d5a/f85 x:0 0 0 2024-05-06T12:14:47.359 INFO:tasks.workunit.client.0.smithi092.stdout:7/999: creat d9/d62/d121/f157 x:0 0 0 2024-05-06T12:14:47.360 INFO:tasks.workunit.client.0.smithi092.stdout:4/987: dread d9/df/d62/d7b/db6/d104/f10a [0,4194304] 0 2024-05-06T12:14:47.361 INFO:tasks.workunit.client.0.smithi092.stdout:2/765: creat df/d17/d51/d96/fe4 x:0 0 0 2024-05-06T12:14:47.362 INFO:tasks.workunit.client.1.smithi190.stdout:8/506: symlink d3/d8/d25/lb4 0 2024-05-06T12:14:47.365 INFO:tasks.workunit.client.0.smithi092.stdout:1/901: unlink d3/d10/d13/d19/d34/d89/lb8 0 2024-05-06T12:14:47.365 INFO:tasks.workunit.client.1.smithi190.stdout:4/283: unlink d2/d10/c27 0 2024-05-06T12:14:47.368 INFO:tasks.workunit.client.1.smithi190.stdout:8/507: stat d3/d8/d25/d3b/f79 0 2024-05-06T12:14:47.370 INFO:tasks.workunit.client.1.smithi190.stdout:5/426: dread - d2/da/d59/d5f/d6e/f7f zero size 2024-05-06T12:14:47.371 INFO:tasks.workunit.client.0.smithi092.stdout:2/766: chown df/d17/f6b 57232775 1 2024-05-06T12:14:47.371 INFO:tasks.workunit.client.1.smithi190.stdout:2/398: dwrite d2/d8/d19/d43/f7c [0,4194304] 0 2024-05-06T12:14:47.373 INFO:tasks.workunit.client.1.smithi190.stdout:3/491: dwrite d5/d7/d15/d39/f9b [0,4194304] 0 2024-05-06T12:14:47.374 INFO:tasks.workunit.client.0.smithi092.stdout:3/755: mkdir d8/db/d4c/d3d/d48/dee 0 2024-05-06T12:14:47.375 INFO:tasks.workunit.client.1.smithi190.stdout:4/284: mkdir d2/d11/d16/d61/d67 0 2024-05-06T12:14:47.376 INFO:tasks.workunit.client.0.smithi092.stdout:2/767: chown df/d17/d51/d96/de2 3946 1 2024-05-06T12:14:47.376 INFO:tasks.workunit.client.0.smithi092.stdout:8/941: dwrite d9/fd3 [0,4194304] 0 2024-05-06T12:14:47.376 INFO:tasks.workunit.client.1.smithi190.stdout:8/508: creat d3/d8/d25/d65/fb5 x:0 0 0 2024-05-06T12:14:47.377 INFO:tasks.workunit.client.1.smithi190.stdout:5/427: mknod d2/da/c86 0 2024-05-06T12:14:47.378 INFO:tasks.workunit.client.1.smithi190.stdout:3/492: symlink d5/d7/d15/d1f/d84/lad 0 2024-05-06T12:14:47.378 INFO:tasks.workunit.client.1.smithi190.stdout:5/428: chown d2/d27/l32 0 1 2024-05-06T12:14:47.382 INFO:tasks.workunit.client.1.smithi190.stdout:3/493: write d5/d7/d15/d1f/d33/d5c/f63 [512934,47836] 0 2024-05-06T12:14:47.384 INFO:tasks.workunit.client.1.smithi190.stdout:4/285: link d2/l49 d2/d11/d16/d61/d66/l68 0 2024-05-06T12:14:47.386 INFO:tasks.workunit.client.0.smithi092.stdout:1/902: dwrite d3/d10/d13/d19/d21/d72/d9a/da4/fba [0,4194304] 0 2024-05-06T12:14:47.386 INFO:tasks.workunit.client.0.smithi092.stdout:2/768: dread - df/d15/d88/f78 zero size 2024-05-06T12:14:47.387 INFO:tasks.workunit.client.0.smithi092.stdout:3/756: rename d8/db/d14/dc7 to d8/d24/d33/da8/def 0 2024-05-06T12:14:47.389 INFO:tasks.workunit.client.1.smithi190.stdout:5/429: dwrite d2/f30 [4194304,4194304] 0 2024-05-06T12:14:47.392 INFO:tasks.workunit.client.0.smithi092.stdout:9/934: sync 2024-05-06T12:14:47.393 INFO:tasks.workunit.client.1.smithi190.stdout:3/494: mknod d5/d14/d11/d21/d44/d80/d82/cae 0 2024-05-06T12:14:47.396 INFO:tasks.workunit.client.1.smithi190.stdout:3/495: write d5/d7/d15/d1f/f96 [101312,18207] 0 2024-05-06T12:14:47.398 INFO:tasks.workunit.client.0.smithi092.stdout:1/903: mkdir d3/d10/d13/d19/d21/d72/d112/d123 0 2024-05-06T12:14:47.399 INFO:tasks.workunit.client.1.smithi190.stdout:3/496: read d5/d14/d11/f13 [790068,2752] 0 2024-05-06T12:14:47.401 INFO:tasks.workunit.client.0.smithi092.stdout:8/942: creat d9/d21/d37/d46/d5c/d71/f145 x:0 0 0 2024-05-06T12:14:47.401 INFO:tasks.workunit.client.1.smithi190.stdout:4/286: dwrite d2/d11/f20 [0,4194304] 0 2024-05-06T12:14:47.402 INFO:tasks.workunit.client.0.smithi092.stdout:3/757: creat d8/db/d14/ff0 x:0 0 0 2024-05-06T12:14:47.402 INFO:tasks.workunit.client.0.smithi092.stdout:4/988: dwrite d9/d27/f7f [0,4194304] 0 2024-05-06T12:14:47.405 INFO:tasks.workunit.client.1.smithi190.stdout:5/430: creat d2/da/d59/d5f/d2e/f87 x:0 0 0 2024-05-06T12:14:47.408 INFO:tasks.workunit.client.0.smithi092.stdout:1/904: creat d3/d10/d13/d19/d21/d88/f124 x:0 0 0 2024-05-06T12:14:47.410 INFO:tasks.workunit.client.1.smithi190.stdout:4/287: symlink d2/d11/d16/d4e/l69 0 2024-05-06T12:14:47.410 INFO:tasks.workunit.client.0.smithi092.stdout:8/943: mkdir d9/d21/d54/d116/d11f/d12e/d146 0 2024-05-06T12:14:47.410 INFO:tasks.workunit.client.0.smithi092.stdout:4/989: creat d9/d2b/d65/d83/f145 x:0 0 0 2024-05-06T12:14:47.412 INFO:tasks.workunit.client.0.smithi092.stdout:3/758: rename d8/d24/c21 to d8/db/d4c/dc1/cf1 0 2024-05-06T12:14:47.413 INFO:tasks.workunit.client.1.smithi190.stdout:4/288: fsync d2/d11/d16/d53/f57 0 2024-05-06T12:14:47.414 INFO:tasks.workunit.client.0.smithi092.stdout:1/905: fsync d3/d42/d84/d4f/d7f/fad 0 2024-05-06T12:14:47.416 INFO:tasks.workunit.client.1.smithi190.stdout:3/497: mknod d5/d14/d11/d21/d44/d92/d9e/da4/caf 0 2024-05-06T12:14:47.417 INFO:tasks.workunit.client.1.smithi190.stdout:8/509: truncate d3/d8/d25/d3b/d90/d6e/f82 2921867 0 2024-05-06T12:14:47.418 INFO:tasks.workunit.client.0.smithi092.stdout:3/759: mkdir d8/d24/d33/d4b/ddf/df2 0 2024-05-06T12:14:47.420 INFO:tasks.workunit.client.1.smithi190.stdout:0/515: dwrite d9/f2c [0,4194304] 0 2024-05-06T12:14:47.420 INFO:tasks.workunit.client.0.smithi092.stdout:1/906: write d3/d10/d13/d19/d21/d88/f124 [41425,27505] 0 2024-05-06T12:14:47.421 INFO:tasks.workunit.client.0.smithi092.stdout:4/990: dwrite d9/dd/f23 [4194304,4194304] 0 2024-05-06T12:14:47.421 INFO:tasks.workunit.client.0.smithi092.stdout:8/944: creat d9/d21/d37/d46/d5c/d62/f147 x:0 0 0 2024-05-06T12:14:47.422 INFO:tasks.workunit.client.1.smithi190.stdout:4/289: symlink d2/d11/d16/d4e/l6a 0 2024-05-06T12:14:47.423 INFO:tasks.workunit.client.0.smithi092.stdout:1/907: readlink d3/d42/d84/d4f/d58/l90 0 2024-05-06T12:14:47.424 INFO:tasks.workunit.client.0.smithi092.stdout:3/760: readlink d8/lb8 0 2024-05-06T12:14:47.425 INFO:tasks.workunit.client.1.smithi190.stdout:3/498: mknod d5/d7/d15/d1f/d84/cb0 0 2024-05-06T12:14:47.426 INFO:tasks.workunit.client.1.smithi190.stdout:5/431: dwrite d2/da/d59/d5f/d2e/d3b/f68 [0,4194304] 0 2024-05-06T12:14:47.428 INFO:tasks.workunit.client.1.smithi190.stdout:4/290: write d2/f48 [303202,71030] 0 2024-05-06T12:14:47.432 INFO:tasks.workunit.client.0.smithi092.stdout:4/991: dwrite d9/d2b/d5b/f98 [0,4194304] 0 2024-05-06T12:14:47.433 INFO:tasks.workunit.client.0.smithi092.stdout:3/761: creat d8/db/d4c/d3d/d48/ff3 x:0 0 0 2024-05-06T12:14:47.434 INFO:tasks.workunit.client.1.smithi190.stdout:4/291: write d2/d10/ff [1968494,75595] 0 2024-05-06T12:14:47.435 INFO:tasks.workunit.client.1.smithi190.stdout:3/499: unlink d5/d7/f1e 0 2024-05-06T12:14:47.438 INFO:tasks.workunit.client.1.smithi190.stdout:4/292: chown d2/d10/f63 300238 1 2024-05-06T12:14:47.438 INFO:tasks.workunit.client.1.smithi190.stdout:8/510: dwrite d3/f71 [0,4194304] 0 2024-05-06T12:14:47.439 INFO:tasks.workunit.client.0.smithi092.stdout:1/908: symlink d3/d42/d84/d4f/d58/d65/d81/d8d/l125 0 2024-05-06T12:14:47.440 INFO:tasks.workunit.client.1.smithi190.stdout:5/432: unlink d2/da/d59/d5f/d2e/d3b/l41 0 2024-05-06T12:14:47.441 INFO:tasks.workunit.client.0.smithi092.stdout:4/992: rename d9/d2b/d65/c108 to d9/dd/d90/c146 0 2024-05-06T12:14:47.444 INFO:tasks.workunit.client.1.smithi190.stdout:0/516: dwrite d9/dc/d48/f4e [0,4194304] 0 2024-05-06T12:14:47.450 INFO:tasks.workunit.client.1.smithi190.stdout:3/500: dread d5/d7/f71 [0,4194304] 0 2024-05-06T12:14:47.455 INFO:tasks.workunit.client.1.smithi190.stdout:4/293: mkdir d2/d11/d16/d61/d66/d6b 0 2024-05-06T12:14:47.455 INFO:tasks.workunit.client.1.smithi190.stdout:3/501: stat d5/d7/d15/c69 0 2024-05-06T12:14:47.458 INFO:tasks.workunit.client.1.smithi190.stdout:3/502: readlink d5/d14/d11/d21/d44/d80/d82/l32 0 2024-05-06T12:14:47.458 INFO:tasks.workunit.client.1.smithi190.stdout:8/511: dread d3/f52 [0,4194304] 0 2024-05-06T12:14:47.459 INFO:tasks.workunit.client.1.smithi190.stdout:8/512: fsync d3/d35/d3a/f91 0 2024-05-06T12:14:47.459 INFO:tasks.workunit.client.0.smithi092.stdout:9/935: write d16/d1e/f28 [4822757,19287] 0 2024-05-06T12:14:47.460 INFO:tasks.workunit.client.1.smithi190.stdout:0/517: link d9/df/l6b d9/dc/d48/d50/d83/lb8 0 2024-05-06T12:14:47.460 INFO:tasks.workunit.client.1.smithi190.stdout:8/513: chown d3/d35/d55/f96 1084635149 1 2024-05-06T12:14:47.462 INFO:tasks.workunit.client.1.smithi190.stdout:3/503: mknod d5/d14/d11/d21/d44/d80/d82/d3d/cb1 0 2024-05-06T12:14:47.462 INFO:tasks.workunit.client.0.smithi092.stdout:9/936: rmdir d16/d1e/da1/d78/db4 39 2024-05-06T12:14:47.463 INFO:tasks.workunit.client.1.smithi190.stdout:0/518: write d9/df/d21/f88 [2730334,5702] 0 2024-05-06T12:14:47.463 INFO:tasks.workunit.client.1.smithi190.stdout:8/514: symlink d3/d8/d25/d3b/d90/lb6 0 2024-05-06T12:14:47.464 INFO:tasks.workunit.client.1.smithi190.stdout:4/294: link d2/d11/d16/d53/c5b d2/d11/d16/d31/c6c 0 2024-05-06T12:14:47.465 INFO:tasks.workunit.client.1.smithi190.stdout:4/295: chown d2/d10/l22 41582 1 2024-05-06T12:14:47.466 INFO:tasks.workunit.client.1.smithi190.stdout:8/515: creat d3/d8/d2b/fb7 x:0 0 0 2024-05-06T12:14:47.468 INFO:tasks.workunit.client.1.smithi190.stdout:8/516: chown d3/d8/d25/d3b/d6b/d7a/fa4 12336879 1 2024-05-06T12:14:47.470 INFO:tasks.workunit.client.1.smithi190.stdout:3/504: dwrite d5/f5e [4194304,4194304] 0 2024-05-06T12:14:47.473 INFO:tasks.workunit.client.1.smithi190.stdout:3/505: symlink d5/d7/d15/d1f/d84/lb2 0 2024-05-06T12:14:47.479 INFO:tasks.workunit.client.1.smithi190.stdout:8/517: dwrite d3/fa3 [0,4194304] 0 2024-05-06T12:14:47.483 INFO:tasks.workunit.client.0.smithi092.stdout:8/945: write d9/d21/d37/dbd/dc5/f13c [119871,61826] 0 2024-05-06T12:14:47.484 INFO:tasks.workunit.client.1.smithi190.stdout:8/518: link d3/d8/c4c d3/d8/d25/d3b/d49/cb8 0 2024-05-06T12:14:47.485 INFO:tasks.workunit.client.1.smithi190.stdout:8/519: chown d3/d8/d25/d3b/d49/l6f 4 1 2024-05-06T12:14:47.486 INFO:tasks.workunit.client.1.smithi190.stdout:8/520: write d3/f71 [4050718,122328] 0 2024-05-06T12:14:47.488 INFO:tasks.workunit.client.0.smithi092.stdout:3/762: truncate d8/d24/d33/d39/d5b/fc4 2963280 0 2024-05-06T12:14:47.489 INFO:tasks.workunit.client.1.smithi190.stdout:8/521: mkdir d3/d35/d3a/db9 0 2024-05-06T12:14:47.492 INFO:tasks.workunit.client.1.smithi190.stdout:8/522: creat d3/d35/d3a/fba x:0 0 0 2024-05-06T12:14:47.495 INFO:tasks.workunit.client.0.smithi092.stdout:4/993: dwrite d9/d2b/f77 [0,4194304] 0 2024-05-06T12:14:47.497 INFO:tasks.workunit.client.1.smithi190.stdout:8/523: dread d3/fa3 [0,4194304] 0 2024-05-06T12:14:47.498 INFO:tasks.workunit.client.0.smithi092.stdout:4/994: creat d9/d27/da8/db3/d124/d12e/f147 x:0 0 0 2024-05-06T12:14:47.499 INFO:tasks.workunit.client.0.smithi092.stdout:4/995: mkdir d9/d2b/d65/d127/d148 0 2024-05-06T12:14:47.501 INFO:tasks.workunit.client.0.smithi092.stdout:9/937: dread d16/d1c/d58/d92/dab/deb/d113/f108 [0,4194304] 0 2024-05-06T12:14:47.502 INFO:tasks.workunit.client.1.smithi190.stdout:5/433: dwrite d2/f5 [0,4194304] 0 2024-05-06T12:14:47.503 INFO:tasks.workunit.client.0.smithi092.stdout:9/938: chown d16/d2c/d2f/d3d/d33/d6b/f101 3745641 1 2024-05-06T12:14:47.503 INFO:tasks.workunit.client.1.smithi190.stdout:4/296: write d2/d11/d16/d4e/f5a [1970306,118057] 0 2024-05-06T12:14:47.504 INFO:tasks.workunit.client.0.smithi092.stdout:3/763: write d8/d24/d33/d4b/f46 [6964238,13685] 0 2024-05-06T12:14:47.505 INFO:tasks.workunit.client.1.smithi190.stdout:5/434: dread - d2/da/d59/d5f/d6e/f7f zero size 2024-05-06T12:14:47.505 INFO:tasks.workunit.client.1.smithi190.stdout:8/524: creat d3/d8/fbb x:0 0 0 2024-05-06T12:14:47.507 INFO:tasks.workunit.client.0.smithi092.stdout:9/939: mknod d16/d1c/d58/d92/d115/c12c 0 2024-05-06T12:14:47.510 INFO:tasks.workunit.client.0.smithi092.stdout:9/940: write d16/d2c/d2f/d3d/ffd [964143,125265] 0 2024-05-06T12:14:47.510 INFO:tasks.workunit.client.0.smithi092.stdout:8/946: dwrite d9/d21/d37/d46/d5c/d62/f6e [0,4194304] 0 2024-05-06T12:14:47.511 INFO:tasks.workunit.client.1.smithi190.stdout:8/525: dread d3/fa3 [0,4194304] 0 2024-05-06T12:14:47.511 INFO:tasks.workunit.client.0.smithi092.stdout:3/764: creat d8/db/d4c/d40/d7a/dbd/ff4 x:0 0 0 2024-05-06T12:14:47.512 INFO:tasks.workunit.client.0.smithi092.stdout:9/941: chown d16/d1c/dd4/dd9 32 1 2024-05-06T12:14:47.512 INFO:tasks.workunit.client.0.smithi092.stdout:8/947: read d9/d21/d54/f7b [3204376,37848] 0 2024-05-06T12:14:47.512 INFO:tasks.workunit.client.1.smithi190.stdout:8/526: truncate d3/d8/d2b/fb7 568422 0 2024-05-06T12:14:47.513 INFO:tasks.workunit.client.1.smithi190.stdout:5/435: mkdir d2/da/d59/d5f/d50/d88 0 2024-05-06T12:14:47.514 INFO:tasks.workunit.client.0.smithi092.stdout:3/765: chown d8/db/d4c/d3d/d49/f4f 29 1 2024-05-06T12:14:47.515 INFO:tasks.workunit.client.1.smithi190.stdout:4/297: fdatasync d2/f37 0 2024-05-06T12:14:47.516 INFO:tasks.workunit.client.1.smithi190.stdout:7/344: sync 2024-05-06T12:14:47.517 INFO:tasks.workunit.client.1.smithi190.stdout:8/527: chown d3/f2d 0 1 2024-05-06T12:14:47.519 INFO:tasks.workunit.client.0.smithi092.stdout:4/996: write d9/f86 [471395,100421] 0 2024-05-06T12:14:47.520 INFO:tasks.workunit.client.0.smithi092.stdout:8/948: symlink d9/l148 0 2024-05-06T12:14:47.522 INFO:tasks.workunit.client.0.smithi092.stdout:9/942: rmdir d16/d1e/d7b/d104 0 2024-05-06T12:14:47.522 INFO:tasks.workunit.client.1.smithi190.stdout:5/436: getdents d2/da/d59/d5f/d2e 0 2024-05-06T12:14:47.523 INFO:tasks.workunit.client.0.smithi092.stdout:4/997: symlink d9/d2b/d5b/l149 0 2024-05-06T12:14:47.524 INFO:tasks.workunit.client.1.smithi190.stdout:4/298: dwrite d2/d10/f58 [0,4194304] 0 2024-05-06T12:14:47.526 INFO:tasks.workunit.client.0.smithi092.stdout:4/998: creat d9/df/d62/d7b/d107/f14a x:0 0 0 2024-05-06T12:14:47.527 INFO:tasks.workunit.client.1.smithi190.stdout:8/528: rmdir d3/d8/d25/d3b/d6b 39 2024-05-06T12:14:47.529 INFO:tasks.workunit.client.0.smithi092.stdout:8/949: dwrite d9/dc/f12d [0,4194304] 0 2024-05-06T12:14:47.529 INFO:tasks.workunit.client.1.smithi190.stdout:5/437: symlink d2/d27/l89 0 2024-05-06T12:14:47.530 INFO:tasks.workunit.client.1.smithi190.stdout:7/345: link d0/da/f7e d0/d1/d18/d2b/f83 0 2024-05-06T12:14:47.531 INFO:tasks.workunit.client.1.smithi190.stdout:8/529: fdatasync d3/fb1 0 2024-05-06T12:14:47.531 INFO:tasks.workunit.client.0.smithi092.stdout:9/943: rename d16/d1e/d7b/dce/de4/cf0 to d16/d1c/d58/c12d 0 2024-05-06T12:14:47.532 INFO:tasks.workunit.client.1.smithi190.stdout:5/438: mknod d2/da/d59/d5f/d2e/d3b/d5a/c8a 0 2024-05-06T12:14:47.534 INFO:tasks.workunit.client.0.smithi092.stdout:4/999: mkdir d9/dd/da6/dca/df2/d14b 0 2024-05-06T12:14:47.534 INFO:tasks.workunit.client.1.smithi190.stdout:5/439: write d2/da/d59/f77 [45856,102389] 0 2024-05-06T12:14:47.535 INFO:tasks.workunit.client.0.smithi092.stdout:8/950: stat d9/d21/d72/lbf 0 2024-05-06T12:14:47.536 INFO:tasks.workunit.client.0.smithi092.stdout:3/766: write d8/db/d4c/d3d/fa9 [1013374,119899] 0 2024-05-06T12:14:47.537 INFO:tasks.workunit.client.1.smithi190.stdout:5/440: chown d2/da/l34 99944 1 2024-05-06T12:14:47.540 INFO:tasks.workunit.client.1.smithi190.stdout:5/441: write d2/da/d59/d5f/d2e/d3b/f42 [1883654,37866] 0 2024-05-06T12:14:47.540 INFO:tasks.workunit.client.1.smithi190.stdout:8/530: dread d3/d8/d25/d3b/d90/d6e/f82 [0,4194304] 0 2024-05-06T12:14:47.541 INFO:tasks.workunit.client.0.smithi092.stdout:9/944: creat d16/f12e x:0 0 0 2024-05-06T12:14:47.542 INFO:tasks.workunit.client.0.smithi092.stdout:3/767: rename d8/db/d4c/d3d/f96 to d8/d24/ff5 0 2024-05-06T12:14:47.543 INFO:tasks.workunit.client.0.smithi092.stdout:3/768: write d8/f12 [1191992,111210] 0 2024-05-06T12:14:47.545 INFO:tasks.workunit.client.0.smithi092.stdout:9/945: mkdir d16/d1c/d12f 0 2024-05-06T12:14:47.545 INFO:tasks.workunit.client.1.smithi190.stdout:8/531: rename d3/d35/d3a/fa7 to d3/d8/d68/d9f/fbc 0 2024-05-06T12:14:47.546 INFO:tasks.workunit.client.0.smithi092.stdout:9/946: chown d16/d1c/d58/le2 106619 1 2024-05-06T12:14:47.548 INFO:tasks.workunit.client.1.smithi190.stdout:8/532: mknod d3/d8/d68/cbd 0 2024-05-06T12:14:47.550 INFO:tasks.workunit.client.1.smithi190.stdout:5/442: dwrite d2/da/d59/d5f/f3c [0,4194304] 0 2024-05-06T12:14:47.550 INFO:tasks.workunit.client.1.smithi190.stdout:8/533: chown d3/d35/d55/d61/c62 48484 1 2024-05-06T12:14:47.551 INFO:tasks.workunit.client.0.smithi092.stdout:9/947: rmdir d16/d2c/d2f/d3d/d33/d57 39 2024-05-06T12:14:47.554 INFO:tasks.workunit.client.0.smithi092.stdout:9/948: mkdir d16/d2c/d2f/d3d/d33/d6b/d130 0 2024-05-06T12:14:47.554 INFO:tasks.workunit.client.0.smithi092.stdout:8/951: link d9/d21/d54/d79/d8a/fb8 d9/d21/d37/d46/d5c/d62/d7a/f149 0 2024-05-06T12:14:47.555 INFO:tasks.workunit.client.0.smithi092.stdout:3/769: link d8/d1b/d73/cc8 d8/dd7/cf6 0 2024-05-06T12:14:47.557 INFO:tasks.workunit.client.0.smithi092.stdout:9/949: unlink d16/d1c/d58/ff3 0 2024-05-06T12:14:47.557 INFO:tasks.workunit.client.0.smithi092.stdout:3/770: chown d8/d24/d33/d39/la1 14179140 1 2024-05-06T12:14:47.559 INFO:tasks.workunit.client.1.smithi190.stdout:4/299: dread d2/d11/d16/f1a [0,4194304] 0 2024-05-06T12:14:47.560 INFO:tasks.workunit.client.0.smithi092.stdout:3/771: truncate d8/d24/d27/f2f 4295113 0 2024-05-06T12:14:47.560 INFO:tasks.workunit.client.0.smithi092.stdout:3/772: read d8/db/f6c [227324,71200] 0 2024-05-06T12:14:47.563 INFO:tasks.workunit.client.1.smithi190.stdout:7/346: dread d0/d1/d2/f41 [0,4194304] 0 2024-05-06T12:14:47.563 INFO:tasks.workunit.client.1.smithi190.stdout:5/443: dwrite d2/da/d59/d5f/d2e/f87 [0,4194304] 0 2024-05-06T12:14:47.565 INFO:tasks.workunit.client.0.smithi092.stdout:9/950: mkdir d16/d1e/d7b/dce/de4/d131 0 2024-05-06T12:14:47.568 INFO:tasks.workunit.client.1.smithi190.stdout:4/300: dread d2/d11/f20 [0,4194304] 0 2024-05-06T12:14:47.570 INFO:tasks.workunit.client.1.smithi190.stdout:4/301: write d2/d10/f1c [3601471,82303] 0 2024-05-06T12:14:47.570 INFO:tasks.workunit.client.0.smithi092.stdout:9/951: dread d16/d1e/f70 [0,4194304] 0 2024-05-06T12:14:47.570 INFO:tasks.workunit.client.1.smithi190.stdout:5/444: rename d2/da/d59/d5f/d50/d88 to d2/da/d59/d5f/d2e/d8b 0 2024-05-06T12:14:47.573 INFO:tasks.workunit.client.1.smithi190.stdout:5/445: read d2/da/d59/d5f/d50/f51 [53897,10254] 0 2024-05-06T12:14:47.575 INFO:tasks.workunit.client.1.smithi190.stdout:5/446: dread - d2/da/d59/d5f/d2e/d3b/d5a/f85 zero size 2024-05-06T12:14:47.576 INFO:tasks.workunit.client.1.smithi190.stdout:8/534: dread d3/d8/d2b/f34 [0,4194304] 0 2024-05-06T12:14:47.577 INFO:tasks.workunit.client.1.smithi190.stdout:4/302: unlink d2/d11/l45 0 2024-05-06T12:14:47.579 INFO:tasks.workunit.client.1.smithi190.stdout:5/447: fsync d2/da/f80 0 2024-05-06T12:14:47.580 INFO:tasks.workunit.client.1.smithi190.stdout:7/347: write d0/d1/f3e [173513,877] 0 2024-05-06T12:14:47.581 INFO:tasks.workunit.client.0.smithi092.stdout:9/952: symlink d16/d2c/d2f/d3d/d33/d57/l132 0 2024-05-06T12:14:47.581 INFO:tasks.workunit.client.0.smithi092.stdout:3/773: link d8/db/d4c/dc1/lbc d8/d1b/d73/db7/lf7 0 2024-05-06T12:14:47.582 INFO:tasks.workunit.client.1.smithi190.stdout:7/348: readlink d0/d1/d18/d37/l7a 0 2024-05-06T12:14:47.582 INFO:tasks.workunit.client.1.smithi190.stdout:8/535: creat d3/d8/d2b/d3e/dad/fbe x:0 0 0 2024-05-06T12:14:47.583 INFO:tasks.workunit.client.0.smithi092.stdout:9/953: chown d16/d2c/d3f/f46 3930 1 2024-05-06T12:14:47.583 INFO:tasks.workunit.client.1.smithi190.stdout:5/448: mknod d2/d27/c8c 0 2024-05-06T12:14:47.584 INFO:tasks.workunit.client.0.smithi092.stdout:8/952: dwrite d9/d21/d37/d46/f132 [0,4194304] 0 2024-05-06T12:14:47.590 INFO:tasks.workunit.client.1.smithi190.stdout:8/536: stat d3/d8/d25/d65/f93 0 2024-05-06T12:14:47.590 INFO:tasks.workunit.client.1.smithi190.stdout:4/303: dwrite d2/d10/f1c [0,4194304] 0 2024-05-06T12:14:47.590 INFO:tasks.workunit.client.1.smithi190.stdout:7/349: truncate d0/da/f4a 1315724 0 2024-05-06T12:14:47.591 INFO:tasks.workunit.client.0.smithi092.stdout:3/774: rename d8/d24/d33/d39 to d8/db/d4c/d3d/d49/df8 0 2024-05-06T12:14:47.591 INFO:tasks.workunit.client.0.smithi092.stdout:9/954: rename d16/d2c/d2f/d3d/d33/d37/f72 to d16/d94/da0/da7/f133 0 2024-05-06T12:14:47.595 INFO:tasks.workunit.client.0.smithi092.stdout:9/955: getdents d16/d1e/d7b/d8d/da4 0 2024-05-06T12:14:47.596 INFO:tasks.workunit.client.0.smithi092.stdout:3/775: dread d8/db/d4c/d3d/fa9 [0,4194304] 0 2024-05-06T12:14:47.596 INFO:tasks.workunit.client.1.smithi190.stdout:8/537: write d3/d35/fa0 [736265,11366] 0 2024-05-06T12:14:47.596 INFO:tasks.workunit.client.1.smithi190.stdout:5/449: dwrite d2/da/d59/d5f/f3c [0,4194304] 0 2024-05-06T12:14:47.597 INFO:tasks.workunit.client.0.smithi092.stdout:9/956: write d16/d1c/dd4/f102 [487871,57817] 0 2024-05-06T12:14:47.599 INFO:tasks.workunit.client.1.smithi190.stdout:4/304: write d2/d11/f14 [1869109,97619] 0 2024-05-06T12:14:47.599 INFO:tasks.workunit.client.1.smithi190.stdout:7/350: mkdir d0/d1/d2/d84 0 2024-05-06T12:14:47.600 INFO:tasks.workunit.client.0.smithi092.stdout:9/957: getdents d16/d1e/d7b/d8d 0 2024-05-06T12:14:47.600 INFO:tasks.workunit.client.0.smithi092.stdout:2/769: sync 2024-05-06T12:14:47.600 INFO:tasks.workunit.client.0.smithi092.stdout:1/909: sync 2024-05-06T12:14:47.601 INFO:tasks.workunit.client.1.smithi190.stdout:7/351: chown d0/d1/f3e 4 1 2024-05-06T12:14:47.601 INFO:tasks.workunit.client.0.smithi092.stdout:9/958: readlink d16/d1e/da1/lb3 0 2024-05-06T12:14:47.602 INFO:tasks.workunit.client.0.smithi092.stdout:1/910: dread - d3/d10/d13/d19/d34/ff5 zero size 2024-05-06T12:14:47.603 INFO:tasks.workunit.client.1.smithi190.stdout:5/450: creat d2/da/f8d x:0 0 0 2024-05-06T12:14:47.605 INFO:tasks.workunit.client.0.smithi092.stdout:2/770: mknod df/dd0/de3/db4/ce5 0 2024-05-06T12:14:47.606 INFO:tasks.workunit.client.0.smithi092.stdout:1/911: chown d3/d10/d13/cc2 118450 1 2024-05-06T12:14:47.608 INFO:tasks.workunit.client.0.smithi092.stdout:1/912: fsync d3/d10/d13/d19/d34/fbe 0 2024-05-06T12:14:47.609 INFO:tasks.workunit.client.0.smithi092.stdout:2/771: symlink df/d15/le6 0 2024-05-06T12:14:47.610 INFO:tasks.workunit.client.0.smithi092.stdout:1/913: chown d3/d42/d84/dcf/le0 2308695 1 2024-05-06T12:14:47.610 INFO:tasks.workunit.client.1.smithi190.stdout:4/305: rmdir d2/d11/d16/d31 39 2024-05-06T12:14:47.610 INFO:tasks.workunit.client.1.smithi190.stdout:7/352: unlink d0/da/d3d/f63 0 2024-05-06T12:14:47.611 INFO:tasks.workunit.client.0.smithi092.stdout:2/772: read df/dd0/de3/fbd [3667892,2286] 0 2024-05-06T12:14:47.611 INFO:tasks.workunit.client.0.smithi092.stdout:1/914: fdatasync d3/d10/d13/d19/d21/d72/d9a/da4/fba 0 2024-05-06T12:14:47.613 INFO:tasks.workunit.client.1.smithi190.stdout:5/451: symlink d2/d65/l8e 0 2024-05-06T12:14:47.613 INFO:tasks.workunit.client.0.smithi092.stdout:2/773: read - df/d17/d53/d85/fba zero size 2024-05-06T12:14:47.615 INFO:tasks.workunit.client.0.smithi092.stdout:1/915: mknod d3/d10/d13/d19/d21/c126 0 2024-05-06T12:14:47.616 INFO:tasks.workunit.client.0.smithi092.stdout:1/916: chown d3/d10/d13/f1f 305237 1 2024-05-06T12:14:47.618 INFO:tasks.workunit.client.0.smithi092.stdout:2/774: creat df/d17/d1d/dc8/fe7 x:0 0 0 2024-05-06T12:14:47.618 INFO:tasks.workunit.client.1.smithi190.stdout:7/353: creat d0/da/d36/d42/f85 x:0 0 0 2024-05-06T12:14:47.619 INFO:tasks.workunit.client.1.smithi190.stdout:7/354: rename d0/d1/d32 to d0/d1/d32/d86 22 2024-05-06T12:14:47.620 INFO:tasks.workunit.client.1.smithi190.stdout:7/355: chown d0/c34 0 1 2024-05-06T12:14:47.623 INFO:tasks.workunit.client.0.smithi092.stdout:1/917: getdents d3/d42/d84/d4f/d58/d65 0 2024-05-06T12:14:47.623 INFO:tasks.workunit.client.1.smithi190.stdout:8/538: getdents d3/d35 0 2024-05-06T12:14:47.625 INFO:tasks.workunit.client.0.smithi092.stdout:9/959: dwrite fc [4194304,4194304] 0 2024-05-06T12:14:47.627 INFO:tasks.workunit.client.1.smithi190.stdout:8/539: mknod d3/d8/d68/d9f/cbf 0 2024-05-06T12:14:47.628 INFO:tasks.workunit.client.0.smithi092.stdout:2/775: dwrite df/d15/d88/fa7 [0,4194304] 0 2024-05-06T12:14:47.629 INFO:tasks.workunit.client.1.smithi190.stdout:4/306: getdents d2/d11 0 2024-05-06T12:14:47.629 INFO:tasks.workunit.client.1.smithi190.stdout:5/452: dread d2/da/d59/d5f/d50/f51 [0,4194304] 0 2024-05-06T12:14:47.631 INFO:tasks.workunit.client.0.smithi092.stdout:9/960: symlink d16/d2c/d4b/d76/l134 0 2024-05-06T12:14:47.632 INFO:tasks.workunit.client.0.smithi092.stdout:3/776: dread d8/db/d14/d5a/f8f [0,4194304] 0 2024-05-06T12:14:47.633 INFO:tasks.workunit.client.1.smithi190.stdout:4/307: symlink d2/d11/d16/d4e/l6d 0 2024-05-06T12:14:47.633 INFO:tasks.workunit.client.0.smithi092.stdout:1/918: truncate d3/d42/d84/d4f/d58/f78 703421 0 2024-05-06T12:14:47.634 INFO:tasks.workunit.client.0.smithi092.stdout:3/777: dread d8/db/d4c/d40/d67/fe9 [0,4194304] 0 2024-05-06T12:14:47.635 INFO:tasks.workunit.client.0.smithi092.stdout:3/778: dread - d8/db/d14/fd4 zero size 2024-05-06T12:14:47.636 INFO:tasks.workunit.client.1.smithi190.stdout:8/540: write d3/d8/d25/d3b/d90/d6e/f82 [1251412,88525] 0 2024-05-06T12:14:47.636 INFO:tasks.workunit.client.0.smithi092.stdout:9/961: creat d16/d1e/d7b/d8d/da4/f135 x:0 0 0 2024-05-06T12:14:47.638 INFO:tasks.workunit.client.0.smithi092.stdout:3/779: fdatasync d8/d24/d33/d4b/fc5 0 2024-05-06T12:14:47.639 INFO:tasks.workunit.client.0.smithi092.stdout:9/962: chown d16/d1c/d58/d92/dab 0 1 2024-05-06T12:14:47.639 INFO:tasks.workunit.client.0.smithi092.stdout:2/776: unlink df/d17/d1d/f41 0 2024-05-06T12:14:47.639 INFO:tasks.workunit.client.1.smithi190.stdout:4/308: dread d2/d10/f1c [0,4194304] 0 2024-05-06T12:14:47.643 INFO:tasks.workunit.client.0.smithi092.stdout:3/780: creat d8/d24/d33/d4b/ff9 x:0 0 0 2024-05-06T12:14:47.643 INFO:tasks.workunit.client.1.smithi190.stdout:5/453: link d2/da/d59/d5f/d2e/l46 d2/l8f 0 2024-05-06T12:14:47.643 INFO:tasks.workunit.client.0.smithi092.stdout:1/919: dwrite d3/d10/d13/d19/d34/d89/ff4 [0,4194304] 0 2024-05-06T12:14:47.644 INFO:tasks.workunit.client.1.smithi190.stdout:4/309: mkdir d2/d11/d16/d61/d67/d6e 0 2024-05-06T12:14:47.646 INFO:tasks.workunit.client.1.smithi190.stdout:8/541: dread d3/d8/f27 [0,4194304] 0 2024-05-06T12:14:47.647 INFO:tasks.workunit.client.0.smithi092.stdout:3/781: symlink d8/d24/d33/d4b/ddf/lfa 0 2024-05-06T12:14:47.648 INFO:tasks.workunit.client.1.smithi190.stdout:4/310: creat d2/d11/d16/d4e/f6f x:0 0 0 2024-05-06T12:14:47.648 INFO:tasks.workunit.client.1.smithi190.stdout:8/542: chown d3/d8/d25/f39 14 1 2024-05-06T12:14:47.651 INFO:tasks.workunit.client.1.smithi190.stdout:8/543: mknod d3/d8/d25/d3b/cc0 0 2024-05-06T12:14:47.651 INFO:tasks.workunit.client.0.smithi092.stdout:1/920: write d3/d42/d84/d4f/dd5/ded/fff [981409,92736] 0 2024-05-06T12:14:47.652 INFO:tasks.workunit.client.0.smithi092.stdout:9/963: dwrite d16/d1e/d7b/dce/f9b [4194304,4194304] 0 2024-05-06T12:14:47.653 INFO:tasks.workunit.client.1.smithi190.stdout:8/544: truncate d3/d8/d25/f39 170779 0 2024-05-06T12:14:47.653 INFO:tasks.workunit.client.0.smithi092.stdout:1/921: readlink d3/d42/d84/d4f/l5c 0 2024-05-06T12:14:47.655 INFO:tasks.workunit.client.0.smithi092.stdout:1/922: stat d3/d10/d13/d19/d34/cee 0 2024-05-06T12:14:47.655 INFO:tasks.workunit.client.0.smithi092.stdout:3/782: mknod d8/db/d14/d5a/cfb 0 2024-05-06T12:14:47.659 INFO:tasks.workunit.client.0.smithi092.stdout:3/783: symlink d8/db/d14/d5a/lfc 0 2024-05-06T12:14:47.661 INFO:tasks.workunit.client.1.smithi190.stdout:4/311: dread d2/d11/d16/f38 [0,4194304] 0 2024-05-06T12:14:47.662 INFO:tasks.workunit.client.1.smithi190.stdout:4/312: stat d2/d11/d16/d53 0 2024-05-06T12:14:47.662 INFO:tasks.workunit.client.1.smithi190.stdout:5/454: dwrite d2/da/d59/d5f/d50/f51 [0,4194304] 0 2024-05-06T12:14:47.663 INFO:tasks.workunit.client.0.smithi092.stdout:1/923: rename d3/d10/d13/d19/d21/l28 to d3/d42/d84/d4f/dd5/l127 0 2024-05-06T12:14:47.663 INFO:tasks.workunit.client.1.smithi190.stdout:8/545: dread d3/d8/f30 [0,4194304] 0 2024-05-06T12:14:47.664 INFO:tasks.workunit.client.0.smithi092.stdout:3/784: write d8/db/d4c/d3d/d49/df8/d5b/f6d [1510604,81610] 0 2024-05-06T12:14:47.664 INFO:tasks.workunit.client.0.smithi092.stdout:1/924: chown d3/d10/c49 61428838 1 2024-05-06T12:14:47.667 INFO:tasks.workunit.client.0.smithi092.stdout:1/925: symlink d3/d10/d13/d19/d57/dd6/l128 0 2024-05-06T12:14:47.669 INFO:tasks.workunit.client.1.smithi190.stdout:8/546: rmdir d3/d35/d55/d7f 0 2024-05-06T12:14:47.670 INFO:tasks.workunit.client.0.smithi092.stdout:3/785: creat d8/db/ffd x:0 0 0 2024-05-06T12:14:47.670 INFO:tasks.workunit.client.1.smithi190.stdout:5/455: rmdir d2/da/d59/d81 0 2024-05-06T12:14:47.671 INFO:tasks.workunit.client.1.smithi190.stdout:4/313: dwrite d2/f48 [0,4194304] 0 2024-05-06T12:14:47.676 INFO:tasks.workunit.client.0.smithi092.stdout:3/786: creat d8/d24/d27/ffe x:0 0 0 2024-05-06T12:14:47.677 INFO:tasks.workunit.client.0.smithi092.stdout:2/777: dread df/dd0/de3/fbd [0,4194304] 0 2024-05-06T12:14:47.678 INFO:tasks.workunit.client.1.smithi190.stdout:5/456: dwrite d2/da/d59/f6a [0,4194304] 0 2024-05-06T12:14:47.678 INFO:tasks.workunit.client.1.smithi190.stdout:5/457: dread - d2/da/f8d zero size 2024-05-06T12:14:47.680 INFO:tasks.workunit.client.0.smithi092.stdout:2/778: write df/d17/d1d/f4f [4087965,115384] 0 2024-05-06T12:14:47.681 INFO:tasks.workunit.client.0.smithi092.stdout:1/926: link d3/d10/l11c d3/d42/d84/dcf/l129 0 2024-05-06T12:14:47.682 INFO:tasks.workunit.client.0.smithi092.stdout:9/964: write d16/d1e/d7b/d8d/f111 [5236342,66692] 0 2024-05-06T12:14:47.683 INFO:tasks.workunit.client.1.smithi190.stdout:5/458: fsync d2/da/f80 0 2024-05-06T12:14:47.687 INFO:tasks.workunit.client.0.smithi092.stdout:9/965: creat d16/d1c/de5/f136 x:0 0 0 2024-05-06T12:14:47.689 INFO:tasks.workunit.client.1.smithi190.stdout:8/547: truncate d3/f52 3226143 0 2024-05-06T12:14:47.690 INFO:tasks.workunit.client.0.smithi092.stdout:2/779: dwrite df/d17/d53/da9/dd9/d83/f9d [0,4194304] 0 2024-05-06T12:14:47.694 INFO:tasks.workunit.client.1.smithi190.stdout:4/314: dwrite d2/f37 [0,4194304] 0 2024-05-06T12:14:47.694 INFO:tasks.workunit.client.0.smithi092.stdout:2/780: truncate df/d17/d53/da9/dd9/d83/f84 941770 0 2024-05-06T12:14:47.694 INFO:tasks.workunit.client.1.smithi190.stdout:5/459: dwrite d2/da/d59/d5f/d2e/d3b/f47 [4194304,4194304] 0 2024-05-06T12:14:47.696 INFO:tasks.workunit.client.0.smithi092.stdout:2/781: truncate df/d5c/f66 3106902 0 2024-05-06T12:14:47.699 INFO:tasks.workunit.client.0.smithi092.stdout:2/782: symlink df/d17/d51/le8 0 2024-05-06T12:14:47.702 INFO:tasks.workunit.client.1.smithi190.stdout:8/548: truncate d3/fb1 814537 0 2024-05-06T12:14:47.704 INFO:tasks.workunit.client.1.smithi190.stdout:4/315: getdents d2/d11/d16/d61/d67/d6e 0 2024-05-06T12:14:47.705 INFO:tasks.workunit.client.1.smithi190.stdout:8/549: fdatasync d3/f17 0 2024-05-06T12:14:47.707 INFO:tasks.workunit.client.1.smithi190.stdout:4/316: fsync d2/f5e 0 2024-05-06T12:14:47.709 INFO:tasks.workunit.client.0.smithi092.stdout:2/783: dwrite df/d15/d88/fa7 [0,4194304] 0 2024-05-06T12:14:47.711 INFO:tasks.workunit.client.1.smithi190.stdout:4/317: mknod d2/d11/d16/d61/d67/d6e/c70 0 2024-05-06T12:14:47.713 INFO:tasks.workunit.client.0.smithi092.stdout:3/787: write d8/db/d4c/d40/f52 [2174897,62872] 0 2024-05-06T12:14:47.721 INFO:tasks.workunit.client.0.smithi092.stdout:2/784: dwrite df/d15/d88/d59/fb7 [0,4194304] 0 2024-05-06T12:14:47.722 INFO:tasks.workunit.client.0.smithi092.stdout:1/927: dwrite d3/d10/d13/d19/d21/f4d [4194304,4194304] 0 2024-05-06T12:14:47.728 INFO:tasks.workunit.client.0.smithi092.stdout:3/788: rename d8/d24/d33/da8/fd3 to d8/db/d4c/d40/fff 0 2024-05-06T12:14:47.732 INFO:tasks.workunit.client.0.smithi092.stdout:2/785: mkdir df/d17/d51/de9 0 2024-05-06T12:14:47.732 INFO:tasks.workunit.client.0.smithi092.stdout:9/966: dwrite d16/d2c/d4b/fae [0,4194304] 0 2024-05-06T12:14:47.744 INFO:tasks.workunit.client.1.smithi190.stdout:5/460: truncate d2/f8 1897094 0 2024-05-06T12:14:47.744 INFO:tasks.workunit.client.1.smithi190.stdout:5/461: readlink d2/d65/l8e 0 2024-05-06T12:14:47.746 INFO:tasks.workunit.client.1.smithi190.stdout:5/462: symlink d2/d6c/l90 0 2024-05-06T12:14:47.748 INFO:tasks.workunit.client.1.smithi190.stdout:5/463: creat d2/da/d59/d5f/d2e/d3b/d5a/f91 x:0 0 0 2024-05-06T12:14:47.749 INFO:tasks.workunit.client.1.smithi190.stdout:5/464: fdatasync d2/da/d59/d5f/f5b 0 2024-05-06T12:14:47.751 INFO:tasks.workunit.client.1.smithi190.stdout:5/465: mknod d2/da/c92 0 2024-05-06T12:14:47.752 INFO:tasks.workunit.client.1.smithi190.stdout:5/466: mknod d2/da/d59/c93 0 2024-05-06T12:14:47.753 INFO:tasks.workunit.client.1.smithi190.stdout:5/467: write d2/da/d59/d5f/d2e/d3b/f68 [5745720,17650] 0 2024-05-06T12:14:47.754 INFO:tasks.workunit.client.1.smithi190.stdout:5/468: chown d2/da/f8d 878941622 1 2024-05-06T12:14:47.755 INFO:tasks.workunit.client.1.smithi190.stdout:5/469: write d2/da/d59/d5f/f36 [689242,63605] 0 2024-05-06T12:14:47.758 INFO:tasks.workunit.client.1.smithi190.stdout:5/470: creat d2/da/d59/f94 x:0 0 0 2024-05-06T12:14:47.758 INFO:tasks.workunit.client.0.smithi092.stdout:2/786: fsync df/d5c/f66 0 2024-05-06T12:14:47.763 INFO:tasks.workunit.client.1.smithi190.stdout:5/471: dwrite d2/f5 [0,4194304] 0 2024-05-06T12:14:47.766 INFO:tasks.workunit.client.1.smithi190.stdout:5/472: mkdir d2/da/d59/d5f/d6e/d95 0 2024-05-06T12:14:47.768 INFO:tasks.workunit.client.1.smithi190.stdout:5/473: stat d2/f44 0 2024-05-06T12:14:47.770 INFO:tasks.workunit.client.1.smithi190.stdout:9/538: sync 2024-05-06T12:14:47.770 INFO:tasks.workunit.client.1.smithi190.stdout:5/474: write d2/da/d59/d5f/d2e/f87 [4601652,44284] 0 2024-05-06T12:14:47.774 INFO:tasks.workunit.client.1.smithi190.stdout:4/318: dwrite d2/f32 [4194304,4194304] 0 2024-05-06T12:14:47.774 INFO:tasks.workunit.client.1.smithi190.stdout:8/550: truncate d3/f52 2848310 0 2024-05-06T12:14:47.774 INFO:tasks.workunit.client.0.smithi092.stdout:1/928: write d3/d10/d13/d46/f92 [1395048,60150] 0 2024-05-06T12:14:47.775 INFO:tasks.workunit.client.1.smithi190.stdout:4/319: chown d2/c4a 55645 1 2024-05-06T12:14:47.775 INFO:tasks.workunit.client.0.smithi092.stdout:3/789: truncate d8/db/d14/d5a/f94 4158855 0 2024-05-06T12:14:47.780 INFO:tasks.workunit.client.1.smithi190.stdout:8/551: fsync d3/d8/d25/d3b/d6b/f99 0 2024-05-06T12:14:47.781 INFO:tasks.workunit.client.1.smithi190.stdout:8/552: write d3/d8/d68/faf [418825,11968] 0 2024-05-06T12:14:47.782 INFO:tasks.workunit.client.1.smithi190.stdout:5/475: getdents d2/d6c/d7d 0 2024-05-06T12:14:47.783 INFO:tasks.workunit.client.0.smithi092.stdout:9/967: dwrite d16/d2c/d2f/d3d/d33/d7e/dad/ff7 [0,4194304] 0 2024-05-06T12:14:47.785 INFO:tasks.workunit.client.1.smithi190.stdout:8/553: symlink d3/d35/d55/lc1 0 2024-05-06T12:14:47.786 INFO:tasks.workunit.client.0.smithi092.stdout:2/787: dwrite f1 [0,4194304] 0 2024-05-06T12:14:47.787 INFO:tasks.workunit.client.1.smithi190.stdout:8/554: chown d3/d8/d25/d3b/d90/d6e/f82 31427102 1 2024-05-06T12:14:47.787 INFO:tasks.workunit.client.0.smithi092.stdout:1/929: dread d3/d42/d84/d4f/d58/f62 [0,4194304] 0 2024-05-06T12:14:47.789 INFO:tasks.workunit.client.0.smithi092.stdout:2/788: truncate df/dd0/de3/f72 1015713 0 2024-05-06T12:14:47.789 INFO:tasks.workunit.client.1.smithi190.stdout:8/555: fdatasync d3/d8/d2b/d3e/f8d 0 2024-05-06T12:14:47.789 INFO:tasks.workunit.client.1.smithi190.stdout:4/320: rename d2/d11/d16/d4e/l6d to d2/d11/d16/l71 0 2024-05-06T12:14:47.791 INFO:tasks.workunit.client.1.smithi190.stdout:5/476: dwrite d2/da/d59/d5f/f36 [0,4194304] 0 2024-05-06T12:14:47.792 INFO:tasks.workunit.client.0.smithi092.stdout:9/968: link d16/d1e/d7b/d8d/f111 d16/d1c/d58/f137 0 2024-05-06T12:14:47.792 INFO:tasks.workunit.client.1.smithi190.stdout:5/477: chown d2/d27/c7c 24111 1 2024-05-06T12:14:47.793 INFO:tasks.workunit.client.1.smithi190.stdout:4/321: truncate d2/d11/d16/d53/f57 1078202 0 2024-05-06T12:14:47.794 INFO:tasks.workunit.client.1.smithi190.stdout:5/478: dread - d2/da/f64 zero size 2024-05-06T12:14:47.794 INFO:tasks.workunit.client.1.smithi190.stdout:5/479: readlink d2/da/l55 0 2024-05-06T12:14:47.795 INFO:tasks.workunit.client.1.smithi190.stdout:8/556: creat d3/d13/fc2 x:0 0 0 2024-05-06T12:14:47.797 INFO:tasks.workunit.client.1.smithi190.stdout:8/557: write d3/d35/f87 [1339351,23544] 0 2024-05-06T12:14:47.799 INFO:tasks.workunit.client.1.smithi190.stdout:5/480: readlink d2/da/d59/d5f/d2e/d3b/l57 0 2024-05-06T12:14:47.800 INFO:tasks.workunit.client.0.smithi092.stdout:1/930: creat d3/d10/d13/d19/d21/d72/d112/d123/f12a x:0 0 0 2024-05-06T12:14:47.802 INFO:tasks.workunit.client.1.smithi190.stdout:9/539: truncate db/d23/f3a 1438470 0 2024-05-06T12:14:47.803 INFO:tasks.workunit.client.0.smithi092.stdout:1/931: chown d3/d10/d13/d19/d21/d72/d9a/da4/d119 2469 1 2024-05-06T12:14:47.806 INFO:tasks.workunit.client.0.smithi092.stdout:9/969: dwrite d16/d2c/d2f/d3d/ffd [0,4194304] 0 2024-05-06T12:14:47.806 INFO:tasks.workunit.client.0.smithi092.stdout:2/789: dwrite df/d5c/fc5 [0,4194304] 0 2024-05-06T12:14:47.807 INFO:tasks.workunit.client.1.smithi190.stdout:4/322: dwrite d2/d10/f35 [0,4194304] 0 2024-05-06T12:14:47.807 INFO:tasks.workunit.client.1.smithi190.stdout:9/540: dread db/dd/d16/f64 [0,4194304] 0 2024-05-06T12:14:47.809 INFO:tasks.workunit.client.0.smithi092.stdout:1/932: symlink d3/d10/d13/d19/d21/d88/l12b 0 2024-05-06T12:14:47.814 INFO:tasks.workunit.client.0.smithi092.stdout:1/933: chown d3/d10/d13/d19/d21/d72/fbd 49 1 2024-05-06T12:14:47.819 INFO:tasks.workunit.client.1.smithi190.stdout:4/323: link d2/d11/d16/f1a d2/d11/d16/d61/d67/d6e/f72 0 2024-05-06T12:14:47.819 INFO:tasks.workunit.client.1.smithi190.stdout:4/324: fdatasync d2/d11/f2e 0 2024-05-06T12:14:47.819 INFO:tasks.workunit.client.0.smithi092.stdout:9/970: creat d16/d2c/d2f/d3d/d33/d37/f138 x:0 0 0 2024-05-06T12:14:47.822 INFO:tasks.workunit.client.0.smithi092.stdout:2/790: dwrite df/d17/d53/da9/dd9/d83/dc2/fcb [0,4194304] 0 2024-05-06T12:14:47.826 INFO:tasks.workunit.client.1.smithi190.stdout:4/325: dwrite d2/f55 [0,4194304] 0 2024-05-06T12:14:47.833 INFO:tasks.workunit.client.0.smithi092.stdout:1/934: rename d3/d42/d84/d4f/d58/d65/d81/c11e to d3/d10/d13/d19/c12c 0 2024-05-06T12:14:47.834 INFO:tasks.workunit.client.0.smithi092.stdout:9/971: link d16/d2c/d2f/d75/l99 d16/d1c/d58/l139 0 2024-05-06T12:14:47.834 INFO:tasks.workunit.client.1.smithi190.stdout:4/326: dwrite d2/d11/d16/f38 [0,4194304] 0 2024-05-06T12:14:47.836 INFO:tasks.workunit.client.0.smithi092.stdout:9/972: write d16/d2c/d2f/d3d/fc2 [1178681,49744] 0 2024-05-06T12:14:47.840 INFO:tasks.workunit.client.1.smithi190.stdout:5/481: dread d2/da/f80 [0,4194304] 0 2024-05-06T12:14:47.841 INFO:tasks.workunit.client.0.smithi092.stdout:2/791: dwrite df/d5c/fc5 [4194304,4194304] 0 2024-05-06T12:14:47.842 INFO:tasks.workunit.client.1.smithi190.stdout:5/482: dread - d2/da/d59/f94 zero size 2024-05-06T12:14:47.843 INFO:tasks.workunit.client.0.smithi092.stdout:2/792: dread - df/d17/d51/d96/fe4 zero size 2024-05-06T12:14:47.844 INFO:tasks.workunit.client.1.smithi190.stdout:4/327: dwrite d2/f50 [0,4194304] 0 2024-05-06T12:14:47.844 INFO:tasks.workunit.client.0.smithi092.stdout:2/793: chown df/dd0/de3/dcf/lb5 945448 1 2024-05-06T12:14:47.847 INFO:tasks.workunit.client.0.smithi092.stdout:2/794: creat df/d17/d53/da9/fea x:0 0 0 2024-05-06T12:14:47.851 INFO:tasks.workunit.client.0.smithi092.stdout:2/795: rename df/d17/d53/da9/fd4 to df/d17/d53/feb 0 2024-05-06T12:14:47.853 INFO:tasks.workunit.client.0.smithi092.stdout:2/796: rmdir df/d17/d53/da9/dd9/d83/dc2 39 2024-05-06T12:14:47.853 INFO:tasks.workunit.client.1.smithi190.stdout:8/558: dread d3/d35/d55/f73 [0,4194304] 0 2024-05-06T12:14:47.854 INFO:tasks.workunit.client.0.smithi092.stdout:2/797: write df/d17/d53/d85/f98 [1814774,77505] 0 2024-05-06T12:14:47.858 INFO:tasks.workunit.client.1.smithi190.stdout:8/559: creat d3/d13/fc3 x:0 0 0 2024-05-06T12:14:47.858 INFO:tasks.workunit.client.1.smithi190.stdout:8/560: mknod d3/d8/d2b/d3e/dad/cc4 0 2024-05-06T12:14:47.859 INFO:tasks.workunit.client.1.smithi190.stdout:8/561: chown d3/d8/d25/f39 18 1 2024-05-06T12:14:47.859 INFO:tasks.workunit.client.1.smithi190.stdout:8/562: write d3/d8/d2b/f47 [4997030,110796] 0 2024-05-06T12:14:47.859 INFO:tasks.workunit.client.0.smithi092.stdout:2/798: fsync df/d17/d53/d85/da0/fc1 0 2024-05-06T12:14:47.859 INFO:tasks.workunit.client.0.smithi092.stdout:2/799: creat df/dd0/fec x:0 0 0 2024-05-06T12:14:47.860 INFO:tasks.workunit.client.0.smithi092.stdout:2/800: dread - df/d17/d7e/fdf zero size 2024-05-06T12:14:47.860 INFO:tasks.workunit.client.1.smithi190.stdout:8/563: mkdir d3/d35/d3a/dc5 0 2024-05-06T12:14:47.878 INFO:tasks.workunit.client.1.smithi190.stdout:9/541: truncate db/dd/d16/d58/d87/d22/f54 442535 0 2024-05-06T12:14:47.879 INFO:tasks.workunit.client.0.smithi092.stdout:1/935: write d3/d42/d84/d4f/d58/d65/d81/fbc [1008308,29879] 0 2024-05-06T12:14:47.881 INFO:tasks.workunit.client.0.smithi092.stdout:2/801: truncate df/dd0/de3/fbd 1460365 0 2024-05-06T12:14:47.881 INFO:tasks.workunit.client.1.smithi190.stdout:9/542: write db/dd/d16/d58/d87/d22/d49/d88/f9c [942314,22091] 0 2024-05-06T12:14:47.882 INFO:tasks.workunit.client.0.smithi092.stdout:1/936: mknod d3/d42/d84/d9f/c12d 0 2024-05-06T12:14:47.883 INFO:tasks.workunit.client.1.smithi190.stdout:9/543: mknod db/dd/d16/d58/d87/d99/cc0 0 2024-05-06T12:14:47.884 INFO:tasks.workunit.client.0.smithi092.stdout:2/802: creat df/dd0/de3/d8e/fed x:0 0 0 2024-05-06T12:14:47.884 INFO:tasks.workunit.client.0.smithi092.stdout:1/937: write d3/d10/d13/f1f [2935447,90595] 0 2024-05-06T12:14:47.885 INFO:tasks.workunit.client.1.smithi190.stdout:4/328: dwrite d2/d11/d16/f1a [0,4194304] 0 2024-05-06T12:14:47.885 INFO:tasks.workunit.client.0.smithi092.stdout:9/973: dwrite d16/d1e/f25 [0,4194304] 0 2024-05-06T12:14:47.886 INFO:tasks.workunit.client.0.smithi092.stdout:2/803: readlink df/d17/d51/le8 0 2024-05-06T12:14:47.886 INFO:tasks.workunit.client.1.smithi190.stdout:9/544: fsync db/dd/d16/d4c/f9b 0 2024-05-06T12:14:47.886 INFO:tasks.workunit.client.0.smithi092.stdout:1/938: chown d3/d42/d84/dcf/le0 215637546 1 2024-05-06T12:14:47.890 INFO:tasks.workunit.client.0.smithi092.stdout:1/939: symlink d3/d10/d13/db0/l12e 0 2024-05-06T12:14:47.894 INFO:tasks.workunit.client.0.smithi092.stdout:2/804: unlink df/f12 0 2024-05-06T12:14:47.894 INFO:tasks.workunit.client.1.smithi190.stdout:8/564: dwrite d3/fb1 [0,4194304] 0 2024-05-06T12:14:47.895 INFO:tasks.workunit.client.1.smithi190.stdout:4/329: dread d2/f32 [4194304,4194304] 0 2024-05-06T12:14:47.896 INFO:tasks.workunit.client.0.smithi092.stdout:9/974: mknod d16/d2c/d2f/d3d/d33/d6b/dbe/c13a 0 2024-05-06T12:14:47.897 INFO:tasks.workunit.client.0.smithi092.stdout:1/940: mknod d3/d10/d13/d19/d57/dd6/c12f 0 2024-05-06T12:14:47.899 INFO:tasks.workunit.client.0.smithi092.stdout:2/805: dread df/d17/d53/da9/dd9/d83/f9d [0,4194304] 0 2024-05-06T12:14:47.900 INFO:tasks.workunit.client.1.smithi190.stdout:4/330: rename d2/c7 to d2/d11/d16/d61/d66/c73 0 2024-05-06T12:14:47.902 INFO:tasks.workunit.client.0.smithi092.stdout:2/806: write df/dd0/fdd [207112,56185] 0 2024-05-06T12:14:47.902 INFO:tasks.workunit.client.1.smithi190.stdout:4/331: write d2/f64 [715168,73524] 0 2024-05-06T12:14:47.903 INFO:tasks.workunit.client.0.smithi092.stdout:9/975: symlink d16/d1e/d7b/dce/de4/d131/l13b 0 2024-05-06T12:14:47.904 INFO:tasks.workunit.client.0.smithi092.stdout:2/807: fdatasync df/d17/d1d/f21 0 2024-05-06T12:14:47.906 INFO:tasks.workunit.client.1.smithi190.stdout:4/332: link d2/d11/d16/d53/f57 d2/d11/d16/d61/d66/f74 0 2024-05-06T12:14:47.908 INFO:tasks.workunit.client.1.smithi190.stdout:4/333: mknod d2/d11/d16/d4e/c75 0 2024-05-06T12:14:47.908 INFO:tasks.workunit.client.0.smithi092.stdout:2/808: unlink df/d17/d1d/c4b 0 2024-05-06T12:14:47.909 INFO:tasks.workunit.client.0.smithi092.stdout:2/809: fdatasync df/d17/d1d/f49 0 2024-05-06T12:14:47.910 INFO:tasks.workunit.client.1.smithi190.stdout:4/334: symlink d2/d10/l76 0 2024-05-06T12:14:47.911 INFO:tasks.workunit.client.0.smithi092.stdout:9/976: link d16/d2c/d2f/l68 d16/d94/ddc/l13c 0 2024-05-06T12:14:47.912 INFO:tasks.workunit.client.0.smithi092.stdout:2/810: creat df/d17/d51/de9/fee x:0 0 0 2024-05-06T12:14:47.915 INFO:tasks.workunit.client.0.smithi092.stdout:9/977: rename d16/f29 to d16/d1e/d7b/dce/de4/d131/f13d 0 2024-05-06T12:14:47.919 INFO:tasks.workunit.client.0.smithi092.stdout:2/811: link df/d17/d53/d85/da0/fc1 df/d17/d51/de9/fef 0 2024-05-06T12:14:47.922 INFO:tasks.workunit.client.0.smithi092.stdout:9/978: rmdir d16/d1c/d12f 0 2024-05-06T12:14:47.926 INFO:tasks.workunit.client.0.smithi092.stdout:2/812: dwrite df/d17/d1d/f4f [0,4194304] 0 2024-05-06T12:14:47.926 INFO:tasks.workunit.client.0.smithi092.stdout:1/941: dwrite d3/d42/d84/d4f/f97 [0,4194304] 0 2024-05-06T12:14:47.932 INFO:tasks.workunit.client.0.smithi092.stdout:1/942: mknod d3/d10/d13/d19/d34/c130 0 2024-05-06T12:14:47.933 INFO:tasks.workunit.client.0.smithi092.stdout:2/813: dread df/d15/d88/fa7 [0,4194304] 0 2024-05-06T12:14:47.935 INFO:tasks.workunit.client.0.smithi092.stdout:1/943: unlink d3/d10/d13/d19/d21/c6c 0 2024-05-06T12:14:47.938 INFO:tasks.workunit.client.0.smithi092.stdout:1/944: creat d3/d10/d13/d19/d34/f131 x:0 0 0 2024-05-06T12:14:47.942 INFO:tasks.workunit.client.0.smithi092.stdout:1/945: link d3/d10/d13/d19/d21/d88/fd7 d3/d10/d13/d19/d21/d72/d9a/f132 0 2024-05-06T12:14:47.944 INFO:tasks.workunit.client.0.smithi092.stdout:1/946: write d3/d42/d84/d4f/d58/d65/d81/f8a [3086074,45818] 0 2024-05-06T12:14:47.948 INFO:tasks.workunit.client.0.smithi092.stdout:9/979: dwrite d16/d2c/d2f/d3d/d33/d6b/dbe/fd0 [0,4194304] 0 2024-05-06T12:14:47.950 INFO:tasks.workunit.client.0.smithi092.stdout:1/947: mknod d3/d42/d84/d4f/dd5/ded/c133 0 2024-05-06T12:14:47.951 INFO:tasks.workunit.client.0.smithi092.stdout:2/814: dwrite df/d17/f6b [0,4194304] 0 2024-05-06T12:14:47.959 INFO:tasks.workunit.client.0.smithi092.stdout:1/948: dwrite d3/d10/f115 [0,4194304] 0 2024-05-06T12:14:47.963 INFO:tasks.workunit.client.0.smithi092.stdout:1/949: symlink d3/d42/d84/d4f/d58/d65/d81/d8d/l134 0 2024-05-06T12:14:47.964 INFO:tasks.workunit.client.0.smithi092.stdout:1/950: chown d3/d10/d13/d19/d57/dd6/c12f 18 1 2024-05-06T12:14:47.969 INFO:tasks.workunit.client.0.smithi092.stdout:9/980: truncate d16/f17 3881942 0 2024-05-06T12:14:47.970 INFO:tasks.workunit.client.0.smithi092.stdout:9/981: fsync d16/d2c/d2f/d3d/d33/d6b/d9e/fba 0 2024-05-06T12:14:47.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:47 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:14:47.975 INFO:tasks.workunit.client.0.smithi092.stdout:9/982: rename d16/d2c/d2f/d3d to d16/d1e/d13e 0 2024-05-06T12:14:47.975 INFO:tasks.workunit.client.0.smithi092.stdout:9/983: dread - d16/d1e/d13e/d33/d37/f138 zero size 2024-05-06T12:14:47.976 INFO:tasks.workunit.client.0.smithi092.stdout:9/984: chown d16/d1e/d13e/d33/d37/cc8 16 1 2024-05-06T12:14:47.979 INFO:tasks.workunit.client.0.smithi092.stdout:9/985: mkdir d16/d1c/d58/d92/dab/deb/d113/d13f 0 2024-05-06T12:14:47.980 INFO:tasks.workunit.client.0.smithi092.stdout:9/986: write d16/d2c/d2f/d75/f10b [635318,96471] 0 2024-05-06T12:14:47.982 INFO:tasks.workunit.client.0.smithi092.stdout:9/987: mknod d16/d1e/d13e/d33/d7e/dad/c140 0 2024-05-06T12:14:47.983 INFO:tasks.workunit.client.0.smithi092.stdout:9/988: symlink d16/d1e/d13e/l141 0 2024-05-06T12:14:47.987 INFO:tasks.workunit.client.0.smithi092.stdout:9/989: creat d16/d1e/d13e/f142 x:0 0 0 2024-05-06T12:14:47.987 INFO:tasks.workunit.client.0.smithi092.stdout:1/951: dread d3/d10/d13/d46/f68 [0,4194304] 0 2024-05-06T12:14:47.991 INFO:tasks.workunit.client.0.smithi092.stdout:1/952: getdents d3/d10/d13/dd2 0 2024-05-06T12:14:47.992 INFO:tasks.workunit.client.0.smithi092.stdout:1/953: write d3/d10/d13/d19/d21/fb4 [3192091,23503] 0 2024-05-06T12:14:48.021 INFO:tasks.workunit.client.1.smithi190.stdout:1/342: sync 2024-05-06T12:14:48.022 INFO:tasks.workunit.client.1.smithi190.stdout:6/345: sync 2024-05-06T12:14:48.022 INFO:tasks.workunit.client.1.smithi190.stdout:9/545: truncate db/dd/d5c/fb0 2348079 0 2024-05-06T12:14:48.023 INFO:tasks.workunit.client.1.smithi190.stdout:6/346: readlink l9 0 2024-05-06T12:14:48.023 INFO:tasks.workunit.client.1.smithi190.stdout:6/347: fdatasync fd 0 2024-05-06T12:14:48.025 INFO:tasks.workunit.client.1.smithi190.stdout:6/348: dread - df/f47 zero size 2024-05-06T12:14:48.026 INFO:tasks.workunit.client.1.smithi190.stdout:4/335: chown d2/d11/d16/d41/f62 649 1 2024-05-06T12:14:48.026 INFO:tasks.workunit.client.1.smithi190.stdout:6/349: creat df/d2c/d61/f78 x:0 0 0 2024-05-06T12:14:48.027 INFO:tasks.workunit.client.1.smithi190.stdout:1/343: mknod d5/d4f/d65/d32/c74 0 2024-05-06T12:14:48.029 INFO:tasks.workunit.client.1.smithi190.stdout:1/344: symlink d5/d15/l75 0 2024-05-06T12:14:48.031 INFO:tasks.workunit.client.1.smithi190.stdout:1/345: creat d5/d4f/d65/d24/d3a/f76 x:0 0 0 2024-05-06T12:14:48.032 INFO:tasks.workunit.client.1.smithi190.stdout:6/350: getdents df/d2c 0 2024-05-06T12:14:48.033 INFO:tasks.workunit.client.1.smithi190.stdout:1/346: creat d5/d4f/d65/d32/d47/f77 x:0 0 0 2024-05-06T12:14:48.033 INFO:tasks.workunit.client.1.smithi190.stdout:6/351: write df/d2c/d1e/d57/f69 [869483,43636] 0 2024-05-06T12:14:48.034 INFO:tasks.workunit.client.1.smithi190.stdout:1/347: dread - d5/d4f/d65/d32/d47/f77 zero size 2024-05-06T12:14:48.036 INFO:tasks.workunit.client.1.smithi190.stdout:1/348: stat d5/d4f/d65/d24/d3a/l3d 0 2024-05-06T12:14:48.038 INFO:tasks.workunit.client.1.smithi190.stdout:1/349: symlink d5/d4f/d65/l78 0 2024-05-06T12:14:48.040 INFO:tasks.workunit.client.1.smithi190.stdout:6/352: dwrite df/d17/f5c [0,4194304] 0 2024-05-06T12:14:48.041 INFO:tasks.workunit.client.1.smithi190.stdout:6/353: readlink df/d2c/l34 0 2024-05-06T12:14:48.041 INFO:tasks.workunit.client.1.smithi190.stdout:6/354: readlink l9 0 2024-05-06T12:14:48.042 INFO:tasks.workunit.client.1.smithi190.stdout:1/350: unlink d5/cc 0 2024-05-06T12:14:48.043 INFO:tasks.workunit.client.1.smithi190.stdout:6/355: creat df/d2c/d61/f79 x:0 0 0 2024-05-06T12:14:48.045 INFO:tasks.workunit.client.1.smithi190.stdout:6/356: fdatasync df/d17/d44/f62 0 2024-05-06T12:14:48.047 INFO:tasks.workunit.client.1.smithi190.stdout:6/357: write df/d17/d44/f48 [636702,13256] 0 2024-05-06T12:14:48.049 INFO:tasks.workunit.client.1.smithi190.stdout:1/351: dwrite d5/fe [4194304,4194304] 0 2024-05-06T12:14:48.050 INFO:tasks.workunit.client.1.smithi190.stdout:1/352: chown d5/d4f/d65/d2b 24 1 2024-05-06T12:14:48.051 INFO:tasks.workunit.client.1.smithi190.stdout:6/358: creat df/d2c/d1e/f7a x:0 0 0 2024-05-06T12:14:48.053 INFO:tasks.workunit.client.1.smithi190.stdout:1/353: creat d5/d4f/d65/d32/d47/d53/d6e/f79 x:0 0 0 2024-05-06T12:14:48.055 INFO:tasks.workunit.client.1.smithi190.stdout:6/359: mknod df/d2c/d1e/c7b 0 2024-05-06T12:14:48.058 INFO:tasks.workunit.client.1.smithi190.stdout:6/360: link df/d2c/d1e/d57/f6b df/d2c/d28/f7c 0 2024-05-06T12:14:48.062 INFO:tasks.workunit.client.1.smithi190.stdout:6/361: write df/d2c/d1e/d57/f69 [1369856,115970] 0 2024-05-06T12:14:48.069 INFO:tasks.workunit.client.1.smithi190.stdout:1/354: dread d5/d4f/d65/d24/f28 [0,4194304] 0 2024-05-06T12:14:48.070 INFO:tasks.workunit.client.1.smithi190.stdout:1/355: stat d5/d4f/d65/f5b 0 2024-05-06T12:14:48.072 INFO:tasks.workunit.client.1.smithi190.stdout:1/356: creat d5/d15/f7a x:0 0 0 2024-05-06T12:14:48.077 INFO:tasks.workunit.client.0.smithi092.stdout:1/954: write d3/d10/d13/f3b [1066990,112165] 0 2024-05-06T12:14:48.079 INFO:tasks.workunit.client.1.smithi190.stdout:4/336: write d2/f1e [970609,1139] 0 2024-05-06T12:14:48.079 INFO:tasks.workunit.client.0.smithi092.stdout:1/955: mknod d3/d10/d13/d19/d21/d72/d9a/d9c/c135 0 2024-05-06T12:14:48.080 INFO:tasks.workunit.client.0.smithi092.stdout:1/956: chown d3/d10/d13/d19/d21/d72/d9a/f132 29 1 2024-05-06T12:14:48.081 INFO:tasks.workunit.client.1.smithi190.stdout:1/357: dwrite d5/d4f/d65/d24/d3a/f73 [4194304,4194304] 0 2024-05-06T12:14:48.083 INFO:tasks.workunit.client.1.smithi190.stdout:4/337: mknod d2/d11/d16/d53/c77 0 2024-05-06T12:14:48.086 INFO:tasks.workunit.client.1.smithi190.stdout:4/338: mkdir d2/d11/d16/d61/d67/d6e/d78 0 2024-05-06T12:14:48.094 INFO:tasks.workunit.client.0.smithi092.stdout:9/990: dread d16/d1e/d13e/d33/f56 [0,4194304] 0 2024-05-06T12:14:48.096 INFO:tasks.workunit.client.0.smithi092.stdout:9/991: creat d16/d1e/da1/f143 x:0 0 0 2024-05-06T12:14:48.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:47 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:14:48.099 INFO:tasks.workunit.client.0.smithi092.stdout:1/957: dread d3/d10/d13/d19/d21/d88/f91 [0,4194304] 0 2024-05-06T12:14:48.102 INFO:tasks.workunit.client.0.smithi092.stdout:1/958: mknod d3/d10/d13/dd2/c136 0 2024-05-06T12:14:48.103 INFO:tasks.workunit.client.0.smithi092.stdout:1/959: write d3/d10/d13/d46/fa1 [719416,101559] 0 2024-05-06T12:14:48.104 INFO:tasks.workunit.client.0.smithi092.stdout:1/960: write d3/d10/d13/f1f [214883,109903] 0 2024-05-06T12:14:48.108 INFO:tasks.workunit.client.0.smithi092.stdout:1/961: truncate d3/d10/d13/d19/d21/d72/d9a/da4/fb9 984972 0 2024-05-06T12:14:48.108 INFO:tasks.workunit.client.0.smithi092.stdout:9/992: dread d16/d2c/d2f/d75/ff6 [0,4194304] 0 2024-05-06T12:14:48.110 INFO:tasks.workunit.client.0.smithi092.stdout:9/993: mknod d16/d1c/d58/d92/dab/deb/c144 0 2024-05-06T12:14:48.115 INFO:tasks.workunit.client.0.smithi092.stdout:9/994: link d16/d1e/da1/d78/db4/fdb d16/d1e/d13e/d33/f145 0 2024-05-06T12:14:48.118 INFO:tasks.workunit.client.0.smithi092.stdout:1/962: dread d3/d10/d13/d46/f7b [0,4194304] 0 2024-05-06T12:14:48.122 INFO:tasks.workunit.client.0.smithi092.stdout:1/963: rename d3/d42/d84/d4f/d58/c93 to d3/d10/d13/d19/d21/d88/d8b/dfc/c137 0 2024-05-06T12:14:48.124 INFO:tasks.workunit.client.0.smithi092.stdout:9/995: write d16/d1e/d13e/d33/d6b/d9e/fc6 [45759,96795] 0 2024-05-06T12:14:48.126 INFO:tasks.workunit.client.0.smithi092.stdout:9/996: read d16/d1e/d7b/d8d/f111 [4028904,6385] 0 2024-05-06T12:14:48.126 INFO:tasks.workunit.client.0.smithi092.stdout:1/964: rename d3/d10/d13/d46/f68 to d3/d10/d13/d46/f138 0 2024-05-06T12:14:48.130 INFO:tasks.workunit.client.0.smithi092.stdout:9/997: creat d16/d1c/d58/d92/dab/deb/d113/d13f/f146 x:0 0 0 2024-05-06T12:14:48.138 INFO:tasks.workunit.client.0.smithi092.stdout:8/953: sync 2024-05-06T12:14:48.143 INFO:tasks.workunit.client.0.smithi092.stdout:1/965: dread d3/d10/d13/d19/d21/d88/fa0 [0,4194304] 0 2024-05-06T12:14:48.151 INFO:tasks.workunit.client.1.smithi190.stdout:0/519: sync 2024-05-06T12:14:48.151 INFO:tasks.workunit.client.1.smithi190.stdout:2/399: sync 2024-05-06T12:14:48.161 INFO:tasks.workunit.client.1.smithi190.stdout:1/358: getdents d5/d4f/d65/d24 0 2024-05-06T12:14:48.161 INFO:tasks.workunit.client.0.smithi092.stdout:9/998: dread - d16/d94/da0/da7/fe7 zero size 2024-05-06T12:14:48.162 INFO:tasks.workunit.client.1.smithi190.stdout:4/339: dread d2/f55 [0,4194304] 0 2024-05-06T12:14:48.163 INFO:tasks.workunit.client.1.smithi190.stdout:2/400: truncate d2/d21/f5e 661111 0 2024-05-06T12:14:48.163 INFO:tasks.workunit.client.0.smithi092.stdout:9/999: chown d16/d1e/d7b/dce/f89 564270239 1 2024-05-06T12:14:48.163 INFO:tasks.workunit.client.0.smithi092.stdout:1/966: truncate d3/d10/d13/d19/d21/d72/d9a/f132 599274 0 2024-05-06T12:14:48.164 INFO:tasks.workunit.client.1.smithi190.stdout:0/520: symlink d9/dc/d48/d50/d87/lb9 0 2024-05-06T12:14:48.165 INFO:tasks.workunit.client.1.smithi190.stdout:1/359: truncate d5/d4f/d65/d24/d3a/f72 1040601 0 2024-05-06T12:14:48.165 INFO:tasks.workunit.client.1.smithi190.stdout:4/340: write d2/d11/d16/d4e/f6f [203530,79714] 0 2024-05-06T12:14:48.166 INFO:tasks.workunit.client.0.smithi092.stdout:8/954: dread d9/dc/fa4 [0,4194304] 0 2024-05-06T12:14:48.167 INFO:tasks.workunit.client.1.smithi190.stdout:0/521: readlink d9/d51/l92 0 2024-05-06T12:14:48.167 INFO:tasks.workunit.client.0.smithi092.stdout:8/955: fdatasync d9/d21/d37/d46/d5c/d71/d75/fef 0 2024-05-06T12:14:48.168 INFO:tasks.workunit.client.1.smithi190.stdout:9/546: dread db/dd/f41 [0,4194304] 0 2024-05-06T12:14:48.169 INFO:tasks.workunit.client.1.smithi190.stdout:0/522: fdatasync d9/dc/f6c 0 2024-05-06T12:14:48.170 INFO:tasks.workunit.client.1.smithi190.stdout:4/341: write d2/d11/d16/f1a [1645777,60178] 0 2024-05-06T12:14:48.173 INFO:tasks.workunit.client.1.smithi190.stdout:4/342: readlink d2/d11/d16/d4e/l69 0 2024-05-06T12:14:48.174 INFO:tasks.workunit.client.1.smithi190.stdout:2/401: dwrite d2/d8/d33/d46/f5f [0,4194304] 0 2024-05-06T12:14:48.174 INFO:tasks.workunit.client.1.smithi190.stdout:1/360: creat d5/d4f/d65/f7b x:0 0 0 2024-05-06T12:14:48.174 INFO:tasks.workunit.client.0.smithi092.stdout:8/956: dwrite d9/f12f [0,4194304] 0 2024-05-06T12:14:48.175 INFO:tasks.workunit.client.0.smithi092.stdout:8/957: fsync d9/d21/d37/dbd/dc5/d11c/ff5 0 2024-05-06T12:14:48.178 INFO:tasks.workunit.client.1.smithi190.stdout:4/343: symlink d2/d11/d16/d61/l79 0 2024-05-06T12:14:48.179 INFO:tasks.workunit.client.0.smithi092.stdout:8/958: rmdir d9/d5a/dd9 39 2024-05-06T12:14:48.184 INFO:tasks.workunit.client.1.smithi190.stdout:2/402: link d2/d8/dc/f16 d2/d8/d19/f83 0 2024-05-06T12:14:48.184 INFO:tasks.workunit.client.1.smithi190.stdout:9/547: dwrite db/d23/d29/f7c [0,4194304] 0 2024-05-06T12:14:48.186 INFO:tasks.workunit.client.1.smithi190.stdout:0/523: link d9/df/d21/d72/f98 d9/df/d56/dab/fba 0 2024-05-06T12:14:48.187 INFO:tasks.workunit.client.1.smithi190.stdout:2/403: chown d2/d8/d33/d46/d5d 2917884 1 2024-05-06T12:14:48.189 INFO:tasks.workunit.client.1.smithi190.stdout:9/548: unlink l8 0 2024-05-06T12:14:48.192 INFO:tasks.workunit.client.0.smithi092.stdout:1/967: truncate d3/d10/d13/d19/d21/d72/d9a/ff9 420716 0 2024-05-06T12:14:48.195 INFO:tasks.workunit.client.1.smithi190.stdout:4/344: link d2/f5e d2/d11/d16/d61/d67/d6e/d78/f7a 0 2024-05-06T12:14:48.198 INFO:tasks.workunit.client.1.smithi190.stdout:2/404: rename d2/c30 to d2/d8/dc/c84 0 2024-05-06T12:14:48.198 INFO:tasks.workunit.client.0.smithi092.stdout:1/968: dwrite d3/d10/d13/d46/f92 [0,4194304] 0 2024-05-06T12:14:48.199 INFO:tasks.workunit.client.0.smithi092.stdout:1/969: chown d3/d10/d13/dd2/c136 6553430 1 2024-05-06T12:14:48.200 INFO:tasks.workunit.client.1.smithi190.stdout:0/524: mknod d9/dc/d48/d50/d87/cbb 0 2024-05-06T12:14:48.201 INFO:tasks.workunit.client.0.smithi092.stdout:1/970: symlink d3/d10/d13/d19/d21/d88/d8b/l139 0 2024-05-06T12:14:48.202 INFO:tasks.workunit.client.1.smithi190.stdout:1/361: link d5/d4f/d65/d32/d47/d53/c5d d5/d4f/d65/d2b/c7c 0 2024-05-06T12:14:48.202 INFO:tasks.workunit.client.0.smithi092.stdout:2/815: sync 2024-05-06T12:14:48.202 INFO:tasks.workunit.client.0.smithi092.stdout:3/790: sync 2024-05-06T12:14:48.203 INFO:tasks.workunit.client.0.smithi092.stdout:3/791: chown d8/d24/d33/f64 444089593 1 2024-05-06T12:14:48.203 INFO:tasks.workunit.client.1.smithi190.stdout:4/345: mkdir d2/d11/d16/d53/d7b 0 2024-05-06T12:14:48.204 INFO:tasks.workunit.client.0.smithi092.stdout:1/971: creat d3/d10/d13/d19/d21/d72/d9a/da4/deb/f13a x:0 0 0 2024-05-06T12:14:48.205 INFO:tasks.workunit.client.1.smithi190.stdout:1/362: dread - d5/d15/f52 zero size 2024-05-06T12:14:48.205 INFO:tasks.workunit.client.0.smithi092.stdout:3/792: chown d8/db/d4c/d40/d7a/d90/fc2 1 1 2024-05-06T12:14:48.207 INFO:tasks.workunit.client.1.smithi190.stdout:9/549: write db/d23/f53 [189366,117187] 0 2024-05-06T12:14:48.208 INFO:tasks.workunit.client.0.smithi092.stdout:3/793: symlink d8/d24/d33/d4b/d74/l100 0 2024-05-06T12:14:48.209 INFO:tasks.workunit.client.1.smithi190.stdout:2/405: dwrite d2/d4c/f5a [0,4194304] 0 2024-05-06T12:14:48.209 INFO:tasks.workunit.client.0.smithi092.stdout:8/959: dwrite d9/d21/d54/d79/d8a/fb8 [0,4194304] 0 2024-05-06T12:14:48.210 INFO:tasks.workunit.client.1.smithi190.stdout:0/525: creat d9/df/d56/dab/fbc x:0 0 0 2024-05-06T12:14:48.210 INFO:tasks.workunit.client.1.smithi190.stdout:1/363: rmdir d5/d4f 39 2024-05-06T12:14:48.213 INFO:tasks.workunit.client.0.smithi092.stdout:1/972: getdents d3/d10/d13/d19/d21/d72/d9a/d9c 0 2024-05-06T12:14:48.215 INFO:tasks.workunit.client.0.smithi092.stdout:1/973: readlink d3/d10/d13/d19/d57/dd6/l128 0 2024-05-06T12:14:48.216 INFO:tasks.workunit.client.0.smithi092.stdout:8/960: link d9/d21/d37/d74/c129 d9/d21/d54/d128/c14a 0 2024-05-06T12:14:48.216 INFO:tasks.workunit.client.0.smithi092.stdout:3/794: dwrite d8/d1b/f81 [0,4194304] 0 2024-05-06T12:14:48.219 INFO:tasks.workunit.client.0.smithi092.stdout:8/961: truncate d9/d21/dc0/fcc 1536149 0 2024-05-06T12:14:48.219 INFO:tasks.workunit.client.1.smithi190.stdout:9/550: rmdir db/dd/d16/d4c/da7 0 2024-05-06T12:14:48.220 INFO:tasks.workunit.client.1.smithi190.stdout:0/526: dwrite d9/df/d21/f4a [0,4194304] 0 2024-05-06T12:14:48.221 INFO:tasks.workunit.client.0.smithi092.stdout:3/795: unlink d8/d1b/f2b 0 2024-05-06T12:14:48.223 INFO:tasks.workunit.client.0.smithi092.stdout:1/974: symlink d3/d42/d84/d4f/d58/d65/d81/l13b 0 2024-05-06T12:14:48.227 INFO:tasks.workunit.client.0.smithi092.stdout:3/796: mkdir d8/db/d4c/d3d/d49/df8/d5b/d101 0 2024-05-06T12:14:48.228 INFO:tasks.workunit.client.0.smithi092.stdout:3/797: readlink d8/db/d4c/d3d/d49/ded/laf 0 2024-05-06T12:14:48.229 INFO:tasks.workunit.client.0.smithi092.stdout:8/962: chown d9/d21/d54/d128/c14a 2 1 2024-05-06T12:14:48.230 INFO:tasks.workunit.client.1.smithi190.stdout:2/406: getdents d2/d8/d19/d43 0 2024-05-06T12:14:48.231 INFO:tasks.workunit.client.0.smithi092.stdout:1/975: mkdir d3/d102/d13c 0 2024-05-06T12:14:48.231 INFO:tasks.workunit.client.1.smithi190.stdout:9/551: creat db/dd/d16/d58/d87/d22/d49/d8c/fc1 x:0 0 0 2024-05-06T12:14:48.231 INFO:tasks.workunit.client.0.smithi092.stdout:1/976: chown d3/d42/fcb 29657 1 2024-05-06T12:14:48.233 INFO:tasks.workunit.client.0.smithi092.stdout:1/977: chown d3/d10/d13/d19/d34/f66 0 1 2024-05-06T12:14:48.233 INFO:tasks.workunit.client.0.smithi092.stdout:8/963: rmdir d9/d21/d54/d79 39 2024-05-06T12:14:48.233 INFO:tasks.workunit.client.0.smithi092.stdout:3/798: symlink d8/de4/l102 0 2024-05-06T12:14:48.234 INFO:tasks.workunit.client.1.smithi190.stdout:0/527: link d9/dc/f1f d9/d51/d93/fbd 0 2024-05-06T12:14:48.235 INFO:tasks.workunit.client.0.smithi092.stdout:8/964: fdatasync d9/dc/f12d 0 2024-05-06T12:14:48.237 INFO:tasks.workunit.client.1.smithi190.stdout:3/506: sync 2024-05-06T12:14:48.242 INFO:tasks.workunit.client.1.smithi190.stdout:1/364: link d5/d4f/d65/d32/d47/c5e d5/d4f/d65/d32/c7d 0 2024-05-06T12:14:48.243 INFO:tasks.workunit.client.0.smithi092.stdout:1/978: dwrite d3/d10/d13/d46/f138 [0,4194304] 0 2024-05-06T12:14:48.244 INFO:tasks.workunit.client.1.smithi190.stdout:1/365: write d5/f34 [100918,11279] 0 2024-05-06T12:14:48.245 INFO:tasks.workunit.client.0.smithi092.stdout:1/979: rmdir d3/d42/d84/d4f/d58/d65 39 2024-05-06T12:14:48.246 INFO:tasks.workunit.client.1.smithi190.stdout:3/507: rename d5/d7/d15/d1f/d33/d40 to d5/d7/d15/d1f/d84/db3 0 2024-05-06T12:14:48.249 INFO:tasks.workunit.client.1.smithi190.stdout:0/528: dwrite d9/dc/d63/fa2 [0,4194304] 0 2024-05-06T12:14:48.249 INFO:tasks.workunit.client.0.smithi092.stdout:1/980: rename d3/f3d to d3/d10/d13/dd2/f13d 0 2024-05-06T12:14:48.252 INFO:tasks.workunit.client.1.smithi190.stdout:1/366: creat d5/d4f/d65/d24/f7e x:0 0 0 2024-05-06T12:14:48.255 INFO:tasks.workunit.client.0.smithi092.stdout:1/981: read d3/d10/d13/d19/d21/d88/fd7 [10172,109857] 0 2024-05-06T12:14:48.255 INFO:tasks.workunit.client.1.smithi190.stdout:3/508: mknod d5/d7/d15/d1f/d33/cb4 0 2024-05-06T12:14:48.255 INFO:tasks.workunit.client.1.smithi190.stdout:0/529: read d9/dc/d48/d50/f85 [125360,23066] 0 2024-05-06T12:14:48.258 INFO:tasks.workunit.client.1.smithi190.stdout:0/530: truncate d9/dc/d48/f9e 584751 0 2024-05-06T12:14:48.259 INFO:tasks.workunit.client.1.smithi190.stdout:3/509: symlink d5/d7/d15/d1f/d33/lb5 0 2024-05-06T12:14:48.262 INFO:tasks.workunit.client.0.smithi092.stdout:3/799: dwrite d8/db/d4c/d3d/d49/df8/f99 [0,4194304] 0 2024-05-06T12:14:48.263 INFO:tasks.workunit.client.0.smithi092.stdout:3/800: chown d8/d1b/f3e 168598 1 2024-05-06T12:14:48.266 INFO:tasks.workunit.client.1.smithi190.stdout:0/531: symlink d9/d91/lbe 0 2024-05-06T12:14:48.268 INFO:tasks.workunit.client.1.smithi190.stdout:9/552: dread db/f3d [0,4194304] 0 2024-05-06T12:14:48.268 INFO:tasks.workunit.client.1.smithi190.stdout:3/510: dwrite d5/d14/f8a [0,4194304] 0 2024-05-06T12:14:48.271 INFO:tasks.workunit.client.1.smithi190.stdout:2/407: truncate d2/f35 2650575 0 2024-05-06T12:14:48.273 INFO:tasks.workunit.client.0.smithi092.stdout:8/965: dwrite d9/d21/d37/d74/f109 [0,4194304] 0 2024-05-06T12:14:48.278 INFO:tasks.workunit.client.1.smithi190.stdout:3/511: dwrite d5/d7/d15/d1f/d33/f66 [0,4194304] 0 2024-05-06T12:14:48.279 INFO:tasks.workunit.client.0.smithi092.stdout:8/966: creat d9/d21/d54/d116/f14b x:0 0 0 2024-05-06T12:14:48.279 INFO:tasks.workunit.client.1.smithi190.stdout:3/512: readlink d5/d14/d11/d21/l48 0 2024-05-06T12:14:48.280 INFO:tasks.workunit.client.0.smithi092.stdout:1/982: truncate d3/d10/d13/d19/d21/d72/d9a/da4/deb/f103 519614 0 2024-05-06T12:14:48.280 INFO:tasks.workunit.client.1.smithi190.stdout:3/513: stat d5/d7/d15/d1f/d33/d5c/f63 0 2024-05-06T12:14:48.282 INFO:tasks.workunit.client.0.smithi092.stdout:3/801: write d8/d24/d27/ffe [279628,88112] 0 2024-05-06T12:14:48.286 INFO:tasks.workunit.client.0.smithi092.stdout:1/983: rename d3/d10/d13/dd2 to d3/d102/d13e 0 2024-05-06T12:14:48.287 INFO:tasks.workunit.client.1.smithi190.stdout:2/408: creat d2/d21/f85 x:0 0 0 2024-05-06T12:14:48.287 INFO:tasks.workunit.client.0.smithi092.stdout:8/967: mkdir d9/d21/d37/d46/d5c/d62/d14c 0 2024-05-06T12:14:48.289 INFO:tasks.workunit.client.0.smithi092.stdout:8/968: chown d9/d21/d37/d46/d5c/d62/d7a/c91 4086 1 2024-05-06T12:14:48.289 INFO:tasks.workunit.client.0.smithi092.stdout:3/802: read d8/db/d4c/d3d/d49/f8a [593458,28106] 0 2024-05-06T12:14:48.289 INFO:tasks.workunit.client.1.smithi190.stdout:2/409: write d2/d8/d19/d43/d79/d7e/f40 [2783606,30363] 0 2024-05-06T12:14:48.291 INFO:tasks.workunit.client.1.smithi190.stdout:9/553: link db/dd/d16/l42 db/d23/d29/lc2 0 2024-05-06T12:14:48.292 INFO:tasks.workunit.client.0.smithi092.stdout:8/969: unlink d9/dc/c2f 0 2024-05-06T12:14:48.292 INFO:tasks.workunit.client.1.smithi190.stdout:3/514: symlink d5/d7/lb6 0 2024-05-06T12:14:48.293 INFO:tasks.workunit.client.0.smithi092.stdout:3/803: truncate d8/db/fad 543625 0 2024-05-06T12:14:48.294 INFO:tasks.workunit.client.1.smithi190.stdout:3/515: read - d5/d14/d11/d21/d44/f5b zero size 2024-05-06T12:14:48.300 INFO:tasks.workunit.client.1.smithi190.stdout:3/516: creat d5/d14/d11/d21/d44/d92/fb7 x:0 0 0 2024-05-06T12:14:48.300 INFO:tasks.workunit.client.1.smithi190.stdout:9/554: getdents db/dd/d16/d4c 0 2024-05-06T12:14:48.303 INFO:tasks.workunit.client.1.smithi190.stdout:3/517: mknod d5/d7/d15/d1f/d33/cb8 0 2024-05-06T12:14:48.304 INFO:tasks.workunit.client.1.smithi190.stdout:9/555: mknod db/dd/d16/d58/d87/d22/d49/cc3 0 2024-05-06T12:14:48.307 INFO:tasks.workunit.client.1.smithi190.stdout:3/518: creat d5/d7/fb9 x:0 0 0 2024-05-06T12:14:48.311 INFO:tasks.workunit.client.1.smithi190.stdout:9/556: dread db/f77 [0,4194304] 0 2024-05-06T12:14:48.312 INFO:tasks.workunit.client.1.smithi190.stdout:3/519: rename d5/d7/d6d/c8e to d5/d14/d11/d21/d44/d92/d9e/da4/cba 0 2024-05-06T12:14:48.313 INFO:tasks.workunit.client.1.smithi190.stdout:3/520: fsync d5/d7/d15/d1f/d33/f76 0 2024-05-06T12:14:48.314 INFO:tasks.workunit.client.0.smithi092.stdout:1/984: dwrite d3/fb2 [0,4194304] 0 2024-05-06T12:14:48.314 INFO:tasks.workunit.client.1.smithi190.stdout:9/557: unlink db/d23/d29/lbc 0 2024-05-06T12:14:48.316 INFO:tasks.workunit.client.1.smithi190.stdout:1/367: write d5/d4f/d65/d24/d43/f60 [1118471,406] 0 2024-05-06T12:14:48.317 INFO:tasks.workunit.client.1.smithi190.stdout:9/558: write db/dd/d16/d4c/f9b [903255,120697] 0 2024-05-06T12:14:48.318 INFO:tasks.workunit.client.0.smithi092.stdout:8/970: dwrite d9/d21/d54/d79/fd8 [0,4194304] 0 2024-05-06T12:14:48.318 INFO:tasks.workunit.client.0.smithi092.stdout:1/985: rmdir d3/d10/d13/d19 39 2024-05-06T12:14:48.319 INFO:tasks.workunit.client.1.smithi190.stdout:3/521: creat d5/d14/d11/d21/d44/d80/d82/fbb x:0 0 0 2024-05-06T12:14:48.319 INFO:tasks.workunit.client.1.smithi190.stdout:0/532: dread d9/dc/d48/d50/f85 [0,4194304] 0 2024-05-06T12:14:48.321 INFO:tasks.workunit.client.1.smithi190.stdout:6/362: sync 2024-05-06T12:14:48.321 INFO:tasks.workunit.client.1.smithi190.stdout:5/483: sync 2024-05-06T12:14:48.322 INFO:tasks.workunit.client.1.smithi190.stdout:8/565: sync 2024-05-06T12:14:48.322 INFO:tasks.workunit.client.1.smithi190.stdout:7/356: sync 2024-05-06T12:14:48.323 INFO:tasks.workunit.client.0.smithi092.stdout:8/971: truncate d9/d21/d37/d46/fae 988810 0 2024-05-06T12:14:48.324 INFO:tasks.workunit.client.1.smithi190.stdout:9/559: unlink db/dd/d16/f24 0 2024-05-06T12:14:48.324 INFO:tasks.workunit.client.0.smithi092.stdout:1/986: mkdir d3/d42/d84/dcf/d13f 0 2024-05-06T12:14:48.325 INFO:tasks.workunit.client.1.smithi190.stdout:5/484: dread d2/da/d59/f77 [0,4194304] 0 2024-05-06T12:14:48.325 INFO:tasks.workunit.client.0.smithi092.stdout:3/804: dread d8/d24/d33/d4b/f46 [0,4194304] 0 2024-05-06T12:14:48.325 INFO:tasks.workunit.client.0.smithi092.stdout:1/987: chown d3/d42/d84/d9f 0 1 2024-05-06T12:14:48.326 INFO:tasks.workunit.client.1.smithi190.stdout:3/522: mkdir d5/d7/d15/d39/dbc 0 2024-05-06T12:14:48.326 INFO:tasks.workunit.client.1.smithi190.stdout:5/485: chown d2/da/d59/d5f 1578702475 1 2024-05-06T12:14:48.328 INFO:tasks.workunit.client.1.smithi190.stdout:0/533: mknod d9/dc/db6/cbf 0 2024-05-06T12:14:48.328 INFO:tasks.workunit.client.1.smithi190.stdout:2/410: truncate d2/d4c/f6f 2368833 0 2024-05-06T12:14:48.328 INFO:tasks.workunit.client.1.smithi190.stdout:6/363: dread df/d2c/f25 [0,4194304] 0 2024-05-06T12:14:48.329 INFO:tasks.workunit.client.1.smithi190.stdout:3/523: write d5/d14/d11/d21/d44/d80/d82/f8b [58890,67893] 0 2024-05-06T12:14:48.329 INFO:tasks.workunit.client.0.smithi092.stdout:1/988: write d3/d10/d13/d19/d21/d88/f124 [70563,119203] 0 2024-05-06T12:14:48.330 INFO:tasks.workunit.client.1.smithi190.stdout:9/560: write db/dd/d16/d58/d87/d22/d49/d8c/fa0 [642110,50168] 0 2024-05-06T12:14:48.330 INFO:tasks.workunit.client.0.smithi092.stdout:8/972: creat d9/d21/d54/f14d x:0 0 0 2024-05-06T12:14:48.331 INFO:tasks.workunit.client.0.smithi092.stdout:3/805: chown d8/d24/d27/l9c 19906567 1 2024-05-06T12:14:48.332 INFO:tasks.workunit.client.0.smithi092.stdout:1/989: stat d3/d10/d13/d19/d21/d72/d112 0 2024-05-06T12:14:48.333 INFO:tasks.workunit.client.0.smithi092.stdout:1/990: readlink d3/d10/d13/d46/lf6 0 2024-05-06T12:14:48.333 INFO:tasks.workunit.client.1.smithi190.stdout:2/411: write d2/d10/f2c [733649,57993] 0 2024-05-06T12:14:48.334 INFO:tasks.workunit.client.1.smithi190.stdout:3/524: write d5/d14/f6a [1622109,70642] 0 2024-05-06T12:14:48.334 INFO:tasks.workunit.client.1.smithi190.stdout:8/566: mkdir d3/d8/d25/d65/dc6 0 2024-05-06T12:14:48.334 INFO:tasks.workunit.client.1.smithi190.stdout:9/561: chown db/dd/d16/d4c/f57 14766425 1 2024-05-06T12:14:48.335 INFO:tasks.workunit.client.1.smithi190.stdout:3/525: write d5/d7/f3a [2703592,39617] 0 2024-05-06T12:14:48.335 INFO:tasks.workunit.client.0.smithi092.stdout:8/973: mkdir d9/d5a/d9f/d14e 0 2024-05-06T12:14:48.336 INFO:tasks.workunit.client.1.smithi190.stdout:9/562: stat db/d23/d29/l5d 0 2024-05-06T12:14:48.337 INFO:tasks.workunit.client.1.smithi190.stdout:6/364: creat df/d17/d44/f7d x:0 0 0 2024-05-06T12:14:48.337 INFO:tasks.workunit.client.0.smithi092.stdout:1/991: creat d3/d10/d13/d19/d21/d88/d105/f140 x:0 0 0 2024-05-06T12:14:48.338 INFO:tasks.workunit.client.1.smithi190.stdout:7/357: mkdir d0/d1/d18/d4e/d87 0 2024-05-06T12:14:48.339 INFO:tasks.workunit.client.0.smithi092.stdout:8/974: symlink d9/d21/d54/d116/d11f/d12e/l14f 0 2024-05-06T12:14:48.340 INFO:tasks.workunit.client.1.smithi190.stdout:6/365: chown df/d17/d44/f7d 21 1 2024-05-06T12:14:48.340 INFO:tasks.workunit.client.0.smithi092.stdout:1/992: mkdir d3/d42/d84/d4f/d7f/d141 0 2024-05-06T12:14:48.342 INFO:tasks.workunit.client.1.smithi190.stdout:1/368: link d5/d4f/d65/d24/c49 d5/d4f/d65/d32/d47/c7f 0 2024-05-06T12:14:48.343 INFO:tasks.workunit.client.0.smithi092.stdout:8/975: truncate d9/d21/d37/d46/ff1 311100 0 2024-05-06T12:14:48.343 INFO:tasks.workunit.client.1.smithi190.stdout:8/567: read d3/f52 [1264926,79194] 0 2024-05-06T12:14:48.344 INFO:tasks.workunit.client.1.smithi190.stdout:2/412: rename d2/d8/dc/c2d to d2/d8/d19/d7b/c86 0 2024-05-06T12:14:48.345 INFO:tasks.workunit.client.1.smithi190.stdout:6/366: write df/f22 [608666,19693] 0 2024-05-06T12:14:48.346 INFO:tasks.workunit.client.0.smithi092.stdout:8/976: creat d9/d21/d37/d46/d5c/d71/d75/f150 x:0 0 0 2024-05-06T12:14:48.347 INFO:tasks.workunit.client.1.smithi190.stdout:1/369: chown d5/d15/f17 15079 1 2024-05-06T12:14:48.348 INFO:tasks.workunit.client.1.smithi190.stdout:9/563: dwrite db/f52 [0,4194304] 0 2024-05-06T12:14:48.348 INFO:tasks.workunit.client.0.smithi092.stdout:3/806: truncate d8/db/d4c/d3d/fdd 796657 0 2024-05-06T12:14:48.351 INFO:tasks.workunit.client.0.smithi092.stdout:3/807: read d8/db/d4c/d3d/f8e [4780477,18116] 0 2024-05-06T12:14:48.352 INFO:tasks.workunit.client.1.smithi190.stdout:5/486: dread d2/da/d59/d5f/d2e/d3b/f52 [0,4194304] 0 2024-05-06T12:14:48.354 INFO:tasks.workunit.client.0.smithi092.stdout:3/808: symlink d8/db/d4c/d3d/d48/l103 0 2024-05-06T12:14:48.355 INFO:tasks.workunit.client.1.smithi190.stdout:8/568: rename d3/d8/d25/d3b/cc0 to d3/d35/cc7 0 2024-05-06T12:14:48.356 INFO:tasks.workunit.client.1.smithi190.stdout:0/534: dread d9/d6f/f7a [0,4194304] 0 2024-05-06T12:14:48.357 INFO:tasks.workunit.client.1.smithi190.stdout:9/564: mkdir db/d23/d29/dc4 0 2024-05-06T12:14:48.359 INFO:tasks.workunit.client.0.smithi092.stdout:1/993: dwrite d3/d42/d84/d4f/d58/d65/d81/f83 [0,4194304] 0 2024-05-06T12:14:48.359 INFO:tasks.workunit.client.1.smithi190.stdout:1/370: write d5/d4f/d65/d24/d3a/f72 [572417,185] 0 2024-05-06T12:14:48.360 INFO:tasks.workunit.client.1.smithi190.stdout:5/487: chown d2/da/d59/d5f/d2e/d3b/d5a/f85 53427 1 2024-05-06T12:14:48.361 INFO:tasks.workunit.client.0.smithi092.stdout:3/809: creat d8/dd7/f104 x:0 0 0 2024-05-06T12:14:48.362 INFO:tasks.workunit.client.1.smithi190.stdout:0/535: chown d9/dc/c89 115888 1 2024-05-06T12:14:48.362 INFO:tasks.workunit.client.1.smithi190.stdout:3/526: dread d5/d14/f8f [0,4194304] 0 2024-05-06T12:14:48.362 INFO:tasks.workunit.client.1.smithi190.stdout:8/569: creat d3/d13/fc8 x:0 0 0 2024-05-06T12:14:48.363 INFO:tasks.workunit.client.0.smithi092.stdout:1/994: fdatasync d3/d10/f115 0 2024-05-06T12:14:48.363 INFO:tasks.workunit.client.0.smithi092.stdout:8/977: dread d9/d21/d37/d46/d5c/d62/d7a/ffe [0,4194304] 0 2024-05-06T12:14:48.364 INFO:tasks.workunit.client.1.smithi190.stdout:7/358: dwrite d0/f22 [0,4194304] 0 2024-05-06T12:14:48.364 INFO:tasks.workunit.client.1.smithi190.stdout:4/346: sync 2024-05-06T12:14:48.367 INFO:tasks.workunit.client.1.smithi190.stdout:5/488: chown d2/da/d59/d5f/d2e/c79 774528811 1 2024-05-06T12:14:48.368 INFO:tasks.workunit.client.0.smithi092.stdout:1/995: mknod d3/d42/d84/d9f/c142 0 2024-05-06T12:14:48.368 INFO:tasks.workunit.client.0.smithi092.stdout:8/978: fsync d9/d21/d37/d74/ff0 0 2024-05-06T12:14:48.368 INFO:tasks.workunit.client.1.smithi190.stdout:0/536: truncate d9/dc/fb1 1494324 0 2024-05-06T12:14:48.369 INFO:tasks.workunit.client.1.smithi190.stdout:4/347: write d2/d10/ff [3469013,102419] 0 2024-05-06T12:14:48.369 INFO:tasks.workunit.client.0.smithi092.stdout:3/810: creat d8/db/d4c/d40/d7a/dbd/dea/f105 x:0 0 0 2024-05-06T12:14:48.369 INFO:tasks.workunit.client.0.smithi092.stdout:1/996: write d3/d10/d13/d19/fde [1509532,16237] 0 2024-05-06T12:14:48.369 INFO:tasks.workunit.client.1.smithi190.stdout:3/527: chown d5/d7/d15/d1f/d33 39186972 1 2024-05-06T12:14:48.371 INFO:tasks.workunit.client.1.smithi190.stdout:6/367: rename df/d2c/d1e/l5b to df/d2c/d1e/d57/d53/l7e 0 2024-05-06T12:14:48.373 INFO:tasks.workunit.client.0.smithi092.stdout:8/979: creat d9/d21/d37/dbd/dc5/d11c/f151 x:0 0 0 2024-05-06T12:14:48.374 INFO:tasks.workunit.client.1.smithi190.stdout:5/489: write d2/d27/f48 [2339022,49856] 0 2024-05-06T12:14:48.375 INFO:tasks.workunit.client.0.smithi092.stdout:2/816: sync 2024-05-06T12:14:48.375 INFO:tasks.workunit.client.1.smithi190.stdout:6/368: write df/d17/f60 [26026,90104] 0 2024-05-06T12:14:48.377 INFO:tasks.workunit.client.0.smithi092.stdout:3/811: creat d8/db/d4c/d40/f106 x:0 0 0 2024-05-06T12:14:48.378 INFO:tasks.workunit.client.1.smithi190.stdout:1/371: dread d5/f8 [0,4194304] 0 2024-05-06T12:14:48.379 INFO:tasks.workunit.client.1.smithi190.stdout:2/413: dread d2/f3b [0,4194304] 0 2024-05-06T12:14:48.379 INFO:tasks.workunit.client.0.smithi092.stdout:1/997: dwrite d3/fdd [0,4194304] 0 2024-05-06T12:14:48.379 INFO:tasks.workunit.client.0.smithi092.stdout:3/812: chown d8/d24/f65 4 1 2024-05-06T12:14:48.380 INFO:tasks.workunit.client.1.smithi190.stdout:9/565: read db/d23/d25/d47/d4b/f9e [161745,22069] 0 2024-05-06T12:14:48.381 INFO:tasks.workunit.client.1.smithi190.stdout:1/372: rename d5/d4f/d65 to d5/d4f/d65/d24/d3a/d80 22 2024-05-06T12:14:48.381 INFO:tasks.workunit.client.1.smithi190.stdout:4/348: creat d2/d11/d16/d61/f7c x:0 0 0 2024-05-06T12:14:48.384 INFO:tasks.workunit.client.1.smithi190.stdout:3/528: creat d5/d7/d15/d1f/d84/fbd x:0 0 0 2024-05-06T12:14:48.384 INFO:tasks.workunit.client.0.smithi092.stdout:1/998: symlink d3/d42/d84/d4f/dd5/ded/l143 0 2024-05-06T12:14:48.385 INFO:tasks.workunit.client.1.smithi190.stdout:8/570: fsync d3/d8/fd 0 2024-05-06T12:14:48.386 INFO:tasks.workunit.client.1.smithi190.stdout:4/349: dread d2/f37 [0,4194304] 0 2024-05-06T12:14:48.386 INFO:tasks.workunit.client.0.smithi092.stdout:3/813: truncate d8/db/d4c/d3d/d49/df8/d5b/d98/fb2 987315 0 2024-05-06T12:14:48.387 INFO:tasks.workunit.client.1.smithi190.stdout:8/571: write d3/d8/d25/f39 [525986,100930] 0 2024-05-06T12:14:48.388 INFO:tasks.workunit.client.0.smithi092.stdout:3/814: truncate d8/d24/d33/d4b/ff9 483938 0 2024-05-06T12:14:48.388 INFO:tasks.workunit.client.0.smithi092.stdout:1/999: read d3/f98 [3756952,21170] 0 2024-05-06T12:14:48.389 INFO:tasks.workunit.client.1.smithi190.stdout:7/359: fsync d0/d1/d2/f5 0 2024-05-06T12:14:48.392 INFO:tasks.workunit.client.1.smithi190.stdout:6/369: creat df/d2c/d28/f7f x:0 0 0 2024-05-06T12:14:48.393 INFO:tasks.workunit.client.1.smithi190.stdout:2/414: creat d2/d10/f87 x:0 0 0 2024-05-06T12:14:48.395 INFO:tasks.workunit.client.1.smithi190.stdout:2/415: chown d2/d21/c49 9345228 1 2024-05-06T12:14:48.395 INFO:tasks.workunit.client.1.smithi190.stdout:9/566: mknod db/dd/d16/d58/d87/d22/d49/d88/d76/cc5 0 2024-05-06T12:14:48.397 INFO:tasks.workunit.client.1.smithi190.stdout:2/416: write d2/d10/f87 [805818,79242] 0 2024-05-06T12:14:48.397 INFO:tasks.workunit.client.1.smithi190.stdout:8/572: dread d3/d8/d2b/f34 [0,4194304] 0 2024-05-06T12:14:48.398 INFO:tasks.workunit.client.1.smithi190.stdout:9/567: write db/dd/d16/d58/d87/d22/d49/d8c/fa0 [1281678,13366] 0 2024-05-06T12:14:48.399 INFO:tasks.workunit.client.1.smithi190.stdout:4/350: unlink d2/d11/c15 0 2024-05-06T12:14:48.400 INFO:tasks.workunit.client.1.smithi190.stdout:8/573: dread d3/d8/d2b/d3e/f54 [0,4194304] 0 2024-05-06T12:14:48.401 INFO:tasks.workunit.client.0.smithi092.stdout:3/815: truncate d8/db/d4c/d40/f52 853809 0 2024-05-06T12:14:48.401 INFO:tasks.workunit.client.1.smithi190.stdout:8/574: readlink d3/d8/d25/lb4 0 2024-05-06T12:14:48.402 INFO:tasks.workunit.client.1.smithi190.stdout:5/490: mknod d2/da/d59/d5f/d2e/d3b/d72/c96 0 2024-05-06T12:14:48.402 INFO:tasks.workunit.client.0.smithi092.stdout:3/816: write d8/db/d4c/d40/f106 [423454,47505] 0 2024-05-06T12:14:48.403 INFO:tasks.workunit.client.1.smithi190.stdout:7/360: creat d0/d1/d2/f88 x:0 0 0 2024-05-06T12:14:48.404 INFO:tasks.workunit.client.0.smithi092.stdout:8/980: dread d9/d21/d37/d46/d5c/d71/d75/fef [0,4194304] 0 2024-05-06T12:14:48.405 INFO:tasks.workunit.client.1.smithi190.stdout:6/370: symlink df/d2c/d61/l80 0 2024-05-06T12:14:48.406 INFO:tasks.workunit.client.0.smithi092.stdout:3/817: mkdir d8/dd7/d107 0 2024-05-06T12:14:48.406 INFO:tasks.workunit.client.0.smithi092.stdout:2/817: dwrite df/d17/d53/da9/dd9/d83/f84 [0,4194304] 0 2024-05-06T12:14:48.406 INFO:tasks.workunit.client.1.smithi190.stdout:1/373: mknod d5/c81 0 2024-05-06T12:14:48.410 INFO:tasks.workunit.client.1.smithi190.stdout:1/374: dread - d5/d15/f4e zero size 2024-05-06T12:14:48.411 INFO:tasks.workunit.client.1.smithi190.stdout:7/361: dwrite d0/d14/f6e [0,4194304] 0 2024-05-06T12:14:48.412 INFO:tasks.workunit.client.0.smithi092.stdout:8/981: unlink d9/d21/d37/dbd/dc5/d11c/ff5 0 2024-05-06T12:14:48.413 INFO:tasks.workunit.client.0.smithi092.stdout:2/818: mkdir df/d17/d7d/df0 0 2024-05-06T12:14:48.414 INFO:tasks.workunit.client.0.smithi092.stdout:8/982: mkdir d9/d21/d54/d79/d8a/d152 0 2024-05-06T12:14:48.416 INFO:tasks.workunit.client.0.smithi092.stdout:8/983: write d9/d21/d54/d79/f82 [1860167,58774] 0 2024-05-06T12:14:48.420 INFO:tasks.workunit.client.1.smithi190.stdout:4/351: dread d2/d10/ff [0,4194304] 0 2024-05-06T12:14:48.420 INFO:tasks.workunit.client.1.smithi190.stdout:4/352: stat d2 0 2024-05-06T12:14:48.421 INFO:tasks.workunit.client.0.smithi092.stdout:8/984: mknod d9/d21/d37/d46/d5c/d62/da5/de6/c153 0 2024-05-06T12:14:48.426 INFO:tasks.workunit.client.0.smithi092.stdout:8/985: rename d9/d21/d37/f3b to d9/d21/d37/d46/d5c/d62/f154 0 2024-05-06T12:14:48.427 INFO:tasks.workunit.client.0.smithi092.stdout:2/819: truncate df/d17/d51/f70 286084 0 2024-05-06T12:14:48.428 INFO:tasks.workunit.client.0.smithi092.stdout:2/820: getdents df/d17/d7d/df0 0 2024-05-06T12:14:48.429 INFO:tasks.workunit.client.0.smithi092.stdout:2/821: fsync df/dd0/de3/dcf/f81 0 2024-05-06T12:14:48.432 INFO:tasks.workunit.client.0.smithi092.stdout:2/822: truncate df/d17/d51/d96/fae 3753887 0 2024-05-06T12:14:48.434 INFO:tasks.workunit.client.0.smithi092.stdout:2/823: write df/d15/f95 [936678,63154] 0 2024-05-06T12:14:48.434 INFO:tasks.workunit.client.0.smithi092.stdout:8/986: dread d9/d21/d37/d46/d5c/d62/f68 [0,4194304] 0 2024-05-06T12:14:48.434 INFO:tasks.workunit.client.0.smithi092.stdout:3/818: dread f4 [0,4194304] 0 2024-05-06T12:14:48.435 INFO:tasks.workunit.client.0.smithi092.stdout:2/824: chown df/d17/d7e/fdf 0 1 2024-05-06T12:14:48.437 INFO:tasks.workunit.client.0.smithi092.stdout:8/987: creat d9/dc/d4a/d5f/da8/f155 x:0 0 0 2024-05-06T12:14:48.438 INFO:tasks.workunit.client.0.smithi092.stdout:2/825: creat df/d17/d1d/dc8/ff1 x:0 0 0 2024-05-06T12:14:48.440 INFO:tasks.workunit.client.0.smithi092.stdout:2/826: symlink df/d17/d53/lf2 0 2024-05-06T12:14:48.442 INFO:tasks.workunit.client.0.smithi092.stdout:3/819: dwrite d8/db/d14/fd9 [0,4194304] 0 2024-05-06T12:14:48.443 INFO:tasks.workunit.client.0.smithi092.stdout:3/820: chown d8/d1b/fd5 12393 1 2024-05-06T12:14:48.444 INFO:tasks.workunit.client.0.smithi092.stdout:2/827: unlink df/d17/d7d/fb2 0 2024-05-06T12:14:48.446 INFO:tasks.workunit.client.0.smithi092.stdout:3/821: chown d8/d24/d33/d4b/cf 0 1 2024-05-06T12:14:48.447 INFO:tasks.workunit.client.0.smithi092.stdout:2/828: rmdir df/dd0/de3/db4/dce 39 2024-05-06T12:14:48.449 INFO:tasks.workunit.client.0.smithi092.stdout:3/822: symlink d8/d24/d33/d91/l108 0 2024-05-06T12:14:48.452 INFO:tasks.workunit.client.0.smithi092.stdout:3/823: rmdir d8/db/d14/d5a/db0 39 2024-05-06T12:14:48.454 INFO:tasks.workunit.client.0.smithi092.stdout:3/824: write d8/db/d4c/d40/d7a/fa3 [1630644,104503] 0 2024-05-06T12:14:48.454 INFO:tasks.workunit.client.1.smithi190.stdout:0/537: write d9/d51/f64 [2447142,126843] 0 2024-05-06T12:14:48.458 INFO:tasks.workunit.client.0.smithi092.stdout:3/825: creat d8/db/d4c/dc1/f109 x:0 0 0 2024-05-06T12:14:48.459 INFO:tasks.workunit.client.1.smithi190.stdout:5/491: symlink d2/d65/l97 0 2024-05-06T12:14:48.459 INFO:tasks.workunit.client.1.smithi190.stdout:0/538: dread d9/df/d21/f4a [0,4194304] 0 2024-05-06T12:14:48.460 INFO:tasks.workunit.client.1.smithi190.stdout:1/375: creat d5/d15/f82 x:0 0 0 2024-05-06T12:14:48.461 INFO:tasks.workunit.client.1.smithi190.stdout:7/362: symlink d0/d4/l89 0 2024-05-06T12:14:48.462 INFO:tasks.workunit.client.0.smithi092.stdout:2/829: write df/d17/d7d/fa1 [236319,52586] 0 2024-05-06T12:14:48.465 INFO:tasks.workunit.client.1.smithi190.stdout:6/371: mknod df/d2c/d61/d73/c81 0 2024-05-06T12:14:48.466 INFO:tasks.workunit.client.0.smithi092.stdout:2/830: symlink df/d17/d7d/lf3 0 2024-05-06T12:14:48.466 INFO:tasks.workunit.client.0.smithi092.stdout:3/826: dwrite d8/dd7/f104 [0,4194304] 0 2024-05-06T12:14:48.467 INFO:tasks.workunit.client.1.smithi190.stdout:5/492: mknod d2/d27/c98 0 2024-05-06T12:14:48.469 INFO:tasks.workunit.client.1.smithi190.stdout:1/376: creat d5/d4f/f83 x:0 0 0 2024-05-06T12:14:48.473 INFO:tasks.workunit.client.0.smithi092.stdout:2/831: creat df/d17/d53/da9/dd9/d83/ff4 x:0 0 0 2024-05-06T12:14:48.473 INFO:tasks.workunit.client.1.smithi190.stdout:3/529: getdents d5 0 2024-05-06T12:14:48.474 INFO:tasks.workunit.client.0.smithi092.stdout:3/827: write d8/db/d4c/d3d/f60 [576161,123] 0 2024-05-06T12:14:48.477 INFO:tasks.workunit.client.1.smithi190.stdout:0/539: mknod d9/dc/d28/cc0 0 2024-05-06T12:14:48.479 INFO:tasks.workunit.client.0.smithi092.stdout:3/828: mkdir d8/d24/d27/d10a 0 2024-05-06T12:14:48.479 INFO:tasks.workunit.client.1.smithi190.stdout:5/493: rename d2/da/d59/d5f/d2e/f87 to d2/d6c/f99 0 2024-05-06T12:14:48.481 INFO:tasks.workunit.client.0.smithi092.stdout:2/832: dwrite df/d17/d53/da9/dd9/d83/ff4 [0,4194304] 0 2024-05-06T12:14:48.482 INFO:tasks.workunit.client.0.smithi092.stdout:3/829: readlink d8/d1b/d73/db7/lf7 0 2024-05-06T12:14:48.483 INFO:tasks.workunit.client.1.smithi190.stdout:4/353: fsync d2/d11/d16/d41/f62 0 2024-05-06T12:14:48.484 INFO:tasks.workunit.client.0.smithi092.stdout:2/833: creat df/d15/ff5 x:0 0 0 2024-05-06T12:14:48.485 INFO:tasks.workunit.client.1.smithi190.stdout:4/354: write d2/d11/d16/d4e/f6f [795834,117418] 0 2024-05-06T12:14:48.485 INFO:tasks.workunit.client.0.smithi092.stdout:2/834: chown df/dd0 1242562936 1 2024-05-06T12:14:48.487 INFO:tasks.workunit.client.1.smithi190.stdout:9/568: getdents db/daa 0 2024-05-06T12:14:48.490 INFO:tasks.workunit.client.0.smithi092.stdout:3/830: rename d8/d24/d27/l9c to d8/db/d4c/d40/l10b 0 2024-05-06T12:14:48.490 INFO:tasks.workunit.client.1.smithi190.stdout:9/569: stat db/dd/d16/d58/d87/d22/d49/d88/db5/fb9 0 2024-05-06T12:14:48.491 INFO:tasks.workunit.client.1.smithi190.stdout:4/355: dread d2/d10/f1c [0,4194304] 0 2024-05-06T12:14:48.493 INFO:tasks.workunit.client.0.smithi092.stdout:3/831: write d8/d24/d27/f2f [1626371,101616] 0 2024-05-06T12:14:48.494 INFO:tasks.workunit.client.0.smithi092.stdout:3/832: write d8/db/d14/f50 [7698257,3351] 0 2024-05-06T12:14:48.495 INFO:tasks.workunit.client.1.smithi190.stdout:2/417: dwrite d2/fe [0,4194304] 0 2024-05-06T12:14:48.495 INFO:tasks.workunit.client.0.smithi092.stdout:2/835: getdents df/dd0/de3/da5 0 2024-05-06T12:14:48.496 INFO:tasks.workunit.client.1.smithi190.stdout:1/377: rmdir d5/d4f 39 2024-05-06T12:14:48.498 INFO:tasks.workunit.client.0.smithi092.stdout:2/836: mknod df/dd0/de3/d8e/dbb/cf6 0 2024-05-06T12:14:48.498 INFO:tasks.workunit.client.1.smithi190.stdout:2/418: dread - d2/f62 zero size 2024-05-06T12:14:48.500 INFO:tasks.workunit.client.1.smithi190.stdout:5/494: mkdir d2/da/d59/d5f/d2e/d3b/d76/d9a 0 2024-05-06T12:14:48.500 INFO:tasks.workunit.client.1.smithi190.stdout:6/372: creat df/f82 x:0 0 0 2024-05-06T12:14:48.502 INFO:tasks.workunit.client.0.smithi092.stdout:3/833: rename d8/db/d4c/dc1/f109 to d8/db/d4c/d3d/f10c 0 2024-05-06T12:14:48.502 INFO:tasks.workunit.client.1.smithi190.stdout:9/570: unlink db/d23/f53 0 2024-05-06T12:14:48.503 INFO:tasks.workunit.client.0.smithi092.stdout:3/834: fdatasync d8/d24/d27/f2f 0 2024-05-06T12:14:48.505 INFO:tasks.workunit.client.1.smithi190.stdout:6/373: write df/d17/f60 [431721,111159] 0 2024-05-06T12:14:48.506 INFO:tasks.workunit.client.0.smithi092.stdout:2/837: dwrite df/d17/d53/feb [0,4194304] 0 2024-05-06T12:14:48.506 INFO:tasks.workunit.client.1.smithi190.stdout:9/571: write f6 [2359579,100521] 0 2024-05-06T12:14:48.508 INFO:tasks.workunit.client.0.smithi092.stdout:3/835: mknod d8/db/d4c/d3d/d49/ded/c10d 0 2024-05-06T12:14:48.509 INFO:tasks.workunit.client.1.smithi190.stdout:4/356: mkdir d2/d7d 0 2024-05-06T12:14:48.509 INFO:tasks.workunit.client.1.smithi190.stdout:6/374: stat df/d2c/d1e/d57/d53/l5d 0 2024-05-06T12:14:48.509 INFO:tasks.workunit.client.1.smithi190.stdout:7/363: dwrite d0/f20 [0,4194304] 0 2024-05-06T12:14:48.510 INFO:tasks.workunit.client.0.smithi092.stdout:3/836: fsync d8/db/d4c/d40/d7a/dbd/dea/f105 0 2024-05-06T12:14:48.511 INFO:tasks.workunit.client.0.smithi092.stdout:2/838: rename df/d15/f30 to df/dd0/de3/dcf/ff7 0 2024-05-06T12:14:48.514 INFO:tasks.workunit.client.0.smithi092.stdout:3/837: unlink d8/db/d4c/d3d/d49/df8/f99 0 2024-05-06T12:14:48.516 INFO:tasks.workunit.client.1.smithi190.stdout:6/375: write df/d17/f5c [3882201,70057] 0 2024-05-06T12:14:48.516 INFO:tasks.workunit.client.1.smithi190.stdout:3/530: write d5/d14/f24 [982032,130754] 0 2024-05-06T12:14:48.517 INFO:tasks.workunit.client.0.smithi092.stdout:3/838: fdatasync d8/d24/d33/d4b/ddf/feb 0 2024-05-06T12:14:48.518 INFO:tasks.workunit.client.0.smithi092.stdout:3/839: stat d8/db/d4c/d3d/ld1 0 2024-05-06T12:14:48.519 INFO:tasks.workunit.client.1.smithi190.stdout:0/540: dwrite d9/dc/f45 [0,4194304] 0 2024-05-06T12:14:48.520 INFO:tasks.workunit.client.1.smithi190.stdout:1/378: dread d5/d4f/d65/d24/d3a/f73 [4194304,4194304] 0 2024-05-06T12:14:48.522 INFO:tasks.workunit.client.1.smithi190.stdout:2/419: rename d2/d8/d19/d43/d79/d7e/f52 to d2/d8/d71/f88 0 2024-05-06T12:14:48.523 INFO:tasks.workunit.client.0.smithi092.stdout:3/840: creat d8/d24/d33/d4b/ddf/f10e x:0 0 0 2024-05-06T12:14:48.523 INFO:tasks.workunit.client.1.smithi190.stdout:7/364: creat d0/d1/d18/f8a x:0 0 0 2024-05-06T12:14:48.526 INFO:tasks.workunit.client.1.smithi190.stdout:6/376: creat df/d12/f83 x:0 0 0 2024-05-06T12:14:48.527 INFO:tasks.workunit.client.1.smithi190.stdout:6/377: chown df/d2c/d1e/d57 3 1 2024-05-06T12:14:48.527 INFO:tasks.workunit.client.1.smithi190.stdout:6/378: chown df/f22 510 1 2024-05-06T12:14:48.528 INFO:tasks.workunit.client.1.smithi190.stdout:9/572: creat db/d23/d29/fc6 x:0 0 0 2024-05-06T12:14:48.532 INFO:tasks.workunit.client.1.smithi190.stdout:1/379: creat d5/d4f/d65/d32/d47/d53/d6e/f84 x:0 0 0 2024-05-06T12:14:48.532 INFO:tasks.workunit.client.0.smithi092.stdout:2/839: write df/d17/d51/f9c [288881,41330] 0 2024-05-06T12:14:48.533 INFO:tasks.workunit.client.1.smithi190.stdout:4/357: mknod d2/d11/d16/d61/d67/d6e/d78/c7e 0 2024-05-06T12:14:48.535 INFO:tasks.workunit.client.0.smithi092.stdout:3/841: write d8/db/d4c/d3d/d49/f8a [2085244,97947] 0 2024-05-06T12:14:48.541 INFO:tasks.workunit.client.0.smithi092.stdout:2/840: dwrite df/dd0/fdd [0,4194304] 0 2024-05-06T12:14:48.544 INFO:tasks.workunit.client.0.smithi092.stdout:3/842: dwrite d8/d24/d27/f2f [4194304,4194304] 0 2024-05-06T12:14:48.548 INFO:tasks.workunit.client.0.smithi092.stdout:3/843: rmdir d8/db/d4c/d40/d7a/dbd/dea 39 2024-05-06T12:14:48.558 INFO:tasks.workunit.client.1.smithi190.stdout:1/380: symlink d5/d4f/d65/d24/d3a/l85 0 2024-05-06T12:14:48.559 INFO:tasks.workunit.client.1.smithi190.stdout:5/495: link d2/da/d59/d5f/d2e/l46 d2/l9b 0 2024-05-06T12:14:48.559 INFO:tasks.workunit.client.1.smithi190.stdout:7/365: creat d0/d1/d18/d4e/d87/f8b x:0 0 0 2024-05-06T12:14:48.559 INFO:tasks.workunit.client.1.smithi190.stdout:9/573: fdatasync db/d23/d7e/fa8 0 2024-05-06T12:14:48.560 INFO:tasks.workunit.client.0.smithi092.stdout:2/841: write df/dd0/fda [492666,86685] 0 2024-05-06T12:14:48.561 INFO:tasks.workunit.client.1.smithi190.stdout:1/381: truncate d5/d4f/d65/d24/d3a/f72 1554444 0 2024-05-06T12:14:48.561 INFO:tasks.workunit.client.1.smithi190.stdout:4/358: unlink d2/d10/f1c 0 2024-05-06T12:14:48.565 INFO:tasks.workunit.client.0.smithi092.stdout:3/844: dwrite d8/db/d4c/d40/d67/f71 [0,4194304] 0 2024-05-06T12:14:48.567 INFO:tasks.workunit.client.1.smithi190.stdout:4/359: unlink d2/d11/d16/d61/l79 0 2024-05-06T12:14:48.567 INFO:tasks.workunit.client.0.smithi092.stdout:3/845: symlink d8/d24/l10f 0 2024-05-06T12:14:48.568 INFO:tasks.workunit.client.1.smithi190.stdout:1/382: creat d5/d4f/d65/d32/d47/d53/d6e/f86 x:0 0 0 2024-05-06T12:14:48.570 INFO:tasks.workunit.client.0.smithi092.stdout:3/846: symlink d8/db/d4c/d3d/d48/l110 0 2024-05-06T12:14:48.571 INFO:tasks.workunit.client.0.smithi092.stdout:3/847: readlink d8/d24/d27/la4 0 2024-05-06T12:14:48.572 INFO:tasks.workunit.client.1.smithi190.stdout:9/574: mknod db/dd/d5c/dac/cc7 0 2024-05-06T12:14:48.572 INFO:tasks.workunit.client.1.smithi190.stdout:7/366: dwrite d0/da/d36/f46 [0,4194304] 0 2024-05-06T12:14:48.573 INFO:tasks.workunit.client.0.smithi092.stdout:3/848: fdatasync d8/db/fad 0 2024-05-06T12:14:48.574 INFO:tasks.workunit.client.1.smithi190.stdout:4/360: symlink d2/d10/l7f 0 2024-05-06T12:14:48.574 INFO:tasks.workunit.client.0.smithi092.stdout:3/849: dread - d8/d1b/fd5 zero size 2024-05-06T12:14:48.577 INFO:tasks.workunit.client.0.smithi092.stdout:3/850: mkdir d8/db/d4c/d3d/d111 0 2024-05-06T12:14:48.578 INFO:tasks.workunit.client.1.smithi190.stdout:3/531: dwrite d5/d14/d11/d21/d44/d80/d82/d3d/f6c [0,4194304] 0 2024-05-06T12:14:48.579 INFO:tasks.workunit.client.0.smithi092.stdout:3/851: write d8/d24/d33/d4b/ff9 [171138,21682] 0 2024-05-06T12:14:48.579 INFO:tasks.workunit.client.1.smithi190.stdout:0/541: dread d9/d51/fa3 [0,4194304] 0 2024-05-06T12:14:48.579 INFO:tasks.workunit.client.1.smithi190.stdout:6/379: dread df/d2c/d1e/d57/f6b [0,4194304] 0 2024-05-06T12:14:48.582 INFO:tasks.workunit.client.1.smithi190.stdout:9/575: rmdir db/dd/d16/d58 39 2024-05-06T12:14:48.585 INFO:tasks.workunit.client.0.smithi092.stdout:3/852: rename d8/db/d4c/d40/d7a/dbd to d8/d1b/d112 0 2024-05-06T12:14:48.588 INFO:tasks.workunit.client.0.smithi092.stdout:3/853: unlink d8/d24/d33/d4b/cf 0 2024-05-06T12:14:48.588 INFO:tasks.workunit.client.1.smithi190.stdout:1/383: rename d5/d4f/d65/d32/d47/c7f to d5/d15/d1e/c87 0 2024-05-06T12:14:48.590 INFO:tasks.workunit.client.0.smithi092.stdout:3/854: rmdir d8/db/d4c/d40/d67 39 2024-05-06T12:14:48.591 INFO:tasks.workunit.client.0.smithi092.stdout:3/855: stat d8/db/d4c/cbf 0 2024-05-06T12:14:48.592 INFO:tasks.workunit.client.1.smithi190.stdout:5/496: write d2/da/f80 [3715224,38044] 0 2024-05-06T12:14:48.592 INFO:tasks.workunit.client.1.smithi190.stdout:6/380: dwrite df/f15 [0,4194304] 0 2024-05-06T12:14:48.595 INFO:tasks.workunit.client.1.smithi190.stdout:3/532: mknod d5/d7/d15/cbe 0 2024-05-06T12:14:48.596 INFO:tasks.workunit.client.1.smithi190.stdout:9/576: rmdir db/d23/d7e 39 2024-05-06T12:14:48.598 INFO:tasks.workunit.client.1.smithi190.stdout:7/367: mkdir d0/da/d36/d53/d77/d8c 0 2024-05-06T12:14:48.602 INFO:tasks.workunit.client.1.smithi190.stdout:7/368: read - d0/d1/d18/d4e/d87/f8b zero size 2024-05-06T12:14:48.602 INFO:tasks.workunit.client.1.smithi190.stdout:5/497: symlink d2/da/d59/l9c 0 2024-05-06T12:14:48.603 INFO:tasks.workunit.client.1.smithi190.stdout:4/361: dwrite d2/d11/d16/d31/f56 [0,4194304] 0 2024-05-06T12:14:48.603 INFO:tasks.workunit.client.1.smithi190.stdout:3/533: dread d5/d7/d15/d1f/d33/f76 [0,4194304] 0 2024-05-06T12:14:48.603 INFO:tasks.workunit.client.1.smithi190.stdout:7/369: chown d0/d1/d18 1203 1 2024-05-06T12:14:48.603 INFO:tasks.workunit.client.1.smithi190.stdout:1/384: symlink d5/d4f/d65/d62/l88 0 2024-05-06T12:14:48.604 INFO:tasks.workunit.client.1.smithi190.stdout:6/381: mkdir df/d2c/d61/d84 0 2024-05-06T12:14:48.606 INFO:tasks.workunit.client.1.smithi190.stdout:4/362: fdatasync d2/f55 0 2024-05-06T12:14:48.613 INFO:tasks.workunit.client.1.smithi190.stdout:6/382: read f1 [1076442,8544] 0 2024-05-06T12:14:48.613 INFO:tasks.workunit.client.0.smithi092.stdout:3/856: write d8/db/d4c/d3d/f8e [4309101,125651] 0 2024-05-06T12:14:48.613 INFO:tasks.workunit.client.1.smithi190.stdout:0/542: write d9/df/d56/dab/fba [404222,121303] 0 2024-05-06T12:14:48.618 INFO:tasks.workunit.client.1.smithi190.stdout:7/370: dwrite d0/d14/f7b [0,4194304] 0 2024-05-06T12:14:48.620 INFO:tasks.workunit.client.0.smithi092.stdout:3/857: link d8/db/d4c/d3d/f60 d8/db/d4c/d40/f113 0 2024-05-06T12:14:48.624 INFO:tasks.workunit.client.1.smithi190.stdout:9/577: truncate db/f13 3262732 0 2024-05-06T12:14:48.625 INFO:tasks.workunit.client.1.smithi190.stdout:4/363: mkdir d2/d11/d16/d53/d80 0 2024-05-06T12:14:48.626 INFO:tasks.workunit.client.1.smithi190.stdout:1/385: symlink d5/l89 0 2024-05-06T12:14:48.627 INFO:tasks.workunit.client.1.smithi190.stdout:1/386: readlink d5/d15/d1e/l41 0 2024-05-06T12:14:48.627 INFO:tasks.workunit.client.1.smithi190.stdout:6/383: dread df/f2e [4194304,4194304] 0 2024-05-06T12:14:48.628 INFO:tasks.workunit.client.1.smithi190.stdout:6/384: chown df/d17/d44 11070096 1 2024-05-06T12:14:48.628 INFO:tasks.workunit.client.1.smithi190.stdout:5/498: mkdir d2/da/d59/d5f/d6e/d95/d9d 0 2024-05-06T12:14:48.630 INFO:tasks.workunit.client.1.smithi190.stdout:0/543: mknod d9/dc/d28/d77/cc1 0 2024-05-06T12:14:48.631 INFO:tasks.workunit.client.0.smithi092.stdout:3/858: link d8/d24/d33/d91/cd8 d8/dd7/d107/c114 0 2024-05-06T12:14:48.631 INFO:tasks.workunit.client.1.smithi190.stdout:7/371: symlink d0/da/d36/d53/l8d 0 2024-05-06T12:14:48.634 INFO:tasks.workunit.client.1.smithi190.stdout:8/575: sync 2024-05-06T12:14:48.635 INFO:tasks.workunit.client.1.smithi190.stdout:1/387: rename d5/f8 to d5/d6b/f8a 0 2024-05-06T12:14:48.636 INFO:tasks.workunit.client.1.smithi190.stdout:1/388: write d5/f12 [428318,84400] 0 2024-05-06T12:14:48.638 INFO:tasks.workunit.client.1.smithi190.stdout:3/534: link d5/d7/d15/d1f/f88 d5/d14/d11/d21/d44/fbf 0 2024-05-06T12:14:48.640 INFO:tasks.workunit.client.0.smithi092.stdout:3/859: truncate d8/db/d4c/d40/d67/f71 1404581 0 2024-05-06T12:14:48.644 INFO:tasks.workunit.client.1.smithi190.stdout:1/389: dwrite d5/d4f/d65/d32/d47/f51 [0,4194304] 0 2024-05-06T12:14:48.645 INFO:tasks.workunit.client.0.smithi092.stdout:3/860: creat d8/db/d4c/d3d/d49/df8/d5b/d101/f115 x:0 0 0 2024-05-06T12:14:48.647 INFO:tasks.workunit.client.0.smithi092.stdout:3/861: mknod d8/db/d4c/d40/c116 0 2024-05-06T12:14:48.649 INFO:tasks.workunit.client.1.smithi190.stdout:6/385: truncate df/d17/d37/f49 4292471 0 2024-05-06T12:14:48.652 INFO:tasks.workunit.client.1.smithi190.stdout:0/544: symlink d9/dc/d28/lc2 0 2024-05-06T12:14:48.654 INFO:tasks.workunit.client.1.smithi190.stdout:4/364: write d2/f37 [614695,30197] 0 2024-05-06T12:14:48.658 INFO:tasks.workunit.client.1.smithi190.stdout:1/390: mkdir d5/d4f/d65/d32/d47/d53/d8b 0 2024-05-06T12:14:48.659 INFO:tasks.workunit.client.1.smithi190.stdout:5/499: write d2/f44 [435347,86496] 0 2024-05-06T12:14:48.662 INFO:tasks.workunit.client.1.smithi190.stdout:3/535: fsync d5/d7/f90 0 2024-05-06T12:14:48.663 INFO:tasks.workunit.client.1.smithi190.stdout:3/536: readlink d5/d7/d15/d39/l70 0 2024-05-06T12:14:48.664 INFO:tasks.workunit.client.1.smithi190.stdout:7/372: rmdir d0/da/d3d 0 2024-05-06T12:14:48.664 INFO:tasks.workunit.client.1.smithi190.stdout:3/537: read - d5/d7/fb9 zero size 2024-05-06T12:14:48.668 INFO:tasks.workunit.client.0.smithi092.stdout:3/862: dwrite d8/db/d4c/d40/d67/f71 [0,4194304] 0 2024-05-06T12:14:48.669 INFO:tasks.workunit.client.1.smithi190.stdout:5/500: mknod d2/da/d59/c9e 0 2024-05-06T12:14:48.674 INFO:tasks.workunit.client.1.smithi190.stdout:0/545: symlink d9/dc/d28/d4c/d80/da7/lc3 0 2024-05-06T12:14:48.674 INFO:tasks.workunit.client.1.smithi190.stdout:9/578: dwrite db/dd/d16/d58/d87/d22/f54 [0,4194304] 0 2024-05-06T12:14:48.676 INFO:tasks.workunit.client.0.smithi092.stdout:3/863: dwrite d8/d1b/d112/dea/f105 [0,4194304] 0 2024-05-06T12:14:48.676 INFO:tasks.workunit.client.1.smithi190.stdout:4/365: mkdir d2/d11/d16/d41/d81 0 2024-05-06T12:14:48.678 INFO:tasks.workunit.client.1.smithi190.stdout:1/391: mkdir d5/d4f/d65/d24/d8c 0 2024-05-06T12:14:48.680 INFO:tasks.workunit.client.1.smithi190.stdout:5/501: rmdir d2/da/d59/d5f/d50/d6d 39 2024-05-06T12:14:48.682 INFO:tasks.workunit.client.0.smithi092.stdout:3/864: dread f4 [0,4194304] 0 2024-05-06T12:14:48.684 INFO:tasks.workunit.client.1.smithi190.stdout:7/373: mknod d0/d4c/d60/c8e 0 2024-05-06T12:14:48.685 INFO:tasks.workunit.client.1.smithi190.stdout:5/502: dread d2/f5 [0,4194304] 0 2024-05-06T12:14:48.686 INFO:tasks.workunit.client.1.smithi190.stdout:0/546: mknod d9/dc/d48/d50/d87/cc4 0 2024-05-06T12:14:48.689 INFO:tasks.workunit.client.1.smithi190.stdout:4/366: chown d2/d11/l2d 375 1 2024-05-06T12:14:48.689 INFO:tasks.workunit.client.1.smithi190.stdout:9/579: chown db/d23/d25/d47/d4b/f9e 6381383 1 2024-05-06T12:14:48.689 INFO:tasks.workunit.client.1.smithi190.stdout:0/547: write d9/dc/d28/d4c/d96/fa0 [950517,41603] 0 2024-05-06T12:14:48.689 INFO:tasks.workunit.client.1.smithi190.stdout:1/392: creat d5/d4f/d65/d62/f8d x:0 0 0 2024-05-06T12:14:48.692 INFO:tasks.workunit.client.1.smithi190.stdout:8/576: link d3/d35/d55/lc1 d3/d35/d55/d61/lc9 0 2024-05-06T12:14:48.693 INFO:tasks.workunit.client.1.smithi190.stdout:7/374: rmdir d0/d1 39 2024-05-06T12:14:48.696 INFO:tasks.workunit.client.1.smithi190.stdout:5/503: dread d2/da/f1b [4194304,4194304] 0 2024-05-06T12:14:48.696 INFO:tasks.workunit.client.1.smithi190.stdout:3/538: rename d5/d14/l19 to d5/d14/d11/d21/d44/d80/d82/lc0 0 2024-05-06T12:14:48.696 INFO:tasks.workunit.client.1.smithi190.stdout:0/548: write d9/dc/d28/f8c [1385169,37480] 0 2024-05-06T12:14:48.703 INFO:tasks.workunit.client.1.smithi190.stdout:9/580: creat db/dd/d16/d58/d87/d22/d49/d88/d76/fc8 x:0 0 0 2024-05-06T12:14:48.704 INFO:tasks.workunit.client.1.smithi190.stdout:7/375: dwrite d0/d4/f7 [0,4194304] 0 2024-05-06T12:14:48.704 INFO:tasks.workunit.client.0.smithi092.stdout:3/865: write d8/db/d4c/d3d/d49/df8/d5b/d98/fb2 [1639396,78811] 0 2024-05-06T12:14:48.705 INFO:tasks.workunit.client.1.smithi190.stdout:8/577: mknod d3/d8/d25/d3b/d6b/d7a/cca 0 2024-05-06T12:14:48.711 INFO:tasks.workunit.client.1.smithi190.stdout:9/581: rmdir db/dd/d16/d58/d87 39 2024-05-06T12:14:48.714 INFO:tasks.workunit.client.1.smithi190.stdout:4/367: dread d2/d10/f58 [0,4194304] 0 2024-05-06T12:14:48.714 INFO:tasks.workunit.client.1.smithi190.stdout:8/578: truncate d3/d35/d3a/f91 343771 0 2024-05-06T12:14:48.715 INFO:tasks.workunit.client.1.smithi190.stdout:4/368: chown d2/d11/d16/d61/f7c 200564108 1 2024-05-06T12:14:48.716 INFO:tasks.workunit.client.1.smithi190.stdout:5/504: symlink d2/d6c/d7d/l9f 0 2024-05-06T12:14:48.718 INFO:tasks.workunit.client.1.smithi190.stdout:7/376: mkdir d0/d14/d6a/d8f 0 2024-05-06T12:14:48.719 INFO:tasks.workunit.client.1.smithi190.stdout:5/505: read - d2/da/d59/d5f/d6e/f7f zero size 2024-05-06T12:14:48.721 INFO:tasks.workunit.client.1.smithi190.stdout:0/549: rename d9/df/d21/d72 to d9/dc/d28/dc5 0 2024-05-06T12:14:48.722 INFO:tasks.workunit.client.1.smithi190.stdout:3/539: creat d5/d14/d11/d21/fc1 x:0 0 0 2024-05-06T12:14:48.723 INFO:tasks.workunit.client.1.smithi190.stdout:8/579: mkdir d3/d8/d25/d3b/d90/dcb 0 2024-05-06T12:14:48.725 INFO:tasks.workunit.client.1.smithi190.stdout:0/550: fdatasync d9/df/d56/f68 0 2024-05-06T12:14:48.725 INFO:tasks.workunit.client.1.smithi190.stdout:4/369: dwrite d2/d11/d16/f1a [4194304,4194304] 0 2024-05-06T12:14:48.727 INFO:tasks.workunit.client.1.smithi190.stdout:9/582: creat db/dd/d16/d58/d87/d99/fc9 x:0 0 0 2024-05-06T12:14:48.729 INFO:tasks.workunit.client.1.smithi190.stdout:9/583: fdatasync db/dd/d16/d4c/f9b 0 2024-05-06T12:14:48.729 INFO:tasks.workunit.client.1.smithi190.stdout:7/377: rename d0/da/d36/d53/l82 to d0/da/l90 0 2024-05-06T12:14:48.731 INFO:tasks.workunit.client.1.smithi190.stdout:5/506: creat d2/da/d59/d5f/d2e/d3b/d5a/fa0 x:0 0 0 2024-05-06T12:14:48.732 INFO:tasks.workunit.client.1.smithi190.stdout:3/540: mknod d5/d79/cc2 0 2024-05-06T12:14:48.732 INFO:tasks.workunit.client.1.smithi190.stdout:0/551: dread d9/dc/d63/fa2 [0,4194304] 0 2024-05-06T12:14:48.735 INFO:tasks.workunit.client.1.smithi190.stdout:4/370: mkdir d2/d11/d16/d61/d82 0 2024-05-06T12:14:48.738 INFO:tasks.workunit.client.1.smithi190.stdout:3/541: dread d5/d14/d11/d21/d44/d80/f83 [0,4194304] 0 2024-05-06T12:14:48.738 INFO:tasks.workunit.client.1.smithi190.stdout:5/507: write d2/da/d59/d5f/f58 [940287,67759] 0 2024-05-06T12:14:48.740 INFO:tasks.workunit.client.1.smithi190.stdout:3/542: stat d5/d14/d11/d21/d44/d80/d82/d3d/f6c 0 2024-05-06T12:14:48.740 INFO:tasks.workunit.client.1.smithi190.stdout:5/508: chown d2/d6c/f99 30420 1 2024-05-06T12:14:48.740 INFO:tasks.workunit.client.1.smithi190.stdout:3/543: chown d5/d7/f73 10982962 1 2024-05-06T12:14:48.741 INFO:tasks.workunit.client.1.smithi190.stdout:3/544: fsync d5/d14/f6a 0 2024-05-06T12:14:48.742 INFO:tasks.workunit.client.1.smithi190.stdout:3/545: readlink d5/d7/d15/d1f/d84/db3/l78 0 2024-05-06T12:14:48.742 INFO:tasks.workunit.client.1.smithi190.stdout:0/552: mkdir d9/dc/d48/d50/d83/dc6 0 2024-05-06T12:14:48.745 INFO:tasks.workunit.client.1.smithi190.stdout:8/580: link d3/d8/f30 d3/d8/d25/d65/dc6/fcc 0 2024-05-06T12:14:48.746 INFO:tasks.workunit.client.1.smithi190.stdout:8/581: chown d3/d8/d2b/d3e/dad/cc4 31541042 1 2024-05-06T12:14:48.747 INFO:tasks.workunit.client.1.smithi190.stdout:7/378: creat d0/d1/d18/d2b/d5a/f91 x:0 0 0 2024-05-06T12:14:48.753 INFO:tasks.workunit.client.1.smithi190.stdout:3/546: read d5/d14/d11/d21/d44/fbf [650345,95526] 0 2024-05-06T12:14:48.753 INFO:tasks.workunit.client.1.smithi190.stdout:9/584: dread db/dd/d16/d58/d87/d22/d49/f89 [0,4194304] 0 2024-05-06T12:14:48.754 INFO:tasks.workunit.client.1.smithi190.stdout:5/509: dwrite d2/da/f4a [4194304,4194304] 0 2024-05-06T12:14:48.755 INFO:tasks.workunit.client.1.smithi190.stdout:9/585: write db/dd/d16/d58/d87/d22/d49/d88/f7a [2754657,57727] 0 2024-05-06T12:14:48.756 INFO:tasks.workunit.client.1.smithi190.stdout:0/553: rename d9/dc/d48/d50/f85 to d9/dc/d28/d4c/d80/da7/fc7 0 2024-05-06T12:14:48.762 INFO:tasks.workunit.client.1.smithi190.stdout:3/547: write d5/d14/d11/d21/d44/d80/f83 [68031,71629] 0 2024-05-06T12:14:48.762 INFO:tasks.workunit.client.1.smithi190.stdout:5/510: write d2/da/f2f [2389092,119460] 0 2024-05-06T12:14:48.762 INFO:tasks.workunit.client.1.smithi190.stdout:5/511: write d2/da/d59/f94 [293811,75341] 0 2024-05-06T12:14:48.763 INFO:tasks.workunit.client.1.smithi190.stdout:8/582: mknod d3/d8/d25/d65/dc6/ccd 0 2024-05-06T12:14:48.764 INFO:tasks.workunit.client.1.smithi190.stdout:7/379: rmdir d0/d1/d81 0 2024-05-06T12:14:48.765 INFO:tasks.workunit.client.1.smithi190.stdout:5/512: write d2/da/d59/d5f/d50/f51 [2909205,10774] 0 2024-05-06T12:14:48.765 INFO:tasks.workunit.client.1.smithi190.stdout:9/586: read db/dd/d16/d4c/d4e/f9d [767722,47983] 0 2024-05-06T12:14:48.766 INFO:tasks.workunit.client.1.smithi190.stdout:7/380: readlink d0/d14/d5d/l28 0 2024-05-06T12:14:48.767 INFO:tasks.workunit.client.1.smithi190.stdout:8/583: chown d3/d8/l37 2100 1 2024-05-06T12:14:48.768 INFO:tasks.workunit.client.1.smithi190.stdout:3/548: creat d5/d7/d15/d1f/d84/db3/da1/fc3 x:0 0 0 2024-05-06T12:14:48.769 INFO:tasks.workunit.client.1.smithi190.stdout:0/554: link d9/dc/d48/d50/d87/cbb d9/dc/d28/d4c/d96/cc8 0 2024-05-06T12:14:48.770 INFO:tasks.workunit.client.1.smithi190.stdout:9/587: rmdir db/d23/d25 39 2024-05-06T12:14:48.773 INFO:tasks.workunit.client.1.smithi190.stdout:9/588: chown db/dd/d16/d58/d87/d22/c75 15 1 2024-05-06T12:14:48.775 INFO:tasks.workunit.client.1.smithi190.stdout:8/584: dwrite d3/d35/d55/fa5 [0,4194304] 0 2024-05-06T12:14:48.776 INFO:tasks.workunit.client.1.smithi190.stdout:0/555: fdatasync d9/d51/d93/fbd 0 2024-05-06T12:14:48.781 INFO:tasks.workunit.client.1.smithi190.stdout:4/371: truncate d2/d11/f4c 50915 0 2024-05-06T12:14:48.781 INFO:tasks.workunit.client.1.smithi190.stdout:9/589: fsync db/d23/d25/fa1 0 2024-05-06T12:14:48.785 INFO:tasks.workunit.client.1.smithi190.stdout:8/585: dwrite d3/d8/d25/f39 [0,4194304] 0 2024-05-06T12:14:48.788 INFO:tasks.workunit.client.1.smithi190.stdout:8/586: dread - d3/d8/d25/d3b/d6b/d7a/fa4 zero size 2024-05-06T12:14:48.789 INFO:tasks.workunit.client.1.smithi190.stdout:7/381: rename d0/da/f7e to d0/d1/f92 0 2024-05-06T12:14:48.790 INFO:tasks.workunit.client.1.smithi190.stdout:0/556: dwrite d9/df/d21/f7e [0,4194304] 0 2024-05-06T12:14:48.795 INFO:tasks.workunit.client.1.smithi190.stdout:3/549: getdents d5/d14/d11/d21/d44 0 2024-05-06T12:14:48.799 INFO:tasks.workunit.client.1.smithi190.stdout:4/372: dwrite d2/d11/d16/f3e [0,4194304] 0 2024-05-06T12:14:48.801 INFO:tasks.workunit.client.1.smithi190.stdout:9/590: creat db/dd/d16/d58/d87/d99/fca x:0 0 0 2024-05-06T12:14:48.801 INFO:tasks.workunit.client.1.smithi190.stdout:9/591: readlink la 0 2024-05-06T12:14:48.801 INFO:tasks.workunit.client.1.smithi190.stdout:4/373: chown d2/d11/d16/d61/d66 185 1 2024-05-06T12:14:48.801 INFO:tasks.workunit.client.1.smithi190.stdout:5/513: rename d2/da/d59/d5f/d2e/l5c to d2/da/d59/d5f/d2e/d3b/d76/d9a/la1 0 2024-05-06T12:14:48.802 INFO:tasks.workunit.client.1.smithi190.stdout:3/550: dread - d5/d14/d11/d21/d44/d80/d82/d3d/f99 zero size 2024-05-06T12:14:48.804 INFO:tasks.workunit.client.1.smithi190.stdout:5/514: chown d2/d27/f48 483928 1 2024-05-06T12:14:48.806 INFO:tasks.workunit.client.1.smithi190.stdout:0/557: rename d9/dc/d63 to d9/d51/dc9 0 2024-05-06T12:14:48.808 INFO:tasks.workunit.client.1.smithi190.stdout:0/558: write d9/df/d21/f4a [2938301,36833] 0 2024-05-06T12:14:48.808 INFO:tasks.workunit.client.1.smithi190.stdout:9/592: creat db/dd/d16/d58/d87/fcb x:0 0 0 2024-05-06T12:14:48.810 INFO:tasks.workunit.client.1.smithi190.stdout:3/551: creat d5/d7/d6d/fc4 x:0 0 0 2024-05-06T12:14:48.812 INFO:tasks.workunit.client.1.smithi190.stdout:5/515: unlink d2/da/f8d 0 2024-05-06T12:14:48.816 INFO:tasks.workunit.client.1.smithi190.stdout:8/587: rename d3/d8/d2b/d3e/f54 to d3/d35/d3a/dc5/fce 0 2024-05-06T12:14:48.818 INFO:tasks.workunit.client.1.smithi190.stdout:9/593: chown db/d23/d7e 123668084 1 2024-05-06T12:14:48.818 INFO:tasks.workunit.client.1.smithi190.stdout:0/559: dread d9/df/d56/f57 [0,4194304] 0 2024-05-06T12:14:48.820 INFO:tasks.workunit.client.1.smithi190.stdout:3/552: dwrite d5/d14/d11/d21/d44/d92/fb7 [0,4194304] 0 2024-05-06T12:14:48.821 INFO:tasks.workunit.client.1.smithi190.stdout:5/516: creat d2/d6c/d7d/fa2 x:0 0 0 2024-05-06T12:14:48.824 INFO:tasks.workunit.client.1.smithi190.stdout:5/517: chown d2/da/d59/d5f/d2e/d3b 5 1 2024-05-06T12:14:48.827 INFO:tasks.workunit.client.1.smithi190.stdout:0/560: creat d9/df/d56/dab/fca x:0 0 0 2024-05-06T12:14:48.829 INFO:tasks.workunit.client.1.smithi190.stdout:4/374: dread d2/f48 [0,4194304] 0 2024-05-06T12:14:48.830 INFO:tasks.workunit.client.1.smithi190.stdout:3/553: mkdir d5/d14/dc5 0 2024-05-06T12:14:48.830 INFO:tasks.workunit.client.1.smithi190.stdout:9/594: truncate db/dd/d5c/fb0 2985742 0 2024-05-06T12:14:48.833 INFO:tasks.workunit.client.1.smithi190.stdout:4/375: dread - d2/d11/d16/d41/f65 zero size 2024-05-06T12:14:48.837 INFO:tasks.workunit.client.1.smithi190.stdout:9/595: rename db/dd/d16/d4c/d4e/d5e/f92 to db/dd/d16/d58/d87/d22/d49/d8c/fcc 0 2024-05-06T12:14:48.839 INFO:tasks.workunit.client.1.smithi190.stdout:9/596: stat db/dd/d16/d58/d87/d99/lb6 0 2024-05-06T12:14:48.839 INFO:tasks.workunit.client.1.smithi190.stdout:4/376: stat d2/l49 0 2024-05-06T12:14:48.841 INFO:tasks.workunit.client.1.smithi190.stdout:2/420: sync 2024-05-06T12:14:48.846 INFO:tasks.workunit.client.1.smithi190.stdout:9/597: dwrite db/dd/d16/d58/d87/f70 [0,4194304] 0 2024-05-06T12:14:48.849 INFO:tasks.workunit.client.1.smithi190.stdout:5/518: truncate d2/da/d59/d5f/d2e/d3b/f68 1968033 0 2024-05-06T12:14:48.850 INFO:tasks.workunit.client.1.smithi190.stdout:5/519: dread - d2/da/d59/f84 zero size 2024-05-06T12:14:48.851 INFO:tasks.workunit.client.1.smithi190.stdout:4/377: unlink d2/d11/d16/d61/d66/c73 0 2024-05-06T12:14:48.852 INFO:tasks.workunit.client.1.smithi190.stdout:9/598: symlink db/dd/d16/d58/d87/d99/lcd 0 2024-05-06T12:14:48.856 INFO:tasks.workunit.client.0.smithi092.stdout:8/988: sync 2024-05-06T12:14:48.876 INFO:tasks.workunit.client.0.smithi092.stdout:8/989: dread d9/dc/f5d [0,4194304] 0 2024-05-06T12:14:48.877 INFO:tasks.workunit.client.0.smithi092.stdout:8/990: unlink d9/d9d/fa0 0 2024-05-06T12:14:48.881 INFO:tasks.workunit.client.0.smithi092.stdout:8/991: write d9/dc/f4d [1765674,10229] 0 2024-05-06T12:14:48.896 INFO:tasks.workunit.client.1.smithi190.stdout:5/520: mkdir d2/da/d59/d5f/d2e/da3 0 2024-05-06T12:14:48.897 INFO:tasks.workunit.client.1.smithi190.stdout:9/599: unlink db/dd/d16/f6c 0 2024-05-06T12:14:48.899 INFO:tasks.workunit.client.1.smithi190.stdout:3/554: dwrite d5/d7/d15/d1f/f42 [0,4194304] 0 2024-05-06T12:14:48.901 INFO:tasks.workunit.client.1.smithi190.stdout:9/600: dread - db/dd/d16/fae zero size 2024-05-06T12:14:48.905 INFO:tasks.workunit.client.1.smithi190.stdout:4/378: link d2/d10/l7f d2/d11/d16/d53/d7b/l83 0 2024-05-06T12:14:48.907 INFO:tasks.workunit.client.1.smithi190.stdout:8/588: dwrite d3/d35/d3a/dc5/fce [0,4194304] 0 2024-05-06T12:14:48.910 INFO:tasks.workunit.client.1.smithi190.stdout:2/421: dwrite d2/d8/d33/d46/d55/f59 [0,4194304] 0 2024-05-06T12:14:48.912 INFO:tasks.workunit.client.1.smithi190.stdout:5/521: dwrite d2/da/f2f [0,4194304] 0 2024-05-06T12:14:48.915 INFO:tasks.workunit.client.1.smithi190.stdout:8/589: fdatasync d3/d8/d25/d3b/d90/d6e/f82 0 2024-05-06T12:14:48.922 INFO:tasks.workunit.client.1.smithi190.stdout:3/555: dread d5/d14/f24 [0,4194304] 0 2024-05-06T12:14:48.922 INFO:tasks.workunit.client.1.smithi190.stdout:4/379: rmdir d2 39 2024-05-06T12:14:48.924 INFO:tasks.workunit.client.1.smithi190.stdout:5/522: chown d2/f8 507 1 2024-05-06T12:14:48.925 INFO:tasks.workunit.client.1.smithi190.stdout:3/556: truncate d5/d7/f3a 5030098 0 2024-05-06T12:14:48.926 INFO:tasks.workunit.client.1.smithi190.stdout:8/590: write d3/d8/d2b/f34 [3714205,116047] 0 2024-05-06T12:14:48.927 INFO:tasks.workunit.client.1.smithi190.stdout:5/523: creat d2/da/d59/d5f/d6e/d95/d9d/fa4 x:0 0 0 2024-05-06T12:14:48.929 INFO:tasks.workunit.client.1.smithi190.stdout:3/557: mkdir d5/d7/d6d/dc6 0 2024-05-06T12:14:48.929 INFO:tasks.workunit.client.1.smithi190.stdout:8/591: dread d3/d8/d25/d3b/d90/f5f [0,4194304] 0 2024-05-06T12:14:48.931 INFO:tasks.workunit.client.1.smithi190.stdout:4/380: truncate d2/d11/d16/f3e 4654189 0 2024-05-06T12:14:48.931 INFO:tasks.workunit.client.1.smithi190.stdout:5/524: creat d2/da/d59/d5f/d2e/d3b/fa5 x:0 0 0 2024-05-06T12:14:48.932 INFO:tasks.workunit.client.1.smithi190.stdout:4/381: stat d2/d11/f20 0 2024-05-06T12:14:48.935 INFO:tasks.workunit.client.1.smithi190.stdout:3/558: rename d5/d14/f8a to d5/d14/d11/fc7 0 2024-05-06T12:14:48.937 INFO:tasks.workunit.client.1.smithi190.stdout:4/382: mknod d2/d11/d16/d53/c84 0 2024-05-06T12:14:48.939 INFO:tasks.workunit.client.1.smithi190.stdout:5/525: dwrite d2/da/d59/d5f/f5b [4194304,4194304] 0 2024-05-06T12:14:48.941 INFO:tasks.workunit.client.1.smithi190.stdout:8/592: rename d3/d8/d25/d3b/d49 to d3/d8/dcf 0 2024-05-06T12:14:48.942 INFO:tasks.workunit.client.1.smithi190.stdout:4/383: write d2/d11/f20 [1888112,86723] 0 2024-05-06T12:14:48.947 INFO:tasks.workunit.client.1.smithi190.stdout:4/384: creat d2/d11/d16/d4e/f85 x:0 0 0 2024-05-06T12:14:48.949 INFO:tasks.workunit.client.1.smithi190.stdout:4/385: read - d2/d11/d16/d31/f5c zero size 2024-05-06T12:14:48.950 INFO:tasks.workunit.client.0.smithi092.stdout:2/842: sync 2024-05-06T12:14:48.950 INFO:tasks.workunit.client.1.smithi190.stdout:3/559: rename d5/d14/d11/d21/d44/d80/c87 to d5/d14/d11/cc8 0 2024-05-06T12:14:48.952 INFO:tasks.workunit.client.0.smithi092.stdout:2/843: fdatasync df/d17/d51/de9/fef 0 2024-05-06T12:14:48.952 INFO:tasks.workunit.client.1.smithi190.stdout:2/422: dread d2/d8/d19/d4f/f57 [0,4194304] 0 2024-05-06T12:14:48.954 INFO:tasks.workunit.client.1.smithi190.stdout:8/593: truncate d3/d8/f2c 3980759 0 2024-05-06T12:14:48.954 INFO:tasks.workunit.client.1.smithi190.stdout:9/601: dread db/d23/f5b [0,4194304] 0 2024-05-06T12:14:48.955 INFO:tasks.workunit.client.0.smithi092.stdout:2/844: truncate df/d17/d51/f70 1097562 0 2024-05-06T12:14:48.955 INFO:tasks.workunit.client.1.smithi190.stdout:4/386: symlink d2/d11/d16/d61/d67/d6e/d78/l86 0 2024-05-06T12:14:48.956 INFO:tasks.workunit.client.1.smithi190.stdout:9/602: chown db/dd/d16/d58/d87/d99/fca 2300 1 2024-05-06T12:14:48.957 INFO:tasks.workunit.client.1.smithi190.stdout:3/560: mkdir d5/d14/d11/dc9 0 2024-05-06T12:14:48.959 INFO:tasks.workunit.client.1.smithi190.stdout:4/387: write d2/d11/f14 [226882,86565] 0 2024-05-06T12:14:48.961 INFO:tasks.workunit.client.1.smithi190.stdout:5/526: dread d2/d6c/f99 [0,4194304] 0 2024-05-06T12:14:48.961 INFO:tasks.workunit.client.0.smithi092.stdout:2/845: dwrite df/d15/fbe [0,4194304] 0 2024-05-06T12:14:48.962 INFO:tasks.workunit.client.1.smithi190.stdout:8/594: creat d3/d35/d3a/db9/fd0 x:0 0 0 2024-05-06T12:14:48.963 INFO:tasks.workunit.client.0.smithi092.stdout:2/846: write df/d15/d88/d59/f9e [903236,2241] 0 2024-05-06T12:14:48.965 INFO:tasks.workunit.client.1.smithi190.stdout:2/423: creat d2/d21/f89 x:0 0 0 2024-05-06T12:14:48.967 INFO:tasks.workunit.client.1.smithi190.stdout:9/603: mkdir db/dd/d16/d4c/d4e/d5e/dce 0 2024-05-06T12:14:48.969 INFO:tasks.workunit.client.1.smithi190.stdout:3/561: mkdir d5/d14/d11/d21/d44/d80/dca 0 2024-05-06T12:14:48.970 INFO:tasks.workunit.client.0.smithi092.stdout:2/847: write df/d17/d1d/f46 [5091538,104875] 0 2024-05-06T12:14:48.971 INFO:tasks.workunit.client.0.smithi092.stdout:2/848: write df/d17/d51/f9c [1020493,53367] 0 2024-05-06T12:14:48.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:48 smithi092 ceph-mon[26954]: pgmap v26: 65 pgs: 65 active+clean; 2.6 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 87 MiB/s rd, 265 MiB/s wr, 554 op/s 2024-05-06T12:14:48.972 INFO:tasks.workunit.client.1.smithi190.stdout:8/595: rmdir d3/d8/d25/d65 39 2024-05-06T12:14:48.973 INFO:tasks.workunit.client.1.smithi190.stdout:2/424: write d2/fa [1168622,18850] 0 2024-05-06T12:14:48.973 INFO:tasks.workunit.client.0.smithi092.stdout:2/849: mkdir df/d17/d1d/df8 0 2024-05-06T12:14:48.974 INFO:tasks.workunit.client.1.smithi190.stdout:2/425: dread - d2/d21/f89 zero size 2024-05-06T12:14:48.975 INFO:tasks.workunit.client.1.smithi190.stdout:2/426: readlink d2/d8/d71/l7d 0 2024-05-06T12:14:48.976 INFO:tasks.workunit.client.0.smithi092.stdout:2/850: mknod df/d17/d7d/df0/cf9 0 2024-05-06T12:14:48.977 INFO:tasks.workunit.client.0.smithi092.stdout:2/851: fdatasync df/d17/d1d/dc8/fe7 0 2024-05-06T12:14:48.978 INFO:tasks.workunit.client.1.smithi190.stdout:4/388: link d2/c4a d2/d11/d16/d61/d67/d6e/d78/c87 0 2024-05-06T12:14:48.979 INFO:tasks.workunit.client.1.smithi190.stdout:3/562: mknod d5/d7/d6d/d9a/ccb 0 2024-05-06T12:14:48.983 INFO:tasks.workunit.client.1.smithi190.stdout:2/427: dwrite d2/f62 [0,4194304] 0 2024-05-06T12:14:48.984 INFO:tasks.workunit.client.1.smithi190.stdout:2/428: dread - d2/d21/f85 zero size 2024-05-06T12:14:48.984 INFO:tasks.workunit.client.1.smithi190.stdout:3/563: symlink d5/d14/d11/d21/d44/d92/lcc 0 2024-05-06T12:14:48.986 INFO:tasks.workunit.client.0.smithi092.stdout:2/852: dwrite df/d15/d88/f78 [0,4194304] 0 2024-05-06T12:14:48.990 INFO:tasks.workunit.client.1.smithi190.stdout:2/429: rename d2/fe to d2/d8/d19/f8a 0 2024-05-06T12:14:48.991 INFO:tasks.workunit.client.0.smithi092.stdout:2/853: symlink df/d17/d51/d96/lfa 0 2024-05-06T12:14:48.991 INFO:tasks.workunit.client.1.smithi190.stdout:9/604: dread db/dd/df/f1b [4194304,4194304] 0 2024-05-06T12:14:48.991 INFO:tasks.workunit.client.1.smithi190.stdout:2/430: write d2/d8/d33/d46/d5d/f74 [19500,5431] 0 2024-05-06T12:14:48.991 INFO:tasks.workunit.client.1.smithi190.stdout:3/564: creat d5/d14/d11/d21/d44/fcd x:0 0 0 2024-05-06T12:14:48.992 INFO:tasks.workunit.client.1.smithi190.stdout:3/565: chown d5 1209449 1 2024-05-06T12:14:48.995 INFO:tasks.workunit.client.1.smithi190.stdout:9/605: creat db/dd/d5c/fcf x:0 0 0 2024-05-06T12:14:48.998 INFO:tasks.workunit.client.1.smithi190.stdout:9/606: chown db/dd/d16/d58/d87/d22 52 1 2024-05-06T12:14:48.998 INFO:tasks.workunit.client.1.smithi190.stdout:2/431: link d2/d8/dc/f81 d2/d8/d71/f8b 0 2024-05-06T12:14:48.999 INFO:tasks.workunit.client.1.smithi190.stdout:5/527: dwrite d2/da/d59/d5f/d2e/d3b/f68 [0,4194304] 0 2024-05-06T12:14:49.004 INFO:tasks.workunit.client.1.smithi190.stdout:3/566: dwrite d5/d7/d15/f28 [0,4194304] 0 2024-05-06T12:14:49.005 INFO:tasks.workunit.client.1.smithi190.stdout:5/528: write d2/d27/f20 [5612020,65344] 0 2024-05-06T12:14:49.006 INFO:tasks.workunit.client.1.smithi190.stdout:9/607: mknod db/dd/d16/d4c/d4e/d5e/dce/cd0 0 2024-05-06T12:14:49.006 INFO:tasks.workunit.client.0.smithi092.stdout:2/854: dread df/dd0/de3/dcf/f68 [0,4194304] 0 2024-05-06T12:14:49.008 INFO:tasks.workunit.client.1.smithi190.stdout:9/608: write db/dd/d16/fa6 [675052,111569] 0 2024-05-06T12:14:49.008 INFO:tasks.workunit.client.1.smithi190.stdout:2/432: link d2/d4c/f5a d2/d8/d33/d46/d5d/f8c 0 2024-05-06T12:14:49.008 INFO:tasks.workunit.client.1.smithi190.stdout:3/567: chown d5/d7/d15/c69 776991 1 2024-05-06T12:14:49.010 INFO:tasks.workunit.client.0.smithi092.stdout:2/855: mkdir df/dfb 0 2024-05-06T12:14:49.013 INFO:tasks.workunit.client.1.smithi190.stdout:3/568: stat d5/d7/d15/d1f/d33/l36 0 2024-05-06T12:14:49.013 INFO:tasks.workunit.client.0.smithi092.stdout:2/856: link df/d17/d53/da9/dd9/c54 df/dd0/de3/dcf/cfc 0 2024-05-06T12:14:49.015 INFO:tasks.workunit.client.1.smithi190.stdout:2/433: creat d2/d8/d33/d46/f8d x:0 0 0 2024-05-06T12:14:49.018 INFO:tasks.workunit.client.1.smithi190.stdout:9/609: unlink db/dd/d5c/fb0 0 2024-05-06T12:14:49.018 INFO:tasks.workunit.client.0.smithi092.stdout:2/857: rename df/d5c/f66 to df/d17/d53/da9/dd9/d83/dc2/ffd 0 2024-05-06T12:14:49.019 INFO:tasks.workunit.client.1.smithi190.stdout:5/529: dwrite d2/da/d59/d5f/d6e/f7f [0,4194304] 0 2024-05-06T12:14:49.019 INFO:tasks.workunit.client.1.smithi190.stdout:3/569: symlink d5/d7/d7f/lce 0 2024-05-06T12:14:49.022 INFO:tasks.workunit.client.1.smithi190.stdout:3/570: fsync d5/d7/d15/d1f/d84/db3/da1/fc3 0 2024-05-06T12:14:49.023 INFO:tasks.workunit.client.0.smithi092.stdout:2/858: dwrite fd [0,4194304] 0 2024-05-06T12:14:49.027 INFO:tasks.workunit.client.0.smithi092.stdout:2/859: link df/d17/d53/lf2 df/d17/lfe 0 2024-05-06T12:14:49.028 INFO:tasks.workunit.client.1.smithi190.stdout:5/530: dwrite d2/f44 [0,4194304] 0 2024-05-06T12:14:49.029 INFO:tasks.workunit.client.0.smithi092.stdout:2/860: mknod df/d17/d1d/cff 0 2024-05-06T12:14:49.030 INFO:tasks.workunit.client.1.smithi190.stdout:9/610: chown db/dd/d16/d58/d87/d22/d49/f98 162330 1 2024-05-06T12:14:49.033 INFO:tasks.workunit.client.0.smithi092.stdout:2/861: getdents df/d17/d53/d85 0 2024-05-06T12:14:49.033 INFO:tasks.workunit.client.1.smithi190.stdout:5/531: truncate d2/d6c/d7d/fa2 916412 0 2024-05-06T12:14:49.035 INFO:tasks.workunit.client.0.smithi092.stdout:2/862: mkdir df/dd0/d100 0 2024-05-06T12:14:49.036 INFO:tasks.workunit.client.1.smithi190.stdout:2/434: truncate d2/d8/dc/f13 1901909 0 2024-05-06T12:14:49.038 INFO:tasks.workunit.client.1.smithi190.stdout:9/611: creat db/d23/d7e/fd1 x:0 0 0 2024-05-06T12:14:49.039 INFO:tasks.workunit.client.1.smithi190.stdout:3/571: symlink d5/d14/d11/d21/d44/d80/d82/lcf 0 2024-05-06T12:14:49.039 INFO:tasks.workunit.client.0.smithi092.stdout:2/863: dread df/d15/d88/d59/f8d [0,4194304] 0 2024-05-06T12:14:49.040 INFO:tasks.workunit.client.1.smithi190.stdout:5/532: symlink d2/da/d59/la6 0 2024-05-06T12:14:49.043 INFO:tasks.workunit.client.0.smithi092.stdout:2/864: getdents df/d17/d53/da9 0 2024-05-06T12:14:49.046 INFO:tasks.workunit.client.1.smithi190.stdout:9/612: truncate db/dd/d16/d58/d87/d22/d49/d8c/fc1 448764 0 2024-05-06T12:14:49.047 INFO:tasks.workunit.client.1.smithi190.stdout:5/533: rename d2/da/d59/d5f/d6e/d95 to d2/da/d59/d5f/d2e/da7 0 2024-05-06T12:14:49.047 INFO:tasks.workunit.client.1.smithi190.stdout:3/572: dwrite d5/d7/f86 [4194304,4194304] 0 2024-05-06T12:14:49.050 INFO:tasks.workunit.client.1.smithi190.stdout:3/573: fdatasync d5/d14/d11/d21/d44/d80/d82/d3d/f49 0 2024-05-06T12:14:49.054 INFO:tasks.workunit.client.1.smithi190.stdout:5/534: symlink d2/da/d59/d5f/d50/d6d/la8 0 2024-05-06T12:14:49.056 INFO:tasks.workunit.client.1.smithi190.stdout:9/613: dwrite db/d23/d25/fa1 [0,4194304] 0 2024-05-06T12:14:49.060 INFO:tasks.workunit.client.1.smithi190.stdout:3/574: symlink d5/d14/d11/d21/ld0 0 2024-05-06T12:14:49.062 INFO:tasks.workunit.client.1.smithi190.stdout:9/614: mknod db/dd/d16/d4c/d4e/d5e/dce/cd2 0 2024-05-06T12:14:49.062 INFO:tasks.workunit.client.1.smithi190.stdout:5/535: mknod d2/da/d59/d5f/d2e/da3/ca9 0 2024-05-06T12:14:49.062 INFO:tasks.workunit.client.1.smithi190.stdout:9/615: chown db/daa 123313 1 2024-05-06T12:14:49.064 INFO:tasks.workunit.client.1.smithi190.stdout:5/536: fsync d2/da/f1b 0 2024-05-06T12:14:49.067 INFO:tasks.workunit.client.1.smithi190.stdout:2/435: dwrite d2/d8/d19/f83 [0,4194304] 0 2024-05-06T12:14:49.069 INFO:tasks.workunit.client.1.smithi190.stdout:5/537: mkdir d2/da/d59/d5f/d6e/daa 0 2024-05-06T12:14:49.071 INFO:tasks.workunit.client.1.smithi190.stdout:5/538: truncate d2/da/f64 36116 0 2024-05-06T12:14:49.072 INFO:tasks.workunit.client.1.smithi190.stdout:2/436: write d2/d21/f70 [80669,72642] 0 2024-05-06T12:14:49.074 INFO:tasks.workunit.client.1.smithi190.stdout:5/539: chown d2/da/d59/d5f/f36 26502088 1 2024-05-06T12:14:49.075 INFO:tasks.workunit.client.1.smithi190.stdout:3/575: dwrite d5/d7/f71 [0,4194304] 0 2024-05-06T12:14:49.076 INFO:tasks.workunit.client.1.smithi190.stdout:3/576: chown d5/d7/d15/c69 14927 1 2024-05-06T12:14:49.078 INFO:tasks.workunit.client.1.smithi190.stdout:2/437: rmdir d2/d8/dc 39 2024-05-06T12:14:49.078 INFO:tasks.workunit.client.1.smithi190.stdout:5/540: symlink d2/da/d59/d5f/d2e/d8b/lab 0 2024-05-06T12:14:49.083 INFO:tasks.workunit.client.0.smithi092.stdout:2/865: dread df/d17/f22 [0,4194304] 0 2024-05-06T12:14:49.084 INFO:tasks.workunit.client.0.smithi092.stdout:2/866: truncate df/d17/d51/f70 1640854 0 2024-05-06T12:14:49.084 INFO:tasks.workunit.client.1.smithi190.stdout:5/541: chown d2/da/d59/d5f/d2e/d3b/d5a/fa0 0 1 2024-05-06T12:14:49.085 INFO:tasks.workunit.client.1.smithi190.stdout:2/438: creat d2/d8/d71/f8e x:0 0 0 2024-05-06T12:14:49.085 INFO:tasks.workunit.client.1.smithi190.stdout:3/577: mknod d5/d7/cd1 0 2024-05-06T12:14:49.085 INFO:tasks.workunit.client.1.smithi190.stdout:5/542: unlink d2/da/d59/d5f/d6e/f7f 0 2024-05-06T12:14:49.087 INFO:tasks.workunit.client.0.smithi092.stdout:2/867: rename df/d15/d88/dc9 to df/d17/d53/da9/dd9/d83/dc2/d101 0 2024-05-06T12:14:49.090 INFO:tasks.workunit.client.0.smithi092.stdout:2/868: getdents df/d15 0 2024-05-06T12:14:49.091 INFO:tasks.workunit.client.0.smithi092.stdout:2/869: fsync df/d15/d88/d59/fb7 0 2024-05-06T12:14:49.092 INFO:tasks.workunit.client.1.smithi190.stdout:3/578: link d5/d14/d11/d21/d44/d80/f83 d5/d7/d6d/dc6/fd2 0 2024-05-06T12:14:49.092 INFO:tasks.workunit.client.1.smithi190.stdout:2/439: dwrite d2/d8/d71/f88 [0,4194304] 0 2024-05-06T12:14:49.093 INFO:tasks.workunit.client.0.smithi092.stdout:2/870: mknod df/d17/d53/da9/dd9/d83/dc2/c102 0 2024-05-06T12:14:49.094 INFO:tasks.workunit.client.0.smithi092.stdout:2/871: dread - df/d17/d53/da9/dd9/d83/dc2/d101/fd6 zero size 2024-05-06T12:14:49.096 INFO:tasks.workunit.client.0.smithi092.stdout:2/872: rmdir df/d17/d7d 39 2024-05-06T12:14:49.097 INFO:tasks.workunit.client.0.smithi092.stdout:2/873: mknod df/d17/d1d/dc8/c103 0 2024-05-06T12:14:49.098 INFO:tasks.workunit.client.1.smithi190.stdout:5/543: dwrite d2/da/d59/d5f/d2e/d3b/f52 [0,4194304] 0 2024-05-06T12:14:49.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:48 smithi190 ceph-mon[33798]: pgmap v26: 65 pgs: 65 active+clean; 2.6 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 87 MiB/s rd, 265 MiB/s wr, 554 op/s 2024-05-06T12:14:49.101 INFO:tasks.workunit.client.0.smithi092.stdout:2/874: rename df/d5c/d60/cca to df/d15/d88/d59/c104 0 2024-05-06T12:14:49.103 INFO:tasks.workunit.client.1.smithi190.stdout:3/579: creat d5/d14/d11/d21/d44/d80/d82/fd3 x:0 0 0 2024-05-06T12:14:49.104 INFO:tasks.workunit.client.1.smithi190.stdout:3/580: fdatasync d5/f27 0 2024-05-06T12:14:49.104 INFO:tasks.workunit.client.1.smithi190.stdout:3/581: stat d5/d7/f71 0 2024-05-06T12:14:49.105 INFO:tasks.workunit.client.0.smithi092.stdout:2/875: mknod df/d17/d53/da9/dd9/d83/c105 0 2024-05-06T12:14:49.108 INFO:tasks.workunit.client.0.smithi092.stdout:2/876: rename df/d17/f42 to df/d17/d53/da9/dd9/d83/dc2/d101/f106 0 2024-05-06T12:14:49.109 INFO:tasks.workunit.client.1.smithi190.stdout:5/544: link d2/l8f d2/da/d59/d5f/d2e/da3/lac 0 2024-05-06T12:14:49.113 INFO:tasks.workunit.client.1.smithi190.stdout:5/545: symlink d2/d6c/d7d/lad 0 2024-05-06T12:14:49.114 INFO:tasks.workunit.client.0.smithi092.stdout:2/877: rename df/d15/d88 to df/dfb/d107 0 2024-05-06T12:14:49.115 INFO:tasks.workunit.client.1.smithi190.stdout:3/582: getdents d5/d14/d11/d21/d44/d92/d9e 0 2024-05-06T12:14:49.115 INFO:tasks.workunit.client.1.smithi190.stdout:5/546: write d2/f44 [2857275,102157] 0 2024-05-06T12:14:49.117 INFO:tasks.workunit.client.0.smithi092.stdout:2/878: creat df/d17/d7d/df0/f108 x:0 0 0 2024-05-06T12:14:49.119 INFO:tasks.workunit.client.1.smithi190.stdout:3/583: mknod d5/d14/d11/d21/d44/d80/cd4 0 2024-05-06T12:14:49.121 INFO:tasks.workunit.client.1.smithi190.stdout:2/440: dwrite d2/d8/d71/f8b [0,4194304] 0 2024-05-06T12:14:49.121 INFO:tasks.workunit.client.0.smithi092.stdout:2/879: link df/c38 df/d17/d53/da9/dd9/d83/dc2/d101/c109 0 2024-05-06T12:14:49.121 INFO:tasks.workunit.client.1.smithi190.stdout:3/584: write d5/d7/d15/d1f/fe [3750585,41855] 0 2024-05-06T12:14:49.123 INFO:tasks.workunit.client.1.smithi190.stdout:5/547: unlink d2/da/d59/d5f/d2e/d3b/d5a/fa0 0 2024-05-06T12:14:49.123 INFO:tasks.workunit.client.0.smithi092.stdout:2/880: unlink df/d17/d1d/l6c 0 2024-05-06T12:14:49.124 INFO:tasks.workunit.client.1.smithi190.stdout:5/548: chown d2/d6c/d7d/lad 453020183 1 2024-05-06T12:14:49.125 INFO:tasks.workunit.client.0.smithi092.stdout:2/881: mknod df/dd0/de3/db4/dce/c10a 0 2024-05-06T12:14:49.127 INFO:tasks.workunit.client.0.smithi092.stdout:2/882: mkdir df/d17/d53/d10b 0 2024-05-06T12:14:49.127 INFO:tasks.workunit.client.1.smithi190.stdout:2/441: dread d2/d8/d71/f88 [0,4194304] 0 2024-05-06T12:14:49.130 INFO:tasks.workunit.client.1.smithi190.stdout:5/549: rename d2/da/c5d to d2/da/d59/d5f/d2e/cae 0 2024-05-06T12:14:49.131 INFO:tasks.workunit.client.0.smithi092.stdout:2/883: rename df/dd0/de3/fbd to df/d17/d7d/f10c 0 2024-05-06T12:14:49.132 INFO:tasks.workunit.client.0.smithi092.stdout:3/866: sync 2024-05-06T12:14:49.135 INFO:tasks.workunit.client.0.smithi092.stdout:3/867: symlink d8/d24/d33/da8/l117 0 2024-05-06T12:14:49.136 INFO:tasks.workunit.client.0.smithi092.stdout:2/884: truncate df/dfb/d107/d59/fb7 1068070 0 2024-05-06T12:14:49.140 INFO:tasks.workunit.client.0.smithi092.stdout:2/885: chown df/d17/d53/d85/la8 6231 1 2024-05-06T12:14:49.140 INFO:tasks.workunit.client.0.smithi092.stdout:2/886: readlink df/d5c/d60/l99 0 2024-05-06T12:14:49.147 INFO:tasks.workunit.client.1.smithi190.stdout:6/386: sync 2024-05-06T12:14:49.148 INFO:tasks.workunit.client.1.smithi190.stdout:2/442: rmdir d2/d8/d19/d43 39 2024-05-06T12:14:49.150 INFO:tasks.workunit.client.1.smithi190.stdout:6/387: rmdir df/d2c/d61 39 2024-05-06T12:14:49.153 INFO:tasks.workunit.client.1.smithi190.stdout:5/550: rename d2/da/d59/d5f/d2e/d3b/f47 to d2/da/d59/d5f/faf 0 2024-05-06T12:14:49.157 INFO:tasks.workunit.client.1.smithi190.stdout:2/443: dwrite d2/d8/d71/f88 [0,4194304] 0 2024-05-06T12:14:49.159 INFO:tasks.workunit.client.1.smithi190.stdout:5/551: dread d2/da/f2f [0,4194304] 0 2024-05-06T12:14:49.160 INFO:tasks.workunit.client.1.smithi190.stdout:3/585: dread d5/d14/d11/d21/d44/d80/d82/d3d/f49 [0,4194304] 0 2024-05-06T12:14:49.162 INFO:tasks.workunit.client.1.smithi190.stdout:2/444: creat d2/d8/d33/f8f x:0 0 0 2024-05-06T12:14:49.162 INFO:tasks.workunit.client.1.smithi190.stdout:2/445: stat d2/d10/f87 0 2024-05-06T12:14:49.168 INFO:tasks.workunit.client.1.smithi190.stdout:3/586: link d5/f51 d5/d14/d11/d21/d44/d92/d9e/fd5 0 2024-05-06T12:14:49.169 INFO:tasks.workunit.client.1.smithi190.stdout:5/552: dwrite d2/da/d59/d5f/f36 [4194304,4194304] 0 2024-05-06T12:14:49.170 INFO:tasks.workunit.client.1.smithi190.stdout:2/446: rename d2/d8/d33/d46/f7a to d2/d8/f90 0 2024-05-06T12:14:49.170 INFO:tasks.workunit.client.0.smithi092.stdout:8/992: sync 2024-05-06T12:14:49.171 INFO:tasks.workunit.client.1.smithi190.stdout:3/587: mknod d5/d14/cd6 0 2024-05-06T12:14:49.173 INFO:tasks.workunit.client.1.smithi190.stdout:2/447: mkdir d2/d8/d33/d46/d55/d91 0 2024-05-06T12:14:49.177 INFO:tasks.workunit.client.0.smithi092.stdout:2/887: write df/f5b [3152101,92558] 0 2024-05-06T12:14:49.178 INFO:tasks.workunit.client.1.smithi190.stdout:5/553: dwrite d2/da/d59/d5f/f3c [0,4194304] 0 2024-05-06T12:14:49.180 INFO:tasks.workunit.client.0.smithi092.stdout:8/993: truncate d9/d21/d37/d74/f109 41898 0 2024-05-06T12:14:49.182 INFO:tasks.workunit.client.1.smithi190.stdout:2/448: read d2/d8/d33/d46/d5d/f74 [798988,116660] 0 2024-05-06T12:14:49.184 INFO:tasks.workunit.client.1.smithi190.stdout:5/554: creat d2/d65/fb0 x:0 0 0 2024-05-06T12:14:49.185 INFO:tasks.workunit.client.1.smithi190.stdout:2/449: chown d2/d8/dc/l17 0 1 2024-05-06T12:14:49.186 INFO:tasks.workunit.client.1.smithi190.stdout:2/450: write d2/d8/d19/f2a [2219956,25224] 0 2024-05-06T12:14:49.189 INFO:tasks.workunit.client.1.smithi190.stdout:5/555: rename d2/da/l16 to d2/da/d59/d5f/d2e/lb1 0 2024-05-06T12:14:49.189 INFO:tasks.workunit.client.1.smithi190.stdout:3/588: write d5/d7/d15/d1f/f88 [618126,19322] 0 2024-05-06T12:14:49.190 INFO:tasks.workunit.client.0.smithi092.stdout:2/888: dwrite f5 [0,4194304] 0 2024-05-06T12:14:49.192 INFO:tasks.workunit.client.1.smithi190.stdout:2/451: creat d2/d8/f92 x:0 0 0 2024-05-06T12:14:49.193 INFO:tasks.workunit.client.1.smithi190.stdout:5/556: symlink d2/da/d59/d5f/d50/lb2 0 2024-05-06T12:14:49.194 INFO:tasks.workunit.client.1.smithi190.stdout:2/452: creat d2/d8/d33/d46/d5d/f93 x:0 0 0 2024-05-06T12:14:49.195 INFO:tasks.workunit.client.0.smithi092.stdout:2/889: fsync df/f1e 0 2024-05-06T12:14:49.195 INFO:tasks.workunit.client.0.smithi092.stdout:2/890: fdatasync df/d15/ff5 0 2024-05-06T12:14:49.195 INFO:tasks.workunit.client.1.smithi190.stdout:2/453: dread - d2/d8/d71/f8e zero size 2024-05-06T12:14:49.196 INFO:tasks.workunit.client.0.smithi092.stdout:2/891: chown df/d15/ff5 14223 1 2024-05-06T12:14:49.198 INFO:tasks.workunit.client.1.smithi190.stdout:5/557: unlink d2/da/d59/d5f/d2e/da3/lac 0 2024-05-06T12:14:49.198 INFO:tasks.workunit.client.0.smithi092.stdout:2/892: creat df/d17/d53/f10d x:0 0 0 2024-05-06T12:14:49.199 INFO:tasks.workunit.client.0.smithi092.stdout:2/893: write df/d17/f6b [3238307,15350] 0 2024-05-06T12:14:49.200 INFO:tasks.workunit.client.0.smithi092.stdout:2/894: chown df/d17/d53/da9/f8b 1100 1 2024-05-06T12:14:49.210 INFO:tasks.workunit.client.1.smithi190.stdout:5/558: write d2/d27/f54 [353417,461] 0 2024-05-06T12:14:49.215 INFO:tasks.workunit.client.0.smithi092.stdout:2/895: dwrite df/d17/d53/da9/f8b [0,4194304] 0 2024-05-06T12:14:49.222 INFO:tasks.workunit.client.0.smithi092.stdout:2/896: dwrite df/d17/d1d/f21 [4194304,4194304] 0 2024-05-06T12:14:49.223 INFO:tasks.workunit.client.1.smithi190.stdout:2/454: dread d2/d8/d19/f24 [0,4194304] 0 2024-05-06T12:14:49.225 INFO:tasks.workunit.client.1.smithi190.stdout:2/455: write d2/d21/f70 [706901,3556] 0 2024-05-06T12:14:49.226 INFO:tasks.workunit.client.1.smithi190.stdout:5/559: dread d2/d27/f48 [0,4194304] 0 2024-05-06T12:14:49.229 INFO:tasks.workunit.client.1.smithi190.stdout:2/456: rename d2/d8/f90 to d2/d8/dc/f94 0 2024-05-06T12:14:49.230 INFO:tasks.workunit.client.1.smithi190.stdout:5/560: mkdir d2/da/d59/d5f/d50/d6d/db3 0 2024-05-06T12:14:49.230 INFO:tasks.workunit.client.0.smithi092.stdout:2/897: write df/d17/f2a [3891993,71506] 0 2024-05-06T12:14:49.230 INFO:tasks.workunit.client.0.smithi092.stdout:2/898: fdatasync fe 0 2024-05-06T12:14:49.232 INFO:tasks.workunit.client.0.smithi092.stdout:2/899: write df/d17/d53/feb [3411935,99583] 0 2024-05-06T12:14:49.232 INFO:tasks.workunit.client.1.smithi190.stdout:5/561: write d2/da/d59/d5f/d2e/d3b/fa5 [739548,72037] 0 2024-05-06T12:14:49.236 INFO:tasks.workunit.client.0.smithi092.stdout:2/900: creat df/d17/d53/da9/dd9/d83/f10e x:0 0 0 2024-05-06T12:14:49.239 INFO:tasks.workunit.client.0.smithi092.stdout:2/901: rename df/d17/d53/d85/da0/cd7 to df/c10f 0 2024-05-06T12:14:49.243 INFO:tasks.workunit.client.1.smithi190.stdout:2/457: getdents d2/d8 0 2024-05-06T12:14:49.243 INFO:tasks.workunit.client.0.smithi092.stdout:2/902: rename df/d17/d53/da9/dd9/d83/ff4 to df/d17/d53/da9/dd9/d83/dc2/d101/f110 0 2024-05-06T12:14:49.247 INFO:tasks.workunit.client.1.smithi190.stdout:2/458: link d2/d8/d33/f8f d2/d8/dc/f95 0 2024-05-06T12:14:49.249 INFO:tasks.workunit.client.1.smithi190.stdout:5/562: dwrite d2/f5 [0,4194304] 0 2024-05-06T12:14:49.250 INFO:tasks.workunit.client.0.smithi092.stdout:2/903: write df/f61 [732605,66002] 0 2024-05-06T12:14:49.252 INFO:tasks.workunit.client.1.smithi190.stdout:5/563: rmdir d2/da/d59/d5f/d2e/da7 39 2024-05-06T12:14:49.258 INFO:tasks.workunit.client.1.smithi190.stdout:5/564: creat d2/da/d59/d5f/d50/d6d/db3/fb4 x:0 0 0 2024-05-06T12:14:49.262 INFO:tasks.workunit.client.0.smithi092.stdout:2/904: dwrite df/dd0/de3/f48 [0,4194304] 0 2024-05-06T12:14:49.262 INFO:tasks.workunit.client.0.smithi092.stdout:2/905: chown df/dd0/de3/dcf 609 1 2024-05-06T12:14:49.265 INFO:tasks.workunit.client.1.smithi190.stdout:5/565: dwrite d2/da/d59/d5f/f5b [4194304,4194304] 0 2024-05-06T12:14:49.265 INFO:tasks.workunit.client.0.smithi092.stdout:2/906: symlink df/d17/d53/d85/da0/de0/l111 0 2024-05-06T12:14:49.269 INFO:tasks.workunit.client.1.smithi190.stdout:5/566: dread - d2/da/d59/d5f/d2e/da7/d9d/fa4 zero size 2024-05-06T12:14:49.270 INFO:tasks.workunit.client.1.smithi190.stdout:5/567: chown d2/da/l34 5607376 1 2024-05-06T12:14:49.271 INFO:tasks.workunit.client.1.smithi190.stdout:5/568: chown d2/da/d59/d5f/d50/d6d/db3/fb4 8 1 2024-05-06T12:14:49.271 INFO:tasks.workunit.client.1.smithi190.stdout:2/459: dread d2/f38 [0,4194304] 0 2024-05-06T12:14:49.273 INFO:tasks.workunit.client.1.smithi190.stdout:5/569: chown d2/da/d59/d5f/d6e 5813769 1 2024-05-06T12:14:49.277 INFO:tasks.workunit.client.1.smithi190.stdout:5/570: link d2/da/d59/d5f/d50/d6d/db3/fb4 d2/da/d59/d5f/d6e/fb5 0 2024-05-06T12:14:49.279 INFO:tasks.workunit.client.1.smithi190.stdout:5/571: mkdir d2/db6 0 2024-05-06T12:14:49.281 INFO:tasks.workunit.client.0.smithi092.stdout:2/907: dread df/dd0/de3/f1c [0,4194304] 0 2024-05-06T12:14:49.282 INFO:tasks.workunit.client.1.smithi190.stdout:5/572: dread d2/f5 [0,4194304] 0 2024-05-06T12:14:49.286 INFO:tasks.workunit.client.0.smithi092.stdout:2/908: mkdir df/d17/d1d/d112 0 2024-05-06T12:14:49.286 INFO:tasks.workunit.client.0.smithi092.stdout:2/909: truncate df/dd0/fdd 4324452 0 2024-05-06T12:14:49.286 INFO:tasks.workunit.client.0.smithi092.stdout:2/910: mknod df/dd0/de3/dcf/c113 0 2024-05-06T12:14:49.286 INFO:tasks.workunit.client.1.smithi190.stdout:2/460: getdents d2/d8/dc 0 2024-05-06T12:14:49.288 INFO:tasks.workunit.client.0.smithi092.stdout:2/911: rename f5 to df/d17/d53/d10b/f114 0 2024-05-06T12:14:49.289 INFO:tasks.workunit.client.1.smithi190.stdout:2/461: rename d2/d8/dc/l66 to d2/d8/d19/d43/d79/d7e/l96 0 2024-05-06T12:14:49.291 INFO:tasks.workunit.client.1.smithi190.stdout:2/462: mknod d2/d8/d33/d46/d55/c97 0 2024-05-06T12:14:49.293 INFO:tasks.workunit.client.0.smithi092.stdout:2/912: dwrite df/d17/d53/da9/fea [0,4194304] 0 2024-05-06T12:14:49.296 INFO:tasks.workunit.client.1.smithi190.stdout:2/463: dwrite d2/d8/d33/f37 [0,4194304] 0 2024-05-06T12:14:49.297 INFO:tasks.workunit.client.0.smithi092.stdout:2/913: creat df/d17/d51/d96/de2/f115 x:0 0 0 2024-05-06T12:14:49.299 INFO:tasks.workunit.client.1.smithi190.stdout:2/464: symlink d2/d8/d19/d43/d79/l98 0 2024-05-06T12:14:49.301 INFO:tasks.workunit.client.1.smithi190.stdout:2/465: symlink d2/d8/d19/l99 0 2024-05-06T12:14:49.302 INFO:tasks.workunit.client.0.smithi092.stdout:2/914: rename df/d15/l28 to df/dd0/l116 0 2024-05-06T12:14:49.302 INFO:tasks.workunit.client.1.smithi190.stdout:2/466: fsync d2/d8/d33/d46/d5d/f93 0 2024-05-06T12:14:49.304 INFO:tasks.workunit.client.1.smithi190.stdout:2/467: chown d2/d10/c61 49699 1 2024-05-06T12:14:49.306 INFO:tasks.workunit.client.1.smithi190.stdout:2/468: mkdir d2/d8/d33/d9a 0 2024-05-06T12:14:49.308 INFO:tasks.workunit.client.0.smithi092.stdout:2/915: dread df/d15/f95 [0,4194304] 0 2024-05-06T12:14:49.309 INFO:tasks.workunit.client.1.smithi190.stdout:2/469: getdents d2/d8/d19/d7b 0 2024-05-06T12:14:49.310 INFO:tasks.workunit.client.0.smithi092.stdout:2/916: fsync df/d17/d53/d85/f98 0 2024-05-06T12:14:49.311 INFO:tasks.workunit.client.1.smithi190.stdout:2/470: fsync d2/d8/d33/d46/d5d/f74 0 2024-05-06T12:14:49.313 INFO:tasks.workunit.client.1.smithi190.stdout:2/471: mkdir d2/d8/d19/d9b 0 2024-05-06T12:14:49.314 INFO:tasks.workunit.client.1.smithi190.stdout:2/472: mknod d2/d21/d50/c9c 0 2024-05-06T12:14:49.316 INFO:tasks.workunit.client.0.smithi092.stdout:2/917: dwrite df/d17/d51/d96/fe4 [0,4194304] 0 2024-05-06T12:14:49.317 INFO:tasks.workunit.client.0.smithi092.stdout:2/918: chown df/d15/c1a 776188821 1 2024-05-06T12:14:49.320 INFO:tasks.workunit.client.1.smithi190.stdout:2/473: dwrite d2/d8/d33/d46/d5d/f93 [0,4194304] 0 2024-05-06T12:14:49.320 INFO:tasks.workunit.client.0.smithi092.stdout:2/919: creat df/d17/d1d/df8/f117 x:0 0 0 2024-05-06T12:14:49.322 INFO:tasks.workunit.client.1.smithi190.stdout:2/474: write d2/d21/f70 [316928,57202] 0 2024-05-06T12:14:49.322 INFO:tasks.workunit.client.1.smithi190.stdout:5/573: dread d2/da/d59/d5f/f36 [0,4194304] 0 2024-05-06T12:14:49.323 INFO:tasks.workunit.client.0.smithi092.stdout:2/920: mkdir df/dd0/de3/d8e/dbb/d118 0 2024-05-06T12:14:49.326 INFO:tasks.workunit.client.0.smithi092.stdout:2/921: truncate df/f37 2626970 0 2024-05-06T12:14:49.327 INFO:tasks.workunit.client.0.smithi092.stdout:2/922: dread - df/dfb/d107/f89 zero size 2024-05-06T12:14:49.327 INFO:tasks.workunit.client.1.smithi190.stdout:1/393: sync 2024-05-06T12:14:49.332 INFO:tasks.workunit.client.1.smithi190.stdout:5/574: mknod d2/da/d59/d5f/d50/d6d/db3/cb7 0 2024-05-06T12:14:49.333 INFO:tasks.workunit.client.1.smithi190.stdout:1/394: creat d5/d4f/d68/f8e x:0 0 0 2024-05-06T12:14:49.334 INFO:tasks.workunit.client.1.smithi190.stdout:2/475: dwrite d2/d8/d19/d43/f7c [0,4194304] 0 2024-05-06T12:14:49.335 INFO:tasks.workunit.client.1.smithi190.stdout:5/575: write d2/f3 [2229030,96419] 0 2024-05-06T12:14:49.335 INFO:tasks.workunit.client.1.smithi190.stdout:1/395: dread - d5/f6d zero size 2024-05-06T12:14:49.336 INFO:tasks.workunit.client.1.smithi190.stdout:5/576: write d2/da/d59/d5f/d2e/d3b/f42 [622253,28149] 0 2024-05-06T12:14:49.341 INFO:tasks.workunit.client.1.smithi190.stdout:5/577: symlink d2/da/d59/d5f/lb8 0 2024-05-06T12:14:49.342 INFO:tasks.workunit.client.1.smithi190.stdout:1/396: dwrite d5/f34 [0,4194304] 0 2024-05-06T12:14:49.342 INFO:tasks.workunit.client.1.smithi190.stdout:5/578: chown d2/da/d59/d5f/d50/f51 634987235 1 2024-05-06T12:14:49.345 INFO:tasks.workunit.client.0.smithi092.stdout:2/923: dread df/dd0/de3/dcf/f81 [0,4194304] 0 2024-05-06T12:14:49.345 INFO:tasks.workunit.client.1.smithi190.stdout:1/397: chown d5/d4f/d65/d24/f2c 8035474 1 2024-05-06T12:14:49.347 INFO:tasks.workunit.client.1.smithi190.stdout:5/579: mknod d2/d27/cb9 0 2024-05-06T12:14:49.351 INFO:tasks.workunit.client.1.smithi190.stdout:5/580: dread d2/f44 [0,4194304] 0 2024-05-06T12:14:49.354 INFO:tasks.workunit.client.1.smithi190.stdout:5/581: symlink d2/da/d59/d5f/d2e/da3/lba 0 2024-05-06T12:14:49.358 INFO:tasks.workunit.client.1.smithi190.stdout:5/582: rename d2/l8f to d2/da/d59/d5f/d2e/da7/lbb 0 2024-05-06T12:14:49.360 INFO:tasks.workunit.client.1.smithi190.stdout:1/398: dread d5/d15/f30 [0,4194304] 0 2024-05-06T12:14:49.361 INFO:tasks.workunit.client.0.smithi092.stdout:2/924: truncate df/d17/d1d/f21 7083937 0 2024-05-06T12:14:49.361 INFO:tasks.workunit.client.0.smithi092.stdout:2/925: fdatasync df/f5b 0 2024-05-06T12:14:49.363 INFO:tasks.workunit.client.0.smithi092.stdout:2/926: unlink df/d17/d51/f9c 0 2024-05-06T12:14:49.364 INFO:tasks.workunit.client.1.smithi190.stdout:1/399: write d5/d4f/d65/d24/f28 [792730,77767] 0 2024-05-06T12:14:49.364 INFO:tasks.workunit.client.1.smithi190.stdout:2/476: dread d2/d8/dc/f13 [0,4194304] 0 2024-05-06T12:14:49.365 INFO:tasks.workunit.client.1.smithi190.stdout:1/400: write d5/d15/f17 [4235936,90590] 0 2024-05-06T12:14:49.368 INFO:tasks.workunit.client.1.smithi190.stdout:2/477: creat d2/d21/d50/f9d x:0 0 0 2024-05-06T12:14:49.369 INFO:tasks.workunit.client.1.smithi190.stdout:5/583: readlink d2/da/d59/d5f/d2e/da7/lbb 0 2024-05-06T12:14:49.370 INFO:tasks.workunit.client.0.smithi092.stdout:2/927: dwrite df/d17/d7e/fc3 [0,4194304] 0 2024-05-06T12:14:49.370 INFO:tasks.workunit.client.0.smithi092.stdout:2/928: readlink df/d17/d53/da9/dd9/l5d 0 2024-05-06T12:14:49.370 INFO:tasks.workunit.client.0.smithi092.stdout:2/929: chown df/dd0/de3/dcf/f81 983748 1 2024-05-06T12:14:49.371 INFO:tasks.workunit.client.1.smithi190.stdout:1/401: mkdir d5/d6b/d8f 0 2024-05-06T12:14:49.371 INFO:tasks.workunit.client.0.smithi092.stdout:2/930: chown df/dd0/de3/dcf/f47 28 1 2024-05-06T12:14:49.372 INFO:tasks.workunit.client.1.smithi190.stdout:2/478: symlink d2/d8/dc/l9e 0 2024-05-06T12:14:49.374 INFO:tasks.workunit.client.1.smithi190.stdout:2/479: creat d2/d21/f9f x:0 0 0 2024-05-06T12:14:49.375 INFO:tasks.workunit.client.0.smithi092.stdout:2/931: dread df/dd0/de3/dcf/f81 [0,4194304] 0 2024-05-06T12:14:49.375 INFO:tasks.workunit.client.0.smithi092.stdout:8/994: sync 2024-05-06T12:14:49.376 INFO:tasks.workunit.client.0.smithi092.stdout:3/868: sync 2024-05-06T12:14:49.376 INFO:tasks.workunit.client.1.smithi190.stdout:2/480: mkdir d2/d8/d19/d7b/da0 0 2024-05-06T12:14:49.378 INFO:tasks.workunit.client.0.smithi092.stdout:8/995: chown d9/d9d/lb5 5115 1 2024-05-06T12:14:49.379 INFO:tasks.workunit.client.0.smithi092.stdout:3/869: creat d8/d24/d33/d91/f118 x:0 0 0 2024-05-06T12:14:49.379 INFO:tasks.workunit.client.0.smithi092.stdout:2/932: mkdir df/dd0/d100/d119 0 2024-05-06T12:14:49.381 INFO:tasks.workunit.client.0.smithi092.stdout:8/996: symlink d9/d21/d37/d46/d5c/d62/d7a/l156 0 2024-05-06T12:14:49.381 INFO:tasks.workunit.client.0.smithi092.stdout:8/997: readlink d9/d21/d54/d79/lad 0 2024-05-06T12:14:49.381 INFO:tasks.workunit.client.0.smithi092.stdout:3/870: mknod d8/d1b/d73/c119 0 2024-05-06T12:14:49.383 INFO:tasks.workunit.client.0.smithi092.stdout:2/933: creat df/dfb/f11a x:0 0 0 2024-05-06T12:14:49.384 INFO:tasks.workunit.client.0.smithi092.stdout:8/998: truncate d9/d5a/dd9/d10b/f13d 483803 0 2024-05-06T12:14:49.385 INFO:tasks.workunit.client.0.smithi092.stdout:2/934: mkdir df/d17/d1d/d11b 0 2024-05-06T12:14:49.386 INFO:tasks.workunit.client.0.smithi092.stdout:8/999: getdents d9/d21/d37/d46/dee 0 2024-05-06T12:14:49.393 INFO:tasks.workunit.client.1.smithi190.stdout:1/402: dread d5/d4f/d65/d24/d43/f46 [0,4194304] 0 2024-05-06T12:14:49.406 INFO:tasks.workunit.client.1.smithi190.stdout:1/403: dread d5/d15/d1e/f27 [0,4194304] 0 2024-05-06T12:14:49.407 INFO:tasks.workunit.client.1.smithi190.stdout:1/404: chown d5/d4f/d65/d32 0 1 2024-05-06T12:14:49.418 INFO:tasks.workunit.client.1.smithi190.stdout:1/405: dwrite d5/d4f/d65/d24/f45 [0,4194304] 0 2024-05-06T12:14:49.418 INFO:tasks.workunit.client.1.smithi190.stdout:1/406: readlink d5/l14 0 2024-05-06T12:14:49.419 INFO:tasks.workunit.client.1.smithi190.stdout:1/407: write d5/fa [1977106,22923] 0 2024-05-06T12:14:49.427 INFO:tasks.workunit.client.1.smithi190.stdout:1/408: dwrite d5/d4f/d65/d32/f61 [0,4194304] 0 2024-05-06T12:14:49.434 INFO:tasks.workunit.client.1.smithi190.stdout:1/409: dwrite d5/d4f/d65/d32/d47/f77 [0,4194304] 0 2024-05-06T12:14:49.440 INFO:tasks.workunit.client.1.smithi190.stdout:1/410: link d5/d15/d1e/c40 d5/d4f/d65/d62/c90 0 2024-05-06T12:14:49.441 INFO:tasks.workunit.client.1.smithi190.stdout:1/411: chown d5/f12 12 1 2024-05-06T12:14:49.444 INFO:tasks.workunit.client.1.smithi190.stdout:1/412: rename d5/d4f/d65/d32/d47/f66 to d5/d4f/d65/d32/d47/d53/f91 0 2024-05-06T12:14:49.446 INFO:tasks.workunit.client.1.smithi190.stdout:1/413: rename d5/d4f/d65/c39 to d5/d15/d1e/c92 0 2024-05-06T12:14:49.448 INFO:tasks.workunit.client.1.smithi190.stdout:1/414: write d5/d4f/d65/d32/f70 [4798972,45153] 0 2024-05-06T12:14:49.449 INFO:tasks.workunit.client.1.smithi190.stdout:1/415: dread d5/d4f/d65/d2b/f35 [0,4194304] 0 2024-05-06T12:14:49.455 INFO:tasks.workunit.client.1.smithi190.stdout:1/416: dwrite d5/d4f/d65/f7b [0,4194304] 0 2024-05-06T12:14:49.456 INFO:tasks.workunit.client.1.smithi190.stdout:1/417: chown d5/d4f/d65/l5c 10440 1 2024-05-06T12:14:49.457 INFO:tasks.workunit.client.1.smithi190.stdout:1/418: truncate d5/f6d 218703 0 2024-05-06T12:14:49.458 INFO:tasks.workunit.client.1.smithi190.stdout:1/419: creat d5/d4f/d65/d62/f93 x:0 0 0 2024-05-06T12:14:49.460 INFO:tasks.workunit.client.1.smithi190.stdout:1/420: mkdir d5/d15/d1e/d94 0 2024-05-06T12:14:49.466 INFO:tasks.workunit.client.1.smithi190.stdout:1/421: dwrite d5/d4f/d65/d32/d47/d53/d6e/f79 [0,4194304] 0 2024-05-06T12:14:49.479 INFO:tasks.workunit.client.1.smithi190.stdout:7/382: sync 2024-05-06T12:14:49.480 INFO:tasks.workunit.client.1.smithi190.stdout:7/383: readlink d0/da/l90 0 2024-05-06T12:14:49.483 INFO:tasks.workunit.client.1.smithi190.stdout:7/384: link d0/d1/f11 d0/da/f93 0 2024-05-06T12:14:49.485 INFO:tasks.workunit.client.1.smithi190.stdout:7/385: creat d0/d1/d32/d45/d70/d3b/f94 x:0 0 0 2024-05-06T12:14:49.490 INFO:tasks.workunit.client.1.smithi190.stdout:7/386: creat d0/d1/f95 x:0 0 0 2024-05-06T12:14:49.500 INFO:tasks.workunit.client.1.smithi190.stdout:7/387: dread d0/da/d36/f46 [0,4194304] 0 2024-05-06T12:14:49.502 INFO:tasks.workunit.client.1.smithi190.stdout:7/388: creat d0/d1/d32/d45/d70/d3b/f96 x:0 0 0 2024-05-06T12:14:49.504 INFO:tasks.workunit.client.1.smithi190.stdout:7/389: unlink d0/d1/l56 0 2024-05-06T12:14:49.505 INFO:tasks.workunit.client.1.smithi190.stdout:7/390: mknod d0/d4c/c97 0 2024-05-06T12:14:49.506 INFO:tasks.workunit.client.1.smithi190.stdout:7/391: dread - d0/d1/d32/f44 zero size 2024-05-06T12:14:49.508 INFO:tasks.workunit.client.1.smithi190.stdout:7/392: creat d0/da/d36/d53/f98 x:0 0 0 2024-05-06T12:14:49.516 INFO:tasks.workunit.client.1.smithi190.stdout:7/393: truncate d0/d1/fb 1309475 0 2024-05-06T12:14:49.520 INFO:tasks.workunit.client.1.smithi190.stdout:7/394: write d0/da/f55 [112887,83887] 0 2024-05-06T12:14:49.523 INFO:tasks.workunit.client.1.smithi190.stdout:7/395: symlink d0/d1/d2/d84/l99 0 2024-05-06T12:14:49.524 INFO:tasks.workunit.client.1.smithi190.stdout:7/396: write d0/d1/fc [1151084,91006] 0 2024-05-06T12:14:49.525 INFO:tasks.workunit.client.1.smithi190.stdout:7/397: read d0/f20 [511023,38362] 0 2024-05-06T12:14:49.532 INFO:tasks.workunit.client.1.smithi190.stdout:7/398: dwrite d0/d1/f11 [0,4194304] 0 2024-05-06T12:14:49.534 INFO:tasks.workunit.client.1.smithi190.stdout:7/399: stat d0/d1/d32/d45/d70/d3b/f48 0 2024-05-06T12:14:49.540 INFO:tasks.workunit.client.1.smithi190.stdout:7/400: dwrite d0/d1/d18/d2b/d5a/f91 [0,4194304] 0 2024-05-06T12:14:49.542 INFO:tasks.workunit.client.1.smithi190.stdout:7/401: fdatasync d0/d4/f7 0 2024-05-06T12:14:49.542 INFO:tasks.workunit.client.1.smithi190.stdout:7/402: dread - d0/f40 zero size 2024-05-06T12:14:49.544 INFO:tasks.workunit.client.1.smithi190.stdout:7/403: write d0/da/d36/d53/f65 [4340571,77711] 0 2024-05-06T12:14:49.551 INFO:tasks.workunit.client.1.smithi190.stdout:7/404: dwrite d0/da/d36/d42/f85 [0,4194304] 0 2024-05-06T12:14:49.584 INFO:tasks.workunit.client.1.smithi190.stdout:7/405: symlink d0/d1/d32/d45/d70/l9a 0 2024-05-06T12:14:49.586 INFO:tasks.workunit.client.1.smithi190.stdout:7/406: creat d0/da/d36/d53/f9b x:0 0 0 2024-05-06T12:14:49.586 INFO:tasks.workunit.client.1.smithi190.stdout:7/407: dread - d0/da/d36/d53/f9b zero size 2024-05-06T12:14:49.587 INFO:tasks.workunit.client.1.smithi190.stdout:7/408: mknod d0/d1/d32/d45/c9c 0 2024-05-06T12:14:49.592 INFO:tasks.workunit.client.1.smithi190.stdout:7/409: dwrite d0/d1/d2/f1d [0,4194304] 0 2024-05-06T12:14:49.593 INFO:tasks.workunit.client.1.smithi190.stdout:7/410: dread - d0/da/d36/d53/f98 zero size 2024-05-06T12:14:49.593 INFO:tasks.workunit.client.1.smithi190.stdout:7/411: dread - d0/d1/d32/d45/d70/d3b/f96 zero size 2024-05-06T12:14:49.599 INFO:tasks.workunit.client.1.smithi190.stdout:7/412: link d0/d1/d2/f88 d0/da/d36/f9d 0 2024-05-06T12:14:49.602 INFO:tasks.workunit.client.1.smithi190.stdout:7/413: link d0/d14/f7b d0/d1/d18/d37/f9e 0 2024-05-06T12:14:49.604 INFO:tasks.workunit.client.1.smithi190.stdout:7/414: creat d0/d1/f9f x:0 0 0 2024-05-06T12:14:49.606 INFO:tasks.workunit.client.1.smithi190.stdout:7/415: mknod d0/d1/d2/d84/ca0 0 2024-05-06T12:14:49.607 INFO:tasks.workunit.client.1.smithi190.stdout:7/416: getdents d0/d1/d18/d6c 0 2024-05-06T12:14:49.607 INFO:tasks.workunit.client.1.smithi190.stdout:7/417: chown d0/d1/d18/f4f 953560 1 2024-05-06T12:14:49.608 INFO:tasks.workunit.client.1.smithi190.stdout:7/418: chown d0/d1/d32/d45/d70/d35 163 1 2024-05-06T12:14:49.617 INFO:tasks.workunit.client.1.smithi190.stdout:7/419: write d0/d1/f92 [4987802,102256] 0 2024-05-06T12:14:49.618 INFO:tasks.workunit.client.1.smithi190.stdout:7/420: fdatasync d0/d1/d18/d2b/f30 0 2024-05-06T12:14:49.618 INFO:tasks.workunit.client.1.smithi190.stdout:7/421: chown d0/d1/d2/l4b 852 1 2024-05-06T12:14:49.621 INFO:tasks.workunit.client.1.smithi190.stdout:0/561: sync 2024-05-06T12:14:49.627 INFO:tasks.workunit.client.1.smithi190.stdout:7/422: rename d0/d1/f11 to d0/d1/d18/d2b/d5a/fa1 0 2024-05-06T12:14:49.628 INFO:tasks.workunit.client.1.smithi190.stdout:0/562: creat d9/dc/d48/fcb x:0 0 0 2024-05-06T12:14:49.634 INFO:tasks.workunit.client.1.smithi190.stdout:0/563: dwrite d9/dc/d28/d4c/d96/fa0 [0,4194304] 0 2024-05-06T12:14:49.635 INFO:tasks.workunit.client.1.smithi190.stdout:0/564: chown d9/dc/d28/d4c/f4b 1618789 1 2024-05-06T12:14:49.641 INFO:tasks.workunit.client.1.smithi190.stdout:7/423: truncate d0/f19 4460020 0 2024-05-06T12:14:49.646 INFO:tasks.workunit.client.1.smithi190.stdout:7/424: dwrite d0/da/f5b [0,4194304] 0 2024-05-06T12:14:49.648 INFO:tasks.workunit.client.1.smithi190.stdout:0/565: write d9/d51/dc9/fa2 [3223722,5771] 0 2024-05-06T12:14:49.651 INFO:tasks.workunit.client.1.smithi190.stdout:0/566: mkdir d9/d95/dcc 0 2024-05-06T12:14:49.653 INFO:tasks.workunit.client.1.smithi190.stdout:0/567: write d9/dc/db6/fb7 [604135,47066] 0 2024-05-06T12:14:49.655 INFO:tasks.workunit.client.1.smithi190.stdout:7/425: write d0/f19 [3477423,39165] 0 2024-05-06T12:14:49.655 INFO:tasks.workunit.client.1.smithi190.stdout:7/426: readlink d0/da/d36/l78 0 2024-05-06T12:14:49.656 INFO:tasks.workunit.client.1.smithi190.stdout:0/568: creat d9/df/fcd x:0 0 0 2024-05-06T12:14:49.657 INFO:tasks.workunit.client.1.smithi190.stdout:7/427: creat d0/d1/d18/fa2 x:0 0 0 2024-05-06T12:14:49.658 INFO:tasks.workunit.client.1.smithi190.stdout:0/569: unlink d9/d51/dc9/c86 0 2024-05-06T12:14:49.660 INFO:tasks.workunit.client.1.smithi190.stdout:0/570: chown d9/dc/d28/d4c/f53 1226743 1 2024-05-06T12:14:49.662 INFO:tasks.workunit.client.1.smithi190.stdout:7/428: dwrite d0/f22 [0,4194304] 0 2024-05-06T12:14:49.663 INFO:tasks.workunit.client.1.smithi190.stdout:7/429: read d0/da/f5b [1081279,4404] 0 2024-05-06T12:14:49.667 INFO:tasks.workunit.client.1.smithi190.stdout:7/430: mknod d0/d1/d32/d45/d70/d35/ca3 0 2024-05-06T12:14:49.672 INFO:tasks.workunit.client.1.smithi190.stdout:7/431: link d0/d1/d18/f68 d0/d1/d32/d45/d70/d35/fa4 0 2024-05-06T12:14:49.673 INFO:tasks.workunit.client.1.smithi190.stdout:7/432: creat d0/d1/d18/d4e/d87/fa5 x:0 0 0 2024-05-06T12:14:49.675 INFO:tasks.workunit.client.1.smithi190.stdout:7/433: mknod d0/d1/d18/d2b/ca6 0 2024-05-06T12:14:49.675 INFO:tasks.workunit.client.1.smithi190.stdout:7/434: write d0/d4/fd [3219158,8042] 0 2024-05-06T12:14:49.678 INFO:tasks.workunit.client.1.smithi190.stdout:7/435: creat d0/da/d36/d42/fa7 x:0 0 0 2024-05-06T12:14:49.682 INFO:tasks.workunit.client.1.smithi190.stdout:7/436: getdents d0/d1 0 2024-05-06T12:14:49.683 INFO:tasks.workunit.client.1.smithi190.stdout:7/437: chown d0/d1/d18/d2b/d5a/fa1 8 1 2024-05-06T12:14:49.693 INFO:tasks.workunit.client.1.smithi190.stdout:7/438: dread d0/da/d36/d53/f65 [0,4194304] 0 2024-05-06T12:14:49.696 INFO:tasks.workunit.client.1.smithi190.stdout:7/439: dread d0/d4c/f52 [0,4194304] 0 2024-05-06T12:14:49.699 INFO:tasks.workunit.client.0.smithi092.stdout:3/871: sync 2024-05-06T12:14:49.699 INFO:tasks.workunit.client.0.smithi092.stdout:3/872: stat d8/d1b/d73/db7 0 2024-05-06T12:14:49.701 INFO:tasks.workunit.client.1.smithi190.stdout:7/440: dwrite d0/f40 [0,4194304] 0 2024-05-06T12:14:49.704 INFO:tasks.workunit.client.1.smithi190.stdout:7/441: creat d0/d4c/d60/fa8 x:0 0 0 2024-05-06T12:14:49.706 INFO:tasks.workunit.client.0.smithi092.stdout:3/873: rename d8/d24/d33/d4b/c55 to d8/d24/d33/d4b/c11a 0 2024-05-06T12:14:49.706 INFO:tasks.workunit.client.1.smithi190.stdout:7/442: rename d0/d14/l61 to d0/d1/d18/d2b/la9 0 2024-05-06T12:14:49.707 INFO:tasks.workunit.client.0.smithi092.stdout:3/874: dread - d8/d24/d33/d4b/fe5 zero size 2024-05-06T12:14:49.707 INFO:tasks.workunit.client.1.smithi190.stdout:7/443: mkdir d0/d1/d32/daa 0 2024-05-06T12:14:49.713 INFO:tasks.workunit.client.0.smithi092.stdout:3/875: dwrite d8/d1b/d73/db7/fdc [0,4194304] 0 2024-05-06T12:14:49.714 INFO:tasks.workunit.client.1.smithi190.stdout:7/444: read d0/d1/d32/d45/d70/d35/fa4 [4439,2622] 0 2024-05-06T12:14:49.716 INFO:tasks.workunit.client.0.smithi092.stdout:3/876: mknod d8/d24/d27/c11b 0 2024-05-06T12:14:49.716 INFO:tasks.workunit.client.1.smithi190.stdout:7/445: mknod d0/da/d36/d53/cab 0 2024-05-06T12:14:49.719 INFO:tasks.workunit.client.1.smithi190.stdout:7/446: mkdir d0/d4c/d60/dac 0 2024-05-06T12:14:49.719 INFO:tasks.workunit.client.0.smithi092.stdout:3/877: symlink d8/db/d4c/d3d/d49/df8/d5b/d101/l11c 0 2024-05-06T12:14:49.722 INFO:tasks.workunit.client.0.smithi092.stdout:3/878: creat d8/db/d4c/d3d/d48/dee/f11d x:0 0 0 2024-05-06T12:14:49.723 INFO:tasks.workunit.client.0.smithi092.stdout:3/879: stat d8/d1b/d73/db7 0 2024-05-06T12:14:49.726 INFO:tasks.workunit.client.0.smithi092.stdout:3/880: mkdir d8/db/d4c/d40/d11e 0 2024-05-06T12:14:49.726 INFO:tasks.workunit.client.1.smithi190.stdout:7/447: dwrite d0/f22 [0,4194304] 0 2024-05-06T12:14:49.728 INFO:tasks.workunit.client.0.smithi092.stdout:2/935: sync 2024-05-06T12:14:49.730 INFO:tasks.workunit.client.0.smithi092.stdout:2/936: truncate df/d17/d53/da9/dd9/dd5/fe1 424981 0 2024-05-06T12:14:49.731 INFO:tasks.workunit.client.0.smithi092.stdout:3/881: getdents d8/db/d14 0 2024-05-06T12:14:49.731 INFO:tasks.workunit.client.1.smithi190.stdout:7/448: write d0/d1/d32/d45/d70/d35/fa4 [3633551,87075] 0 2024-05-06T12:14:49.734 INFO:tasks.workunit.client.0.smithi092.stdout:2/937: mknod df/c11c 0 2024-05-06T12:14:49.734 INFO:tasks.workunit.client.1.smithi190.stdout:7/449: unlink d0/da/d36/d42/f76 0 2024-05-06T12:14:49.736 INFO:tasks.workunit.client.0.smithi092.stdout:3/882: truncate d8/d24/d33/d4b/ddf/feb 408657 0 2024-05-06T12:14:49.737 INFO:tasks.workunit.client.0.smithi092.stdout:2/938: creat df/d17/d7d/f11d x:0 0 0 2024-05-06T12:14:49.740 INFO:tasks.workunit.client.0.smithi092.stdout:3/883: rmdir d8/db/d4c/dc1 39 2024-05-06T12:14:49.740 INFO:tasks.workunit.client.1.smithi190.stdout:7/450: dwrite d0/d14/f6e [0,4194304] 0 2024-05-06T12:14:49.740 INFO:tasks.workunit.client.0.smithi092.stdout:2/939: dread - df/d17/d51/d96/fac zero size 2024-05-06T12:14:49.741 INFO:tasks.workunit.client.0.smithi092.stdout:3/884: dread d8/db/f6c [0,4194304] 0 2024-05-06T12:14:49.743 INFO:tasks.workunit.client.0.smithi092.stdout:2/940: mkdir df/dd0/de3/db4/dce/d11e 0 2024-05-06T12:14:49.744 INFO:tasks.workunit.client.0.smithi092.stdout:3/885: mknod d8/db/d4c/d3d/d49/df8/d5b/d98/c11f 0 2024-05-06T12:14:49.745 INFO:tasks.workunit.client.1.smithi190.stdout:7/451: symlink d0/d14/lad 0 2024-05-06T12:14:49.747 INFO:tasks.workunit.client.0.smithi092.stdout:2/941: dread df/d17/f22 [0,4194304] 0 2024-05-06T12:14:49.750 INFO:tasks.workunit.client.0.smithi092.stdout:3/886: creat d8/db/d4c/d3d/f120 x:0 0 0 2024-05-06T12:14:49.755 INFO:tasks.workunit.client.1.smithi190.stdout:8/596: sync 2024-05-06T12:14:49.755 INFO:tasks.workunit.client.1.smithi190.stdout:4/389: sync 2024-05-06T12:14:49.757 INFO:tasks.workunit.client.0.smithi092.stdout:2/942: write df/d17/d51/de9/fef [1003306,115152] 0 2024-05-06T12:14:49.757 INFO:tasks.workunit.client.0.smithi092.stdout:2/943: readlink df/dfb/d107/d59/l7f 0 2024-05-06T12:14:49.759 INFO:tasks.workunit.client.1.smithi190.stdout:8/597: rename d3/d8/d2b/d3e/cab to d3/d8/d2b/d3e/dad/cd1 0 2024-05-06T12:14:49.759 INFO:tasks.workunit.client.1.smithi190.stdout:4/390: creat d2/d11/d16/d61/d66/d6b/f88 x:0 0 0 2024-05-06T12:14:49.761 INFO:tasks.workunit.client.1.smithi190.stdout:7/452: dread d0/d14/f7b [0,4194304] 0 2024-05-06T12:14:49.762 INFO:tasks.workunit.client.1.smithi190.stdout:8/598: symlink d3/d8/dcf/ld2 0 2024-05-06T12:14:49.762 INFO:tasks.workunit.client.1.smithi190.stdout:4/391: read d2/d11/d16/d53/f57 [1072870,17737] 0 2024-05-06T12:14:49.764 INFO:tasks.workunit.client.0.smithi092.stdout:2/944: write df/dfb/d107/d59/fb0 [2318205,95936] 0 2024-05-06T12:14:49.764 INFO:tasks.workunit.client.1.smithi190.stdout:8/599: creat d3/d8/d2b/d3e/dad/fd3 x:0 0 0 2024-05-06T12:14:49.765 INFO:tasks.workunit.client.1.smithi190.stdout:7/453: rename d0/da/d36/d53/f65 to d0/d4c/d60/dac/fae 0 2024-05-06T12:14:49.768 INFO:tasks.workunit.client.1.smithi190.stdout:7/454: symlink d0/d1/laf 0 2024-05-06T12:14:49.774 INFO:tasks.workunit.client.0.smithi092.stdout:3/887: dread d8/d24/d27/fa0 [4194304,4194304] 0 2024-05-06T12:14:49.776 INFO:tasks.workunit.client.1.smithi190.stdout:7/455: truncate d0/d1/d18/d4e/d87/fa5 380159 0 2024-05-06T12:14:49.776 INFO:tasks.workunit.client.1.smithi190.stdout:8/600: dwrite d3/d8/fd [4194304,4194304] 0 2024-05-06T12:14:49.778 INFO:tasks.workunit.client.1.smithi190.stdout:7/456: readlink d0/da/d36/l75 0 2024-05-06T12:14:49.779 INFO:tasks.workunit.client.0.smithi092.stdout:2/945: dread f1 [0,4194304] 0 2024-05-06T12:14:49.779 INFO:tasks.workunit.client.0.smithi092.stdout:3/888: write d8/db/d14/d5a/f94 [5101088,57312] 0 2024-05-06T12:14:49.780 INFO:tasks.workunit.client.0.smithi092.stdout:2/946: write df/d17/d53/f10d [395208,6186] 0 2024-05-06T12:14:49.783 INFO:tasks.workunit.client.1.smithi190.stdout:7/457: mkdir d0/d1/d32/db0 0 2024-05-06T12:14:49.784 INFO:tasks.workunit.client.0.smithi092.stdout:2/947: mknod df/d17/d1d/d112/c11f 0 2024-05-06T12:14:49.785 INFO:tasks.workunit.client.1.smithi190.stdout:7/458: chown d0/d1/f92 424112874 1 2024-05-06T12:14:49.787 INFO:tasks.workunit.client.0.smithi092.stdout:2/948: unlink df/d17/d7d/df0/cf9 0 2024-05-06T12:14:49.787 INFO:tasks.workunit.client.1.smithi190.stdout:7/459: rmdir d0/da/d36 39 2024-05-06T12:14:49.788 INFO:tasks.workunit.client.1.smithi190.stdout:7/460: stat d0/da/f55 0 2024-05-06T12:14:49.789 INFO:tasks.workunit.client.0.smithi092.stdout:2/949: stat df/dd0/de3/dcf/ff7 0 2024-05-06T12:14:49.791 INFO:tasks.workunit.client.1.smithi190.stdout:7/461: unlink d0/d4/l1f 0 2024-05-06T12:14:49.794 INFO:tasks.workunit.client.1.smithi190.stdout:7/462: creat d0/d1/d32/daa/fb1 x:0 0 0 2024-05-06T12:14:49.797 INFO:tasks.workunit.client.1.smithi190.stdout:7/463: symlink d0/da/d36/d42/lb2 0 2024-05-06T12:14:49.798 INFO:tasks.workunit.client.1.smithi190.stdout:8/601: dread d3/f2d [0,4194304] 0 2024-05-06T12:14:49.799 INFO:tasks.workunit.client.0.smithi092.stdout:2/950: dwrite df/d17/d51/f57 [0,4194304] 0 2024-05-06T12:14:49.800 INFO:tasks.workunit.client.1.smithi190.stdout:8/602: symlink d3/d13/ld4 0 2024-05-06T12:14:49.801 INFO:tasks.workunit.client.1.smithi190.stdout:8/603: readlink d3/l1e 0 2024-05-06T12:14:49.801 INFO:tasks.workunit.client.0.smithi092.stdout:2/951: creat df/d17/d1d/df8/f120 x:0 0 0 2024-05-06T12:14:49.802 INFO:tasks.workunit.client.1.smithi190.stdout:8/604: dread - d3/d8/fa2 zero size 2024-05-06T12:14:49.804 INFO:tasks.workunit.client.1.smithi190.stdout:7/464: dwrite d0/f20 [4194304,4194304] 0 2024-05-06T12:14:49.804 INFO:tasks.workunit.client.0.smithi092.stdout:2/952: rmdir df/dd0/de3/d8e 39 2024-05-06T12:14:49.805 INFO:tasks.workunit.client.1.smithi190.stdout:7/465: write d0/d4/f6b [520357,11906] 0 2024-05-06T12:14:49.807 INFO:tasks.workunit.client.0.smithi092.stdout:2/953: creat df/dd0/de3/db4/dce/f121 x:0 0 0 2024-05-06T12:14:49.809 INFO:tasks.workunit.client.0.smithi092.stdout:3/889: dread d8/db/d14/d5a/f8f [0,4194304] 0 2024-05-06T12:14:49.811 INFO:tasks.workunit.client.1.smithi190.stdout:8/605: creat d3/d8/d25/d3b/d6b/fd5 x:0 0 0 2024-05-06T12:14:49.811 INFO:tasks.workunit.client.0.smithi092.stdout:3/890: chown d8/d24/d27/cba 2892020 1 2024-05-06T12:14:49.812 INFO:tasks.workunit.client.0.smithi092.stdout:2/954: dread df/d17/d7e/fc3 [0,4194304] 0 2024-05-06T12:14:49.813 INFO:tasks.workunit.client.0.smithi092.stdout:2/955: stat fd 0 2024-05-06T12:14:49.813 INFO:tasks.workunit.client.0.smithi092.stdout:3/891: write d8/d24/d27/fb6 [1287063,104580] 0 2024-05-06T12:14:49.813 INFO:tasks.workunit.client.1.smithi190.stdout:8/606: symlink d3/d8/d25/d3b/d90/ld6 0 2024-05-06T12:14:49.817 INFO:tasks.workunit.client.1.smithi190.stdout:8/607: link d3/d8/d25/d3b/d6b/fd5 d3/fd7 0 2024-05-06T12:14:49.817 INFO:tasks.workunit.client.0.smithi092.stdout:3/892: rmdir d8/db/d4c/d3d 39 2024-05-06T12:14:49.818 INFO:tasks.workunit.client.1.smithi190.stdout:8/608: fsync d3/d8/fa2 0 2024-05-06T12:14:49.818 INFO:tasks.workunit.client.1.smithi190.stdout:8/609: chown d3/d35/d55/d61 0 1 2024-05-06T12:14:49.818 INFO:tasks.workunit.client.0.smithi092.stdout:3/893: write d8/d24/d27/fb6 [117134,72492] 0 2024-05-06T12:14:49.819 INFO:tasks.workunit.client.1.smithi190.stdout:8/610: creat d3/d8/d68/fd8 x:0 0 0 2024-05-06T12:14:49.820 INFO:tasks.workunit.client.1.smithi190.stdout:8/611: readlink d3/l20 0 2024-05-06T12:14:49.821 INFO:tasks.workunit.client.0.smithi092.stdout:3/894: fdatasync d8/db/d4c/d3d/f8e 0 2024-05-06T12:14:49.823 INFO:tasks.workunit.client.0.smithi092.stdout:3/895: truncate d8/db/d4c/d40/d67/fe9 337959 0 2024-05-06T12:14:49.824 INFO:tasks.workunit.client.0.smithi092.stdout:2/956: dread df/d17/d51/f67 [0,4194304] 0 2024-05-06T12:14:49.825 INFO:tasks.workunit.client.0.smithi092.stdout:2/957: write df/fd1 [335823,37581] 0 2024-05-06T12:14:49.826 INFO:tasks.workunit.client.0.smithi092.stdout:3/896: mkdir d8/db/d4c/d3d/d49/ded/d121 0 2024-05-06T12:14:49.828 INFO:tasks.workunit.client.1.smithi190.stdout:7/466: truncate d0/d1/f21 1538967 0 2024-05-06T12:14:49.830 INFO:tasks.workunit.client.1.smithi190.stdout:7/467: write d0/d1/f95 [310374,95903] 0 2024-05-06T12:14:49.831 INFO:tasks.workunit.client.1.smithi190.stdout:7/468: chown d0/da/c47 9 1 2024-05-06T12:14:49.834 INFO:tasks.workunit.client.0.smithi092.stdout:3/897: truncate d8/db/d14/d5a/f94 1225407 0 2024-05-06T12:14:49.835 INFO:tasks.workunit.client.1.smithi190.stdout:7/469: unlink d0/d4c/d60/dac/fae 0 2024-05-06T12:14:49.837 INFO:tasks.workunit.client.1.smithi190.stdout:7/470: write d0/d1/d32/d45/d70/d3b/f94 [456932,58664] 0 2024-05-06T12:14:49.838 INFO:tasks.workunit.client.1.smithi190.stdout:7/471: write d0/d1/d32/d45/d70/d35/fa4 [3398484,39020] 0 2024-05-06T12:14:49.840 INFO:tasks.workunit.client.1.smithi190.stdout:7/472: mkdir d0/d1/d18/d4e/db3 0 2024-05-06T12:14:49.842 INFO:tasks.workunit.client.1.smithi190.stdout:7/473: rmdir d0/d1/d32/d45 39 2024-05-06T12:14:49.844 INFO:tasks.workunit.client.1.smithi190.stdout:7/474: fdatasync d0/d1/d32/d45/d70/d35/fa4 0 2024-05-06T12:14:49.846 INFO:tasks.workunit.client.1.smithi190.stdout:7/475: chown d0/d4/f39 31248 1 2024-05-06T12:14:49.848 INFO:tasks.workunit.client.1.smithi190.stdout:7/476: mkdir d0/db4 0 2024-05-06T12:14:49.850 INFO:tasks.workunit.client.1.smithi190.stdout:7/477: dread d0/da/f93 [0,4194304] 0 2024-05-06T12:14:49.851 INFO:tasks.workunit.client.1.smithi190.stdout:7/478: dread - d0/d1/d18/d4e/d87/f8b zero size 2024-05-06T12:14:49.854 INFO:tasks.workunit.client.1.smithi190.stdout:7/479: creat d0/da/d36/d53/fb5 x:0 0 0 2024-05-06T12:14:49.858 INFO:tasks.workunit.client.1.smithi190.stdout:7/480: dwrite d0/d1/d18/d2b/f80 [0,4194304] 0 2024-05-06T12:14:49.861 INFO:tasks.workunit.client.1.smithi190.stdout:7/481: mknod d0/d4c/d60/dac/cb6 0 2024-05-06T12:14:49.862 INFO:tasks.workunit.client.1.smithi190.stdout:7/482: chown d0/d1/d2/f7c 5003631 1 2024-05-06T12:14:49.865 INFO:tasks.workunit.client.1.smithi190.stdout:7/483: mknod d0/da/d36/d53/d77/cb7 0 2024-05-06T12:14:49.867 INFO:tasks.workunit.client.1.smithi190.stdout:7/484: mkdir d0/d1/d32/d45/d70/d35/db8 0 2024-05-06T12:14:49.874 INFO:tasks.workunit.client.1.smithi190.stdout:7/485: truncate d0/da/f55 98084 0 2024-05-06T12:14:49.875 INFO:tasks.workunit.client.1.smithi190.stdout:7/486: chown d0/d1/d2/l4b 229 1 2024-05-06T12:14:49.878 INFO:tasks.workunit.client.1.smithi190.stdout:7/487: dread d0/ff [0,4194304] 0 2024-05-06T12:14:49.891 INFO:tasks.workunit.client.1.smithi190.stdout:7/488: dread d0/da/d36/d53/f62 [0,4194304] 0 2024-05-06T12:14:49.893 INFO:tasks.workunit.client.1.smithi190.stdout:7/489: getdents d0/db4 0 2024-05-06T12:14:49.895 INFO:tasks.workunit.client.1.smithi190.stdout:7/490: link d0/d1/d18/d2b/f83 d0/da/d36/d53/d5c/fb9 0 2024-05-06T12:14:49.898 INFO:tasks.workunit.client.1.smithi190.stdout:7/491: dread d0/f20 [0,4194304] 0 2024-05-06T12:14:49.908 INFO:tasks.workunit.client.1.smithi190.stdout:7/492: dwrite d0/d1/d32/d45/f58 [0,4194304] 0 2024-05-06T12:14:49.912 INFO:tasks.workunit.client.1.smithi190.stdout:7/493: creat d0/d1/d18/d6c/fba x:0 0 0 2024-05-06T12:14:49.913 INFO:tasks.workunit.client.1.smithi190.stdout:7/494: chown d0/da/d36/d53/f98 815456 1 2024-05-06T12:14:49.920 INFO:tasks.workunit.client.1.smithi190.stdout:7/495: truncate d0/d1/d18/d2b/d5a/f5e 2365199 0 2024-05-06T12:14:49.921 INFO:tasks.workunit.client.1.smithi190.stdout:7/496: mknod d0/d4c/cbb 0 2024-05-06T12:14:49.922 INFO:tasks.workunit.client.1.smithi190.stdout:7/497: fsync d0/f20 0 2024-05-06T12:14:49.926 INFO:tasks.workunit.client.1.smithi190.stdout:9/616: sync 2024-05-06T12:14:49.929 INFO:tasks.workunit.client.1.smithi190.stdout:9/617: fsync db/dd/d16/d58/d87/d22/f55 0 2024-05-06T12:14:49.938 INFO:tasks.workunit.client.1.smithi190.stdout:9/618: dwrite db/dd/f41 [0,4194304] 0 2024-05-06T12:14:49.941 INFO:tasks.workunit.client.1.smithi190.stdout:9/619: rename db/d23/d25/c48 to db/dd/d16/d58/d87/d99/cd3 0 2024-05-06T12:14:49.943 INFO:tasks.workunit.client.1.smithi190.stdout:9/620: fdatasync db/dd/f8d 0 2024-05-06T12:14:49.945 INFO:tasks.workunit.client.1.smithi190.stdout:9/621: write db/dd/d16/d58/d87/d22/d49/d8c/fc1 [914940,121437] 0 2024-05-06T12:14:49.947 INFO:tasks.workunit.client.1.smithi190.stdout:9/622: write db/d23/d25/fa1 [2506259,48320] 0 2024-05-06T12:14:49.947 INFO:tasks.workunit.client.1.smithi190.stdout:7/498: dread d0/d1/d32/d45/d70/d35/fa4 [0,4194304] 0 2024-05-06T12:14:49.950 INFO:tasks.workunit.client.1.smithi190.stdout:7/499: mknod d0/d1/d18/d4e/d87/cbc 0 2024-05-06T12:14:49.951 INFO:tasks.workunit.client.1.smithi190.stdout:9/623: truncate db/d23/f68 2844130 0 2024-05-06T12:14:49.953 INFO:tasks.workunit.client.1.smithi190.stdout:7/500: link d0/da/l90 d0/d1/lbd 0 2024-05-06T12:14:49.955 INFO:tasks.workunit.client.1.smithi190.stdout:7/501: stat d0/d1/d32/d45/d70 0 2024-05-06T12:14:49.957 INFO:tasks.workunit.client.1.smithi190.stdout:7/502: write d0/d1/fc [2180164,14196] 0 2024-05-06T12:14:49.959 INFO:tasks.workunit.client.1.smithi190.stdout:7/503: creat d0/d1/fbe x:0 0 0 2024-05-06T12:14:49.964 INFO:tasks.workunit.client.1.smithi190.stdout:9/624: dread db/dd/f34 [0,4194304] 0 2024-05-06T12:14:49.967 INFO:tasks.workunit.client.1.smithi190.stdout:7/504: dwrite d0/d1/d18/f8a [0,4194304] 0 2024-05-06T12:14:49.971 INFO:tasks.workunit.client.1.smithi190.stdout:7/505: unlink d0/d1/d18/f4f 0 2024-05-06T12:14:49.977 INFO:tasks.workunit.client.1.smithi190.stdout:9/625: dread db/d23/f3a [0,4194304] 0 2024-05-06T12:14:49.982 INFO:tasks.workunit.client.1.smithi190.stdout:7/506: dwrite d0/f20 [4194304,4194304] 0 2024-05-06T12:14:49.983 INFO:tasks.workunit.client.1.smithi190.stdout:7/507: write d0/d1/d18/f68 [785563,52918] 0 2024-05-06T12:14:49.989 INFO:tasks.workunit.client.1.smithi190.stdout:7/508: rename d0/d1/f95 to d0/d1/d2/d84/fbf 0 2024-05-06T12:14:49.995 INFO:tasks.workunit.client.1.smithi190.stdout:7/509: dwrite d0/d1/d2/f41 [0,4194304] 0 2024-05-06T12:14:50.001 INFO:tasks.workunit.client.1.smithi190.stdout:7/510: getdents d0/d4c/d60 0 2024-05-06T12:14:50.001 INFO:tasks.workunit.client.1.smithi190.stdout:7/511: dread - d0/f69 zero size 2024-05-06T12:14:50.002 INFO:tasks.workunit.client.1.smithi190.stdout:7/512: fsync d0/d1/d2/f1d 0 2024-05-06T12:14:50.003 INFO:tasks.workunit.client.1.smithi190.stdout:7/513: chown d0/d1/d18/d2b/ca6 212320841 1 2024-05-06T12:14:50.005 INFO:tasks.workunit.client.1.smithi190.stdout:9/626: dread db/d23/d25/fa1 [0,4194304] 0 2024-05-06T12:14:50.008 INFO:tasks.workunit.client.1.smithi190.stdout:7/514: link d0/d1/f3e d0/d1/d32/d45/d70/d33/d57/fc0 0 2024-05-06T12:14:50.009 INFO:tasks.workunit.client.1.smithi190.stdout:9/627: truncate db/dd/d16/d58/d87/d22/f51 1864644 0 2024-05-06T12:14:50.010 INFO:tasks.workunit.client.1.smithi190.stdout:7/515: mknod d0/da/d36/cc1 0 2024-05-06T12:14:50.012 INFO:tasks.workunit.client.1.smithi190.stdout:9/628: write db/dd/d16/d58/d87/d22/d49/d88/f86 [1267607,97232] 0 2024-05-06T12:14:50.015 INFO:tasks.workunit.client.1.smithi190.stdout:7/516: link d0/d1/d2/f7c d0/d4c/d60/fc2 0 2024-05-06T12:14:50.015 INFO:tasks.workunit.client.1.smithi190.stdout:9/629: truncate db/dd/d16/d58/d87/d22/d49/d8c/fa0 2140905 0 2024-05-06T12:14:50.016 INFO:tasks.workunit.client.1.smithi190.stdout:7/517: fsync d0/d1/d2/f5 0 2024-05-06T12:14:50.017 INFO:tasks.workunit.client.1.smithi190.stdout:7/518: fsync d0/da/d36/d53/f98 0 2024-05-06T12:14:50.018 INFO:tasks.workunit.client.1.smithi190.stdout:9/630: mknod db/dd/d5c/dac/cd4 0 2024-05-06T12:14:50.020 INFO:tasks.workunit.client.1.smithi190.stdout:9/631: dread db/dd/d16/d58/d87/d22/d49/d88/db5/fbd [0,4194304] 0 2024-05-06T12:14:50.022 INFO:tasks.workunit.client.1.smithi190.stdout:9/632: read db/dd/d16/d58/d87/d22/d49/d8c/fa0 [124529,88510] 0 2024-05-06T12:14:50.028 INFO:tasks.workunit.client.1.smithi190.stdout:9/633: dwrite db/d23/d7e/fa8 [0,4194304] 0 2024-05-06T12:14:50.029 INFO:tasks.workunit.client.1.smithi190.stdout:7/519: write d0/d1/fb [1067099,18660] 0 2024-05-06T12:14:50.031 INFO:tasks.workunit.client.1.smithi190.stdout:9/634: truncate db/f1f 1209041 0 2024-05-06T12:14:50.032 INFO:tasks.workunit.client.1.smithi190.stdout:7/520: mkdir d0/d1/d32/d45/d70/d33/d57/dc3 0 2024-05-06T12:14:50.033 INFO:tasks.workunit.client.0.smithi092.stdout:3/898: sync 2024-05-06T12:14:50.033 INFO:tasks.workunit.client.0.smithi092.stdout:2/958: sync 2024-05-06T12:14:50.035 INFO:tasks.workunit.client.0.smithi092.stdout:2/959: chown df/d17/d1d/d112/c11f 5086 1 2024-05-06T12:14:50.036 INFO:tasks.workunit.client.0.smithi092.stdout:3/899: fdatasync d8/d24/d27/f2e 0 2024-05-06T12:14:50.037 INFO:tasks.workunit.client.1.smithi190.stdout:9/635: link db/d23/d25/d47/d4b/f66 db/dd/d16/d4c/d4e/d5e/dce/fd5 0 2024-05-06T12:14:50.038 INFO:tasks.workunit.client.0.smithi092.stdout:3/900: symlink d8/dd7/l122 0 2024-05-06T12:14:50.039 INFO:tasks.workunit.client.0.smithi092.stdout:2/960: truncate df/d17/d1d/f21 7894878 0 2024-05-06T12:14:50.039 INFO:tasks.workunit.client.1.smithi190.stdout:7/521: dwrite d0/da/f2e [0,4194304] 0 2024-05-06T12:14:50.040 INFO:tasks.workunit.client.1.smithi190.stdout:3/589: sync 2024-05-06T12:14:50.040 INFO:tasks.workunit.client.1.smithi190.stdout:6/388: sync 2024-05-06T12:14:50.040 INFO:tasks.workunit.client.1.smithi190.stdout:7/522: dread - d0/f69 zero size 2024-05-06T12:14:50.041 INFO:tasks.workunit.client.1.smithi190.stdout:3/590: dread - d5/d79/f95 zero size 2024-05-06T12:14:50.042 INFO:tasks.workunit.client.0.smithi092.stdout:3/901: link d8/db/ffd d8/db/d4c/d3d/d49/ded/d121/f123 0 2024-05-06T12:14:50.043 INFO:tasks.workunit.client.1.smithi190.stdout:9/636: symlink db/daa/ld6 0 2024-05-06T12:14:50.045 INFO:tasks.workunit.client.1.smithi190.stdout:6/389: mknod df/d12/c85 0 2024-05-06T12:14:50.046 INFO:tasks.workunit.client.1.smithi190.stdout:3/591: creat d5/d79/fd7 x:0 0 0 2024-05-06T12:14:50.048 INFO:tasks.workunit.client.1.smithi190.stdout:7/523: dread d0/d1/d32/d45/d70/d33/d57/fc0 [0,4194304] 0 2024-05-06T12:14:50.048 INFO:tasks.workunit.client.1.smithi190.stdout:9/637: rename db/d23/d25/d47/d4b/la2 to db/d23/d25/ld7 0 2024-05-06T12:14:50.049 INFO:tasks.workunit.client.1.smithi190.stdout:9/638: chown db/f26 95501 1 2024-05-06T12:14:50.050 INFO:tasks.workunit.client.1.smithi190.stdout:9/639: stat db/d23/d7e/fd1 0 2024-05-06T12:14:50.050 INFO:tasks.workunit.client.1.smithi190.stdout:7/524: truncate d0/d1/d2/f88 174171 0 2024-05-06T12:14:50.052 INFO:tasks.workunit.client.1.smithi190.stdout:6/390: truncate f7 3251815 0 2024-05-06T12:14:50.054 INFO:tasks.workunit.client.1.smithi190.stdout:9/640: chown db/dd/d16/l42 937152 1 2024-05-06T12:14:50.054 INFO:tasks.workunit.client.1.smithi190.stdout:6/391: write df/d17/d44/f7d [868270,102315] 0 2024-05-06T12:14:50.057 INFO:tasks.workunit.client.1.smithi190.stdout:7/525: link d0/d1/fbe d0/da/d36/d42/fc4 0 2024-05-06T12:14:50.062 INFO:tasks.workunit.client.1.smithi190.stdout:6/392: symlink df/d17/l86 0 2024-05-06T12:14:50.062 INFO:tasks.workunit.client.0.smithi092.stdout:2/961: dwrite df/f1e [0,4194304] 0 2024-05-06T12:14:50.064 INFO:tasks.workunit.client.1.smithi190.stdout:7/526: symlink d0/da/d36/lc5 0 2024-05-06T12:14:50.064 INFO:tasks.workunit.client.0.smithi092.stdout:3/902: dwrite d8/db/d4c/d3d/d49/df8/d5b/fc4 [0,4194304] 0 2024-05-06T12:14:50.065 INFO:tasks.workunit.client.1.smithi190.stdout:7/527: dread - d0/d4c/d60/fa8 zero size 2024-05-06T12:14:50.066 INFO:tasks.workunit.client.0.smithi092.stdout:2/962: mkdir df/dd0/d100/d119/d122 0 2024-05-06T12:14:50.067 INFO:tasks.workunit.client.1.smithi190.stdout:6/393: mkdir df/d2c/d61/d87 0 2024-05-06T12:14:50.068 INFO:tasks.workunit.client.1.smithi190.stdout:6/394: chown df/d2c/f45 2 1 2024-05-06T12:14:50.069 INFO:tasks.workunit.client.0.smithi092.stdout:2/963: mkdir df/d17/d1d/d112/d123 0 2024-05-06T12:14:50.069 INFO:tasks.workunit.client.1.smithi190.stdout:6/395: write df/d2c/d28/f2a [86930,123828] 0 2024-05-06T12:14:50.070 INFO:tasks.workunit.client.0.smithi092.stdout:2/964: chown df/d15/l2d 543 1 2024-05-06T12:14:50.072 INFO:tasks.workunit.client.0.smithi092.stdout:3/903: dwrite d8/db/d4c/d40/d7a/d90/fd0 [4194304,4194304] 0 2024-05-06T12:14:50.073 INFO:tasks.workunit.client.1.smithi190.stdout:3/592: dread d5/f5e [0,4194304] 0 2024-05-06T12:14:50.073 INFO:tasks.workunit.client.0.smithi092.stdout:2/965: truncate df/d17/d53/da9/dd9/d83/dc2/d101/fd6 103828 0 2024-05-06T12:14:50.075 INFO:tasks.workunit.client.0.smithi092.stdout:2/966: mkdir df/d15/d124 0 2024-05-06T12:14:50.075 INFO:tasks.workunit.client.1.smithi190.stdout:7/528: rmdir d0/d1/d32/d45/d70/d35/db8 0 2024-05-06T12:14:50.076 INFO:tasks.workunit.client.0.smithi092.stdout:3/904: creat d8/d24/d33/d4b/d74/f124 x:0 0 0 2024-05-06T12:14:50.079 INFO:tasks.workunit.client.0.smithi092.stdout:2/967: rename df/d17/f6b to df/d17/d7d/f125 0 2024-05-06T12:14:50.082 INFO:tasks.workunit.client.1.smithi190.stdout:7/529: mkdir d0/d1/d2/d84/dc6 0 2024-05-06T12:14:50.083 INFO:tasks.workunit.client.1.smithi190.stdout:3/593: dwrite d5/d7/d15/d1f/d33/f53 [0,4194304] 0 2024-05-06T12:14:50.085 INFO:tasks.workunit.client.1.smithi190.stdout:3/594: chown d5/d14/d11/d21/f30 1948854 1 2024-05-06T12:14:50.090 INFO:tasks.workunit.client.1.smithi190.stdout:6/396: link l9 df/l88 0 2024-05-06T12:14:50.091 INFO:tasks.workunit.client.1.smithi190.stdout:6/397: stat df/f82 0 2024-05-06T12:14:50.093 INFO:tasks.workunit.client.1.smithi190.stdout:7/530: dwrite d0/f69 [0,4194304] 0 2024-05-06T12:14:50.098 INFO:tasks.workunit.client.1.smithi190.stdout:3/595: dwrite d5/d7/d15/d1f/d33/f76 [0,4194304] 0 2024-05-06T12:14:50.099 INFO:tasks.workunit.client.1.smithi190.stdout:6/398: mkdir df/d2c/d1e/d57/d89 0 2024-05-06T12:14:50.101 INFO:tasks.workunit.client.1.smithi190.stdout:3/596: chown d5/d14/d11/d21/d44/d80/d82/d3d/l3e 901307 1 2024-05-06T12:14:50.107 INFO:tasks.workunit.client.0.smithi092.stdout:3/905: truncate f4 3189139 0 2024-05-06T12:14:50.108 INFO:tasks.workunit.client.1.smithi190.stdout:3/597: unlink d5/d7/d15/d39/l70 0 2024-05-06T12:14:50.108 INFO:tasks.workunit.client.1.smithi190.stdout:7/531: link d0/d1/d2/f1b d0/da/d36/d53/d5c/fc7 0 2024-05-06T12:14:50.109 INFO:tasks.workunit.client.1.smithi190.stdout:7/532: getdents d0/d1/d18/d4e/db3 0 2024-05-06T12:14:50.109 INFO:tasks.workunit.client.0.smithi092.stdout:3/906: link d8/db/d4c/d40/f113 d8/db/d4c/d3d/d49/ded/f125 0 2024-05-06T12:14:50.111 INFO:tasks.workunit.client.1.smithi190.stdout:3/598: creat d5/d7/fd8 x:0 0 0 2024-05-06T12:14:50.111 INFO:tasks.workunit.client.1.smithi190.stdout:7/533: chown d0/d1/d18/d6c/fba 43457 1 2024-05-06T12:14:50.112 INFO:tasks.workunit.client.1.smithi190.stdout:7/534: dread d0/d1/d2/f88 [0,4194304] 0 2024-05-06T12:14:50.117 INFO:tasks.workunit.client.1.smithi190.stdout:3/599: getdents d5/d7/d6d/d9a 0 2024-05-06T12:14:50.117 INFO:tasks.workunit.client.0.smithi092.stdout:2/968: dread df/d17/d53/da9/dd9/d83/f9d [0,4194304] 0 2024-05-06T12:14:50.118 INFO:tasks.workunit.client.0.smithi092.stdout:2/969: write df/dd0/de3/f48 [2226543,89884] 0 2024-05-06T12:14:50.120 INFO:tasks.workunit.client.1.smithi190.stdout:3/600: read d5/fc [254316,39105] 0 2024-05-06T12:14:50.121 INFO:tasks.workunit.client.1.smithi190.stdout:3/601: stat d5/d14/d11/d21/d44/f85 0 2024-05-06T12:14:50.122 INFO:tasks.workunit.client.0.smithi092.stdout:2/970: rename df/dd0/de3/dcf to df/d17/d51/d96/d126 0 2024-05-06T12:14:50.124 INFO:tasks.workunit.client.0.smithi092.stdout:2/971: symlink df/d15/d124/l127 0 2024-05-06T12:14:50.128 INFO:tasks.workunit.client.0.smithi092.stdout:2/972: dread df/d15/fdc [0,4194304] 0 2024-05-06T12:14:50.132 INFO:tasks.workunit.client.1.smithi190.stdout:7/535: truncate d0/d1/d32/d45/d70/d35/fa4 4155477 0 2024-05-06T12:14:50.133 INFO:tasks.workunit.client.1.smithi190.stdout:3/602: truncate d5/d14/d11/d21/d44/d92/fb7 3059915 0 2024-05-06T12:14:50.134 INFO:tasks.workunit.client.1.smithi190.stdout:7/536: dread - d0/d1/d32/d45/f7d zero size 2024-05-06T12:14:50.135 INFO:tasks.workunit.client.1.smithi190.stdout:6/399: dwrite f7 [0,4194304] 0 2024-05-06T12:14:50.136 INFO:tasks.workunit.client.0.smithi092.stdout:2/973: dwrite df/d17/d51/d96/d126/f6f [0,4194304] 0 2024-05-06T12:14:50.136 INFO:tasks.workunit.client.1.smithi190.stdout:3/603: read d5/d14/f56 [54060,29522] 0 2024-05-06T12:14:50.137 INFO:tasks.workunit.client.1.smithi190.stdout:3/604: dread - d5/d79/fd7 zero size 2024-05-06T12:14:50.137 INFO:tasks.workunit.client.1.smithi190.stdout:7/537: symlink d0/db4/lc8 0 2024-05-06T12:14:50.138 INFO:tasks.workunit.client.1.smithi190.stdout:6/400: stat df/d17/d44/f62 0 2024-05-06T12:14:50.138 INFO:tasks.workunit.client.0.smithi092.stdout:2/974: mkdir df/d15/d124/d128 0 2024-05-06T12:14:50.138 INFO:tasks.workunit.client.1.smithi190.stdout:6/401: dread - df/f82 zero size 2024-05-06T12:14:50.139 INFO:tasks.workunit.client.1.smithi190.stdout:7/538: stat d0/d1/d18/d4e/d87/f8b 0 2024-05-06T12:14:50.141 INFO:tasks.workunit.client.1.smithi190.stdout:3/605: rename d5/d14/dc5 to d5/d94/dd9 0 2024-05-06T12:14:50.142 INFO:tasks.workunit.client.1.smithi190.stdout:3/606: fsync d5/d7/d15/d1f/d33/d5c/f63 0 2024-05-06T12:14:50.143 INFO:tasks.workunit.client.0.smithi092.stdout:2/975: mkdir df/d17/d53/d85/da0/de0/d129 0 2024-05-06T12:14:50.144 INFO:tasks.workunit.client.1.smithi190.stdout:3/607: chown d5/d14/d11/d21/l7a 24851860 1 2024-05-06T12:14:50.145 INFO:tasks.workunit.client.1.smithi190.stdout:6/402: link df/d2c/d1e/f2d df/d2c/d1e/d57/f8a 0 2024-05-06T12:14:50.148 INFO:tasks.workunit.client.1.smithi190.stdout:3/608: creat d5/d7/d6d/d9a/fda x:0 0 0 2024-05-06T12:14:50.148 INFO:tasks.workunit.client.1.smithi190.stdout:6/403: write df/d17/f26 [5051084,60945] 0 2024-05-06T12:14:50.150 INFO:tasks.workunit.client.1.smithi190.stdout:6/404: dread - df/d2c/d61/f79 zero size 2024-05-06T12:14:50.152 INFO:tasks.workunit.client.1.smithi190.stdout:3/609: rename d5/d14/d11/d21/d44/d80/dca to d5/d14/d11/d21/d44/d80/ddb 0 2024-05-06T12:14:50.152 INFO:tasks.workunit.client.1.smithi190.stdout:6/405: creat df/d12/f8b x:0 0 0 2024-05-06T12:14:50.154 INFO:tasks.workunit.client.1.smithi190.stdout:3/610: symlink d5/d7/d15/d1f/d84/db3/da1/ldc 0 2024-05-06T12:14:50.155 INFO:tasks.workunit.client.1.smithi190.stdout:6/406: creat df/d17/d37/f8c x:0 0 0 2024-05-06T12:14:50.156 INFO:tasks.workunit.client.1.smithi190.stdout:6/407: chown df/d17/f5c 100 1 2024-05-06T12:14:50.170 INFO:tasks.workunit.client.1.smithi190.stdout:7/539: dwrite d0/d1/f3e [0,4194304] 0 2024-05-06T12:14:50.175 INFO:tasks.workunit.client.1.smithi190.stdout:3/611: dread d5/d14/d11/d21/d44/d80/d82/d3d/f49 [0,4194304] 0 2024-05-06T12:14:50.176 INFO:tasks.workunit.client.1.smithi190.stdout:7/540: unlink d0/d1/d32/d45/f58 0 2024-05-06T12:14:50.178 INFO:tasks.workunit.client.1.smithi190.stdout:6/408: dwrite df/d12/f2b [0,4194304] 0 2024-05-06T12:14:50.178 INFO:tasks.workunit.client.1.smithi190.stdout:3/612: chown d5/d7/d15/d1f/d84/db3/da1/ldc 8083685 1 2024-05-06T12:14:50.179 INFO:tasks.workunit.client.0.smithi092.stdout:2/976: creat df/d17/d53/d85/da0/de0/f12a x:0 0 0 2024-05-06T12:14:50.179 INFO:tasks.workunit.client.1.smithi190.stdout:6/409: chown df/d2c/l43 67616 1 2024-05-06T12:14:50.181 INFO:tasks.workunit.client.0.smithi092.stdout:2/977: mkdir df/d17/d51/d96/d12b 0 2024-05-06T12:14:50.182 INFO:tasks.workunit.client.0.smithi092.stdout:2/978: write df/d17/d51/f57 [3337142,94805] 0 2024-05-06T12:14:50.183 INFO:tasks.workunit.client.1.smithi190.stdout:1/422: sync 2024-05-06T12:14:50.183 INFO:tasks.workunit.client.1.smithi190.stdout:2/481: sync 2024-05-06T12:14:50.183 INFO:tasks.workunit.client.0.smithi092.stdout:2/979: read - df/dfb/d107/f89 zero size 2024-05-06T12:14:50.184 INFO:tasks.workunit.client.1.smithi190.stdout:4/392: sync 2024-05-06T12:14:50.184 INFO:tasks.workunit.client.1.smithi190.stdout:0/571: sync 2024-05-06T12:14:50.184 INFO:tasks.workunit.client.1.smithi190.stdout:8/612: sync 2024-05-06T12:14:50.184 INFO:tasks.workunit.client.1.smithi190.stdout:5/584: sync 2024-05-06T12:14:50.184 INFO:tasks.workunit.client.0.smithi092.stdout:2/980: write df/d17/d7d/f10c [1172772,116103] 0 2024-05-06T12:14:50.187 INFO:tasks.workunit.client.0.smithi092.stdout:2/981: rmdir df/d17/d1d/dc8 39 2024-05-06T12:14:50.188 INFO:tasks.workunit.client.1.smithi190.stdout:8/613: fsync d3/d35/d3a/dc5/fce 0 2024-05-06T12:14:50.188 INFO:tasks.workunit.client.1.smithi190.stdout:1/423: creat d5/d4f/d65/d2b/f95 x:0 0 0 2024-05-06T12:14:50.191 INFO:tasks.workunit.client.1.smithi190.stdout:2/482: symlink d2/d21/la1 0 2024-05-06T12:14:50.191 INFO:tasks.workunit.client.1.smithi190.stdout:0/572: rmdir d9/df/d21 39 2024-05-06T12:14:50.191 INFO:tasks.workunit.client.1.smithi190.stdout:3/613: dwrite d5/d14/d11/d21/d44/d80/d82/fbb [0,4194304] 0 2024-05-06T12:14:50.192 INFO:tasks.workunit.client.1.smithi190.stdout:1/424: read - d5/d4f/d65/d32/d47/d53/f91 zero size 2024-05-06T12:14:50.192 INFO:tasks.workunit.client.0.smithi092.stdout:2/982: rename df/d17/d1d/f46 to df/d17/d51/d96/f12c 0 2024-05-06T12:14:50.195 INFO:tasks.workunit.client.0.smithi092.stdout:2/983: getdents df 0 2024-05-06T12:14:50.196 INFO:tasks.workunit.client.0.smithi092.stdout:2/984: write df/d17/f2a [741521,94174] 0 2024-05-06T12:14:50.198 INFO:tasks.workunit.client.1.smithi190.stdout:1/425: creat d5/d4f/d65/d32/d47/f96 x:0 0 0 2024-05-06T12:14:50.198 INFO:tasks.workunit.client.1.smithi190.stdout:0/573: truncate d9/dc/f6c 599793 0 2024-05-06T12:14:50.200 INFO:tasks.workunit.client.1.smithi190.stdout:2/483: creat d2/d8/d19/d4f/fa2 x:0 0 0 2024-05-06T12:14:50.201 INFO:tasks.workunit.client.1.smithi190.stdout:3/614: creat d5/d14/d11/d21/d44/d80/d82/d3d/fdd x:0 0 0 2024-05-06T12:14:50.205 INFO:tasks.workunit.client.1.smithi190.stdout:1/426: chown d5/d4f/d65/d32/f37 101913 1 2024-05-06T12:14:50.209 INFO:tasks.workunit.client.1.smithi190.stdout:8/614: dwrite d3/d35/d55/f96 [0,4194304] 0 2024-05-06T12:14:50.209 INFO:tasks.workunit.client.1.smithi190.stdout:4/393: truncate d2/f64 497439 0 2024-05-06T12:14:50.210 INFO:tasks.workunit.client.1.smithi190.stdout:9/641: sync 2024-05-06T12:14:50.212 INFO:tasks.workunit.client.1.smithi190.stdout:2/484: getdents d2/d8/d33 0 2024-05-06T12:14:50.213 INFO:tasks.workunit.client.1.smithi190.stdout:7/541: dwrite d0/da/d36/d42/fc4 [0,4194304] 0 2024-05-06T12:14:50.214 INFO:tasks.workunit.client.1.smithi190.stdout:2/485: chown d2/d8/d33/c7f 2 1 2024-05-06T12:14:50.214 INFO:tasks.workunit.client.1.smithi190.stdout:7/542: chown d0/f22 31052724 1 2024-05-06T12:14:50.215 INFO:tasks.workunit.client.1.smithi190.stdout:1/427: symlink d5/d6b/l97 0 2024-05-06T12:14:50.217 INFO:tasks.workunit.client.1.smithi190.stdout:4/394: read d2/d11/d16/d61/d67/d6e/f72 [2833270,47232] 0 2024-05-06T12:14:50.218 INFO:tasks.workunit.client.1.smithi190.stdout:3/615: symlink d5/d7/d6d/dc6/lde 0 2024-05-06T12:14:50.219 INFO:tasks.workunit.client.1.smithi190.stdout:8/615: chown d3/d8/d2b/d3e/dad/fd3 867 1 2024-05-06T12:14:50.221 INFO:tasks.workunit.client.1.smithi190.stdout:4/395: dread - d2/d11/d16/d31/f5c zero size 2024-05-06T12:14:50.224 INFO:tasks.workunit.client.1.smithi190.stdout:5/585: dread d2/d6c/f99 [0,4194304] 0 2024-05-06T12:14:50.224 INFO:tasks.workunit.client.1.smithi190.stdout:7/543: read d0/da/d36/f43 [233686,63753] 0 2024-05-06T12:14:50.227 INFO:tasks.workunit.client.0.smithi092.stdout:2/985: write df/d17/d53/da9/dd9/d83/fb9 [845963,87810] 0 2024-05-06T12:14:50.227 INFO:tasks.workunit.client.1.smithi190.stdout:8/616: write d3/f98 [3901333,76501] 0 2024-05-06T12:14:50.228 INFO:tasks.workunit.client.0.smithi092.stdout:2/986: write df/d17/d53/d85/da0/de0/f12a [580469,11272] 0 2024-05-06T12:14:50.229 INFO:tasks.workunit.client.1.smithi190.stdout:1/428: fdatasync d5/d4f/d65/d24/d3a/f72 0 2024-05-06T12:14:50.229 INFO:tasks.workunit.client.0.smithi092.stdout:2/987: chown df/c10f 281086 1 2024-05-06T12:14:50.229 INFO:tasks.workunit.client.1.smithi190.stdout:6/410: dread df/d2c/d1e/f2d [0,4194304] 0 2024-05-06T12:14:50.230 INFO:tasks.workunit.client.1.smithi190.stdout:7/544: creat d0/d1/d2/fc9 x:0 0 0 2024-05-06T12:14:50.230 INFO:tasks.workunit.client.0.smithi092.stdout:2/988: fsync df/dfb/d107/d59/f9e 0 2024-05-06T12:14:50.232 INFO:tasks.workunit.client.0.smithi092.stdout:2/989: chown df/dfb/d107/d59/fb7 1 1 2024-05-06T12:14:50.232 INFO:tasks.workunit.client.0.smithi092.stdout:2/990: dread - df/dfb/f11a zero size 2024-05-06T12:14:50.232 INFO:tasks.workunit.client.1.smithi190.stdout:5/586: truncate d2/f30 9159360 0 2024-05-06T12:14:50.233 INFO:tasks.workunit.client.0.smithi092.stdout:2/991: fdatasync df/d17/d51/d96/f97 0 2024-05-06T12:14:50.234 INFO:tasks.workunit.client.1.smithi190.stdout:0/574: dread d9/df/f2e [0,4194304] 0 2024-05-06T12:14:50.235 INFO:tasks.workunit.client.1.smithi190.stdout:9/642: dread db/dd/d16/d4c/f9b [0,4194304] 0 2024-05-06T12:14:50.235 INFO:tasks.workunit.client.1.smithi190.stdout:0/575: readlink d9/dc/d28/d4c/d80/lad 0 2024-05-06T12:14:50.238 INFO:tasks.workunit.client.1.smithi190.stdout:6/411: symlink df/d2c/d61/l8d 0 2024-05-06T12:14:50.243 INFO:tasks.workunit.client.1.smithi190.stdout:1/429: mkdir d5/d4f/d65/d32/d47/d53/d98 0 2024-05-06T12:14:50.247 INFO:tasks.workunit.client.1.smithi190.stdout:0/576: rename d9/dc/f1f to d9/dc/d48/fce 0 2024-05-06T12:14:50.247 INFO:tasks.workunit.client.1.smithi190.stdout:0/577: readlink d9/dc/d28/d77/la8 0 2024-05-06T12:14:50.251 INFO:tasks.workunit.client.1.smithi190.stdout:7/545: dwrite d0/d1/d18/d2b/f83 [4194304,4194304] 0 2024-05-06T12:14:50.252 INFO:tasks.workunit.client.1.smithi190.stdout:5/587: mknod d2/da/d59/d5f/cbc 0 2024-05-06T12:14:50.255 INFO:tasks.workunit.client.0.smithi092.stdout:2/992: dread df/d17/d1d/f21 [0,4194304] 0 2024-05-06T12:14:50.255 INFO:tasks.workunit.client.1.smithi190.stdout:7/546: dread d0/f69 [0,4194304] 0 2024-05-06T12:14:50.257 INFO:tasks.workunit.client.0.smithi092.stdout:2/993: mknod df/d17/d7e/c12d 0 2024-05-06T12:14:50.257 INFO:tasks.workunit.client.1.smithi190.stdout:3/616: link d5/d7/d15/cbe d5/d14/d11/d21/d44/d80/cdf 0 2024-05-06T12:14:50.257 INFO:tasks.workunit.client.1.smithi190.stdout:3/617: stat d5/d79/cc2 0 2024-05-06T12:14:50.259 INFO:tasks.workunit.client.0.smithi092.stdout:2/994: creat df/d17/d53/da9/dd9/dd5/f12e x:0 0 0 2024-05-06T12:14:50.260 INFO:tasks.workunit.client.1.smithi190.stdout:3/618: write d5/d14/d11/d21/d44/d80/d82/fd3 [829265,10394] 0 2024-05-06T12:14:50.262 INFO:tasks.workunit.client.1.smithi190.stdout:3/619: write d5/d7/d15/d1f/d33/f53 [2499084,76788] 0 2024-05-06T12:14:50.263 INFO:tasks.workunit.client.1.smithi190.stdout:4/396: link d2/d11/d16/d53/d7b/l83 d2/d11/d16/d61/l89 0 2024-05-06T12:14:50.266 INFO:tasks.workunit.client.1.smithi190.stdout:5/588: mknod d2/da/d59/d5f/d2e/d8b/cbd 0 2024-05-06T12:14:50.266 INFO:tasks.workunit.client.0.smithi092.stdout:2/995: link df/d17/d1d/df8/f120 df/dd0/de3/d8e/f12f 0 2024-05-06T12:14:50.267 INFO:tasks.workunit.client.1.smithi190.stdout:8/617: truncate d3/f98 2191598 0 2024-05-06T12:14:50.267 INFO:tasks.workunit.client.1.smithi190.stdout:5/589: write d2/da/d59/d5f/d50/f51 [3553916,110080] 0 2024-05-06T12:14:50.268 INFO:tasks.workunit.client.1.smithi190.stdout:8/618: write d3/d13/fc8 [765389,91921] 0 2024-05-06T12:14:50.269 INFO:tasks.workunit.client.0.smithi092.stdout:2/996: getdents df/d17/d1d/df8 0 2024-05-06T12:14:50.270 INFO:tasks.workunit.client.0.smithi092.stdout:2/997: chown df/d17/d1d/cff 5850 1 2024-05-06T12:14:50.271 INFO:tasks.workunit.client.1.smithi190.stdout:6/412: rename fc to df/d2c/d61/d84/f8e 0 2024-05-06T12:14:50.271 INFO:tasks.workunit.client.0.smithi092.stdout:2/998: fsync df/d17/d53/da9/fea 0 2024-05-06T12:14:50.271 INFO:tasks.workunit.client.1.smithi190.stdout:6/413: chown df/d2c/d1e/d57/f8a 18749413 1 2024-05-06T12:14:50.272 INFO:tasks.workunit.client.1.smithi190.stdout:0/578: rmdir d9/df/d21/d31 39 2024-05-06T12:14:50.272 INFO:tasks.workunit.client.0.smithi092.stdout:2/999: rename df/dd0/de3/db4/dce to df/dd0/de3/db4/dce/d130 22 2024-05-06T12:14:50.275 INFO:tasks.workunit.client.1.smithi190.stdout:4/397: fsync d2/f32 0 2024-05-06T12:14:50.278 INFO:tasks.workunit.client.0.smithi092.stdout:3/907: sync 2024-05-06T12:14:50.279 INFO:tasks.workunit.client.1.smithi190.stdout:9/643: link db/dd/d16/d58/d87/d99/lcd db/dd/d16/d58/ld8 0 2024-05-06T12:14:50.281 INFO:tasks.workunit.client.1.smithi190.stdout:5/590: dread d2/f5 [0,4194304] 0 2024-05-06T12:14:50.281 INFO:tasks.workunit.client.1.smithi190.stdout:5/591: chown d2/c1d 8039528 1 2024-05-06T12:14:50.283 INFO:tasks.workunit.client.1.smithi190.stdout:5/592: write d2/da/d59/d5f/d2e/d3b/f68 [5000124,38418] 0 2024-05-06T12:14:50.283 INFO:tasks.workunit.client.0.smithi092.stdout:3/908: getdents d8/d1b/d73 0 2024-05-06T12:14:50.284 INFO:tasks.workunit.client.0.smithi092.stdout:3/909: dread - d8/d24/d33/d4b/fe5 zero size 2024-05-06T12:14:50.284 INFO:tasks.workunit.client.1.smithi190.stdout:8/619: mknod d3/d8/cd9 0 2024-05-06T12:14:50.285 INFO:tasks.workunit.client.1.smithi190.stdout:8/620: dread - d3/d8/d25/d3b/d6b/f99 zero size 2024-05-06T12:14:50.285 INFO:tasks.workunit.client.1.smithi190.stdout:9/644: fdatasync db/dd/d16/d58/d87/d22/f55 0 2024-05-06T12:14:50.287 INFO:tasks.workunit.client.1.smithi190.stdout:7/547: getdents d0/d1/d32/d45 0 2024-05-06T12:14:50.288 INFO:tasks.workunit.client.1.smithi190.stdout:7/548: read - d0/da/d36/d42/fa7 zero size 2024-05-06T12:14:50.289 INFO:tasks.workunit.client.1.smithi190.stdout:7/549: readlink d0/d1/d32/d45/d70/d35/l51 0 2024-05-06T12:14:50.289 INFO:tasks.workunit.client.1.smithi190.stdout:1/430: write d5/d4f/d65/d32/d47/f4c [1204326,85498] 0 2024-05-06T12:14:50.291 INFO:tasks.workunit.client.0.smithi092.stdout:3/910: dwrite d8/d24/d33/d4b/d74/f9e [0,4194304] 0 2024-05-06T12:14:50.292 INFO:tasks.workunit.client.0.smithi092.stdout:3/911: chown d8/db/d4c/d40/d7a/d90/f93 1 1 2024-05-06T12:14:50.294 INFO:tasks.workunit.client.1.smithi190.stdout:5/593: mkdir d2/da/d59/d5f/d2e/d8b/dbe 0 2024-05-06T12:14:50.297 INFO:tasks.workunit.client.1.smithi190.stdout:9/645: rmdir db/daa 39 2024-05-06T12:14:50.298 INFO:tasks.workunit.client.1.smithi190.stdout:1/431: dwrite d5/d4f/d65/d32/f37 [0,4194304] 0 2024-05-06T12:14:50.299 INFO:tasks.workunit.client.1.smithi190.stdout:4/398: link d2/c3f d2/d11/d16/d53/c8a 0 2024-05-06T12:14:50.302 INFO:tasks.workunit.client.1.smithi190.stdout:6/414: getdents df/d17 0 2024-05-06T12:14:50.304 INFO:tasks.workunit.client.1.smithi190.stdout:5/594: fdatasync d2/f44 0 2024-05-06T12:14:50.306 INFO:tasks.workunit.client.1.smithi190.stdout:8/621: mknod d3/d8/d25/d3b/cda 0 2024-05-06T12:14:50.311 INFO:tasks.workunit.client.1.smithi190.stdout:3/620: dwrite d5/d7/f90 [0,4194304] 0 2024-05-06T12:14:50.311 INFO:tasks.workunit.client.1.smithi190.stdout:5/595: dread d2/d6c/d7d/fa2 [0,4194304] 0 2024-05-06T12:14:50.311 INFO:tasks.workunit.client.1.smithi190.stdout:6/415: mkdir df/d2c/d61/d84/d8f 0 2024-05-06T12:14:50.312 INFO:tasks.workunit.client.1.smithi190.stdout:3/621: fdatasync d5/f27 0 2024-05-06T12:14:50.315 INFO:tasks.workunit.client.1.smithi190.stdout:9/646: fdatasync db/d23/d25/d47/fbf 0 2024-05-06T12:14:50.317 INFO:tasks.workunit.client.1.smithi190.stdout:0/579: dwrite d9/df/f75 [0,4194304] 0 2024-05-06T12:14:50.317 INFO:tasks.workunit.client.1.smithi190.stdout:0/580: fdatasync d9/f2c 0 2024-05-06T12:14:50.319 INFO:tasks.workunit.client.1.smithi190.stdout:5/596: mkdir d2/da/d59/dbf 0 2024-05-06T12:14:50.322 INFO:tasks.workunit.client.1.smithi190.stdout:6/416: mknod df/d2c/d61/d84/c90 0 2024-05-06T12:14:50.329 INFO:tasks.workunit.client.1.smithi190.stdout:5/597: rename d2/da/c92 to d2/d65/cc0 0 2024-05-06T12:14:50.332 INFO:tasks.workunit.client.1.smithi190.stdout:8/622: write d3/d8/f27 [120379,20889] 0 2024-05-06T12:14:50.333 INFO:tasks.workunit.client.1.smithi190.stdout:5/598: write d2/f44 [3011287,34580] 0 2024-05-06T12:14:50.335 INFO:tasks.workunit.client.1.smithi190.stdout:9/647: creat db/dd/d16/d58/d87/d22/d49/fd9 x:0 0 0 2024-05-06T12:14:50.335 INFO:tasks.workunit.client.1.smithi190.stdout:1/432: dwrite d5/d15/f3f [0,4194304] 0 2024-05-06T12:14:50.339 INFO:tasks.workunit.client.1.smithi190.stdout:4/399: dwrite d2/d11/f4c [0,4194304] 0 2024-05-06T12:14:50.339 INFO:tasks.workunit.client.1.smithi190.stdout:6/417: rename df/d2c/d1e/d57/l66 to df/d17/l91 0 2024-05-06T12:14:50.340 INFO:tasks.workunit.client.1.smithi190.stdout:8/623: chown d3/d8/d2b/l33 12175096 1 2024-05-06T12:14:50.341 INFO:tasks.workunit.client.1.smithi190.stdout:8/624: dread - d3/d35/d3a/db9/fd0 zero size 2024-05-06T12:14:50.343 INFO:tasks.workunit.client.1.smithi190.stdout:0/581: rename d9/df to d9/dcf 0 2024-05-06T12:14:50.348 INFO:tasks.workunit.client.1.smithi190.stdout:1/433: unlink d5/l14 0 2024-05-06T12:14:50.352 INFO:tasks.workunit.client.1.smithi190.stdout:4/400: rename d2/l49 to d2/d11/d16/d41/d81/l8b 0 2024-05-06T12:14:50.353 INFO:tasks.workunit.client.1.smithi190.stdout:8/625: symlink d3/d8/d2b/d3e/ldb 0 2024-05-06T12:14:50.355 INFO:tasks.workunit.client.1.smithi190.stdout:0/582: mkdir d9/dc/dd0 0 2024-05-06T12:14:50.355 INFO:tasks.workunit.client.1.smithi190.stdout:0/583: readlink d9/d91/lbe 0 2024-05-06T12:14:50.357 INFO:tasks.workunit.client.1.smithi190.stdout:1/434: symlink d5/d4f/d65/d24/d3a/l99 0 2024-05-06T12:14:50.359 INFO:tasks.workunit.client.1.smithi190.stdout:9/648: mknod db/d23/d29/dc4/cda 0 2024-05-06T12:14:50.363 INFO:tasks.workunit.client.1.smithi190.stdout:6/418: readlink df/d17/l6e 0 2024-05-06T12:14:50.363 INFO:tasks.workunit.client.1.smithi190.stdout:8/626: chown d3/d8/d25/d65/dc6/ccd 1667900 1 2024-05-06T12:14:50.365 INFO:tasks.workunit.client.1.smithi190.stdout:3/622: link d5/d14/d11/cc8 d5/d14/d11/d21/d44/d92/d9e/ce0 0 2024-05-06T12:14:50.365 INFO:tasks.workunit.client.1.smithi190.stdout:9/649: dwrite db/d23/d7e/fa8 [0,4194304] 0 2024-05-06T12:14:50.365 INFO:tasks.workunit.client.1.smithi190.stdout:6/419: write df/d2c/d1e/d57/f69 [1941472,32926] 0 2024-05-06T12:14:50.367 INFO:tasks.workunit.client.1.smithi190.stdout:0/584: fdatasync d9/d51/dc9/f69 0 2024-05-06T12:14:50.367 INFO:tasks.workunit.client.1.smithi190.stdout:3/623: write d5/d79/fd7 [204110,28402] 0 2024-05-06T12:14:50.371 INFO:tasks.workunit.client.1.smithi190.stdout:0/585: write d9/dcf/d21/f2a [243278,23406] 0 2024-05-06T12:14:50.374 INFO:tasks.workunit.client.1.smithi190.stdout:8/627: dwrite d3/d8/d2b/f9e [0,4194304] 0 2024-05-06T12:14:50.376 INFO:tasks.workunit.client.1.smithi190.stdout:4/401: write d2/f5e [766312,84776] 0 2024-05-06T12:14:50.381 INFO:tasks.workunit.client.1.smithi190.stdout:0/586: dwrite d9/dc/d48/f4e [4194304,4194304] 0 2024-05-06T12:14:50.384 INFO:tasks.workunit.client.1.smithi190.stdout:1/435: symlink d5/d6b/d8f/l9a 0 2024-05-06T12:14:50.395 INFO:tasks.workunit.client.1.smithi190.stdout:4/402: symlink d2/d10/l8c 0 2024-05-06T12:14:50.396 INFO:tasks.workunit.client.1.smithi190.stdout:3/624: dwrite d5/d14/f24 [0,4194304] 0 2024-05-06T12:14:50.397 INFO:tasks.workunit.client.1.smithi190.stdout:8/628: rename d3/d8/l37 to d3/d8/d2b/ldc 0 2024-05-06T12:14:50.398 INFO:tasks.workunit.client.1.smithi190.stdout:6/420: mkdir df/d2c/d92 0 2024-05-06T12:14:50.400 INFO:tasks.workunit.client.1.smithi190.stdout:6/421: write f7 [4031247,122024] 0 2024-05-06T12:14:50.400 INFO:tasks.workunit.client.1.smithi190.stdout:5/599: link d2/da/c6f d2/da/d59/d5f/d2e/cc1 0 2024-05-06T12:14:50.403 INFO:tasks.workunit.client.1.smithi190.stdout:8/629: rmdir d3/d8/dcf 39 2024-05-06T12:14:50.405 INFO:tasks.workunit.client.1.smithi190.stdout:4/403: chown d2/d11/d16/d41/d81/l8b 131783 1 2024-05-06T12:14:50.406 INFO:tasks.workunit.client.1.smithi190.stdout:3/625: write d5/f5e [5922573,21435] 0 2024-05-06T12:14:50.407 INFO:tasks.workunit.client.1.smithi190.stdout:0/587: link d9/dcf/d21/c25 d9/dc/cd1 0 2024-05-06T12:14:50.408 INFO:tasks.workunit.client.1.smithi190.stdout:9/650: dread db/dd/df/f2c [0,4194304] 0 2024-05-06T12:14:50.411 INFO:tasks.workunit.client.1.smithi190.stdout:6/422: read df/d17/f26 [2144902,74877] 0 2024-05-06T12:14:50.413 INFO:tasks.workunit.client.1.smithi190.stdout:1/436: dwrite d5/d4f/d65/d32/d47/d53/f91 [0,4194304] 0 2024-05-06T12:14:50.416 INFO:tasks.workunit.client.1.smithi190.stdout:1/437: chown d5/d4f/d65/d2b/f95 1 1 2024-05-06T12:14:50.419 INFO:tasks.workunit.client.1.smithi190.stdout:5/600: link d2/da/c86 d2/da/d59/d5f/d2e/d8b/cc2 0 2024-05-06T12:14:50.421 INFO:tasks.workunit.client.1.smithi190.stdout:3/626: dwrite d5/d7/f3a [0,4194304] 0 2024-05-06T12:14:50.422 INFO:tasks.workunit.client.1.smithi190.stdout:6/423: creat df/d2c/d92/f93 x:0 0 0 2024-05-06T12:14:50.424 INFO:tasks.workunit.client.1.smithi190.stdout:3/627: dread - d5/d7/d6d/fc4 zero size 2024-05-06T12:14:50.428 INFO:tasks.workunit.client.1.smithi190.stdout:3/628: stat d5/d14/d11/d21/d44/d80/d82/l72 0 2024-05-06T12:14:50.430 INFO:tasks.workunit.client.1.smithi190.stdout:1/438: read d5/d4f/d65/d24/d3a/f73 [5152336,32647] 0 2024-05-06T12:14:50.430 INFO:tasks.workunit.client.1.smithi190.stdout:5/601: truncate d2/da/d59/d5f/d2e/d3b/d5a/f91 849321 0 2024-05-06T12:14:50.432 INFO:tasks.workunit.client.1.smithi190.stdout:8/630: write d3/f18 [1164689,74022] 0 2024-05-06T12:14:50.433 INFO:tasks.workunit.client.1.smithi190.stdout:9/651: dwrite db/dd/d16/d58/d87/d22/d49/f89 [0,4194304] 0 2024-05-06T12:14:50.435 INFO:tasks.workunit.client.1.smithi190.stdout:3/629: unlink d5/d7/d15/d1f/d84/ca8 0 2024-05-06T12:14:50.436 INFO:tasks.workunit.client.1.smithi190.stdout:8/631: truncate d3/d13/fc3 410859 0 2024-05-06T12:14:50.438 INFO:tasks.workunit.client.1.smithi190.stdout:2/486: sync 2024-05-06T12:14:50.439 INFO:tasks.workunit.client.1.smithi190.stdout:0/588: dread d9/d51/f64 [0,4194304] 0 2024-05-06T12:14:50.439 INFO:tasks.workunit.client.1.smithi190.stdout:5/602: mknod d2/da/d59/d5f/cc3 0 2024-05-06T12:14:50.442 INFO:tasks.workunit.client.1.smithi190.stdout:1/439: creat d5/d4f/d65/d32/d47/d53/d8b/f9b x:0 0 0 2024-05-06T12:14:50.443 INFO:tasks.workunit.client.1.smithi190.stdout:4/404: dread d2/d11/f14 [0,4194304] 0 2024-05-06T12:14:50.444 INFO:tasks.workunit.client.1.smithi190.stdout:3/630: read d5/d14/d11/d21/d44/d92/fb7 [1326038,13224] 0 2024-05-06T12:14:50.453 INFO:tasks.workunit.client.1.smithi190.stdout:1/440: rename d5/fe to d5/d4f/d65/d24/d3a/f9c 0 2024-05-06T12:14:50.454 INFO:tasks.workunit.client.1.smithi190.stdout:2/487: dwrite d2/d21/f5e [0,4194304] 0 2024-05-06T12:14:50.454 INFO:tasks.workunit.client.1.smithi190.stdout:5/603: mknod d2/db6/cc4 0 2024-05-06T12:14:50.456 INFO:tasks.workunit.client.1.smithi190.stdout:0/589: fdatasync d9/d51/d93/fbd 0 2024-05-06T12:14:50.456 INFO:tasks.workunit.client.1.smithi190.stdout:2/488: stat d2/c76 0 2024-05-06T12:14:50.458 INFO:tasks.workunit.client.1.smithi190.stdout:7/550: sync 2024-05-06T12:14:50.460 INFO:tasks.workunit.client.1.smithi190.stdout:6/424: dread df/d17/d44/f48 [0,4194304] 0 2024-05-06T12:14:50.461 INFO:tasks.workunit.client.1.smithi190.stdout:9/652: dwrite db/d23/d29/f6b [4194304,4194304] 0 2024-05-06T12:14:50.480 INFO:tasks.workunit.client.0.smithi092.stdout:3/912: sync 2024-05-06T12:14:50.483 INFO:tasks.workunit.client.0.smithi092.stdout:3/913: getdents d8/db/d4c/d3d/d49/dd2 0 2024-05-06T12:14:50.484 INFO:tasks.workunit.client.0.smithi092.stdout:3/914: truncate d8/db/d4c/d40/f106 745145 0 2024-05-06T12:14:50.485 INFO:tasks.workunit.client.0.smithi092.stdout:3/915: rmdir d8/dd7 39 2024-05-06T12:14:50.485 INFO:tasks.workunit.client.0.smithi092.stdout:3/916: stat d8/db/d4c/d40/d67 0 2024-05-06T12:14:50.491 INFO:tasks.workunit.client.0.smithi092.stdout:3/917: dwrite d8/d24/d33/f62 [8388608,4194304] 0 2024-05-06T12:14:50.493 INFO:tasks.workunit.client.0.smithi092.stdout:3/918: creat d8/d1b/d73/f126 x:0 0 0 2024-05-06T12:14:50.494 INFO:tasks.workunit.client.0.smithi092.stdout:3/919: write d8/d24/d27/f2e [589767,125438] 0 2024-05-06T12:14:50.499 INFO:tasks.workunit.client.0.smithi092.stdout:3/920: unlink d8/dd7/d107/c114 0 2024-05-06T12:14:50.500 INFO:tasks.workunit.client.0.smithi092.stdout:3/921: symlink d8/db/d14/d5a/l127 0 2024-05-06T12:14:50.501 INFO:tasks.workunit.client.0.smithi092.stdout:3/922: write d8/d24/f77 [335606,8061] 0 2024-05-06T12:14:50.521 INFO:tasks.workunit.client.0.smithi092.stdout:3/923: dread d8/db/d4c/d40/d7a/d90/fd0 [0,4194304] 0 2024-05-06T12:14:50.521 INFO:tasks.workunit.client.0.smithi092.stdout:3/924: chown d8/db/d4c/d40/d67/f80 15080 1 2024-05-06T12:14:50.522 INFO:tasks.workunit.client.0.smithi092.stdout:3/925: read d8/db/d4c/d3d/d49/df8/f70 [3812552,31294] 0 2024-05-06T12:14:50.525 INFO:tasks.workunit.client.0.smithi092.stdout:3/926: mknod d8/db/d4c/d3d/d49/ded/d121/c128 0 2024-05-06T12:14:50.525 INFO:tasks.workunit.client.1.smithi190.stdout:3/631: dread d5/d7/f90 [0,4194304] 0 2024-05-06T12:14:50.526 INFO:tasks.workunit.client.1.smithi190.stdout:0/590: write d9/dcf/d56/f57 [437056,33178] 0 2024-05-06T12:14:50.526 INFO:tasks.workunit.client.1.smithi190.stdout:9/653: chown db/dd/df/c14 7973692 1 2024-05-06T12:14:50.528 INFO:tasks.workunit.client.0.smithi092.stdout:3/927: dread f4 [0,4194304] 0 2024-05-06T12:14:50.528 INFO:tasks.workunit.client.1.smithi190.stdout:2/489: symlink d2/d8/d33/d46/la3 0 2024-05-06T12:14:50.529 INFO:tasks.workunit.client.1.smithi190.stdout:5/604: unlink d2/da/d59/d5f/d50/l67 0 2024-05-06T12:14:50.531 INFO:tasks.workunit.client.1.smithi190.stdout:2/490: chown d2/d8/dc/c2f 862549 1 2024-05-06T12:14:50.532 INFO:tasks.workunit.client.1.smithi190.stdout:8/632: dread d3/f6 [4194304,4194304] 0 2024-05-06T12:14:50.533 INFO:tasks.workunit.client.1.smithi190.stdout:3/632: rename d5/f27 to d5/d14/fe1 0 2024-05-06T12:14:50.533 INFO:tasks.workunit.client.1.smithi190.stdout:5/605: chown d2/da/d59/dbf 2615 1 2024-05-06T12:14:50.535 INFO:tasks.workunit.client.1.smithi190.stdout:6/425: creat df/d2c/d61/d84/d8f/f94 x:0 0 0 2024-05-06T12:14:50.535 INFO:tasks.workunit.client.1.smithi190.stdout:9/654: mknod db/daa/cdb 0 2024-05-06T12:14:50.536 INFO:tasks.workunit.client.1.smithi190.stdout:4/405: dwrite d2/d11/d16/d61/d66/f74 [0,4194304] 0 2024-05-06T12:14:50.538 INFO:tasks.workunit.client.0.smithi092.stdout:3/928: dwrite d8/db/d4c/d3d/d49/ded/f125 [0,4194304] 0 2024-05-06T12:14:50.540 INFO:tasks.workunit.client.1.smithi190.stdout:7/551: mknod d0/d14/d5d/cca 0 2024-05-06T12:14:50.541 INFO:tasks.workunit.client.0.smithi092.stdout:3/929: symlink d8/dd7/l129 0 2024-05-06T12:14:50.542 INFO:tasks.workunit.client.0.smithi092.stdout:3/930: creat d8/d24/d33/d4b/ddf/f12a x:0 0 0 2024-05-06T12:14:50.546 INFO:tasks.workunit.client.1.smithi190.stdout:5/606: creat d2/d6c/fc5 x:0 0 0 2024-05-06T12:14:50.546 INFO:tasks.workunit.client.1.smithi190.stdout:8/633: unlink d3/d8/d25/d3b/d90/f5f 0 2024-05-06T12:14:50.548 INFO:tasks.workunit.client.1.smithi190.stdout:0/591: dwrite d9/f5f [0,4194304] 0 2024-05-06T12:14:50.549 INFO:tasks.workunit.client.1.smithi190.stdout:7/552: dread - d0/d1/d32/f44 zero size 2024-05-06T12:14:50.552 INFO:tasks.workunit.client.1.smithi190.stdout:4/406: rmdir d2/d11/d16/d53/d7b 39 2024-05-06T12:14:50.553 INFO:tasks.workunit.client.1.smithi190.stdout:6/426: dwrite df/d2c/d61/f79 [0,4194304] 0 2024-05-06T12:14:50.553 INFO:tasks.workunit.client.1.smithi190.stdout:9/655: rmdir db/dd/d16/d58/d87/d22/d49/d88/db5 39 2024-05-06T12:14:50.553 INFO:tasks.workunit.client.0.smithi092.stdout:3/931: dwrite d8/d24/ff5 [0,4194304] 0 2024-05-06T12:14:50.554 INFO:tasks.workunit.client.1.smithi190.stdout:8/634: symlink d3/d8/d2b/d3e/ldd 0 2024-05-06T12:14:50.555 INFO:tasks.workunit.client.0.smithi092.stdout:3/932: fdatasync d8/d24/d27/ffe 0 2024-05-06T12:14:50.556 INFO:tasks.workunit.client.0.smithi092.stdout:3/933: readlink d8/d24/d33/d4b/ddf/lfa 0 2024-05-06T12:14:50.557 INFO:tasks.workunit.client.1.smithi190.stdout:3/633: readlink d5/d7/d6d/dc6/lde 0 2024-05-06T12:14:50.557 INFO:tasks.workunit.client.1.smithi190.stdout:6/427: write df/f82 [706257,76759] 0 2024-05-06T12:14:50.559 INFO:tasks.workunit.client.1.smithi190.stdout:7/553: creat d0/d1/d32/daa/fcb x:0 0 0 2024-05-06T12:14:50.561 INFO:tasks.workunit.client.0.smithi092.stdout:3/934: dread d8/db/d4c/d3d/f60 [0,4194304] 0 2024-05-06T12:14:50.562 INFO:tasks.workunit.client.1.smithi190.stdout:0/592: rename d9/d95 to d9/dc/d48/d50/d87/dd2 0 2024-05-06T12:14:50.562 INFO:tasks.workunit.client.1.smithi190.stdout:7/554: readlink d0/l16 0 2024-05-06T12:14:50.563 INFO:tasks.workunit.client.0.smithi092.stdout:3/935: readlink d8/db/d4c/d3d/d49/l8b 0 2024-05-06T12:14:50.563 INFO:tasks.workunit.client.1.smithi190.stdout:5/607: read d2/f8 [1587091,130819] 0 2024-05-06T12:14:50.565 INFO:tasks.workunit.client.0.smithi092.stdout:3/936: mknod d8/db/d4c/d3d/d49/df8/d5b/d98/c12b 0 2024-05-06T12:14:50.569 INFO:tasks.workunit.client.1.smithi190.stdout:4/407: dread d2/d11/d16/d61/d66/f74 [0,4194304] 0 2024-05-06T12:14:50.570 INFO:tasks.workunit.client.0.smithi092.stdout:3/937: dread d8/d24/d27/fa0 [4194304,4194304] 0 2024-05-06T12:14:50.570 INFO:tasks.workunit.client.1.smithi190.stdout:2/491: dwrite d2/d21/f3d [0,4194304] 0 2024-05-06T12:14:50.572 INFO:tasks.workunit.client.1.smithi190.stdout:9/656: rename db/d23/d25 to db/dd/d16/d58/ddc 0 2024-05-06T12:14:50.573 INFO:tasks.workunit.client.1.smithi190.stdout:7/555: fdatasync d0/d1/d2/f1b 0 2024-05-06T12:14:50.574 INFO:tasks.workunit.client.0.smithi092.stdout:3/938: link d8/d24/d33/d91/cb9 d8/d1b/d112/c12c 0 2024-05-06T12:14:50.574 INFO:tasks.workunit.client.1.smithi190.stdout:9/657: chown db/dd/df/l28 16151 1 2024-05-06T12:14:50.578 INFO:tasks.workunit.client.1.smithi190.stdout:7/556: write d0/f40 [4093885,51527] 0 2024-05-06T12:14:50.579 INFO:tasks.workunit.client.0.smithi092.stdout:3/939: dread d8/d24/d33/d4b/d74/f9e [0,4194304] 0 2024-05-06T12:14:50.579 INFO:tasks.workunit.client.1.smithi190.stdout:6/428: rename df/d17/d44/f62 to df/d2c/d1e/f95 0 2024-05-06T12:14:50.579 INFO:tasks.workunit.client.1.smithi190.stdout:4/408: symlink d2/d11/l8d 0 2024-05-06T12:14:50.580 INFO:tasks.workunit.client.1.smithi190.stdout:7/557: chown d0/d4c/d60/c8e 55963 1 2024-05-06T12:14:50.583 INFO:tasks.workunit.client.0.smithi092.stdout:3/940: rename d8/d24/d33/d4b/fc5 to d8/d24/d27/f12d 0 2024-05-06T12:14:50.585 INFO:tasks.workunit.client.0.smithi092.stdout:3/941: chown d8/db/d14/d5a/ce0 0 1 2024-05-06T12:14:50.585 INFO:tasks.workunit.client.1.smithi190.stdout:9/658: creat db/d23/fdd x:0 0 0 2024-05-06T12:14:50.588 INFO:tasks.workunit.client.1.smithi190.stdout:4/409: rename d2/d11/d16/d53 to d2/d11/d16/d8e 0 2024-05-06T12:14:50.589 INFO:tasks.workunit.client.1.smithi190.stdout:6/429: getdents df/d2c/d61/d84 0 2024-05-06T12:14:50.595 INFO:tasks.workunit.client.0.smithi092.stdout:3/942: dwrite d8/db/f9f [0,4194304] 0 2024-05-06T12:14:50.599 INFO:tasks.workunit.client.0.smithi092.stdout:3/943: chown d8/db/d14/d5a/db0/cc3 239251 1 2024-05-06T12:14:50.599 INFO:tasks.workunit.client.0.smithi092.stdout:3/944: stat d8/db/d14/f50 0 2024-05-06T12:14:50.601 INFO:tasks.workunit.client.0.smithi092.stdout:3/945: symlink d8/db/d4c/d3d/d49/ded/d121/l12e 0 2024-05-06T12:14:50.603 INFO:tasks.workunit.client.0.smithi092.stdout:3/946: rmdir d8/db/d4c/d3d/d49/df8/d5b/d101 39 2024-05-06T12:14:50.606 INFO:tasks.workunit.client.0.smithi092.stdout:3/947: symlink d8/db/d4c/d40/l12f 0 2024-05-06T12:14:50.607 INFO:tasks.workunit.client.0.smithi092.stdout:3/948: chown d8/d24/d33/d91/cd8 1323539 1 2024-05-06T12:14:50.607 INFO:tasks.workunit.client.0.smithi092.stdout:3/949: fdatasync d8/d24/ff5 0 2024-05-06T12:14:50.613 INFO:tasks.workunit.client.0.smithi092.stdout:3/950: dread d8/db/d4c/d40/d7a/d90/fc2 [0,4194304] 0 2024-05-06T12:14:50.620 INFO:tasks.workunit.client.0.smithi092.stdout:3/951: dwrite d8/d24/d33/f64 [0,4194304] 0 2024-05-06T12:14:50.621 INFO:tasks.workunit.client.0.smithi092.stdout:3/952: chown d8/d24/d27/cba 12 1 2024-05-06T12:14:50.625 INFO:tasks.workunit.client.0.smithi092.stdout:3/953: link d8/d1b/fd5 d8/d24/d27/f130 0 2024-05-06T12:14:50.641 INFO:tasks.workunit.client.0.smithi092.stdout:3/954: dread d8/db/d14/f50 [0,4194304] 0 2024-05-06T12:14:50.643 INFO:tasks.workunit.client.0.smithi092.stdout:3/955: write d8/db/d4c/d40/d7a/d90/fc2 [760453,71482] 0 2024-05-06T12:14:50.645 INFO:tasks.workunit.client.0.smithi092.stdout:3/956: getdents d8/db/d4c/d3d/d111 0 2024-05-06T12:14:50.647 INFO:tasks.workunit.client.0.smithi092.stdout:3/957: creat d8/dd7/d107/f131 x:0 0 0 2024-05-06T12:14:50.649 INFO:tasks.workunit.client.1.smithi190.stdout:1/441: sync 2024-05-06T12:14:50.652 INFO:tasks.workunit.client.0.smithi092.stdout:3/958: write d8/db/d4c/d3d/d49/ded/d121/f123 [723578,31971] 0 2024-05-06T12:14:50.653 INFO:tasks.workunit.client.0.smithi092.stdout:3/959: fdatasync d8/db/d4c/d3d/d49/f8a 0 2024-05-06T12:14:50.654 INFO:tasks.workunit.client.1.smithi190.stdout:8/635: dread d3/fc [4194304,4194304] 0 2024-05-06T12:14:50.655 INFO:tasks.workunit.client.0.smithi092.stdout:3/960: mkdir d8/d24/d33/d4b/d74/d132 0 2024-05-06T12:14:50.655 INFO:tasks.workunit.client.1.smithi190.stdout:5/608: write d2/da/d59/d5f/d6e/fb5 [751103,85743] 0 2024-05-06T12:14:50.657 INFO:tasks.workunit.client.0.smithi092.stdout:3/961: unlink d8/db/d4c/d3d/d48/ff3 0 2024-05-06T12:14:50.658 INFO:tasks.workunit.client.1.smithi190.stdout:7/558: write d0/d1/d18/f68 [4082326,89138] 0 2024-05-06T12:14:50.659 INFO:tasks.workunit.client.1.smithi190.stdout:4/410: dwrite d2/d11/f14 [0,4194304] 0 2024-05-06T12:14:50.661 INFO:tasks.workunit.client.0.smithi092.stdout:3/962: dread d8/db/d4c/d3d/f60 [0,4194304] 0 2024-05-06T12:14:50.663 INFO:tasks.workunit.client.1.smithi190.stdout:8/636: chown d3/d8/d25/d3b/d90/d6e/f94 1057625317 1 2024-05-06T12:14:50.664 INFO:tasks.workunit.client.0.smithi092.stdout:3/963: rename d8/d1b/d112/lec to d8/d24/d33/da8/l133 0 2024-05-06T12:14:50.665 INFO:tasks.workunit.client.1.smithi190.stdout:5/609: write d2/da/d59/d5f/d2e/d3b/fa5 [148630,123596] 0 2024-05-06T12:14:50.665 INFO:tasks.workunit.client.0.smithi092.stdout:3/964: truncate d8/db/d4c/d3d/fa9 1370767 0 2024-05-06T12:14:50.666 INFO:tasks.workunit.client.0.smithi092.stdout:3/965: stat d8/db/d14 0 2024-05-06T12:14:50.667 INFO:tasks.workunit.client.1.smithi190.stdout:9/659: dread db/dd/d16/d4c/d4e/f82 [0,4194304] 0 2024-05-06T12:14:50.669 INFO:tasks.workunit.client.1.smithi190.stdout:2/492: dwrite d2/d8/dc/f94 [0,4194304] 0 2024-05-06T12:14:50.669 INFO:tasks.workunit.client.0.smithi092.stdout:3/966: creat d8/db/d4c/d3d/d49/df8/d5b/d101/f134 x:0 0 0 2024-05-06T12:14:50.670 INFO:tasks.workunit.client.1.smithi190.stdout:2/493: stat d2/d8/d33/d46 0 2024-05-06T12:14:50.671 INFO:tasks.workunit.client.1.smithi190.stdout:9/660: readlink db/dd/d16/d4c/d4e/d5e/lbb 0 2024-05-06T12:14:50.671 INFO:tasks.workunit.client.1.smithi190.stdout:2/494: readlink d2/d8/d19/d43/d79/l98 0 2024-05-06T12:14:50.673 INFO:tasks.workunit.client.1.smithi190.stdout:2/495: chown d2/c7 1 1 2024-05-06T12:14:50.673 INFO:tasks.workunit.client.1.smithi190.stdout:2/496: dread - d2/d21/f85 zero size 2024-05-06T12:14:50.674 INFO:tasks.workunit.client.1.smithi190.stdout:1/442: rename d5/d4f/d65/d24/d8c to d5/d4f/d65/d62/d9d 0 2024-05-06T12:14:50.677 INFO:tasks.workunit.client.1.smithi190.stdout:6/430: link df/d17/d44/l5f df/d2c/d1e/d57/l96 0 2024-05-06T12:14:50.678 INFO:tasks.workunit.client.1.smithi190.stdout:9/661: unlink db/dd/df/l28 0 2024-05-06T12:14:50.681 INFO:tasks.workunit.client.0.smithi092.stdout:3/967: write f4 [1606268,32285] 0 2024-05-06T12:14:50.683 INFO:tasks.workunit.client.1.smithi190.stdout:4/411: rename d2/d11/d16/d4e/f5a to d2/d11/d16/d61/d66/d6b/f8f 0 2024-05-06T12:14:50.684 INFO:tasks.workunit.client.0.smithi092.stdout:3/968: fdatasync d8/db/d4c/d3d/fdd 0 2024-05-06T12:14:50.684 INFO:tasks.workunit.client.0.smithi092.stdout:3/969: stat d8/d1b/d112/dea 0 2024-05-06T12:14:50.685 INFO:tasks.workunit.client.1.smithi190.stdout:6/431: symlink df/d17/d44/l97 0 2024-05-06T12:14:50.685 INFO:tasks.workunit.client.0.smithi092.stdout:3/970: chown d8/db/d4c/d40/l12f 5 1 2024-05-06T12:14:50.687 INFO:tasks.workunit.client.1.smithi190.stdout:7/559: rmdir d0/d1/d32/d45/d70/d33/d57/dc3 0 2024-05-06T12:14:50.688 INFO:tasks.workunit.client.0.smithi092.stdout:3/971: read d8/db/d4c/d40/f52 [843113,126008] 0 2024-05-06T12:14:50.688 INFO:tasks.workunit.client.0.smithi092.stdout:3/972: dread - d8/db/d4c/d3d/f10c zero size 2024-05-06T12:14:50.690 INFO:tasks.workunit.client.1.smithi190.stdout:4/412: unlink d2/d11/d16/f3e 0 2024-05-06T12:14:50.690 INFO:tasks.workunit.client.0.smithi092.stdout:3/973: mknod d8/d24/d33/d91/c135 0 2024-05-06T12:14:50.690 INFO:tasks.workunit.client.1.smithi190.stdout:2/497: mknod d2/d8/d33/d46/d55/d91/ca4 0 2024-05-06T12:14:50.691 INFO:tasks.workunit.client.1.smithi190.stdout:6/432: mknod df/d2c/d1e/d57/c98 0 2024-05-06T12:14:50.692 INFO:tasks.workunit.client.0.smithi092.stdout:3/974: chown d8/d24/d33/d4b/c1e 400 1 2024-05-06T12:14:50.694 INFO:tasks.workunit.client.0.smithi092.stdout:3/975: read d8/db/ffd [589161,115569] 0 2024-05-06T12:14:50.696 INFO:tasks.workunit.client.0.smithi092.stdout:3/976: fsync d8/d24/d33/d4b/ff9 0 2024-05-06T12:14:50.697 INFO:tasks.workunit.client.1.smithi190.stdout:5/610: rename d2/da/d59/dbf to d2/da/d59/d5f/d2e/dc6 0 2024-05-06T12:14:50.699 INFO:tasks.workunit.client.0.smithi092.stdout:3/977: rename f4 to d8/dd7/f136 0 2024-05-06T12:14:50.700 INFO:tasks.workunit.client.1.smithi190.stdout:7/560: creat d0/d1/d32/fcc x:0 0 0 2024-05-06T12:14:50.700 INFO:tasks.workunit.client.0.smithi092.stdout:3/978: chown d8/d24/d33/d91/f118 48095 1 2024-05-06T12:14:50.702 INFO:tasks.workunit.client.1.smithi190.stdout:1/443: rename d5/d4f/d65/d32/c55 to d5/d4f/d65/d24/d43/c9e 0 2024-05-06T12:14:50.703 INFO:tasks.workunit.client.0.smithi092.stdout:3/979: rmdir d8/db/d4c/d3d/d48 39 2024-05-06T12:14:50.703 INFO:tasks.workunit.client.1.smithi190.stdout:4/413: creat d2/d11/d16/f90 x:0 0 0 2024-05-06T12:14:50.704 INFO:tasks.workunit.client.1.smithi190.stdout:2/498: creat d2/fa5 x:0 0 0 2024-05-06T12:14:50.705 INFO:tasks.workunit.client.0.smithi092.stdout:3/980: unlink d8/db/fad 0 2024-05-06T12:14:50.706 INFO:tasks.workunit.client.1.smithi190.stdout:6/433: creat df/f99 x:0 0 0 2024-05-06T12:14:50.707 INFO:tasks.workunit.client.1.smithi190.stdout:6/434: readlink df/d2c/d1e/d57/d53/l5d 0 2024-05-06T12:14:50.708 INFO:tasks.workunit.client.0.smithi092.stdout:3/981: mkdir d8/d137 0 2024-05-06T12:14:50.709 INFO:tasks.workunit.client.1.smithi190.stdout:5/611: rename d2/f5 to d2/da/d59/d5f/d2e/d3b/d5a/fc7 0 2024-05-06T12:14:50.711 INFO:tasks.workunit.client.1.smithi190.stdout:1/444: mknod d5/d4f/d65/d32/d47/d53/d8b/c9f 0 2024-05-06T12:14:50.711 INFO:tasks.workunit.client.0.smithi092.stdout:3/982: mknod d8/db/d4c/d3d/d49/df8/d5b/c138 0 2024-05-06T12:14:50.712 INFO:tasks.workunit.client.1.smithi190.stdout:4/414: symlink d2/d11/d16/d8e/d7b/l91 0 2024-05-06T12:14:50.713 INFO:tasks.workunit.client.1.smithi190.stdout:2/499: dwrite d2/d8/d19/d43/d79/f82 [0,4194304] 0 2024-05-06T12:14:50.713 INFO:tasks.workunit.client.0.smithi092.stdout:3/983: truncate d8/db/d4c/d3d/d49/df8/d5b/d101/f134 774627 0 2024-05-06T12:14:50.715 INFO:tasks.workunit.client.1.smithi190.stdout:6/435: write df/d2c/d1e/d57/f8a [4471678,105744] 0 2024-05-06T12:14:50.722 INFO:tasks.workunit.client.1.smithi190.stdout:5/612: dread d2/d6c/d7d/fa2 [0,4194304] 0 2024-05-06T12:14:50.722 INFO:tasks.workunit.client.0.smithi092.stdout:3/984: dwrite d8/db/f9f [0,4194304] 0 2024-05-06T12:14:50.724 INFO:tasks.workunit.client.0.smithi092.stdout:3/985: mknod d8/d24/d33/da8/c139 0 2024-05-06T12:14:50.724 INFO:tasks.workunit.client.1.smithi190.stdout:4/415: creat d2/d11/d16/d61/d67/d6e/f92 x:0 0 0 2024-05-06T12:14:50.730 INFO:tasks.workunit.client.1.smithi190.stdout:8/637: dwrite d3/d8/f2c [0,4194304] 0 2024-05-06T12:14:50.730 INFO:tasks.workunit.client.0.smithi092.stdout:3/986: dwrite d8/db/d4c/d40/fff [0,4194304] 0 2024-05-06T12:14:50.731 INFO:tasks.workunit.client.1.smithi190.stdout:6/436: dwrite df/d12/f76 [0,4194304] 0 2024-05-06T12:14:50.734 INFO:tasks.workunit.client.0.smithi092.stdout:3/987: rename d8/d1b/f81 to d8/d24/d33/d4b/d74/d132/f13a 0 2024-05-06T12:14:50.735 INFO:tasks.workunit.client.1.smithi190.stdout:5/613: creat d2/da/d59/d5f/d6e/fc8 x:0 0 0 2024-05-06T12:14:50.737 INFO:tasks.workunit.client.1.smithi190.stdout:8/638: chown d3/l7 23929816 1 2024-05-06T12:14:50.742 INFO:tasks.workunit.client.1.smithi190.stdout:4/416: dread d2/f64 [0,4194304] 0 2024-05-06T12:14:50.742 INFO:tasks.workunit.client.1.smithi190.stdout:6/437: creat df/d2c/d92/f9a x:0 0 0 2024-05-06T12:14:50.743 INFO:tasks.workunit.client.1.smithi190.stdout:4/417: write d2/f59 [79525,120474] 0 2024-05-06T12:14:50.747 INFO:tasks.workunit.client.1.smithi190.stdout:5/614: rmdir d2/da/d59/d5f/d50 39 2024-05-06T12:14:50.750 INFO:tasks.workunit.client.1.smithi190.stdout:2/500: getdents d2/d21 0 2024-05-06T12:14:50.752 INFO:tasks.workunit.client.1.smithi190.stdout:8/639: dread d3/d8/d68/d9f/fbc [0,4194304] 0 2024-05-06T12:14:50.753 INFO:tasks.workunit.client.1.smithi190.stdout:4/418: symlink d2/d11/d16/d61/d66/d6b/l93 0 2024-05-06T12:14:50.755 INFO:tasks.workunit.client.1.smithi190.stdout:5/615: write d2/d27/f48 [1018427,88241] 0 2024-05-06T12:14:50.755 INFO:tasks.workunit.client.1.smithi190.stdout:6/438: mknod df/d17/c9b 0 2024-05-06T12:14:50.756 INFO:tasks.workunit.client.0.smithi092.stdout:3/988: getdents d8/d24/d33/d4b/d74/d132 0 2024-05-06T12:14:50.757 INFO:tasks.workunit.client.1.smithi190.stdout:5/616: chown d2/da/l45 226894931 1 2024-05-06T12:14:50.757 INFO:tasks.workunit.client.1.smithi190.stdout:2/501: rename d2/c4 to d2/d8/d19/d9b/ca6 0 2024-05-06T12:14:50.758 INFO:tasks.workunit.client.1.smithi190.stdout:6/439: chown df/d12/f83 28040418 1 2024-05-06T12:14:50.758 INFO:tasks.workunit.client.0.smithi092.stdout:3/989: symlink d8/db/d4c/l13b 0 2024-05-06T12:14:50.759 INFO:tasks.workunit.client.1.smithi190.stdout:2/502: chown d2/d4c/f5a 480 1 2024-05-06T12:14:50.761 INFO:tasks.workunit.client.1.smithi190.stdout:7/561: dwrite d0/da/d36/d53/d5c/fc7 [0,4194304] 0 2024-05-06T12:14:50.761 INFO:tasks.workunit.client.1.smithi190.stdout:6/440: write df/f22 [1592399,17838] 0 2024-05-06T12:14:50.761 INFO:tasks.workunit.client.0.smithi092.stdout:3/990: rename d8/d24/d33/d4b/l3b to d8/d1b/d73/l13c 0 2024-05-06T12:14:50.763 INFO:tasks.workunit.client.1.smithi190.stdout:4/419: mknod d2/d11/d16/d8e/c94 0 2024-05-06T12:14:50.765 INFO:tasks.workunit.client.1.smithi190.stdout:8/640: getdents d3/d8/d25/d3b/d90/dcb 0 2024-05-06T12:14:50.765 INFO:tasks.workunit.client.0.smithi092.stdout:3/991: dread d8/d1b/d112/dea/f105 [0,4194304] 0 2024-05-06T12:14:50.767 INFO:tasks.workunit.client.0.smithi092.stdout:3/992: mkdir d8/d24/d33/d13d 0 2024-05-06T12:14:50.769 INFO:tasks.workunit.client.0.smithi092.stdout:3/993: stat d8/db/d14/d5a/c9d 0 2024-05-06T12:14:50.770 INFO:tasks.workunit.client.0.smithi092.stdout:3/994: stat d8/d24/d33/d4b/l35 0 2024-05-06T12:14:50.771 INFO:tasks.workunit.client.1.smithi190.stdout:4/420: chown d2/d11/l1d 1541 1 2024-05-06T12:14:50.772 INFO:tasks.workunit.client.1.smithi190.stdout:1/445: dwrite d5/d4f/d65/f3c [0,4194304] 0 2024-05-06T12:14:50.772 INFO:tasks.workunit.client.1.smithi190.stdout:2/503: creat d2/d8/d19/fa7 x:0 0 0 2024-05-06T12:14:50.773 INFO:tasks.workunit.client.1.smithi190.stdout:2/504: dread - d2/f6a zero size 2024-05-06T12:14:50.774 INFO:tasks.workunit.client.1.smithi190.stdout:7/562: symlink d0/d1/d32/d45/d70/d35/lcd 0 2024-05-06T12:14:50.775 INFO:tasks.workunit.client.1.smithi190.stdout:2/505: chown d2/d8/d19/d43 349049 1 2024-05-06T12:14:50.776 INFO:tasks.workunit.client.0.smithi092.stdout:3/995: rename d8/d24/d33/d4b/l35 to d8/db/d4c/d3d/d48/l13e 0 2024-05-06T12:14:50.776 INFO:tasks.workunit.client.1.smithi190.stdout:8/641: fdatasync d3/f52 0 2024-05-06T12:14:50.778 INFO:tasks.workunit.client.1.smithi190.stdout:1/446: creat d5/d4f/d65/d62/d9d/fa0 x:0 0 0 2024-05-06T12:14:50.781 INFO:tasks.workunit.client.1.smithi190.stdout:2/506: symlink d2/d8/dc/la8 0 2024-05-06T12:14:50.781 INFO:tasks.workunit.client.0.smithi092.stdout:3/996: read d8/db/d4c/d3d/fdd [295464,63147] 0 2024-05-06T12:14:50.783 INFO:tasks.workunit.client.1.smithi190.stdout:7/563: mknod d0/da/d36/d53/d5c/cce 0 2024-05-06T12:14:50.783 INFO:tasks.workunit.client.1.smithi190.stdout:2/507: rmdir d2/d21 39 2024-05-06T12:14:50.784 INFO:tasks.workunit.client.1.smithi190.stdout:2/508: chown d2/c6 1338896275 1 2024-05-06T12:14:50.786 INFO:tasks.workunit.client.0.smithi092.stdout:3/997: creat d8/db/d4c/d40/d7a/f13f x:0 0 0 2024-05-06T12:14:50.787 INFO:tasks.workunit.client.1.smithi190.stdout:2/509: read d2/d21/f5e [1772035,4704] 0 2024-05-06T12:14:50.787 INFO:tasks.workunit.client.1.smithi190.stdout:1/447: fsync d5/d15/d1e/f5a 0 2024-05-06T12:14:50.788 INFO:tasks.workunit.client.1.smithi190.stdout:3/634: sync 2024-05-06T12:14:50.792 INFO:tasks.workunit.client.1.smithi190.stdout:2/510: dread d2/f12 [0,4194304] 0 2024-05-06T12:14:50.793 INFO:tasks.workunit.client.1.smithi190.stdout:1/448: rmdir d5/d4f/d65/d24/d3a 39 2024-05-06T12:14:50.796 INFO:tasks.workunit.client.1.smithi190.stdout:6/441: write df/d12/f21 [271715,35050] 0 2024-05-06T12:14:50.797 INFO:tasks.workunit.client.0.smithi092.stdout:3/998: write d8/d24/d27/f130 [989072,67321] 0 2024-05-06T12:14:50.798 INFO:tasks.workunit.client.0.smithi092.stdout:3/999: fdatasync d8/db/d4c/d3d/d49/df8/d5b/d98/fb2 0 2024-05-06T12:14:50.800 INFO:tasks.workunit.client.1.smithi190.stdout:4/421: dread d2/f55 [0,4194304] 0 2024-05-06T12:14:50.802 INFO:tasks.workunit.client.1.smithi190.stdout:3/635: dread d5/d7/f23 [0,4194304] 0 2024-05-06T12:14:50.802 INFO:tasks.workunit.client.0.smithi092.stderr:+ rm -rf -- ./tmp.yrc4lArKdJ 2024-05-06T12:14:50.803 INFO:tasks.workunit.client.1.smithi190.stdout:5/617: dread d2/da/d59/d5f/f3c [0,4194304] 0 2024-05-06T12:14:50.803 INFO:tasks.workunit.client.1.smithi190.stdout:1/449: mknod d5/d4f/d65/d24/d3a/ca1 0 2024-05-06T12:14:50.805 INFO:tasks.workunit.client.1.smithi190.stdout:6/442: write df/d17/d44/f48 [3762743,55201] 0 2024-05-06T12:14:50.808 INFO:tasks.workunit.client.1.smithi190.stdout:8/642: dread d3/d35/d55/f73 [0,4194304] 0 2024-05-06T12:14:50.810 INFO:tasks.workunit.client.1.smithi190.stdout:4/422: creat d2/d11/d16/d8e/d80/f95 x:0 0 0 2024-05-06T12:14:50.811 INFO:tasks.workunit.client.1.smithi190.stdout:5/618: creat d2/db6/fc9 x:0 0 0 2024-05-06T12:14:50.812 INFO:tasks.workunit.client.1.smithi190.stdout:0/593: sync 2024-05-06T12:14:50.812 INFO:tasks.workunit.client.1.smithi190.stdout:9/662: sync 2024-05-06T12:14:50.814 INFO:tasks.workunit.client.1.smithi190.stdout:8/643: mkdir d3/d8/d68/dde 0 2024-05-06T12:14:50.815 INFO:tasks.workunit.client.1.smithi190.stdout:0/594: chown d9/dc/dd0 0 1 2024-05-06T12:14:50.819 INFO:tasks.workunit.client.1.smithi190.stdout:1/450: dwrite d5/d4f/d65/d24/f7e [0,4194304] 0 2024-05-06T12:14:50.819 INFO:tasks.workunit.client.1.smithi190.stdout:4/423: write d2/d11/d16/d41/f62 [882313,106923] 0 2024-05-06T12:14:50.823 INFO:tasks.workunit.client.1.smithi190.stdout:3/636: link d5/d14/d11/cc8 d5/d7/d7f/ce2 0 2024-05-06T12:14:50.825 INFO:tasks.workunit.client.1.smithi190.stdout:5/619: read d2/da/d59/d5f/d6e/fb5 [607561,112435] 0 2024-05-06T12:14:50.826 INFO:tasks.workunit.client.1.smithi190.stdout:4/424: write d2/f32 [4779941,51933] 0 2024-05-06T12:14:50.828 INFO:tasks.workunit.client.1.smithi190.stdout:6/443: dwrite df/d12/f76 [0,4194304] 0 2024-05-06T12:14:50.828 INFO:tasks.workunit.client.1.smithi190.stdout:8/644: chown d3/d35/f3c 11622178 1 2024-05-06T12:14:50.830 INFO:tasks.workunit.client.1.smithi190.stdout:4/425: chown d2/d11/d16/d61/d66 14996990 1 2024-05-06T12:14:50.833 INFO:tasks.workunit.client.1.smithi190.stdout:3/637: fdatasync d5/d14/d11/d21/d44/fcd 0 2024-05-06T12:14:50.834 INFO:tasks.workunit.client.1.smithi190.stdout:7/564: dwrite d0/d4/f39 [0,4194304] 0 2024-05-06T12:14:50.837 INFO:tasks.workunit.client.1.smithi190.stdout:5/620: chown d2/da/d59/d5f/f36 449305 1 2024-05-06T12:14:50.840 INFO:tasks.workunit.client.1.smithi190.stdout:2/511: dwrite d2/d8/d19/d4f/f57 [0,4194304] 0 2024-05-06T12:14:50.840 INFO:tasks.workunit.client.1.smithi190.stdout:1/451: creat d5/d15/fa2 x:0 0 0 2024-05-06T12:14:50.843 INFO:tasks.workunit.client.1.smithi190.stdout:0/595: rename d9/l65 to d9/dc/d28/d4c/ld3 0 2024-05-06T12:14:50.849 INFO:tasks.workunit.client.1.smithi190.stdout:8/645: mknod d3/d8/d25/d65/cdf 0 2024-05-06T12:14:50.849 INFO:tasks.workunit.client.1.smithi190.stdout:7/565: mknod d0/d1/d32/daa/ccf 0 2024-05-06T12:14:50.856 INFO:tasks.workunit.client.1.smithi190.stdout:6/444: dwrite df/d17/d37/f58 [0,4194304] 0 2024-05-06T12:14:50.857 INFO:tasks.workunit.client.1.smithi190.stdout:8/646: write d3/d35/d3a/f91 [767527,52523] 0 2024-05-06T12:14:50.858 INFO:tasks.workunit.client.1.smithi190.stdout:9/663: rename db/dd/d16/fb2 to db/dd/d16/d58/d87/d22/d49/d8c/fde 0 2024-05-06T12:14:50.860 INFO:tasks.workunit.client.1.smithi190.stdout:9/664: stat db/d23/d7e 0 2024-05-06T12:14:50.860 INFO:tasks.workunit.client.1.smithi190.stdout:1/452: write d5/d4f/d65/d24/f25 [1872697,5022] 0 2024-05-06T12:14:50.867 INFO:tasks.workunit.client.1.smithi190.stdout:6/445: creat df/d17/d37/f9c x:0 0 0 2024-05-06T12:14:50.868 INFO:tasks.workunit.client.1.smithi190.stdout:2/512: write d2/f1f [469337,64682] 0 2024-05-06T12:14:50.871 INFO:tasks.workunit.client.1.smithi190.stdout:0/596: link d9/dc/db6/fb7 d9/dc/d28/d77/fd4 0 2024-05-06T12:14:50.873 INFO:tasks.workunit.client.1.smithi190.stdout:8/647: readlink d3/d8/d2b/ldc 0 2024-05-06T12:14:50.876 INFO:tasks.workunit.client.1.smithi190.stdout:3/638: truncate d5/d7/d15/d1f/fe 4015597 0 2024-05-06T12:14:50.876 INFO:tasks.workunit.client.1.smithi190.stdout:3/639: dread - d5/d7/fb9 zero size 2024-05-06T12:14:50.878 INFO:tasks.workunit.client.1.smithi190.stdout:3/640: write d5/d7/f3a [4007787,72592] 0 2024-05-06T12:14:50.878 INFO:tasks.workunit.client.1.smithi190.stdout:4/426: rename d2/d11/c21 to d2/d11/d16/d61/d67/d6e/d78/c96 0 2024-05-06T12:14:50.880 INFO:tasks.workunit.client.1.smithi190.stdout:6/446: creat df/d17/d44/f9d x:0 0 0 2024-05-06T12:14:50.882 INFO:tasks.workunit.client.1.smithi190.stdout:1/453: mknod d5/d15/ca3 0 2024-05-06T12:14:50.883 INFO:tasks.workunit.client.1.smithi190.stdout:5/621: dwrite d2/da/d59/d5f/d50/d6d/db3/fb4 [0,4194304] 0 2024-05-06T12:14:50.884 INFO:tasks.workunit.client.1.smithi190.stdout:7/566: dread d0/da/d36/d53/d5c/fb9 [0,4194304] 0 2024-05-06T12:14:50.884 INFO:tasks.workunit.client.1.smithi190.stdout:9/665: creat db/dd/d16/d58/d87/d22/d49/d88/db5/fdf x:0 0 0 2024-05-06T12:14:50.884 INFO:tasks.workunit.client.1.smithi190.stdout:8/648: chown d3/d8/dcf/l86 16240831 1 2024-05-06T12:14:50.887 INFO:tasks.workunit.client.1.smithi190.stdout:5/622: write d2/da/d59/d5f/d2e/d3b/f42 [739479,39308] 0 2024-05-06T12:14:50.888 INFO:tasks.workunit.client.1.smithi190.stdout:7/567: dread - d0/da/d36/d42/fa7 zero size 2024-05-06T12:14:50.889 INFO:tasks.workunit.client.1.smithi190.stdout:7/568: chown d0/d1/f3e 1732778987 1 2024-05-06T12:14:50.892 INFO:tasks.workunit.client.1.smithi190.stdout:8/649: dread d3/d35/d55/f96 [0,4194304] 0 2024-05-06T12:14:50.892 INFO:tasks.workunit.client.1.smithi190.stdout:7/569: dread - d0/d1/d32/daa/fb1 zero size 2024-05-06T12:14:50.892 INFO:tasks.workunit.client.1.smithi190.stdout:4/427: creat d2/d11/d16/d8e/d80/f97 x:0 0 0 2024-05-06T12:14:50.893 INFO:tasks.workunit.client.1.smithi190.stdout:3/641: creat d5/d7/d15/d1f/d33/fe3 x:0 0 0 2024-05-06T12:14:50.894 INFO:tasks.workunit.client.1.smithi190.stdout:0/597: write d9/dc/d48/f8b [629055,99826] 0 2024-05-06T12:14:50.894 INFO:tasks.workunit.client.1.smithi190.stdout:4/428: chown d2/d10/l28 60924890 1 2024-05-06T12:14:50.896 INFO:tasks.workunit.client.1.smithi190.stdout:0/598: write d9/dc/d48/f8b [697558,54048] 0 2024-05-06T12:14:50.897 INFO:tasks.workunit.client.1.smithi190.stdout:1/454: rmdir d5/d4f/d65/d32/d47 39 2024-05-06T12:14:50.900 INFO:tasks.workunit.client.1.smithi190.stdout:4/429: read d2/f37 [2867265,4626] 0 2024-05-06T12:14:50.900 INFO:tasks.workunit.client.1.smithi190.stdout:9/666: symlink db/dd/d16/d4c/d4e/d5e/dce/le0 0 2024-05-06T12:14:50.902 INFO:tasks.workunit.client.1.smithi190.stdout:2/513: truncate d2/d10/f87 819437 0 2024-05-06T12:14:50.906 INFO:tasks.workunit.client.1.smithi190.stdout:8/650: rmdir d3/d8 39 2024-05-06T12:14:50.908 INFO:tasks.workunit.client.1.smithi190.stdout:7/570: creat d0/d4c/d60/dac/fd0 x:0 0 0 2024-05-06T12:14:50.910 INFO:tasks.workunit.client.1.smithi190.stdout:0/599: dread d9/dc/d28/d4c/f4b [0,4194304] 0 2024-05-06T12:14:50.910 INFO:tasks.workunit.client.1.smithi190.stdout:6/447: creat df/d2c/d61/d87/f9e x:0 0 0 2024-05-06T12:14:50.910 INFO:tasks.workunit.client.1.smithi190.stdout:7/571: write d0/d4/f39 [190579,87807] 0 2024-05-06T12:14:50.911 INFO:tasks.workunit.client.1.smithi190.stdout:3/642: unlink d5/d7/d15/d1f/d33/f53 0 2024-05-06T12:14:50.919 INFO:tasks.workunit.client.1.smithi190.stdout:2/514: unlink d2/d8/dc/l9e 0 2024-05-06T12:14:50.924 INFO:tasks.workunit.client.1.smithi190.stdout:0/600: mkdir d9/dc/d28/dc5/dd5 0 2024-05-06T12:14:50.925 INFO:tasks.workunit.client.1.smithi190.stdout:4/430: write d2/d11/d16/f1a [1172782,86972] 0 2024-05-06T12:14:50.927 INFO:tasks.workunit.client.1.smithi190.stdout:7/572: rename d0/d4c to d0/da/d36/d53/d77/d8c/dd1 0 2024-05-06T12:14:50.928 INFO:tasks.workunit.client.1.smithi190.stdout:5/623: truncate d2/da/f4a 3160750 0 2024-05-06T12:14:50.928 INFO:tasks.workunit.client.1.smithi190.stdout:7/573: read d0/d14/f7b [838321,53199] 0 2024-05-06T12:14:50.929 INFO:tasks.workunit.client.1.smithi190.stdout:7/574: stat d0/d14/lad 0 2024-05-06T12:14:50.931 INFO:tasks.workunit.client.1.smithi190.stdout:1/455: mknod d5/d15/ca4 0 2024-05-06T12:14:50.934 INFO:tasks.workunit.client.1.smithi190.stdout:8/651: mknod d3/d8/d2b/ce0 0 2024-05-06T12:14:50.936 INFO:tasks.workunit.client.1.smithi190.stdout:6/448: fdatasync df/d17/f26 0 2024-05-06T12:14:50.938 INFO:tasks.workunit.client.1.smithi190.stdout:3/643: symlink d5/d7/d15/d1f/le4 0 2024-05-06T12:14:50.940 INFO:tasks.workunit.client.1.smithi190.stdout:7/575: mkdir d0/d1/d18/dd2 0 2024-05-06T12:14:50.942 INFO:tasks.workunit.client.1.smithi190.stdout:1/456: write d5/f12 [3608888,19475] 0 2024-05-06T12:14:50.943 INFO:tasks.workunit.client.1.smithi190.stdout:1/457: fdatasync d5/fa 0 2024-05-06T12:14:50.943 INFO:tasks.workunit.client.1.smithi190.stdout:9/667: creat db/dd/d16/d4c/d4e/fe1 x:0 0 0 2024-05-06T12:14:50.944 INFO:tasks.workunit.client.1.smithi190.stdout:2/515: mkdir d2/d8/d19/da9 0 2024-05-06T12:14:50.945 INFO:tasks.workunit.client.1.smithi190.stdout:2/516: chown d2/d21/f42 839 1 2024-05-06T12:14:50.946 INFO:tasks.workunit.client.1.smithi190.stdout:8/652: rename d3/d8/d2b/ldc to d3/d8/d25/d65/dc6/le1 0 2024-05-06T12:14:50.946 INFO:tasks.workunit.client.1.smithi190.stdout:0/601: write d9/dc/d48/fce [2505392,41715] 0 2024-05-06T12:14:50.948 INFO:tasks.workunit.client.1.smithi190.stdout:4/431: link d2/f1e d2/d11/d16/d61/d82/f98 0 2024-05-06T12:14:50.950 INFO:tasks.workunit.client.1.smithi190.stdout:6/449: unlink df/d2c/d61/d87/f9e 0 2024-05-06T12:14:50.954 INFO:tasks.workunit.client.1.smithi190.stdout:7/576: creat d0/d1/d2/d84/fd3 x:0 0 0 2024-05-06T12:14:50.959 INFO:tasks.workunit.client.1.smithi190.stdout:6/450: dwrite df/f47 [0,4194304] 0 2024-05-06T12:14:50.962 INFO:tasks.workunit.client.1.smithi190.stdout:6/451: write df/f15 [734787,99820] 0 2024-05-06T12:14:50.962 INFO:tasks.workunit.client.1.smithi190.stdout:8/653: creat d3/d8/d2b/d3e/fe2 x:0 0 0 2024-05-06T12:14:50.963 INFO:tasks.workunit.client.1.smithi190.stdout:2/517: dread d2/f1f [0,4194304] 0 2024-05-06T12:14:50.965 INFO:tasks.workunit.client.1.smithi190.stdout:0/602: fdatasync d9/dc/d28/d4c/f9b 0 2024-05-06T12:14:50.967 INFO:tasks.workunit.client.1.smithi190.stdout:4/432: creat d2/d10/f99 x:0 0 0 2024-05-06T12:14:50.969 INFO:tasks.workunit.client.1.smithi190.stdout:5/624: creat d2/da/d59/d5f/fca x:0 0 0 2024-05-06T12:14:50.970 INFO:tasks.workunit.client.1.smithi190.stdout:1/458: dwrite d5/d4f/d65/d32/d47/f4c [0,4194304] 0 2024-05-06T12:14:50.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:50 smithi092 ceph-mon[26954]: pgmap v27: 65 pgs: 65 active+clean; 2.9 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 107 MiB/s rd, 288 MiB/s wr, 597 op/s 2024-05-06T12:14:50.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:50 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:50.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:50 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:50.976 INFO:tasks.workunit.client.1.smithi190.stdout:9/668: unlink db/dd/d16/d58/d87/d22/f51 0 2024-05-06T12:14:50.978 INFO:tasks.workunit.client.1.smithi190.stdout:1/459: write d5/d4f/d65/d32/d47/d53/d6e/f84 [421100,1514] 0 2024-05-06T12:14:50.980 INFO:tasks.workunit.client.1.smithi190.stdout:3/644: dwrite d5/d14/d11/d21/f30 [0,4194304] 0 2024-05-06T12:14:50.982 INFO:tasks.workunit.client.1.smithi190.stdout:8/654: creat d3/d8/d25/d3b/d6b/d7a/fe3 x:0 0 0 2024-05-06T12:14:50.984 INFO:tasks.workunit.client.1.smithi190.stdout:2/518: unlink d2/d8/d19/d4f/fa2 0 2024-05-06T12:14:50.985 INFO:tasks.workunit.client.1.smithi190.stdout:4/433: dwrite d2/d11/d16/d61/d66/d6b/f88 [0,4194304] 0 2024-05-06T12:14:50.986 INFO:tasks.workunit.client.1.smithi190.stdout:2/519: write d2/f62 [227169,68722] 0 2024-05-06T12:14:50.993 INFO:tasks.workunit.client.1.smithi190.stdout:7/577: link d0/d1/fbe d0/d1/d18/dd2/fd4 0 2024-05-06T12:14:50.996 INFO:tasks.workunit.client.1.smithi190.stdout:7/578: truncate d0/d1/d18/d2b/d5a/fa1 4762303 0 2024-05-06T12:14:50.998 INFO:tasks.workunit.client.1.smithi190.stdout:9/669: fdatasync db/dd/d16/d58/d87/d22/f55 0 2024-05-06T12:14:51.000 INFO:tasks.workunit.client.1.smithi190.stdout:4/434: creat d2/d11/d16/d8e/d80/f9a x:0 0 0 2024-05-06T12:14:51.001 INFO:tasks.workunit.client.1.smithi190.stdout:4/435: write d2/d11/d16/f90 [38285,94332] 0 2024-05-06T12:14:51.002 INFO:tasks.workunit.client.1.smithi190.stdout:0/603: rename d9/dc/d28/d4c/d96/cc8 to d9/dc/d28/d4c/d80/da7/cd6 0 2024-05-06T12:14:51.004 INFO:tasks.workunit.client.1.smithi190.stdout:4/436: symlink d2/d10/l9b 0 2024-05-06T12:14:51.008 INFO:tasks.workunit.client.1.smithi190.stdout:7/579: unlink d0/d1/f21 0 2024-05-06T12:14:51.011 INFO:tasks.workunit.client.1.smithi190.stdout:0/604: link d9/dcf/d21/f2a d9/dcf/d56/dab/fd7 0 2024-05-06T12:14:51.012 INFO:tasks.workunit.client.1.smithi190.stdout:4/437: dwrite d2/d11/d16/f90 [0,4194304] 0 2024-05-06T12:14:51.013 INFO:tasks.workunit.client.1.smithi190.stdout:5/625: rename d2/da/d59/c93 to d2/da/d59/d5f/d50/ccb 0 2024-05-06T12:14:51.017 INFO:tasks.workunit.client.1.smithi190.stdout:0/605: creat d9/dc/dd0/fd8 x:0 0 0 2024-05-06T12:14:51.018 INFO:tasks.workunit.client.1.smithi190.stdout:4/438: readlink d2/d11/d16/d41/l46 0 2024-05-06T12:14:51.020 INFO:tasks.workunit.client.1.smithi190.stdout:2/520: rename d2/d8/dc/c31 to d2/d8/d19/d7b/da0/caa 0 2024-05-06T12:14:51.023 INFO:tasks.workunit.client.1.smithi190.stdout:4/439: chown d2/d10/f58 17196 1 2024-05-06T12:14:51.027 INFO:tasks.workunit.client.1.smithi190.stdout:7/580: dwrite d0/da/d36/d53/f9b [0,4194304] 0 2024-05-06T12:14:51.028 INFO:tasks.workunit.client.1.smithi190.stdout:3/645: truncate d5/d14/d11/d21/d44/d80/d82/fbb 1979414 0 2024-05-06T12:14:51.030 INFO:tasks.workunit.client.1.smithi190.stdout:9/670: rename db/dd/d16/d58/d87/f70 to db/dd/d16/d4c/fe2 0 2024-05-06T12:14:51.033 INFO:tasks.workunit.client.1.smithi190.stdout:5/626: dwrite d2/db6/fc9 [0,4194304] 0 2024-05-06T12:14:51.035 INFO:tasks.workunit.client.1.smithi190.stdout:1/460: dwrite d5/d15/d1e/f5a [0,4194304] 0 2024-05-06T12:14:51.036 INFO:tasks.workunit.client.1.smithi190.stdout:3/646: write d5/d7/d15/f28 [4953489,125727] 0 2024-05-06T12:14:51.036 INFO:tasks.workunit.client.1.smithi190.stdout:2/521: rename d2/d8/d33/d46/d55 to d2/d8/d33/d46/dab 0 2024-05-06T12:14:51.038 INFO:tasks.workunit.client.1.smithi190.stdout:1/461: chown d5/d6b/l97 5 1 2024-05-06T12:14:51.041 INFO:tasks.workunit.client.1.smithi190.stdout:2/522: chown d2/d8/d19/d43/d79 2795740 1 2024-05-06T12:14:51.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:50 smithi190 ceph-mon[33798]: pgmap v27: 65 pgs: 65 active+clean; 2.9 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 107 MiB/s rd, 288 MiB/s wr, 597 op/s 2024-05-06T12:14:51.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:50 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:51.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:50 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:51.109 INFO:tasks.workunit.client.1.smithi190.stdout:1/462: fsync d5/d4f/d65/d24/d3a/f72 0 2024-05-06T12:14:51.109 INFO:tasks.workunit.client.1.smithi190.stdout:7/581: creat d0/d1/d18/d4e/db3/fd5 x:0 0 0 2024-05-06T12:14:51.111 INFO:tasks.workunit.client.1.smithi190.stdout:4/440: dwrite d2/d11/d16/d41/f62 [0,4194304] 0 2024-05-06T12:14:51.113 INFO:tasks.workunit.client.1.smithi190.stdout:3/647: write d5/d14/d11/d21/d44/d80/f83 [1301363,15275] 0 2024-05-06T12:14:51.114 INFO:tasks.workunit.client.1.smithi190.stdout:2/523: mkdir d2/d8/d33/d46/dab/dac 0 2024-05-06T12:14:51.114 INFO:tasks.workunit.client.1.smithi190.stdout:7/582: creat d0/d1/d32/d45/d70/fd6 x:0 0 0 2024-05-06T12:14:51.116 INFO:tasks.workunit.client.1.smithi190.stdout:1/463: chown d5/d4f/d65/d32/d47/d53/c5d 80698 1 2024-05-06T12:14:51.116 INFO:tasks.workunit.client.1.smithi190.stdout:3/648: chown d5/d7/d15/d1f/d33/lb5 211344 1 2024-05-06T12:14:51.117 INFO:tasks.workunit.client.1.smithi190.stdout:0/606: dread d9/dc/d28/d4c/d96/fa0 [0,4194304] 0 2024-05-06T12:14:51.117 INFO:tasks.workunit.client.1.smithi190.stdout:7/583: write d0/d1/d32/d45/d70/fd6 [543824,16699] 0 2024-05-06T12:14:51.118 INFO:tasks.workunit.client.1.smithi190.stdout:9/671: link db/dd/d16/d58/ddc/d47/d4b/f66 db/dd/d16/d58/d87/d22/d49/d88/db5/fe3 0 2024-05-06T12:14:51.118 INFO:tasks.workunit.client.1.smithi190.stdout:4/441: symlink d2/d11/d16/d8e/d80/l9c 0 2024-05-06T12:14:51.123 INFO:tasks.workunit.client.1.smithi190.stdout:3/649: creat d5/d7/d15/d1f/d84/fe5 x:0 0 0 2024-05-06T12:14:51.123 INFO:tasks.workunit.client.1.smithi190.stdout:0/607: truncate d9/d6f/f7a 2390414 0 2024-05-06T12:14:51.124 INFO:tasks.workunit.client.1.smithi190.stdout:9/672: mkdir db/dd/d16/d58/ddc/d47/de4 0 2024-05-06T12:14:51.126 INFO:tasks.workunit.client.1.smithi190.stdout:1/464: dread d5/d4f/d65/d24/f2c [0,4194304] 0 2024-05-06T12:14:51.131 INFO:tasks.workunit.client.1.smithi190.stdout:4/442: symlink d2/d11/d16/d61/d66/d6b/l9d 0 2024-05-06T12:14:51.131 INFO:tasks.workunit.client.1.smithi190.stdout:0/608: unlink d9/dc/d28/d4c/c79 0 2024-05-06T12:14:51.131 INFO:tasks.workunit.client.1.smithi190.stdout:7/584: dwrite d0/d1/d18/fa2 [0,4194304] 0 2024-05-06T12:14:51.132 INFO:tasks.workunit.client.1.smithi190.stdout:3/650: creat d5/d14/d11/d21/d44/d92/fe6 x:0 0 0 2024-05-06T12:14:51.133 INFO:tasks.workunit.client.1.smithi190.stdout:2/524: write d2/d4c/f5a [1648865,114134] 0 2024-05-06T12:14:51.134 INFO:tasks.workunit.client.1.smithi190.stdout:2/525: read - d2/d21/f42 zero size 2024-05-06T12:14:51.135 INFO:tasks.workunit.client.1.smithi190.stdout:2/526: dread - d2/f6a zero size 2024-05-06T12:14:51.137 INFO:tasks.workunit.client.1.smithi190.stdout:4/443: dread d2/d11/d16/f2f [0,4194304] 0 2024-05-06T12:14:51.141 INFO:tasks.workunit.client.1.smithi190.stdout:1/465: symlink d5/d15/la5 0 2024-05-06T12:14:51.143 INFO:tasks.workunit.client.1.smithi190.stdout:9/673: creat db/dd/fe5 x:0 0 0 2024-05-06T12:14:51.144 INFO:tasks.workunit.client.1.smithi190.stdout:2/527: creat d2/d8/dc/fad x:0 0 0 2024-05-06T12:14:51.145 INFO:tasks.workunit.client.1.smithi190.stdout:4/444: readlink d2/l2c 0 2024-05-06T12:14:51.146 INFO:tasks.workunit.client.1.smithi190.stdout:4/445: dread - d2/d11/d16/d4e/f85 zero size 2024-05-06T12:14:51.149 INFO:tasks.workunit.client.1.smithi190.stdout:1/466: mkdir d5/d4f/d65/d24/d43/da6 0 2024-05-06T12:14:51.149 INFO:tasks.workunit.client.1.smithi190.stdout:4/446: truncate d2/d11/d16/d61/d67/d6e/d78/f7a 1679626 0 2024-05-06T12:14:51.154 INFO:tasks.workunit.client.1.smithi190.stdout:0/609: creat d9/dcf/d21/fd9 x:0 0 0 2024-05-06T12:14:51.154 INFO:tasks.workunit.client.1.smithi190.stdout:7/585: truncate d0/f40 2913880 0 2024-05-06T12:14:51.156 INFO:tasks.workunit.client.1.smithi190.stdout:3/651: write d5/d14/d11/d21/f38 [679341,77680] 0 2024-05-06T12:14:51.200 INFO:tasks.workunit.client.1.smithi190.stdout:4/447: rename d2/d11/d16/d61/d67 to d2/d11/d16/d61/d9e 0 2024-05-06T12:14:51.200 INFO:tasks.workunit.client.1.smithi190.stdout:2/528: write d2/f12 [267762,128732] 0 2024-05-06T12:14:51.201 INFO:tasks.workunit.client.1.smithi190.stdout:7/586: symlink d0/d14/d6a/ld7 0 2024-05-06T12:14:51.201 INFO:tasks.workunit.client.1.smithi190.stdout:0/610: write d9/d51/f9d [3017658,47584] 0 2024-05-06T12:14:51.204 INFO:tasks.workunit.client.1.smithi190.stdout:3/652: rename d5/d14/d11/d21/c2d to d5/d14/ce7 0 2024-05-06T12:14:51.205 INFO:tasks.workunit.client.1.smithi190.stdout:0/611: chown d9/d51/dc9/f8d 18 1 2024-05-06T12:14:51.205 INFO:tasks.workunit.client.1.smithi190.stdout:2/529: creat d2/d8/d19/d43/fae x:0 0 0 2024-05-06T12:14:51.207 INFO:tasks.workunit.client.1.smithi190.stdout:2/530: readlink d2/d10/l77 0 2024-05-06T12:14:51.207 INFO:tasks.workunit.client.1.smithi190.stdout:7/587: unlink d0/d1/d32/d45/d70/d35/fa4 0 2024-05-06T12:14:51.207 INFO:tasks.workunit.client.1.smithi190.stdout:9/674: dwrite db/d23/f5b [4194304,4194304] 0 2024-05-06T12:14:51.208 INFO:tasks.workunit.client.1.smithi190.stdout:2/531: chown d2/l9 10357992 1 2024-05-06T12:14:51.209 INFO:tasks.workunit.client.1.smithi190.stdout:4/448: rename d2/f1e to d2/d11/d16/d61/d66/d6b/f9f 0 2024-05-06T12:14:51.214 INFO:tasks.workunit.client.1.smithi190.stdout:4/449: mkdir d2/d11/d16/d8e/d80/da0 0 2024-05-06T12:14:51.216 INFO:tasks.workunit.client.1.smithi190.stdout:3/653: mknod d5/d7/d15/d39/dbc/ce8 0 2024-05-06T12:14:51.218 INFO:tasks.workunit.client.1.smithi190.stdout:7/588: truncate d0/d4/f7 1926520 0 2024-05-06T12:14:51.219 INFO:tasks.workunit.client.1.smithi190.stdout:7/589: fsync d0/d4/fd 0 2024-05-06T12:14:51.221 INFO:tasks.workunit.client.1.smithi190.stdout:7/590: truncate d0/da/d36/d53/f9b 4328592 0 2024-05-06T12:14:51.221 INFO:tasks.workunit.client.1.smithi190.stdout:4/450: rmdir d2/d11/d16/d8e/d7b 39 2024-05-06T12:14:51.224 INFO:tasks.workunit.client.1.smithi190.stdout:3/654: symlink d5/d14/d11/d21/d44/le9 0 2024-05-06T12:14:51.225 INFO:tasks.workunit.client.1.smithi190.stdout:3/655: fsync d5/d7/d15/d1f/f96 0 2024-05-06T12:14:51.226 INFO:tasks.workunit.client.1.smithi190.stdout:0/612: rename d9/dcf/d56/f68 to d9/fda 0 2024-05-06T12:14:51.232 INFO:tasks.workunit.client.1.smithi190.stdout:3/656: dwrite d5/d7/d15/d39/f9b [0,4194304] 0 2024-05-06T12:14:51.236 INFO:tasks.workunit.client.1.smithi190.stdout:2/532: rename d2/d8/d33/d46/dab/d91/ca4 to d2/d8/d33/caf 0 2024-05-06T12:14:51.236 INFO:tasks.workunit.client.1.smithi190.stdout:9/675: dwrite db/dd/d16/d58/d87/d22/d49/d88/faf [0,4194304] 0 2024-05-06T12:14:51.236 INFO:tasks.workunit.client.1.smithi190.stdout:0/613: symlink d9/d51/ldb 0 2024-05-06T12:14:51.238 INFO:tasks.workunit.client.1.smithi190.stdout:2/533: readlink d2/d21/la1 0 2024-05-06T12:14:51.239 INFO:tasks.workunit.client.1.smithi190.stdout:7/591: mkdir d0/d1/d32/dd8 0 2024-05-06T12:14:51.247 INFO:tasks.workunit.client.1.smithi190.stdout:4/451: dread d2/d10/f58 [0,4194304] 0 2024-05-06T12:14:51.250 INFO:tasks.workunit.client.1.smithi190.stdout:2/534: dwrite d2/d10/f2c [0,4194304] 0 2024-05-06T12:14:51.251 INFO:tasks.workunit.client.1.smithi190.stdout:4/452: dread d2/f64 [0,4194304] 0 2024-05-06T12:14:51.251 INFO:tasks.workunit.client.1.smithi190.stdout:9/676: mknod db/dd/d16/d58/d87/d99/ce6 0 2024-05-06T12:14:51.252 INFO:tasks.workunit.client.1.smithi190.stdout:0/614: dwrite d9/dc/d48/fce [0,4194304] 0 2024-05-06T12:14:51.252 INFO:tasks.workunit.client.1.smithi190.stdout:7/592: fsync d0/da/d36/f43 0 2024-05-06T12:14:51.255 INFO:tasks.workunit.client.1.smithi190.stdout:3/657: rename d5/d7/d15/d39/f55 to d5/d14/d11/d21/d44/d80/d82/fea 0 2024-05-06T12:14:51.255 INFO:tasks.workunit.client.1.smithi190.stdout:7/593: read d0/d1/d32/d45/d70/fd6 [78885,87891] 0 2024-05-06T12:14:51.257 INFO:tasks.workunit.client.1.smithi190.stdout:4/453: creat d2/d11/d16/d4e/fa1 x:0 0 0 2024-05-06T12:14:51.263 INFO:tasks.workunit.client.1.smithi190.stdout:0/615: write d9/dc/d28/d4c/d80/da7/fc7 [947978,18057] 0 2024-05-06T12:14:51.264 INFO:tasks.workunit.client.1.smithi190.stdout:0/616: chown c4 358536828 1 2024-05-06T12:14:51.267 INFO:tasks.workunit.client.1.smithi190.stdout:4/454: dwrite d2/d11/d16/d61/d9e/d6e/f72 [0,4194304] 0 2024-05-06T12:14:51.268 INFO:tasks.workunit.client.1.smithi190.stdout:0/617: write d9/dcf/d56/dab/fba [1285697,7586] 0 2024-05-06T12:14:51.275 INFO:tasks.workunit.client.1.smithi190.stdout:4/455: dread d2/d11/d16/f90 [0,4194304] 0 2024-05-06T12:14:51.277 INFO:tasks.workunit.client.1.smithi190.stdout:3/658: rmdir d5/d14/d11/d21/d44/d80 39 2024-05-06T12:14:51.279 INFO:tasks.workunit.client.1.smithi190.stdout:0/618: rename d9/c3d to d9/dc/d48/d50/d83/dc6/cdc 0 2024-05-06T12:14:51.282 INFO:tasks.workunit.client.1.smithi190.stdout:3/659: truncate d5/d14/d11/fc7 4852842 0 2024-05-06T12:14:51.284 INFO:tasks.workunit.client.1.smithi190.stdout:0/619: write d9/dc/dd0/fd8 [767810,97252] 0 2024-05-06T12:14:51.285 INFO:tasks.workunit.client.1.smithi190.stdout:7/594: getdents d0/d1/d32/d45/d70/d33 0 2024-05-06T12:14:51.287 INFO:tasks.workunit.client.1.smithi190.stdout:2/535: dwrite d2/d8/dc/f13 [0,4194304] 0 2024-05-06T12:14:51.287 INFO:tasks.workunit.client.1.smithi190.stdout:4/456: rename d2/d11/f2e to d2/d11/d16/d61/d82/fa2 0 2024-05-06T12:14:51.290 INFO:tasks.workunit.client.1.smithi190.stdout:9/677: dwrite db/dd/d16/d4c/d4e/f9d [0,4194304] 0 2024-05-06T12:14:51.292 INFO:tasks.workunit.client.1.smithi190.stdout:3/660: mkdir d5/d14/d11/d21/deb 0 2024-05-06T12:14:51.296 INFO:tasks.workunit.client.1.smithi190.stdout:2/536: chown d2/d10/f87 230 1 2024-05-06T12:14:51.297 INFO:tasks.workunit.client.1.smithi190.stdout:3/661: truncate d5/d7/d15/d1f/f96 387623 0 2024-05-06T12:14:51.298 INFO:tasks.workunit.client.1.smithi190.stdout:0/620: rename d9/l43 to d9/dc/d48/d50/d87/ldd 0 2024-05-06T12:14:51.299 INFO:tasks.workunit.client.1.smithi190.stdout:7/595: dwrite d0/d1/d32/daa/fb1 [0,4194304] 0 2024-05-06T12:14:51.299 INFO:tasks.workunit.client.1.smithi190.stdout:2/537: fsync d2/d8/d71/f8e 0 2024-05-06T12:14:51.301 INFO:tasks.workunit.client.1.smithi190.stdout:2/538: fdatasync d2/d4c/f75 0 2024-05-06T12:14:51.301 INFO:tasks.workunit.client.1.smithi190.stdout:0/621: write d9/dc/d48/fcb [873666,109560] 0 2024-05-06T12:14:51.306 INFO:tasks.workunit.client.1.smithi190.stdout:4/457: getdents d2/d11/d16/d8e/d80/da0 0 2024-05-06T12:14:51.307 INFO:tasks.workunit.client.1.smithi190.stdout:3/662: symlink d5/d7/d6d/dc6/lec 0 2024-05-06T12:14:51.314 INFO:tasks.workunit.client.1.smithi190.stdout:9/678: getdents db/dd/d16/d4c 0 2024-05-06T12:14:51.315 INFO:tasks.workunit.client.1.smithi190.stdout:4/458: dwrite d2/d11/f14 [0,4194304] 0 2024-05-06T12:14:51.317 INFO:tasks.workunit.client.1.smithi190.stdout:3/663: rename d5/d7/d15/d1f to d5/d14/d11/d21/d44/d80/d82/d3d/ded 0 2024-05-06T12:14:51.319 INFO:tasks.workunit.client.1.smithi190.stdout:2/539: link d2/d8/d19/d43/d79/l98 d2/d21/d50/lb0 0 2024-05-06T12:14:51.321 INFO:tasks.workunit.client.1.smithi190.stdout:9/679: chown db/dd/d16/d4c/d4e/ca3 2671838 1 2024-05-06T12:14:51.325 INFO:tasks.workunit.client.1.smithi190.stdout:2/540: rename d2/f6a to d2/d8/d19/d43/fb1 0 2024-05-06T12:14:51.326 INFO:tasks.workunit.client.1.smithi190.stdout:7/596: write d0/da/d36/d53/d77/d8c/dd1/d60/fc2 [932663,80338] 0 2024-05-06T12:14:51.327 INFO:tasks.workunit.client.1.smithi190.stdout:0/622: write d9/d51/f55 [531988,95736] 0 2024-05-06T12:14:51.328 INFO:tasks.workunit.client.1.smithi190.stdout:7/597: fsync d0/d1/d32/daa/fb1 0 2024-05-06T12:14:51.333 INFO:tasks.workunit.client.1.smithi190.stdout:0/623: fdatasync d9/dc/d48/d50/d83/f94 0 2024-05-06T12:14:51.334 INFO:tasks.workunit.client.1.smithi190.stdout:0/624: chown d9/dc/d28/d77/la8 54 1 2024-05-06T12:14:51.335 INFO:tasks.workunit.client.1.smithi190.stdout:7/598: creat d0/d14/d6a/fd9 x:0 0 0 2024-05-06T12:14:51.336 INFO:tasks.workunit.client.1.smithi190.stdout:4/459: getdents d2/d11/d16/d61/d66/d6b 0 2024-05-06T12:14:51.338 INFO:tasks.workunit.client.1.smithi190.stdout:3/664: creat d5/d14/d11/d21/d44/d92/d9e/fee x:0 0 0 2024-05-06T12:14:51.340 INFO:tasks.workunit.client.1.smithi190.stdout:2/541: dwrite d2/fd [4194304,4194304] 0 2024-05-06T12:14:51.344 INFO:tasks.workunit.client.1.smithi190.stdout:7/599: rmdir d0/d1 39 2024-05-06T12:14:51.348 INFO:tasks.workunit.client.1.smithi190.stdout:3/665: dwrite d5/d7/d6d/fc4 [0,4194304] 0 2024-05-06T12:14:51.349 INFO:tasks.workunit.client.1.smithi190.stdout:3/666: fdatasync d5/d7/fb9 0 2024-05-06T12:14:51.349 INFO:tasks.workunit.client.1.smithi190.stdout:3/667: chown d5/d14/d11/d21/d44/d80/ddb 1718135 1 2024-05-06T12:14:51.350 INFO:tasks.workunit.client.1.smithi190.stdout:9/680: rename db/d23/c36 to db/dd/ce7 0 2024-05-06T12:14:51.351 INFO:tasks.workunit.client.1.smithi190.stdout:4/460: symlink d2/d11/d16/d8e/la3 0 2024-05-06T12:14:51.356 INFO:tasks.workunit.client.1.smithi190.stdout:5/627: sync 2024-05-06T12:14:51.356 INFO:tasks.workunit.client.1.smithi190.stdout:1/467: sync 2024-05-06T12:14:51.356 INFO:tasks.workunit.client.1.smithi190.stdout:6/452: sync 2024-05-06T12:14:51.356 INFO:tasks.workunit.client.1.smithi190.stdout:8/655: sync 2024-05-06T12:14:51.360 INFO:tasks.workunit.client.1.smithi190.stdout:2/542: rename d2/d21/d50 to d2/d8/d19/d43/d79/db2 0 2024-05-06T12:14:51.360 INFO:tasks.workunit.client.1.smithi190.stdout:1/468: chown d5/d4f/d65/d2b/c7c 0 1 2024-05-06T12:14:51.362 INFO:tasks.workunit.client.1.smithi190.stdout:4/461: mkdir d2/d11/d16/d61/d66/d6b/da4 0 2024-05-06T12:14:51.362 INFO:tasks.workunit.client.1.smithi190.stdout:1/469: dread - d5/d4f/d65/d32/d47/d53/d8b/f9b zero size 2024-05-06T12:14:51.364 INFO:tasks.workunit.client.1.smithi190.stdout:6/453: creat df/d2c/d28/f9f x:0 0 0 2024-05-06T12:14:51.365 INFO:tasks.workunit.client.1.smithi190.stdout:5/628: rmdir d2/da/d59/d5f/d2e 39 2024-05-06T12:14:51.365 INFO:tasks.workunit.client.1.smithi190.stdout:2/543: read - d2/d8/d19/f45 zero size 2024-05-06T12:14:51.367 INFO:tasks.workunit.client.1.smithi190.stdout:1/470: truncate d5/d4f/d65/d32/d47/d53/d6e/f86 928983 0 2024-05-06T12:14:51.367 INFO:tasks.workunit.client.1.smithi190.stdout:6/454: chown df/d2c/d1e/f52 1084740367 1 2024-05-06T12:14:51.367 INFO:tasks.workunit.client.1.smithi190.stdout:1/471: fdatasync d5/f34 0 2024-05-06T12:14:51.369 INFO:tasks.workunit.client.1.smithi190.stdout:0/625: dread d9/dc/d48/f4e [4194304,4194304] 0 2024-05-06T12:14:51.369 INFO:tasks.workunit.client.1.smithi190.stdout:8/656: creat d3/d8/d68/dde/fe4 x:0 0 0 2024-05-06T12:14:51.371 INFO:tasks.workunit.client.1.smithi190.stdout:6/455: truncate df/d2c/d92/f93 40098 0 2024-05-06T12:14:51.371 INFO:tasks.workunit.client.1.smithi190.stdout:1/472: dread - d5/d4f/d65/d32/d47/d53/d8b/f9b zero size 2024-05-06T12:14:51.372 INFO:tasks.workunit.client.1.smithi190.stdout:8/657: fdatasync d3/d13/fc8 0 2024-05-06T12:14:51.372 INFO:tasks.workunit.client.1.smithi190.stdout:4/462: dread d2/d11/d16/f1f [0,4194304] 0 2024-05-06T12:14:51.373 INFO:tasks.workunit.client.1.smithi190.stdout:1/473: stat d5/d15/la5 0 2024-05-06T12:14:51.373 INFO:tasks.workunit.client.1.smithi190.stdout:3/668: link d5/c60 d5/d14/d11/d21/d44/d92/cef 0 2024-05-06T12:14:51.375 INFO:tasks.workunit.client.1.smithi190.stdout:6/456: write df/d2c/d28/f2a [296536,89960] 0 2024-05-06T12:14:51.375 INFO:tasks.workunit.client.1.smithi190.stdout:2/544: unlink d2/d8/d33/l64 0 2024-05-06T12:14:51.379 INFO:tasks.workunit.client.1.smithi190.stdout:1/474: dread - d5/d15/f7a zero size 2024-05-06T12:14:51.380 INFO:tasks.workunit.client.1.smithi190.stdout:1/475: write d5/d15/fa2 [45596,92186] 0 2024-05-06T12:14:51.380 INFO:tasks.workunit.client.1.smithi190.stdout:9/681: dread db/d23/f83 [0,4194304] 0 2024-05-06T12:14:51.381 INFO:tasks.workunit.client.1.smithi190.stdout:3/669: mknod d5/d7/d7f/cf0 0 2024-05-06T12:14:51.383 INFO:tasks.workunit.client.1.smithi190.stdout:9/682: chown db/dd/d16/d4c/d4e/d5e 1 1 2024-05-06T12:14:51.385 INFO:tasks.workunit.client.1.smithi190.stdout:2/545: unlink d2/fa 0 2024-05-06T12:14:51.387 INFO:tasks.workunit.client.1.smithi190.stdout:2/546: write d2/d8/d33/d46/d5d/f93 [1258894,76706] 0 2024-05-06T12:14:51.391 INFO:tasks.workunit.client.1.smithi190.stdout:9/683: truncate db/dd/d16/d58/ddc/d47/fbf 1300661 0 2024-05-06T12:14:51.393 INFO:tasks.workunit.client.1.smithi190.stdout:0/626: link d9/dc/l60 d9/dc/d48/d50/lde 0 2024-05-06T12:14:51.394 INFO:tasks.workunit.client.1.smithi190.stdout:8/658: dread d3/d8/d25/d3b/d90/f7c [0,4194304] 0 2024-05-06T12:14:51.395 INFO:tasks.workunit.client.1.smithi190.stdout:6/457: symlink df/d2c/d1e/d57/d89/la0 0 2024-05-06T12:14:51.397 INFO:tasks.workunit.client.1.smithi190.stdout:9/684: write db/d23/d7e/fa8 [2095710,9404] 0 2024-05-06T12:14:51.397 INFO:tasks.workunit.client.1.smithi190.stdout:3/670: getdents d5/d14/d11/d21/d44/d80/ddb 0 2024-05-06T12:14:51.400 INFO:tasks.workunit.client.1.smithi190.stdout:2/547: dwrite d2/d8/d33/d46/d5d/f93 [0,4194304] 0 2024-05-06T12:14:51.400 INFO:tasks.workunit.client.1.smithi190.stdout:0/627: symlink d9/dc/dd0/ldf 0 2024-05-06T12:14:51.401 INFO:tasks.workunit.client.1.smithi190.stdout:1/476: dwrite d5/d4f/d65/d24/f2c [0,4194304] 0 2024-05-06T12:14:51.401 INFO:tasks.workunit.client.1.smithi190.stdout:8/659: write d3/d8/f2c [4602264,61865] 0 2024-05-06T12:14:51.405 INFO:tasks.workunit.client.1.smithi190.stdout:5/629: truncate d2/da/d59/d5f/d2e/d3b/f42 487090 0 2024-05-06T12:14:51.406 INFO:tasks.workunit.client.1.smithi190.stdout:1/477: readlink d5/l4b 0 2024-05-06T12:14:51.410 INFO:tasks.workunit.client.1.smithi190.stdout:2/548: creat d2/d8/d19/d43/fb3 x:0 0 0 2024-05-06T12:14:51.413 INFO:tasks.workunit.client.1.smithi190.stdout:3/671: truncate d5/d14/d11/d21/d44/f5b 458253 0 2024-05-06T12:14:51.414 INFO:tasks.workunit.client.1.smithi190.stdout:9/685: mknod db/d23/ce8 0 2024-05-06T12:14:51.416 INFO:tasks.workunit.client.1.smithi190.stdout:8/660: creat d3/d8/d68/fe5 x:0 0 0 2024-05-06T12:14:51.419 INFO:tasks.workunit.client.1.smithi190.stdout:8/661: fdatasync d3/d13/f7d 0 2024-05-06T12:14:51.423 INFO:tasks.workunit.client.1.smithi190.stdout:3/672: mknod d5/d14/d11/d21/d44/d92/cf1 0 2024-05-06T12:14:51.423 INFO:tasks.workunit.client.1.smithi190.stdout:5/630: dwrite d2/d6c/f99 [0,4194304] 0 2024-05-06T12:14:51.425 INFO:tasks.workunit.client.1.smithi190.stdout:9/686: symlink db/dd/d5c/dac/le9 0 2024-05-06T12:14:51.430 INFO:tasks.workunit.client.1.smithi190.stdout:6/458: link df/f15 df/d2c/d1e/d57/d89/fa1 0 2024-05-06T12:14:51.431 INFO:tasks.workunit.client.1.smithi190.stdout:0/628: dwrite d9/dc/d28/dc5/f7f [0,4194304] 0 2024-05-06T12:14:51.433 INFO:tasks.workunit.client.1.smithi190.stdout:6/459: fsync df/d2c/d61/f79 0 2024-05-06T12:14:51.435 INFO:tasks.workunit.client.1.smithi190.stdout:1/478: mkdir d5/d4f/d65/d32/d47/d53/d98/da7 0 2024-05-06T12:14:51.436 INFO:tasks.workunit.client.1.smithi190.stdout:8/662: rmdir d3/d35 39 2024-05-06T12:14:51.436 INFO:tasks.workunit.client.1.smithi190.stdout:5/631: mknod d2/da/d59/d5f/d2e/da3/ccc 0 2024-05-06T12:14:51.438 INFO:tasks.workunit.client.1.smithi190.stdout:3/673: symlink d5/d14/d11/d21/d44/d80/lf2 0 2024-05-06T12:14:51.443 INFO:tasks.workunit.client.1.smithi190.stdout:9/687: creat db/d23/d7e/fea x:0 0 0 2024-05-06T12:14:51.443 INFO:tasks.workunit.client.1.smithi190.stdout:0/629: mknod d9/dc/d28/dc5/ce0 0 2024-05-06T12:14:51.448 INFO:tasks.workunit.client.1.smithi190.stdout:8/663: chown d3/d8/d25/d3b/la8 41081 1 2024-05-06T12:14:51.448 INFO:tasks.workunit.client.1.smithi190.stdout:5/632: dwrite d2/f60 [0,4194304] 0 2024-05-06T12:14:51.448 INFO:tasks.workunit.client.1.smithi190.stdout:0/630: chown d9/dcf/d21/f2a 3602464 1 2024-05-06T12:14:51.450 INFO:tasks.workunit.client.1.smithi190.stdout:3/674: write d5/d14/f8f [4294802,84637] 0 2024-05-06T12:14:51.450 INFO:tasks.workunit.client.1.smithi190.stdout:1/479: creat d5/d4f/d65/d2b/fa8 x:0 0 0 2024-05-06T12:14:51.452 INFO:tasks.workunit.client.1.smithi190.stdout:3/675: fdatasync d5/d7/d6d/f9d 0 2024-05-06T12:14:51.453 INFO:tasks.workunit.client.1.smithi190.stdout:9/688: mkdir db/dd/d5c/dac/deb 0 2024-05-06T12:14:51.454 INFO:tasks.workunit.client.1.smithi190.stdout:1/480: readlink d5/d15/la5 0 2024-05-06T12:14:51.458 INFO:tasks.workunit.client.1.smithi190.stdout:5/633: symlink d2/da/d59/lcd 0 2024-05-06T12:14:51.460 INFO:tasks.workunit.client.1.smithi190.stdout:8/664: dwrite d3/d8/d2b/d3e/dad/fbe [0,4194304] 0 2024-05-06T12:14:51.465 INFO:tasks.workunit.client.1.smithi190.stdout:8/665: read - d3/d8/d68/dde/fe4 zero size 2024-05-06T12:14:51.469 INFO:tasks.workunit.client.1.smithi190.stdout:2/549: truncate d2/d8/d33/f37 3834869 0 2024-05-06T12:14:51.472 INFO:tasks.workunit.client.1.smithi190.stdout:0/631: dread d9/dcf/f62 [0,4194304] 0 2024-05-06T12:14:51.473 INFO:tasks.workunit.client.1.smithi190.stdout:5/634: creat d2/da/d59/d5f/d2e/da7/d9d/fce x:0 0 0 2024-05-06T12:14:51.475 INFO:tasks.workunit.client.1.smithi190.stdout:3/676: dwrite d5/d7/f23 [0,4194304] 0 2024-05-06T12:14:51.477 INFO:tasks.workunit.client.1.smithi190.stdout:5/635: chown d2/da/d59/d5f/d2e/l29 21003 1 2024-05-06T12:14:51.479 INFO:tasks.workunit.client.1.smithi190.stdout:8/666: mknod d3/d8/d68/d9f/ce6 0 2024-05-06T12:14:51.481 INFO:tasks.workunit.client.1.smithi190.stdout:8/667: fdatasync d3/d8/d68/fd8 0 2024-05-06T12:14:51.483 INFO:tasks.workunit.client.1.smithi190.stdout:5/636: fdatasync d2/da/d59/d5f/d2e/da7/d9d/fa4 0 2024-05-06T12:14:51.485 INFO:tasks.workunit.client.1.smithi190.stdout:8/668: truncate d3/d8/d2b/d3e/f84 1093800 0 2024-05-06T12:14:51.485 INFO:tasks.workunit.client.1.smithi190.stdout:5/637: fsync d2/f60 0 2024-05-06T12:14:51.485 INFO:tasks.workunit.client.1.smithi190.stdout:1/481: creat d5/d4f/d65/fa9 x:0 0 0 2024-05-06T12:14:51.486 INFO:tasks.workunit.client.1.smithi190.stdout:6/460: write df/d17/d44/f54 [951826,116486] 0 2024-05-06T12:14:51.487 INFO:tasks.workunit.client.1.smithi190.stdout:1/482: chown d5/d4f/d65/d32/d47/d53 4147770 1 2024-05-06T12:14:51.489 INFO:tasks.workunit.client.1.smithi190.stdout:6/461: truncate df/d2c/d1e/d57/d53/f6f 194611 0 2024-05-06T12:14:51.496 INFO:tasks.workunit.client.1.smithi190.stdout:9/689: dwrite db/dd/d16/d58/d87/d22/f56 [0,4194304] 0 2024-05-06T12:14:51.502 INFO:tasks.workunit.client.1.smithi190.stdout:6/462: creat df/d2c/d1e/d57/d89/fa2 x:0 0 0 2024-05-06T12:14:51.505 INFO:tasks.workunit.client.1.smithi190.stdout:8/669: symlink d3/d35/d55/d61/le7 0 2024-05-06T12:14:51.505 INFO:tasks.workunit.client.1.smithi190.stdout:6/463: dread - df/d2c/d92/f9a zero size 2024-05-06T12:14:51.506 INFO:tasks.workunit.client.1.smithi190.stdout:0/632: link d9/dcf/f33 d9/dc/d48/fe1 0 2024-05-06T12:14:51.509 INFO:tasks.workunit.client.1.smithi190.stdout:5/638: creat d2/d65/d83/fcf x:0 0 0 2024-05-06T12:14:51.511 INFO:tasks.workunit.client.1.smithi190.stdout:1/483: dwrite d5/d4f/d65/d24/d3a/f73 [0,4194304] 0 2024-05-06T12:14:51.513 INFO:tasks.workunit.client.1.smithi190.stdout:2/550: dwrite d2/d8/d19/d43/d79/d7e/f63 [0,4194304] 0 2024-05-06T12:14:51.514 INFO:tasks.workunit.client.1.smithi190.stdout:0/633: dread d9/dc/f6c [0,4194304] 0 2024-05-06T12:14:51.514 INFO:tasks.workunit.client.1.smithi190.stdout:3/677: truncate d5/d14/d11/d21/d44/d80/d82/f8b 790390 0 2024-05-06T12:14:51.517 INFO:tasks.workunit.client.1.smithi190.stdout:8/670: creat d3/d8/d2b/d3e/dad/fe8 x:0 0 0 2024-05-06T12:14:51.523 INFO:tasks.workunit.client.1.smithi190.stdout:5/639: mknod d2/d6c/cd0 0 2024-05-06T12:14:51.523 INFO:tasks.workunit.client.1.smithi190.stdout:9/690: unlink db/d23/l6a 0 2024-05-06T12:14:51.526 INFO:tasks.workunit.client.1.smithi190.stdout:1/484: write d5/d4f/d65/d62/f8d [502836,87073] 0 2024-05-06T12:14:51.526 INFO:tasks.workunit.client.1.smithi190.stdout:3/678: write d5/d7/d6d/fc4 [2919448,95797] 0 2024-05-06T12:14:51.531 INFO:tasks.workunit.client.1.smithi190.stdout:8/671: unlink d3/l5 0 2024-05-06T12:14:51.533 INFO:tasks.workunit.client.1.smithi190.stdout:1/485: mknod d5/d4f/d65/d62/d9d/caa 0 2024-05-06T12:14:51.537 INFO:tasks.workunit.client.1.smithi190.stdout:8/672: write d3/d8/d25/d65/fb5 [1027232,130646] 0 2024-05-06T12:14:51.538 INFO:tasks.workunit.client.1.smithi190.stdout:2/551: dread d2/d10/f2c [0,4194304] 0 2024-05-06T12:14:51.538 INFO:tasks.workunit.client.1.smithi190.stdout:9/691: creat db/fec x:0 0 0 2024-05-06T12:14:51.539 INFO:tasks.workunit.client.1.smithi190.stdout:0/634: rename d9/dc/d28/d4c/d80/da7/cd6 to d9/dc/d28/d4c/d96/ce2 0 2024-05-06T12:14:51.540 INFO:tasks.workunit.client.1.smithi190.stdout:2/552: write d2/d8/d19/d43/fb3 [567386,117276] 0 2024-05-06T12:14:51.543 INFO:tasks.workunit.client.1.smithi190.stdout:1/486: mknod d5/d15/d1e/cab 0 2024-05-06T12:14:51.548 INFO:tasks.workunit.client.1.smithi190.stdout:9/692: unlink db/dd/d16/d58/c6f 0 2024-05-06T12:14:51.548 INFO:tasks.workunit.client.1.smithi190.stdout:3/679: dwrite d5/d14/d11/d21/d44/d80/d82/d3d/ded/d84/db3/f7d [4194304,4194304] 0 2024-05-06T12:14:51.552 INFO:tasks.workunit.client.1.smithi190.stdout:0/635: symlink d9/d51/d93/le3 0 2024-05-06T12:14:51.555 INFO:tasks.workunit.client.1.smithi190.stdout:8/673: dwrite d3/fa3 [0,4194304] 0 2024-05-06T12:14:51.556 INFO:tasks.workunit.client.1.smithi190.stdout:3/680: dread d5/d14/f56 [0,4194304] 0 2024-05-06T12:14:51.562 INFO:tasks.workunit.client.1.smithi190.stdout:6/464: truncate df/d2c/d28/f7c 6298923 0 2024-05-06T12:14:51.564 INFO:tasks.workunit.client.1.smithi190.stdout:1/487: dwrite d5/d4f/d65/d24/d43/f60 [0,4194304] 0 2024-05-06T12:14:51.566 INFO:tasks.workunit.client.1.smithi190.stdout:1/488: write d5/d4f/d68/f8e [961790,2125] 0 2024-05-06T12:14:51.567 INFO:tasks.workunit.client.1.smithi190.stdout:3/681: write d5/d14/d11/d21/d44/d80/d82/d3d/ded/d84/db3/f45 [3842566,44093] 0 2024-05-06T12:14:51.570 INFO:tasks.workunit.client.1.smithi190.stdout:3/682: write d5/d7/f3a [2337479,7869] 0 2024-05-06T12:14:51.576 INFO:tasks.workunit.client.1.smithi190.stdout:5/640: dwrite d2/da/f1b [0,4194304] 0 2024-05-06T12:14:51.578 INFO:tasks.workunit.client.1.smithi190.stdout:6/465: mkdir df/d2c/d92/da3 0 2024-05-06T12:14:51.582 INFO:tasks.workunit.client.1.smithi190.stdout:3/683: write d5/d14/fe1 [8755878,60474] 0 2024-05-06T12:14:51.591 INFO:tasks.workunit.client.1.smithi190.stdout:6/466: rmdir df/d2c/d28 39 2024-05-06T12:14:51.596 INFO:tasks.workunit.client.1.smithi190.stdout:9/693: dwrite db/dd/d16/d58/ddc/d47/fbf [0,4194304] 0 2024-05-06T12:14:51.598 INFO:tasks.workunit.client.1.smithi190.stdout:1/489: mkdir d5/d4f/d65/d32/dac 0 2024-05-06T12:14:51.602 INFO:tasks.workunit.client.1.smithi190.stdout:0/636: dread d9/dcf/f33 [0,4194304] 0 2024-05-06T12:14:51.603 INFO:tasks.workunit.client.1.smithi190.stdout:6/467: creat df/d2c/d1e/d57/d53/fa4 x:0 0 0 2024-05-06T12:14:51.624 INFO:tasks.workunit.client.1.smithi190.stdout:1/490: dread d5/d4f/d65/d32/d47/d53/d6e/f79 [0,4194304] 0 2024-05-06T12:14:51.659 INFO:tasks.workunit.client.1.smithi190.stdout:1/491: rename d5/d4f/d65/d62/l67 to d5/d6b/d8f/lad 0 2024-05-06T12:14:51.660 INFO:tasks.workunit.client.1.smithi190.stdout:6/468: rename df to df/d2c/d28/da5 22 2024-05-06T12:14:51.661 INFO:tasks.workunit.client.1.smithi190.stdout:9/694: fdatasync db/d23/fdd 0 2024-05-06T12:14:51.682 INFO:tasks.workunit.client.1.smithi190.stdout:3/684: write d5/fc [346071,99281] 0 2024-05-06T12:14:51.690 INFO:tasks.workunit.client.1.smithi190.stdout:9/695: truncate db/d23/fa9 613787 0 2024-05-06T12:14:51.691 INFO:tasks.workunit.client.1.smithi190.stdout:6/469: truncate df/d2c/d61/f79 2937237 0 2024-05-06T12:14:51.695 INFO:tasks.workunit.client.1.smithi190.stdout:1/492: link d5/d4f/d65/d24/d3a/f3e d5/d4f/d65/d32/d47/d53/d6e/fae 0 2024-05-06T12:14:51.697 INFO:tasks.workunit.client.1.smithi190.stdout:1/493: fdatasync d5/d4f/d65/d32/d47/d53/d8b/f9b 0 2024-05-06T12:14:51.698 INFO:tasks.workunit.client.1.smithi190.stdout:1/494: fsync d5/d4f/d65/fa9 0 2024-05-06T12:14:51.698 INFO:tasks.workunit.client.1.smithi190.stdout:3/685: write d5/d14/d11/d21/d44/d92/d9e/fa2 [39280,81886] 0 2024-05-06T12:14:51.701 INFO:tasks.workunit.client.1.smithi190.stdout:1/495: mknod d5/d6b/d8f/caf 0 2024-05-06T12:14:51.704 INFO:tasks.workunit.client.1.smithi190.stdout:1/496: write d5/d15/f3f [2747096,111984] 0 2024-05-06T12:14:51.706 INFO:tasks.workunit.client.1.smithi190.stdout:3/686: mkdir d5/d14/d11/d21/d44/d92/d9e/da4/df3 0 2024-05-06T12:14:51.708 INFO:tasks.workunit.client.1.smithi190.stdout:3/687: write d5/d14/d11/d21/d44/d80/f83 [1548638,90351] 0 2024-05-06T12:14:51.710 INFO:tasks.workunit.client.1.smithi190.stdout:3/688: write d5/d14/d11/d21/d44/d80/f83 [2535035,93506] 0 2024-05-06T12:14:51.711 INFO:tasks.workunit.client.1.smithi190.stdout:7/600: sync 2024-05-06T12:14:51.713 INFO:tasks.workunit.client.1.smithi190.stdout:3/689: creat d5/d14/d11/d21/d44/d80/d82/d3d/ff4 x:0 0 0 2024-05-06T12:14:51.715 INFO:tasks.workunit.client.1.smithi190.stdout:7/601: creat d0/da/d36/d53/d5c/fda x:0 0 0 2024-05-06T12:14:51.720 INFO:tasks.workunit.client.1.smithi190.stdout:7/602: dwrite d0/da/f93 [0,4194304] 0 2024-05-06T12:14:51.746 INFO:tasks.workunit.client.1.smithi190.stdout:3/690: creat d5/d94/dd9/ff5 x:0 0 0 2024-05-06T12:14:51.747 INFO:tasks.workunit.client.1.smithi190.stdout:3/691: write d5/d7/fd8 [526969,116187] 0 2024-05-06T12:14:51.749 INFO:tasks.workunit.client.1.smithi190.stdout:3/692: chown d5/d14/d11/d21/d44/le9 8447 1 2024-05-06T12:14:51.750 INFO:tasks.workunit.client.1.smithi190.stdout:7/603: truncate d0/f40 3875543 0 2024-05-06T12:14:51.755 INFO:tasks.workunit.client.1.smithi190.stdout:7/604: unlink d0/d14/f6e 0 2024-05-06T12:14:51.755 INFO:tasks.workunit.client.1.smithi190.stdout:0/637: write d9/dc/f74 [407262,3041] 0 2024-05-06T12:14:51.757 INFO:tasks.workunit.client.1.smithi190.stdout:0/638: chown d9/d51/dc9/f8d 13938 1 2024-05-06T12:14:51.759 INFO:tasks.workunit.client.1.smithi190.stdout:0/639: read d9/d51/f9d [2937414,116984] 0 2024-05-06T12:14:51.761 INFO:tasks.workunit.client.1.smithi190.stdout:7/605: creat d0/d14/d6a/d8f/fdb x:0 0 0 2024-05-06T12:14:51.764 INFO:tasks.workunit.client.1.smithi190.stdout:1/497: link d5/d6b/d8f/lad d5/d15/d1e/d94/lb0 0 2024-05-06T12:14:51.768 INFO:tasks.workunit.client.1.smithi190.stdout:7/606: dwrite d0/d14/d6a/d8f/fdb [0,4194304] 0 2024-05-06T12:14:51.772 INFO:tasks.workunit.client.1.smithi190.stdout:3/693: dread d5/d7/f86 [0,4194304] 0 2024-05-06T12:14:51.774 INFO:tasks.workunit.client.1.smithi190.stdout:7/607: creat d0/d1/d18/d2b/fdc x:0 0 0 2024-05-06T12:14:51.776 INFO:tasks.workunit.client.1.smithi190.stdout:7/608: chown d0/da/d36/d53/d77/d8c/dd1/cbb 467063 1 2024-05-06T12:14:51.777 INFO:tasks.workunit.client.1.smithi190.stdout:7/609: truncate d0/da/d36/d42/fa7 316401 0 2024-05-06T12:14:51.778 INFO:tasks.workunit.client.1.smithi190.stdout:3/694: dread d5/d14/d11/f13 [0,4194304] 0 2024-05-06T12:14:51.779 INFO:tasks.workunit.client.1.smithi190.stdout:7/610: rmdir d0/d1/d2 39 2024-05-06T12:14:51.782 INFO:tasks.workunit.client.1.smithi190.stdout:7/611: readlink d0/d1/d2/l4b 0 2024-05-06T12:14:51.783 INFO:tasks.workunit.client.1.smithi190.stdout:3/695: mknod d5/cf6 0 2024-05-06T12:14:51.793 INFO:tasks.workunit.client.1.smithi190.stdout:7/612: rename d0/d14/d5d/l3f to d0/d14/ldd 0 2024-05-06T12:14:51.793 INFO:tasks.workunit.client.1.smithi190.stdout:7/613: fdatasync d0/da/f93 0 2024-05-06T12:14:51.797 INFO:tasks.workunit.client.1.smithi190.stdout:7/614: unlink d0/d1/d32/daa/ccf 0 2024-05-06T12:14:51.803 INFO:tasks.workunit.client.1.smithi190.stdout:7/615: dwrite d0/d1/d2/fc9 [0,4194304] 0 2024-05-06T12:14:51.805 INFO:tasks.workunit.client.1.smithi190.stdout:7/616: dread - d0/da/d36/d53/d77/d8c/dd1/d60/fa8 zero size 2024-05-06T12:14:51.807 INFO:tasks.workunit.client.1.smithi190.stdout:9/696: write db/dd/d16/d58/d87/d22/d49/d88/db5/fdf [1043347,126837] 0 2024-05-06T12:14:51.811 INFO:tasks.workunit.client.1.smithi190.stdout:7/617: rename d0/da/d36/d53/d5c/fb9 to d0/d4/fde 0 2024-05-06T12:14:51.814 INFO:tasks.workunit.client.1.smithi190.stdout:7/618: truncate d0/d1/d32/daa/fcb 474044 0 2024-05-06T12:14:51.815 INFO:tasks.workunit.client.1.smithi190.stdout:7/619: write d0/da/d36/d53/fb5 [948860,52914] 0 2024-05-06T12:14:51.815 INFO:tasks.workunit.client.1.smithi190.stdout:9/697: symlink db/dd/d5c/dac/deb/led 0 2024-05-06T12:14:51.817 INFO:tasks.workunit.client.1.smithi190.stdout:1/498: truncate d5/d4f/d65/d24/d43/f60 4112297 0 2024-05-06T12:14:51.818 INFO:tasks.workunit.client.1.smithi190.stdout:9/698: rmdir db/daa 39 2024-05-06T12:14:51.821 INFO:tasks.workunit.client.1.smithi190.stdout:9/699: read db/daa/fab [64846,47504] 0 2024-05-06T12:14:51.821 INFO:tasks.workunit.client.1.smithi190.stdout:1/499: mknod d5/d4f/d65/d32/d47/d53/d6e/cb1 0 2024-05-06T12:14:51.823 INFO:tasks.workunit.client.1.smithi190.stdout:3/696: write d5/d14/d11/d21/d44/f85 [402477,78133] 0 2024-05-06T12:14:51.828 INFO:tasks.workunit.client.1.smithi190.stdout:7/620: dwrite d0/d1/f92 [0,4194304] 0 2024-05-06T12:14:51.831 INFO:tasks.workunit.client.1.smithi190.stdout:3/697: creat d5/d14/d11/d21/d44/d92/d9e/da4/ff7 x:0 0 0 2024-05-06T12:14:51.833 INFO:tasks.workunit.client.1.smithi190.stdout:3/698: fsync d5/d14/d11/d21/d44/fbf 0 2024-05-06T12:14:51.834 INFO:tasks.workunit.client.1.smithi190.stdout:7/621: getdents d0/d1/d32/dd8 0 2024-05-06T12:14:51.834 INFO:tasks.workunit.client.1.smithi190.stdout:3/699: write d5/d14/d11/d21/fc1 [29706,109099] 0 2024-05-06T12:14:51.835 INFO:tasks.workunit.client.1.smithi190.stdout:1/500: getdents d5/d4f/d65/d62/d9d 0 2024-05-06T12:14:51.841 INFO:tasks.workunit.client.1.smithi190.stdout:7/622: getdents d0/d1/d2/d84/dc6 0 2024-05-06T12:14:51.843 INFO:tasks.workunit.client.1.smithi190.stdout:3/700: mkdir d5/d14/d11/d21/d44/df8 0 2024-05-06T12:14:51.846 INFO:tasks.workunit.client.1.smithi190.stdout:9/700: truncate db/d23/f8a 795989 0 2024-05-06T12:14:51.847 INFO:tasks.workunit.client.1.smithi190.stdout:1/501: getdents d5/d4f 0 2024-05-06T12:14:51.848 INFO:tasks.workunit.client.1.smithi190.stdout:3/701: creat d5/d7/d15/d39/ff9 x:0 0 0 2024-05-06T12:14:51.849 INFO:tasks.workunit.client.1.smithi190.stdout:7/623: dwrite d0/d1/d2/f7c [0,4194304] 0 2024-05-06T12:14:51.889 INFO:tasks.workunit.client.1.smithi190.stdout:7/624: symlink d0/d1/d32/daa/ldf 0 2024-05-06T12:14:51.890 INFO:tasks.workunit.client.1.smithi190.stdout:9/701: rename db/dd/d16/d58/d87/c72 to db/dd/d16/d58/ddc/d47/d4b/cee 0 2024-05-06T12:14:51.891 INFO:tasks.workunit.client.1.smithi190.stdout:4/463: sync 2024-05-06T12:14:51.891 INFO:tasks.workunit.client.1.smithi190.stdout:1/502: dread d5/d4f/d65/d32/d47/d53/d6e/f79 [0,4194304] 0 2024-05-06T12:14:51.892 INFO:tasks.workunit.client.1.smithi190.stdout:3/702: rmdir d5/d14/d11/d21/d44/d80/d82/d3d 39 2024-05-06T12:14:51.960 INFO:tasks.workunit.client.1.smithi190.stdout:4/464: chown d2/d11/d16/d8e/c5b 4 1 2024-05-06T12:14:51.961 INFO:tasks.workunit.client.1.smithi190.stdout:7/625: rename d0/d14/c31 to d0/d14/d5d/ce0 0 2024-05-06T12:14:51.962 INFO:tasks.workunit.client.1.smithi190.stdout:1/503: creat d5/d15/d1e/d94/fb2 x:0 0 0 2024-05-06T12:14:51.963 INFO:tasks.workunit.client.1.smithi190.stdout:7/626: read d0/d1/d2/f1d [3580855,107203] 0 2024-05-06T12:14:51.965 INFO:tasks.workunit.client.1.smithi190.stdout:4/465: fdatasync d2/d11/d16/f1f 0 2024-05-06T12:14:51.967 INFO:tasks.workunit.client.1.smithi190.stdout:7/627: mkdir d0/d1/d18/d6c/de1 0 2024-05-06T12:14:51.969 INFO:tasks.workunit.client.1.smithi190.stdout:7/628: fsync d0/d1/d18/d2b/f80 0 2024-05-06T12:14:51.971 INFO:tasks.workunit.client.1.smithi190.stdout:3/703: dread d5/d14/d11/d21/d44/d80/d82/d3d/ded/d84/db3/f7d [4194304,4194304] 0 2024-05-06T12:14:51.971 INFO:tasks.workunit.client.1.smithi190.stdout:1/504: dwrite d5/d4f/d65/d32/f4a [0,4194304] 0 2024-05-06T12:14:51.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:51 smithi092 ceph-mon[26954]: Reconfiguring alertmanager.smithi092 (dependencies changed)... 2024-05-06T12:14:51.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:51 smithi092 ceph-mon[26954]: Reconfiguring daemon alertmanager.smithi092 on smithi092 2024-05-06T12:14:51.975 INFO:tasks.workunit.client.1.smithi190.stdout:1/505: mknod d5/d15/d1e/cb3 0 2024-05-06T12:14:51.978 INFO:tasks.workunit.client.1.smithi190.stdout:3/704: mkdir d5/d14/d11/d21/d44/d80/d82/d3d/ded/d84/db3/dfa 0 2024-05-06T12:14:51.980 INFO:tasks.workunit.client.1.smithi190.stdout:3/705: write d5/d7/d6d/fc4 [3855423,36920] 0 2024-05-06T12:14:51.980 INFO:tasks.workunit.client.1.smithi190.stdout:7/629: rename d0/d1/d18/d2b/d5a/f5e to d0/d1/d18/d4e/db3/fe2 0 2024-05-06T12:14:51.982 INFO:tasks.workunit.client.1.smithi190.stdout:7/630: write d0/d4/f6b [760047,122624] 0 2024-05-06T12:14:51.983 INFO:tasks.workunit.client.1.smithi190.stdout:4/466: dread d2/d10/f35 [0,4194304] 0 2024-05-06T12:14:51.983 INFO:tasks.workunit.client.1.smithi190.stdout:4/467: chown d2/d11/d16/f1f 13643337 1 2024-05-06T12:14:51.984 INFO:tasks.workunit.client.1.smithi190.stdout:7/631: mknod d0/d1/d18/d4e/db3/ce3 0 2024-05-06T12:14:51.993 INFO:tasks.workunit.client.1.smithi190.stdout:7/632: write d0/f69 [4132486,28570] 0 2024-05-06T12:14:51.994 INFO:tasks.workunit.client.1.smithi190.stdout:7/633: fsync d0/da/d36/d53/f98 0 2024-05-06T12:14:51.996 INFO:tasks.workunit.client.1.smithi190.stdout:7/634: mknod d0/d1/d18/d2b/ce4 0 2024-05-06T12:14:51.996 INFO:tasks.workunit.client.1.smithi190.stdout:7/635: chown d0/d1/d18/d2b/d5a/fa1 1 1 2024-05-06T12:14:51.999 INFO:tasks.workunit.client.1.smithi190.stdout:7/636: unlink d0/d14/c27 0 2024-05-06T12:14:52.001 INFO:tasks.workunit.client.1.smithi190.stdout:7/637: creat d0/da/d36/fe5 x:0 0 0 2024-05-06T12:14:52.002 INFO:tasks.workunit.client.1.smithi190.stdout:7/638: write d0/da/d36/d53/d77/d8c/dd1/d60/fc2 [3948950,28507] 0 2024-05-06T12:14:52.004 INFO:tasks.workunit.client.1.smithi190.stdout:7/639: symlink d0/db4/le6 0 2024-05-06T12:14:52.006 INFO:tasks.workunit.client.1.smithi190.stdout:7/640: rmdir d0/d1/d18/d4e/d87 39 2024-05-06T12:14:52.009 INFO:tasks.workunit.client.1.smithi190.stdout:7/641: unlink d0/d1/d32/daa/fb1 0 2024-05-06T12:14:52.013 INFO:tasks.workunit.client.1.smithi190.stdout:7/642: rmdir d0/d14/d6a/d8f 39 2024-05-06T12:14:52.017 INFO:tasks.workunit.client.1.smithi190.stdout:7/643: mkdir d0/d1/d18/d6c/de1/de7 0 2024-05-06T12:14:52.021 INFO:tasks.workunit.client.1.smithi190.stdout:7/644: dwrite d0/d1/d18/d2b/f80 [0,4194304] 0 2024-05-06T12:14:52.026 INFO:tasks.workunit.client.1.smithi190.stdout:7/645: dread d0/d1/d2/fc9 [0,4194304] 0 2024-05-06T12:14:52.029 INFO:tasks.workunit.client.1.smithi190.stdout:7/646: write d0/da/d36/d53/f9b [514046,129824] 0 2024-05-06T12:14:52.032 INFO:tasks.workunit.client.1.smithi190.stdout:7/647: creat d0/d14/d6a/d8f/fe8 x:0 0 0 2024-05-06T12:14:52.036 INFO:tasks.workunit.client.1.smithi190.stdout:7/648: rename d0/da/d36/d53/cab to d0/da/d36/d53/d77/d8c/dd1/d60/dac/ce9 0 2024-05-06T12:14:52.037 INFO:tasks.workunit.client.1.smithi190.stdout:7/649: fdatasync d0/d1/d18/d37/f9e 0 2024-05-06T12:14:52.040 INFO:tasks.workunit.client.1.smithi190.stdout:7/650: link d0/d4/l89 d0/d1/d32/d45/d70/d3b/lea 0 2024-05-06T12:14:52.044 INFO:tasks.workunit.client.1.smithi190.stdout:7/651: dwrite d0/f22 [0,4194304] 0 2024-05-06T12:14:52.048 INFO:tasks.workunit.client.1.smithi190.stdout:7/652: symlink d0/db4/leb 0 2024-05-06T12:14:52.048 INFO:tasks.workunit.client.1.smithi190.stdout:7/653: fsync d0/da/d36/d53/f62 0 2024-05-06T12:14:52.054 INFO:tasks.workunit.client.1.smithi190.stdout:7/654: dwrite d0/d1/d18/d6c/fba [0,4194304] 0 2024-05-06T12:14:52.058 INFO:tasks.workunit.client.1.smithi190.stdout:7/655: getdents d0/d1/d32/d45/d70/d35 0 2024-05-06T12:14:52.059 INFO:tasks.workunit.client.1.smithi190.stdout:7/656: read - d0/d14/d6a/fd9 zero size 2024-05-06T12:14:52.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:51 smithi190 ceph-mon[33798]: Reconfiguring alertmanager.smithi092 (dependencies changed)... 2024-05-06T12:14:52.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:51 smithi190 ceph-mon[33798]: Reconfiguring daemon alertmanager.smithi092 on smithi092 2024-05-06T12:14:52.166 INFO:tasks.workunit.client.1.smithi190.stdout:2/553: sync 2024-05-06T12:14:52.167 INFO:tasks.workunit.client.1.smithi190.stdout:2/554: truncate d2/d8/f92 576850 0 2024-05-06T12:14:52.168 INFO:tasks.workunit.client.1.smithi190.stdout:2/555: mknod d2/d8/d71/cb4 0 2024-05-06T12:14:52.173 INFO:tasks.workunit.client.1.smithi190.stdout:2/556: dwrite d2/d8/d19/d43/d79/d7e/f40 [4194304,4194304] 0 2024-05-06T12:14:52.177 INFO:tasks.workunit.client.1.smithi190.stdout:2/557: link d2/d8/d33/d46/f4e d2/d8/d19/d7b/da0/fb5 0 2024-05-06T12:14:52.185 INFO:tasks.workunit.client.1.smithi190.stdout:2/558: truncate d2/d8/dc/f16 4145572 0 2024-05-06T12:14:52.190 INFO:tasks.workunit.client.1.smithi190.stdout:2/559: dwrite d2/d4c/f6f [0,4194304] 0 2024-05-06T12:14:52.265 INFO:tasks.workunit.client.1.smithi190.stdout:2/560: creat d2/fb6 x:0 0 0 2024-05-06T12:14:52.265 INFO:tasks.workunit.client.1.smithi190.stdout:2/561: dread - d2/d8/d71/f8e zero size 2024-05-06T12:14:52.267 INFO:tasks.workunit.client.1.smithi190.stdout:2/562: dread - d2/d8/d19/d43/d79/db2/f9d zero size 2024-05-06T12:14:52.269 INFO:tasks.workunit.client.1.smithi190.stdout:2/563: mkdir d2/d8/d33/d46/db7 0 2024-05-06T12:14:52.272 INFO:tasks.workunit.client.1.smithi190.stdout:2/564: dread d2/d10/f87 [0,4194304] 0 2024-05-06T12:14:52.273 INFO:tasks.workunit.client.1.smithi190.stdout:2/565: read d2/f12 [59173,64313] 0 2024-05-06T12:14:52.274 INFO:tasks.workunit.client.1.smithi190.stdout:2/566: readlink d2/l3 0 2024-05-06T12:14:52.276 INFO:tasks.workunit.client.1.smithi190.stdout:2/567: mkdir d2/d8/d19/db8 0 2024-05-06T12:14:52.276 INFO:tasks.workunit.client.1.smithi190.stdout:2/568: fdatasync d2/d8/d19/d4f/f57 0 2024-05-06T12:14:52.278 INFO:tasks.workunit.client.1.smithi190.stdout:2/569: mkdir d2/d8/d19/db8/db9 0 2024-05-06T12:14:52.279 INFO:tasks.workunit.client.1.smithi190.stdout:2/570: stat d2/d8/d33 0 2024-05-06T12:14:52.279 INFO:tasks.workunit.client.1.smithi190.stdout:2/571: chown d2/d8/d33/d46/d5d/l73 100746 1 2024-05-06T12:14:52.281 INFO:tasks.workunit.client.1.smithi190.stdout:2/572: write d2/d8/d19/f8a [398725,130169] 0 2024-05-06T12:14:52.292 INFO:tasks.workunit.client.1.smithi190.stdout:2/573: dwrite d2/d8/d33/f37 [0,4194304] 0 2024-05-06T12:14:52.295 INFO:tasks.workunit.client.1.smithi190.stdout:2/574: fdatasync d2/d21/f9f 0 2024-05-06T12:14:52.299 INFO:tasks.workunit.client.1.smithi190.stdout:2/575: dwrite d2/d8/d19/f8a [0,4194304] 0 2024-05-06T12:14:52.300 INFO:tasks.workunit.client.1.smithi190.stdout:2/576: chown d2/d8/d19/d43/d79/d7e 331261 1 2024-05-06T12:14:52.303 INFO:tasks.workunit.client.1.smithi190.stdout:2/577: creat d2/d8/d33/d46/dab/dac/fba x:0 0 0 2024-05-06T12:14:52.305 INFO:tasks.workunit.client.1.smithi190.stdout:2/578: mkdir d2/d8/d33/d46/dab/dbb 0 2024-05-06T12:14:52.307 INFO:tasks.workunit.client.1.smithi190.stdout:2/579: mknod d2/d8/d19/d43/cbc 0 2024-05-06T12:14:52.314 INFO:tasks.workunit.client.1.smithi190.stdout:2/580: dread d2/f12 [0,4194304] 0 2024-05-06T12:14:52.315 INFO:tasks.workunit.client.1.smithi190.stdout:8/674: sync 2024-05-06T12:14:52.316 INFO:tasks.workunit.client.1.smithi190.stdout:8/675: readlink d3/d8/d25/d3b/la8 0 2024-05-06T12:14:52.316 INFO:tasks.workunit.client.1.smithi190.stdout:2/581: stat d2/d8/c53 0 2024-05-06T12:14:52.317 INFO:tasks.workunit.client.1.smithi190.stdout:2/582: chown d2/lb 609615 1 2024-05-06T12:14:52.319 INFO:tasks.workunit.client.1.smithi190.stdout:8/676: creat d3/d8/fe9 x:0 0 0 2024-05-06T12:14:52.320 INFO:tasks.workunit.client.1.smithi190.stdout:2/583: mkdir d2/d8/d19/dbd 0 2024-05-06T12:14:52.321 INFO:tasks.workunit.client.1.smithi190.stdout:2/584: rmdir d2/d8/d33/d46 39 2024-05-06T12:14:52.322 INFO:tasks.workunit.client.1.smithi190.stdout:8/677: truncate d3/d8/d2b/d3e/dad/fd3 734610 0 2024-05-06T12:14:52.324 INFO:tasks.workunit.client.1.smithi190.stdout:2/585: mknod d2/d8/d33/d46/dab/dbb/cbe 0 2024-05-06T12:14:52.325 INFO:tasks.workunit.client.1.smithi190.stdout:2/586: write d2/d4c/f6f [1551471,10975] 0 2024-05-06T12:14:52.326 INFO:tasks.workunit.client.1.smithi190.stdout:8/678: rename d3/f6 to d3/d8/d25/d65/dc6/fea 0 2024-05-06T12:14:52.336 INFO:tasks.workunit.client.1.smithi190.stdout:2/587: dwrite d2/d8/dc/f95 [0,4194304] 0 2024-05-06T12:14:52.340 INFO:tasks.workunit.client.1.smithi190.stdout:2/588: creat d2/d8/d33/d46/dab/dbb/fbf x:0 0 0 2024-05-06T12:14:52.343 INFO:tasks.workunit.client.1.smithi190.stdout:2/589: rename l0 to d2/d8/d19/d4f/lc0 0 2024-05-06T12:14:52.347 INFO:tasks.workunit.client.1.smithi190.stdout:2/590: rename d2/d8/dc/c2f to d2/d8/d19/cc1 0 2024-05-06T12:14:52.354 INFO:tasks.workunit.client.1.smithi190.stdout:2/591: dwrite d2/d8/d71/f8b [4194304,4194304] 0 2024-05-06T12:14:52.374 INFO:tasks.workunit.client.1.smithi190.stdout:2/592: dread d2/d8/d33/d46/f5f [0,4194304] 0 2024-05-06T12:14:52.375 INFO:tasks.workunit.client.1.smithi190.stdout:2/593: write d2/d8/d19/d4f/f57 [226873,84300] 0 2024-05-06T12:14:52.377 INFO:tasks.workunit.client.1.smithi190.stdout:2/594: mkdir d2/d8/d33/dc2 0 2024-05-06T12:14:52.378 INFO:tasks.workunit.client.1.smithi190.stdout:2/595: symlink d2/d8/d33/d46/dab/dbb/lc3 0 2024-05-06T12:14:52.381 INFO:tasks.workunit.client.1.smithi190.stdout:2/596: rename d2/d8/d19/d43/d79/db2/c9c to d2/d8/d33/d46/dab/d91/cc4 0 2024-05-06T12:14:52.382 INFO:tasks.workunit.client.1.smithi190.stdout:2/597: mknod d2/d8/d33/d46/db7/cc5 0 2024-05-06T12:14:52.383 INFO:tasks.workunit.client.1.smithi190.stdout:2/598: read - d2/d8/d19/f45 zero size 2024-05-06T12:14:52.383 INFO:tasks.workunit.client.1.smithi190.stdout:2/599: chown d2/d8/d71/f8b 553599 1 2024-05-06T12:14:52.393 INFO:tasks.workunit.client.1.smithi190.stdout:2/600: dwrite d2/d8/d19/f83 [4194304,4194304] 0 2024-05-06T12:14:52.395 INFO:tasks.workunit.client.1.smithi190.stdout:2/601: mknod d2/d8/d19/da9/cc6 0 2024-05-06T12:14:52.397 INFO:tasks.workunit.client.1.smithi190.stdout:2/602: mknod d2/d8/d33/dc2/cc7 0 2024-05-06T12:14:52.398 INFO:tasks.workunit.client.1.smithi190.stdout:2/603: chown d2/d8/d19/d9b/ca6 1 1 2024-05-06T12:14:52.400 INFO:tasks.workunit.client.1.smithi190.stdout:2/604: creat d2/d8/d19/d43/fc8 x:0 0 0 2024-05-06T12:14:52.402 INFO:tasks.workunit.client.1.smithi190.stdout:2/605: mknod d2/d8/d33/d46/dab/cc9 0 2024-05-06T12:14:52.404 INFO:tasks.workunit.client.1.smithi190.stdout:2/606: mknod d2/d8/d33/d46/dab/cca 0 2024-05-06T12:14:52.405 INFO:tasks.workunit.client.1.smithi190.stdout:2/607: mkdir d2/d8/d33/dc2/dcb 0 2024-05-06T12:14:52.407 INFO:tasks.workunit.client.1.smithi190.stdout:2/608: mknod d2/d8/d19/dbd/ccc 0 2024-05-06T12:14:52.408 INFO:tasks.workunit.client.1.smithi190.stdout:2/609: mkdir d2/d8/d33/dc2/dcd 0 2024-05-06T12:14:52.409 INFO:tasks.workunit.client.1.smithi190.stdout:2/610: chown d2/d8/d19/dbd 41394 1 2024-05-06T12:14:52.414 INFO:tasks.workunit.client.1.smithi190.stdout:2/611: read d2/d8/d71/f88 [3008554,129279] 0 2024-05-06T12:14:52.416 INFO:tasks.workunit.client.1.smithi190.stdout:2/612: creat d2/d8/d19/d9b/fce x:0 0 0 2024-05-06T12:14:52.419 INFO:tasks.workunit.client.1.smithi190.stdout:2/613: dread d2/d8/d19/d43/d79/d7e/f63 [0,4194304] 0 2024-05-06T12:14:52.421 INFO:tasks.workunit.client.1.smithi190.stdout:2/614: stat d2/d21/la1 0 2024-05-06T12:14:52.422 INFO:tasks.workunit.client.1.smithi190.stdout:2/615: unlink d2/l26 0 2024-05-06T12:14:52.423 INFO:tasks.workunit.client.1.smithi190.stdout:2/616: write d2/d8/d19/f8a [429733,121474] 0 2024-05-06T12:14:52.426 INFO:tasks.workunit.client.1.smithi190.stdout:2/617: write d2/d21/f89 [1010090,85459] 0 2024-05-06T12:14:52.429 INFO:tasks.workunit.client.1.smithi190.stdout:2/618: unlink d2/d8/d33/d46/d5d/f74 0 2024-05-06T12:14:52.429 INFO:tasks.workunit.client.1.smithi190.stdout:2/619: chown d2/d8/dc/c78 313037 1 2024-05-06T12:14:52.430 INFO:tasks.workunit.client.1.smithi190.stdout:2/620: write d2/d8/d71/f8b [4284159,55522] 0 2024-05-06T12:14:52.432 INFO:tasks.workunit.client.1.smithi190.stdout:2/621: symlink d2/d8/d33/lcf 0 2024-05-06T12:14:52.434 INFO:tasks.workunit.client.1.smithi190.stdout:2/622: stat d2/d8/d19/cc1 0 2024-05-06T12:14:52.438 INFO:tasks.workunit.client.1.smithi190.stdout:6/470: sync 2024-05-06T12:14:52.438 INFO:tasks.workunit.client.1.smithi190.stdout:0/640: sync 2024-05-06T12:14:52.438 INFO:tasks.workunit.client.1.smithi190.stdout:5/641: sync 2024-05-06T12:14:52.439 INFO:tasks.workunit.client.1.smithi190.stdout:6/471: chown df/d12/f8b 455451857 1 2024-05-06T12:14:52.439 INFO:tasks.workunit.client.1.smithi190.stdout:2/623: rmdir d2 39 2024-05-06T12:14:52.440 INFO:tasks.workunit.client.1.smithi190.stdout:0/641: fdatasync d9/d51/f64 0 2024-05-06T12:14:52.440 INFO:tasks.workunit.client.1.smithi190.stdout:5/642: chown d2/da/d59/d5f/d2e/da3/ccc 10012 1 2024-05-06T12:14:52.441 INFO:tasks.workunit.client.1.smithi190.stdout:6/472: write df/d2c/d92/f93 [346537,72229] 0 2024-05-06T12:14:52.442 INFO:tasks.workunit.client.1.smithi190.stdout:5/643: write d2/f30 [5365373,25278] 0 2024-05-06T12:14:52.444 INFO:tasks.workunit.client.1.smithi190.stdout:0/642: mkdir d9/dc/d28/dc5/dd5/de4 0 2024-05-06T12:14:52.448 INFO:tasks.workunit.client.1.smithi190.stdout:2/624: dwrite d2/f35 [0,4194304] 0 2024-05-06T12:14:52.450 INFO:tasks.workunit.client.1.smithi190.stdout:5/644: getdents d2/da/d59/d5f/d2e/d3b/d5a 0 2024-05-06T12:14:52.450 INFO:tasks.workunit.client.1.smithi190.stdout:0/643: dread d9/dc/d28/dc5/f7f [0,4194304] 0 2024-05-06T12:14:52.455 INFO:tasks.workunit.client.1.smithi190.stdout:5/645: creat d2/da/d59/d5f/d50/d6d/fd1 x:0 0 0 2024-05-06T12:14:52.457 INFO:tasks.workunit.client.1.smithi190.stdout:5/646: chown d2/d65/c6b 0 1 2024-05-06T12:14:52.459 INFO:tasks.workunit.client.1.smithi190.stdout:0/644: link d9/dc/d28/f8c d9/d51/dc9/fe5 0 2024-05-06T12:14:52.461 INFO:tasks.workunit.client.1.smithi190.stdout:5/647: unlink d2/d6c/f99 0 2024-05-06T12:14:52.463 INFO:tasks.workunit.client.1.smithi190.stdout:2/625: dread d2/d21/f3d [0,4194304] 0 2024-05-06T12:14:52.465 INFO:tasks.workunit.client.1.smithi190.stdout:5/648: creat d2/da/d59/d5f/d2e/d3b/d76/fd2 x:0 0 0 2024-05-06T12:14:52.465 INFO:tasks.workunit.client.1.smithi190.stdout:5/649: stat d2/da/d59/d5f/d2e/l29 0 2024-05-06T12:14:52.466 INFO:tasks.workunit.client.1.smithi190.stdout:2/626: mknod d2/d21/cd0 0 2024-05-06T12:14:52.469 INFO:tasks.workunit.client.1.smithi190.stdout:2/627: mknod d2/d8/d33/dc2/dcd/cd1 0 2024-05-06T12:14:52.469 INFO:tasks.workunit.client.1.smithi190.stdout:2/628: dread - d2/d8/d33/d46/f8d zero size 2024-05-06T12:14:52.470 INFO:tasks.workunit.client.1.smithi190.stdout:5/650: link d2/da/d59/l9c d2/da/d59/d5f/d50/ld3 0 2024-05-06T12:14:52.471 INFO:tasks.workunit.client.1.smithi190.stdout:5/651: chown d2/d6c/cd0 219740457 1 2024-05-06T12:14:52.472 INFO:tasks.workunit.client.1.smithi190.stdout:2/629: creat d2/d4c/fd2 x:0 0 0 2024-05-06T12:14:52.474 INFO:tasks.workunit.client.1.smithi190.stdout:2/630: write d2/d8/d33/d46/dab/f59 [1796290,110425] 0 2024-05-06T12:14:52.474 INFO:tasks.workunit.client.1.smithi190.stdout:5/652: dread d2/da/d59/d5f/d2e/d3b/d5a/f91 [0,4194304] 0 2024-05-06T12:14:52.476 INFO:tasks.workunit.client.1.smithi190.stdout:5/653: write d2/d65/fb0 [265807,111581] 0 2024-05-06T12:14:52.477 INFO:tasks.workunit.client.1.smithi190.stdout:2/631: creat d2/d8/d19/d7b/da0/fd3 x:0 0 0 2024-05-06T12:14:52.478 INFO:tasks.workunit.client.1.smithi190.stdout:0/645: dread d9/dc/d48/d50/f66 [0,4194304] 0 2024-05-06T12:14:52.478 INFO:tasks.workunit.client.1.smithi190.stdout:5/654: fdatasync d2/da/f2f 0 2024-05-06T12:14:52.479 INFO:tasks.workunit.client.1.smithi190.stdout:2/632: write d2/d21/f3d [4080579,72376] 0 2024-05-06T12:14:52.480 INFO:tasks.workunit.client.1.smithi190.stdout:5/655: read d2/da/d59/d5f/f5b [9111300,23291] 0 2024-05-06T12:14:52.482 INFO:tasks.workunit.client.1.smithi190.stdout:5/656: write d2/d27/f20 [5612079,123785] 0 2024-05-06T12:14:52.483 INFO:tasks.workunit.client.1.smithi190.stdout:5/657: chown d2/da/l45 31153 1 2024-05-06T12:14:52.484 INFO:tasks.workunit.client.1.smithi190.stdout:2/633: creat d2/d8/d19/d43/d79/d7e/fd4 x:0 0 0 2024-05-06T12:14:52.493 INFO:tasks.workunit.client.1.smithi190.stdout:2/634: dwrite d2/f12 [0,4194304] 0 2024-05-06T12:14:52.496 INFO:tasks.workunit.client.1.smithi190.stdout:2/635: mknod d2/d8/d19/d7b/cd5 0 2024-05-06T12:14:52.497 INFO:tasks.workunit.client.1.smithi190.stdout:0/646: write d9/f1b [12904,16135] 0 2024-05-06T12:14:52.500 INFO:tasks.workunit.client.1.smithi190.stdout:5/658: truncate d2/da/f1b 1059070 0 2024-05-06T12:14:52.502 INFO:tasks.workunit.client.1.smithi190.stdout:2/636: creat d2/d8/d19/d43/d79/fd6 x:0 0 0 2024-05-06T12:14:52.504 INFO:tasks.workunit.client.1.smithi190.stdout:2/637: mknod d2/d8/d33/d46/db7/cd7 0 2024-05-06T12:14:52.505 INFO:tasks.workunit.client.1.smithi190.stdout:0/647: dwrite d9/dc/d48/f4e [0,4194304] 0 2024-05-06T12:14:52.509 INFO:tasks.workunit.client.1.smithi190.stdout:0/648: rename d9/dc/d28/f76 to d9/d51/d93/fe6 0 2024-05-06T12:14:52.511 INFO:tasks.workunit.client.1.smithi190.stdout:2/638: getdents d2/d8/d33/d46/dab/dbb 0 2024-05-06T12:14:52.515 INFO:tasks.workunit.client.1.smithi190.stdout:2/639: creat d2/d8/d19/d4f/fd8 x:0 0 0 2024-05-06T12:14:52.516 INFO:tasks.workunit.client.1.smithi190.stdout:0/649: dwrite d9/f1b [4194304,4194304] 0 2024-05-06T12:14:52.517 INFO:tasks.workunit.client.1.smithi190.stdout:0/650: dread - d9/dcf/f71 zero size 2024-05-06T12:14:52.518 INFO:tasks.workunit.client.1.smithi190.stdout:0/651: dread d9/dcf/f62 [0,4194304] 0 2024-05-06T12:14:52.519 INFO:tasks.workunit.client.1.smithi190.stdout:0/652: chown d9/dc/d28/d4c/d80/da7/lc3 87580604 1 2024-05-06T12:14:52.531 INFO:tasks.workunit.client.1.smithi190.stdout:0/653: truncate d9/dcf/d21/f7e 3690509 0 2024-05-06T12:14:52.542 INFO:tasks.workunit.client.1.smithi190.stdout:0/654: dread d9/dcf/d56/dab/fba [0,4194304] 0 2024-05-06T12:14:52.543 INFO:tasks.workunit.client.1.smithi190.stdout:0/655: write d9/dc/f74 [267399,86136] 0 2024-05-06T12:14:52.546 INFO:tasks.workunit.client.1.smithi190.stdout:0/656: rename d9/dcf/l6e to d9/dc/le7 0 2024-05-06T12:14:52.547 INFO:tasks.workunit.client.1.smithi190.stdout:0/657: rmdir d9/dc/d28/d4c/d80/da7 39 2024-05-06T12:14:52.549 INFO:tasks.workunit.client.1.smithi190.stdout:0/658: fdatasync d9/dc/d48/fe1 0 2024-05-06T12:14:52.550 INFO:tasks.workunit.client.1.smithi190.stdout:0/659: write d9/dc/d48/fce [591325,9001] 0 2024-05-06T12:14:52.552 INFO:tasks.workunit.client.1.smithi190.stdout:0/660: truncate d9/dcf/d56/dab/fca 1042721 0 2024-05-06T12:14:52.554 INFO:tasks.workunit.client.1.smithi190.stdout:0/661: mknod d9/d6f/ce8 0 2024-05-06T12:14:52.556 INFO:tasks.workunit.client.1.smithi190.stdout:0/662: creat d9/dcf/d56/fe9 x:0 0 0 2024-05-06T12:14:52.556 INFO:tasks.workunit.client.1.smithi190.stdout:0/663: dread - d9/dcf/d56/dab/fbc zero size 2024-05-06T12:14:52.558 INFO:tasks.workunit.client.1.smithi190.stdout:1/506: sync 2024-05-06T12:14:52.558 INFO:tasks.workunit.client.1.smithi190.stdout:4/468: sync 2024-05-06T12:14:52.558 INFO:tasks.workunit.client.1.smithi190.stdout:9/702: sync 2024-05-06T12:14:52.558 INFO:tasks.workunit.client.1.smithi190.stdout:7/657: sync 2024-05-06T12:14:52.558 INFO:tasks.workunit.client.1.smithi190.stdout:8/679: sync 2024-05-06T12:14:52.558 INFO:tasks.workunit.client.1.smithi190.stdout:3/706: sync 2024-05-06T12:14:52.560 INFO:tasks.workunit.client.1.smithi190.stdout:3/707: readlink d5/l93 0 2024-05-06T12:14:52.561 INFO:tasks.workunit.client.1.smithi190.stdout:0/664: read d9/dcf/d21/d31/f7b [181474,111112] 0 2024-05-06T12:14:52.561 INFO:tasks.workunit.client.1.smithi190.stdout:4/469: stat d2/d11/d16/d8e/c8a 0 2024-05-06T12:14:52.563 INFO:tasks.workunit.client.1.smithi190.stdout:7/658: mknod d0/d1/d18/d2b/cec 0 2024-05-06T12:14:52.563 INFO:tasks.workunit.client.1.smithi190.stdout:4/470: dread - d2/d11/d16/d4e/fa1 zero size 2024-05-06T12:14:52.568 INFO:tasks.workunit.client.1.smithi190.stdout:3/708: mkdir d5/d14/d11/d21/d44/d80/d82/d3d/ded/d84/dfb 0 2024-05-06T12:14:52.570 INFO:tasks.workunit.client.1.smithi190.stdout:4/471: chown d2/f64 608854514 1 2024-05-06T12:14:52.570 INFO:tasks.workunit.client.1.smithi190.stdout:9/703: creat db/dd/d16/d58/d87/d22/d49/d88/d9f/fef x:0 0 0 2024-05-06T12:14:52.570 INFO:tasks.workunit.client.1.smithi190.stdout:0/665: mkdir d9/dc/d28/dc5/dd5/de4/dea 0 2024-05-06T12:14:52.571 INFO:tasks.workunit.client.1.smithi190.stdout:0/666: chown d9/dc/f74 335 1 2024-05-06T12:14:52.571 INFO:tasks.workunit.client.1.smithi190.stdout:3/709: chown d5/ca7 117 1 2024-05-06T12:14:52.572 INFO:tasks.workunit.client.1.smithi190.stdout:8/680: dwrite d3/d8/d2b/f34 [0,4194304] 0 2024-05-06T12:14:52.572 INFO:tasks.workunit.client.1.smithi190.stdout:4/472: symlink d2/d11/d16/d61/la5 0 2024-05-06T12:14:52.577 INFO:tasks.workunit.client.1.smithi190.stdout:9/704: mknod db/dd/d16/d58/cf0 0 2024-05-06T12:14:52.579 INFO:tasks.workunit.client.1.smithi190.stdout:4/473: dread - d2/d11/d16/d41/f65 zero size 2024-05-06T12:14:52.580 INFO:tasks.workunit.client.1.smithi190.stdout:4/474: fdatasync d2/d11/f20 0 2024-05-06T12:14:52.580 INFO:tasks.workunit.client.1.smithi190.stdout:1/507: dread d5/d4f/d65/f7b [0,4194304] 0 2024-05-06T12:14:52.582 INFO:tasks.workunit.client.1.smithi190.stdout:9/705: read - db/d23/d7e/fea zero size 2024-05-06T12:14:52.583 INFO:tasks.workunit.client.1.smithi190.stdout:3/710: mkdir d5/d14/d11/d21/d44/d80/d82/dfc 0 2024-05-06T12:14:52.584 INFO:tasks.workunit.client.1.smithi190.stdout:0/667: creat d9/feb x:0 0 0 2024-05-06T12:14:52.585 INFO:tasks.workunit.client.1.smithi190.stdout:1/508: chown d5/d15/d1e/d94/lb0 65078 1 2024-05-06T12:14:52.586 INFO:tasks.workunit.client.1.smithi190.stdout:4/475: symlink d2/d11/d16/d61/d9e/la6 0 2024-05-06T12:14:52.587 INFO:tasks.workunit.client.1.smithi190.stdout:9/706: symlink db/dd/d16/d4c/lf1 0 2024-05-06T12:14:52.589 INFO:tasks.workunit.client.1.smithi190.stdout:1/509: truncate d5/d15/f4e 186071 0 2024-05-06T12:14:52.590 INFO:tasks.workunit.client.1.smithi190.stdout:0/668: creat d9/dc/d28/dc5/dd5/de4/dea/fec x:0 0 0 2024-05-06T12:14:52.590 INFO:tasks.workunit.client.1.smithi190.stdout:1/510: write d5/d15/d1e/d94/fb2 [35456,23980] 0 2024-05-06T12:14:52.590 INFO:tasks.workunit.client.1.smithi190.stdout:4/476: truncate d2/d11/d16/d31/f43 1779326 0 2024-05-06T12:14:52.593 INFO:tasks.workunit.client.1.smithi190.stdout:1/511: readlink d5/d4f/d65/d24/d3a/l3d 0 2024-05-06T12:14:52.593 INFO:tasks.workunit.client.1.smithi190.stdout:3/711: link d5/d14/d11/f9f d5/d14/d11/d21/d44/d80/d82/d3d/ded/d33/ffd 0 2024-05-06T12:14:52.594 INFO:tasks.workunit.client.1.smithi190.stdout:1/512: fsync d5/d15/f4e 0 2024-05-06T12:14:52.595 INFO:tasks.workunit.client.1.smithi190.stdout:3/712: readlink d5/d14/d11/d21/d44/d80/d82/l97 0 2024-05-06T12:14:52.596 INFO:tasks.workunit.client.1.smithi190.stdout:9/707: rename db/dd/d16/d58/d87/d22/d49/d88/db5 to db/dd/d16/d58/d87/d22/d49/d88/d9f/df2 0 2024-05-06T12:14:52.597 INFO:tasks.workunit.client.1.smithi190.stdout:3/713: chown d5/d14/fe1 1688 1 2024-05-06T12:14:52.599 INFO:tasks.workunit.client.1.smithi190.stdout:4/477: dwrite d2/d10/f63 [4194304,4194304] 0 2024-05-06T12:14:52.600 INFO:tasks.workunit.client.1.smithi190.stdout:4/478: fsync d2/d11/d16/d61/f7c 0 2024-05-06T12:14:52.601 INFO:tasks.workunit.client.1.smithi190.stdout:1/513: creat d5/d4f/d65/d24/d43/fb4 x:0 0 0 2024-05-06T12:14:52.603 INFO:tasks.workunit.client.1.smithi190.stdout:3/714: truncate d5/d14/d11/d21/d44/d80/d82/fd3 1409762 0 2024-05-06T12:14:52.604 INFO:tasks.workunit.client.1.smithi190.stdout:4/479: write d2/d11/d16/d61/d66/d6b/f88 [5054279,89806] 0 2024-05-06T12:14:52.606 INFO:tasks.workunit.client.1.smithi190.stdout:9/708: creat db/dd/d16/d4c/d4e/d5e/ff3 x:0 0 0 2024-05-06T12:14:52.610 INFO:tasks.workunit.client.1.smithi190.stdout:1/514: creat d5/d4f/d65/d32/d47/d53/d6e/fb5 x:0 0 0 2024-05-06T12:14:52.618 INFO:tasks.workunit.client.1.smithi190.stdout:8/681: dread d3/d8/d25/f7b [0,4194304] 0 2024-05-06T12:14:52.618 INFO:tasks.workunit.client.1.smithi190.stdout:3/715: rename d5/d7/f71 to d5/ffe 0 2024-05-06T12:14:52.620 INFO:tasks.workunit.client.1.smithi190.stdout:1/515: creat d5/d4f/d65/d32/d47/d53/d98/da7/fb6 x:0 0 0 2024-05-06T12:14:52.620 INFO:tasks.workunit.client.1.smithi190.stdout:3/716: write d5/d14/d11/d21/fc1 [177113,115574] 0 2024-05-06T12:14:52.626 INFO:tasks.workunit.client.1.smithi190.stdout:0/669: dread d9/d51/dc9/fe5 [0,4194304] 0 2024-05-06T12:14:52.627 INFO:tasks.workunit.client.1.smithi190.stdout:3/717: truncate d5/d79/f95 318635 0 2024-05-06T12:14:52.629 INFO:tasks.workunit.client.1.smithi190.stdout:8/682: creat d3/d8/d25/d3b/d90/dcb/feb x:0 0 0 2024-05-06T12:14:52.630 INFO:tasks.workunit.client.1.smithi190.stdout:0/670: creat d9/dc/d28/d4c/d96/fed x:0 0 0 2024-05-06T12:14:52.633 INFO:tasks.workunit.client.1.smithi190.stdout:4/480: truncate d2/d11/d16/d31/f56 3908723 0 2024-05-06T12:14:52.634 INFO:tasks.workunit.client.1.smithi190.stdout:8/683: mknod d3/d8/dcf/cec 0 2024-05-06T12:14:52.634 INFO:tasks.workunit.client.1.smithi190.stdout:3/718: dwrite d5/d7/f73 [0,4194304] 0 2024-05-06T12:14:52.636 INFO:tasks.workunit.client.1.smithi190.stdout:8/684: write d3/d8/d68/fd8 [177668,63734] 0 2024-05-06T12:14:52.638 INFO:tasks.workunit.client.1.smithi190.stdout:0/671: rename d9/d6f/ce8 to d9/dc/d28/dc5/dd5/de4/cee 0 2024-05-06T12:14:52.639 INFO:tasks.workunit.client.1.smithi190.stdout:3/719: rmdir d5/d14 39 2024-05-06T12:14:52.645 INFO:tasks.workunit.client.1.smithi190.stdout:1/516: truncate d5/d4f/d65/d32/f61 571372 0 2024-05-06T12:14:52.645 INFO:tasks.workunit.client.1.smithi190.stdout:4/481: dwrite d2/d11/d16/d61/d66/f74 [0,4194304] 0 2024-05-06T12:14:52.651 INFO:tasks.workunit.client.1.smithi190.stdout:1/517: mknod d5/d15/d1e/cb7 0 2024-05-06T12:14:52.652 INFO:tasks.workunit.client.1.smithi190.stdout:1/518: chown d5/d6b/l97 522177 1 2024-05-06T12:14:52.653 INFO:tasks.workunit.client.1.smithi190.stdout:3/720: rename d5/d14/d11/d21/d44/d80/d82/lcf to d5/d14/d11/d21/d44/df8/lff 0 2024-05-06T12:14:52.655 INFO:tasks.workunit.client.1.smithi190.stdout:4/482: unlink d2/d11/d16/d61/d9e/d6e/f72 0 2024-05-06T12:14:52.661 INFO:tasks.workunit.client.1.smithi190.stdout:0/672: dread d9/d6f/f7a [0,4194304] 0 2024-05-06T12:14:52.674 INFO:tasks.workunit.client.1.smithi190.stdout:1/519: dread d5/f34 [0,4194304] 0 2024-05-06T12:14:52.681 INFO:tasks.workunit.client.1.smithi190.stdout:1/520: creat d5/d15/d1e/fb8 x:0 0 0 2024-05-06T12:14:52.682 INFO:tasks.workunit.client.1.smithi190.stdout:3/721: creat d5/d14/d11/d21/d44/d80/d82/d3d/ded/f100 x:0 0 0 2024-05-06T12:14:52.683 INFO:tasks.workunit.client.1.smithi190.stdout:3/722: fdatasync d5/d14/fe1 0 2024-05-06T12:14:52.684 INFO:tasks.workunit.client.1.smithi190.stdout:1/521: creat d5/d4f/d65/d24/d43/da6/fb9 x:0 0 0 2024-05-06T12:14:52.686 INFO:tasks.workunit.client.1.smithi190.stdout:3/723: rename d5/d14/d11/d21/d44/d92/fe6 to d5/d14/d11/d21/d44/d92/f101 0 2024-05-06T12:14:52.687 INFO:tasks.workunit.client.1.smithi190.stdout:1/522: dread d5/d15/fa2 [0,4194304] 0 2024-05-06T12:14:52.688 INFO:tasks.workunit.client.1.smithi190.stdout:1/523: dread - d5/d4f/f83 zero size 2024-05-06T12:14:52.689 INFO:tasks.workunit.client.1.smithi190.stdout:3/724: write d5/fc [1815089,12647] 0 2024-05-06T12:14:52.691 INFO:tasks.workunit.client.1.smithi190.stdout:1/524: mkdir d5/d4f/d65/d62/d9d/dba 0 2024-05-06T12:14:52.692 INFO:tasks.workunit.client.1.smithi190.stdout:1/525: chown d5/d4f/d65 1859 1 2024-05-06T12:14:52.693 INFO:tasks.workunit.client.1.smithi190.stdout:1/526: write d5/d4f/d65/d62/f93 [203480,87407] 0 2024-05-06T12:14:52.694 INFO:tasks.workunit.client.1.smithi190.stdout:3/725: creat d5/d14/d11/d21/d44/d80/d82/f102 x:0 0 0 2024-05-06T12:14:52.695 INFO:tasks.workunit.client.1.smithi190.stdout:3/726: dread - d5/d14/d11/d21/d44/fcd zero size 2024-05-06T12:14:52.705 INFO:tasks.workunit.client.1.smithi190.stdout:3/727: dwrite d5/d7/d15/fa9 [0,4194304] 0 2024-05-06T12:14:52.711 INFO:tasks.workunit.client.1.smithi190.stdout:0/673: dwrite d9/dcf/d21/d31/f7b [4194304,4194304] 0 2024-05-06T12:14:52.712 INFO:tasks.workunit.client.1.smithi190.stdout:3/728: mknod d5/d14/d11/d21/d44/d80/d82/c103 0 2024-05-06T12:14:52.713 INFO:tasks.workunit.client.1.smithi190.stdout:1/527: rmdir d5 39 2024-05-06T12:14:52.722 INFO:tasks.workunit.client.1.smithi190.stdout:0/674: truncate d9/dcf/f75 1331832 0 2024-05-06T12:14:52.722 INFO:tasks.workunit.client.1.smithi190.stdout:3/729: dwrite d5/d14/d11/d21/d44/d92/d9e/fee [0,4194304] 0 2024-05-06T12:14:52.724 INFO:tasks.workunit.client.1.smithi190.stdout:0/675: mknod d9/d6f/cef 0 2024-05-06T12:14:52.724 INFO:tasks.workunit.client.1.smithi190.stdout:3/730: write d5/d14/d11/d21/d44/d80/d82/f102 [1030872,65513] 0 2024-05-06T12:14:52.725 INFO:tasks.workunit.client.1.smithi190.stdout:0/676: chown d9/dcf/f62 0 1 2024-05-06T12:14:52.726 INFO:tasks.workunit.client.1.smithi190.stdout:0/677: readlink d9/d51/dc9/l8a 0 2024-05-06T12:14:52.728 INFO:tasks.workunit.client.1.smithi190.stdout:3/731: rmdir d5/d14/d11 39 2024-05-06T12:14:52.730 INFO:tasks.workunit.client.1.smithi190.stdout:0/678: creat d9/dc/d48/ff0 x:0 0 0 2024-05-06T12:14:52.730 INFO:tasks.workunit.client.1.smithi190.stdout:0/679: stat d9/dc/l46 0 2024-05-06T12:14:52.731 INFO:tasks.workunit.client.1.smithi190.stdout:3/732: fdatasync d5/d14/d11/d21/d44/d92/d9e/fa2 0 2024-05-06T12:14:52.736 INFO:tasks.workunit.client.1.smithi190.stdout:0/680: symlink d9/dc/d48/d50/d87/lf1 0 2024-05-06T12:14:52.736 INFO:tasks.workunit.client.1.smithi190.stdout:0/681: chown d9/d51/d93 0 1 2024-05-06T12:14:52.738 INFO:tasks.workunit.client.1.smithi190.stdout:0/682: write d9/dc/fb1 [1699320,15002] 0 2024-05-06T12:14:52.739 INFO:tasks.workunit.client.1.smithi190.stdout:3/733: dwrite d5/d14/d11/d21/d44/d80/d82/d3d/ded/f88 [0,4194304] 0 2024-05-06T12:14:52.744 INFO:tasks.workunit.client.1.smithi190.stdout:0/683: mkdir d9/dcf/df2 0 2024-05-06T12:14:52.746 INFO:tasks.workunit.client.1.smithi190.stdout:3/734: rename d5/d14/d11/d21/d44/d80/lf2 to d5/d79/l104 0 2024-05-06T12:14:52.748 INFO:tasks.workunit.client.1.smithi190.stdout:3/735: readlink d5/d14/d11/d21/d44/d80/d82/d3d/ded/d33/l37 0 2024-05-06T12:14:52.751 INFO:tasks.workunit.client.1.smithi190.stdout:3/736: rename d5/d14/d11/d21/d44/d80/d82/d3d/ded/d84 to d5/d14/d11/d21/d44/d92/d9e/da4/d105 0 2024-05-06T12:14:52.753 INFO:tasks.workunit.client.1.smithi190.stdout:3/737: mkdir d5/d7/d106 0 2024-05-06T12:14:52.753 INFO:tasks.workunit.client.1.smithi190.stdout:0/684: dread d9/dc/db6/fb7 [0,4194304] 0 2024-05-06T12:14:52.754 INFO:tasks.workunit.client.1.smithi190.stdout:0/685: write d9/dc/f74 [1111068,42623] 0 2024-05-06T12:14:52.756 INFO:tasks.workunit.client.1.smithi190.stdout:3/738: rmdir d5/d7/d15/d39 39 2024-05-06T12:14:52.759 INFO:tasks.workunit.client.1.smithi190.stdout:3/739: dread d5/d14/d11/d21/d44/d92/d9e/fa2 [0,4194304] 0 2024-05-06T12:14:52.759 INFO:tasks.workunit.client.1.smithi190.stdout:0/686: chown d9/dc/le7 34 1 2024-05-06T12:14:52.759 INFO:tasks.workunit.client.1.smithi190.stdout:3/740: write d5/d14/f6a [4251056,45864] 0 2024-05-06T12:14:52.762 INFO:tasks.workunit.client.1.smithi190.stdout:0/687: mknod d9/d6f/cf3 0 2024-05-06T12:14:52.763 INFO:tasks.workunit.client.1.smithi190.stdout:3/741: symlink d5/d7/d6d/l107 0 2024-05-06T12:14:52.766 INFO:tasks.workunit.client.1.smithi190.stdout:3/742: creat d5/d14/d11/d21/d44/d92/d9e/da4/df3/f108 x:0 0 0 2024-05-06T12:14:52.767 INFO:tasks.workunit.client.1.smithi190.stdout:0/688: dwrite d9/dc/fb1 [0,4194304] 0 2024-05-06T12:14:52.779 INFO:tasks.workunit.client.1.smithi190.stdout:3/743: dwrite d5/d7/d7f/f8d [0,4194304] 0 2024-05-06T12:14:52.779 INFO:tasks.workunit.client.1.smithi190.stdout:0/689: dwrite d9/dc/d28/f3e [0,4194304] 0 2024-05-06T12:14:52.789 INFO:tasks.workunit.client.1.smithi190.stdout:3/744: dwrite d5/d14/d11/d21/d44/d92/d9e/da4/d105/db3/da1/fc3 [0,4194304] 0 2024-05-06T12:14:52.789 INFO:tasks.workunit.client.1.smithi190.stdout:0/690: dwrite d9/dcf/d21/f2a [0,4194304] 0 2024-05-06T12:14:52.812 INFO:tasks.workunit.client.1.smithi190.stdout:3/745: read d5/d14/d11/fc7 [2847829,196] 0 2024-05-06T12:14:52.814 INFO:tasks.workunit.client.1.smithi190.stdout:3/746: chown d5/l7e 45 1 2024-05-06T12:14:52.815 INFO:tasks.workunit.client.1.smithi190.stdout:3/747: dread - d5/d14/d11/d21/d44/d80/d82/d3d/ded/d33/fe3 zero size 2024-05-06T12:14:52.816 INFO:tasks.workunit.client.1.smithi190.stdout:3/748: stat d5/d14/d11/d21/d44/d92/d9e/da4/d105/db3/f7d 0 2024-05-06T12:14:52.817 INFO:tasks.workunit.client.1.smithi190.stdout:3/749: truncate d5/d14/d11/d21/d44/d80/d82/fd3 1519126 0 2024-05-06T12:14:52.820 INFO:tasks.workunit.client.1.smithi190.stdout:3/750: mkdir d5/d14/d11/d21/deb/d109 0 2024-05-06T12:14:52.821 INFO:tasks.workunit.client.1.smithi190.stdout:3/751: fdatasync d5/d7/d15/fa9 0 2024-05-06T12:14:52.823 INFO:tasks.workunit.client.1.smithi190.stdout:0/691: dwrite d9/dcf/d56/dae/fb2 [0,4194304] 0 2024-05-06T12:14:52.824 INFO:tasks.workunit.client.1.smithi190.stdout:0/692: read - d9/dcf/d21/fd9 zero size 2024-05-06T12:14:52.829 INFO:tasks.workunit.client.1.smithi190.stdout:0/693: write d9/d51/dc9/fe5 [150442,12939] 0 2024-05-06T12:14:52.829 INFO:tasks.workunit.client.1.smithi190.stdout:3/752: creat d5/d14/d11/d21/d44/d80/d82/dfc/f10a x:0 0 0 2024-05-06T12:14:52.830 INFO:tasks.workunit.client.1.smithi190.stdout:3/753: write d5/d7/f23 [5194108,126019] 0 2024-05-06T12:14:52.832 INFO:tasks.workunit.client.1.smithi190.stdout:3/754: dread d5/d14/d11/d21/d44/d92/d9e/fa2 [0,4194304] 0 2024-05-06T12:14:52.832 INFO:tasks.workunit.client.1.smithi190.stdout:0/694: mknod d9/cf4 0 2024-05-06T12:14:52.835 INFO:tasks.workunit.client.1.smithi190.stdout:3/755: unlink d5/d14/d11/d21/d44/d80/d82/d3d/l3e 0 2024-05-06T12:14:52.836 INFO:tasks.workunit.client.1.smithi190.stdout:0/695: mkdir d9/dc/d48/d50/d87/dd2/dcc/df5 0 2024-05-06T12:14:52.837 INFO:tasks.workunit.client.1.smithi190.stdout:0/696: chown d9/dc/dd0/fd8 163093769 1 2024-05-06T12:14:52.837 INFO:tasks.workunit.client.1.smithi190.stdout:3/756: symlink d5/d7/d15/l10b 0 2024-05-06T12:14:52.847 INFO:tasks.workunit.client.1.smithi190.stdout:0/697: dwrite d9/d51/f55 [0,4194304] 0 2024-05-06T12:14:52.850 INFO:tasks.workunit.client.1.smithi190.stdout:3/757: dwrite d5/d14/d11/d21/d44/d80/d82/d3d/ded/d33/fe3 [0,4194304] 0 2024-05-06T12:14:52.854 INFO:tasks.workunit.client.1.smithi190.stdout:0/698: mknod d9/dcf/d56/dae/cf6 0 2024-05-06T12:14:52.883 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:52 smithi092 ceph-mon[26954]: pgmap v28: 65 pgs: 65 active+clean; 2.9 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 95 MiB/s rd, 250 MiB/s wr, 521 op/s 2024-05-06T12:14:52.883 INFO:tasks.workunit.client.1.smithi190.stdout:5/659: sync 2024-05-06T12:14:52.883 INFO:tasks.workunit.client.1.smithi190.stdout:6/473: sync 2024-05-06T12:14:52.884 INFO:tasks.workunit.client.1.smithi190.stdout:6/474: write df/f22 [279235,129841] 0 2024-05-06T12:14:52.887 INFO:tasks.workunit.client.1.smithi190.stdout:6/475: unlink df/d2c/f39 0 2024-05-06T12:14:52.888 INFO:tasks.workunit.client.1.smithi190.stdout:3/758: dread d5/d14/d11/d21/d44/d92/fb7 [0,4194304] 0 2024-05-06T12:14:52.897 INFO:tasks.workunit.client.1.smithi190.stdout:5/660: dread d2/da/f2f [0,4194304] 0 2024-05-06T12:14:52.898 INFO:tasks.workunit.client.1.smithi190.stdout:5/661: truncate d2/da/d59/d5f/d6e/fc8 756631 0 2024-05-06T12:14:52.999 INFO:tasks.workunit.client.1.smithi190.stdout:3/759: truncate d5/d14/d11/d21/d44/d80/d82/d3d/ded/fe 3083840 0 2024-05-06T12:14:53.001 INFO:tasks.workunit.client.1.smithi190.stdout:5/662: unlink d2/d27/f48 0 2024-05-06T12:14:53.006 INFO:tasks.workunit.client.1.smithi190.stdout:3/760: dwrite d5/d7/fb9 [0,4194304] 0 2024-05-06T12:14:53.007 INFO:tasks.workunit.client.1.smithi190.stdout:3/761: stat d5/d7/l29 0 2024-05-06T12:14:53.047 INFO:tasks.workunit.client.1.smithi190.stdout:2/640: sync 2024-05-06T12:14:53.048 INFO:tasks.workunit.client.1.smithi190.stdout:6/476: dwrite df/d2c/f36 [0,4194304] 0 2024-05-06T12:14:53.049 INFO:tasks.workunit.client.1.smithi190.stdout:3/762: fdatasync d5/d7/fb9 0 2024-05-06T12:14:53.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:52 smithi190 ceph-mon[33798]: pgmap v28: 65 pgs: 65 active+clean; 2.9 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 95 MiB/s rd, 250 MiB/s wr, 521 op/s 2024-05-06T12:14:53.141 INFO:tasks.workunit.client.1.smithi190.stdout:3/763: read d5/d14/d11/d21/d44/d80/d82/d3d/ded/f3b [2544546,31773] 0 2024-05-06T12:14:53.141 INFO:tasks.workunit.client.1.smithi190.stdout:5/663: dwrite d2/da/d59/d5f/d2e/d3b/d5a/f85 [0,4194304] 0 2024-05-06T12:14:53.142 INFO:tasks.workunit.client.1.smithi190.stdout:2/641: rename d2/d21/f85 to d2/d8/d19/fd9 0 2024-05-06T12:14:53.143 INFO:tasks.workunit.client.1.smithi190.stdout:3/764: write d5/d14/d11/d21/d44/d92/d9e/fa2 [930837,5501] 0 2024-05-06T12:14:53.147 INFO:tasks.workunit.client.1.smithi190.stdout:2/642: creat d2/d8/d19/fda x:0 0 0 2024-05-06T12:14:53.148 INFO:tasks.workunit.client.1.smithi190.stdout:5/664: getdents d2/da/d59/d5f/d2e/da3 0 2024-05-06T12:14:53.148 INFO:tasks.workunit.client.1.smithi190.stdout:2/643: chown d2/d8/d19/d43/fb3 11841284 1 2024-05-06T12:14:53.152 INFO:tasks.workunit.client.1.smithi190.stdout:3/765: dwrite d5/d14/d11/d21/d44/d92/d9e/fa2 [0,4194304] 0 2024-05-06T12:14:53.152 INFO:tasks.workunit.client.1.smithi190.stdout:2/644: symlink d2/d8/d33/ldb 0 2024-05-06T12:14:53.154 INFO:tasks.workunit.client.1.smithi190.stdout:5/665: rename d2/da/d59/d5f/f36 to d2/da/d59/d5f/d2e/d3b/fd4 0 2024-05-06T12:14:53.154 INFO:tasks.workunit.client.1.smithi190.stdout:2/645: truncate d2/d8/d33/d46/f8d 574401 0 2024-05-06T12:14:53.155 INFO:tasks.workunit.client.1.smithi190.stdout:3/766: mknod d5/d79/c10c 0 2024-05-06T12:14:53.157 INFO:tasks.workunit.client.1.smithi190.stdout:2/646: creat d2/d8/fdc x:0 0 0 2024-05-06T12:14:53.158 INFO:tasks.workunit.client.1.smithi190.stdout:3/767: rename d5/d14/d11/c5d to d5/d14/d11/d21/d44/d80/ddb/c10d 0 2024-05-06T12:14:53.159 INFO:tasks.workunit.client.1.smithi190.stdout:5/666: getdents d2/da/d59/d5f/d2e/dc6 0 2024-05-06T12:14:53.162 INFO:tasks.workunit.client.1.smithi190.stdout:2/647: write d2/d8/d33/d46/f8d [1573345,96354] 0 2024-05-06T12:14:53.165 INFO:tasks.workunit.client.1.smithi190.stdout:3/768: dread d5/d14/d11/f9f [0,4194304] 0 2024-05-06T12:14:53.168 INFO:tasks.workunit.client.1.smithi190.stdout:6/477: dread df/d2c/d1e/d57/f6a [0,4194304] 0 2024-05-06T12:14:53.170 INFO:tasks.workunit.client.1.smithi190.stdout:3/769: stat d5/d14/d11/d21/d44/d80/d82/d3d/ded/d33/l37 0 2024-05-06T12:14:53.170 INFO:tasks.workunit.client.1.smithi190.stdout:6/478: write df/d17/d37/f58 [1016553,29446] 0 2024-05-06T12:14:53.170 INFO:tasks.workunit.client.1.smithi190.stdout:2/648: dwrite d2/d21/f9f [0,4194304] 0 2024-05-06T12:14:53.176 INFO:tasks.workunit.client.1.smithi190.stdout:6/479: mknod df/ca6 0 2024-05-06T12:14:53.178 INFO:tasks.workunit.client.1.smithi190.stdout:7/659: sync 2024-05-06T12:14:53.180 INFO:tasks.workunit.client.1.smithi190.stdout:2/649: getdents d2/d8/d33/d46/dab/d91 0 2024-05-06T12:14:53.184 INFO:tasks.workunit.client.1.smithi190.stdout:6/480: dwrite df/f82 [0,4194304] 0 2024-05-06T12:14:53.190 INFO:tasks.workunit.client.1.smithi190.stdout:7/660: getdents d0/da 0 2024-05-06T12:14:53.191 INFO:tasks.workunit.client.1.smithi190.stdout:5/667: dread d2/da/d59/d5f/d50/d6d/db3/fb4 [0,4194304] 0 2024-05-06T12:14:53.193 INFO:tasks.workunit.client.1.smithi190.stdout:6/481: dread df/d17/d44/f54 [0,4194304] 0 2024-05-06T12:14:53.194 INFO:tasks.workunit.client.1.smithi190.stdout:2/650: dwrite d2/d8/dc/f95 [0,4194304] 0 2024-05-06T12:14:53.196 INFO:tasks.workunit.client.1.smithi190.stdout:5/668: unlink d2/c37 0 2024-05-06T12:14:53.196 INFO:tasks.workunit.client.1.smithi190.stdout:5/669: chown d2/da/d59/d5f/d50 0 1 2024-05-06T12:14:53.198 INFO:tasks.workunit.client.1.smithi190.stdout:7/661: symlink d0/d1/d18/d6c/de1/de7/led 0 2024-05-06T12:14:53.198 INFO:tasks.workunit.client.1.smithi190.stdout:2/651: mknod d2/d8/d19/d4f/cdd 0 2024-05-06T12:14:53.200 INFO:tasks.workunit.client.1.smithi190.stdout:6/482: link df/d2c/d61/f70 df/d17/d44/fa7 0 2024-05-06T12:14:53.202 INFO:tasks.workunit.client.1.smithi190.stdout:6/483: dread - df/d17/d37/f9c zero size 2024-05-06T12:14:53.204 INFO:tasks.workunit.client.1.smithi190.stdout:5/670: dwrite d2/da/d59/d5f/d6e/fc8 [0,4194304] 0 2024-05-06T12:14:53.208 INFO:tasks.workunit.client.1.smithi190.stdout:5/671: rename d2/d6c/cd0 to d2/d65/cd5 0 2024-05-06T12:14:53.208 INFO:tasks.workunit.client.1.smithi190.stdout:5/672: stat d2/da/d59/d5f/d2e/da7 0 2024-05-06T12:14:53.211 INFO:tasks.workunit.client.1.smithi190.stdout:5/673: creat d2/da/d59/d5f/d2e/d8b/fd6 x:0 0 0 2024-05-06T12:14:53.212 INFO:tasks.workunit.client.1.smithi190.stdout:5/674: mknod d2/da/d59/cd7 0 2024-05-06T12:14:53.224 INFO:tasks.workunit.client.1.smithi190.stdout:6/484: dwrite df/d2c/d61/f79 [0,4194304] 0 2024-05-06T12:14:53.227 INFO:tasks.workunit.client.1.smithi190.stdout:6/485: link df/d2c/d61/f79 df/d17/d44/fa8 0 2024-05-06T12:14:53.231 INFO:tasks.workunit.client.1.smithi190.stdout:6/486: link f1 df/d2c/fa9 0 2024-05-06T12:14:53.233 INFO:tasks.workunit.client.1.smithi190.stdout:5/675: dread d2/d27/f54 [0,4194304] 0 2024-05-06T12:14:53.234 INFO:tasks.workunit.client.1.smithi190.stdout:7/662: dread d0/d14/d6a/d8f/fdb [0,4194304] 0 2024-05-06T12:14:53.236 INFO:tasks.workunit.client.1.smithi190.stdout:6/487: link df/d2c/d92/f93 df/d2c/d61/d84/faa 0 2024-05-06T12:14:53.236 INFO:tasks.workunit.client.1.smithi190.stdout:6/488: chown df/d17/f26 77656799 1 2024-05-06T12:14:53.237 INFO:tasks.workunit.client.1.smithi190.stdout:7/663: rmdir d0/d1/d18/d4e 39 2024-05-06T12:14:53.239 INFO:tasks.workunit.client.1.smithi190.stdout:5/676: symlink d2/da/d59/d5f/ld8 0 2024-05-06T12:14:53.241 INFO:tasks.workunit.client.1.smithi190.stdout:7/664: creat d0/d1/d18/d37/fee x:0 0 0 2024-05-06T12:14:53.242 INFO:tasks.workunit.client.1.smithi190.stdout:5/677: creat d2/da/d59/d5f/d50/d6d/db3/fd9 x:0 0 0 2024-05-06T12:14:53.245 INFO:tasks.workunit.client.1.smithi190.stdout:6/489: getdents df/d17/d37 0 2024-05-06T12:14:53.246 INFO:tasks.workunit.client.1.smithi190.stdout:7/665: dread d0/d4/f6b [0,4194304] 0 2024-05-06T12:14:53.247 INFO:tasks.workunit.client.1.smithi190.stdout:6/490: dread df/d2c/d1e/d57/f6a [4194304,4194304] 0 2024-05-06T12:14:53.249 INFO:tasks.workunit.client.1.smithi190.stdout:6/491: dread - df/d17/d37/f8c zero size 2024-05-06T12:14:53.250 INFO:tasks.workunit.client.1.smithi190.stdout:7/666: symlink d0/d1/d32/lef 0 2024-05-06T12:14:53.252 INFO:tasks.workunit.client.1.smithi190.stdout:6/492: creat df/d2c/d61/fab x:0 0 0 2024-05-06T12:14:53.256 INFO:tasks.workunit.client.1.smithi190.stdout:5/678: dwrite d2/da/d59/d5f/faf [0,4194304] 0 2024-05-06T12:14:53.280 INFO:tasks.workunit.client.1.smithi190.stdout:5/679: dread d2/f30 [0,4194304] 0 2024-05-06T12:14:53.282 INFO:tasks.workunit.client.1.smithi190.stdout:5/680: creat d2/da/d59/d5f/d2e/da7/d9d/fda x:0 0 0 2024-05-06T12:14:53.318 INFO:tasks.workunit.client.1.smithi190.stdout:8/685: sync 2024-05-06T12:14:53.319 INFO:tasks.workunit.client.1.smithi190.stdout:9/709: sync 2024-05-06T12:14:53.319 INFO:tasks.workunit.client.1.smithi190.stdout:9/710: chown db/dd/d16/d58/d87/d99/cc0 436292 1 2024-05-06T12:14:53.320 INFO:tasks.workunit.client.1.smithi190.stdout:9/711: truncate db/d23/d7e/fea 704597 0 2024-05-06T12:14:53.322 INFO:tasks.workunit.client.1.smithi190.stdout:8/686: rename d3/d8/d25/d3b/d90/dcb/feb to d3/d8/d25/d65/dc6/fed 0 2024-05-06T12:14:53.327 INFO:tasks.workunit.client.1.smithi190.stdout:9/712: mknod db/dd/cf4 0 2024-05-06T12:14:53.328 INFO:tasks.workunit.client.1.smithi190.stdout:8/687: creat d3/d35/d55/d61/fee x:0 0 0 2024-05-06T12:14:53.329 INFO:tasks.workunit.client.1.smithi190.stdout:9/713: chown db/dd/d16/d58/f91 4472535 1 2024-05-06T12:14:53.329 INFO:tasks.workunit.client.1.smithi190.stdout:8/688: symlink d3/d35/lef 0 2024-05-06T12:14:53.331 INFO:tasks.workunit.client.1.smithi190.stdout:9/714: mknod db/dd/d5c/cf5 0 2024-05-06T12:14:53.333 INFO:tasks.workunit.client.1.smithi190.stdout:9/715: symlink db/d23/lf6 0 2024-05-06T12:14:53.410 INFO:tasks.workunit.client.1.smithi190.stdout:8/689: dread d3/f18 [0,4194304] 0 2024-05-06T12:14:53.411 INFO:tasks.workunit.client.1.smithi190.stdout:8/690: dread - d3/d35/d3a/fba zero size 2024-05-06T12:14:53.412 INFO:tasks.workunit.client.1.smithi190.stdout:9/716: dwrite db/f1f [0,4194304] 0 2024-05-06T12:14:53.418 INFO:tasks.workunit.client.1.smithi190.stdout:8/691: link d3/fd7 d3/d8/d25/d65/dc6/ff0 0 2024-05-06T12:14:53.419 INFO:tasks.workunit.client.1.smithi190.stdout:8/692: chown d3/d8/d25/d3b/d6b/ca1 21497 1 2024-05-06T12:14:53.426 INFO:tasks.workunit.client.1.smithi190.stdout:9/717: dwrite db/dd/d16/d58/f7d [0,4194304] 0 2024-05-06T12:14:53.430 INFO:tasks.workunit.client.1.smithi190.stdout:4/483: sync 2024-05-06T12:14:53.431 INFO:tasks.workunit.client.1.smithi190.stdout:1/528: sync 2024-05-06T12:14:53.433 INFO:tasks.workunit.client.1.smithi190.stdout:8/693: dwrite d3/d8/d25/d65/f93 [0,4194304] 0 2024-05-06T12:14:53.435 INFO:tasks.workunit.client.1.smithi190.stdout:8/694: write d3/d8/fd [9288908,54929] 0 2024-05-06T12:14:53.435 INFO:tasks.workunit.client.1.smithi190.stdout:4/484: symlink d2/d11/d16/d61/la7 0 2024-05-06T12:14:53.437 INFO:tasks.workunit.client.1.smithi190.stdout:1/529: creat d5/d4f/d65/d32/d47/d53/d98/fbb x:0 0 0 2024-05-06T12:14:53.438 INFO:tasks.workunit.client.1.smithi190.stdout:8/695: chown d3/lae 686273884 1 2024-05-06T12:14:53.439 INFO:tasks.workunit.client.1.smithi190.stdout:4/485: mknod d2/d11/d16/d4e/ca8 0 2024-05-06T12:14:53.440 INFO:tasks.workunit.client.1.smithi190.stdout:4/486: truncate d2/f59 671222 0 2024-05-06T12:14:53.440 INFO:tasks.workunit.client.1.smithi190.stdout:8/696: fsync d3/fc 0 2024-05-06T12:14:53.442 INFO:tasks.workunit.client.1.smithi190.stdout:4/487: write d2/d11/d16/d61/d66/d6b/f88 [5084844,105683] 0 2024-05-06T12:14:53.442 INFO:tasks.workunit.client.1.smithi190.stdout:8/697: creat d3/d35/d55/d61/ff1 x:0 0 0 2024-05-06T12:14:53.446 INFO:tasks.workunit.client.1.smithi190.stdout:4/488: symlink d2/d10/la9 0 2024-05-06T12:14:53.446 INFO:tasks.workunit.client.1.smithi190.stdout:1/530: link d5/d4f/d65/d32/d47/d53/d6e/fae d5/d4f/d65/d2b/fbc 0 2024-05-06T12:14:53.447 INFO:tasks.workunit.client.1.smithi190.stdout:8/698: chown d3/d35/c57 29626066 1 2024-05-06T12:14:53.448 INFO:tasks.workunit.client.1.smithi190.stdout:9/718: dread db/f26 [0,4194304] 0 2024-05-06T12:14:53.450 INFO:tasks.workunit.client.1.smithi190.stdout:4/489: mkdir d2/d11/d16/d61/d9e/d6e/daa 0 2024-05-06T12:14:53.450 INFO:tasks.workunit.client.1.smithi190.stdout:9/719: chown db/dd/d16/d4c/d4e/d5e/c73 6348 1 2024-05-06T12:14:53.455 INFO:tasks.workunit.client.1.smithi190.stdout:1/531: dwrite d5/d4f/f83 [0,4194304] 0 2024-05-06T12:14:53.457 INFO:tasks.workunit.client.1.smithi190.stdout:9/720: truncate db/dd/d16/d4c/d4e/f9d 4714512 0 2024-05-06T12:14:53.458 INFO:tasks.workunit.client.1.smithi190.stdout:8/699: dwrite d3/d8/d25/d3b/d6b/d7a/fe3 [0,4194304] 0 2024-05-06T12:14:53.460 INFO:tasks.workunit.client.1.smithi190.stdout:4/490: rename d2/d11/d16/d8e/c8a to d2/d11/d16/d61/d9e/d6e/d78/cab 0 2024-05-06T12:14:53.463 INFO:tasks.workunit.client.1.smithi190.stdout:9/721: truncate db/dd/d16/d58/ddc/d47/d4b/f43 913982 0 2024-05-06T12:14:53.466 INFO:tasks.workunit.client.1.smithi190.stdout:1/532: mknod d5/d15/cbd 0 2024-05-06T12:14:53.468 INFO:tasks.workunit.client.1.smithi190.stdout:4/491: rename d2/d11/l1d to d2/d11/d16/d31/lac 0 2024-05-06T12:14:53.470 INFO:tasks.workunit.client.1.smithi190.stdout:4/492: chown d2/d11/d16/d61/d9e/la6 260623595 1 2024-05-06T12:14:53.471 INFO:tasks.workunit.client.1.smithi190.stdout:1/533: read d5/d4f/d65/d62/f8d [564458,126568] 0 2024-05-06T12:14:53.471 INFO:tasks.workunit.client.1.smithi190.stdout:1/534: chown d5/d15/d1e/fb8 232 1 2024-05-06T12:14:53.475 INFO:tasks.workunit.client.1.smithi190.stdout:8/700: getdents d3 0 2024-05-06T12:14:53.477 INFO:tasks.workunit.client.1.smithi190.stdout:9/722: link db/dd/d16/d58/d87/d22/d49/d88/d76/fc8 db/d23/da4/ff7 0 2024-05-06T12:14:53.478 INFO:tasks.workunit.client.1.smithi190.stdout:4/493: symlink d2/d11/d16/lad 0 2024-05-06T12:14:53.479 INFO:tasks.workunit.client.1.smithi190.stdout:4/494: write d2/d11/d16/d41/f62 [471031,64117] 0 2024-05-06T12:14:53.480 INFO:tasks.workunit.client.1.smithi190.stdout:9/723: readlink db/d23/d29/l67 0 2024-05-06T12:14:53.481 INFO:tasks.workunit.client.1.smithi190.stdout:1/535: getdents d5/d4f/d65/d2b 0 2024-05-06T12:14:53.481 INFO:tasks.workunit.client.1.smithi190.stdout:0/699: sync 2024-05-06T12:14:53.481 INFO:tasks.workunit.client.1.smithi190.stdout:3/770: sync 2024-05-06T12:14:53.483 INFO:tasks.workunit.client.1.smithi190.stdout:3/771: stat d5/d14/d11/d21/d44/d92/fb7 0 2024-05-06T12:14:53.483 INFO:tasks.workunit.client.1.smithi190.stdout:8/701: link d3/d8/d25/d3b/d90/f7c d3/d35/d3a/db9/ff2 0 2024-05-06T12:14:53.485 INFO:tasks.workunit.client.1.smithi190.stdout:4/495: creat d2/d11/d16/d61/d66/fae x:0 0 0 2024-05-06T12:14:53.490 INFO:tasks.workunit.client.1.smithi190.stdout:3/772: chown d5/d14/ce7 7435969 1 2024-05-06T12:14:53.491 INFO:tasks.workunit.client.1.smithi190.stdout:9/724: mknod db/d23/cf8 0 2024-05-06T12:14:53.497 INFO:tasks.workunit.client.1.smithi190.stdout:0/700: truncate d9/f2c 543869 0 2024-05-06T12:14:53.498 INFO:tasks.workunit.client.1.smithi190.stdout:3/773: mknod d5/d14/d11/d21/deb/c10e 0 2024-05-06T12:14:53.502 INFO:tasks.workunit.client.1.smithi190.stdout:4/496: dread d2/d11/f14 [0,4194304] 0 2024-05-06T12:14:53.503 INFO:tasks.workunit.client.1.smithi190.stdout:4/497: write d2/d10/f99 [921905,126678] 0 2024-05-06T12:14:53.503 INFO:tasks.workunit.client.1.smithi190.stdout:1/536: dread d5/d4f/d65/d24/d43/f60 [0,4194304] 0 2024-05-06T12:14:53.504 INFO:tasks.workunit.client.1.smithi190.stdout:0/701: unlink d9/dc/d48/fce 0 2024-05-06T12:14:53.505 INFO:tasks.workunit.client.1.smithi190.stdout:0/702: dread - d9/feb zero size 2024-05-06T12:14:53.509 INFO:tasks.workunit.client.1.smithi190.stdout:3/774: creat d5/d14/d11/dc9/f10f x:0 0 0 2024-05-06T12:14:53.511 INFO:tasks.workunit.client.1.smithi190.stdout:0/703: fsync d9/dcf/d21/f49 0 2024-05-06T12:14:53.512 INFO:tasks.workunit.client.1.smithi190.stdout:1/537: dread d5/d4f/d65/d24/f25 [0,4194304] 0 2024-05-06T12:14:53.512 INFO:tasks.workunit.client.1.smithi190.stdout:4/498: mknod d2/d11/d16/caf 0 2024-05-06T12:14:53.514 INFO:tasks.workunit.client.1.smithi190.stdout:1/538: stat d5/d4f/d65/d32/d47/f96 0 2024-05-06T12:14:53.515 INFO:tasks.workunit.client.1.smithi190.stdout:3/775: rename d5/d14/d11/d21/d44/fcd to d5/d14/d11/d21/d44/d92/f110 0 2024-05-06T12:14:53.516 INFO:tasks.workunit.client.1.smithi190.stdout:9/725: dread db/dd/d16/d4c/d4e/f62 [0,4194304] 0 2024-05-06T12:14:53.517 INFO:tasks.workunit.client.1.smithi190.stdout:9/726: chown db/dd/d16/d58/d87/d22/d49/f98 36768 1 2024-05-06T12:14:53.523 INFO:tasks.workunit.client.1.smithi190.stdout:3/776: mkdir d5/d14/d11/d21/d44/d92/d9e/da4/d105/d111 0 2024-05-06T12:14:53.525 INFO:tasks.workunit.client.1.smithi190.stdout:0/704: symlink d9/dcf/d21/lf7 0 2024-05-06T12:14:53.525 INFO:tasks.workunit.client.1.smithi190.stdout:0/705: chown d9 6 1 2024-05-06T12:14:53.525 INFO:tasks.workunit.client.1.smithi190.stdout:3/777: dread d5/d14/d11/d21/d44/f5b [0,4194304] 0 2024-05-06T12:14:53.526 INFO:tasks.workunit.client.1.smithi190.stdout:9/727: mkdir db/dd/d16/d58/d87/d22/d49/d8c/df9 0 2024-05-06T12:14:53.527 INFO:tasks.workunit.client.1.smithi190.stdout:1/539: dwrite d5/d4f/d65/d24/f25 [0,4194304] 0 2024-05-06T12:14:53.529 INFO:tasks.workunit.client.1.smithi190.stdout:9/728: dread - db/dd/d16/d58/d87/d99/fc9 zero size 2024-05-06T12:14:53.538 INFO:tasks.workunit.client.1.smithi190.stdout:0/706: dwrite d9/dc/d48/d50/f66 [0,4194304] 0 2024-05-06T12:14:53.544 INFO:tasks.workunit.client.1.smithi190.stdout:0/707: creat d9/dc/d28/dc5/dd5/ff8 x:0 0 0 2024-05-06T12:14:53.548 INFO:tasks.workunit.client.1.smithi190.stdout:4/499: truncate d2/d11/d16/f38 2835479 0 2024-05-06T12:14:53.556 INFO:tasks.workunit.client.1.smithi190.stdout:4/500: dread d2/d11/d16/d61/d66/f74 [0,4194304] 0 2024-05-06T12:14:53.559 INFO:tasks.workunit.client.1.smithi190.stdout:1/540: dwrite d5/d4f/d65/d32/d47/f57 [0,4194304] 0 2024-05-06T12:14:53.562 INFO:tasks.workunit.client.1.smithi190.stdout:1/541: readlink d5/d15/la5 0 2024-05-06T12:14:53.564 INFO:tasks.workunit.client.1.smithi190.stdout:9/729: dwrite db/f26 [0,4194304] 0 2024-05-06T12:14:53.564 INFO:tasks.workunit.client.1.smithi190.stdout:3/778: dwrite d5/f62 [0,4194304] 0 2024-05-06T12:14:53.567 INFO:tasks.workunit.client.1.smithi190.stdout:9/730: fsync db/dd/d16/fa6 0 2024-05-06T12:14:53.572 INFO:tasks.workunit.client.1.smithi190.stdout:4/501: rename d2/d11/f20 to d2/fb0 0 2024-05-06T12:14:53.573 INFO:tasks.workunit.client.1.smithi190.stdout:1/542: unlink d5/d4f/d65/d32/d47/d53/d6e/f6f 0 2024-05-06T12:14:53.574 INFO:tasks.workunit.client.1.smithi190.stdout:9/731: dread - db/dd/fe5 zero size 2024-05-06T12:14:53.574 INFO:tasks.workunit.client.1.smithi190.stdout:0/708: dread d9/f5f [0,4194304] 0 2024-05-06T12:14:53.575 INFO:tasks.workunit.client.1.smithi190.stdout:3/779: symlink d5/d7/d15/l112 0 2024-05-06T12:14:53.582 INFO:tasks.workunit.client.1.smithi190.stdout:4/502: rename d2/c3f to d2/d7d/cb1 0 2024-05-06T12:14:53.587 INFO:tasks.workunit.client.1.smithi190.stdout:3/780: creat d5/d14/d11/d21/d44/d92/d9e/da4/d105/db3/dfa/f113 x:0 0 0 2024-05-06T12:14:53.589 INFO:tasks.workunit.client.1.smithi190.stdout:1/543: dwrite d5/f6d [0,4194304] 0 2024-05-06T12:14:53.590 INFO:tasks.workunit.client.1.smithi190.stdout:6/493: sync 2024-05-06T12:14:53.590 INFO:tasks.workunit.client.1.smithi190.stdout:5/681: sync 2024-05-06T12:14:53.590 INFO:tasks.workunit.client.1.smithi190.stdout:7/667: sync 2024-05-06T12:14:53.590 INFO:tasks.workunit.client.1.smithi190.stdout:2/652: sync 2024-05-06T12:14:53.590 INFO:tasks.workunit.client.1.smithi190.stdout:9/732: dwrite db/dd/d16/d4c/f9b [0,4194304] 0 2024-05-06T12:14:53.590 INFO:tasks.workunit.client.1.smithi190.stdout:0/709: creat d9/d51/d93/ff9 x:0 0 0 2024-05-06T12:14:53.597 INFO:tasks.workunit.client.1.smithi190.stdout:4/503: dread d2/d11/d16/d8e/f57 [0,4194304] 0 2024-05-06T12:14:53.598 INFO:tasks.workunit.client.1.smithi190.stdout:7/668: write d0/d1/f3e [5001242,27547] 0 2024-05-06T12:14:53.598 INFO:tasks.workunit.client.1.smithi190.stdout:3/781: write d5/d7/d15/fa9 [4538378,22294] 0 2024-05-06T12:14:53.603 INFO:tasks.workunit.client.1.smithi190.stdout:6/494: dread df/f2e [0,4194304] 0 2024-05-06T12:14:53.604 INFO:tasks.workunit.client.1.smithi190.stdout:0/710: creat d9/dc/d48/d50/d87/dd2/ffa x:0 0 0 2024-05-06T12:14:53.604 INFO:tasks.workunit.client.1.smithi190.stdout:7/669: chown d0/d1/d18/d37 22532 1 2024-05-06T12:14:53.612 INFO:tasks.workunit.client.1.smithi190.stdout:2/653: dwrite d2/d8/f92 [0,4194304] 0 2024-05-06T12:14:53.613 INFO:tasks.workunit.client.1.smithi190.stdout:4/504: getdents d2/d11/d16/d8e/d80/da0 0 2024-05-06T12:14:53.617 INFO:tasks.workunit.client.1.smithi190.stdout:5/682: write d2/d6c/d7d/fa2 [168095,30203] 0 2024-05-06T12:14:53.617 INFO:tasks.workunit.client.1.smithi190.stdout:6/495: truncate f1 23535 0 2024-05-06T12:14:53.618 INFO:tasks.workunit.client.1.smithi190.stdout:5/683: chown d2/da/d59/d5f/d50/d6d 292 1 2024-05-06T12:14:53.619 INFO:tasks.workunit.client.1.smithi190.stdout:2/654: mkdir d2/d8/d19/d7b/dde 0 2024-05-06T12:14:53.621 INFO:tasks.workunit.client.1.smithi190.stdout:7/670: creat d0/da/d36/ff0 x:0 0 0 2024-05-06T12:14:53.622 INFO:tasks.workunit.client.1.smithi190.stdout:9/733: link db/dd/d16/d58/ddc/l27 db/d23/d29/lfa 0 2024-05-06T12:14:53.625 INFO:tasks.workunit.client.1.smithi190.stdout:4/505: mkdir d2/d11/d16/d8e/d80/da0/db2 0 2024-05-06T12:14:53.629 INFO:tasks.workunit.client.1.smithi190.stdout:2/655: symlink d2/d4c/ldf 0 2024-05-06T12:14:53.630 INFO:tasks.workunit.client.1.smithi190.stdout:7/671: rmdir d0/d1/d2 39 2024-05-06T12:14:53.632 INFO:tasks.workunit.client.1.smithi190.stdout:9/734: symlink db/dd/d16/d4c/d4e/d5e/lfb 0 2024-05-06T12:14:53.633 INFO:tasks.workunit.client.1.smithi190.stdout:6/496: dwrite df/d17/f5c [4194304,4194304] 0 2024-05-06T12:14:53.635 INFO:tasks.workunit.client.1.smithi190.stdout:2/656: fsync d2/d8/d19/d43/d79/f82 0 2024-05-06T12:14:53.646 INFO:tasks.workunit.client.1.smithi190.stdout:4/506: dwrite d2/d11/d16/d8e/f57 [0,4194304] 0 2024-05-06T12:14:53.653 INFO:tasks.workunit.client.1.smithi190.stdout:3/782: dread d5/d7/d15/d39/f9b [0,4194304] 0 2024-05-06T12:14:53.655 INFO:tasks.workunit.client.1.smithi190.stdout:9/735: dwrite db/dd/d16/d58/d87/fcb [0,4194304] 0 2024-05-06T12:14:53.655 INFO:tasks.workunit.client.1.smithi190.stdout:5/684: write d2/da/d59/d5f/d2e/d3b/f42 [1227724,81932] 0 2024-05-06T12:14:53.658 INFO:tasks.workunit.client.1.smithi190.stdout:7/672: truncate d0/da/d36/d53/d77/d8c/dd1/d60/fc2 411407 0 2024-05-06T12:14:53.658 INFO:tasks.workunit.client.1.smithi190.stdout:4/507: symlink d2/d11/lb3 0 2024-05-06T12:14:53.660 INFO:tasks.workunit.client.1.smithi190.stdout:2/657: dwrite d2/f12 [4194304,4194304] 0 2024-05-06T12:14:53.669 INFO:tasks.workunit.client.1.smithi190.stdout:5/685: rmdir d2/da/d59/d5f/d2e/da7 39 2024-05-06T12:14:53.672 INFO:tasks.workunit.client.1.smithi190.stdout:2/658: write d2/d8/d19/fa7 [980537,47775] 0 2024-05-06T12:14:53.674 INFO:tasks.workunit.client.1.smithi190.stdout:6/497: dread df/d2c/d1e/f52 [0,4194304] 0 2024-05-06T12:14:53.688 INFO:tasks.workunit.client.1.smithi190.stdout:8/702: sync 2024-05-06T12:14:53.688 INFO:tasks.workunit.client.1.smithi190.stdout:8/703: chown d3/fb1 11098 1 2024-05-06T12:14:53.690 INFO:tasks.workunit.client.1.smithi190.stdout:3/783: symlink d5/d7/d15/d39/l114 0 2024-05-06T12:14:53.694 INFO:tasks.workunit.client.1.smithi190.stdout:2/659: symlink d2/d8/d19/da9/le0 0 2024-05-06T12:14:53.696 INFO:tasks.workunit.client.1.smithi190.stdout:6/498: creat df/d17/d44/fac x:0 0 0 2024-05-06T12:14:53.696 INFO:tasks.workunit.client.1.smithi190.stdout:2/660: chown d2/d8/d19/db8/db9 41505 1 2024-05-06T12:14:53.697 INFO:tasks.workunit.client.1.smithi190.stdout:4/508: dread d2/fb0 [0,4194304] 0 2024-05-06T12:14:53.698 INFO:tasks.workunit.client.1.smithi190.stdout:8/704: chown d3/d35/fa0 3 1 2024-05-06T12:14:53.699 INFO:tasks.workunit.client.1.smithi190.stdout:3/784: dwrite d5/d14/d11/d21/d44/d80/f83 [0,4194304] 0 2024-05-06T12:14:53.701 INFO:tasks.workunit.client.1.smithi190.stdout:2/661: truncate d2/fb6 309169 0 2024-05-06T12:14:53.705 INFO:tasks.workunit.client.1.smithi190.stdout:4/509: read d2/d11/d16/d61/d66/d6b/f9f [3334527,107333] 0 2024-05-06T12:14:53.705 INFO:tasks.workunit.client.1.smithi190.stdout:7/673: mkdir d0/d1/d18/d4e/d87/df1 0 2024-05-06T12:14:53.705 INFO:tasks.workunit.client.1.smithi190.stdout:2/662: rmdir d2/d4c 39 2024-05-06T12:14:53.707 INFO:tasks.workunit.client.1.smithi190.stdout:8/705: chown d3/d8/d2b/ce0 275776 1 2024-05-06T12:14:53.712 INFO:tasks.workunit.client.1.smithi190.stdout:4/510: rename d2/d11/d16/d41/f62 to d2/d11/d16/d31/fb4 0 2024-05-06T12:14:53.716 INFO:tasks.workunit.client.1.smithi190.stdout:8/706: creat d3/ff3 x:0 0 0 2024-05-06T12:14:53.717 INFO:tasks.workunit.client.1.smithi190.stdout:7/674: symlink d0/d1/d2/d84/lf2 0 2024-05-06T12:14:53.720 INFO:tasks.workunit.client.1.smithi190.stdout:2/663: dread d2/d21/f70 [0,4194304] 0 2024-05-06T12:14:53.721 INFO:tasks.workunit.client.1.smithi190.stdout:6/499: getdents df 0 2024-05-06T12:14:53.722 INFO:tasks.workunit.client.1.smithi190.stdout:6/500: dread - df/d2c/d1e/d57/d53/fa4 zero size 2024-05-06T12:14:53.723 INFO:tasks.workunit.client.1.smithi190.stdout:6/501: readlink df/d2c/d1e/d57/d53/l5d 0 2024-05-06T12:14:53.725 INFO:tasks.workunit.client.1.smithi190.stdout:7/675: symlink d0/da/d36/d53/d77/d8c/lf3 0 2024-05-06T12:14:53.726 INFO:tasks.workunit.client.1.smithi190.stdout:4/511: creat d2/d11/d16/d61/d9e/d6e/daa/fb5 x:0 0 0 2024-05-06T12:14:53.727 INFO:tasks.workunit.client.1.smithi190.stdout:9/736: truncate db/dd/d16/d4c/fe2 3047632 0 2024-05-06T12:14:53.728 INFO:tasks.workunit.client.1.smithi190.stdout:2/664: rename d2/d21/f9f to d2/d8/d33/d46/d5d/fe1 0 2024-05-06T12:14:53.729 INFO:tasks.workunit.client.1.smithi190.stdout:5/686: truncate d2/da/d59/d5f/f3c 1142866 0 2024-05-06T12:14:53.730 INFO:tasks.workunit.client.1.smithi190.stdout:3/785: getdents d5/d14/d11/d21/d44/d92/d9e/da4/d105 0 2024-05-06T12:14:53.731 INFO:tasks.workunit.client.1.smithi190.stdout:6/502: mkdir df/d2c/d61/d73/dad 0 2024-05-06T12:14:53.733 INFO:tasks.workunit.client.1.smithi190.stdout:4/512: symlink d2/d11/d16/d61/d9e/d6e/d78/lb6 0 2024-05-06T12:14:53.758 INFO:tasks.workunit.client.1.smithi190.stdout:2/665: rmdir d2/d8/d19/d43 39 2024-05-06T12:14:53.759 INFO:tasks.workunit.client.1.smithi190.stdout:5/687: symlink d2/d27/ldb 0 2024-05-06T12:14:53.762 INFO:tasks.workunit.client.1.smithi190.stdout:5/688: read d2/d27/f20 [5153042,58341] 0 2024-05-06T12:14:53.764 INFO:tasks.workunit.client.1.smithi190.stdout:6/503: dwrite f7 [0,4194304] 0 2024-05-06T12:14:53.764 INFO:tasks.workunit.client.1.smithi190.stdout:4/513: creat d2/d11/d16/d61/d66/fb7 x:0 0 0 2024-05-06T12:14:53.765 INFO:tasks.workunit.client.1.smithi190.stdout:4/514: chown d2/d10/l28 0 1 2024-05-06T12:14:53.766 INFO:tasks.workunit.client.1.smithi190.stdout:9/737: creat db/dd/d16/d4c/d4e/ffc x:0 0 0 2024-05-06T12:14:53.769 INFO:tasks.workunit.client.1.smithi190.stdout:2/666: creat d2/d8/d19/d43/d79/d7e/fe2 x:0 0 0 2024-05-06T12:14:53.771 INFO:tasks.workunit.client.1.smithi190.stdout:8/707: mknod d3/d35/d55/d61/cf4 0 2024-05-06T12:14:53.773 INFO:tasks.workunit.client.1.smithi190.stdout:5/689: creat d2/da/d59/d5f/d2e/d3b/d5a/fdc x:0 0 0 2024-05-06T12:14:53.774 INFO:tasks.workunit.client.1.smithi190.stdout:3/786: getdents d5/d7/d7f 0 2024-05-06T12:14:53.775 INFO:tasks.workunit.client.1.smithi190.stdout:2/667: dwrite d2/fd [0,4194304] 0 2024-05-06T12:14:53.776 INFO:tasks.workunit.client.1.smithi190.stdout:9/738: rename db/dd/cf4 to db/dd/d16/d58/ddc/cfd 0 2024-05-06T12:14:53.777 INFO:tasks.workunit.client.1.smithi190.stdout:6/504: mkdir df/d2c/dae 0 2024-05-06T12:14:53.781 INFO:tasks.workunit.client.1.smithi190.stdout:9/739: readlink db/d23/l84 0 2024-05-06T12:14:53.782 INFO:tasks.workunit.client.1.smithi190.stdout:5/690: fdatasync d2/da/d59/d5f/f43 0 2024-05-06T12:14:53.783 INFO:tasks.workunit.client.1.smithi190.stdout:8/708: mknod d3/d8/d25/d65/cf5 0 2024-05-06T12:14:53.784 INFO:tasks.workunit.client.1.smithi190.stdout:3/787: symlink d5/d14/d11/d21/d44/d80/d82/d3d/ded/l115 0 2024-05-06T12:14:53.786 INFO:tasks.workunit.client.1.smithi190.stdout:2/668: link d2/d8/d19/fd9 d2/d8/d19/d4f/fe3 0 2024-05-06T12:14:53.787 INFO:tasks.workunit.client.1.smithi190.stdout:9/740: unlink db/d23/d7e/fea 0 2024-05-06T12:14:53.789 INFO:tasks.workunit.client.1.smithi190.stdout:6/505: link df/d2c/d1e/d57/f69 df/d17/d44/faf 0 2024-05-06T12:14:53.792 INFO:tasks.workunit.client.1.smithi190.stdout:5/691: creat d2/da/d59/d5f/d2e/d8b/fdd x:0 0 0 2024-05-06T12:14:53.793 INFO:tasks.workunit.client.1.smithi190.stdout:8/709: creat d3/d35/d55/d61/ff6 x:0 0 0 2024-05-06T12:14:53.795 INFO:tasks.workunit.client.1.smithi190.stdout:2/669: readlink d2/d8/d19/d43/d79/l98 0 2024-05-06T12:14:53.802 INFO:tasks.workunit.client.1.smithi190.stdout:9/741: rename db/dd/d16/d58 to db/dd/df/dfe 0 2024-05-06T12:14:53.804 INFO:tasks.workunit.client.1.smithi190.stdout:2/670: mknod d2/d8/d33/dc2/dcd/ce4 0 2024-05-06T12:14:53.805 INFO:tasks.workunit.client.1.smithi190.stdout:8/710: truncate d3/f71 5092544 0 2024-05-06T12:14:53.805 INFO:tasks.workunit.client.1.smithi190.stdout:8/711: chown d3/d35/d55/d61/c8c 6092184 1 2024-05-06T12:14:53.807 INFO:tasks.workunit.client.1.smithi190.stdout:5/692: link d2/da/f80 d2/da/d59/d5f/d6e/fde 0 2024-05-06T12:14:53.810 INFO:tasks.workunit.client.1.smithi190.stdout:9/742: rename db/dd/df/dfe/ddc/d47/de4 to db/dd/df/dfe/ddc/d47/d4b/dff 0 2024-05-06T12:14:53.812 INFO:tasks.workunit.client.1.smithi190.stdout:2/671: mknod d2/d8/d19/d7b/da0/ce5 0 2024-05-06T12:14:53.815 INFO:tasks.workunit.client.1.smithi190.stdout:8/712: chown d3/d13/ld4 0 1 2024-05-06T12:14:53.821 INFO:tasks.workunit.client.1.smithi190.stdout:6/506: dread df/d2c/d28/f7c [0,4194304] 0 2024-05-06T12:14:53.824 INFO:tasks.workunit.client.1.smithi190.stdout:8/713: creat d3/d35/d55/ff7 x:0 0 0 2024-05-06T12:14:53.826 INFO:tasks.workunit.client.1.smithi190.stdout:9/743: rename db/dd/l15 to db/dd/d16/d4c/d4e/d5e/dce/l100 0 2024-05-06T12:14:53.826 INFO:tasks.workunit.client.1.smithi190.stdout:3/788: dwrite d5/d14/d11/d21/d44/d80/d82/d3d/ded/fe [0,4194304] 0 2024-05-06T12:14:53.827 INFO:tasks.workunit.client.1.smithi190.stdout:8/714: readlink d3/d8/d2b/d3e/ldd 0 2024-05-06T12:14:53.828 INFO:tasks.workunit.client.1.smithi190.stdout:3/789: chown d5/d7/d6d/d9a 10829 1 2024-05-06T12:14:53.829 INFO:tasks.workunit.client.1.smithi190.stdout:5/693: link d2/da/c19 d2/da/d59/d5f/d50/d6d/cdf 0 2024-05-06T12:14:53.831 INFO:tasks.workunit.client.1.smithi190.stdout:5/694: fsync d2/da/d59/d5f/d50/f51 0 2024-05-06T12:14:53.833 INFO:tasks.workunit.client.1.smithi190.stdout:3/790: write d5/d14/d11/d21/d44/d92/d9e/da4/d105/fe5 [627712,22054] 0 2024-05-06T12:14:53.834 INFO:tasks.workunit.client.1.smithi190.stdout:5/695: write d2/d65/d83/fcf [126364,75428] 0 2024-05-06T12:14:53.836 INFO:tasks.workunit.client.1.smithi190.stdout:3/791: dread - d5/f59 zero size 2024-05-06T12:14:53.837 INFO:tasks.workunit.client.1.smithi190.stdout:6/507: link fd df/d2c/d61/d73/dad/fb0 0 2024-05-06T12:14:53.840 INFO:tasks.workunit.client.1.smithi190.stdout:8/715: getdents d3/d13 0 2024-05-06T12:14:53.841 INFO:tasks.workunit.client.1.smithi190.stdout:6/508: chown df/d12/c59 568620 1 2024-05-06T12:14:53.844 INFO:tasks.workunit.client.1.smithi190.stdout:5/696: dwrite d2/da/f64 [0,4194304] 0 2024-05-06T12:14:53.845 INFO:tasks.workunit.client.1.smithi190.stdout:6/509: mknod df/d2c/d1e/d57/cb1 0 2024-05-06T12:14:53.848 INFO:tasks.workunit.client.1.smithi190.stdout:5/697: creat d2/d65/d83/fe0 x:0 0 0 2024-05-06T12:14:53.853 INFO:tasks.workunit.client.1.smithi190.stdout:1/544: sync 2024-05-06T12:14:53.855 INFO:tasks.workunit.client.1.smithi190.stdout:2/672: dread d2/d8/d19/f2a [4194304,4194304] 0 2024-05-06T12:14:53.856 INFO:tasks.workunit.client.1.smithi190.stdout:1/545: symlink d5/d4f/d65/d24/d3a/lbe 0 2024-05-06T12:14:53.857 INFO:tasks.workunit.client.1.smithi190.stdout:2/673: mkdir d2/d8/d33/de6 0 2024-05-06T12:14:53.858 INFO:tasks.workunit.client.1.smithi190.stdout:1/546: write d5/d4f/d65/d32/d47/d53/d98/fbb [161763,118877] 0 2024-05-06T12:14:53.861 INFO:tasks.workunit.client.1.smithi190.stdout:1/547: creat d5/d15/fbf x:0 0 0 2024-05-06T12:14:53.862 INFO:tasks.workunit.client.1.smithi190.stdout:1/548: write d5/d4f/d65/d32/d47/d53/d6e/fb5 [554419,110735] 0 2024-05-06T12:14:53.864 INFO:tasks.workunit.client.1.smithi190.stdout:1/549: creat d5/d4f/d65/d32/d47/d53/fc0 x:0 0 0 2024-05-06T12:14:53.866 INFO:tasks.workunit.client.1.smithi190.stdout:1/550: unlink d5/d4f/d65/d32/f70 0 2024-05-06T12:14:53.868 INFO:tasks.workunit.client.1.smithi190.stdout:1/551: mknod d5/d4f/d65/d32/d47/cc1 0 2024-05-06T12:14:53.868 INFO:tasks.workunit.client.1.smithi190.stdout:1/552: chown d5/d4f/d65/d32/d47/cc1 1 1 2024-05-06T12:14:53.877 INFO:tasks.workunit.client.1.smithi190.stdout:1/553: getdents d5/d4f/d65/d32/d47/d53/d98/da7 0 2024-05-06T12:14:53.881 INFO:tasks.workunit.client.1.smithi190.stdout:5/698: rmdir d2/da 39 2024-05-06T12:14:53.884 INFO:tasks.workunit.client.1.smithi190.stdout:3/792: dread d5/fc [0,4194304] 0 2024-05-06T12:14:53.885 INFO:tasks.workunit.client.1.smithi190.stdout:5/699: read d2/da/f64 [3422280,20337] 0 2024-05-06T12:14:53.888 INFO:tasks.workunit.client.1.smithi190.stdout:9/744: write db/d23/f83 [2526126,122516] 0 2024-05-06T12:14:53.896 INFO:tasks.workunit.client.1.smithi190.stdout:3/793: dwrite d5/f5e [0,4194304] 0 2024-05-06T12:14:53.900 INFO:tasks.workunit.client.1.smithi190.stdout:9/745: dread db/dd/df/dfe/d87/fcb [0,4194304] 0 2024-05-06T12:14:53.902 INFO:tasks.workunit.client.1.smithi190.stdout:6/510: write df/d17/f60 [619413,81508] 0 2024-05-06T12:14:53.905 INFO:tasks.workunit.client.1.smithi190.stdout:8/716: dwrite d3/d8/d68/faf [0,4194304] 0 2024-05-06T12:14:53.907 INFO:tasks.workunit.client.1.smithi190.stdout:5/700: creat d2/da/d59/d5f/d50/fe1 x:0 0 0 2024-05-06T12:14:53.908 INFO:tasks.workunit.client.1.smithi190.stdout:6/511: write df/f82 [618589,19203] 0 2024-05-06T12:14:53.910 INFO:tasks.workunit.client.1.smithi190.stdout:3/794: dread d5/d14/f56 [0,4194304] 0 2024-05-06T12:14:53.912 INFO:tasks.workunit.client.1.smithi190.stdout:1/554: truncate d5/d4f/d68/f8e 859026 0 2024-05-06T12:14:53.916 INFO:tasks.workunit.client.1.smithi190.stdout:2/674: dread d2/d8/d19/d43/d79/d7e/f63 [0,4194304] 0 2024-05-06T12:14:53.917 INFO:tasks.workunit.client.1.smithi190.stdout:5/701: creat d2/da/d59/d5f/d50/d6d/fe2 x:0 0 0 2024-05-06T12:14:53.917 INFO:tasks.workunit.client.1.smithi190.stdout:3/795: readlink d5/d14/d11/d21/d44/d80/d82/d3d/ded/d33/lb5 0 2024-05-06T12:14:53.917 INFO:tasks.workunit.client.1.smithi190.stdout:9/746: mknod db/dd/df/dfe/d87/d22/c101 0 2024-05-06T12:14:53.919 INFO:tasks.workunit.client.1.smithi190.stdout:1/555: mknod d5/d4f/d65/cc2 0 2024-05-06T12:14:53.919 INFO:tasks.workunit.client.1.smithi190.stdout:6/512: dwrite df/d17/d37/f8c [0,4194304] 0 2024-05-06T12:14:53.922 INFO:tasks.workunit.client.1.smithi190.stdout:1/556: readlink d5/d4f/d65/d24/d3a/l85 0 2024-05-06T12:14:53.922 INFO:tasks.workunit.client.1.smithi190.stdout:8/717: rename d3/d8/d2b/c43 to d3/d8/d2b/cf8 0 2024-05-06T12:14:53.925 INFO:tasks.workunit.client.1.smithi190.stdout:1/557: write d5/d4f/d65/d24/f28 [452870,121947] 0 2024-05-06T12:14:53.925 INFO:tasks.workunit.client.1.smithi190.stdout:6/513: rmdir df/d2c/d61 39 2024-05-06T12:14:53.927 INFO:tasks.workunit.client.1.smithi190.stdout:2/675: symlink d2/d8/d19/d7b/dde/le7 0 2024-05-06T12:14:53.929 INFO:tasks.workunit.client.1.smithi190.stdout:8/718: mkdir d3/d35/d55/d61/df9 0 2024-05-06T12:14:53.931 INFO:tasks.workunit.client.1.smithi190.stdout:5/702: truncate d2/da/d59/d5f/f3c 2164018 0 2024-05-06T12:14:53.933 INFO:tasks.workunit.client.1.smithi190.stdout:0/711: sync 2024-05-06T12:14:53.935 INFO:tasks.workunit.client.1.smithi190.stdout:2/676: unlink d2/d8/c4b 0 2024-05-06T12:14:53.936 INFO:tasks.workunit.client.1.smithi190.stdout:5/703: fsync d2/da/d59/d5f/d2e/d3b/d5a/f85 0 2024-05-06T12:14:53.936 INFO:tasks.workunit.client.1.smithi190.stdout:8/719: dread - d3/d8/d25/d3b/d6b/d7a/fb2 zero size 2024-05-06T12:14:53.937 INFO:tasks.workunit.client.1.smithi190.stdout:6/514: mknod df/d2c/d61/d84/cb2 0 2024-05-06T12:14:53.938 INFO:tasks.workunit.client.1.smithi190.stdout:1/558: dwrite d5/d15/f7a [0,4194304] 0 2024-05-06T12:14:53.940 INFO:tasks.workunit.client.1.smithi190.stdout:2/677: mknod d2/d8/d33/d46/db7/ce8 0 2024-05-06T12:14:53.941 INFO:tasks.workunit.client.1.smithi190.stdout:0/712: truncate d9/dc/db6/fb7 740523 0 2024-05-06T12:14:53.944 INFO:tasks.workunit.client.1.smithi190.stdout:8/720: getdents d3 0 2024-05-06T12:14:53.945 INFO:tasks.workunit.client.1.smithi190.stdout:2/678: dread d2/d21/f70 [0,4194304] 0 2024-05-06T12:14:53.949 INFO:tasks.workunit.client.1.smithi190.stdout:1/559: dwrite d5/d4f/d65/f7b [0,4194304] 0 2024-05-06T12:14:53.950 INFO:tasks.workunit.client.1.smithi190.stdout:5/704: dread d2/da/d59/d5f/f58 [0,4194304] 0 2024-05-06T12:14:53.952 INFO:tasks.workunit.client.1.smithi190.stdout:5/705: write d2/da/d59/d5f/d50/d6d/fe2 [95216,73123] 0 2024-05-06T12:14:53.953 INFO:tasks.workunit.client.1.smithi190.stdout:5/706: chown d2/d65/l8e 226186245 1 2024-05-06T12:14:53.958 INFO:tasks.workunit.client.1.smithi190.stdout:9/747: dread db/dd/d16/d4c/d4e/d5e/dce/fd5 [4194304,4194304] 0 2024-05-06T12:14:53.966 INFO:tasks.workunit.client.1.smithi190.stdout:3/796: dread d5/d14/d11/fc7 [0,4194304] 0 2024-05-06T12:14:53.967 INFO:tasks.workunit.client.1.smithi190.stdout:3/797: fdatasync d5/d14/d11/d21/f30 0 2024-05-06T12:14:53.999 INFO:tasks.workunit.client.1.smithi190.stdout:0/713: mkdir d9/dc/d48/d50/d83/dfb 0 2024-05-06T12:14:54.005 INFO:tasks.workunit.client.1.smithi190.stdout:2/679: rename d2/f12 to d2/d8/d33/d46/fe9 0 2024-05-06T12:14:54.014 INFO:tasks.workunit.client.1.smithi190.stdout:0/714: creat d9/dc/d48/d50/d87/ffc x:0 0 0 2024-05-06T12:14:54.017 INFO:tasks.workunit.client.1.smithi190.stdout:2/680: write d2/d8/d33/d46/d5d/fe1 [4843019,63894] 0 2024-05-06T12:14:54.018 INFO:tasks.workunit.client.1.smithi190.stdout:2/681: stat d2/d8/d33/d46/la3 0 2024-05-06T12:14:54.020 INFO:tasks.workunit.client.1.smithi190.stdout:3/798: rmdir d5/d94/dd9 39 2024-05-06T12:14:54.022 INFO:tasks.workunit.client.1.smithi190.stdout:0/715: mkdir d9/dc/d28/dc5/dd5/de4/dea/dfd 0 2024-05-06T12:14:54.023 INFO:tasks.workunit.client.1.smithi190.stdout:1/560: creat d5/d4f/d65/d32/fc3 x:0 0 0 2024-05-06T12:14:54.024 INFO:tasks.workunit.client.1.smithi190.stdout:1/561: readlink d5/d4f/d65/l6a 0 2024-05-06T12:14:54.027 INFO:tasks.workunit.client.1.smithi190.stdout:2/682: creat d2/d8/d33/d46/dab/dac/fea x:0 0 0 2024-05-06T12:14:54.030 INFO:tasks.workunit.client.1.smithi190.stdout:1/562: dwrite d5/f12 [0,4194304] 0 2024-05-06T12:14:54.034 INFO:tasks.workunit.client.1.smithi190.stdout:8/721: write d3/d35/d55/d61/f63 [337495,79461] 0 2024-05-06T12:14:54.035 INFO:tasks.workunit.client.1.smithi190.stdout:9/748: getdents db/dd 0 2024-05-06T12:14:54.035 INFO:tasks.workunit.client.1.smithi190.stdout:3/799: symlink d5/d14/d11/d21/deb/d109/l116 0 2024-05-06T12:14:54.035 INFO:tasks.workunit.client.1.smithi190.stdout:9/749: readlink la 0 2024-05-06T12:14:54.036 INFO:tasks.workunit.client.1.smithi190.stdout:8/722: dread - d3/d35/d55/d61/ff6 zero size 2024-05-06T12:14:54.040 INFO:tasks.workunit.client.1.smithi190.stdout:2/683: symlink d2/d8/d71/leb 0 2024-05-06T12:14:54.043 INFO:tasks.workunit.client.1.smithi190.stdout:2/684: chown d2/d8/d19/dbd/ccc 22 1 2024-05-06T12:14:54.043 INFO:tasks.workunit.client.1.smithi190.stdout:5/707: dwrite d2/da/d59/d5f/f5b [4194304,4194304] 0 2024-05-06T12:14:54.043 INFO:tasks.workunit.client.1.smithi190.stdout:7/676: sync 2024-05-06T12:14:54.043 INFO:tasks.workunit.client.1.smithi190.stdout:4/515: sync 2024-05-06T12:14:54.045 INFO:tasks.workunit.client.1.smithi190.stdout:1/563: dread d5/d4f/d65/d24/f28 [0,4194304] 0 2024-05-06T12:14:54.046 INFO:tasks.workunit.client.1.smithi190.stdout:3/800: rename d5/d14/ce7 to d5/d14/d11/d21/d44/d80/d82/dfc/c117 0 2024-05-06T12:14:54.048 INFO:tasks.workunit.client.1.smithi190.stdout:9/750: creat db/dd/df/dfe/d87/d22/d49/d88/d9f/df2/f102 x:0 0 0 2024-05-06T12:14:54.049 INFO:tasks.workunit.client.1.smithi190.stdout:0/716: creat d9/d51/d93/ffe x:0 0 0 2024-05-06T12:14:54.049 INFO:tasks.workunit.client.1.smithi190.stdout:7/677: dread - d0/d1/d18/d4e/d87/f8b zero size 2024-05-06T12:14:54.049 INFO:tasks.workunit.client.1.smithi190.stdout:2/685: stat d2/d21/c49 0 2024-05-06T12:14:54.050 INFO:tasks.workunit.client.1.smithi190.stdout:5/708: truncate d2/fc 4606104 0 2024-05-06T12:14:54.051 INFO:tasks.workunit.client.1.smithi190.stdout:1/564: symlink d5/d4f/d65/d62/d9d/lc4 0 2024-05-06T12:14:54.053 INFO:tasks.workunit.client.1.smithi190.stdout:9/751: chown db/dd/d16/d4c/d4e/d5e/c73 24909896 1 2024-05-06T12:14:54.054 INFO:tasks.workunit.client.1.smithi190.stdout:8/723: dwrite d3/f18 [4194304,4194304] 0 2024-05-06T12:14:54.054 INFO:tasks.workunit.client.1.smithi190.stdout:3/801: truncate d5/d14/d11/d21/f38 2604400 0 2024-05-06T12:14:54.057 INFO:tasks.workunit.client.1.smithi190.stdout:0/717: symlink d9/dc/d28/dc5/dd5/de4/dea/lff 0 2024-05-06T12:14:54.057 INFO:tasks.workunit.client.1.smithi190.stdout:1/565: truncate d5/d4f/d65/fa9 445953 0 2024-05-06T12:14:54.058 INFO:tasks.workunit.client.1.smithi190.stdout:5/709: mknod d2/da/d59/d5f/d2e/d8b/ce3 0 2024-05-06T12:14:54.061 INFO:tasks.workunit.client.1.smithi190.stdout:7/678: rename d0/da/d36/d53/d5c/fc7 to d0/da/ff4 0 2024-05-06T12:14:54.062 INFO:tasks.workunit.client.1.smithi190.stdout:4/516: link d2/d11/d16/d31/f43 d2/d10/fb8 0 2024-05-06T12:14:54.064 INFO:tasks.workunit.client.1.smithi190.stdout:7/679: chown d0/d1/d18/d2b/ce4 16 1 2024-05-06T12:14:54.065 INFO:tasks.workunit.client.1.smithi190.stdout:9/752: write db/dd/df/dfe/d87/fcb [4785373,122054] 0 2024-05-06T12:14:54.065 INFO:tasks.workunit.client.1.smithi190.stdout:2/686: dwrite d2/d8/d71/f88 [0,4194304] 0 2024-05-06T12:14:54.065 INFO:tasks.workunit.client.1.smithi190.stdout:4/517: write d2/d11/d16/d61/f7c [562437,109248] 0 2024-05-06T12:14:54.067 INFO:tasks.workunit.client.1.smithi190.stdout:8/724: creat d3/d8/d68/d9f/ffa x:0 0 0 2024-05-06T12:14:54.075 INFO:tasks.workunit.client.1.smithi190.stdout:7/680: creat d0/d1/d18/d2b/ff5 x:0 0 0 2024-05-06T12:14:54.076 INFO:tasks.workunit.client.1.smithi190.stdout:3/802: fdatasync d5/d14/d11/f1b 0 2024-05-06T12:14:54.079 INFO:tasks.workunit.client.1.smithi190.stdout:9/753: creat db/dd/df/dfe/d87/d22/d49/d8c/f103 x:0 0 0 2024-05-06T12:14:54.081 INFO:tasks.workunit.client.1.smithi190.stdout:5/710: creat d2/da/d59/d5f/d6e/daa/fe4 x:0 0 0 2024-05-06T12:14:54.084 INFO:tasks.workunit.client.1.smithi190.stdout:2/687: dwrite d2/d8/dc/f13 [0,4194304] 0 2024-05-06T12:14:54.085 INFO:tasks.workunit.client.1.smithi190.stdout:8/725: dwrite d3/d8/d68/dde/fe4 [0,4194304] 0 2024-05-06T12:14:54.085 INFO:tasks.workunit.client.1.smithi190.stdout:5/711: dread - d2/da/d59/d5f/d2e/d8b/fd6 zero size 2024-05-06T12:14:54.087 INFO:tasks.workunit.client.1.smithi190.stdout:9/754: mknod db/dd/df/dfe/ddc/d47/d4b/c104 0 2024-05-06T12:14:54.089 INFO:tasks.workunit.client.1.smithi190.stdout:5/712: fsync d2/d6c/f7a 0 2024-05-06T12:14:54.092 INFO:tasks.workunit.client.1.smithi190.stdout:7/681: dread d0/d1/d2/d84/fbf [0,4194304] 0 2024-05-06T12:14:54.092 INFO:tasks.workunit.client.1.smithi190.stdout:9/755: chown db/dd/df/dfe/d87/d22/d49/fd9 83443377 1 2024-05-06T12:14:54.093 INFO:tasks.workunit.client.1.smithi190.stdout:2/688: symlink d2/d8/d33/d46/dab/dac/lec 0 2024-05-06T12:14:54.093 INFO:tasks.workunit.client.1.smithi190.stdout:5/713: chown d2/d6c/l90 14492 1 2024-05-06T12:14:54.094 INFO:tasks.workunit.client.1.smithi190.stdout:7/682: chown d0/d1/d32/f44 34881538 1 2024-05-06T12:14:54.096 INFO:tasks.workunit.client.1.smithi190.stdout:2/689: rmdir d2/d8/d19/d9b 39 2024-05-06T12:14:54.096 INFO:tasks.workunit.client.1.smithi190.stdout:5/714: dread d2/d65/d83/fcf [0,4194304] 0 2024-05-06T12:14:54.097 INFO:tasks.workunit.client.1.smithi190.stdout:2/690: chown d2/d8/dc/l17 240204469 1 2024-05-06T12:14:54.098 INFO:tasks.workunit.client.1.smithi190.stdout:9/756: symlink db/dd/d5c/dac/l105 0 2024-05-06T12:14:54.098 INFO:tasks.workunit.client.1.smithi190.stdout:2/691: write d2/d8/d19/f8a [3126643,120079] 0 2024-05-06T12:14:54.102 INFO:tasks.workunit.client.1.smithi190.stdout:1/566: dread d5/d4f/d65/d24/f7e [0,4194304] 0 2024-05-06T12:14:54.104 INFO:tasks.workunit.client.1.smithi190.stdout:5/715: stat d2/da/d59/d5f/d2e/cc1 0 2024-05-06T12:14:54.105 INFO:tasks.workunit.client.1.smithi190.stdout:0/718: dread d9/dcf/d56/dae/fb2 [0,4194304] 0 2024-05-06T12:14:54.107 INFO:tasks.workunit.client.1.smithi190.stdout:9/757: rmdir db/dd/d16/d4c/d4e 39 2024-05-06T12:14:54.109 INFO:tasks.workunit.client.1.smithi190.stdout:7/683: dwrite d0/d14/d6a/d8f/fdb [0,4194304] 0 2024-05-06T12:14:54.115 INFO:tasks.workunit.client.1.smithi190.stdout:9/758: creat db/dd/df/f106 x:0 0 0 2024-05-06T12:14:54.116 INFO:tasks.workunit.client.1.smithi190.stdout:7/684: symlink d0/db4/lf6 0 2024-05-06T12:14:54.117 INFO:tasks.workunit.client.1.smithi190.stdout:5/716: mknod d2/da/d59/d5f/d2e/da7/ce5 0 2024-05-06T12:14:54.119 INFO:tasks.workunit.client.1.smithi190.stdout:9/759: creat db/dd/df/f107 x:0 0 0 2024-05-06T12:14:54.119 INFO:tasks.workunit.client.1.smithi190.stdout:6/515: sync 2024-05-06T12:14:54.120 INFO:tasks.workunit.client.1.smithi190.stdout:5/717: write d2/da/d59/d5f/d6e/daa/fe4 [235915,1653] 0 2024-05-06T12:14:54.125 INFO:tasks.workunit.client.1.smithi190.stdout:0/719: link d9/dcf/d21/f7e d9/dc/d48/d50/d87/dd2/db3/f100 0 2024-05-06T12:14:54.126 INFO:tasks.workunit.client.1.smithi190.stdout:3/803: dread d5/f5e [4194304,4194304] 0 2024-05-06T12:14:54.126 INFO:tasks.workunit.client.1.smithi190.stdout:7/685: mkdir d0/d1/d32/d45/d70/d33/df7 0 2024-05-06T12:14:54.126 INFO:tasks.workunit.client.1.smithi190.stdout:6/516: rmdir df/d2c/d1e/d57/d53 39 2024-05-06T12:14:54.127 INFO:tasks.workunit.client.1.smithi190.stdout:6/517: fsync df/f99 0 2024-05-06T12:14:54.130 INFO:tasks.workunit.client.1.smithi190.stdout:7/686: unlink d0/d14/d6a/fd9 0 2024-05-06T12:14:54.132 INFO:tasks.workunit.client.1.smithi190.stdout:3/804: symlink d5/d14/d11/d21/d44/d80/ddb/l118 0 2024-05-06T12:14:54.134 INFO:tasks.workunit.client.1.smithi190.stdout:3/805: fdatasync d5/d14/d11/d21/d44/d92/d9e/fee 0 2024-05-06T12:14:54.136 INFO:tasks.workunit.client.1.smithi190.stdout:4/518: dwrite d2/f64 [0,4194304] 0 2024-05-06T12:14:54.140 INFO:tasks.workunit.client.1.smithi190.stdout:7/687: symlink d0/d1/d32/d45/d70/d33/df7/lf8 0 2024-05-06T12:14:54.142 INFO:tasks.workunit.client.1.smithi190.stdout:7/688: fdatasync d0/d1/d18/d2b/fdc 0 2024-05-06T12:14:54.143 INFO:tasks.workunit.client.1.smithi190.stdout:4/519: symlink d2/d11/d16/d61/d66/d6b/da4/lb9 0 2024-05-06T12:14:54.143 INFO:tasks.workunit.client.1.smithi190.stdout:2/692: dread d2/d8/d33/f37 [0,4194304] 0 2024-05-06T12:14:54.146 INFO:tasks.workunit.client.1.smithi190.stdout:1/567: write d5/d4f/d65/d32/d47/f4c [4978631,34117] 0 2024-05-06T12:14:54.148 INFO:tasks.workunit.client.1.smithi190.stdout:4/520: creat d2/d10/fba x:0 0 0 2024-05-06T12:14:54.148 INFO:tasks.workunit.client.1.smithi190.stdout:8/726: dwrite d3/d8/d25/d3b/d6b/d7a/fb2 [0,4194304] 0 2024-05-06T12:14:54.149 INFO:tasks.workunit.client.1.smithi190.stdout:7/689: creat d0/d1/d32/d45/d70/d35/ff9 x:0 0 0 2024-05-06T12:14:54.152 INFO:tasks.workunit.client.1.smithi190.stdout:2/693: mknod d2/d8/d33/d46/dab/dbb/ced 0 2024-05-06T12:14:54.156 INFO:tasks.workunit.client.1.smithi190.stdout:8/727: rmdir d3/d8/d2b 39 2024-05-06T12:14:54.159 INFO:tasks.workunit.client.1.smithi190.stdout:9/760: write db/dd/d16/d4c/f69 [3836108,50362] 0 2024-05-06T12:14:54.160 INFO:tasks.workunit.client.1.smithi190.stdout:4/521: dwrite d2/d11/d16/d8e/d80/f97 [0,4194304] 0 2024-05-06T12:14:54.160 INFO:tasks.workunit.client.1.smithi190.stdout:7/690: mkdir d0/d1/d18/dfa 0 2024-05-06T12:14:54.162 INFO:tasks.workunit.client.1.smithi190.stdout:0/720: write d9/f52 [2710448,90029] 0 2024-05-06T12:14:54.165 INFO:tasks.workunit.client.1.smithi190.stdout:8/728: dread - d3/d8/d2b/d3e/fe2 zero size 2024-05-06T12:14:54.165 INFO:tasks.workunit.client.1.smithi190.stdout:6/518: write df/d2c/d61/d84/f8e [788257,28822] 0 2024-05-06T12:14:54.167 INFO:tasks.workunit.client.1.smithi190.stdout:1/568: getdents d5/d15 0 2024-05-06T12:14:54.169 INFO:tasks.workunit.client.1.smithi190.stdout:1/569: chown d5/d4f/d65/d32/d47/d53/d98/da7/fb6 169 1 2024-05-06T12:14:54.169 INFO:tasks.workunit.client.1.smithi190.stdout:0/721: mkdir d9/dcf/d56/d101 0 2024-05-06T12:14:54.173 INFO:tasks.workunit.client.1.smithi190.stdout:9/761: creat db/d23/da4/f108 x:0 0 0 2024-05-06T12:14:54.175 INFO:tasks.workunit.client.1.smithi190.stdout:2/694: link d2/d8/d33/d46/d5d/fe1 d2/d8/d33/d46/dab/fee 0 2024-05-06T12:14:54.177 INFO:tasks.workunit.client.1.smithi190.stdout:6/519: rename df/f22 to df/d2c/dae/fb3 0 2024-05-06T12:14:54.178 INFO:tasks.workunit.client.1.smithi190.stdout:7/691: dwrite d0/da/d36/d53/d77/d8c/dd1/d60/fa8 [0,4194304] 0 2024-05-06T12:14:54.178 INFO:tasks.workunit.client.1.smithi190.stdout:1/570: creat d5/d6b/fc5 x:0 0 0 2024-05-06T12:14:54.179 INFO:tasks.workunit.client.1.smithi190.stdout:1/571: stat d5/d15/d1e/f5a 0 2024-05-06T12:14:54.180 INFO:tasks.workunit.client.1.smithi190.stdout:5/718: dread d2/fc [0,4194304] 0 2024-05-06T12:14:54.180 INFO:tasks.workunit.client.1.smithi190.stdout:0/722: creat d9/d51/dc9/f102 x:0 0 0 2024-05-06T12:14:54.187 INFO:tasks.workunit.client.1.smithi190.stdout:5/719: dread d2/da/d59/d5f/f58 [0,4194304] 0 2024-05-06T12:14:54.188 INFO:tasks.workunit.client.1.smithi190.stdout:0/723: fdatasync d9/dcf/d21/f4f 0 2024-05-06T12:14:54.189 INFO:tasks.workunit.client.1.smithi190.stdout:2/695: creat d2/d8/d19/d9b/fef x:0 0 0 2024-05-06T12:14:54.190 INFO:tasks.workunit.client.1.smithi190.stdout:5/720: write d2/f44 [2232166,19320] 0 2024-05-06T12:14:54.191 INFO:tasks.workunit.client.1.smithi190.stdout:9/762: fsync db/dd/d16/d4c/d4e/f62 0 2024-05-06T12:14:54.194 INFO:tasks.workunit.client.1.smithi190.stdout:1/572: dwrite d5/d15/d1e/d94/fb2 [0,4194304] 0 2024-05-06T12:14:54.194 INFO:tasks.workunit.client.1.smithi190.stdout:4/522: getdents d2/d11/d16/d61/d66 0 2024-05-06T12:14:54.194 INFO:tasks.workunit.client.1.smithi190.stdout:7/692: mkdir d0/d1/d2/d84/dc6/dfb 0 2024-05-06T12:14:54.200 INFO:tasks.workunit.client.1.smithi190.stdout:7/693: write d0/d1/d32/daa/fcb [1180410,61289] 0 2024-05-06T12:14:54.201 INFO:tasks.workunit.client.1.smithi190.stdout:0/724: creat d9/dc/d28/d4c/d80/da7/f103 x:0 0 0 2024-05-06T12:14:54.203 INFO:tasks.workunit.client.1.smithi190.stdout:5/721: creat d2/fe6 x:0 0 0 2024-05-06T12:14:54.204 INFO:tasks.workunit.client.1.smithi190.stdout:9/763: symlink db/dd/df/dfe/d87/d22/d49/d8c/l109 0 2024-05-06T12:14:54.204 INFO:tasks.workunit.client.1.smithi190.stdout:0/725: truncate d9/dc/d28/f8c 4200601 0 2024-05-06T12:14:54.205 INFO:tasks.workunit.client.1.smithi190.stdout:4/523: dwrite d2/f59 [0,4194304] 0 2024-05-06T12:14:54.206 INFO:tasks.workunit.client.1.smithi190.stdout:0/726: stat d9/dc/d28/l41 0 2024-05-06T12:14:54.208 INFO:tasks.workunit.client.1.smithi190.stdout:5/722: mknod d2/da/d59/d5f/d6e/daa/ce7 0 2024-05-06T12:14:54.211 INFO:tasks.workunit.client.1.smithi190.stdout:5/723: chown d2/da/d59/d5f/d2e/d3b/d5a/fc7 977749644 1 2024-05-06T12:14:54.212 INFO:tasks.workunit.client.1.smithi190.stdout:2/696: dread d2/f1f [0,4194304] 0 2024-05-06T12:14:54.213 INFO:tasks.workunit.client.1.smithi190.stdout:5/724: chown d2/d27/l89 11 1 2024-05-06T12:14:54.214 INFO:tasks.workunit.client.1.smithi190.stdout:0/727: dread d9/dc/dd0/fd8 [0,4194304] 0 2024-05-06T12:14:54.217 INFO:tasks.workunit.client.1.smithi190.stdout:9/764: rename db/dd/df/dfe/d87/f85 to db/d23/da4/f10a 0 2024-05-06T12:14:54.219 INFO:tasks.workunit.client.1.smithi190.stdout:5/725: symlink d2/db6/le8 0 2024-05-06T12:14:54.220 INFO:tasks.workunit.client.1.smithi190.stdout:8/729: write d3/d8/d25/f9b [531448,62943] 0 2024-05-06T12:14:54.221 INFO:tasks.workunit.client.1.smithi190.stdout:2/697: symlink d2/d8/d33/d46/db7/lf0 0 2024-05-06T12:14:54.223 INFO:tasks.workunit.client.1.smithi190.stdout:0/728: dwrite d9/dc/d48/d50/d83/f94 [0,4194304] 0 2024-05-06T12:14:54.225 INFO:tasks.workunit.client.1.smithi190.stdout:5/726: symlink d2/da/le9 0 2024-05-06T12:14:54.225 INFO:tasks.workunit.client.1.smithi190.stdout:0/729: truncate d9/dc/d48/ff0 5173 0 2024-05-06T12:14:54.230 INFO:tasks.workunit.client.1.smithi190.stdout:9/765: link db/dd/d16/l20 db/dd/df/l10b 0 2024-05-06T12:14:54.236 INFO:tasks.workunit.client.1.smithi190.stdout:2/698: creat d2/d4c/ff1 x:0 0 0 2024-05-06T12:14:54.238 INFO:tasks.workunit.client.1.smithi190.stdout:7/694: dread d0/f19 [0,4194304] 0 2024-05-06T12:14:54.239 INFO:tasks.workunit.client.1.smithi190.stdout:9/766: mkdir db/dd/df/dfe/ddc/d47/d10c 0 2024-05-06T12:14:54.240 INFO:tasks.workunit.client.1.smithi190.stdout:5/727: mkdir d2/dea 0 2024-05-06T12:14:54.240 INFO:tasks.workunit.client.1.smithi190.stdout:5/728: chown d2/da/d59/d5f/d2e/d8b 425 1 2024-05-06T12:14:54.242 INFO:tasks.workunit.client.1.smithi190.stdout:2/699: unlink d2/d8/d33/d46/la3 0 2024-05-06T12:14:54.243 INFO:tasks.workunit.client.1.smithi190.stdout:6/520: creat df/d2c/d61/d84/fb4 x:0 0 0 2024-05-06T12:14:54.244 INFO:tasks.workunit.client.1.smithi190.stdout:7/695: creat d0/d1/d18/d4e/db3/ffc x:0 0 0 2024-05-06T12:14:54.250 INFO:tasks.workunit.client.1.smithi190.stdout:7/696: write d0/da/d36/f43 [4983129,79274] 0 2024-05-06T12:14:54.252 INFO:tasks.workunit.client.1.smithi190.stdout:2/700: creat d2/d8/d33/d9a/ff2 x:0 0 0 2024-05-06T12:14:54.253 INFO:tasks.workunit.client.1.smithi190.stdout:7/697: dread d0/d1/d2/d84/fbf [0,4194304] 0 2024-05-06T12:14:54.255 INFO:tasks.workunit.client.1.smithi190.stdout:2/701: read d2/d4c/f5a [3629956,88807] 0 2024-05-06T12:14:54.256 INFO:tasks.workunit.client.1.smithi190.stdout:0/730: dread d9/dcf/d21/f7e [0,4194304] 0 2024-05-06T12:14:54.257 INFO:tasks.workunit.client.1.smithi190.stdout:7/698: symlink d0/da/d36/d53/d77/d8c/dd1/d60/dac/lfd 0 2024-05-06T12:14:54.258 INFO:tasks.workunit.client.1.smithi190.stdout:4/524: write d2/d11/d16/d4e/f6f [361200,20318] 0 2024-05-06T12:14:54.261 INFO:tasks.workunit.client.1.smithi190.stdout:0/731: fdatasync d9/dc/dd0/fd8 0 2024-05-06T12:14:54.262 INFO:tasks.workunit.client.1.smithi190.stdout:7/699: mkdir d0/db4/dfe 0 2024-05-06T12:14:54.262 INFO:tasks.workunit.client.1.smithi190.stdout:6/521: dread df/d2c/d1e/f2d [0,4194304] 0 2024-05-06T12:14:54.264 INFO:tasks.workunit.client.1.smithi190.stdout:7/700: fdatasync d0/f69 0 2024-05-06T12:14:54.265 INFO:tasks.workunit.client.1.smithi190.stdout:8/730: write d3/f1c [653008,11885] 0 2024-05-06T12:14:54.270 INFO:tasks.workunit.client.1.smithi190.stdout:7/701: dread d0/d1/d32/daa/fcb [0,4194304] 0 2024-05-06T12:14:54.271 INFO:tasks.workunit.client.1.smithi190.stdout:0/732: symlink d9/dc/d28/dc5/dd5/de4/dea/dfd/l104 0 2024-05-06T12:14:54.273 INFO:tasks.workunit.client.1.smithi190.stdout:8/731: read - d3/d8/d25/d3b/d6b/fd5 zero size 2024-05-06T12:14:54.274 INFO:tasks.workunit.client.1.smithi190.stdout:2/702: dread d2/d8/d19/d43/d79/f82 [0,4194304] 0 2024-05-06T12:14:54.279 INFO:tasks.workunit.client.1.smithi190.stdout:5/729: write d2/fc [4167493,122048] 0 2024-05-06T12:14:54.280 INFO:tasks.workunit.client.1.smithi190.stdout:5/730: readlink d2/d27/l32 0 2024-05-06T12:14:54.281 INFO:tasks.workunit.client.1.smithi190.stdout:8/732: mkdir d3/d8/d25/d3b/d90/dcb/dfb 0 2024-05-06T12:14:54.282 INFO:tasks.workunit.client.1.smithi190.stdout:9/767: dwrite db/dd/d16/d4c/d4e/d5e/dce/fd5 [0,4194304] 0 2024-05-06T12:14:54.282 INFO:tasks.workunit.client.1.smithi190.stdout:2/703: rmdir d2/d8/dc 39 2024-05-06T12:14:54.284 INFO:tasks.workunit.client.1.smithi190.stdout:7/702: fdatasync d0/da/d36/d42/fc4 0 2024-05-06T12:14:54.286 INFO:tasks.workunit.client.1.smithi190.stdout:7/703: write d0/d1/fc [2868244,7513] 0 2024-05-06T12:14:54.287 INFO:tasks.workunit.client.1.smithi190.stdout:0/733: rename d9/dc/l60 to d9/dc/d48/d50/d87/dd2/dcc/df5/l105 0 2024-05-06T12:14:54.287 INFO:tasks.workunit.client.1.smithi190.stdout:7/704: fdatasync d0/d1/d18/d4e/d87/f8b 0 2024-05-06T12:14:54.288 INFO:tasks.workunit.client.1.smithi190.stdout:7/705: chown d0/d1/d32/d45/d70/d3b 7 1 2024-05-06T12:14:54.289 INFO:tasks.workunit.client.1.smithi190.stdout:7/706: readlink d0/db4/lc8 0 2024-05-06T12:14:54.292 INFO:tasks.workunit.client.1.smithi190.stdout:2/704: fsync d2/d4c/f5a 0 2024-05-06T12:14:54.294 INFO:tasks.workunit.client.1.smithi190.stdout:4/525: write d2/d11/d16/f90 [1023683,5557] 0 2024-05-06T12:14:54.300 INFO:tasks.workunit.client.1.smithi190.stdout:9/768: dread db/dd/f41 [0,4194304] 0 2024-05-06T12:14:54.300 INFO:tasks.workunit.client.1.smithi190.stdout:5/731: rename d2/da/d59/d5f/fca to d2/d6c/feb 0 2024-05-06T12:14:54.300 INFO:tasks.workunit.client.1.smithi190.stdout:9/769: readlink db/dd/d16/d4c/l8e 0 2024-05-06T12:14:54.301 INFO:tasks.workunit.client.1.smithi190.stdout:0/734: symlink d9/dcf/d56/l106 0 2024-05-06T12:14:54.303 INFO:tasks.workunit.client.1.smithi190.stdout:6/522: link df/d2c/d1e/d57/d53/l5d df/d2c/d1e/d57/lb5 0 2024-05-06T12:14:54.304 INFO:tasks.workunit.client.1.smithi190.stdout:2/705: creat d2/d8/d33/d46/ff3 x:0 0 0 2024-05-06T12:14:54.309 INFO:tasks.workunit.client.1.smithi190.stdout:7/707: rename d0/da/d36/d53/d77/d8c/dd1/f52 to d0/d1/d2/d84/dc6/fff 0 2024-05-06T12:14:54.310 INFO:tasks.workunit.client.1.smithi190.stdout:7/708: chown d0/da/d36/d42/lb2 0 1 2024-05-06T12:14:54.311 INFO:tasks.workunit.client.1.smithi190.stdout:9/770: creat db/dd/df/dfe/d87/d22/d49/d8c/f10d x:0 0 0 2024-05-06T12:14:54.313 INFO:tasks.workunit.client.1.smithi190.stdout:8/733: write d3/fd7 [980957,105048] 0 2024-05-06T12:14:54.316 INFO:tasks.workunit.client.1.smithi190.stdout:9/771: rmdir db/dd/df/dfe 39 2024-05-06T12:14:54.320 INFO:tasks.workunit.client.1.smithi190.stdout:5/732: rmdir d2/da/d59/d5f/d2e/dc6 0 2024-05-06T12:14:54.323 INFO:tasks.workunit.client.1.smithi190.stdout:4/526: dwrite d2/d11/d16/d61/d66/d6b/f9f [0,4194304] 0 2024-05-06T12:14:54.325 INFO:tasks.workunit.client.1.smithi190.stdout:2/706: creat d2/d8/ff4 x:0 0 0 2024-05-06T12:14:54.325 INFO:tasks.workunit.client.1.smithi190.stdout:6/523: truncate df/d17/d44/fa7 382928 0 2024-05-06T12:14:54.327 INFO:tasks.workunit.client.1.smithi190.stdout:0/735: link d9/dcf/d21/d31/c39 d9/c107 0 2024-05-06T12:14:54.328 INFO:tasks.workunit.client.1.smithi190.stdout:6/524: truncate df/d17/d44/fa8 5098590 0 2024-05-06T12:14:54.328 INFO:tasks.workunit.client.1.smithi190.stdout:5/733: mkdir d2/da/d59/d5f/d2e/d3b/d72/dec 0 2024-05-06T12:14:54.329 INFO:tasks.workunit.client.1.smithi190.stdout:4/527: mknod d2/d11/d16/d61/d66/d6b/cbb 0 2024-05-06T12:14:54.332 INFO:tasks.workunit.client.1.smithi190.stdout:8/734: creat d3/d8/d25/d3b/d6b/ffc x:0 0 0 2024-05-06T12:14:54.334 INFO:tasks.workunit.client.1.smithi190.stdout:4/528: write d2/d11/d16/d8e/d80/f9a [890658,90451] 0 2024-05-06T12:14:54.334 INFO:tasks.workunit.client.1.smithi190.stdout:0/736: mknod d9/dc/d48/d50/d87/dd2/dcc/c108 0 2024-05-06T12:14:54.336 INFO:tasks.workunit.client.1.smithi190.stdout:7/709: truncate d0/d1/d18/d2b/d5a/f91 556790 0 2024-05-06T12:14:54.338 INFO:tasks.workunit.client.1.smithi190.stdout:6/525: mkdir df/d2c/d61/d84/d8f/db6 0 2024-05-06T12:14:54.339 INFO:tasks.workunit.client.1.smithi190.stdout:2/707: dwrite d2/fd [8388608,4194304] 0 2024-05-06T12:14:54.339 INFO:tasks.workunit.client.1.smithi190.stdout:4/529: truncate d2/d10/ff 4790590 0 2024-05-06T12:14:54.342 INFO:tasks.workunit.client.1.smithi190.stdout:8/735: getdents d3/d8/d2b/d3e/d8a 0 2024-05-06T12:14:54.344 INFO:tasks.workunit.client.1.smithi190.stdout:5/734: dwrite d2/da/d59/d5f/faf [4194304,4194304] 0 2024-05-06T12:14:54.344 INFO:tasks.workunit.client.1.smithi190.stdout:7/710: write d0/d1/d32/d45/d70/d3b/f48 [7984931,39008] 0 2024-05-06T12:14:54.346 INFO:tasks.workunit.client.1.smithi190.stdout:8/736: write d3/d8/d2b/d3e/f88 [1100300,75887] 0 2024-05-06T12:14:54.350 INFO:tasks.workunit.client.1.smithi190.stdout:0/737: rename d9/dc/d48/d50/d83/dfb to d9/dc/d48/d50/d83/dc6/d109 0 2024-05-06T12:14:54.357 INFO:tasks.workunit.client.1.smithi190.stdout:7/711: creat d0/da/d36/d53/d77/d8c/dd1/d60/dac/f100 x:0 0 0 2024-05-06T12:14:54.357 INFO:tasks.workunit.client.1.smithi190.stdout:0/738: creat d9/dcf/d21/d31/f10a x:0 0 0 2024-05-06T12:14:54.359 INFO:tasks.workunit.client.1.smithi190.stdout:8/737: mkdir d3/d8/d2b/d3e/dfd 0 2024-05-06T12:14:54.361 INFO:tasks.workunit.client.1.smithi190.stdout:2/708: link d2/d8/d19/d43/c47 d2/d8/d19/d43/d79/db2/cf5 0 2024-05-06T12:14:54.362 INFO:tasks.workunit.client.1.smithi190.stdout:4/530: dwrite d2/d11/d16/d31/fb4 [4194304,4194304] 0 2024-05-06T12:14:54.364 INFO:tasks.workunit.client.1.smithi190.stdout:6/526: dwrite df/d2c/f45 [0,4194304] 0 2024-05-06T12:14:54.364 INFO:tasks.workunit.client.1.smithi190.stdout:7/712: creat d0/d1/d18/d4e/d87/df1/f101 x:0 0 0 2024-05-06T12:14:54.368 INFO:tasks.workunit.client.1.smithi190.stdout:0/739: mknod d9/dc/d48/d50/d87/dd2/dcc/df5/c10b 0 2024-05-06T12:14:54.369 INFO:tasks.workunit.client.1.smithi190.stdout:8/738: write d3/d8/d2b/d3e/dad/fe8 [212813,43510] 0 2024-05-06T12:14:54.372 INFO:tasks.workunit.client.1.smithi190.stdout:6/527: creat df/d2c/d61/d73/fb7 x:0 0 0 2024-05-06T12:14:54.375 INFO:tasks.workunit.client.1.smithi190.stdout:0/740: symlink d9/dc/d28/d4c/d80/da7/l10c 0 2024-05-06T12:14:54.375 INFO:tasks.workunit.client.1.smithi190.stdout:6/528: stat df/d17/d37/f42 0 2024-05-06T12:14:54.376 INFO:tasks.workunit.client.1.smithi190.stdout:8/739: mknod d3/d8/d2b/d3e/d8a/cfe 0 2024-05-06T12:14:54.376 INFO:tasks.workunit.client.1.smithi190.stdout:2/709: dwrite d2/f35 [0,4194304] 0 2024-05-06T12:14:54.378 INFO:tasks.workunit.client.1.smithi190.stdout:7/713: dwrite d0/d1/d2/f1d [0,4194304] 0 2024-05-06T12:14:54.380 INFO:tasks.workunit.client.1.smithi190.stdout:0/741: chown d9/dc/d48/d50/d87 236880 1 2024-05-06T12:14:54.381 INFO:tasks.workunit.client.1.smithi190.stdout:0/742: chown d9/dc/d28/l41 155743747 1 2024-05-06T12:14:54.381 INFO:tasks.workunit.client.1.smithi190.stdout:8/740: chown d3/d8/d25/d3b/d6b 39 1 2024-05-06T12:14:54.384 INFO:tasks.workunit.client.1.smithi190.stdout:5/735: truncate d2/db6/fc9 260000 0 2024-05-06T12:14:54.384 INFO:tasks.workunit.client.1.smithi190.stdout:6/529: symlink df/d2c/d1e/d57/d89/lb8 0 2024-05-06T12:14:54.389 INFO:tasks.workunit.client.1.smithi190.stdout:2/710: write d2/d21/f70 [1643550,32012] 0 2024-05-06T12:14:54.389 INFO:tasks.workunit.client.1.smithi190.stdout:0/743: fsync d9/d6f/f7a 0 2024-05-06T12:14:54.390 INFO:tasks.workunit.client.1.smithi190.stdout:7/714: dwrite d0/d1/d18/d4e/db3/fd5 [0,4194304] 0 2024-05-06T12:14:54.390 INFO:tasks.workunit.client.1.smithi190.stdout:8/741: fsync d3/d8/d68/d9f/fbc 0 2024-05-06T12:14:54.393 INFO:tasks.workunit.client.1.smithi190.stdout:6/530: readlink df/d17/d44/l5f 0 2024-05-06T12:14:54.393 INFO:tasks.workunit.client.1.smithi190.stdout:8/742: fdatasync d3/d8/fd 0 2024-05-06T12:14:54.398 INFO:tasks.workunit.client.1.smithi190.stdout:0/744: creat d9/dcf/d21/d31/f10d x:0 0 0 2024-05-06T12:14:54.398 INFO:tasks.workunit.client.1.smithi190.stdout:2/711: truncate d2/d8/d33/d46/d5d/f93 4714529 0 2024-05-06T12:14:54.398 INFO:tasks.workunit.client.1.smithi190.stdout:8/743: chown d3/d8/d2b/d3e 17817832 1 2024-05-06T12:14:54.400 INFO:tasks.workunit.client.1.smithi190.stdout:6/531: symlink df/d17/lb9 0 2024-05-06T12:14:54.401 INFO:tasks.workunit.client.1.smithi190.stdout:2/712: mknod d2/d8/d19/d4f/cf6 0 2024-05-06T12:14:54.402 INFO:tasks.workunit.client.1.smithi190.stdout:5/736: link d2/da/c19 d2/da/d59/d5f/d6e/ced 0 2024-05-06T12:14:54.403 INFO:tasks.workunit.client.1.smithi190.stdout:7/715: dwrite d0/d1/d18/d2b/f80 [0,4194304] 0 2024-05-06T12:14:54.409 INFO:tasks.workunit.client.1.smithi190.stdout:8/744: getdents d3/d35/d55/d61 0 2024-05-06T12:14:54.410 INFO:tasks.workunit.client.1.smithi190.stdout:6/532: creat df/d2c/d61/d87/fba x:0 0 0 2024-05-06T12:14:54.412 INFO:tasks.workunit.client.1.smithi190.stdout:0/745: dwrite d9/fda [0,4194304] 0 2024-05-06T12:14:54.413 INFO:tasks.workunit.client.1.smithi190.stdout:5/737: fdatasync d2/da/d59/d5f/f58 0 2024-05-06T12:14:54.415 INFO:tasks.workunit.client.1.smithi190.stdout:7/716: fdatasync d0/da/d36/f9d 0 2024-05-06T12:14:54.417 INFO:tasks.workunit.client.1.smithi190.stdout:8/745: mkdir d3/d8/d25/d3b/d90/dff 0 2024-05-06T12:14:54.419 INFO:tasks.workunit.client.1.smithi190.stdout:4/531: dread d2/f37 [0,4194304] 0 2024-05-06T12:14:54.420 INFO:tasks.workunit.client.1.smithi190.stdout:0/746: mknod d9/dc/d48/d50/d83/dc6/d109/c10e 0 2024-05-06T12:14:54.422 INFO:tasks.workunit.client.1.smithi190.stdout:4/532: rmdir d2 39 2024-05-06T12:14:54.423 INFO:tasks.workunit.client.1.smithi190.stdout:0/747: fdatasync d9/f5f 0 2024-05-06T12:14:54.425 INFO:tasks.workunit.client.1.smithi190.stdout:8/746: read d3/d8/d25/fac [1876487,45238] 0 2024-05-06T12:14:54.427 INFO:tasks.workunit.client.1.smithi190.stdout:4/533: read d2/d11/d16/f90 [1694198,25516] 0 2024-05-06T12:14:54.428 INFO:tasks.workunit.client.1.smithi190.stdout:5/738: dwrite d2/da/d59/d5f/d2e/da7/d9d/fda [0,4194304] 0 2024-05-06T12:14:54.429 INFO:tasks.workunit.client.1.smithi190.stdout:7/717: link d0/d1/d32/d45/d70/d33/d57/fc0 d0/d14/d6a/f102 0 2024-05-06T12:14:54.431 INFO:tasks.workunit.client.1.smithi190.stdout:6/533: dwrite df/d17/d44/fa8 [0,4194304] 0 2024-05-06T12:14:54.431 INFO:tasks.workunit.client.1.smithi190.stdout:7/718: chown d0/da/d36/d53/d5c/cce 0 1 2024-05-06T12:14:54.436 INFO:tasks.workunit.client.1.smithi190.stdout:4/534: mknod d2/d11/d16/d8e/d80/cbc 0 2024-05-06T12:14:54.437 INFO:tasks.workunit.client.1.smithi190.stdout:7/719: write d0/d1/d32/d45/d70/d3b/f48 [3372590,30585] 0 2024-05-06T12:14:54.437 INFO:tasks.workunit.client.1.smithi190.stdout:8/747: creat d3/d8/d25/d3b/d90/dff/f100 x:0 0 0 2024-05-06T12:14:54.438 INFO:tasks.workunit.client.1.smithi190.stdout:6/534: dread - df/d2c/d28/f40 zero size 2024-05-06T12:14:54.439 INFO:tasks.workunit.client.1.smithi190.stdout:4/535: fdatasync d2/d11/f14 0 2024-05-06T12:14:54.440 INFO:tasks.workunit.client.1.smithi190.stdout:5/739: dwrite d2/f44 [0,4194304] 0 2024-05-06T12:14:54.453 INFO:tasks.workunit.client.1.smithi190.stdout:7/720: dwrite d0/d4/f6b [0,4194304] 0 2024-05-06T12:14:54.453 INFO:tasks.workunit.client.1.smithi190.stdout:8/748: link d3/d8/d25/d3b/d6b/d7a/f97 d3/d35/d55/f101 0 2024-05-06T12:14:54.454 INFO:tasks.workunit.client.1.smithi190.stdout:3/806: sync 2024-05-06T12:14:54.456 INFO:tasks.workunit.client.1.smithi190.stdout:6/535: dwrite df/d17/d37/f8c [0,4194304] 0 2024-05-06T12:14:54.457 INFO:tasks.workunit.client.1.smithi190.stdout:4/536: dwrite d2/f64 [0,4194304] 0 2024-05-06T12:14:54.460 INFO:tasks.workunit.client.1.smithi190.stdout:6/536: readlink df/d17/l86 0 2024-05-06T12:14:54.462 INFO:tasks.workunit.client.1.smithi190.stdout:3/807: rmdir d5/d14/d11/d21/d44/d92/d9e/da4/d105/db3/dfa 39 2024-05-06T12:14:54.464 INFO:tasks.workunit.client.1.smithi190.stdout:3/808: chown d5/d14/d11/d21/d44/d80/d82/fd3 859221 1 2024-05-06T12:14:54.465 INFO:tasks.workunit.client.1.smithi190.stdout:6/537: getdents df/d2c/d61/d84/d8f/db6 0 2024-05-06T12:14:54.466 INFO:tasks.workunit.client.1.smithi190.stdout:6/538: chown df/d2c/d1e/d57 28287 1 2024-05-06T12:14:54.467 INFO:tasks.workunit.client.1.smithi190.stdout:6/539: truncate df/d2c/f36 4666180 0 2024-05-06T12:14:54.467 INFO:tasks.workunit.client.1.smithi190.stdout:4/537: link d2/d11/d16/d8e/c5b d2/d11/d16/d8e/d7b/cbd 0 2024-05-06T12:14:54.468 INFO:tasks.workunit.client.1.smithi190.stdout:6/540: write df/d17/d37/f58 [1186781,97657] 0 2024-05-06T12:14:54.469 INFO:tasks.workunit.client.1.smithi190.stdout:4/538: write d2/d11/d16/d4e/f85 [27247,84846] 0 2024-05-06T12:14:54.470 INFO:tasks.workunit.client.1.smithi190.stdout:3/809: link d5/d14/d11/d21/d44/d80/d82/l72 d5/d7/d6d/dc6/l119 0 2024-05-06T12:14:54.474 INFO:tasks.workunit.client.1.smithi190.stdout:4/539: creat d2/d7d/fbe x:0 0 0 2024-05-06T12:14:54.475 INFO:tasks.workunit.client.1.smithi190.stdout:6/541: truncate df/d17/d44/fa7 433993 0 2024-05-06T12:14:54.476 INFO:tasks.workunit.client.1.smithi190.stdout:4/540: mknod d2/d11/d16/d61/cbf 0 2024-05-06T12:14:54.479 INFO:tasks.workunit.client.1.smithi190.stdout:3/810: rename d5/d14/d11/d21/d44/d80/d82/d3d/ded/f42 to d5/d7/d15/f11a 0 2024-05-06T12:14:54.484 INFO:tasks.workunit.client.1.smithi190.stdout:4/541: chown d2/d11/d16/d8e/d7b/cbd 4808 1 2024-05-06T12:14:54.485 INFO:tasks.workunit.client.1.smithi190.stdout:3/811: symlink d5/d14/d11/d21/d44/d92/d9e/da4/d105/d111/l11b 0 2024-05-06T12:14:54.489 INFO:tasks.workunit.client.1.smithi190.stdout:4/542: truncate d2/d10/fb8 785037 0 2024-05-06T12:14:54.493 INFO:tasks.workunit.client.1.smithi190.stdout:3/812: link d5/d7/d7f/f8d d5/d14/d11/d21/d44/d92/d9e/da4/d105/db3/dfa/f11c 0 2024-05-06T12:14:54.495 INFO:tasks.workunit.client.1.smithi190.stdout:3/813: truncate d5/d14/d11/d21/d44/d80/d82/d3d/ff4 6495 0 2024-05-06T12:14:54.497 INFO:tasks.workunit.client.1.smithi190.stdout:3/814: creat d5/d14/d11/d21/d44/d92/d9e/da4/d105/dfb/f11d x:0 0 0 2024-05-06T12:14:54.499 INFO:tasks.workunit.client.1.smithi190.stdout:3/815: creat d5/d7/d106/f11e x:0 0 0 2024-05-06T12:14:54.500 INFO:tasks.workunit.client.1.smithi190.stdout:3/816: creat d5/d14/d11/d21/d44/d80/f11f x:0 0 0 2024-05-06T12:14:54.504 INFO:tasks.workunit.client.1.smithi190.stdout:4/543: dread d2/d11/d16/d61/d9e/d6e/d78/f7a [0,4194304] 0 2024-05-06T12:14:54.521 INFO:tasks.workunit.client.1.smithi190.stdout:4/544: fsync d2/d11/d16/d4e/f85 0 2024-05-06T12:14:54.522 INFO:tasks.workunit.client.1.smithi190.stdout:5/740: write d2/da/d59/d5f/d2e/da7/d9d/fda [4591618,11742] 0 2024-05-06T12:14:54.525 INFO:tasks.workunit.client.1.smithi190.stdout:4/545: write d2/d11/d16/d61/d66/d6b/f88 [4593682,30137] 0 2024-05-06T12:14:54.525 INFO:tasks.workunit.client.1.smithi190.stdout:5/741: read d2/f60 [1340269,10202] 0 2024-05-06T12:14:54.528 INFO:tasks.workunit.client.1.smithi190.stdout:2/713: dwrite d2/d21/f5e [0,4194304] 0 2024-05-06T12:14:54.530 INFO:tasks.workunit.client.1.smithi190.stdout:4/546: creat d2/d10/fc0 x:0 0 0 2024-05-06T12:14:54.530 INFO:tasks.workunit.client.1.smithi190.stdout:0/748: dwrite d9/f37 [0,4194304] 0 2024-05-06T12:14:54.531 INFO:tasks.workunit.client.1.smithi190.stdout:5/742: mknod d2/db6/cee 0 2024-05-06T12:14:54.536 INFO:tasks.workunit.client.1.smithi190.stdout:4/547: creat d2/d11/d16/d61/d9e/fc1 x:0 0 0 2024-05-06T12:14:54.538 INFO:tasks.workunit.client.1.smithi190.stdout:5/743: creat d2/d6c/fef x:0 0 0 2024-05-06T12:14:54.543 INFO:tasks.workunit.client.1.smithi190.stdout:7/721: dwrite d0/d1/d2/f7c [0,4194304] 0 2024-05-06T12:14:54.545 INFO:tasks.workunit.client.1.smithi190.stdout:0/749: rename d9/dc/db6/fb7 to d9/d91/f10f 0 2024-05-06T12:14:54.547 INFO:tasks.workunit.client.1.smithi190.stdout:2/714: truncate d2/d8/d33/d46/dab/fee 1673892 0 2024-05-06T12:14:54.549 INFO:tasks.workunit.client.1.smithi190.stdout:4/548: dread d2/f37 [0,4194304] 0 2024-05-06T12:14:54.549 INFO:tasks.workunit.client.1.smithi190.stdout:7/722: readlink d0/d14/d5d/l28 0 2024-05-06T12:14:54.557 INFO:tasks.workunit.client.1.smithi190.stdout:0/750: dread d9/dcf/d56/dab/fba [0,4194304] 0 2024-05-06T12:14:54.558 INFO:tasks.workunit.client.1.smithi190.stdout:2/715: write d2/d4c/fd2 [517494,98837] 0 2024-05-06T12:14:54.558 INFO:tasks.workunit.client.1.smithi190.stdout:5/744: dwrite d2/f30 [4194304,4194304] 0 2024-05-06T12:14:54.558 INFO:tasks.workunit.client.1.smithi190.stdout:6/542: dwrite df/d2c/d28/f40 [0,4194304] 0 2024-05-06T12:14:54.560 INFO:tasks.workunit.client.1.smithi190.stdout:6/543: readlink df/d17/l86 0 2024-05-06T12:14:54.568 INFO:tasks.workunit.client.1.smithi190.stdout:7/723: mknod d0/d1/d2/d84/dc6/c103 0 2024-05-06T12:14:54.569 INFO:tasks.workunit.client.1.smithi190.stdout:5/745: fdatasync d2/da/d59/d5f/d6e/daa/fe4 0 2024-05-06T12:14:54.570 INFO:tasks.workunit.client.1.smithi190.stdout:2/716: creat d2/d8/d33/dc2/dcd/ff7 x:0 0 0 2024-05-06T12:14:54.576 INFO:tasks.workunit.client.1.smithi190.stdout:3/817: dwrite d5/d14/d11/d21/d44/d80/d82/d3d/ded/d33/f66 [0,4194304] 0 2024-05-06T12:14:54.580 INFO:tasks.workunit.client.1.smithi190.stdout:4/549: dwrite d2/d10/fba [0,4194304] 0 2024-05-06T12:14:54.580 INFO:tasks.workunit.client.1.smithi190.stdout:2/717: dread d2/fb6 [0,4194304] 0 2024-05-06T12:14:54.582 INFO:tasks.workunit.client.1.smithi190.stdout:6/544: dwrite df/d2c/d28/f9f [0,4194304] 0 2024-05-06T12:14:54.587 INFO:tasks.workunit.client.1.smithi190.stdout:6/545: mkdir df/d2c/dae/dbb 0 2024-05-06T12:14:54.589 INFO:tasks.workunit.client.1.smithi190.stdout:5/746: chown d2/da/d59/d5f/d2e/l46 11 1 2024-05-06T12:14:54.593 INFO:tasks.workunit.client.1.smithi190.stdout:5/747: chown d2/c14 8 1 2024-05-06T12:14:54.597 INFO:tasks.workunit.client.1.smithi190.stdout:2/718: fsync d2/d8/d33/d46/dab/f6d 0 2024-05-06T12:14:54.597 INFO:tasks.workunit.client.1.smithi190.stdout:7/724: dwrite d0/d1/d18/f8a [0,4194304] 0 2024-05-06T12:14:54.603 INFO:tasks.workunit.client.1.smithi190.stdout:7/725: creat d0/db4/f104 x:0 0 0 2024-05-06T12:14:54.607 INFO:tasks.workunit.client.1.smithi190.stdout:6/546: dwrite df/d17/d44/f54 [0,4194304] 0 2024-05-06T12:14:54.607 INFO:tasks.workunit.client.1.smithi190.stdout:2/719: link d2/f38 d2/d8/d33/dc2/ff8 0 2024-05-06T12:14:54.608 INFO:tasks.workunit.client.1.smithi190.stdout:0/751: dread d9/dc/f45 [0,4194304] 0 2024-05-06T12:14:54.608 INFO:tasks.workunit.client.1.smithi190.stdout:6/547: chown df/d2c/d1e/f7a 3 1 2024-05-06T12:14:54.613 INFO:tasks.workunit.client.1.smithi190.stdout:2/720: creat d2/d8/d19/da9/ff9 x:0 0 0 2024-05-06T12:14:54.614 INFO:tasks.workunit.client.1.smithi190.stdout:7/726: dread d0/da/d36/d53/d77/d8c/dd1/d60/fa8 [0,4194304] 0 2024-05-06T12:14:54.615 INFO:tasks.workunit.client.1.smithi190.stdout:0/752: mkdir d9/dc/d28/dc5/dd5/de4/dea/dfd/d110 0 2024-05-06T12:14:54.616 INFO:tasks.workunit.client.1.smithi190.stdout:7/727: write d0/d1/d18/fa2 [5222613,42036] 0 2024-05-06T12:14:54.617 INFO:tasks.workunit.client.1.smithi190.stdout:2/721: stat d2/d8/d71/leb 0 2024-05-06T12:14:54.618 INFO:tasks.workunit.client.1.smithi190.stdout:6/548: rename df/d2c/d61/d84/cb2 to df/d12/cbc 0 2024-05-06T12:14:54.619 INFO:tasks.workunit.client.1.smithi190.stdout:2/722: read d2/d8/d33/d46/dab/f59 [3154241,157] 0 2024-05-06T12:14:54.621 INFO:tasks.workunit.client.1.smithi190.stdout:0/753: rmdir d9/dc/d48 39 2024-05-06T12:14:54.623 INFO:tasks.workunit.client.1.smithi190.stdout:6/549: rmdir df/d2c/d61 39 2024-05-06T12:14:54.626 INFO:tasks.workunit.client.1.smithi190.stdout:0/754: rename d9/dc/d28/cc0 to d9/dc/d28/d4c/c111 0 2024-05-06T12:14:54.627 INFO:tasks.workunit.client.1.smithi190.stdout:7/728: mknod d0/da/c105 0 2024-05-06T12:14:54.628 INFO:tasks.workunit.client.1.smithi190.stdout:5/748: dread d2/da/d59/d5f/d2e/d3b/d5a/f85 [0,4194304] 0 2024-05-06T12:14:54.629 INFO:tasks.workunit.client.1.smithi190.stdout:0/755: fdatasync d9/dcf/d56/dab/fbc 0 2024-05-06T12:14:54.630 INFO:tasks.workunit.client.1.smithi190.stdout:6/550: fdatasync df/f99 0 2024-05-06T12:14:54.631 INFO:tasks.workunit.client.1.smithi190.stdout:2/723: mknod d2/d8/d33/dc2/dcb/cfa 0 2024-05-06T12:14:54.631 INFO:tasks.workunit.client.1.smithi190.stdout:3/818: dread d5/d7/d6d/dc6/fd2 [0,4194304] 0 2024-05-06T12:14:54.635 INFO:tasks.workunit.client.1.smithi190.stdout:7/729: chown d0/da/c47 1341967 1 2024-05-06T12:14:54.637 INFO:tasks.workunit.client.1.smithi190.stdout:4/550: dwrite d2/d10/ff [4194304,4194304] 0 2024-05-06T12:14:54.640 INFO:tasks.workunit.client.1.smithi190.stdout:4/551: fdatasync d2/d11/d16/d41/f65 0 2024-05-06T12:14:54.643 INFO:tasks.workunit.client.1.smithi190.stdout:1/573: sync 2024-05-06T12:14:54.644 INFO:tasks.workunit.client.1.smithi190.stdout:4/552: write d2/d11/f14 [3466746,71466] 0 2024-05-06T12:14:54.645 INFO:tasks.workunit.client.1.smithi190.stdout:6/551: dwrite df/d2c/d28/f40 [0,4194304] 0 2024-05-06T12:14:54.647 INFO:tasks.workunit.client.1.smithi190.stdout:0/756: link d9/dc/d28/dc5/dd5/de4/dea/fec d9/dc/d28/dc5/dd5/de4/dea/dfd/d110/f112 0 2024-05-06T12:14:54.648 INFO:tasks.workunit.client.1.smithi190.stdout:5/749: rename d2/da/d59/d5f/f3c to d2/da/d59/d5f/ff0 0 2024-05-06T12:14:54.651 INFO:tasks.workunit.client.1.smithi190.stdout:6/552: mkdir df/d12/dbd 0 2024-05-06T12:14:54.651 INFO:tasks.workunit.client.1.smithi190.stdout:0/757: rmdir d9/dc/db6 39 2024-05-06T12:14:54.654 INFO:tasks.workunit.client.1.smithi190.stdout:3/819: dwrite d5/d7/d7f/f8d [4194304,4194304] 0 2024-05-06T12:14:54.657 INFO:tasks.workunit.client.1.smithi190.stdout:6/553: fdatasync df/d17/d37/f49 0 2024-05-06T12:14:54.657 INFO:tasks.workunit.client.1.smithi190.stdout:5/750: creat d2/dea/ff1 x:0 0 0 2024-05-06T12:14:54.658 INFO:tasks.workunit.client.1.smithi190.stdout:0/758: rmdir d9/dcf/d21 39 2024-05-06T12:14:54.658 INFO:tasks.workunit.client.1.smithi190.stdout:5/751: stat d2/c4d 0 2024-05-06T12:14:54.658 INFO:tasks.workunit.client.1.smithi190.stdout:2/724: dread d2/d8/d19/f8a [0,4194304] 0 2024-05-06T12:14:54.661 INFO:tasks.workunit.client.1.smithi190.stdout:1/574: getdents d5/d4f/d65/d32/d47 0 2024-05-06T12:14:54.663 INFO:tasks.workunit.client.1.smithi190.stdout:6/554: unlink df/d12/f83 0 2024-05-06T12:14:54.664 INFO:tasks.workunit.client.1.smithi190.stdout:0/759: mkdir d9/d51/d113 0 2024-05-06T12:14:54.665 INFO:tasks.workunit.client.1.smithi190.stdout:2/725: symlink d2/d8/d33/d46/dab/dbb/lfb 0 2024-05-06T12:14:54.672 INFO:tasks.workunit.client.1.smithi190.stdout:3/820: creat d5/d14/d11/d21/d44/d80/d82/d3d/ded/d33/f120 x:0 0 0 2024-05-06T12:14:54.673 INFO:tasks.workunit.client.1.smithi190.stdout:5/752: mknod d2/da/d59/d5f/d2e/d3b/cf2 0 2024-05-06T12:14:54.673 INFO:tasks.workunit.client.1.smithi190.stdout:1/575: creat d5/d4f/d65/d32/d47/d53/d98/da7/fc6 x:0 0 0 2024-05-06T12:14:54.675 INFO:tasks.workunit.client.1.smithi190.stdout:2/726: truncate d2/d8/d19/d43/d79/fd6 653103 0 2024-05-06T12:14:54.675 INFO:tasks.workunit.client.1.smithi190.stdout:8/749: sync 2024-05-06T12:14:54.675 INFO:tasks.workunit.client.1.smithi190.stdout:9/772: sync 2024-05-06T12:14:54.676 INFO:tasks.workunit.client.1.smithi190.stdout:6/555: dread - df/d2c/d61/d84/fb4 zero size 2024-05-06T12:14:54.678 INFO:tasks.workunit.client.1.smithi190.stdout:6/556: write f7 [3338186,64733] 0 2024-05-06T12:14:54.678 INFO:tasks.workunit.client.1.smithi190.stdout:8/750: write d3/fd7 [220957,110496] 0 2024-05-06T12:14:54.678 INFO:tasks.workunit.client.1.smithi190.stdout:7/730: dread d0/da/d36/d42/fc4 [0,4194304] 0 2024-05-06T12:14:54.685 INFO:tasks.workunit.client.1.smithi190.stdout:7/731: dwrite d0/d1/d18/d2b/f30 [0,4194304] 0 2024-05-06T12:14:54.686 INFO:tasks.workunit.client.1.smithi190.stdout:7/732: fsync d0/da/d36/d53/d77/d8c/dd1/d60/dac/f100 0 2024-05-06T12:14:54.691 INFO:tasks.workunit.client.1.smithi190.stdout:4/553: getdents d2/d11/d16/d61/d9e/d6e/d78 0 2024-05-06T12:14:54.692 INFO:tasks.workunit.client.1.smithi190.stdout:5/753: mkdir d2/d65/d83/df3 0 2024-05-06T12:14:54.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.692+0000 7fc22b9cf700 1 -- 172.21.15.92:0/1225954513 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2240fbcb0 msgr2=0x7fc2240fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:54.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.692+0000 7fc22b9cf700 1 --2- 172.21.15.92:0/1225954513 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2240fbcb0 0x7fc2240fc0c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc220004940 tx=0x7fc220009e30 comp rx=0 tx=0).stop 2024-05-06T12:14:54.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.692+0000 7fc22b9cf700 1 -- 172.21.15.92:0/1225954513 shutdown_connections 2024-05-06T12:14:54.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.692+0000 7fc22b9cf700 1 --2- 172.21.15.92:0/1225954513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2240fcf60 0x7fc2240fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:54.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.692+0000 7fc22b9cf700 1 --2- 172.21.15.92:0/1225954513 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2240fbcb0 0x7fc2240fc0c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:54.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.692+0000 7fc22b9cf700 1 -- 172.21.15.92:0/1225954513 >> 172.21.15.92:0/1225954513 conn(0x7fc2240f7430 msgr2=0x7fc2240f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:14:54.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.692+0000 7fc22b9cf700 1 -- 172.21.15.92:0/1225954513 shutdown_connections 2024-05-06T12:14:54.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.692+0000 7fc22b9cf700 1 -- 172.21.15.92:0/1225954513 wait complete. 2024-05-06T12:14:54.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.693+0000 7fc22b9cf700 1 Processor -- start 2024-05-06T12:14:54.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.693+0000 7fc22b9cf700 1 -- start start 2024-05-06T12:14:54.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.693+0000 7fc22b9cf700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2240fbcb0 0x7fc22410dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:54.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.693+0000 7fc22b9cf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2240fcf60 0x7fc22410dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:54.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.694+0000 7fc22b9cf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22410e5f0 con 0x7fc2240fcf60 2024-05-06T12:14:54.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.694+0000 7fc22b9cf700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22410eda0 con 0x7fc2240fbcb0 2024-05-06T12:14:54.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.694+0000 7fc22976b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2240fbcb0 0x7fc22410dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:54.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.694+0000 7fc228f6a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2240fcf60 0x7fc22410dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:54.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.694+0000 7fc228f6a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2240fcf60 0x7fc22410dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36090/0 (socket says 172.21.15.92:36090) 2024-05-06T12:14:54.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.694+0000 7fc228f6a700 1 -- 172.21.15.92:0/554454768 learned_addr learned my addr 172.21.15.92:0/554454768 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:14:54.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.694+0000 7fc228f6a700 1 -- 172.21.15.92:0/554454768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2240fbcb0 msgr2=0x7fc22410dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:54.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.694+0000 7fc228f6a700 1 --2- 172.21.15.92:0/554454768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2240fbcb0 0x7fc22410dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:54.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.694+0000 7fc228f6a700 1 -- 172.21.15.92:0/554454768 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc220005040 con 0x7fc2240fcf60 2024-05-06T12:14:54.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.695+0000 7fc228f6a700 1 --2- 172.21.15.92:0/554454768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2240fcf60 0x7fc22410dff0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc214002a50 tx=0x7fc2140046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:14:54.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.695+0000 7fc219ffb700 1 -- 172.21.15.92:0/554454768 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc214007cc0 con 0x7fc2240fcf60 2024-05-06T12:14:54.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.695+0000 7fc22b9cf700 1 -- 172.21.15.92:0/554454768 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc22410f080 con 0x7fc2240fcf60 2024-05-06T12:14:54.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.695+0000 7fc219ffb700 1 -- 172.21.15.92:0/554454768 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc21400f040 con 0x7fc2240fcf60 2024-05-06T12:14:54.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.695+0000 7fc219ffb700 1 -- 172.21.15.92:0/554454768 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc21400da30 con 0x7fc2240fcf60 2024-05-06T12:14:54.700 INFO:tasks.workunit.client.1.smithi190.stdout:3/821: rename d5/d14/d11/f13 to d5/d14/d11/d21/d44/d80/d82/d3d/ded/d33/d5c/f121 0 2024-05-06T12:14:54.700 INFO:tasks.workunit.client.1.smithi190.stdout:4/554: mknod d2/d11/d16/d61/d66/cc2 0 2024-05-06T12:14:54.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.695+0000 7fc22b9cf700 1 -- 172.21.15.92:0/554454768 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc22410f5c0 con 0x7fc2240fcf60 2024-05-06T12:14:54.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.696+0000 7fc22b9cf700 1 -- 172.21.15.92:0/554454768 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc224101af0 con 0x7fc2240fcf60 2024-05-06T12:14:54.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.696+0000 7fc219ffb700 1 -- 172.21.15.92:0/554454768 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7fc21400db90 con 0x7fc2240fcf60 2024-05-06T12:14:54.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.697+0000 7fc219ffb700 1 --2- 172.21.15.92:0/554454768 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fc21006b7f0 0x7fc21006dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:54.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.697+0000 7fc219ffb700 1 -- 172.21.15.92:0/554454768 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fc21408a1b0 con 0x7fc2240fcf60 2024-05-06T12:14:54.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.697+0000 7fc22976b700 1 --2- 172.21.15.92:0/554454768 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fc21006b7f0 0x7fc21006dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:54.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.698+0000 7fc22976b700 1 --2- 172.21.15.92:0/554454768 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fc21006b7f0 0x7fc21006dca0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc22000b3b0 tx=0x7fc220009e30 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:14:54.702 INFO:tasks.workunit.client.1.smithi190.stdout:5/754: creat d2/dea/ff4 x:0 0 0 2024-05-06T12:14:54.703 INFO:tasks.workunit.client.1.smithi190.stdout:0/760: link d9/dc/d28/lc2 d9/d51/l114 0 2024-05-06T12:14:54.703 INFO:tasks.workunit.client.1.smithi190.stdout:9/773: dwrite db/dd/df/dfe/d87/d22/d49/fd9 [0,4194304] 0 2024-05-06T12:14:54.703 INFO:tasks.workunit.client.1.smithi190.stdout:4/555: dread - d2/d11/d16/d61/d66/fae zero size 2024-05-06T12:14:54.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:54.701+0000 7fc219ffb700 1 -- 172.21.15.92:0/554454768 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fc2140577a0 con 0x7fc2240fcf60 2024-05-06T12:14:54.705 INFO:tasks.workunit.client.1.smithi190.stdout:6/557: unlink df/d17/l91 0 2024-05-06T12:14:54.705 INFO:tasks.workunit.client.1.smithi190.stdout:4/556: fdatasync d2/f64 0 2024-05-06T12:14:54.709 INFO:tasks.workunit.client.1.smithi190.stdout:2/727: link d2/d8/d19/f8a d2/d8/ffc 0 2024-05-06T12:14:54.710 INFO:tasks.workunit.client.1.smithi190.stdout:0/761: rmdir d9/dc/d28/d77 39 2024-05-06T12:14:54.711 INFO:tasks.workunit.client.1.smithi190.stdout:3/822: dread d5/d14/d11/d21/d44/d80/d82/d3d/ded/d33/f66 [0,4194304] 0 2024-05-06T12:14:54.714 INFO:tasks.workunit.client.1.smithi190.stdout:9/774: write db/dd/df/dfe/d87/d22/d49/d88/f7a [4576508,53787] 0 2024-05-06T12:14:54.718 INFO:tasks.workunit.client.1.smithi190.stdout:6/558: rename df/d2c/d28/l6c to df/d2c/d1e/d57/d89/lbe 0 2024-05-06T12:14:54.719 INFO:tasks.workunit.client.1.smithi190.stdout:3/823: dwrite d5/d14/d11/d21/d44/d92/d9e/da4/d105/db3/da1/fc3 [0,4194304] 0 2024-05-06T12:14:54.720 INFO:tasks.workunit.client.1.smithi190.stdout:6/559: chown df/d17/d37/f49 122127 1 2024-05-06T12:14:54.720 INFO:tasks.workunit.client.1.smithi190.stdout:3/824: fsync d5/d79/f95 0 2024-05-06T12:14:54.723 INFO:tasks.workunit.client.1.smithi190.stdout:2/728: unlink d2/l9 0 2024-05-06T12:14:54.723 INFO:tasks.workunit.client.1.smithi190.stdout:8/751: getdents d3/d8/d68/d9f 0 2024-05-06T12:14:54.725 INFO:tasks.workunit.client.1.smithi190.stdout:2/729: write d2/d4c/ff1 [76889,49050] 0 2024-05-06T12:14:54.726 INFO:tasks.workunit.client.1.smithi190.stdout:8/752: read d3/d8/d25/fac [1393049,106180] 0 2024-05-06T12:14:54.728 INFO:tasks.workunit.client.1.smithi190.stdout:8/753: write d3/d8/f2c [3225089,71268] 0 2024-05-06T12:14:54.729 INFO:tasks.workunit.client.1.smithi190.stdout:7/733: getdents d0/d1/d2 0 2024-05-06T12:14:54.731 INFO:tasks.workunit.client.1.smithi190.stdout:8/754: truncate d3/d8/d25/d65/dc6/fed 693982 0 2024-05-06T12:14:54.731 INFO:tasks.workunit.client.1.smithi190.stdout:9/775: fdatasync db/dd/df/dfe/d87/d22/f55 0 2024-05-06T12:14:54.731 INFO:tasks.workunit.client.1.smithi190.stdout:7/734: truncate d0/d14/d6a/d8f/fe8 905108 0 2024-05-06T12:14:54.733 INFO:tasks.workunit.client.1.smithi190.stdout:9/776: read - db/dd/df/dfe/d87/d22/d49/d8c/fde zero size 2024-05-06T12:14:54.733 INFO:tasks.workunit.client.1.smithi190.stdout:7/735: chown d0/d1/d18/d2b/f83 0 1 2024-05-06T12:14:54.734 INFO:tasks.workunit.client.1.smithi190.stdout:6/560: mknod df/d2c/d1e/d57/d89/cbf 0 2024-05-06T12:14:54.734 INFO:tasks.workunit.client.1.smithi190.stdout:3/825: rmdir d5/d14/d11/d21/d44 39 2024-05-06T12:14:54.740 INFO:tasks.workunit.client.1.smithi190.stdout:8/755: unlink d3/d8/d25/f9b 0 2024-05-06T12:14:54.742 INFO:tasks.workunit.client.1.smithi190.stdout:6/561: mkdir df/d2c/dae/dc0 0 2024-05-06T12:14:54.743 INFO:tasks.workunit.client.1.smithi190.stdout:9/777: mkdir db/dd/d5c/dac/deb/d10e 0 2024-05-06T12:14:54.745 INFO:tasks.workunit.client.1.smithi190.stdout:0/762: unlink d9/dc/d28/d77/fd4 0 2024-05-06T12:14:54.747 INFO:tasks.workunit.client.1.smithi190.stdout:4/557: getdents d2/d11/d16/d61/d9e 0 2024-05-06T12:14:54.749 INFO:tasks.workunit.client.1.smithi190.stdout:7/736: symlink d0/d1/l106 0 2024-05-06T12:14:54.750 INFO:tasks.workunit.client.1.smithi190.stdout:3/826: dwrite d5/d14/d11/d21/d44/d92/d9e/da4/df3/f108 [0,4194304] 0 2024-05-06T12:14:54.752 INFO:tasks.workunit.client.1.smithi190.stdout:8/756: symlink d3/d8/d25/d3b/d90/dff/l102 0 2024-05-06T12:14:54.755 INFO:tasks.workunit.client.1.smithi190.stdout:8/757: dread - d3/d8/fe9 zero size 2024-05-06T12:14:54.762 INFO:tasks.workunit.client.1.smithi190.stdout:7/737: dwrite d0/d1/d32/d45/d70/d3b/f94 [0,4194304] 0 2024-05-06T12:14:54.765 INFO:tasks.workunit.client.1.smithi190.stdout:2/730: getdents d2/d8/d33/d46/dab 0 2024-05-06T12:14:54.766 INFO:tasks.workunit.client.1.smithi190.stdout:2/731: readlink d2/d4c/ldf 0 2024-05-06T12:14:54.767 INFO:tasks.workunit.client.1.smithi190.stdout:6/562: creat df/fc1 x:0 0 0 2024-05-06T12:14:54.768 INFO:tasks.workunit.client.1.smithi190.stdout:3/827: dwrite d5/d14/d11/d21/d44/d92/d9e/da4/d105/db3/dfa/f11c [8388608,4194304] 0 2024-05-06T12:14:54.770 INFO:tasks.workunit.client.1.smithi190.stdout:7/738: mkdir d0/d1/d18/d4e/d87/d107 0 2024-05-06T12:14:54.771 INFO:tasks.workunit.client.1.smithi190.stdout:7/739: chown d0/db4/lc8 1 1 2024-05-06T12:14:54.778 INFO:tasks.workunit.client.1.smithi190.stdout:4/558: dread d2/f50 [0,4194304] 0 2024-05-06T12:14:54.781 INFO:tasks.workunit.client.1.smithi190.stdout:2/732: rename d2/d4c/ff1 to d2/d8/d33/dc2/ffd 0 2024-05-06T12:14:54.781 INFO:tasks.workunit.client.1.smithi190.stdout:6/563: dread df/d12/f21 [0,4194304] 0 2024-05-06T12:14:54.784 INFO:tasks.workunit.client.1.smithi190.stdout:2/733: write d2/d8/d33/d46/dab/f6d [356460,26148] 0 2024-05-06T12:14:54.786 INFO:tasks.workunit.client.1.smithi190.stdout:8/758: link d3/d35/d55/fa5 d3/d8/db0/f103 0 2024-05-06T12:14:54.788 INFO:tasks.workunit.client.1.smithi190.stdout:3/828: dwrite d5/d7/d6d/dc6/fd2 [4194304,4194304] 0 2024-05-06T12:14:54.791 INFO:tasks.workunit.client.1.smithi190.stdout:4/559: dwrite d2/d11/d16/d61/d66/d6b/f88 [0,4194304] 0 2024-05-06T12:14:54.792 INFO:tasks.workunit.client.1.smithi190.stdout:3/829: readlink d5/d14/d11/d21/d44/d80/d82/d3d/ded/d33/l37 0 2024-05-06T12:14:54.792 INFO:tasks.workunit.client.1.smithi190.stdout:6/564: write df/d17/f26 [2703294,5602] 0 2024-05-06T12:14:54.795 INFO:tasks.workunit.client.1.smithi190.stdout:4/560: read - d2/d11/d16/d41/f65 zero size 2024-05-06T12:14:54.795 INFO:tasks.workunit.client.1.smithi190.stdout:6/565: write df/d17/d37/f9c [771971,103844] 0 2024-05-06T12:14:54.796 INFO:tasks.workunit.client.1.smithi190.stdout:0/763: link d9/d51/d93/le3 d9/l115 0 2024-05-06T12:14:54.797 INFO:tasks.workunit.client.1.smithi190.stdout:4/561: write d2/d11/d16/d8e/f57 [31363,59731] 0 2024-05-06T12:14:54.799 INFO:tasks.workunit.client.1.smithi190.stdout:7/740: getdents d0/d1/d2/d84/dc6/dfb 0 2024-05-06T12:14:54.806 INFO:tasks.workunit.client.1.smithi190.stdout:2/734: rename d2/d10 to d2/d8/d33/dc2/dcd/dfe 0 2024-05-06T12:14:54.808 INFO:tasks.workunit.client.1.smithi190.stdout:3/830: mknod d5/d14/d11/d21/d44/d80/ddb/c122 0 2024-05-06T12:14:54.808 INFO:tasks.workunit.client.1.smithi190.stdout:9/778: dwrite db/dd/d16/f4f [0,4194304] 0 2024-05-06T12:14:54.810 INFO:tasks.workunit.client.1.smithi190.stdout:3/831: truncate d5/ffe 5080202 0 2024-05-06T12:14:54.811 INFO:tasks.workunit.client.1.smithi190.stdout:0/764: mknod d9/dc/d28/d4c/c116 0 2024-05-06T12:14:54.812 INFO:tasks.workunit.client.1.smithi190.stdout:0/765: chown d9/dc/d28/d4c/f4b 6246 1 2024-05-06T12:14:54.813 INFO:tasks.workunit.client.1.smithi190.stdout:4/562: creat d2/d11/d16/d61/d9e/d6e/d78/fc3 x:0 0 0 2024-05-06T12:14:54.815 INFO:tasks.workunit.client.1.smithi190.stdout:7/741: rmdir d0/da/d36/d53 39 2024-05-06T12:14:54.817 INFO:tasks.workunit.client.1.smithi190.stdout:8/759: creat d3/d8/d25/d3b/d90/dcb/dfb/f104 x:0 0 0 2024-05-06T12:14:54.820 INFO:tasks.workunit.client.1.smithi190.stdout:4/563: dwrite d2/d11/d16/d8e/f57 [0,4194304] 0 2024-05-06T12:14:54.820 INFO:tasks.workunit.client.1.smithi190.stdout:8/760: truncate d3/d35/d55/d61/ff1 860852 0 2024-05-06T12:14:54.820 INFO:tasks.workunit.client.1.smithi190.stdout:2/735: rename d2/d8/d33/dc2/dcd/ff7 to d2/d8/d19/d43/d79/d7e/fff 0 2024-05-06T12:14:54.825 INFO:tasks.workunit.client.1.smithi190.stdout:3/832: unlink d5/d7/d6d/c7b 0 2024-05-06T12:14:54.829 INFO:tasks.workunit.client.1.smithi190.stdout:0/766: write d9/dcf/f71 [824492,76192] 0 2024-05-06T12:14:54.830 INFO:tasks.workunit.client.1.smithi190.stdout:8/761: dwrite d3/d8/d25/d3b/d6b/f99 [0,4194304] 0 2024-05-06T12:14:54.836 INFO:tasks.workunit.client.1.smithi190.stdout:4/564: write d2/fb0 [4704651,25679] 0 2024-05-06T12:14:54.837 INFO:tasks.workunit.client.1.smithi190.stdout:9/779: mknod db/c10f 0 2024-05-06T12:14:54.842 INFO:tasks.workunit.client.1.smithi190.stdout:3/833: fsync d5/d14/f56 0 2024-05-06T12:14:54.843 INFO:tasks.workunit.client.1.smithi190.stdout:0/767: chown d9/dc/d48/d50/d87 89997 1 2024-05-06T12:14:54.844 INFO:tasks.workunit.client.1.smithi190.stdout:8/762: creat d3/d8/d25/d3b/d90/dcb/dfb/f105 x:0 0 0 2024-05-06T12:14:54.845 INFO:tasks.workunit.client.1.smithi190.stdout:7/742: creat d0/d1/d18/d4e/d87/d107/f108 x:0 0 0 2024-05-06T12:14:54.846 INFO:tasks.workunit.client.1.smithi190.stdout:4/565: rmdir d2/d7d 39 2024-05-06T12:14:54.848 INFO:tasks.workunit.client.1.smithi190.stdout:6/566: truncate df/d2c/f36 4659974 0 2024-05-06T12:14:54.849 INFO:tasks.workunit.client.1.smithi190.stdout:9/780: read db/dd/df/dfe/d87/d22/d49/f98 [1986910,98515] 0 2024-05-06T12:14:54.849 INFO:tasks.workunit.client.1.smithi190.stdout:4/566: read d2/d11/f14 [676857,13327] 0 2024-05-06T12:14:54.851 INFO:tasks.workunit.client.1.smithi190.stdout:3/834: mknod d5/d14/d11/dc9/c123 0 2024-05-06T12:14:54.852 INFO:tasks.workunit.client.1.smithi190.stdout:0/768: rmdir d9/d91 39 2024-05-06T12:14:54.856 INFO:tasks.workunit.client.1.smithi190.stdout:2/736: rename d2/d8/d33/d46/ff3 to d2/d8/d33/f100 0 2024-05-06T12:14:54.860 INFO:tasks.workunit.client.1.smithi190.stdout:4/567: creat d2/d11/d16/d61/d9e/d6e/fc4 x:0 0 0 2024-05-06T12:14:54.861 INFO:tasks.workunit.client.1.smithi190.stdout:8/763: mknod d3/d8/db0/c106 0 2024-05-06T12:14:54.863 INFO:tasks.workunit.client.1.smithi190.stdout:0/769: symlink d9/dc/d48/d50/d87/l117 0 2024-05-06T12:14:54.864 INFO:tasks.workunit.client.1.smithi190.stdout:0/770: stat d9/d51/d93/ff9 0 2024-05-06T12:14:54.865 INFO:tasks.workunit.client.1.smithi190.stdout:3/835: rename d5/d7/d15/d39 to d5/d14/d11/d21/d44/d80/ddb/d124 0 2024-05-06T12:14:54.865 INFO:tasks.workunit.client.1.smithi190.stdout:0/771: chown d9/dc/dd0 568 1 2024-05-06T12:14:54.869 INFO:tasks.workunit.client.1.smithi190.stdout:1/576: sync 2024-05-06T12:14:54.870 INFO:tasks.workunit.client.1.smithi190.stdout:4/568: dwrite d2/d11/d16/d61/d9e/d6e/daa/fb5 [0,4194304] 0 2024-05-06T12:14:54.874 INFO:tasks.workunit.client.1.smithi190.stdout:9/781: creat db/dd/d16/d4c/d4e/f110 x:0 0 0 2024-05-06T12:14:54.876 INFO:tasks.workunit.client.1.smithi190.stdout:0/772: unlink d9/dc/d28/d4c/f9b 0 2024-05-06T12:14:54.879 INFO:tasks.workunit.client.1.smithi190.stdout:2/737: creat d2/d8/d33/de6/f101 x:0 0 0 2024-05-06T12:14:54.882 INFO:tasks.workunit.client.1.smithi190.stdout:3/836: dread d5/d14/d11/d21/d44/d92/d9e/da4/d105/db3/dfa/f11c [4194304,4194304] 0 2024-05-06T12:14:54.884 INFO:tasks.workunit.client.1.smithi190.stdout:7/743: rename d0/da/d36/d53/d77/d8c/dd1/d60/dac/ce9 to d0/d1/d32/db0/c109 0 2024-05-06T12:14:54.886 INFO:tasks.workunit.client.1.smithi190.stdout:6/567: getdents df/d2c/d61/d73/dad 0 2024-05-06T12:14:54.891 INFO:tasks.workunit.client.1.smithi190.stdout:2/738: read d2/d4c/f5a [1867417,55513] 0 2024-05-06T12:14:54.895 INFO:tasks.workunit.client.1.smithi190.stdout:2/739: dread d2/d21/f5e [0,4194304] 0 2024-05-06T12:14:54.896 INFO:tasks.workunit.client.1.smithi190.stdout:9/782: rmdir db/d23/d29 39 2024-05-06T12:14:54.898 INFO:tasks.workunit.client.1.smithi190.stdout:3/837: mknod d5/d14/d11/d21/d44/d80/d82/d3d/ded/d33/c125 0 2024-05-06T12:14:54.904 INFO:tasks.workunit.client.1.smithi190.stdout:2/740: dwrite d2/d4c/fd2 [0,4194304] 0 2024-05-06T12:14:54.909 INFO:tasks.workunit.client.1.smithi190.stdout:1/577: link d5/d4f/d65/d24/d43/fb4 d5/d4f/d65/d32/d47/d53/d98/da7/fc7 0 2024-05-06T12:14:54.909 INFO:tasks.workunit.client.1.smithi190.stdout:5/755: sync 2024-05-06T12:14:54.910 INFO:tasks.workunit.client.1.smithi190.stdout:4/569: creat d2/d11/d16/d8e/d80/da0/db2/fc5 x:0 0 0 2024-05-06T12:14:54.913 INFO:tasks.workunit.client.1.smithi190.stdout:5/756: chown d2/da/d59/d5f/faf 316863789 1 2024-05-06T12:14:54.914 INFO:tasks.workunit.client.1.smithi190.stdout:9/783: mkdir db/dd/d5c/d111 0 2024-05-06T12:14:54.916 INFO:tasks.workunit.client.1.smithi190.stdout:3/838: unlink d5/d7/fb9 0 2024-05-06T12:14:54.917 INFO:tasks.workunit.client.1.smithi190.stdout:2/741: readlink d2/l3 0 2024-05-06T12:14:54.919 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:54 smithi092 ceph-mon[26954]: pgmap v29: 65 pgs: 65 active+clean; 3.5 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 130 MiB/s rd, 339 MiB/s wr, 727 op/s 2024-05-06T12:14:54.921 INFO:tasks.workunit.client.1.smithi190.stdout:8/764: dwrite d3/d35/f87 [0,4194304] 0 2024-05-06T12:14:54.924 INFO:tasks.workunit.client.1.smithi190.stdout:5/757: write d2/da/d59/d5f/d2e/da7/d9d/fda [2704369,77992] 0 2024-05-06T12:14:54.924 INFO:tasks.workunit.client.1.smithi190.stdout:6/568: dwrite df/d2c/d1e/f95 [0,4194304] 0 2024-05-06T12:14:54.928 INFO:tasks.workunit.client.1.smithi190.stdout:5/758: write d2/d65/fb0 [955319,58427] 0 2024-05-06T12:14:54.928 INFO:tasks.workunit.client.1.smithi190.stdout:3/839: read d5/d7/f3a [1482015,57738] 0 2024-05-06T12:14:54.930 INFO:tasks.workunit.client.1.smithi190.stdout:6/569: dread - df/d17/d44/f9d zero size 2024-05-06T12:14:54.930 INFO:tasks.workunit.client.1.smithi190.stdout:8/765: chown d3/d8/dcf/cec 19931734 1 2024-05-06T12:14:54.932 INFO:tasks.workunit.client.1.smithi190.stdout:9/784: mknod db/dd/df/dfe/c112 0 2024-05-06T12:14:54.933 INFO:tasks.workunit.client.1.smithi190.stdout:0/773: dwrite d9/dcf/f62 [0,4194304] 0 2024-05-06T12:14:54.935 INFO:tasks.workunit.client.1.smithi190.stdout:6/570: fdatasync df/d2c/d61/d84/d8f/f94 0 2024-05-06T12:14:54.937 INFO:tasks.workunit.client.1.smithi190.stdout:2/742: creat d2/d8/d19/d43/d79/f102 x:0 0 0 2024-05-06T12:14:54.940 INFO:tasks.workunit.client.1.smithi190.stdout:8/766: read d3/d8/f27 [892897,60294] 0 2024-05-06T12:14:54.942 INFO:tasks.workunit.client.1.smithi190.stdout:3/840: truncate d5/d14/d11/d21/d44/d92/d9e/da4/d105/dfb/f11d 113040 0 2024-05-06T12:14:54.943 INFO:tasks.workunit.client.1.smithi190.stdout:3/841: fdatasync d5/d79/f95 0 2024-05-06T12:14:54.946 INFO:tasks.workunit.client.1.smithi190.stdout:5/759: unlink d2/da/d59/d5f/f43 0 2024-05-06T12:14:54.948 INFO:tasks.workunit.client.1.smithi190.stdout:5/760: chown d2/da/l71 3902216 1 2024-05-06T12:14:54.949 INFO:tasks.workunit.client.1.smithi190.stdout:6/571: unlink df/d2c/d61/l80 0 2024-05-06T12:14:54.952 INFO:tasks.workunit.client.1.smithi190.stdout:0/774: dwrite d9/dcf/d21/f2a [0,4194304] 0 2024-05-06T12:14:54.956 INFO:tasks.workunit.client.1.smithi190.stdout:9/785: mknod db/dd/df/dfe/d87/c113 0 2024-05-06T12:14:54.958 INFO:tasks.workunit.client.1.smithi190.stdout:7/744: dread d0/d1/d18/f68 [0,4194304] 0 2024-05-06T12:14:54.961 INFO:tasks.workunit.client.1.smithi190.stdout:3/842: mkdir d5/d7/d6d/d9a/d126 0 2024-05-06T12:14:54.964 INFO:tasks.workunit.client.1.smithi190.stdout:6/572: readlink df/l68 0 2024-05-06T12:14:54.965 INFO:tasks.workunit.client.1.smithi190.stdout:4/570: link d2/l2a d2/d11/d16/d61/d66/lc6 0 2024-05-06T12:14:54.969 INFO:tasks.workunit.client.1.smithi190.stdout:0/775: creat d9/d51/dc9/f118 x:0 0 0 2024-05-06T12:14:54.971 INFO:tasks.workunit.client.1.smithi190.stdout:9/786: dwrite db/dd/d16/d4c/d4e/d5e/ff3 [0,4194304] 0 2024-05-06T12:14:54.972 INFO:tasks.workunit.client.1.smithi190.stdout:7/745: symlink d0/d1/d18/d2b/d5a/l10a 0 2024-05-06T12:14:54.972 INFO:tasks.workunit.client.1.smithi190.stdout:4/571: dread - d2/d10/fc0 zero size 2024-05-06T12:14:54.973 INFO:tasks.workunit.client.1.smithi190.stdout:8/767: link d3/d8/dcf/l86 d3/d8/d2b/d3e/dad/l107 0 2024-05-06T12:14:54.973 INFO:tasks.workunit.client.1.smithi190.stdout:7/746: dread - d0/d1/d32/fcc zero size 2024-05-06T12:14:54.977 INFO:tasks.workunit.client.1.smithi190.stdout:0/776: symlink d9/dcf/d56/dab/l119 0 2024-05-06T12:14:54.978 INFO:tasks.workunit.client.1.smithi190.stdout:9/787: fdatasync db/dd/df/dfe/d87/d99/fc9 0 2024-05-06T12:14:54.980 INFO:tasks.workunit.client.1.smithi190.stdout:6/573: creat df/d2c/d61/d73/dad/fc2 x:0 0 0 2024-05-06T12:14:54.980 INFO:tasks.workunit.client.1.smithi190.stdout:1/578: dwrite d5/d4f/d65/d32/f61 [0,4194304] 0 2024-05-06T12:14:54.985 INFO:tasks.workunit.client.1.smithi190.stdout:1/579: chown d5/d15/cbd 198201056 1 2024-05-06T12:14:54.985 INFO:tasks.workunit.client.1.smithi190.stdout:4/572: creat d2/d11/d16/d31/fc7 x:0 0 0 2024-05-06T12:14:54.988 INFO:tasks.workunit.client.1.smithi190.stdout:6/574: creat df/d17/d44/fc3 x:0 0 0 2024-05-06T12:14:54.991 INFO:tasks.workunit.client.1.smithi190.stdout:3/843: creat d5/d14/d11/d21/d44/d92/d9e/da4/d105/db3/f127 x:0 0 0 2024-05-06T12:14:54.991 INFO:tasks.workunit.client.1.smithi190.stdout:4/573: write d2/d10/fc0 [968013,83211] 0 2024-05-06T12:14:54.993 INFO:tasks.workunit.client.1.smithi190.stdout:4/574: stat d2/d10/ff 0 2024-05-06T12:14:54.998 INFO:tasks.workunit.client.1.smithi190.stdout:0/777: symlink d9/dc/d28/dc5/dd5/de4/dea/dfd/l11a 0 2024-05-06T12:14:54.999 INFO:tasks.workunit.client.1.smithi190.stdout:4/575: write d2/d11/d16/d31/fc7 [423325,56801] 0 2024-05-06T12:14:54.999 INFO:tasks.workunit.client.1.smithi190.stdout:8/768: dwrite d3/d8/d25/d3b/d6b/fd5 [0,4194304] 0 2024-05-06T12:14:55.001 INFO:tasks.workunit.client.1.smithi190.stdout:6/575: mkdir df/d2c/d61/d87/dc4 0 2024-05-06T12:14:55.003 INFO:tasks.workunit.client.1.smithi190.stdout:3/844: rmdir d5/d14/d11/d21/d44/d80/d82/dfc 39 2024-05-06T12:14:55.003 INFO:tasks.workunit.client.1.smithi190.stdout:2/743: dread d2/d8/dc/f16 [4194304,4194304] 0 2024-05-06T12:14:55.006 INFO:tasks.workunit.client.1.smithi190.stdout:7/747: dwrite d0/da/f5b [4194304,4194304] 0 2024-05-06T12:14:55.009 INFO:tasks.workunit.client.1.smithi190.stdout:8/769: symlink d3/d35/d55/l108 0 2024-05-06T12:14:55.011 INFO:tasks.workunit.client.1.smithi190.stdout:7/748: write d0/d1/f92 [9118322,50296] 0 2024-05-06T12:14:55.012 INFO:tasks.workunit.client.1.smithi190.stdout:3/845: write d5/d7/f3a [5375895,65120] 0 2024-05-06T12:14:55.013 INFO:tasks.workunit.client.1.smithi190.stdout:9/788: getdents db/dd/d16/d4c/d4e/d5e/dce 0 2024-05-06T12:14:55.016 INFO:tasks.workunit.client.1.smithi190.stdout:4/576: chown d2/d11/d16/d61/d9e/d6e/d78/c87 19599854 1 2024-05-06T12:14:55.017 INFO:tasks.workunit.client.1.smithi190.stdout:7/749: read - d0/da/d36/ff0 zero size 2024-05-06T12:14:55.021 INFO:tasks.workunit.client.1.smithi190.stdout:2/744: mknod d2/d4c/c103 0 2024-05-06T12:14:55.026 INFO:tasks.workunit.client.1.smithi190.stdout:9/789: dread db/f77 [0,4194304] 0 2024-05-06T12:14:55.026 INFO:tasks.workunit.client.1.smithi190.stdout:3/846: rmdir d5/d14/d11/d21/d44/d80/ddb/d124/dbc 39 2024-05-06T12:14:55.028 INFO:tasks.workunit.client.1.smithi190.stdout:1/580: dwrite d5/d4f/d65/d24/d3a/f3b [0,4194304] 0 2024-05-06T12:14:55.028 INFO:tasks.workunit.client.1.smithi190.stdout:4/577: creat d2/d11/d16/d61/d9e/d6e/daa/fc8 x:0 0 0 2024-05-06T12:14:55.031 INFO:tasks.workunit.client.1.smithi190.stdout:6/576: link df/d17/d44/f54 df/d2c/d61/d84/d8f/db6/fc5 0 2024-05-06T12:14:55.034 INFO:tasks.workunit.client.1.smithi190.stdout:8/770: write d3/f2e [4597425,78827] 0 2024-05-06T12:14:55.036 INFO:tasks.workunit.client.1.smithi190.stdout:0/778: getdents d9/dc/d48 0 2024-05-06T12:14:55.037 INFO:tasks.workunit.client.1.smithi190.stdout:3/847: unlink d5/d14/d11/d21/fc1 0 2024-05-06T12:14:55.039 INFO:tasks.workunit.client.1.smithi190.stdout:7/750: truncate d0/f69 830101 0 2024-05-06T12:14:55.041 INFO:tasks.workunit.client.1.smithi190.stdout:9/790: fsync db/dd/df/dfe/d87/d22/d49/f98 0 2024-05-06T12:14:55.044 INFO:tasks.workunit.client.1.smithi190.stdout:4/578: rename d2/d11/d16/l34 to d2/d11/d16/d41/d81/lc9 0 2024-05-06T12:14:55.049 INFO:tasks.workunit.client.1.smithi190.stdout:7/751: dread d0/f19 [0,4194304] 0 2024-05-06T12:14:55.053 INFO:tasks.workunit.client.1.smithi190.stdout:8/771: unlink d3/d13/fc3 0 2024-05-06T12:14:55.054 INFO:tasks.workunit.client.1.smithi190.stdout:2/745: truncate d2/d8/d19/f2a 4109112 0 2024-05-06T12:14:55.055 INFO:tasks.workunit.client.1.smithi190.stdout:7/752: dread d0/da/d36/d53/d77/d8c/dd1/d60/fa8 [0,4194304] 0 2024-05-06T12:14:55.060 INFO:tasks.workunit.client.1.smithi190.stdout:4/579: write d2/d7d/fbe [747046,119488] 0 2024-05-06T12:14:55.066 INFO:tasks.workunit.client.1.smithi190.stdout:8/772: rename d3/f18 to d3/d35/d3a/db9/f109 0 2024-05-06T12:14:55.068 INFO:tasks.workunit.client.1.smithi190.stdout:7/753: truncate d0/da/d36/d42/fc4 2917298 0 2024-05-06T12:14:55.071 INFO:tasks.workunit.client.1.smithi190.stdout:0/779: mkdir d9/dc/db6/d11b 0 2024-05-06T12:14:55.073 INFO:tasks.workunit.client.1.smithi190.stdout:3/848: truncate d5/d14/f56 2136065 0 2024-05-06T12:14:55.076 INFO:tasks.workunit.client.1.smithi190.stdout:9/791: creat db/dd/df/dfe/d87/d22/d49/d8c/df9/f114 x:0 0 0 2024-05-06T12:14:55.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.073+0000 7fc22b9cf700 1 -- 172.21.15.92:0/554454768 --> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc22410f7e0 con 0x7fc21006b7f0 2024-05-06T12:14:55.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.075+0000 7fc219ffb700 1 -- 172.21.15.92:0/554454768 <== mgr.14636 v2:172.21.15.190:6828/3484477933 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fc22410f7e0 con 0x7fc21006b7f0 2024-05-06T12:14:55.078 INFO:tasks.workunit.client.1.smithi190.stdout:9/792: write db/dd/df/dfe/d87/d22/d49/d88/d9f/df2/f102 [569893,34407] 0 2024-05-06T12:14:55.079 INFO:tasks.workunit.client.1.smithi190.stdout:6/577: rmdir df/d2c/dae/dc0 0 2024-05-06T12:14:55.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.078+0000 7fc22b9cf700 1 -- 172.21.15.92:0/554454768 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fc21006b7f0 msgr2=0x7fc21006dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:55.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.078+0000 7fc22b9cf700 1 --2- 172.21.15.92:0/554454768 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fc21006b7f0 0x7fc21006dca0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc22000b3b0 tx=0x7fc220009e30 comp rx=0 tx=0).stop 2024-05-06T12:14:55.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.078+0000 7fc22b9cf700 1 -- 172.21.15.92:0/554454768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2240fcf60 msgr2=0x7fc22410dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:55.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.078+0000 7fc22b9cf700 1 --2- 172.21.15.92:0/554454768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2240fcf60 0x7fc22410dff0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc214002a50 tx=0x7fc2140046b0 comp rx=0 tx=0).stop 2024-05-06T12:14:55.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.079+0000 7fc22b9cf700 1 -- 172.21.15.92:0/554454768 shutdown_connections 2024-05-06T12:14:55.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.079+0000 7fc22b9cf700 1 --2- 172.21.15.92:0/554454768 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fc21006b7f0 0x7fc21006dca0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:55.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.079+0000 7fc22b9cf700 1 --2- 172.21.15.92:0/554454768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2240fbcb0 0x7fc22410dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:55.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.079+0000 7fc22b9cf700 1 --2- 172.21.15.92:0/554454768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2240fcf60 0x7fc22410dff0 secure :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc214002a50 tx=0x7fc2140046b0 comp rx=0 tx=0).stop 2024-05-06T12:14:55.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.079+0000 7fc22b9cf700 1 -- 172.21.15.92:0/554454768 >> 172.21.15.92:0/554454768 conn(0x7fc2240f7430 msgr2=0x7fc224065b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:14:55.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.079+0000 7fc22b9cf700 1 -- 172.21.15.92:0/554454768 shutdown_connections 2024-05-06T12:14:55.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.079+0000 7fc22b9cf700 1 -- 172.21.15.92:0/554454768 wait complete. 2024-05-06T12:14:55.088 INFO:tasks.workunit.client.1.smithi190.stdout:0/780: rename d9/dcf/d56/dab/fca to d9/dc/d28/dc5/f11c 0 2024-05-06T12:14:55.093 INFO:tasks.workunit.client.1.smithi190.stdout:6/578: symlink df/d2c/d61/d73/dad/lc6 0 2024-05-06T12:14:55.093 INFO:teuthology.orchestra.run.smithi092.stdout:true 2024-05-06T12:14:55.096 INFO:tasks.workunit.client.1.smithi190.stdout:9/793: dread db/d23/fa9 [0,4194304] 0 2024-05-06T12:14:55.096 INFO:tasks.workunit.client.1.smithi190.stdout:2/746: dread d2/d8/d19/d4f/f57 [0,4194304] 0 2024-05-06T12:14:55.097 INFO:tasks.workunit.client.1.smithi190.stdout:4/580: link d2/d11/d16/d8e/d80/da0/db2/fc5 d2/d11/d16/d61/fca 0 2024-05-06T12:14:55.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:54 smithi190 ceph-mon[33798]: pgmap v29: 65 pgs: 65 active+clean; 3.5 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 130 MiB/s rd, 339 MiB/s wr, 727 op/s 2024-05-06T12:14:55.098 INFO:tasks.workunit.client.1.smithi190.stdout:9/794: chown db/dd/df/dfe/d87/d22/d49/d88/d9f/fef 55495299 1 2024-05-06T12:14:55.100 INFO:tasks.workunit.client.1.smithi190.stdout:8/773: truncate d3/d8/d25/d65/dc6/fea 9329105 0 2024-05-06T12:14:55.100 INFO:tasks.workunit.client.1.smithi190.stdout:0/781: fdatasync d9/dc/d28/dc5/f11c 0 2024-05-06T12:14:55.105 INFO:tasks.workunit.client.1.smithi190.stdout:4/581: readlink d2/d11/d16/d31/l54 0 2024-05-06T12:14:55.107 INFO:tasks.workunit.client.1.smithi190.stdout:3/849: rename d5/l7e to d5/l128 0 2024-05-06T12:14:55.108 INFO:tasks.workunit.client.1.smithi190.stdout:2/747: dwrite d2/f1f [4194304,4194304] 0 2024-05-06T12:14:55.111 INFO:tasks.workunit.client.1.smithi190.stdout:8/774: creat d3/d8/d25/d3b/d90/dcb/f10a x:0 0 0 2024-05-06T12:14:55.114 INFO:tasks.workunit.client.1.smithi190.stdout:3/850: creat d5/d14/d11/d21/d44/d80/d82/d3d/ded/d33/d5c/f129 x:0 0 0 2024-05-06T12:14:55.115 INFO:tasks.workunit.client.1.smithi190.stdout:0/782: dwrite d9/dcf/d21/f4a [0,4194304] 0 2024-05-06T12:14:55.115 INFO:tasks.workunit.client.1.smithi190.stdout:2/748: creat d2/d8/d33/d46/dab/dac/f104 x:0 0 0 2024-05-06T12:14:55.117 INFO:tasks.workunit.client.1.smithi190.stdout:6/579: getdents df/d2c/d1e/d57/d53 0 2024-05-06T12:14:55.120 INFO:tasks.workunit.client.1.smithi190.stdout:3/851: symlink d5/d7/d6d/d9a/l12a 0 2024-05-06T12:14:55.121 INFO:tasks.workunit.client.1.smithi190.stdout:6/580: read df/d2c/d28/f2a [375831,65803] 0 2024-05-06T12:14:55.122 INFO:tasks.workunit.client.1.smithi190.stdout:2/749: rmdir d2/d8/d19/d43/d79/d7e 39 2024-05-06T12:14:55.124 INFO:tasks.workunit.client.1.smithi190.stdout:4/582: getdents d2/d11/d16/d8e/d7b 0 2024-05-06T12:14:55.124 INFO:tasks.workunit.client.1.smithi190.stdout:3/852: creat d5/d7/d6d/d9a/f12b x:0 0 0 2024-05-06T12:14:55.125 INFO:tasks.workunit.client.1.smithi190.stdout:8/775: link d3/d8/d25/lb4 d3/d8/d25/d3b/d90/d6e/l10b 0 2024-05-06T12:14:55.127 INFO:tasks.workunit.client.1.smithi190.stdout:2/750: write d2/d8/d19/d43/fb1 [730216,111387] 0 2024-05-06T12:14:55.127 INFO:tasks.workunit.client.1.smithi190.stdout:2/751: read - d2/d8/fdc zero size 2024-05-06T12:14:55.130 INFO:tasks.workunit.client.1.smithi190.stdout:3/853: mknod d5/d14/d11/c12c 0 2024-05-06T12:14:55.131 INFO:tasks.workunit.client.1.smithi190.stdout:2/752: symlink d2/d4c/l105 0 2024-05-06T12:14:55.132 INFO:tasks.workunit.client.1.smithi190.stdout:8/776: creat d3/d35/d3a/db9/f10c x:0 0 0 2024-05-06T12:14:55.134 INFO:tasks.workunit.client.1.smithi190.stdout:2/753: write d2/d8/d19/f83 [1997265,74234] 0 2024-05-06T12:14:55.134 INFO:tasks.workunit.client.1.smithi190.stdout:3/854: symlink d5/d7/d15/l12d 0 2024-05-06T12:14:55.136 INFO:tasks.workunit.client.1.smithi190.stdout:8/777: chown d3/l7 12604 1 2024-05-06T12:14:55.136 INFO:tasks.workunit.client.1.smithi190.stdout:4/583: link d2/l2a d2/d11/lcb 0 2024-05-06T12:14:55.137 INFO:tasks.workunit.client.1.smithi190.stdout:3/855: fsync d5/d14/d11/d21/d44/d92/fb7 0 2024-05-06T12:14:55.142 INFO:tasks.workunit.client.1.smithi190.stdout:8/778: creat d3/d8/d68/d9f/f10d x:0 0 0 2024-05-06T12:14:55.144 INFO:tasks.workunit.client.1.smithi190.stdout:2/754: creat d2/d8/d19/d43/d79/d7e/f106 x:0 0 0 2024-05-06T12:14:55.147 INFO:tasks.workunit.client.1.smithi190.stdout:3/856: rename d5/d7/d6d/f9d to d5/d14/d11/d21/d44/d92/d9e/da4/d105/dfb/f12e 0 2024-05-06T12:14:55.151 INFO:tasks.workunit.client.1.smithi190.stdout:8/779: rename d3/d35/d3a/ca6 to d3/d8/d2b/d3e/dfd/c10e 0 2024-05-06T12:14:55.151 INFO:tasks.workunit.client.1.smithi190.stdout:4/584: dread d2/d11/d16/f38 [0,4194304] 0 2024-05-06T12:14:55.152 INFO:tasks.workunit.client.1.smithi190.stdout:2/755: truncate d2/d8/ffc 1492364 0 2024-05-06T12:14:55.152 INFO:tasks.workunit.client.1.smithi190.stdout:8/780: dread - d3/d35/d3a/db9/f10c zero size 2024-05-06T12:14:55.153 INFO:tasks.workunit.client.1.smithi190.stdout:2/756: readlink d2/d21/la1 0 2024-05-06T12:14:55.153 INFO:tasks.workunit.client.1.smithi190.stdout:6/581: dread fd [4194304,4194304] 0 2024-05-06T12:14:55.157 INFO:tasks.workunit.client.1.smithi190.stdout:6/582: dread df/d2c/d61/d84/d8f/db6/fc5 [0,4194304] 0 2024-05-06T12:14:55.158 INFO:tasks.workunit.client.1.smithi190.stdout:2/757: symlink d2/d8/d19/d43/d79/l107 0 2024-05-06T12:14:55.162 INFO:tasks.workunit.client.1.smithi190.stdout:2/758: rename d2/f35 to d2/d8/d33/d46/dab/dbb/f108 0 2024-05-06T12:14:55.164 INFO:tasks.workunit.client.1.smithi190.stdout:4/585: write d2/f50 [152495,801] 0 2024-05-06T12:14:55.165 INFO:tasks.workunit.client.1.smithi190.stdout:8/781: write d3/d8/d25/d3b/d6b/d7a/f9d [661113,26987] 0 2024-05-06T12:14:55.168 INFO:tasks.workunit.client.1.smithi190.stdout:8/782: chown d3/d8/dcf/l6f 9493835 1 2024-05-06T12:14:55.170 INFO:tasks.workunit.client.1.smithi190.stdout:2/759: dwrite d2/f1f [8388608,4194304] 0 2024-05-06T12:14:55.171 INFO:tasks.workunit.client.1.smithi190.stdout:2/760: chown d2/d8/d33/d46/d5d/l73 2311433 1 2024-05-06T12:14:55.172 INFO:tasks.workunit.client.1.smithi190.stdout:2/761: truncate d2/d8/d33/f100 82923 0 2024-05-06T12:14:55.174 INFO:tasks.workunit.client.1.smithi190.stdout:2/762: write d2/d8/d19/d43/d79/d7e/f63 [1299822,20291] 0 2024-05-06T12:14:55.188 INFO:tasks.workunit.client.1.smithi190.stdout:8/783: dread d3/d8/d25/fac [0,4194304] 0 2024-05-06T12:14:55.189 INFO:tasks.workunit.client.1.smithi190.stdout:8/784: chown d3/d35/d55/d61/c85 34926 1 2024-05-06T12:14:55.189 INFO:tasks.workunit.client.1.smithi190.stdout:2/763: truncate d2/d4c/fd2 2442490 0 2024-05-06T12:14:55.190 INFO:tasks.workunit.client.1.smithi190.stdout:8/785: dread - d3/d13/f7d zero size 2024-05-06T12:14:55.191 INFO:tasks.workunit.client.1.smithi190.stdout:2/764: truncate d2/d4c/f5a 173750 0 2024-05-06T12:14:55.194 INFO:tasks.workunit.client.1.smithi190.stdout:2/765: dread d2/d8/ffc [0,4194304] 0 2024-05-06T12:14:55.196 INFO:tasks.workunit.client.1.smithi190.stdout:4/586: dread d2/d11/d16/d61/d66/d6b/f8f [0,4194304] 0 2024-05-06T12:14:55.203 INFO:tasks.workunit.client.1.smithi190.stdout:2/766: dwrite d2/fd [0,4194304] 0 2024-05-06T12:14:55.205 INFO:tasks.workunit.client.1.smithi190.stdout:8/786: dwrite d3/d8/d25/d3b/d6b/fd5 [4194304,4194304] 0 2024-05-06T12:14:55.206 INFO:tasks.workunit.client.1.smithi190.stdout:2/767: truncate d2/d8/d19/fd9 942640 0 2024-05-06T12:14:55.209 INFO:tasks.workunit.client.1.smithi190.stdout:2/768: creat d2/d8/d33/d46/f109 x:0 0 0 2024-05-06T12:14:55.210 INFO:tasks.workunit.client.1.smithi190.stdout:8/787: mkdir d3/d35/d55/d61/df9/d10f 0 2024-05-06T12:14:55.214 INFO:tasks.workunit.client.1.smithi190.stdout:2/769: rmdir d2/d8/d33/d46/dab/d91 39 2024-05-06T12:14:55.214 INFO:tasks.workunit.client.1.smithi190.stdout:4/587: dwrite d2/d10/fb8 [0,4194304] 0 2024-05-06T12:14:55.216 INFO:tasks.workunit.client.1.smithi190.stdout:4/588: fdatasync d2/d10/f99 0 2024-05-06T12:14:55.217 INFO:tasks.workunit.client.1.smithi190.stdout:4/589: fdatasync d2/d11/d16/d8e/d80/f95 0 2024-05-06T12:14:55.218 INFO:tasks.workunit.client.1.smithi190.stdout:4/590: stat d2/d11/d16/d4e/f6f 0 2024-05-06T12:14:55.220 INFO:tasks.workunit.client.1.smithi190.stdout:4/591: creat d2/d11/d16/d61/d82/fcc x:0 0 0 2024-05-06T12:14:55.222 INFO:tasks.workunit.client.1.smithi190.stdout:4/592: truncate d2/d11/d16/d61/d66/f74 5221586 0 2024-05-06T12:14:55.224 INFO:tasks.workunit.client.1.smithi190.stdout:4/593: creat d2/d11/d16/d61/d66/d6b/fcd x:0 0 0 2024-05-06T12:14:55.227 INFO:tasks.workunit.client.1.smithi190.stdout:4/594: symlink d2/d11/d16/d61/d9e/d6e/d78/lce 0 2024-05-06T12:14:55.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.228+0000 7efbff04e700 1 -- 172.21.15.92:0/1115152215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efbf80fbcc0 msgr2=0x7efbf80fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:55.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.228+0000 7efbff04e700 1 --2- 172.21.15.92:0/1115152215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efbf80fbcc0 0x7efbf80fc0d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7efbec004940 tx=0x7efbec009e30 comp rx=0 tx=0).stop 2024-05-06T12:14:55.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.228+0000 7efbff04e700 1 -- 172.21.15.92:0/1115152215 shutdown_connections 2024-05-06T12:14:55.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.228+0000 7efbff04e700 1 --2- 172.21.15.92:0/1115152215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efbf80fcf70 0x7efbf80fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:55.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.228+0000 7efbff04e700 1 --2- 172.21.15.92:0/1115152215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efbf80fbcc0 0x7efbf80fc0d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:55.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.228+0000 7efbff04e700 1 -- 172.21.15.92:0/1115152215 >> 172.21.15.92:0/1115152215 conn(0x7efbf80f7420 msgr2=0x7efbf80f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:14:55.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.228+0000 7efbff04e700 1 -- 172.21.15.92:0/1115152215 shutdown_connections 2024-05-06T12:14:55.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.228+0000 7efbff04e700 1 -- 172.21.15.92:0/1115152215 wait complete. 2024-05-06T12:14:55.233 INFO:tasks.workunit.client.1.smithi190.stdout:8/788: truncate d3/d8/d25/d3b/d6b/d7a/fe3 1437314 0 2024-05-06T12:14:55.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.229+0000 7efbff04e700 1 Processor -- start 2024-05-06T12:14:55.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.229+0000 7efbff04e700 1 -- start start 2024-05-06T12:14:55.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.229+0000 7efbff04e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efbf80fbcc0 0x7efbf8112b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:55.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.229+0000 7efbff04e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efbf80fcf70 0x7efbf8115070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:55.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.229+0000 7efbff04e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf8066ab0 con 0x7efbf80fcf70 2024-05-06T12:14:55.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.229+0000 7efbff04e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf8066bf0 con 0x7efbf80fbcc0 2024-05-06T12:14:55.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.229+0000 7efbf7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efbf80fcf70 0x7efbf8115070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:55.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.229+0000 7efbf7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efbf80fcf70 0x7efbf8115070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36112/0 (socket says 172.21.15.92:36112) 2024-05-06T12:14:55.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.229+0000 7efbf7fff700 1 -- 172.21.15.92:0/430289845 learned_addr learned my addr 172.21.15.92:0/430289845 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:14:55.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.229+0000 7efbfcdea700 1 --2- 172.21.15.92:0/430289845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efbf80fbcc0 0x7efbf8112b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:55.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.230+0000 7efbf7fff700 1 -- 172.21.15.92:0/430289845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efbf80fbcc0 msgr2=0x7efbf8112b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:55.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.230+0000 7efbf7fff700 1 --2- 172.21.15.92:0/430289845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efbf80fbcc0 0x7efbf8112b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:55.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.230+0000 7efbf7fff700 1 -- 172.21.15.92:0/430289845 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbec005040 con 0x7efbf80fcf70 2024-05-06T12:14:55.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.230+0000 7efbf7fff700 1 --2- 172.21.15.92:0/430289845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efbf80fcf70 0x7efbf8115070 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7efbe400b790 tx=0x7efbe4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:14:55.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.230+0000 7efbf57fa700 1 -- 172.21.15.92:0/430289845 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efbe400f6f0 con 0x7efbf80fcf70 2024-05-06T12:14:55.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.230+0000 7efbff04e700 1 -- 172.21.15.92:0/430289845 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbf81155b0 con 0x7efbf80fcf70 2024-05-06T12:14:55.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.230+0000 7efbff04e700 1 -- 172.21.15.92:0/430289845 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbf8115a90 con 0x7efbf80fcf70 2024-05-06T12:14:55.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.230+0000 7efbf57fa700 1 -- 172.21.15.92:0/430289845 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efbe400fd20 con 0x7efbf80fcf70 2024-05-06T12:14:55.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.230+0000 7efbf57fa700 1 -- 172.21.15.92:0/430289845 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efbe40103a0 con 0x7efbf80fcf70 2024-05-06T12:14:55.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.231+0000 7efbff04e700 1 -- 172.21.15.92:0/430289845 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbdc0052e0 con 0x7efbf80fcf70 2024-05-06T12:14:55.236 INFO:tasks.workunit.client.1.smithi190.stdout:4/595: dwrite d2/f55 [0,4194304] 0 2024-05-06T12:14:55.236 INFO:tasks.workunit.client.1.smithi190.stdout:8/789: mkdir d3/d8/d25/d3b/d90/dcb/dfb/d110 0 2024-05-06T12:14:55.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.232+0000 7efbf57fa700 1 -- 172.21.15.92:0/430289845 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7efbe4010500 con 0x7efbf80fcf70 2024-05-06T12:14:55.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.232+0000 7efbf57fa700 1 --2- 172.21.15.92:0/430289845 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7efbe806b790 0x7efbe806dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:55.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.232+0000 7efbf57fa700 1 -- 172.21.15.92:0/430289845 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7efbe4089f20 con 0x7efbf80fcf70 2024-05-06T12:14:55.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.233+0000 7efbfcdea700 1 --2- 172.21.15.92:0/430289845 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7efbe806b790 0x7efbe806dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:55.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.235+0000 7efbfcdea700 1 --2- 172.21.15.92:0/430289845 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7efbe806b790 0x7efbe806dc40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efbec004890 tx=0x7efbec009e30 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:14:55.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.239+0000 7efbf57fa700 1 -- 172.21.15.92:0/430289845 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7efbe4053c30 con 0x7efbf80fcf70 2024-05-06T12:14:55.249 INFO:tasks.workunit.client.1.smithi190.stdout:8/790: dwrite d3/fb1 [4194304,4194304] 0 2024-05-06T12:14:55.251 INFO:tasks.workunit.client.1.smithi190.stdout:8/791: rmdir d3/d8/d25/d3b/d6b/d7a 39 2024-05-06T12:14:55.252 INFO:tasks.workunit.client.1.smithi190.stdout:8/792: chown d3/d8/d2b/d3e/fe2 556 1 2024-05-06T12:14:55.254 INFO:tasks.workunit.client.1.smithi190.stdout:8/793: mkdir d3/d8/d25/d3b/d111 0 2024-05-06T12:14:55.263 INFO:tasks.workunit.client.1.smithi190.stdout:4/596: dread d2/d11/d16/d31/f56 [0,4194304] 0 2024-05-06T12:14:55.265 INFO:tasks.workunit.client.1.smithi190.stdout:8/794: dwrite d3/d13/fc2 [0,4194304] 0 2024-05-06T12:14:55.270 INFO:tasks.workunit.client.1.smithi190.stdout:4/597: rename d2/d11/d16/d4e/f6f to d2/d11/d16/d4e/fcf 0 2024-05-06T12:14:55.271 INFO:tasks.workunit.client.1.smithi190.stdout:8/795: creat d3/d35/d55/d61/f112 x:0 0 0 2024-05-06T12:14:55.273 INFO:tasks.workunit.client.1.smithi190.stdout:4/598: fdatasync d2/d11/d16/d61/fca 0 2024-05-06T12:14:55.287 INFO:tasks.workunit.client.1.smithi190.stdout:8/796: dwrite d3/d35/f74 [0,4194304] 0 2024-05-06T12:14:55.290 INFO:tasks.workunit.client.1.smithi190.stdout:8/797: creat d3/d8/d68/f113 x:0 0 0 2024-05-06T12:14:55.294 INFO:tasks.workunit.client.1.smithi190.stdout:8/798: link d3/l51 d3/d8/d25/d3b/d111/l114 0 2024-05-06T12:14:55.308 INFO:tasks.workunit.client.1.smithi190.stdout:8/799: dread d3/d8/d25/f39 [0,4194304] 0 2024-05-06T12:14:55.311 INFO:tasks.workunit.client.1.smithi190.stdout:8/800: mkdir d3/d8/d2b/d3e/d115 0 2024-05-06T12:14:55.321 INFO:tasks.workunit.client.1.smithi190.stdout:8/801: dread d3/d8/d2b/d3e/dad/fbe [0,4194304] 0 2024-05-06T12:14:55.323 INFO:tasks.workunit.client.1.smithi190.stdout:8/802: write d3/d8/d25/d3b/d90/dcb/dfb/f105 [209608,108350] 0 2024-05-06T12:14:55.328 INFO:tasks.workunit.client.1.smithi190.stdout:8/803: truncate d3/d8/d25/d65/dc6/fea 9317476 0 2024-05-06T12:14:55.330 INFO:tasks.workunit.client.1.smithi190.stdout:8/804: write d3/f98 [337685,123566] 0 2024-05-06T12:14:55.331 INFO:tasks.workunit.client.1.smithi190.stdout:8/805: fsync d3/fa3 0 2024-05-06T12:14:55.334 INFO:tasks.workunit.client.1.smithi190.stdout:8/806: rename d3/d8/db0 to d3/d8/d25/d3b/d90/dcb/dfb/d116 0 2024-05-06T12:14:55.335 INFO:tasks.workunit.client.1.smithi190.stdout:8/807: truncate d3/d35/d3a/db9/f10c 605955 0 2024-05-06T12:14:55.345 INFO:tasks.workunit.client.1.smithi190.stdout:8/808: dread d3/d8/fd [4194304,4194304] 0 2024-05-06T12:14:55.346 INFO:tasks.workunit.client.1.smithi190.stdout:8/809: dread - d3/d35/d55/ff7 zero size 2024-05-06T12:14:55.347 INFO:tasks.workunit.client.1.smithi190.stdout:8/810: dread - d3/d35/d55/d61/f112 zero size 2024-05-06T12:14:55.348 INFO:tasks.workunit.client.1.smithi190.stdout:8/811: mknod d3/d13/c117 0 2024-05-06T12:14:55.349 INFO:tasks.workunit.client.1.smithi190.stdout:8/812: dread - d3/d35/d55/d61/ff6 zero size 2024-05-06T12:14:55.367 INFO:tasks.workunit.client.1.smithi190.stdout:8/813: dread d3/f2e [0,4194304] 0 2024-05-06T12:14:55.372 INFO:tasks.workunit.client.1.smithi190.stdout:8/814: write d3/d8/d25/d3b/d90/dcb/dfb/d116/f103 [435610,4353] 0 2024-05-06T12:14:55.375 INFO:tasks.workunit.client.1.smithi190.stdout:8/815: rename d3/d8/d2b/d3e/dad/l107 to d3/d8/d2b/l118 0 2024-05-06T12:14:55.377 INFO:tasks.workunit.client.1.smithi190.stdout:8/816: creat d3/f119 x:0 0 0 2024-05-06T12:14:55.378 INFO:tasks.workunit.client.1.smithi190.stdout:8/817: dread - d3/d8/d68/f113 zero size 2024-05-06T12:14:55.386 INFO:tasks.workunit.client.1.smithi190.stdout:5/761: sync 2024-05-06T12:14:55.386 INFO:tasks.workunit.client.1.smithi190.stdout:8/818: dwrite d3/d8/d25/d65/dc6/fcc [0,4194304] 0 2024-05-06T12:14:55.389 INFO:tasks.workunit.client.1.smithi190.stdout:8/819: mkdir d3/d35/d3a/db9/d11a 0 2024-05-06T12:14:55.392 INFO:tasks.workunit.client.1.smithi190.stdout:5/762: dwrite d2/f60 [0,4194304] 0 2024-05-06T12:14:55.395 INFO:tasks.workunit.client.1.smithi190.stdout:5/763: mknod d2/da/d59/d5f/d50/d6d/db3/cf5 0 2024-05-06T12:14:55.397 INFO:tasks.workunit.client.1.smithi190.stdout:8/820: dwrite d3/d8/d25/d65/dc6/fed [0,4194304] 0 2024-05-06T12:14:55.401 INFO:tasks.workunit.client.1.smithi190.stdout:8/821: creat d3/d8/d25/d3b/d90/f11b x:0 0 0 2024-05-06T12:14:55.403 INFO:tasks.workunit.client.1.smithi190.stdout:5/764: rename d2/da/d59/d5f/d2e/da7/d9d/fce to d2/da/d59/d5f/d2e/d8b/ff6 0 2024-05-06T12:14:55.407 INFO:tasks.workunit.client.1.smithi190.stdout:8/822: creat d3/d8/d25/d3b/d90/f11c x:0 0 0 2024-05-06T12:14:55.407 INFO:tasks.workunit.client.1.smithi190.stdout:5/765: dread d2/da/d59/d5f/d6e/daa/fe4 [0,4194304] 0 2024-05-06T12:14:55.432 INFO:tasks.workunit.client.1.smithi190.stdout:5/766: dread d2/da/d59/d5f/d2e/d3b/d5a/f85 [0,4194304] 0 2024-05-06T12:14:55.434 INFO:tasks.workunit.client.1.smithi190.stdout:5/767: creat d2/ff7 x:0 0 0 2024-05-06T12:14:55.436 INFO:tasks.workunit.client.1.smithi190.stdout:8/823: dread d3/d35/d55/fa5 [0,4194304] 0 2024-05-06T12:14:55.438 INFO:tasks.workunit.client.1.smithi190.stdout:8/824: mkdir d3/d8/d25/d65/d11d 0 2024-05-06T12:14:55.438 INFO:tasks.workunit.client.1.smithi190.stdout:8/825: stat d3/d8/d25/d3b/d90/f11b 0 2024-05-06T12:14:55.439 INFO:tasks.workunit.client.1.smithi190.stdout:8/826: chown d3/d8/d25/d3b/d90/d6e/f82 8 1 2024-05-06T12:14:55.440 INFO:tasks.workunit.client.1.smithi190.stdout:8/827: fsync d3/d35/d55/d61/ff6 0 2024-05-06T12:14:55.444 INFO:tasks.workunit.client.1.smithi190.stdout:8/828: dread d3/f17 [0,4194304] 0 2024-05-06T12:14:55.447 INFO:tasks.workunit.client.1.smithi190.stdout:8/829: unlink d3/d35/d55/f73 0 2024-05-06T12:14:55.447 INFO:tasks.workunit.client.1.smithi190.stdout:8/830: fdatasync d3/d8/d68/f113 0 2024-05-06T12:14:55.449 INFO:tasks.workunit.client.1.smithi190.stdout:5/768: dread d2/da/d59/d5f/d2e/da7/d9d/fda [0,4194304] 0 2024-05-06T12:14:55.451 INFO:tasks.workunit.client.1.smithi190.stdout:5/769: creat d2/da/d59/d5f/d2e/da7/d9d/ff8 x:0 0 0 2024-05-06T12:14:55.451 INFO:tasks.workunit.client.1.smithi190.stdout:5/770: readlink d2/l9b 0 2024-05-06T12:14:55.463 INFO:tasks.workunit.client.1.smithi190.stdout:5/771: dread d2/da/d59/d5f/d6e/fde [0,4194304] 0 2024-05-06T12:14:55.466 INFO:tasks.workunit.client.1.smithi190.stdout:5/772: dread d2/fc [4194304,4194304] 0 2024-05-06T12:14:55.467 INFO:tasks.workunit.client.1.smithi190.stdout:5/773: getdents d2/da/d59/d5f/d2e/d3b/d72/dec 0 2024-05-06T12:14:55.474 INFO:tasks.workunit.client.1.smithi190.stdout:5/774: dwrite d2/da/d59/f77 [0,4194304] 0 2024-05-06T12:14:55.475 INFO:tasks.workunit.client.1.smithi190.stdout:5/775: fdatasync d2/dea/ff1 0 2024-05-06T12:14:55.480 INFO:tasks.workunit.client.1.smithi190.stdout:5/776: dwrite d2/da/d59/d5f/d2e/d3b/d5a/f85 [0,4194304] 0 2024-05-06T12:14:55.481 INFO:tasks.workunit.client.1.smithi190.stdout:5/777: read d2/f60 [1638220,89018] 0 2024-05-06T12:14:55.483 INFO:tasks.workunit.client.1.smithi190.stdout:5/778: rmdir d2/d27 39 2024-05-06T12:14:55.485 INFO:tasks.workunit.client.1.smithi190.stdout:5/779: mknod d2/da/d59/d5f/d50/d6d/db3/cf9 0 2024-05-06T12:14:55.487 INFO:tasks.workunit.client.1.smithi190.stdout:5/780: symlink d2/da/d59/d5f/d6e/lfa 0 2024-05-06T12:14:55.488 INFO:tasks.workunit.client.1.smithi190.stdout:5/781: stat d2/da/d59/d5f/d6e/lfa 0 2024-05-06T12:14:55.490 INFO:tasks.workunit.client.1.smithi190.stdout:5/782: mkdir d2/da/d59/d5f/dfb 0 2024-05-06T12:14:55.498 INFO:tasks.workunit.client.1.smithi190.stdout:5/783: write d2/f8 [197570,64617] 0 2024-05-06T12:14:55.499 INFO:tasks.workunit.client.1.smithi190.stdout:5/784: write d2/da/d59/d5f/faf [6282197,125299] 0 2024-05-06T12:14:55.501 INFO:tasks.workunit.client.1.smithi190.stdout:5/785: mkdir d2/db6/dfc 0 2024-05-06T12:14:55.503 INFO:tasks.workunit.client.1.smithi190.stdout:5/786: symlink d2/da/d59/d5f/d2e/da7/d9d/lfd 0 2024-05-06T12:14:55.510 INFO:tasks.workunit.client.1.smithi190.stdout:5/787: dwrite d2/da/d59/d5f/d2e/d8b/ff6 [0,4194304] 0 2024-05-06T12:14:55.514 INFO:tasks.workunit.client.1.smithi190.stdout:5/788: rename d2/da/d59/d5f/lb8 to d2/da/d59/d5f/d2e/d3b/d76/d9a/lfe 0 2024-05-06T12:14:55.515 INFO:tasks.workunit.client.1.smithi190.stdout:5/789: symlink d2/da/d59/d5f/d2e/da3/lff 0 2024-05-06T12:14:55.517 INFO:tasks.workunit.client.1.smithi190.stdout:5/790: creat d2/d65/d83/f100 x:0 0 0 2024-05-06T12:14:55.519 INFO:tasks.workunit.client.1.smithi190.stdout:5/791: rmdir d2/da/d59/d5f/d50 39 2024-05-06T12:14:55.520 INFO:tasks.workunit.client.1.smithi190.stdout:5/792: read d2/da/d59/d5f/d6e/daa/fe4 [44319,20366] 0 2024-05-06T12:14:55.522 INFO:tasks.workunit.client.1.smithi190.stdout:5/793: symlink d2/da/d59/d5f/d2e/d3b/d72/l101 0 2024-05-06T12:14:55.523 INFO:tasks.workunit.client.1.smithi190.stdout:5/794: chown d2/da/d59/d5f/d2e/d3b/d5a 16925 1 2024-05-06T12:14:55.525 INFO:tasks.workunit.client.1.smithi190.stdout:5/795: mknod d2/da/d59/d5f/d50/c102 0 2024-05-06T12:14:55.531 INFO:tasks.workunit.client.1.smithi190.stdout:5/796: dwrite d2/d6c/d7d/fa2 [0,4194304] 0 2024-05-06T12:14:55.533 INFO:tasks.workunit.client.1.smithi190.stdout:5/797: creat d2/da/d59/d5f/d6e/daa/f103 x:0 0 0 2024-05-06T12:14:55.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.546+0000 7efbff04e700 1 -- 172.21.15.92:0/430289845 --> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efbdc000bf0 con 0x7efbe806b790 2024-05-06T12:14:55.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.547+0000 7efbf57fa700 1 -- 172.21.15.92:0/430289845 <== mgr.14636 v2:172.21.15.190:6828/3484477933 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7efbdc000bf0 con 0x7efbe806b790 2024-05-06T12:14:55.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.551+0000 7efbff04e700 1 -- 172.21.15.92:0/430289845 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7efbe806b790 msgr2=0x7efbe806dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:55.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.551+0000 7efbff04e700 1 --2- 172.21.15.92:0/430289845 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7efbe806b790 0x7efbe806dc40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efbec004890 tx=0x7efbec009e30 comp rx=0 tx=0).stop 2024-05-06T12:14:55.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.551+0000 7efbff04e700 1 -- 172.21.15.92:0/430289845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efbf80fcf70 msgr2=0x7efbf8115070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:55.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.551+0000 7efbff04e700 1 --2- 172.21.15.92:0/430289845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efbf80fcf70 0x7efbf8115070 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7efbe400b790 tx=0x7efbe4009250 comp rx=0 tx=0).stop 2024-05-06T12:14:55.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.551+0000 7efbff04e700 1 -- 172.21.15.92:0/430289845 shutdown_connections 2024-05-06T12:14:55.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.551+0000 7efbff04e700 1 --2- 172.21.15.92:0/430289845 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7efbe806b790 0x7efbe806dc40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:55.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.551+0000 7efbff04e700 1 --2- 172.21.15.92:0/430289845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efbf80fbcc0 0x7efbf8112b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:55.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.551+0000 7efbff04e700 1 --2- 172.21.15.92:0/430289845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efbf80fcf70 0x7efbf8115070 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:55.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.551+0000 7efbff04e700 1 -- 172.21.15.92:0/430289845 >> 172.21.15.92:0/430289845 conn(0x7efbf80f7420 msgr2=0x7efbf80f8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:14:55.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.551+0000 7efbff04e700 1 -- 172.21.15.92:0/430289845 shutdown_connections 2024-05-06T12:14:55.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.551+0000 7efbff04e700 1 -- 172.21.15.92:0/430289845 wait complete. 2024-05-06T12:14:55.560 INFO:tasks.workunit.client.1.smithi190.stdout:5/798: dread d2/da/d59/d5f/d2e/d3b/fd4 [0,4194304] 0 2024-05-06T12:14:55.561 INFO:tasks.workunit.client.1.smithi190.stdout:5/799: stat d2/da/d59/d5f/d2e/d3b/f40 0 2024-05-06T12:14:55.563 INFO:tasks.workunit.client.1.smithi190.stdout:5/800: mkdir d2/da/d59/d5f/d2e/da7/d104 0 2024-05-06T12:14:55.565 INFO:tasks.workunit.client.1.smithi190.stdout:5/801: truncate d2/da/d59/d5f/f58 1968480 0 2024-05-06T12:14:55.565 INFO:tasks.workunit.client.1.smithi190.stdout:1/581: sync 2024-05-06T12:14:55.565 INFO:tasks.workunit.client.1.smithi190.stdout:3/857: sync 2024-05-06T12:14:55.565 INFO:tasks.workunit.client.1.smithi190.stdout:5/802: write d2/da/d59/d5f/d2e/da7/d9d/fa4 [914033,35024] 0 2024-05-06T12:14:55.566 INFO:tasks.workunit.client.1.smithi190.stdout:3/858: write d5/d14/d11/dc9/f10f [466561,67647] 0 2024-05-06T12:14:55.568 INFO:tasks.workunit.client.1.smithi190.stdout:1/582: creat d5/d4f/d65/d32/d47/d53/d98/da7/fc8 x:0 0 0 2024-05-06T12:14:55.572 INFO:tasks.workunit.client.1.smithi190.stdout:1/583: mknod d5/d4f/d65/cc9 0 2024-05-06T12:14:55.574 INFO:tasks.workunit.client.1.smithi190.stdout:5/803: dwrite d2/da/d59/d5f/d2e/da7/d9d/fa4 [0,4194304] 0 2024-05-06T12:14:55.575 INFO:tasks.workunit.client.1.smithi190.stdout:1/584: read d5/d4f/d65/d32/d47/f77 [1970451,47671] 0 2024-05-06T12:14:55.575 INFO:tasks.workunit.client.1.smithi190.stdout:5/804: stat d2/da/d59/d5f/d50/d6d/fe2 0 2024-05-06T12:14:55.576 INFO:tasks.workunit.client.1.smithi190.stdout:5/805: stat d2/da/d59/f84 0 2024-05-06T12:14:55.577 INFO:tasks.workunit.client.1.smithi190.stdout:3/859: truncate d5/d14/d11/d21/d44/fbf 2461165 0 2024-05-06T12:14:55.578 INFO:tasks.workunit.client.1.smithi190.stdout:5/806: stat d2/da/d59/d5f/d2e/d3b/d72/dec 0 2024-05-06T12:14:55.579 INFO:tasks.workunit.client.1.smithi190.stdout:5/807: write d2/dea/ff1 [89302,82858] 0 2024-05-06T12:14:55.584 INFO:tasks.workunit.client.1.smithi190.stdout:1/585: truncate d5/d4f/d65/d24/f2c 200984 0 2024-05-06T12:14:55.585 INFO:tasks.workunit.client.1.smithi190.stdout:3/860: dwrite d5/d14/d11/d21/d44/d92/f101 [0,4194304] 0 2024-05-06T12:14:55.586 INFO:tasks.workunit.client.1.smithi190.stdout:3/861: chown d5/d14/d11/dc9/f10f 33193945 1 2024-05-06T12:14:55.588 INFO:tasks.workunit.client.1.smithi190.stdout:1/586: mknod d5/d4f/d68/cca 0 2024-05-06T12:14:55.589 INFO:tasks.workunit.client.1.smithi190.stdout:1/587: fsync d5/d4f/d65/d32/d47/f96 0 2024-05-06T12:14:55.590 INFO:tasks.workunit.client.1.smithi190.stdout:1/588: chown d5/d4f/f83 45 1 2024-05-06T12:14:55.679 INFO:tasks.workunit.client.1.smithi190.stdout:1/589: creat d5/d4f/d65/d32/dac/fcb x:0 0 0 2024-05-06T12:14:55.684 INFO:tasks.workunit.client.1.smithi190.stdout:1/590: dwrite d5/fa [0,4194304] 0 2024-05-06T12:14:55.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.682+0000 7ff0500d3700 1 -- 172.21.15.92:0/1638366725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff048066af0 msgr2=0x7ff0480ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:55.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.682+0000 7ff0500d3700 1 --2- 172.21.15.92:0/1638366725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff048066af0 0x7ff0480ffa30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff044004940 tx=0x7ff044009e30 comp rx=0 tx=0).stop 2024-05-06T12:14:55.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.682+0000 7ff0500d3700 1 -- 172.21.15.92:0/1638366725 shutdown_connections 2024-05-06T12:14:55.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.682+0000 7ff0500d3700 1 --2- 172.21.15.92:0/1638366725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0480fff70 0x7ff048102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:55.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.682+0000 7ff0500d3700 1 --2- 172.21.15.92:0/1638366725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff048066af0 0x7ff0480ffa30 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:55.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.682+0000 7ff0500d3700 1 -- 172.21.15.92:0/1638366725 >> 172.21.15.92:0/1638366725 conn(0x7ff0480f7440 msgr2=0x7ff0480f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:14:55.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.682+0000 7ff0500d3700 1 -- 172.21.15.92:0/1638366725 shutdown_connections 2024-05-06T12:14:55.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.682+0000 7ff0500d3700 1 -- 172.21.15.92:0/1638366725 wait complete. 2024-05-06T12:14:55.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.683+0000 7ff0500d3700 1 Processor -- start 2024-05-06T12:14:55.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.683+0000 7ff0500d3700 1 -- start start 2024-05-06T12:14:55.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.683+0000 7ff0500d3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff048066af0 0x7ff04810da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:55.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.683+0000 7ff0500d3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0480fff70 0x7ff04810df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:55.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.683+0000 7ff0500d3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04810e550 con 0x7ff048066af0 2024-05-06T12:14:55.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.683+0000 7ff0500d3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04810ed50 con 0x7ff0480fff70 2024-05-06T12:14:55.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.683+0000 7ff04de6f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff048066af0 0x7ff04810da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:55.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.683+0000 7ff04de6f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff048066af0 0x7ff04810da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36126/0 (socket says 172.21.15.92:36126) 2024-05-06T12:14:55.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.683+0000 7ff04de6f700 1 -- 172.21.15.92:0/776618301 learned_addr learned my addr 172.21.15.92:0/776618301 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:14:55.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.683+0000 7ff04de6f700 1 -- 172.21.15.92:0/776618301 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0480fff70 msgr2=0x7ff04810df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:55.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.683+0000 7ff04de6f700 1 --2- 172.21.15.92:0/776618301 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0480fff70 0x7ff04810df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:55.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.684+0000 7ff04de6f700 1 -- 172.21.15.92:0/776618301 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff044005040 con 0x7ff048066af0 2024-05-06T12:14:55.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.684+0000 7ff04de6f700 1 --2- 172.21.15.92:0/776618301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff048066af0 0x7ff04810da10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff044004090 tx=0x7ff044009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:14:55.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.684+0000 7ff03e7fc700 1 -- 172.21.15.92:0/776618301 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff044021070 con 0x7ff048066af0 2024-05-06T12:14:55.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.684+0000 7ff03e7fc700 1 -- 172.21.15.92:0/776618301 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff04401c400 con 0x7ff048066af0 2024-05-06T12:14:55.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.684+0000 7ff0500d3700 1 -- 172.21.15.92:0/776618301 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff04810efd0 con 0x7ff048066af0 2024-05-06T12:14:55.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.684+0000 7ff0500d3700 1 -- 172.21.15.92:0/776618301 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff04810f4b0 con 0x7ff048066af0 2024-05-06T12:14:55.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.684+0000 7ff03e7fc700 1 -- 172.21.15.92:0/776618301 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff04400e070 con 0x7ff048066af0 2024-05-06T12:14:55.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.686+0000 7ff0500d3700 1 -- 172.21.15.92:0/776618301 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff04805f5c0 con 0x7ff048066af0 2024-05-06T12:14:55.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.686+0000 7ff03e7fc700 1 -- 172.21.15.92:0/776618301 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7ff04401c9e0 con 0x7ff048066af0 2024-05-06T12:14:55.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.687+0000 7ff03e7fc700 1 --2- 172.21.15.92:0/776618301 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7ff03406b790 0x7ff03406dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:55.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.688+0000 7ff03e7fc700 1 -- 172.21.15.92:0/776618301 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7ff04408b5d0 con 0x7ff048066af0 2024-05-06T12:14:55.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.688+0000 7ff04d66e700 1 --2- 172.21.15.92:0/776618301 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7ff03406b790 0x7ff03406dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:55.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.690+0000 7ff04d66e700 1 --2- 172.21.15.92:0/776618301 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7ff03406b790 0x7ff03406dc40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff038009fd0 tx=0x7ff03800b040 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:14:55.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:55.695+0000 7ff03e7fc700 1 -- 172.21.15.92:0/776618301 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7ff044058d00 con 0x7ff048066af0 2024-05-06T12:14:55.702 INFO:tasks.workunit.client.1.smithi190.stdout:1/591: rename d5/d4f/d65/d62/d9d/fa0 to d5/d4f/d65/d62/fcc 0 2024-05-06T12:14:55.703 INFO:tasks.workunit.client.1.smithi190.stdout:3/862: link d5/d14/d11/d21/d44/d80/d82/d3d/ded/d33/d5c/f121 d5/d14/d11/d21/d44/d80/d82/f12f 0 2024-05-06T12:14:55.704 INFO:tasks.workunit.client.1.smithi190.stdout:1/592: truncate d5/d4f/f50 583221 0 2024-05-06T12:14:55.707 INFO:tasks.workunit.client.1.smithi190.stdout:1/593: unlink d5/d4f/f83 0 2024-05-06T12:14:55.708 INFO:tasks.workunit.client.1.smithi190.stdout:3/863: dread d5/d14/d11/d21/d44/d80/d82/d3d/ded/d33/ffd [0,4194304] 0 2024-05-06T12:14:55.709 INFO:tasks.workunit.client.1.smithi190.stdout:1/594: mknod d5/d4f/d65/d24/d3a/ccd 0 2024-05-06T12:14:55.710 INFO:tasks.workunit.client.1.smithi190.stdout:1/595: write d5/d4f/d65/d32/d47/d53/d6e/f86 [1940716,38340] 0 2024-05-06T12:14:55.711 INFO:tasks.workunit.client.1.smithi190.stdout:7/754: sync 2024-05-06T12:14:55.712 INFO:tasks.workunit.client.1.smithi190.stdout:7/755: fdatasync d0/d1/d18/d6c/fba 0 2024-05-06T12:14:55.713 INFO:tasks.workunit.client.1.smithi190.stdout:7/756: dread - d0/d1/d18/d4e/db3/ffc zero size 2024-05-06T12:14:55.713 INFO:tasks.workunit.client.1.smithi190.stdout:1/596: creat d5/fce x:0 0 0 2024-05-06T12:14:55.714 INFO:tasks.workunit.client.1.smithi190.stdout:1/597: read - d5/d4f/d65/d32/d47/d53/d98/da7/fc7 zero size 2024-05-06T12:14:55.718 INFO:tasks.workunit.client.1.smithi190.stdout:1/598: read d5/d4f/d65/d24/f45 [2695483,23528] 0 2024-05-06T12:14:55.721 INFO:tasks.workunit.client.1.smithi190.stdout:1/599: creat d5/d6b/d8f/fcf x:0 0 0 2024-05-06T12:14:55.726 INFO:tasks.workunit.client.1.smithi190.stdout:7/757: dread d0/d1/d18/d37/f9e [0,4194304] 0 2024-05-06T12:14:55.729 INFO:tasks.workunit.client.1.smithi190.stdout:7/758: creat d0/f10b x:0 0 0 2024-05-06T12:14:55.731 INFO:tasks.workunit.client.1.smithi190.stdout:7/759: rmdir d0/d1/d18/d4e/d87/df1 39 2024-05-06T12:14:55.733 INFO:tasks.workunit.client.1.smithi190.stdout:7/760: mkdir d0/d1/d2/d84/dc6/d10c 0 2024-05-06T12:14:55.734 INFO:tasks.workunit.client.1.smithi190.stdout:7/761: dread - d0/d1/d32/d45/f7d zero size 2024-05-06T12:14:55.735 INFO:tasks.workunit.client.1.smithi190.stdout:7/762: chown d0/da/d36/d53/d5c/fda 9923 1 2024-05-06T12:14:55.739 INFO:tasks.workunit.client.1.smithi190.stdout:7/763: getdents d0/da/d36/d53/d77 0 2024-05-06T12:14:55.741 INFO:tasks.workunit.client.1.smithi190.stdout:7/764: creat d0/da/d36/d53/d77/f10d x:0 0 0 2024-05-06T12:14:55.741 INFO:tasks.workunit.client.1.smithi190.stdout:1/600: write d5/d4f/d65/d62/f8d [822661,73324] 0 2024-05-06T12:14:55.742 INFO:tasks.workunit.client.1.smithi190.stdout:1/601: write d5/fa [1366653,6549] 0 2024-05-06T12:14:55.744 INFO:tasks.workunit.client.1.smithi190.stdout:1/602: chown d5/d4f/d65/d32/d47/f77 3593 1 2024-05-06T12:14:55.746 INFO:tasks.workunit.client.1.smithi190.stdout:7/765: rename d0/d1/d18/d2b/d5a to d0/d1/d32/d45/d70/d33/df7/d10e 0 2024-05-06T12:14:55.749 INFO:tasks.workunit.client.1.smithi190.stdout:7/766: link d0/d1/d2/d84/l99 d0/d1/d32/db0/l10f 0 2024-05-06T12:14:55.750 INFO:tasks.workunit.client.1.smithi190.stdout:7/767: truncate d0/d1/d2/d84/fbf 1198025 0 2024-05-06T12:14:55.752 INFO:tasks.workunit.client.1.smithi190.stdout:7/768: symlink d0/da/d36/d53/d5c/l110 0 2024-05-06T12:14:55.753 INFO:tasks.workunit.client.1.smithi190.stdout:1/603: write d5/d4f/d65/d24/d43/f60 [1061218,39281] 0 2024-05-06T12:14:55.753 INFO:tasks.workunit.client.1.smithi190.stdout:7/769: stat d0/l16 0 2024-05-06T12:14:55.756 INFO:tasks.workunit.client.1.smithi190.stdout:1/604: rename d5/d4f/d65/d24/d3a/f73 to d5/d4f/d65/d62/d9d/dba/fd0 0 2024-05-06T12:14:55.757 INFO:tasks.workunit.client.1.smithi190.stdout:1/605: chown d5/d4f/d65/d24/d3a/f76 0 1 2024-05-06T12:14:55.758 INFO:tasks.workunit.client.1.smithi190.stdout:1/606: stat d5/c81 0 2024-05-06T12:14:55.760 INFO:tasks.workunit.client.1.smithi190.stdout:1/607: getdents d5/d4f/d65/d62/d9d 0 2024-05-06T12:14:55.766 INFO:tasks.workunit.client.1.smithi190.stdout:1/608: dwrite d5/d4f/d65/d32/d47/d53/d98/da7/fc6 [0,4194304] 0 2024-05-06T12:14:55.773 INFO:tasks.workunit.client.1.smithi190.stdout:7/770: dread d0/d1/d18/d2b/f80 [0,4194304] 0 2024-05-06T12:14:55.774 INFO:tasks.workunit.client.1.smithi190.stdout:7/771: fdatasync d0/da/f4a 0 2024-05-06T12:14:55.777 INFO:tasks.workunit.client.1.smithi190.stdout:1/609: dwrite d5/d4f/d65/d24/f45 [0,4194304] 0 2024-05-06T12:14:55.779 INFO:tasks.workunit.client.1.smithi190.stdout:1/610: creat d5/d4f/fd1 x:0 0 0 2024-05-06T12:14:55.787 INFO:tasks.workunit.client.1.smithi190.stdout:1/611: write d5/d4f/d65/d32/d47/d53/f91 [31073,24000] 0 2024-05-06T12:14:55.789 INFO:tasks.workunit.client.1.smithi190.stdout:1/612: mknod d5/d4f/d65/d62/d9d/cd2 0 2024-05-06T12:14:55.790 INFO:tasks.workunit.client.1.smithi190.stdout:1/613: mknod d5/d6b/cd3 0 2024-05-06T12:14:55.803 INFO:tasks.workunit.client.1.smithi190.stdout:1/614: rename d5/d4f/d65/d32/d47/d53/d6e to d5/d4f/d65/d24/dd4 0 2024-05-06T12:14:55.807 INFO:tasks.workunit.client.1.smithi190.stdout:1/615: rmdir d5 39 2024-05-06T12:14:55.815 INFO:tasks.workunit.client.1.smithi190.stdout:1/616: dwrite d5/d4f/d65/d24/f2c [0,4194304] 0 2024-05-06T12:14:55.817 INFO:tasks.workunit.client.1.smithi190.stdout:1/617: dread - d5/d4f/d65/d24/d43/fb4 zero size 2024-05-06T12:14:55.818 INFO:tasks.workunit.client.1.smithi190.stdout:1/618: creat d5/d15/fd5 x:0 0 0 2024-05-06T12:14:55.825 INFO:tasks.workunit.client.1.smithi190.stdout:1/619: creat d5/d15/d1e/fd6 x:0 0 0 2024-05-06T12:14:55.826 INFO:tasks.workunit.client.1.smithi190.stdout:1/620: truncate d5/d4f/fd1 694291 0 2024-05-06T12:14:55.826 INFO:tasks.workunit.client.1.smithi190.stdout:1/621: readlink d5/d15/d1e/l41 0 2024-05-06T12:14:55.827 INFO:tasks.workunit.client.1.smithi190.stdout:1/622: dread d5/d4f/fd1 [0,4194304] 0 2024-05-06T12:14:55.838 INFO:tasks.workunit.client.1.smithi190.stdout:1/623: write d5/d15/f17 [1492081,116284] 0 2024-05-06T12:14:55.842 INFO:tasks.workunit.client.1.smithi190.stdout:1/624: unlink d5/d4f/d65/d32/d47/c5e 0 2024-05-06T12:14:55.845 INFO:tasks.workunit.client.1.smithi190.stdout:0/783: sync 2024-05-06T12:14:55.845 INFO:tasks.workunit.client.1.smithi190.stdout:9/795: sync 2024-05-06T12:14:55.848 INFO:tasks.workunit.client.1.smithi190.stdout:1/625: dwrite d5/d15/f17 [4194304,4194304] 0 2024-05-06T12:14:55.850 INFO:tasks.workunit.client.1.smithi190.stdout:0/784: creat d9/d6f/f11d x:0 0 0 2024-05-06T12:14:55.853 INFO:tasks.workunit.client.1.smithi190.stdout:1/626: creat d5/d4f/d65/d32/d47/d53/fd7 x:0 0 0 2024-05-06T12:14:55.856 INFO:tasks.workunit.client.1.smithi190.stdout:9/796: mknod db/d23/d29/dc4/c115 0 2024-05-06T12:14:55.857 INFO:tasks.workunit.client.1.smithi190.stdout:1/627: creat d5/d4f/d65/d32/d47/d53/d8b/fd8 x:0 0 0 2024-05-06T12:14:55.859 INFO:tasks.workunit.client.1.smithi190.stdout:9/797: symlink db/daa/l116 0 2024-05-06T12:14:55.860 INFO:tasks.workunit.client.1.smithi190.stdout:0/785: fdatasync d9/dc/d48/d50/d83/faf 0 2024-05-06T12:14:55.861 INFO:tasks.workunit.client.1.smithi190.stdout:9/798: write db/dd/df/f107 [10845,117172] 0 2024-05-06T12:14:55.861 INFO:tasks.workunit.client.1.smithi190.stdout:1/628: chown d5/d4f/d68/f8e 804 1 2024-05-06T12:14:55.862 INFO:tasks.workunit.client.1.smithi190.stdout:0/786: truncate d9/feb 443804 0 2024-05-06T12:14:55.863 INFO:tasks.workunit.client.1.smithi190.stdout:1/629: stat d5/d4f/d65/d24/dd4/f84 0 2024-05-06T12:14:55.865 INFO:tasks.workunit.client.1.smithi190.stdout:0/787: symlink d9/d51/dc9/l11e 0 2024-05-06T12:14:55.867 INFO:tasks.workunit.client.1.smithi190.stdout:9/799: getdents db/dd/df/dfe/d87/d99 0 2024-05-06T12:14:55.868 INFO:tasks.workunit.client.1.smithi190.stdout:0/788: symlink d9/dc/d28/dc5/l11f 0 2024-05-06T12:14:55.870 INFO:tasks.workunit.client.1.smithi190.stdout:1/630: rename d5/d4f/d65/d62/l88 to d5/d4f/d65/d32/ld9 0 2024-05-06T12:14:55.874 INFO:tasks.workunit.client.1.smithi190.stdout:0/789: mkdir d9/dc/d28/d4c/d80/da7/d120 0 2024-05-06T12:14:55.878 INFO:tasks.workunit.client.1.smithi190.stdout:0/790: creat d9/dc/d28/dc5/dd5/de4/dea/f121 x:0 0 0 2024-05-06T12:14:55.879 INFO:tasks.workunit.client.1.smithi190.stdout:0/791: fsync d9/dcf/d21/d31/f10d 0 2024-05-06T12:14:55.885 INFO:tasks.workunit.client.1.smithi190.stdout:9/800: write db/dd/df/dfe/d87/d22/d49/f98 [856057,7524] 0 2024-05-06T12:14:55.887 INFO:tasks.workunit.client.1.smithi190.stdout:1/631: write d5/d4f/d65/d24/dd4/f79 [1668281,95658] 0 2024-05-06T12:14:55.889 INFO:tasks.workunit.client.1.smithi190.stdout:9/801: fdatasync db/dd/df/dfe/d87/d22/d49/d8c/fc1 0 2024-05-06T12:14:55.890 INFO:tasks.workunit.client.1.smithi190.stdout:0/792: write d9/d6f/f7a [176422,42744] 0 2024-05-06T12:14:55.892 INFO:tasks.workunit.client.1.smithi190.stdout:1/632: getdents d5/d4f/d65/d32/d47/d53/d98 0 2024-05-06T12:14:55.893 INFO:tasks.workunit.client.1.smithi190.stdout:0/793: stat d9/dcf/l6b 0 2024-05-06T12:14:55.896 INFO:tasks.workunit.client.1.smithi190.stdout:0/794: write d9/dcf/d21/f4a [1663169,55307] 0 2024-05-06T12:14:55.900 INFO:tasks.workunit.client.1.smithi190.stdout:1/633: link d5/d4f/d65/d24/d43/da6/fb9 d5/d4f/d65/d32/d47/d53/d8b/fda 0 2024-05-06T12:14:55.900 INFO:tasks.workunit.client.1.smithi190.stdout:0/795: symlink d9/dc/d28/d4c/d80/da7/l122 0 2024-05-06T12:14:55.905 INFO:tasks.workunit.client.1.smithi190.stdout:1/634: creat d5/d4f/d65/d62/d9d/fdb x:0 0 0 2024-05-06T12:14:55.905 INFO:tasks.workunit.client.1.smithi190.stdout:0/796: read d9/dc/d28/d4c/f4b [3605028,124186] 0 2024-05-06T12:14:55.908 INFO:tasks.workunit.client.1.smithi190.stdout:1/635: write d5/d4f/d65/d62/fcc [570972,121833] 0 2024-05-06T12:14:55.910 INFO:tasks.workunit.client.1.smithi190.stdout:0/797: getdents d9/dc/d28/dc5/dd5 0 2024-05-06T12:14:55.913 INFO:tasks.workunit.client.1.smithi190.stdout:1/636: rename d5/d4f/d65/d24/d3a/f3b to d5/d4f/d65/d32/d47/d53/fdc 0 2024-05-06T12:14:55.919 INFO:tasks.workunit.client.1.smithi190.stdout:1/637: mkdir d5/d4f/d65/d32/dac/ddd 0 2024-05-06T12:14:55.924 INFO:tasks.workunit.client.1.smithi190.stdout:0/798: link d9/dc/d48/d50/d87/lb9 d9/dc/d28/d4c/d80/da7/l123 0 2024-05-06T12:14:55.928 INFO:tasks.workunit.client.1.smithi190.stdout:0/799: rmdir d9/d51 39 2024-05-06T12:14:55.930 INFO:tasks.workunit.client.1.smithi190.stdout:1/638: creat d5/d15/fde x:0 0 0 2024-05-06T12:14:55.937 INFO:tasks.workunit.client.1.smithi190.stdout:0/800: write d9/f2f [3184322,17851] 0 2024-05-06T12:14:55.937 INFO:tasks.workunit.client.1.smithi190.stdout:1/639: write d5/d4f/d65/d24/d43/da6/fb9 [842837,3954] 0 2024-05-06T12:14:55.942 INFO:tasks.workunit.client.1.smithi190.stdout:0/801: rename d9/dc/f74 to d9/dc/db6/d11b/f124 0 2024-05-06T12:14:55.942 INFO:tasks.workunit.client.1.smithi190.stdout:0/802: chown d9/dcf/l32 10789272 1 2024-05-06T12:14:55.943 INFO:tasks.workunit.client.1.smithi190.stdout:1/640: dread d5/d4f/d65/d24/dd4/fb5 [0,4194304] 0 2024-05-06T12:14:55.944 INFO:tasks.workunit.client.1.smithi190.stdout:1/641: fdatasync d5/d4f/d65/d32/d47/d53/fc0 0 2024-05-06T12:14:55.945 INFO:tasks.workunit.client.1.smithi190.stdout:0/803: getdents d9/dcf/d56/dae 0 2024-05-06T12:14:55.945 INFO:tasks.workunit.client.1.smithi190.stdout:1/642: chown d5/d4f/d65/d24/c49 6 1 2024-05-06T12:14:55.947 INFO:tasks.workunit.client.1.smithi190.stdout:1/643: chown d5/d15/f17 463666607 1 2024-05-06T12:14:55.949 INFO:tasks.workunit.client.1.smithi190.stdout:1/644: truncate d5/d15/fbf 570695 0 2024-05-06T12:14:55.950 INFO:tasks.workunit.client.1.smithi190.stdout:0/804: rename d9/d51/d93/ff9 to d9/dcf/d21/f125 0 2024-05-06T12:14:55.952 INFO:tasks.workunit.client.1.smithi190.stdout:1/645: unlink d5/d4f/d65/d24/d43/c9e 0 2024-05-06T12:14:55.953 INFO:tasks.workunit.client.1.smithi190.stdout:0/805: symlink d9/dc/d28/d77/l126 0 2024-05-06T12:14:55.953 INFO:tasks.workunit.client.1.smithi190.stdout:0/806: fsync d9/d51/dc9/fe5 0 2024-05-06T12:14:55.959 INFO:tasks.workunit.client.1.smithi190.stdout:1/646: write d5/d4f/d65/d24/d43/fb4 [166078,82732] 0 2024-05-06T12:14:55.963 INFO:tasks.workunit.client.1.smithi190.stdout:1/647: rename d5/d4f/d65/l2e to d5/d4f/d65/d32/dac/ldf 0 2024-05-06T12:14:55.964 INFO:tasks.workunit.client.1.smithi190.stdout:0/807: dwrite d9/dcf/d21/f97 [0,4194304] 0 2024-05-06T12:14:55.969 INFO:tasks.workunit.client.1.smithi190.stdout:0/808: symlink d9/dc/d28/d4c/d80/da7/l127 0 2024-05-06T12:14:55.969 INFO:tasks.workunit.client.1.smithi190.stdout:1/648: dread d5/d4f/d65/d32/d47/d53/d98/da7/fc6 [0,4194304] 0 2024-05-06T12:14:55.971 INFO:tasks.workunit.client.1.smithi190.stdout:0/809: write d9/dc/d28/dc5/f98 [1484003,93493] 0 2024-05-06T12:14:55.971 INFO:tasks.workunit.client.1.smithi190.stdout:4/599: sync 2024-05-06T12:14:55.971 INFO:tasks.workunit.client.1.smithi190.stdout:2/770: sync 2024-05-06T12:14:55.972 INFO:tasks.workunit.client.1.smithi190.stdout:6/583: sync 2024-05-06T12:14:55.972 INFO:tasks.workunit.client.1.smithi190.stdout:0/810: readlink d9/dcf/l32 0 2024-05-06T12:14:55.975 INFO:tasks.workunit.client.1.smithi190.stdout:0/811: chown d9/dc/d28/d4c/d80/da7/l122 747785 1 2024-05-06T12:14:55.976 INFO:tasks.workunit.client.1.smithi190.stdout:4/600: unlink d2/d7d/cb1 0 2024-05-06T12:14:55.978 INFO:tasks.workunit.client.1.smithi190.stdout:0/812: chown d9/dc/d28/dc5/dd5/ff8 0 1 2024-05-06T12:14:55.979 INFO:tasks.workunit.client.1.smithi190.stdout:6/584: dread df/d12/f6d [0,4194304] 0 2024-05-06T12:14:55.979 INFO:tasks.workunit.client.1.smithi190.stdout:4/601: write d2/d10/ff [2807872,45355] 0 2024-05-06T12:14:55.982 INFO:tasks.workunit.client.1.smithi190.stdout:0/813: write d9/d51/dc9/fe5 [3983120,56806] 0 2024-05-06T12:14:55.984 INFO:tasks.workunit.client.1.smithi190.stdout:2/771: dwrite d2/d8/d19/d43/d79/d7e/fff [0,4194304] 0 2024-05-06T12:14:55.984 INFO:tasks.workunit.client.1.smithi190.stdout:6/585: dread df/d17/d44/fa7 [0,4194304] 0 2024-05-06T12:14:55.989 INFO:tasks.workunit.client.1.smithi190.stdout:4/602: dread d2/d11/d16/d61/f7c [0,4194304] 0 2024-05-06T12:14:55.990 INFO:tasks.workunit.client.1.smithi190.stdout:6/586: creat df/d2c/d61/d73/dad/fc7 x:0 0 0 2024-05-06T12:14:55.993 INFO:tasks.workunit.client.1.smithi190.stdout:0/814: dwrite d9/d51/dc9/f102 [0,4194304] 0 2024-05-06T12:14:55.993 INFO:tasks.workunit.client.1.smithi190.stdout:0/815: readlink d9/dc/l46 0 2024-05-06T12:14:55.995 INFO:tasks.workunit.client.1.smithi190.stdout:0/816: write d9/f37 [2083955,103829] 0 2024-05-06T12:14:55.997 INFO:tasks.workunit.client.1.smithi190.stdout:6/587: read df/d17/d37/f3c [518416,123296] 0 2024-05-06T12:14:56.001 INFO:tasks.workunit.client.1.smithi190.stdout:0/817: creat d9/dc/d28/d4c/d96/f128 x:0 0 0 2024-05-06T12:14:56.002 INFO:tasks.workunit.client.1.smithi190.stdout:6/588: creat df/d12/fc8 x:0 0 0 2024-05-06T12:14:56.002 INFO:tasks.workunit.client.1.smithi190.stdout:2/772: getdents d2/d8/d19/db8 0 2024-05-06T12:14:56.004 INFO:tasks.workunit.client.1.smithi190.stdout:6/589: truncate df/d17/d44/fc3 239905 0 2024-05-06T12:14:56.004 INFO:tasks.workunit.client.1.smithi190.stdout:2/773: mkdir d2/d4c/d10a 0 2024-05-06T12:14:56.005 INFO:tasks.workunit.client.1.smithi190.stdout:0/818: dread d9/dc/f45 [0,4194304] 0 2024-05-06T12:14:56.006 INFO:tasks.workunit.client.1.smithi190.stdout:6/590: mknod df/d12/dbd/cc9 0 2024-05-06T12:14:56.008 INFO:tasks.workunit.client.1.smithi190.stdout:2/774: rename d2/d4c/c68 to d2/d8/d19/d7b/dde/c10b 0 2024-05-06T12:14:56.012 INFO:tasks.workunit.client.1.smithi190.stdout:6/591: rename df/d12/f76 to df/d2c/d61/d73/dad/fca 0 2024-05-06T12:14:56.013 INFO:tasks.workunit.client.1.smithi190.stdout:4/603: chown d2/d11/d16/d61/d9e/d6e/d78/l86 13961 1 2024-05-06T12:14:56.015 INFO:tasks.workunit.client.1.smithi190.stdout:2/775: unlink d2/d8/d19/c5c 0 2024-05-06T12:14:56.015 INFO:tasks.workunit.client.1.smithi190.stdout:4/604: truncate d2/d11/d16/d61/d82/fcc 102393 0 2024-05-06T12:14:56.017 INFO:tasks.workunit.client.1.smithi190.stdout:2/776: fsync d2/d8/d33/dc2/ffd 0 2024-05-06T12:14:56.019 INFO:tasks.workunit.client.1.smithi190.stdout:4/605: creat d2/d11/d16/d61/d9e/d6e/fd0 x:0 0 0 2024-05-06T12:14:56.019 INFO:tasks.workunit.client.1.smithi190.stdout:0/819: getdents d9/dc/d48/d50/d87/dd2/db3 0 2024-05-06T12:14:56.021 INFO:tasks.workunit.client.1.smithi190.stdout:2/777: creat d2/d4c/d10a/f10c x:0 0 0 2024-05-06T12:14:56.021 INFO:tasks.workunit.client.1.smithi190.stdout:4/606: rmdir d2/d11/d16/d41/d81 39 2024-05-06T12:14:56.030 INFO:tasks.workunit.client.1.smithi190.stdout:4/607: getdents d2/d11/d16/d4e 0 2024-05-06T12:14:56.031 INFO:tasks.workunit.client.1.smithi190.stdout:6/592: truncate df/d2c/d1e/f95 2591661 0 2024-05-06T12:14:56.033 INFO:tasks.workunit.client.1.smithi190.stdout:6/593: rmdir df/d17/d37 39 2024-05-06T12:14:56.038 INFO:tasks.workunit.client.1.smithi190.stdout:0/820: dwrite d9/dc/d48/d50/d87/dd2/db3/f100 [0,4194304] 0 2024-05-06T12:14:56.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.039+0000 7ff0500d3700 1 -- 172.21.15.92:0/776618301 --> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff04810f830 con 0x7ff03406b790 2024-05-06T12:14:56.043 INFO:tasks.workunit.client.1.smithi190.stdout:2/778: dread d2/d8/dc/f1a [0,4194304] 0 2024-05-06T12:14:56.043 INFO:tasks.workunit.client.1.smithi190.stdout:0/821: mkdir d9/dc/d48/d50/d87/d129 0 2024-05-06T12:14:56.044 INFO:tasks.workunit.client.1.smithi190.stdout:4/608: dwrite d2/d11/d16/d61/d9e/d6e/fc4 [0,4194304] 0 2024-05-06T12:14:56.049 INFO:tasks.workunit.client.1.smithi190.stdout:6/594: dwrite df/d2c/d61/d73/dad/fc7 [0,4194304] 0 2024-05-06T12:14:56.050 INFO:tasks.workunit.client.1.smithi190.stdout:4/609: rmdir d2/d11/d16/d61/d66/d6b/da4 39 2024-05-06T12:14:56.052 INFO:tasks.workunit.client.1.smithi190.stdout:4/610: write d2/d11/d16/d61/d9e/d6e/daa/fb5 [298552,27207] 0 2024-05-06T12:14:56.053 INFO:tasks.workunit.client.1.smithi190.stdout:2/779: dread d2/d8/dc/f1a [0,4194304] 0 2024-05-06T12:14:56.056 INFO:tasks.workunit.client.1.smithi190.stdout:2/780: write d2/d8/d19/d43/d79/f102 [718939,4280] 0 2024-05-06T12:14:56.059 INFO:tasks.workunit.client.1.smithi190.stdout:4/611: stat d2/d11/d16/d8e/d7b/cbd 0 2024-05-06T12:14:56.062 INFO:tasks.workunit.client.1.smithi190.stdout:2/781: creat d2/d8/d19/dbd/f10d x:0 0 0 2024-05-06T12:14:56.062 INFO:tasks.workunit.client.1.smithi190.stdout:6/595: getdents df/d2c/d1e/d57/d89 0 2024-05-06T12:14:56.062 INFO:tasks.workunit.client.1.smithi190.stdout:4/612: rmdir d2/d11/d16/d61 39 2024-05-06T12:14:56.062 INFO:tasks.workunit.client.1.smithi190.stdout:4/613: chown d2/d7d 234 1 2024-05-06T12:14:56.062 INFO:tasks.workunit.client.1.smithi190.stdout:4/614: fdatasync d2/f64 0 2024-05-06T12:14:56.065 INFO:tasks.workunit.client.1.smithi190.stdout:2/782: creat d2/d8/d33/de6/f10e x:0 0 0 2024-05-06T12:14:56.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.063+0000 7ff03e7fc700 1 -- 172.21.15.92:0/776618301 <== mgr.14636 v2:172.21.15.190:6828/3484477933 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7ff04810f830 con 0x7ff03406b790 2024-05-06T12:14:56.066 INFO:teuthology.orchestra.run.smithi092.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-06T12:14:56.066 INFO:teuthology.orchestra.run.smithi092.stdout:alertmanager.smithi092 smithi092 *:9093,9094 running (7m) 35s ago 8m 25.6M - 0.25.0 c8568f914cd2 dab380a7352e 2024-05-06T12:14:56.066 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi092 smithi092 running (8m) 35s ago 8m 6660k - 17.2.7-925-ge81df420 96d5ae7fd4b4 95dc7e14bd9a 2024-05-06T12:14:56.066 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi190 smithi190 running (7m) 26s ago 7m 6665k - 17.2.7-925-ge81df420 96d5ae7fd4b4 6cf97785406f 2024-05-06T12:14:56.066 INFO:teuthology.orchestra.run.smithi092.stdout:grafana.smithi092 smithi092 *:3000 running (7m) 35s ago 8m 85.5M - 9.4.7 954c08fa6188 841f34687c12 2024-05-06T12:14:56.067 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.ijsqgp smithi092 running (3m) 35s ago 3m 79.3M - 17.2.7-925-ge81df420 96d5ae7fd4b4 b4c2ed6d2c82 2024-05-06T12:14:56.067 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.rosbll smithi092 running (3m) 35s ago 2m 101M - 17.2.7-925-ge81df420 96d5ae7fd4b4 5f5426cd1bf5 2024-05-06T12:14:56.067 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.byjvgf smithi190 running (3m) 26s ago 3m 112M - 17.2.7-925-ge81df420 96d5ae7fd4b4 24581d2e1c5d 2024-05-06T12:14:56.067 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.idzoff smithi190 running (2m) 26s ago 2m 69.7M - 17.2.7-925-ge81df420 96d5ae7fd4b4 20f2641696ef 2024-05-06T12:14:56.067 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi092.gyzeqb smithi092 *:9283 running (9m) 35s ago 9m 419M - 17.2.7-925-ge81df420 96d5ae7fd4b4 820ba9ba8a82 2024-05-06T12:14:56.067 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi190.kczxkh smithi190 *:8443,9283 running (63s) 26s ago 7m 491M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 efed230a6048 2024-05-06T12:14:56.067 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi092 smithi092 running (9m) 35s ago 10m 46.6M 2048M 17.2.7-925-ge81df420 96d5ae7fd4b4 df5b322e9719 2024-05-06T12:14:56.067 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi190 smithi190 running (7m) 26s ago 7m 37.7M 2048M 17.2.7-925-ge81df420 96d5ae7fd4b4 912ded072601 2024-05-06T12:14:56.067 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi092 smithi092 *:9100 starting - - - - 2024-05-06T12:14:56.067 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi190 smithi190 *:9100 running (7m) 26s ago 7m 15.6M - 1.5.0 0da6a335fe13 da5c2a030c69 2024-05-06T12:14:56.067 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 smithi092 running (6m) 35s ago 6m 77.5M 1557M 17.2.7-925-ge81df420 96d5ae7fd4b4 ce84e7af9084 2024-05-06T12:14:56.067 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 smithi092 running (6m) 35s ago 6m 81.5M 1557M 17.2.7-925-ge81df420 96d5ae7fd4b4 8ab870f57eb2 2024-05-06T12:14:56.067 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 smithi092 running (5m) 35s ago 5m 67.4M 1557M 17.2.7-925-ge81df420 96d5ae7fd4b4 1e6ed04a0688 2024-05-06T12:14:56.067 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 smithi190 running (5m) 26s ago 5m 87.2M 2581M 17.2.7-925-ge81df420 96d5ae7fd4b4 52f9521286d5 2024-05-06T12:14:56.068 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 smithi190 running (5m) 26s ago 5m 72.8M 2581M 17.2.7-925-ge81df420 96d5ae7fd4b4 3a7da09e4c48 2024-05-06T12:14:56.068 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 smithi190 running (4m) 26s ago 4m 73.6M 2581M 17.2.7-925-ge81df420 96d5ae7fd4b4 e7452c6b5cec 2024-05-06T12:14:56.068 INFO:teuthology.orchestra.run.smithi092.stdout:prometheus.smithi092 smithi092 *:9095 running (6m) 35s ago 7m 58.8M - 2.43.0 a07b618ecd1d ef92f3572328 2024-05-06T12:14:56.068 INFO:tasks.workunit.client.1.smithi190.stdout:2/783: mkdir d2/d8/d19/d7b/d10f 0 2024-05-06T12:14:56.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.066+0000 7ff0500d3700 1 -- 172.21.15.92:0/776618301 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7ff03406b790 msgr2=0x7ff03406dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:56.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.066+0000 7ff0500d3700 1 --2- 172.21.15.92:0/776618301 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7ff03406b790 0x7ff03406dc40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff038009fd0 tx=0x7ff03800b040 comp rx=0 tx=0).stop 2024-05-06T12:14:56.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.066+0000 7ff0500d3700 1 -- 172.21.15.92:0/776618301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff048066af0 msgr2=0x7ff04810da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:56.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.066+0000 7ff0500d3700 1 --2- 172.21.15.92:0/776618301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff048066af0 0x7ff04810da10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff044004090 tx=0x7ff044009e30 comp rx=0 tx=0).stop 2024-05-06T12:14:56.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.066+0000 7ff0500d3700 1 -- 172.21.15.92:0/776618301 shutdown_connections 2024-05-06T12:14:56.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.066+0000 7ff0500d3700 1 --2- 172.21.15.92:0/776618301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff048066af0 0x7ff04810da10 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:56.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.066+0000 7ff0500d3700 1 --2- 172.21.15.92:0/776618301 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7ff03406b790 0x7ff03406dc40 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:56.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.066+0000 7ff0500d3700 1 --2- 172.21.15.92:0/776618301 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0480fff70 0x7ff04810df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:56.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.066+0000 7ff0500d3700 1 -- 172.21.15.92:0/776618301 >> 172.21.15.92:0/776618301 conn(0x7ff0480f7440 msgr2=0x7ff0480f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:14:56.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.066+0000 7ff0500d3700 1 -- 172.21.15.92:0/776618301 shutdown_connections 2024-05-06T12:14:56.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.066+0000 7ff0500d3700 1 -- 172.21.15.92:0/776618301 wait complete. 2024-05-06T12:14:56.070 INFO:tasks.workunit.client.1.smithi190.stdout:4/615: mkdir d2/d11/d16/d61/dd1 0 2024-05-06T12:14:56.071 INFO:tasks.workunit.client.1.smithi190.stdout:2/784: mkdir d2/d110 0 2024-05-06T12:14:56.073 INFO:tasks.workunit.client.1.smithi190.stdout:6/596: write df/d17/d44/f54 [4124210,90219] 0 2024-05-06T12:14:56.076 INFO:tasks.workunit.client.1.smithi190.stdout:6/597: creat df/d2c/d61/d73/dad/fcb x:0 0 0 2024-05-06T12:14:56.078 INFO:tasks.workunit.client.1.smithi190.stdout:4/616: getdents d2/d11/d16/d61/d66/d6b 0 2024-05-06T12:14:56.078 INFO:tasks.workunit.client.1.smithi190.stdout:6/598: stat df/d17/l63 0 2024-05-06T12:14:56.079 INFO:tasks.workunit.client.1.smithi190.stdout:0/822: dread d9/f2f [0,4194304] 0 2024-05-06T12:14:56.081 INFO:tasks.workunit.client.1.smithi190.stdout:4/617: mknod d2/d7d/cd2 0 2024-05-06T12:14:56.082 INFO:tasks.workunit.client.1.smithi190.stdout:4/618: chown d2/d11/d16/d4e/fcf 24822 1 2024-05-06T12:14:56.083 INFO:tasks.workunit.client.1.smithi190.stdout:4/619: chown d2/d11/d16/d8e/d80/f97 13363 1 2024-05-06T12:14:56.084 INFO:tasks.workunit.client.1.smithi190.stdout:4/620: dread - d2/d11/d16/d8e/d80/da0/db2/fc5 zero size 2024-05-06T12:14:56.085 INFO:tasks.workunit.client.1.smithi190.stdout:0/823: mknod d9/dc/d48/d50/d83/dc6/c12a 0 2024-05-06T12:14:56.085 INFO:tasks.workunit.client.1.smithi190.stdout:4/621: chown d2/d11/d16/d61/d66/l68 25568143 1 2024-05-06T12:14:56.086 INFO:tasks.workunit.client.1.smithi190.stdout:6/599: symlink df/lcc 0 2024-05-06T12:14:56.087 INFO:tasks.workunit.client.1.smithi190.stdout:2/785: link d2/d8/d33/caf d2/d8/d33/d46/dab/dac/c111 0 2024-05-06T12:14:56.087 INFO:tasks.workunit.client.1.smithi190.stdout:0/824: write d9/dcf/d21/d31/f7b [1915928,17538] 0 2024-05-06T12:14:56.094 INFO:tasks.workunit.client.1.smithi190.stdout:4/622: dwrite d2/d11/d16/d4e/fa1 [0,4194304] 0 2024-05-06T12:14:56.097 INFO:tasks.workunit.client.1.smithi190.stdout:2/786: dwrite d2/d8/d33/d46/d5d/f93 [0,4194304] 0 2024-05-06T12:14:56.097 INFO:tasks.workunit.client.1.smithi190.stdout:6/600: link df/c4a df/d2c/d92/ccd 0 2024-05-06T12:14:56.099 INFO:tasks.workunit.client.1.smithi190.stdout:2/787: stat d2/d8/d19/d43/d79/d7e/f40 0 2024-05-06T12:14:56.101 INFO:tasks.workunit.client.1.smithi190.stdout:2/788: dread - d2/d8/d71/f8e zero size 2024-05-06T12:14:56.101 INFO:tasks.workunit.client.1.smithi190.stdout:4/623: readlink d2/d11/d16/d61/d66/lc6 0 2024-05-06T12:14:56.103 INFO:tasks.workunit.client.1.smithi190.stdout:0/825: write d9/f2c [1021627,34445] 0 2024-05-06T12:14:56.105 INFO:tasks.workunit.client.1.smithi190.stdout:2/789: dread - d2/d8/d33/d46/dab/dac/f104 zero size 2024-05-06T12:14:56.106 INFO:tasks.workunit.client.1.smithi190.stdout:6/601: unlink f1 0 2024-05-06T12:14:56.107 INFO:tasks.workunit.client.1.smithi190.stdout:2/790: write d2/d8/d19/d9b/fef [974787,76689] 0 2024-05-06T12:14:56.110 INFO:tasks.workunit.client.1.smithi190.stdout:6/602: mkdir df/d2c/d92/dce 0 2024-05-06T12:14:56.112 INFO:tasks.workunit.client.1.smithi190.stdout:3/864: sync 2024-05-06T12:14:56.112 INFO:tasks.workunit.client.1.smithi190.stdout:5/808: sync 2024-05-06T12:14:56.112 INFO:tasks.workunit.client.1.smithi190.stdout:7/772: sync 2024-05-06T12:14:56.112 INFO:tasks.workunit.client.1.smithi190.stdout:9/802: sync 2024-05-06T12:14:56.113 INFO:tasks.workunit.client.1.smithi190.stdout:8/831: sync 2024-05-06T12:14:56.113 INFO:tasks.workunit.client.1.smithi190.stdout:4/624: dwrite d2/d11/d16/d61/d9e/fc1 [0,4194304] 0 2024-05-06T12:14:56.114 INFO:tasks.workunit.client.1.smithi190.stdout:0/826: link d9/f1b d9/dc/d28/dc5/dd5/de4/dea/dfd/f12b 0 2024-05-06T12:14:56.115 INFO:tasks.workunit.client.1.smithi190.stdout:2/791: read d2/d8/d19/f2a [508703,58285] 0 2024-05-06T12:14:56.117 INFO:tasks.workunit.client.1.smithi190.stdout:7/773: chown d0/d1/d2/f88 236048696 1 2024-05-06T12:14:56.117 INFO:tasks.workunit.client.1.smithi190.stdout:6/603: creat df/d2c/d92/da3/fcf x:0 0 0 2024-05-06T12:14:56.117 INFO:tasks.workunit.client.1.smithi190.stdout:5/809: rmdir d2/da 39 2024-05-06T12:14:56.119 INFO:tasks.workunit.client.1.smithi190.stdout:0/827: mkdir d9/dc/d48/d50/d83/d12c 0 2024-05-06T12:14:56.121 INFO:tasks.workunit.client.1.smithi190.stdout:3/865: readlink d5/d14/d11/d21/d44/d80/ddb/l118 0 2024-05-06T12:14:56.130 INFO:tasks.workunit.client.1.smithi190.stdout:9/803: dwrite db/dd/d16/d4c/d4e/fe1 [0,4194304] 0 2024-05-06T12:14:56.130 INFO:tasks.workunit.client.1.smithi190.stdout:3/866: chown d5/d14/d11/fab 5567 1 2024-05-06T12:14:56.132 INFO:tasks.workunit.client.1.smithi190.stdout:0/828: fsync d9/dc/f6c 0 2024-05-06T12:14:56.132 INFO:tasks.workunit.client.1.smithi190.stdout:5/810: unlink d2/d27/l21 0 2024-05-06T12:14:56.138 INFO:tasks.workunit.client.1.smithi190.stdout:4/625: creat d2/d11/d16/d8e/fd3 x:0 0 0 2024-05-06T12:14:56.139 INFO:tasks.workunit.client.1.smithi190.stdout:6/604: dread df/d2c/d1e/f20 [4194304,4194304] 0 2024-05-06T12:14:56.139 INFO:tasks.workunit.client.1.smithi190.stdout:1/649: sync 2024-05-06T12:14:56.140 INFO:tasks.workunit.client.1.smithi190.stdout:2/792: dwrite d2/d8/d33/d46/f5f [0,4194304] 0 2024-05-06T12:14:56.144 INFO:tasks.workunit.client.1.smithi190.stdout:9/804: rmdir db/dd/df/dfe/d87/d99 39 2024-05-06T12:14:56.147 INFO:tasks.workunit.client.1.smithi190.stdout:5/811: dread d2/da/d59/d5f/d2e/da7/d9d/fda [4194304,4194304] 0 2024-05-06T12:14:56.148 INFO:tasks.workunit.client.1.smithi190.stdout:0/829: mkdir d9/dc/d28/dc5/dd5/de4/d12d 0 2024-05-06T12:14:56.155 INFO:tasks.workunit.client.1.smithi190.stdout:3/867: creat d5/d14/d11/d21/d44/d80/ddb/f130 x:0 0 0 2024-05-06T12:14:56.158 INFO:tasks.workunit.client.1.smithi190.stdout:3/868: write d5/d7/d15/f28 [3657144,83084] 0 2024-05-06T12:14:56.158 INFO:tasks.workunit.client.1.smithi190.stdout:5/812: dread d2/da/d59/d5f/d2e/d3b/f42 [0,4194304] 0 2024-05-06T12:14:56.160 INFO:tasks.workunit.client.1.smithi190.stdout:3/869: stat d5/d14/d11/l5f 0 2024-05-06T12:14:56.160 INFO:tasks.workunit.client.1.smithi190.stdout:7/774: link d0/d1/d32/daa/ldf d0/da/d36/d53/d77/d8c/dd1/l111 0 2024-05-06T12:14:56.161 INFO:tasks.workunit.client.1.smithi190.stdout:5/813: chown d2/da/d59/d5f/d50/d6d/la8 149036617 1 2024-05-06T12:14:56.164 INFO:tasks.workunit.client.1.smithi190.stdout:8/832: dread d3/d8/d25/d3b/d90/d6e/f82 [0,4194304] 0 2024-05-06T12:14:56.165 INFO:tasks.workunit.client.1.smithi190.stdout:3/870: fsync d5/d14/d11/d21/d44/d92/d9e/da4/d105/db3/f127 0 2024-05-06T12:14:56.165 INFO:tasks.workunit.client.1.smithi190.stdout:1/650: creat d5/d4f/d65/d32/d47/d53/fe0 x:0 0 0 2024-05-06T12:14:56.170 INFO:tasks.workunit.client.1.smithi190.stdout:4/626: dwrite d2/d11/d16/d61/d82/fa2 [0,4194304] 0 2024-05-06T12:14:56.171 INFO:tasks.workunit.client.1.smithi190.stdout:7/775: unlink d0/da/d36/d53/f9b 0 2024-05-06T12:14:56.173 INFO:tasks.workunit.client.1.smithi190.stdout:2/793: mkdir d2/d112 0 2024-05-06T12:14:56.175 INFO:tasks.workunit.client.1.smithi190.stdout:9/805: fdatasync db/d23/f2a 0 2024-05-06T12:14:56.176 INFO:tasks.workunit.client.1.smithi190.stdout:9/806: stat db/dd/f34 0 2024-05-06T12:14:56.179 INFO:tasks.workunit.client.1.smithi190.stdout:7/776: dwrite d0/d1/d18/d6c/fba [0,4194304] 0 2024-05-06T12:14:56.182 INFO:tasks.workunit.client.1.smithi190.stdout:1/651: mknod d5/d4f/ce1 0 2024-05-06T12:14:56.183 INFO:tasks.workunit.client.1.smithi190.stdout:3/871: write d5/d14/d11/d21/d44/d92/d9e/da4/d105/dfb/f12e [6380261,62285] 0 2024-05-06T12:14:56.184 INFO:tasks.workunit.client.1.smithi190.stdout:4/627: rmdir d2 39 2024-05-06T12:14:56.186 INFO:tasks.workunit.client.1.smithi190.stdout:3/872: readlink d5/d14/d11/d21/d44/d80/d82/l97 0 2024-05-06T12:14:56.188 INFO:tasks.workunit.client.1.smithi190.stdout:3/873: chown d5/d14/d11/d21/d44/d80/ddb/d124 3 1 2024-05-06T12:14:56.193 INFO:tasks.workunit.client.1.smithi190.stdout:3/874: write d5/d14/d11/d21/d44/d92/d9e/da4/ff7 [703636,30431] 0 2024-05-06T12:14:56.193 INFO:tasks.workunit.client.1.smithi190.stdout:2/794: symlink d2/l113 0 2024-05-06T12:14:56.194 INFO:tasks.workunit.client.1.smithi190.stdout:7/777: dread d0/da/f93 [4194304,4194304] 0 2024-05-06T12:14:56.195 INFO:tasks.workunit.client.1.smithi190.stdout:5/814: unlink d2/da/d59/d5f/d2e/da7/ce5 0 2024-05-06T12:14:56.196 INFO:tasks.workunit.client.1.smithi190.stdout:6/605: creat df/d2c/d1e/d57/fd0 x:0 0 0 2024-05-06T12:14:56.197 INFO:tasks.workunit.client.1.smithi190.stdout:2/795: chown d2/d8/d19/d43/fae 23 1 2024-05-06T12:14:56.199 INFO:tasks.workunit.client.1.smithi190.stdout:4/628: unlink d2/d11/d16/d61/la7 0 2024-05-06T12:14:56.200 INFO:tasks.workunit.client.1.smithi190.stdout:5/815: symlink d2/da/l105 0 2024-05-06T12:14:56.200 INFO:tasks.workunit.client.1.smithi190.stdout:8/833: dwrite d3/d8/fbb [0,4194304] 0 2024-05-06T12:14:56.202 INFO:tasks.workunit.client.1.smithi190.stdout:2/796: read d2/d21/f70 [881026,130322] 0 2024-05-06T12:14:56.203 INFO:tasks.workunit.client.1.smithi190.stdout:3/875: fdatasync d5/d14/d11/d21/d44/d80/d82/d3d/ded/f3b 0 2024-05-06T12:14:56.203 INFO:tasks.workunit.client.1.smithi190.stdout:9/807: write db/f77 [318983,80493] 0 2024-05-06T12:14:56.204 INFO:tasks.workunit.client.1.smithi190.stdout:7/778: symlink d0/d1/d32/d45/d70/d33/df7/d10e/l112 0 2024-05-06T12:14:56.206 INFO:tasks.workunit.client.1.smithi190.stdout:6/606: mkdir df/d2c/d28/dd1 0 2024-05-06T12:14:56.207 INFO:tasks.workunit.client.1.smithi190.stdout:7/779: stat d0/d1/d2/d84/dc6/c103 0 2024-05-06T12:14:56.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.206+0000 7f9b641ae700 1 -- 172.21.15.92:0/3920491528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b5c0fce80 msgr2=0x7f9b5c0fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:56.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.206+0000 7f9b641ae700 1 --2- 172.21.15.92:0/3920491528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b5c0fce80 0x7f9b5c0fd290 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9b58004940 tx=0x7f9b58009e30 comp rx=0 tx=0).stop 2024-05-06T12:14:56.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.206+0000 7f9b641ae700 1 -- 172.21.15.92:0/3920491528 shutdown_connections 2024-05-06T12:14:56.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.206+0000 7f9b641ae700 1 --2- 172.21.15.92:0/3920491528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b5c0fdff0 0x7f9b5c0fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:56.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.206+0000 7f9b641ae700 1 --2- 172.21.15.92:0/3920491528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b5c0fce80 0x7f9b5c0fd290 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:56.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.206+0000 7f9b641ae700 1 -- 172.21.15.92:0/3920491528 >> 172.21.15.92:0/3920491528 conn(0x7f9b5c0f8540 msgr2=0x7f9b5c0fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:14:56.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.206+0000 7f9b641ae700 1 -- 172.21.15.92:0/3920491528 shutdown_connections 2024-05-06T12:14:56.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.207+0000 7f9b641ae700 1 -- 172.21.15.92:0/3920491528 wait complete. 2024-05-06T12:14:56.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.208+0000 7f9b641ae700 1 Processor -- start 2024-05-06T12:14:56.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.208+0000 7f9b641ae700 1 -- start start 2024-05-06T12:14:56.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.209+0000 7f9b641ae700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b5c0fce80 0x7f9b5c10b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:56.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.209+0000 7f9b641ae700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b5c0fdff0 0x7f9b5c10bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:56.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.209+0000 7f9b641ae700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c10c320 con 0x7f9b5c0fce80 2024-05-06T12:14:56.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.209+0000 7f9b641ae700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c10cb70 con 0x7f9b5c0fdff0 2024-05-06T12:14:56.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.209+0000 7f9b61f4a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b5c0fce80 0x7f9b5c10b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:56.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.209+0000 7f9b61f4a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b5c0fce80 0x7f9b5c10b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36152/0 (socket says 172.21.15.92:36152) 2024-05-06T12:14:56.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.209+0000 7f9b61f4a700 1 -- 172.21.15.92:0/2480165038 learned_addr learned my addr 172.21.15.92:0/2480165038 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:14:56.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.209+0000 7f9b61749700 1 --2- 172.21.15.92:0/2480165038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b5c0fdff0 0x7f9b5c10bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:56.213 INFO:tasks.workunit.client.1.smithi190.stdout:4/629: mkdir d2/d11/d16/d8e/d7b/dd4 0 2024-05-06T12:14:56.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.210+0000 7f9b61f4a700 1 -- 172.21.15.92:0/2480165038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b5c0fdff0 msgr2=0x7f9b5c10bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:56.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.210+0000 7f9b61f4a700 1 --2- 172.21.15.92:0/2480165038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b5c0fdff0 0x7f9b5c10bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:56.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.210+0000 7f9b61f4a700 1 -- 172.21.15.92:0/2480165038 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b58005040 con 0x7f9b5c0fce80 2024-05-06T12:14:56.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.210+0000 7f9b61749700 1 --2- 172.21.15.92:0/2480165038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b5c0fdff0 0x7f9b5c10bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-06T12:14:56.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.210+0000 7f9b61f4a700 1 --2- 172.21.15.92:0/2480165038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b5c0fce80 0x7f9b5c10b790 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9b58013870 tx=0x7f9b58009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:14:56.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.210+0000 7f9b527fc700 1 -- 172.21.15.92:0/2480165038 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b58021070 con 0x7f9b5c0fce80 2024-05-06T12:14:56.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.210+0000 7f9b641ae700 1 -- 172.21.15.92:0/2480165038 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b5c10cdf0 con 0x7f9b5c0fce80 2024-05-06T12:14:56.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.210+0000 7f9b527fc700 1 -- 172.21.15.92:0/2480165038 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b5801c400 con 0x7f9b5c0fce80 2024-05-06T12:14:56.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.210+0000 7f9b527fc700 1 -- 172.21.15.92:0/2480165038 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b5800e070 con 0x7f9b5c0fce80 2024-05-06T12:14:56.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.210+0000 7f9b641ae700 1 -- 172.21.15.92:0/2480165038 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b5c10d330 con 0x7f9b5c0fce80 2024-05-06T12:14:56.215 INFO:tasks.workunit.client.1.smithi190.stdout:1/652: dread d5/d4f/d65/d2b/fbc [0,4194304] 0 2024-05-06T12:14:56.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.212+0000 7f9b641ae700 1 -- 172.21.15.92:0/2480165038 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b5c101980 con 0x7f9b5c0fce80 2024-05-06T12:14:56.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.212+0000 7f9b527fc700 1 -- 172.21.15.92:0/2480165038 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7f9b58013af0 con 0x7f9b5c0fce80 2024-05-06T12:14:56.216 INFO:tasks.workunit.client.1.smithi190.stdout:6/607: chown df/d2c/d61/d73/dad/fb0 11 1 2024-05-06T12:14:56.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.213+0000 7f9b527fc700 1 --2- 172.21.15.92:0/2480165038 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f9b4806b790 0x7f9b4806dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:56.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.213+0000 7f9b527fc700 1 -- 172.21.15.92:0/2480165038 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f9b5808b6c0 con 0x7f9b5c0fce80 2024-05-06T12:14:56.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.214+0000 7f9b61749700 1 --2- 172.21.15.92:0/2480165038 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f9b4806b790 0x7f9b4806dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:56.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.214+0000 7f9b61749700 1 --2- 172.21.15.92:0/2480165038 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f9b4806b790 0x7f9b4806dc40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c004600 tx=0x7f9b4c00b040 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:14:56.218 INFO:tasks.workunit.client.1.smithi190.stdout:9/808: mkdir db/dd/df/dfe/d87/d22/d49/d88/d76/d117 0 2024-05-06T12:14:56.221 INFO:tasks.workunit.client.1.smithi190.stdout:1/653: read d5/d4f/d65/d24/f25 [3025138,47862] 0 2024-05-06T12:14:56.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.219+0000 7f9b527fc700 1 -- 172.21.15.92:0/2480165038 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f9b58055320 con 0x7f9b5c0fce80 2024-05-06T12:14:56.222 INFO:tasks.workunit.client.1.smithi190.stdout:3/876: mkdir d5/d14/d11/d21/d44/d80/d82/d3d/ded/d131 0 2024-05-06T12:14:56.224 INFO:tasks.workunit.client.1.smithi190.stdout:4/630: creat d2/d11/d16/d8e/d80/da0/db2/fd5 x:0 0 0 2024-05-06T12:14:56.225 INFO:tasks.workunit.client.1.smithi190.stdout:3/877: write d5/d14/d11/d21/d44/d92/d9e/fee [863306,119419] 0 2024-05-06T12:14:56.226 INFO:tasks.workunit.client.1.smithi190.stdout:9/809: rmdir db/dd/d16/d4c/d4e 39 2024-05-06T12:14:56.229 INFO:tasks.workunit.client.1.smithi190.stdout:8/834: dwrite d3/d8/d2b/d3e/f8d [0,4194304] 0 2024-05-06T12:14:56.229 INFO:tasks.workunit.client.1.smithi190.stdout:1/654: write d5/d4f/d65/d24/dd4/fb5 [1301637,116328] 0 2024-05-06T12:14:56.230 INFO:tasks.workunit.client.1.smithi190.stdout:3/878: stat d5/d14/d11/d21/d44/d92/d9e/da4/d105/lb2 0 2024-05-06T12:14:56.230 INFO:tasks.workunit.client.1.smithi190.stdout:7/780: creat d0/d14/d6a/f113 x:0 0 0 2024-05-06T12:14:56.233 INFO:tasks.workunit.client.1.smithi190.stdout:5/816: truncate d2/da/d59/f77 851011 0 2024-05-06T12:14:56.236 INFO:tasks.workunit.client.1.smithi190.stdout:5/817: readlink d2/d6c/d7d/lad 0 2024-05-06T12:14:56.237 INFO:tasks.workunit.client.1.smithi190.stdout:2/797: rename d2/d8/d19/d43/c47 to d2/c114 0 2024-05-06T12:14:56.237 INFO:tasks.workunit.client.1.smithi190.stdout:4/631: chown d2/l8 6 1 2024-05-06T12:14:56.242 INFO:tasks.workunit.client.1.smithi190.stdout:3/879: chown d5/d14/d11/d21/d44/d80/ddb/d124/dbc/ce8 930034 1 2024-05-06T12:14:56.246 INFO:tasks.workunit.client.1.smithi190.stdout:6/608: rename df/d12/f65 to df/d2c/d61/d84/d8f/fd2 0 2024-05-06T12:14:56.248 INFO:tasks.workunit.client.1.smithi190.stdout:2/798: dread d2/fb6 [0,4194304] 0 2024-05-06T12:14:56.250 INFO:tasks.workunit.client.1.smithi190.stdout:2/799: truncate d2/d8/d33/de6/f10e 709898 0 2024-05-06T12:14:56.252 INFO:tasks.workunit.client.1.smithi190.stdout:4/632: creat d2/d11/d16/d4e/fd6 x:0 0 0 2024-05-06T12:14:56.253 INFO:tasks.workunit.client.1.smithi190.stdout:8/835: dread d3/d8/d68/faf [0,4194304] 0 2024-05-06T12:14:56.254 INFO:tasks.workunit.client.1.smithi190.stdout:5/818: stat d2/da/d59/d5f/d2e/d8b/ce3 0 2024-05-06T12:14:56.257 INFO:tasks.workunit.client.1.smithi190.stdout:6/609: mknod df/d2c/d61/d73/dad/cd3 0 2024-05-06T12:14:56.259 INFO:tasks.workunit.client.1.smithi190.stdout:2/800: creat d2/d8/d19/d9b/f115 x:0 0 0 2024-05-06T12:14:56.260 INFO:tasks.workunit.client.1.smithi190.stdout:4/633: truncate d2/d11/d16/d31/f5c 791827 0 2024-05-06T12:14:56.260 INFO:tasks.workunit.client.1.smithi190.stdout:7/781: dwrite d0/d1/d18/f68 [0,4194304] 0 2024-05-06T12:14:56.262 INFO:tasks.workunit.client.1.smithi190.stdout:8/836: mkdir d3/d35/d55/d61/d11e 0 2024-05-06T12:14:56.263 INFO:tasks.workunit.client.1.smithi190.stdout:1/655: dread d5/f6d [0,4194304] 0 2024-05-06T12:14:56.263 INFO:tasks.workunit.client.1.smithi190.stdout:5/819: creat d2/da/d59/d5f/d2e/d8b/f106 x:0 0 0 2024-05-06T12:14:56.264 INFO:tasks.workunit.client.1.smithi190.stdout:7/782: chown d0/da/d36/d53/f98 1602288 1 2024-05-06T12:14:56.264 INFO:tasks.workunit.client.1.smithi190.stdout:6/610: read df/d2c/d1e/d57/d89/fa1 [3431134,109847] 0 2024-05-06T12:14:56.266 INFO:tasks.workunit.client.1.smithi190.stdout:1/656: chown d5/d4f/fd1 247324 1 2024-05-06T12:14:56.267 INFO:tasks.workunit.client.1.smithi190.stdout:1/657: stat d5 0 2024-05-06T12:14:56.267 INFO:tasks.workunit.client.1.smithi190.stdout:3/880: dread d5/d14/d11/d21/d44/d80/d82/d3d/ded/fe [0,4194304] 0 2024-05-06T12:14:56.269 INFO:tasks.workunit.client.1.smithi190.stdout:3/881: chown d5/d14/d11/d21/d44/d80/ddb/f130 12217 1 2024-05-06T12:14:56.271 INFO:tasks.workunit.client.1.smithi190.stdout:9/810: rename db/dd/df/dfe/d87/d22/d49/d88/d9f/df2/fbd to db/dd/d16/d4c/f118 0 2024-05-06T12:14:56.271 INFO:tasks.workunit.client.1.smithi190.stdout:4/634: read d2/d10/fc0 [711475,67532] 0 2024-05-06T12:14:56.274 INFO:tasks.workunit.client.1.smithi190.stdout:8/837: creat d3/d35/d55/d61/df9/f11f x:0 0 0 2024-05-06T12:14:56.276 INFO:tasks.workunit.client.1.smithi190.stdout:7/783: readlink d0/d1/d32/db0/l10f 0 2024-05-06T12:14:56.278 INFO:tasks.workunit.client.1.smithi190.stdout:5/820: mknod d2/da/d59/d5f/d50/d6d/db3/c107 0 2024-05-06T12:14:56.279 INFO:tasks.workunit.client.1.smithi190.stdout:1/658: mkdir d5/d4f/d65/d24/d43/de2 0 2024-05-06T12:14:56.281 INFO:tasks.workunit.client.1.smithi190.stdout:3/882: mkdir d5/d14/d11/dc9/d132 0 2024-05-06T12:14:56.282 INFO:tasks.workunit.client.1.smithi190.stdout:2/801: dread d2/d21/f5e [0,4194304] 0 2024-05-06T12:14:56.283 INFO:tasks.workunit.client.1.smithi190.stdout:4/635: mkdir d2/d11/d16/d61/d82/dd7 0 2024-05-06T12:14:56.284 INFO:tasks.workunit.client.1.smithi190.stdout:2/802: chown d2/d8/d33/dc2/ff8 450 1 2024-05-06T12:14:56.286 INFO:tasks.workunit.client.1.smithi190.stdout:8/838: rename d3/d8/d2b/d3e/f84 to d3/d8/d2b/d3e/dad/f120 0 2024-05-06T12:14:56.287 INFO:tasks.workunit.client.1.smithi190.stdout:7/784: symlink d0/d1/d32/d45/d70/d3b/l114 0 2024-05-06T12:14:56.289 INFO:tasks.workunit.client.1.smithi190.stdout:6/611: getdents df/d17/d37 0 2024-05-06T12:14:56.294 INFO:tasks.workunit.client.1.smithi190.stdout:2/803: mkdir d2/d8/dc/d116 0 2024-05-06T12:14:56.295 INFO:tasks.workunit.client.1.smithi190.stdout:8/839: symlink d3/d8/d25/d3b/d90/dcb/dfb/l121 0 2024-05-06T12:14:56.298 INFO:tasks.workunit.client.1.smithi190.stdout:7/785: creat d0/da/d36/d53/d77/d8c/dd1/d60/dac/f115 x:0 0 0 2024-05-06T12:14:56.299 INFO:tasks.workunit.client.1.smithi190.stdout:5/821: dread d2/da/d59/d5f/f5b [0,4194304] 0 2024-05-06T12:14:56.300 INFO:tasks.workunit.client.1.smithi190.stdout:1/659: link d5/f16 d5/d4f/d65/d24/d43/fe3 0 2024-05-06T12:14:56.301 INFO:tasks.workunit.client.1.smithi190.stdout:1/660: stat d5/d15/f17 0 2024-05-06T12:14:56.301 INFO:tasks.workunit.client.1.smithi190.stdout:5/822: truncate d2/da/d59/d5f/d2e/d8b/ff6 4514111 0 2024-05-06T12:14:56.304 INFO:tasks.workunit.client.1.smithi190.stdout:6/612: creat df/d2c/d1e/d57/d53/fd4 x:0 0 0 2024-05-06T12:14:56.305 INFO:tasks.workunit.client.1.smithi190.stdout:6/613: write df/d17/d37/f9c [1807436,48423] 0 2024-05-06T12:14:56.305 INFO:tasks.workunit.client.1.smithi190.stdout:4/636: symlink d2/d11/d16/d41/d81/ld8 0 2024-05-06T12:14:56.306 INFO:tasks.workunit.client.1.smithi190.stdout:2/804: symlink d2/d8/d19/d7b/dde/l117 0 2024-05-06T12:14:56.308 INFO:tasks.workunit.client.1.smithi190.stdout:6/614: write df/d17/d37/f8c [968550,122128] 0 2024-05-06T12:14:56.308 INFO:tasks.workunit.client.1.smithi190.stdout:1/661: creat d5/d4f/fe4 x:0 0 0 2024-05-06T12:14:56.310 INFO:tasks.workunit.client.1.smithi190.stdout:5/823: mknod d2/da/d59/d5f/d50/d6d/db3/c108 0 2024-05-06T12:14:56.311 INFO:tasks.workunit.client.1.smithi190.stdout:9/811: write db/dd/d16/d4c/fe2 [3137955,51329] 0 2024-05-06T12:14:56.312 INFO:tasks.workunit.client.1.smithi190.stdout:9/812: readlink db/dd/df/dfe/ddc/l93 0 2024-05-06T12:14:56.313 INFO:tasks.workunit.client.1.smithi190.stdout:4/637: symlink d2/d11/d16/d41/d81/ld9 0 2024-05-06T12:14:56.315 INFO:tasks.workunit.client.1.smithi190.stdout:7/786: mkdir d0/d1/d32/d116 0 2024-05-06T12:14:56.316 INFO:tasks.workunit.client.1.smithi190.stdout:7/787: write d0/d1/d18/d37/fee [508392,113983] 0 2024-05-06T12:14:56.317 INFO:tasks.workunit.client.1.smithi190.stdout:7/788: write d0/d1/d18/d37/f9e [3071226,5557] 0 2024-05-06T12:14:56.319 INFO:tasks.workunit.client.1.smithi190.stdout:1/662: mkdir d5/d4f/d65/d32/d47/de5 0 2024-05-06T12:14:56.324 INFO:tasks.workunit.client.1.smithi190.stdout:7/789: symlink d0/d14/d6a/d8f/l117 0 2024-05-06T12:14:56.325 INFO:tasks.workunit.client.1.smithi190.stdout:5/824: rmdir d2/da/d59/d5f/d2e/d3b/d76 39 2024-05-06T12:14:56.329 INFO:tasks.workunit.client.1.smithi190.stdout:2/805: dread d2/d8/d33/f8f [0,4194304] 0 2024-05-06T12:14:56.330 INFO:tasks.workunit.client.1.smithi190.stdout:3/883: dwrite d5/d14/d11/d21/d44/d80/d82/fbb [0,4194304] 0 2024-05-06T12:14:56.331 INFO:tasks.workunit.client.1.smithi190.stdout:5/825: write d2/d6c/d7d/fa2 [3240337,93268] 0 2024-05-06T12:14:56.332 INFO:tasks.workunit.client.1.smithi190.stdout:7/790: chown d0/d1/d32/d45/d70/d33/d57/fc0 1941444 1 2024-05-06T12:14:56.334 INFO:tasks.workunit.client.1.smithi190.stdout:1/663: dwrite d5/d4f/d65/d32/d47/d53/d98/da7/fc8 [0,4194304] 0 2024-05-06T12:14:56.337 INFO:tasks.workunit.client.1.smithi190.stdout:4/638: getdents d2/d11/d16/d31 0 2024-05-06T12:14:56.339 INFO:tasks.workunit.client.1.smithi190.stdout:4/639: write d2/d11/d16/d31/f43 [5063087,9449] 0 2024-05-06T12:14:56.339 INFO:tasks.workunit.client.1.smithi190.stdout:9/813: rename db/d23/l50 to db/dd/df/dfe/d87/d22/d49/d88/l119 0 2024-05-06T12:14:56.342 INFO:tasks.workunit.client.1.smithi190.stdout:2/806: creat d2/d8/d19/d7b/f118 x:0 0 0 2024-05-06T12:14:56.343 INFO:tasks.workunit.client.1.smithi190.stdout:6/615: write df/d2c/f25 [1458697,10910] 0 2024-05-06T12:14:56.344 INFO:tasks.workunit.client.1.smithi190.stdout:3/884: creat d5/d94/dd9/f133 x:0 0 0 2024-05-06T12:14:56.345 INFO:tasks.workunit.client.1.smithi190.stdout:2/807: chown d2/l3 3656420 1 2024-05-06T12:14:56.350 INFO:tasks.workunit.client.1.smithi190.stdout:4/640: dwrite d2/d11/d16/d8e/d80/f95 [0,4194304] 0 2024-05-06T12:14:56.350 INFO:tasks.workunit.client.1.smithi190.stdout:6/616: chown df/c5e 27322098 1 2024-05-06T12:14:56.351 INFO:tasks.workunit.client.1.smithi190.stdout:9/814: dread db/f5a [0,4194304] 0 2024-05-06T12:14:56.351 INFO:tasks.workunit.client.1.smithi190.stdout:1/664: link d5/d4f/d65/d62/f93 d5/d4f/d65/d62/d9d/fe6 0 2024-05-06T12:14:56.353 INFO:tasks.workunit.client.1.smithi190.stdout:3/885: creat d5/d7/d106/f134 x:0 0 0 2024-05-06T12:14:56.359 INFO:tasks.workunit.client.1.smithi190.stdout:5/826: getdents d2/d65/d83 0 2024-05-06T12:14:56.360 INFO:tasks.workunit.client.1.smithi190.stdout:9/815: chown db/dd/d16/d4c/d4e/d5e/c73 441025542 1 2024-05-06T12:14:56.363 INFO:tasks.workunit.client.1.smithi190.stdout:1/665: rename d5/d4f/d65/d32/d47/d53/d8b to d5/d4f/d65/d32/d47/d53/d98/da7/de7 0 2024-05-06T12:14:56.364 INFO:tasks.workunit.client.1.smithi190.stdout:6/617: dwrite df/d2c/d61/f79 [4194304,4194304] 0 2024-05-06T12:14:56.368 INFO:tasks.workunit.client.1.smithi190.stdout:9/816: mkdir db/dd/d5c/d11a 0 2024-05-06T12:14:56.369 INFO:tasks.workunit.client.1.smithi190.stdout:2/808: dwrite d2/d8/dc/f95 [4194304,4194304] 0 2024-05-06T12:14:56.370 INFO:tasks.workunit.client.1.smithi190.stdout:4/641: write d2/d11/d16/f90 [2031129,127715] 0 2024-05-06T12:14:56.372 INFO:tasks.workunit.client.1.smithi190.stdout:5/827: dread - d2/da/d59/d5f/d2e/d3b/d76/fd2 zero size 2024-05-06T12:14:56.373 INFO:tasks.workunit.client.1.smithi190.stdout:9/817: write db/d23/da4/f108 [1036609,53587] 0 2024-05-06T12:14:56.373 INFO:tasks.workunit.client.1.smithi190.stdout:6/618: creat df/d2c/d1e/d57/d89/fd5 x:0 0 0 2024-05-06T12:14:56.374 INFO:tasks.workunit.client.1.smithi190.stdout:2/809: fdatasync d2/d8/d33/d46/f8d 0 2024-05-06T12:14:56.374 INFO:tasks.workunit.client.1.smithi190.stdout:1/666: mkdir d5/d4f/d65/d32/d47/d53/de8 0 2024-05-06T12:14:56.377 INFO:tasks.workunit.client.1.smithi190.stdout:3/886: dwrite d5/ffe [4194304,4194304] 0 2024-05-06T12:14:56.379 INFO:tasks.workunit.client.1.smithi190.stdout:1/667: dread - d5/d4f/d65/d32/fc3 zero size 2024-05-06T12:14:56.379 INFO:tasks.workunit.client.1.smithi190.stdout:4/642: rmdir d2/d11/d16/d61/d9e 39 2024-05-06T12:14:56.383 INFO:tasks.workunit.client.1.smithi190.stdout:9/818: getdents db/d23/d29/dc4 0 2024-05-06T12:14:56.385 INFO:tasks.workunit.client.1.smithi190.stdout:5/828: dread d2/da/d59/d5f/d2e/d3b/f42 [0,4194304] 0 2024-05-06T12:14:56.385 INFO:tasks.workunit.client.1.smithi190.stdout:1/668: fdatasync d5/d4f/d65/d24/d43/fe3 0 2024-05-06T12:14:56.387 INFO:tasks.workunit.client.1.smithi190.stdout:2/810: dread d2/d8/d19/d43/fb1 [0,4194304] 0 2024-05-06T12:14:56.388 INFO:tasks.workunit.client.1.smithi190.stdout:4/643: symlink d2/d11/d16/d8e/d7b/dd4/lda 0 2024-05-06T12:14:56.393 INFO:tasks.workunit.client.1.smithi190.stdout:4/644: fsync d2/d11/d16/d61/d9e/d6e/fc4 0 2024-05-06T12:14:56.393 INFO:tasks.workunit.client.1.smithi190.stdout:2/811: mknod d2/d8/d33/d9a/c119 0 2024-05-06T12:14:56.394 INFO:tasks.workunit.client.1.smithi190.stdout:4/645: read d2/d11/d16/d4e/fcf [214849,49465] 0 2024-05-06T12:14:56.395 INFO:tasks.workunit.client.1.smithi190.stdout:4/646: chown d2/d11/d16/d61/d9e/la6 1670 1 2024-05-06T12:14:56.396 INFO:tasks.workunit.client.1.smithi190.stdout:9/819: unlink db/dd/df/dfe/ddc/d47/cad 0 2024-05-06T12:14:56.398 INFO:tasks.workunit.client.1.smithi190.stdout:5/829: rename d2/d65/cd5 to d2/da/d59/d5f/d2e/d3b/d76/d9a/c109 0 2024-05-06T12:14:56.402 INFO:tasks.workunit.client.1.smithi190.stdout:4/647: read d2/d11/d16/d61/d66/d6b/f9f [3009363,101844] 0 2024-05-06T12:14:56.405 INFO:tasks.workunit.client.1.smithi190.stdout:3/887: truncate d5/d14/d11/d21/d44/d92/d9e/fa2 172967 0 2024-05-06T12:14:56.405 INFO:tasks.workunit.client.1.smithi190.stdout:9/820: dwrite db/dd/df/dfe/d87/d22/d49/d88/d9f/df2/fdf [0,4194304] 0 2024-05-06T12:14:56.408 INFO:tasks.workunit.client.1.smithi190.stdout:7/791: dread d0/da/d36/d42/f85 [0,4194304] 0 2024-05-06T12:14:56.412 INFO:tasks.workunit.client.1.smithi190.stdout:3/888: write d5/d14/d11/d21/d44/d80/d82/d3d/ff4 [679226,30813] 0 2024-05-06T12:14:56.412 INFO:tasks.workunit.client.1.smithi190.stdout:1/669: write d5/d4f/f50 [1071234,67044] 0 2024-05-06T12:14:56.413 INFO:tasks.workunit.client.1.smithi190.stdout:2/812: write d2/f38 [586262,62827] 0 2024-05-06T12:14:56.421 INFO:tasks.workunit.client.1.smithi190.stdout:3/889: creat d5/d14/d11/d21/d44/d92/d9e/da4/d105/db3/da1/f135 x:0 0 0 2024-05-06T12:14:56.421 INFO:tasks.workunit.client.1.smithi190.stdout:4/648: mknod d2/cdb 0 2024-05-06T12:14:56.423 INFO:tasks.workunit.client.1.smithi190.stdout:1/670: readlink d5/d15/d1e/l31 0 2024-05-06T12:14:56.423 INFO:tasks.workunit.client.1.smithi190.stdout:2/813: stat d2/d8/d19/d4f/cdd 0 2024-05-06T12:14:56.425 INFO:tasks.workunit.client.1.smithi190.stdout:2/814: write d2/d8/d19/fa7 [1254922,114713] 0 2024-05-06T12:14:56.427 INFO:tasks.workunit.client.1.smithi190.stdout:1/671: mknod d5/d6b/d8f/ce9 0 2024-05-06T12:14:56.431 INFO:tasks.workunit.client.1.smithi190.stdout:3/890: creat d5/d7/d6d/d9a/d126/f136 x:0 0 0 2024-05-06T12:14:56.433 INFO:tasks.workunit.client.1.smithi190.stdout:2/815: creat d2/d8/d33/d46/dab/dac/f11a x:0 0 0 2024-05-06T12:14:56.437 INFO:tasks.workunit.client.1.smithi190.stdout:4/649: write d2/d11/d16/d31/f5c [1688542,127360] 0 2024-05-06T12:14:56.441 INFO:tasks.workunit.client.1.smithi190.stdout:2/816: read d2/d8/f92 [3191006,58516] 0 2024-05-06T12:14:56.448 INFO:tasks.workunit.client.1.smithi190.stdout:7/792: dwrite d0/da/d36/f9d [0,4194304] 0 2024-05-06T12:14:56.450 INFO:tasks.workunit.client.1.smithi190.stdout:1/672: dwrite d5/d4f/d65/d32/d47/d53/d98/da7/fb6 [0,4194304] 0 2024-05-06T12:14:56.452 INFO:tasks.workunit.client.1.smithi190.stdout:9/821: dwrite db/dd/df/dfe/d87/d22/d49/d8c/fc1 [0,4194304] 0 2024-05-06T12:14:56.453 INFO:tasks.workunit.client.1.smithi190.stdout:5/830: dwrite d2/da/d59/f77 [0,4194304] 0 2024-05-06T12:14:56.453 INFO:tasks.workunit.client.1.smithi190.stdout:2/817: symlink d2/d8/d33/dc2/dcd/l11b 0 2024-05-06T12:14:56.454 INFO:tasks.workunit.client.1.smithi190.stdout:4/650: read - d2/d11/d16/d4e/fd6 zero size 2024-05-06T12:14:56.454 INFO:tasks.workunit.client.1.smithi190.stdout:3/891: symlink d5/d14/d11/d21/d44/d80/d82/d3d/ded/l137 0 2024-05-06T12:14:56.456 INFO:tasks.workunit.client.1.smithi190.stdout:7/793: rmdir d0/da/d36/d42 39 2024-05-06T12:14:56.458 INFO:tasks.workunit.client.1.smithi190.stdout:5/831: rmdir d2/d65/d83 39 2024-05-06T12:14:56.462 INFO:tasks.workunit.client.1.smithi190.stdout:2/818: creat d2/d8/d19/d43/d79/db2/f11c x:0 0 0 2024-05-06T12:14:56.465 INFO:tasks.workunit.client.1.smithi190.stdout:9/822: creat db/dd/df/dfe/d87/d22/d49/d88/d9f/f11b x:0 0 0 2024-05-06T12:14:56.468 INFO:tasks.workunit.client.1.smithi190.stdout:7/794: getdents d0/d1 0 2024-05-06T12:14:56.472 INFO:tasks.workunit.client.1.smithi190.stdout:2/819: dwrite d2/d8/d19/d43/d79/d7e/f63 [0,4194304] 0 2024-05-06T12:14:56.472 INFO:tasks.workunit.client.1.smithi190.stdout:9/823: mkdir db/dd/df/dfe/d87/d99/d11c 0 2024-05-06T12:14:56.478 INFO:tasks.workunit.client.1.smithi190.stdout:2/820: creat d2/d110/f11d x:0 0 0 2024-05-06T12:14:56.479 INFO:tasks.workunit.client.1.smithi190.stdout:7/795: getdents d0/d1/d18/d6c/de1/de7 0 2024-05-06T12:14:56.481 INFO:tasks.workunit.client.1.smithi190.stdout:3/892: dread d5/d7/f86 [4194304,4194304] 0 2024-05-06T12:14:56.482 INFO:tasks.workunit.client.1.smithi190.stdout:7/796: dread - d0/da/d36/ff0 zero size 2024-05-06T12:14:56.486 INFO:tasks.workunit.client.1.smithi190.stdout:3/893: truncate d5/d7/d6d/d9a/fda 119978 0 2024-05-06T12:14:56.489 INFO:tasks.workunit.client.1.smithi190.stdout:2/821: rename d2/d8/d19/d43/d79/l98 to d2/d8/d19/d4f/l11e 0 2024-05-06T12:14:56.491 INFO:tasks.workunit.client.1.smithi190.stdout:4/651: write d2/f5e [1313464,74605] 0 2024-05-06T12:14:56.492 INFO:tasks.workunit.client.1.smithi190.stdout:3/894: creat d5/d14/d11/d21/d44/d92/d9e/da4/df3/f138 x:0 0 0 2024-05-06T12:14:56.492 INFO:tasks.workunit.client.1.smithi190.stdout:2/822: creat d2/d8/d19/d9b/f11f x:0 0 0 2024-05-06T12:14:56.495 INFO:tasks.workunit.client.1.smithi190.stdout:4/652: mkdir d2/d11/d16/d61/d9e/d6e/daa/ddc 0 2024-05-06T12:14:56.496 INFO:tasks.workunit.client.1.smithi190.stdout:3/895: symlink d5/l139 0 2024-05-06T12:14:56.498 INFO:tasks.workunit.client.1.smithi190.stdout:5/832: dread d2/da/d59/d5f/d6e/fb5 [0,4194304] 0 2024-05-06T12:14:56.500 INFO:tasks.workunit.client.1.smithi190.stdout:2/823: symlink d2/d8/d33/d46/dab/l120 0 2024-05-06T12:14:56.500 INFO:tasks.workunit.client.1.smithi190.stdout:4/653: chown d2/d11/d16/d61/d9e/d6e/d78/c96 334 1 2024-05-06T12:14:56.502 INFO:tasks.workunit.client.1.smithi190.stdout:3/896: creat d5/d14/d11/d21/d44/d92/d9e/da4/df3/f13a x:0 0 0 2024-05-06T12:14:56.504 INFO:tasks.workunit.client.1.smithi190.stdout:5/833: mknod d2/da/d59/d5f/d2e/d3b/c10a 0 2024-05-06T12:14:56.505 INFO:tasks.workunit.client.1.smithi190.stdout:2/824: symlink d2/d8/d19/d7b/d10f/l121 0 2024-05-06T12:14:56.507 INFO:tasks.workunit.client.1.smithi190.stdout:4/654: rename d2/d11/d16/d61/d82/fcc to d2/d11/d16/d61/d9e/d6e/daa/fdd 0 2024-05-06T12:14:56.507 INFO:tasks.workunit.client.1.smithi190.stdout:2/825: read d2/d8/d71/f88 [2191415,78531] 0 2024-05-06T12:14:56.511 INFO:tasks.workunit.client.1.smithi190.stdout:2/826: creat d2/d8/d19/d7b/f122 x:0 0 0 2024-05-06T12:14:56.512 INFO:tasks.workunit.client.1.smithi190.stdout:9/824: dread db/dd/df/dfe/d87/d22/d49/f79 [0,4194304] 0 2024-05-06T12:14:56.517 INFO:tasks.workunit.client.1.smithi190.stdout:9/825: write db/dd/df/dfe/d87/d99/fc9 [919891,99615] 0 2024-05-06T12:14:56.518 INFO:tasks.workunit.client.1.smithi190.stdout:7/797: dread d0/da/d36/d42/fc4 [0,4194304] 0 2024-05-06T12:14:56.522 INFO:tasks.workunit.client.1.smithi190.stdout:2/827: dwrite d2/d8/d33/d46/f5f [4194304,4194304] 0 2024-05-06T12:14:56.529 INFO:tasks.workunit.client.1.smithi190.stdout:0/830: sync 2024-05-06T12:14:56.533 INFO:tasks.workunit.client.1.smithi190.stdout:7/798: dread d0/d1/d18/fa2 [4194304,4194304] 0 2024-05-06T12:14:56.533 INFO:tasks.workunit.client.1.smithi190.stdout:5/834: dread d2/da/f1b [0,4194304] 0 2024-05-06T12:14:56.534 INFO:tasks.workunit.client.1.smithi190.stdout:9/826: getdents db/dd/d5c 0 2024-05-06T12:14:56.535 INFO:tasks.workunit.client.1.smithi190.stdout:0/831: read d9/dc/d28/f5d [2254411,103248] 0 2024-05-06T12:14:56.535 INFO:tasks.workunit.client.1.smithi190.stdout:4/655: dread d2/f59 [0,4194304] 0 2024-05-06T12:14:56.537 INFO:tasks.workunit.client.1.smithi190.stdout:2/828: write d2/d21/f70 [73866,121195] 0 2024-05-06T12:14:56.539 INFO:tasks.workunit.client.1.smithi190.stdout:4/656: write d2/d11/d16/d61/d9e/d6e/f92 [148088,63187] 0 2024-05-06T12:14:56.540 INFO:tasks.workunit.client.1.smithi190.stdout:5/835: truncate d2/da/d59/d5f/d50/d6d/fe2 1041453 0 2024-05-06T12:14:56.543 INFO:tasks.workunit.client.1.smithi190.stdout:7/799: write d0/d1/f3e [5602761,92473] 0 2024-05-06T12:14:56.545 INFO:tasks.workunit.client.1.smithi190.stdout:5/836: stat d2/da/c86 0 2024-05-06T12:14:56.546 INFO:tasks.workunit.client.1.smithi190.stdout:4/657: symlink d2/d11/d16/d61/d9e/d6e/daa/ddc/lde 0 2024-05-06T12:14:56.547 INFO:tasks.workunit.client.1.smithi190.stdout:5/837: chown d2/d6c/f7a 2 1 2024-05-06T12:14:56.550 INFO:tasks.workunit.client.1.smithi190.stdout:2/829: rename d2/d8/d19/d43 to d2/d21/d123 0 2024-05-06T12:14:56.553 INFO:tasks.workunit.client.1.smithi190.stdout:2/830: chown d2/d8/d33/d46/fe9 203 1 2024-05-06T12:14:56.554 INFO:tasks.workunit.client.1.smithi190.stdout:4/658: dwrite d2/d10/fb8 [0,4194304] 0 2024-05-06T12:14:56.554 INFO:tasks.workunit.client.1.smithi190.stdout:5/838: unlink d2/da/d59/d5f/d2e/d3b/c63 0 2024-05-06T12:14:56.582 INFO:tasks.workunit.client.1.smithi190.stdout:7/800: truncate d0/da/d36/f43 3138172 0 2024-05-06T12:14:56.583 INFO:tasks.workunit.client.1.smithi190.stdout:0/832: dread d9/dc/d48/d50/d83/f94 [0,4194304] 0 2024-05-06T12:14:56.584 INFO:tasks.workunit.client.1.smithi190.stdout:9/827: dread db/dd/df/dfe/d87/d22/d49/f89 [0,4194304] 0 2024-05-06T12:14:56.687 INFO:tasks.workunit.client.1.smithi190.stdout:4/659: creat d2/d11/d16/d61/d9e/d6e/d78/fdf x:0 0 0 2024-05-06T12:14:56.688 INFO:tasks.workunit.client.1.smithi190.stdout:0/833: chown d9/dc/db6/d11b/f124 3 1 2024-05-06T12:14:56.689 INFO:tasks.workunit.client.1.smithi190.stdout:2/831: dwrite d2/d8/fdc [0,4194304] 0 2024-05-06T12:14:56.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.689+0000 7f9b641ae700 1 -- 172.21.15.92:0/2480165038 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9b5c105980 con 0x7f9b5c0fce80 2024-05-06T12:14:56.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.691+0000 7f9b527fc700 1 -- 172.21.15.92:0/2480165038 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f9b58055320 con 0x7f9b5c0fce80 2024-05-06T12:14:56.694 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:14:56.694 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": { 2024-05-06T12:14:56.694 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 2 2024-05-06T12:14:56.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:14:56.694 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": { 2024-05-06T12:14:56.694 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 1, 2024-05-06T12:14:56.694 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 1 2024-05-06T12:14:56.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:14:56.694 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": { 2024-05-06T12:14:56.695 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 6 2024-05-06T12:14:56.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:14:56.695 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-05-06T12:14:56.695 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 4 2024-05-06T12:14:56.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:14:56.695 INFO:teuthology.orchestra.run.smithi092.stdout: "overall": { 2024-05-06T12:14:56.695 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 13, 2024-05-06T12:14:56.695 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 1 2024-05-06T12:14:56.695 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-06T12:14:56.695 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:14:56.696 INFO:tasks.workunit.client.1.smithi190.stdout:7/801: dwrite d0/d4/fde [0,4194304] 0 2024-05-06T12:14:56.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.694+0000 7f9b641ae700 1 -- 172.21.15.92:0/2480165038 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f9b4806b790 msgr2=0x7f9b4806dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:56.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.694+0000 7f9b641ae700 1 --2- 172.21.15.92:0/2480165038 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f9b4806b790 0x7f9b4806dc40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c004600 tx=0x7f9b4c00b040 comp rx=0 tx=0).stop 2024-05-06T12:14:56.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.694+0000 7f9b641ae700 1 -- 172.21.15.92:0/2480165038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b5c0fce80 msgr2=0x7f9b5c10b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:56.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.694+0000 7f9b641ae700 1 --2- 172.21.15.92:0/2480165038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b5c0fce80 0x7f9b5c10b790 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9b58013870 tx=0x7f9b58009e30 comp rx=0 tx=0).stop 2024-05-06T12:14:56.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.694+0000 7f9b641ae700 1 -- 172.21.15.92:0/2480165038 shutdown_connections 2024-05-06T12:14:56.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.694+0000 7f9b641ae700 1 --2- 172.21.15.92:0/2480165038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b5c0fce80 0x7f9b5c10b790 secure :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9b58013870 tx=0x7f9b58009e30 comp rx=0 tx=0).stop 2024-05-06T12:14:56.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.694+0000 7f9b641ae700 1 --2- 172.21.15.92:0/2480165038 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f9b4806b790 0x7f9b4806dc40 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:56.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.694+0000 7f9b641ae700 1 --2- 172.21.15.92:0/2480165038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b5c0fdff0 0x7f9b5c10bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:56.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.694+0000 7f9b641ae700 1 -- 172.21.15.92:0/2480165038 >> 172.21.15.92:0/2480165038 conn(0x7f9b5c0f8540 msgr2=0x7f9b5c0f9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:14:56.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.695+0000 7f9b641ae700 1 -- 172.21.15.92:0/2480165038 shutdown_connections 2024-05-06T12:14:56.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.695+0000 7f9b641ae700 1 -- 172.21.15.92:0/2480165038 wait complete. 2024-05-06T12:14:56.710 INFO:tasks.workunit.client.1.smithi190.stdout:8/840: sync 2024-05-06T12:14:56.711 INFO:tasks.workunit.client.1.smithi190.stdout:8/841: readlink d3/d13/l70 0 2024-05-06T12:14:56.717 INFO:tasks.workunit.client.1.smithi190.stdout:8/842: dwrite d3/d8/d25/d3b/d90/dcb/dfb/f105 [0,4194304] 0 2024-05-06T12:14:56.723 INFO:tasks.workunit.client.1.smithi190.stdout:5/839: symlink d2/da/d59/d5f/d2e/d3b/d72/dec/l10b 0 2024-05-06T12:14:56.729 INFO:tasks.workunit.client.1.smithi190.stdout:4/660: unlink d2/d11/d16/d4e/c51 0 2024-05-06T12:14:56.730 INFO:tasks.workunit.client.1.smithi190.stdout:2/832: creat d2/d8/d19/da9/f124 x:0 0 0 2024-05-06T12:14:56.731 INFO:tasks.workunit.client.1.smithi190.stdout:7/802: rmdir d0/d1/d18/d4e 39 2024-05-06T12:14:56.731 INFO:tasks.workunit.client.1.smithi190.stdout:2/833: write d2/d21/d123/d79/f82 [2679017,107711] 0 2024-05-06T12:14:56.732 INFO:tasks.workunit.client.1.smithi190.stdout:7/803: chown d0/da/f5b 258115 1 2024-05-06T12:14:56.735 INFO:tasks.workunit.client.1.smithi190.stdout:8/843: unlink d3/d8/d25/d3b/d90/dcb/dfb/l121 0 2024-05-06T12:14:56.736 INFO:tasks.workunit.client.1.smithi190.stdout:9/828: symlink db/dd/df/dfe/d87/l11d 0 2024-05-06T12:14:56.738 INFO:tasks.workunit.client.1.smithi190.stdout:5/840: symlink d2/da/d59/d5f/d2e/da7/d9d/l10c 0 2024-05-06T12:14:56.739 INFO:tasks.workunit.client.1.smithi190.stdout:7/804: dwrite d0/d1/d2/f1d [0,4194304] 0 2024-05-06T12:14:56.740 INFO:tasks.workunit.client.1.smithi190.stdout:5/841: chown d2/da/d59/f84 486288844 1 2024-05-06T12:14:56.741 INFO:tasks.workunit.client.1.smithi190.stdout:4/661: symlink d2/d11/d16/d61/d9e/d6e/daa/le0 0 2024-05-06T12:14:56.742 INFO:tasks.workunit.client.1.smithi190.stdout:4/662: readlink d2/d11/d16/d31/lac 0 2024-05-06T12:14:56.745 INFO:tasks.workunit.client.1.smithi190.stdout:4/663: chown d2/d11/d16/d61/d9e/d6e/daa/le0 2095 1 2024-05-06T12:14:56.746 INFO:tasks.workunit.client.1.smithi190.stdout:8/844: rmdir d3/d35/d55/d61/df9 39 2024-05-06T12:14:56.748 INFO:tasks.workunit.client.1.smithi190.stdout:6/619: sync 2024-05-06T12:14:56.748 INFO:tasks.workunit.client.1.smithi190.stdout:1/673: sync 2024-05-06T12:14:56.748 INFO:tasks.workunit.client.1.smithi190.stdout:3/897: sync 2024-05-06T12:14:56.751 INFO:tasks.workunit.client.1.smithi190.stdout:5/842: symlink d2/d6c/l10d 0 2024-05-06T12:14:56.752 INFO:tasks.workunit.client.1.smithi190.stdout:8/845: fdatasync d3/d8/d25/f39 0 2024-05-06T12:14:56.752 INFO:tasks.workunit.client.1.smithi190.stdout:3/898: chown d5/d14/d11/d21/d44/d92/lcc 52788 1 2024-05-06T12:14:56.754 INFO:tasks.workunit.client.1.smithi190.stdout:8/846: read d3/d8/d68/faf [3827564,116491] 0 2024-05-06T12:14:56.755 INFO:tasks.workunit.client.1.smithi190.stdout:4/664: mkdir d2/d11/d16/d61/d66/d6b/da4/de1 0 2024-05-06T12:14:56.755 INFO:tasks.workunit.client.1.smithi190.stdout:3/899: chown d5/d7/cd1 0 1 2024-05-06T12:14:56.756 INFO:tasks.workunit.client.1.smithi190.stdout:7/805: dread d0/d1/d32/d45/d70/fd6 [0,4194304] 0 2024-05-06T12:14:56.757 INFO:tasks.workunit.client.1.smithi190.stdout:6/620: unlink df/d2c/d1e/d57/d53/fa4 0 2024-05-06T12:14:56.757 INFO:tasks.workunit.client.1.smithi190.stdout:2/834: dread d2/d8/d33/d46/d5d/fe1 [0,4194304] 0 2024-05-06T12:14:56.758 INFO:tasks.workunit.client.1.smithi190.stdout:1/674: rmdir d5/d4f/d65/d32/d47/d53/d98/da7/de7 39 2024-05-06T12:14:56.758 INFO:tasks.workunit.client.1.smithi190.stdout:4/665: read d2/d11/d16/d61/d82/f98 [1802965,67844] 0 2024-05-06T12:14:56.759 INFO:tasks.workunit.client.1.smithi190.stdout:1/675: read d5/d15/fbf [353347,76985] 0 2024-05-06T12:14:56.760 INFO:tasks.workunit.client.1.smithi190.stdout:5/843: dread - d2/da/d59/f84 zero size 2024-05-06T12:14:56.763 INFO:tasks.workunit.client.1.smithi190.stdout:3/900: write d5/d14/d11/d21/d44/d80/d82/d3d/ded/fe [3323446,22180] 0 2024-05-06T12:14:56.766 INFO:tasks.workunit.client.1.smithi190.stdout:6/621: mkdir df/d2c/d1e/d57/d53/dd6 0 2024-05-06T12:14:56.766 INFO:tasks.workunit.client.1.smithi190.stdout:3/901: write d5/d14/d11/d21/d44/d92/d9e/da4/d105/db3/dfa/f113 [76784,14195] 0 2024-05-06T12:14:56.769 INFO:tasks.workunit.client.1.smithi190.stdout:7/806: mknod d0/d1/d32/d45/d70/d33/df7/c118 0 2024-05-06T12:14:56.770 INFO:tasks.workunit.client.1.smithi190.stdout:2/835: dwrite d2/d8/f92 [0,4194304] 0 2024-05-06T12:14:56.771 INFO:tasks.workunit.client.1.smithi190.stdout:1/676: symlink d5/d4f/d65/d32/d47/d53/d98/lea 0 2024-05-06T12:14:56.773 INFO:tasks.workunit.client.1.smithi190.stdout:8/847: mkdir d3/d8/d2b/d122 0 2024-05-06T12:14:56.777 INFO:tasks.workunit.client.1.smithi190.stdout:3/902: rmdir d5/d14/d11/d21/d44/d80/ddb/d124 39 2024-05-06T12:14:56.779 INFO:tasks.workunit.client.1.smithi190.stdout:4/666: symlink d2/d11/d16/d61/d9e/d6e/d78/le2 0 2024-05-06T12:14:56.783 INFO:tasks.workunit.client.1.smithi190.stdout:2/836: symlink d2/d8/d33/de6/l125 0 2024-05-06T12:14:56.784 INFO:tasks.workunit.client.1.smithi190.stdout:8/848: mknod d3/d8/d68/c123 0 2024-05-06T12:14:56.787 INFO:tasks.workunit.client.1.smithi190.stdout:3/903: dread - d5/d14/d11/fab zero size 2024-05-06T12:14:56.787 INFO:tasks.workunit.client.1.smithi190.stdout:1/677: dread d5/f16 [0,4194304] 0 2024-05-06T12:14:56.789 INFO:tasks.workunit.client.1.smithi190.stdout:7/807: mkdir d0/da/d36/d119 0 2024-05-06T12:14:56.791 INFO:tasks.workunit.client.1.smithi190.stdout:4/667: mkdir d2/d11/d16/d61/d66/d6b/de3 0 2024-05-06T12:14:56.792 INFO:tasks.workunit.client.1.smithi190.stdout:2/837: dwrite d2/d8/d19/d7b/f122 [0,4194304] 0 2024-05-06T12:14:56.796 INFO:tasks.workunit.client.1.smithi190.stdout:0/834: write d9/dc/f6c [488103,66278] 0 2024-05-06T12:14:56.797 INFO:tasks.workunit.client.1.smithi190.stdout:1/678: symlink d5/d4f/d65/d32/dac/leb 0 2024-05-06T12:14:56.798 INFO:tasks.workunit.client.1.smithi190.stdout:0/835: stat d9/c82 0 2024-05-06T12:14:56.798 INFO:tasks.workunit.client.1.smithi190.stdout:3/904: mknod d5/d94/dd9/c13b 0 2024-05-06T12:14:56.801 INFO:tasks.workunit.client.1.smithi190.stdout:5/844: getdents d2/da/d59/d5f/d2e 0 2024-05-06T12:14:56.804 INFO:tasks.workunit.client.1.smithi190.stdout:2/838: dwrite d2/d8/f92 [0,4194304] 0 2024-05-06T12:14:56.805 INFO:tasks.workunit.client.1.smithi190.stdout:3/905: truncate d5/d14/d11/d21/d44/d92/d9e/fee 4906208 0 2024-05-06T12:14:56.812 INFO:tasks.workunit.client.1.smithi190.stdout:9/829: write db/dd/d16/d4c/d4e/d5e/fbe [859269,14489] 0 2024-05-06T12:14:56.821 INFO:tasks.workunit.client.1.smithi190.stdout:2/839: mkdir d2/d8/d33/d46/db7/d126 0 2024-05-06T12:14:56.821 INFO:tasks.workunit.client.1.smithi190.stdout:3/906: dwrite d5/d7/d6d/d9a/d126/f136 [0,4194304] 0 2024-05-06T12:14:56.823 INFO:tasks.workunit.client.1.smithi190.stdout:2/840: read - d2/d21/d123/d79/d7e/fe2 zero size 2024-05-06T12:14:56.823 INFO:tasks.workunit.client.1.smithi190.stdout:0/836: dread d9/dcf/d21/f7e [0,4194304] 0 2024-05-06T12:14:56.824 INFO:tasks.workunit.client.1.smithi190.stdout:5/845: unlink d2/d27/c74 0 2024-05-06T12:14:56.829 INFO:tasks.workunit.client.1.smithi190.stdout:9/830: dwrite db/dd/df/dfe/d87/d22/d49/d88/d9f/f11b [0,4194304] 0 2024-05-06T12:14:56.832 INFO:tasks.workunit.client.1.smithi190.stdout:6/622: write df/d17/d37/f42 [10606,16539] 0 2024-05-06T12:14:56.835 INFO:tasks.workunit.client.1.smithi190.stdout:3/907: mknod d5/d14/d11/d21/d44/d92/d9e/da4/d105/d111/c13c 0 2024-05-06T12:14:56.835 INFO:tasks.workunit.client.1.smithi190.stdout:3/908: chown d5/d7/d15/f28 18 1 2024-05-06T12:14:56.836 INFO:tasks.workunit.client.1.smithi190.stdout:8/849: link d3/d35/d55/d61/lc9 d3/d8/d25/l124 0 2024-05-06T12:14:56.839 INFO:tasks.workunit.client.1.smithi190.stdout:2/841: mkdir d2/d8/d33/d127 0 2024-05-06T12:14:56.840 INFO:tasks.workunit.client.1.smithi190.stdout:2/842: fdatasync d2/d21/f70 0 2024-05-06T12:14:56.841 INFO:tasks.workunit.client.1.smithi190.stdout:2/843: write d2/f38 [4745862,120180] 0 2024-05-06T12:14:56.843 INFO:tasks.workunit.client.1.smithi190.stdout:2/844: fsync d2/d8/dc/f13 0 2024-05-06T12:14:56.843 INFO:tasks.workunit.client.1.smithi190.stdout:9/831: rename db/dd/df/dfe/d87/d22/d49/d88/d9f/df2/fb9 to db/dd/df/dfe/d87/d22/d49/d88/d9f/f11e 0 2024-05-06T12:14:56.845 INFO:tasks.workunit.client.1.smithi190.stdout:6/623: symlink df/d2c/d61/ld7 0 2024-05-06T12:14:56.847 INFO:tasks.workunit.client.1.smithi190.stdout:0/837: truncate d9/dc/dd0/fd8 583270 0 2024-05-06T12:14:56.849 INFO:tasks.workunit.client.1.smithi190.stdout:3/909: creat d5/d14/d11/d21/d44/d80/d82/d3d/ded/d33/f13d x:0 0 0 2024-05-06T12:14:56.855 INFO:tasks.workunit.client.1.smithi190.stdout:4/668: truncate d2/d11/d16/d61/d82/fa2 1876339 0 2024-05-06T12:14:56.860 INFO:tasks.workunit.client.1.smithi190.stdout:8/850: rename d3/d8/d68/cbd to d3/d8/d25/d3b/d111/c125 0 2024-05-06T12:14:56.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.857+0000 7f6f9affd700 1 -- 172.21.15.92:0/2237988894 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f8c013b70 con 0x7f6f9c06ed90 2024-05-06T12:14:56.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.858+0000 7f6fa2702700 1 -- 172.21.15.92:0/2237988894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f9c06ed90 msgr2=0x7f6f9c108f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:56.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.858+0000 7f6fa2702700 1 --2- 172.21.15.92:0/2237988894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f9c06ed90 0x7f6f9c108f80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c004940 tx=0x7f6f8c009e30 comp rx=0 tx=0).stop 2024-05-06T12:14:56.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.858+0000 7f6fa2702700 1 -- 172.21.15.92:0/2237988894 shutdown_connections 2024-05-06T12:14:56.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.858+0000 7f6fa2702700 1 --2- 172.21.15.92:0/2237988894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f9c06ed90 0x7f6f9c108f80 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c004940 tx=0x7f6f8c009e30 comp rx=0 tx=0).stop 2024-05-06T12:14:56.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.858+0000 7f6fa2702700 1 --2- 172.21.15.92:0/2237988894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f9c06e440 0x7f6f9c06e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:56.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.858+0000 7f6fa2702700 1 -- 172.21.15.92:0/2237988894 >> 172.21.15.92:0/2237988894 conn(0x7f6f9c069dc0 msgr2=0x7f6f9c06c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:14:56.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.858+0000 7f6fa2702700 1 -- 172.21.15.92:0/2237988894 shutdown_connections 2024-05-06T12:14:56.862 INFO:tasks.workunit.client.1.smithi190.stdout:7/808: dwrite d0/d1/d18/d4e/d87/fa5 [0,4194304] 0 2024-05-06T12:14:56.862 INFO:tasks.workunit.client.1.smithi190.stdout:5/846: dread d2/da/d59/d5f/ff0 [0,4194304] 0 2024-05-06T12:14:56.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.859+0000 7f6fa2702700 1 -- 172.21.15.92:0/2237988894 wait complete. 2024-05-06T12:14:56.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.859+0000 7f6fa2702700 1 Processor -- start 2024-05-06T12:14:56.864 INFO:tasks.workunit.client.1.smithi190.stdout:2/845: symlink d2/d8/d33/d46/l128 0 2024-05-06T12:14:56.864 INFO:tasks.workunit.client.1.smithi190.stdout:1/679: write d5/d15/f30 [408730,21789] 0 2024-05-06T12:14:56.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.860+0000 7f6fa2702700 1 -- start start 2024-05-06T12:14:56.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.860+0000 7f6fa2702700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f9c06e440 0x7f6f9c11c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:56.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.860+0000 7f6fa2702700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f9c11cd50 0x7f6f9c121d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:56.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.860+0000 7f6fa2702700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c11d210 con 0x7f6f9c11cd50 2024-05-06T12:14:56.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.860+0000 7f6fa2702700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c11d350 con 0x7f6f9c06e440 2024-05-06T12:14:56.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.860+0000 7f6f9bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f9c06e440 0x7f6f9c11c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:56.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.860+0000 7f6f9bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f9c06e440 0x7f6f9c11c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.92:53130/0 (socket says 172.21.15.92:53130) 2024-05-06T12:14:56.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.860+0000 7f6f9bfff700 1 -- 172.21.15.92:0/113730957 learned_addr learned my addr 172.21.15.92:0/113730957 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:14:56.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.861+0000 7f6f9b7fe700 1 --2- 172.21.15.92:0/113730957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f9c11cd50 0x7f6f9c121d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:56.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.861+0000 7f6f9b7fe700 1 -- 172.21.15.92:0/113730957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f9c06e440 msgr2=0x7f6f9c11c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:56.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.861+0000 7f6f9b7fe700 1 --2- 172.21.15.92:0/113730957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f9c06e440 0x7f6f9c11c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:56.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.861+0000 7f6f9b7fe700 1 -- 172.21.15.92:0/113730957 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f8c005040 con 0x7f6f9c11cd50 2024-05-06T12:14:56.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.862+0000 7f6f9b7fe700 1 --2- 172.21.15.92:0/113730957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f9c11cd50 0x7f6f9c121d80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c008000 tx=0x7f6f8c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:14:56.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.862+0000 7f6f98ff9700 1 -- 172.21.15.92:0/113730957 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f8c020070 con 0x7f6f9c11cd50 2024-05-06T12:14:56.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.862+0000 7f6f98ff9700 1 -- 172.21.15.92:0/113730957 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6f8c013de0 con 0x7f6f9c11cd50 2024-05-06T12:14:56.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.862+0000 7f6fa2702700 1 -- 172.21.15.92:0/113730957 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f9c10b580 con 0x7f6f9c11cd50 2024-05-06T12:14:56.867 INFO:tasks.workunit.client.1.smithi190.stdout:0/838: dwrite d9/dc/d28/dc5/dd5/ff8 [0,4194304] 0 2024-05-06T12:14:56.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.862+0000 7f6fa2702700 1 -- 172.21.15.92:0/113730957 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f9c10ba60 con 0x7f6f9c11cd50 2024-05-06T12:14:56.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.862+0000 7f6f98ff9700 1 -- 172.21.15.92:0/113730957 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f8c00e070 con 0x7f6f9c11cd50 2024-05-06T12:14:56.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.864+0000 7f6fa2702700 1 -- 172.21.15.92:0/113730957 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f7c0052e0 con 0x7f6f9c11cd50 2024-05-06T12:14:56.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.865+0000 7f6f98ff9700 1 -- 172.21.15.92:0/113730957 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7f6f8c01cd50 con 0x7f6f9c11cd50 2024-05-06T12:14:56.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.865+0000 7f6f98ff9700 1 --2- 172.21.15.92:0/113730957 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f6f8806b8a0 0x7f6f8806dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:56.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.865+0000 7f6f98ff9700 1 -- 172.21.15.92:0/113730957 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f6f8c08b160 con 0x7f6f9c11cd50 2024-05-06T12:14:56.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.868+0000 7f6f9bfff700 1 --2- 172.21.15.92:0/113730957 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f6f8806b8a0 0x7f6f8806dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:56.871 INFO:tasks.workunit.client.1.smithi190.stdout:2/846: dread d2/d21/d123/d79/d7e/f63 [0,4194304] 0 2024-05-06T12:14:56.872 INFO:tasks.workunit.client.1.smithi190.stdout:2/847: chown d2/d8/d33/dc2/cc7 2 1 2024-05-06T12:14:56.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.870+0000 7f6f9bfff700 1 --2- 172.21.15.92:0/113730957 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f6f8806b8a0 0x7f6f8806dd50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6f84008fd0 tx=0x7f6f8400b040 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:14:56.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:56.872+0000 7f6f98ff9700 1 -- 172.21.15.92:0/113730957 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f6f8c0587d0 con 0x7f6f9c11cd50 2024-05-06T12:14:56.882 INFO:tasks.workunit.client.1.smithi190.stdout:2/848: dwrite d2/d8/d19/f83 [4194304,4194304] 0 2024-05-06T12:14:56.891 INFO:tasks.workunit.client.1.smithi190.stdout:7/809: unlink d0/d1/d18/d4e/d87/fa5 0 2024-05-06T12:14:56.892 INFO:tasks.workunit.client.1.smithi190.stdout:1/680: mknod d5/d4f/d65/d24/d43/da6/cec 0 2024-05-06T12:14:56.903 INFO:tasks.workunit.client.1.smithi190.stdout:2/849: creat d2/d8/d33/d46/db7/f129 x:0 0 0 2024-05-06T12:14:56.905 INFO:tasks.workunit.client.1.smithi190.stdout:5/847: mkdir d2/d10e 0 2024-05-06T12:14:56.911 INFO:tasks.workunit.client.1.smithi190.stdout:6/624: dwrite df/d2c/d61/d84/faa [0,4194304] 0 2024-05-06T12:14:56.916 INFO:tasks.workunit.client.1.smithi190.stdout:9/832: write db/d23/f3a [1568420,18248] 0 2024-05-06T12:14:56.921 INFO:tasks.workunit.client.1.smithi190.stdout:3/910: creat d5/d14/d11/d21/d44/d80/ddb/f13e x:0 0 0 2024-05-06T12:14:56.921 INFO:tasks.workunit.client.1.smithi190.stdout:8/851: dwrite d3/d35/d3a/f3d [0,4194304] 0 2024-05-06T12:14:56.929 INFO:tasks.workunit.client.1.smithi190.stdout:7/810: write d0/da/ff4 [3821234,90670] 0 2024-05-06T12:14:56.929 INFO:tasks.workunit.client.1.smithi190.stdout:0/839: dwrite d9/d51/dc9/fa2 [0,4194304] 0 2024-05-06T12:14:56.930 INFO:tasks.workunit.client.1.smithi190.stdout:4/669: creat d2/d11/d16/d61/d66/d6b/da4/de1/fe4 x:0 0 0 2024-05-06T12:14:56.932 INFO:tasks.workunit.client.1.smithi190.stdout:4/670: readlink d2/d11/l8d 0 2024-05-06T12:14:56.934 INFO:tasks.workunit.client.1.smithi190.stdout:5/848: symlink d2/da/d59/d5f/d2e/d8b/dbe/l10f 0 2024-05-06T12:14:56.937 INFO:tasks.workunit.client.1.smithi190.stdout:3/911: truncate d5/d14/d11/d21/d44/d80/f11f 53009 0 2024-05-06T12:14:56.942 INFO:tasks.workunit.client.1.smithi190.stdout:6/625: creat df/d2c/dae/fd8 x:0 0 0 2024-05-06T12:14:56.943 INFO:tasks.workunit.client.1.smithi190.stdout:4/671: dread d2/d11/d16/d61/d9e/d6e/f92 [0,4194304] 0 2024-05-06T12:14:56.944 INFO:tasks.workunit.client.1.smithi190.stdout:2/850: link d2/d8/d33/d46/dab/fee d2/d21/d123/f12a 0 2024-05-06T12:14:56.946 INFO:tasks.workunit.client.1.smithi190.stdout:7/811: creat d0/da/d36/d53/d77/d8c/dd1/d60/dac/f11a x:0 0 0 2024-05-06T12:14:56.949 INFO:tasks.workunit.client.1.smithi190.stdout:7/812: chown d0/da/d36/d53/d77/d8c/dd1/d60/dac 100084095 1 2024-05-06T12:14:56.951 INFO:tasks.workunit.client.1.smithi190.stdout:5/849: dwrite d2/d65/fb0 [0,4194304] 0 2024-05-06T12:14:56.952 INFO:tasks.workunit.client.1.smithi190.stdout:1/681: getdents d5/d4f/d65/d32/d47/d53/d98 0 2024-05-06T12:14:56.956 INFO:tasks.workunit.client.1.smithi190.stdout:8/852: creat d3/d35/d55/d61/f126 x:0 0 0 2024-05-06T12:14:56.956 INFO:tasks.workunit.client.1.smithi190.stdout:2/851: truncate d2/d8/d33/d46/dab/f6d 973721 0 2024-05-06T12:14:56.960 INFO:tasks.workunit.client.1.smithi190.stdout:3/912: link d5/d14/d11/dc9/f10f d5/d14/d11/d21/f13f 0 2024-05-06T12:14:56.966 INFO:tasks.workunit.client.1.smithi190.stdout:3/913: write d5/d14/d11/d21/d44/d92/d9e/da4/d105/db3/da1/fc3 [5016877,90392] 0 2024-05-06T12:14:56.968 INFO:tasks.workunit.client.1.smithi190.stdout:2/852: mkdir d2/d4c/d10a/d12b 0 2024-05-06T12:14:56.971 INFO:tasks.workunit.client.1.smithi190.stdout:5/850: dwrite d2/da/d59/d5f/d2e/d3b/d5a/f85 [0,4194304] 0 2024-05-06T12:14:56.973 INFO:tasks.workunit.client.1.smithi190.stdout:1/682: mkdir d5/d4f/d65/d24/d43/da6/ded 0 2024-05-06T12:14:56.975 INFO:tasks.workunit.client.1.smithi190.stdout:0/840: dwrite d9/dc/d28/dc5/dd5/de4/dea/dfd/d110/f112 [0,4194304] 0 2024-05-06T12:14:56.978 INFO:tasks.workunit.client.1.smithi190.stdout:6/626: getdents df/d2c/dae 0 2024-05-06T12:14:56.980 INFO:tasks.workunit.client.1.smithi190.stdout:8/853: symlink d3/d8/d68/d9f/l127 0 2024-05-06T12:14:56.984 INFO:tasks.workunit.client.1.smithi190.stdout:9/833: dwrite db/dd/df/dfe/d87/d22/d49/d88/faf [0,4194304] 0 2024-05-06T12:14:56.987 INFO:tasks.workunit.client.1.smithi190.stdout:7/813: truncate d0/d1/d18/d2b/f83 595515 0 2024-05-06T12:14:56.988 INFO:tasks.workunit.client.1.smithi190.stdout:2/853: creat d2/d8/d33/dc2/dcb/f12c x:0 0 0 2024-05-06T12:14:56.989 INFO:tasks.workunit.client.1.smithi190.stdout:1/683: unlink d5/d4f/d65/d24/d43/f60 0 2024-05-06T12:14:56.991 INFO:tasks.workunit.client.1.smithi190.stdout:5/851: rename d2/da/l71 to d2/da/d59/d5f/d2e/d8b/l110 0 2024-05-06T12:14:56.993 INFO:tasks.workunit.client.1.smithi190.stdout:3/914: chown d5/d14/d11/d21/d44/d92/d9e/fa2 0 1 2024-05-06T12:14:56.995 INFO:tasks.workunit.client.1.smithi190.stdout:6/627: read df/d2c/d92/f93 [1559974,5160] 0 2024-05-06T12:14:57.002 INFO:tasks.workunit.client.1.smithi190.stdout:3/915: chown d5/d14/d11/d21/d44/d80/d82/l72 1648639 1 2024-05-06T12:14:57.002 INFO:tasks.workunit.client.1.smithi190.stdout:3/916: chown d5/d7/lb6 40260 1 2024-05-06T12:14:57.004 INFO:tasks.workunit.client.1.smithi190.stdout:5/852: read d2/d65/d83/fcf [49254,119199] 0 2024-05-06T12:14:57.008 INFO:tasks.workunit.client.1.smithi190.stdout:6/628: creat df/d2c/d92/dce/fd9 x:0 0 0 2024-05-06T12:14:57.010 INFO:tasks.workunit.client.1.smithi190.stdout:3/917: write d5/d14/d11/d21/d44/d80/d82/fea [1660438,54525] 0 2024-05-06T12:14:57.012 INFO:tasks.workunit.client.1.smithi190.stdout:8/854: dwrite d3/f52 [0,4194304] 0 2024-05-06T12:14:57.015 INFO:tasks.workunit.client.1.smithi190.stdout:4/672: dread d2/d10/f58 [0,4194304] 0 2024-05-06T12:14:57.015 INFO:tasks.workunit.client.1.smithi190.stdout:8/855: write d3/f98 [399516,77846] 0 2024-05-06T12:14:57.015 INFO:tasks.workunit.client.1.smithi190.stdout:3/918: chown d5/d79/cc2 15685 1 2024-05-06T12:14:57.017 INFO:tasks.workunit.client.1.smithi190.stdout:5/853: rename d2/da/d59/d5f/d2e/d3b/d5a/fc7 to d2/f111 0 2024-05-06T12:14:57.020 INFO:tasks.workunit.client.1.smithi190.stdout:1/684: write d5/d4f/d65/d24/d3a/f3e [6633814,51205] 0 2024-05-06T12:14:57.022 INFO:tasks.workunit.client.1.smithi190.stdout:8/856: dread d3/d13/f21 [0,4194304] 0 2024-05-06T12:14:57.022 INFO:tasks.workunit.client.1.smithi190.stdout:0/841: write d9/d91/f10f [951770,117362] 0 2024-05-06T12:14:57.029 INFO:tasks.workunit.client.1.smithi190.stdout:5/854: mknod d2/da/d59/d5f/d2e/da7/d9d/c112 0 2024-05-06T12:14:57.030 INFO:tasks.workunit.client.1.smithi190.stdout:0/842: write d9/fda [2889103,57744] 0 2024-05-06T12:14:57.030 INFO:tasks.workunit.client.1.smithi190.stdout:8/857: readlink d3/d8/d25/d3b/d111/l114 0 2024-05-06T12:14:57.034 INFO:tasks.workunit.client.1.smithi190.stdout:1/685: symlink d5/d4f/d65/d32/d47/d53/de8/lee 0 2024-05-06T12:14:57.035 INFO:tasks.workunit.client.1.smithi190.stdout:0/843: dread - d9/dcf/d56/dab/fbc zero size 2024-05-06T12:14:57.036 INFO:tasks.workunit.client.1.smithi190.stdout:9/834: dwrite db/f5a [0,4194304] 0 2024-05-06T12:14:57.039 INFO:tasks.workunit.client.1.smithi190.stdout:4/673: dwrite d2/d10/ff [0,4194304] 0 2024-05-06T12:14:57.040 INFO:tasks.workunit.client.1.smithi190.stdout:3/919: dwrite d5/d14/d11/d21/f30 [0,4194304] 0 2024-05-06T12:14:57.043 INFO:tasks.workunit.client.1.smithi190.stdout:0/844: dread - d9/dc/d28/d4c/d96/fed zero size 2024-05-06T12:14:57.043 INFO:tasks.workunit.client.1.smithi190.stdout:5/855: dread d2/da/d59/d5f/d6e/daa/fe4 [0,4194304] 0 2024-05-06T12:14:57.049 INFO:tasks.workunit.client.1.smithi190.stdout:9/835: rmdir db/dd/df/dfe/d87/d22/d49/d88/d9f 39 2024-05-06T12:14:57.050 INFO:tasks.workunit.client.1.smithi190.stdout:0/845: readlink d9/dc/d48/d50/d87/l117 0 2024-05-06T12:14:57.053 INFO:tasks.workunit.client.1.smithi190.stdout:8/858: creat d3/f128 x:0 0 0 2024-05-06T12:14:57.054 INFO:tasks.workunit.client.1.smithi190.stdout:5/856: symlink d2/d10e/l113 0 2024-05-06T12:14:57.056 INFO:tasks.workunit.client.1.smithi190.stdout:0/846: readlink d9/dc/d48/d50/lde 0 2024-05-06T12:14:57.058 INFO:tasks.workunit.client.1.smithi190.stdout:3/920: dwrite d5/d14/d11/d21/d44/d80/d82/fd3 [0,4194304] 0 2024-05-06T12:14:57.060 INFO:tasks.workunit.client.1.smithi190.stdout:6/629: dread df/d2c/d28/f9f [0,4194304] 0 2024-05-06T12:14:57.061 INFO:tasks.workunit.client.1.smithi190.stdout:1/686: dwrite d5/d4f/d65/d32/f61 [0,4194304] 0 2024-05-06T12:14:57.061 INFO:tasks.workunit.client.1.smithi190.stdout:0/847: chown d9/dc/d28/f8c 0 1 2024-05-06T12:14:57.061 INFO:tasks.workunit.client.1.smithi190.stdout:8/859: dread d3/d13/f21 [0,4194304] 0 2024-05-06T12:14:57.063 INFO:tasks.workunit.client.1.smithi190.stdout:4/674: rename d2/d11/d16/d61/d66/d6b/l93 to d2/d11/d16/d8e/d80/le5 0 2024-05-06T12:14:57.065 INFO:tasks.workunit.client.1.smithi190.stdout:4/675: fsync d2/d11/d16/d61/d9e/d6e/fc4 0 2024-05-06T12:14:57.067 INFO:tasks.workunit.client.1.smithi190.stdout:3/921: write d5/d14/d11/d21/d44/d92/d9e/da4/d105/fe5 [1242545,119304] 0 2024-05-06T12:14:57.067 INFO:tasks.workunit.client.1.smithi190.stdout:0/848: stat d9/dc/d48/fe1 0 2024-05-06T12:14:57.071 INFO:tasks.workunit.client.1.smithi190.stdout:5/857: creat d2/d65/f114 x:0 0 0 2024-05-06T12:14:57.071 INFO:tasks.workunit.client.1.smithi190.stdout:1/687: creat d5/d4f/d65/d2b/fef x:0 0 0 2024-05-06T12:14:57.074 INFO:tasks.workunit.client.1.smithi190.stdout:8/860: creat d3/d8/d25/d65/f129 x:0 0 0 2024-05-06T12:14:57.075 INFO:tasks.workunit.client.1.smithi190.stdout:3/922: chown d5/d14/d11/d21/d44/d80/d82/d3d/ded/f3b 23454 1 2024-05-06T12:14:57.076 INFO:tasks.workunit.client.1.smithi190.stdout:4/676: creat d2/d11/d16/d61/d9e/d6e/daa/fe6 x:0 0 0 2024-05-06T12:14:57.077 INFO:tasks.workunit.client.1.smithi190.stdout:8/861: dread - d3/d35/d3a/fba zero size 2024-05-06T12:14:57.078 INFO:tasks.workunit.client.1.smithi190.stdout:9/836: getdents db/dd/df/dfe/ddc 0 2024-05-06T12:14:57.082 INFO:tasks.workunit.client.1.smithi190.stdout:6/630: dwrite df/f99 [0,4194304] 0 2024-05-06T12:14:57.086 INFO:tasks.workunit.client.1.smithi190.stdout:3/923: truncate d5/d14/d11/d21/d44/d80/ddb/f130 640589 0 2024-05-06T12:14:57.088 INFO:tasks.workunit.client.1.smithi190.stdout:0/849: dread d9/d51/dc9/fa2 [0,4194304] 0 2024-05-06T12:14:57.088 INFO:tasks.workunit.client.1.smithi190.stdout:6/631: read df/d2c/d61/d84/f8e [641033,18430] 0 2024-05-06T12:14:57.089 INFO:tasks.workunit.client.1.smithi190.stdout:4/677: creat d2/d11/d16/fe7 x:0 0 0 2024-05-06T12:14:57.094 INFO:tasks.workunit.client.1.smithi190.stdout:0/850: write d9/dcf/f2e [3653000,128566] 0 2024-05-06T12:14:57.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:56 smithi190 ceph-mon[33798]: from='client.14692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:14:57.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:56 smithi190 ceph-mon[33798]: from='client.14696 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:14:57.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:56 smithi190 ceph-mon[33798]: pgmap v30: 65 pgs: 65 active+clean; 3.5 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 96 MiB/s rd, 221 MiB/s wr, 506 op/s 2024-05-06T12:14:57.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:56 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:57.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:56 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:57.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:56 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/2480165038' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:14:57.100 INFO:tasks.workunit.client.1.smithi190.stdout:3/924: symlink d5/d14/d11/d21/d44/d92/d9e/da4/d105/db3/l140 0 2024-05-06T12:14:57.102 INFO:tasks.workunit.client.1.smithi190.stdout:0/851: unlink d9/dc/f45 0 2024-05-06T12:14:57.103 INFO:tasks.workunit.client.1.smithi190.stdout:3/925: write d5/d7/d15/f28 [5719977,93218] 0 2024-05-06T12:14:57.105 INFO:tasks.workunit.client.1.smithi190.stdout:8/862: truncate d3/d13/fc2 1256085 0 2024-05-06T12:14:57.105 INFO:tasks.workunit.client.1.smithi190.stdout:1/688: dwrite d5/d4f/d65/d2b/f35 [0,4194304] 0 2024-05-06T12:14:57.109 INFO:tasks.workunit.client.1.smithi190.stdout:9/837: truncate db/dd/df/dfe/d87/d22/d49/d88/f7a 4281115 0 2024-05-06T12:14:57.110 INFO:tasks.workunit.client.1.smithi190.stdout:8/863: fsync d3/d8/d25/d65/fb5 0 2024-05-06T12:14:57.115 INFO:tasks.workunit.client.1.smithi190.stdout:1/689: dwrite d5/d15/d1e/fd6 [0,4194304] 0 2024-05-06T12:14:57.115 INFO:tasks.workunit.client.1.smithi190.stdout:8/864: readlink d3/l53 0 2024-05-06T12:14:57.118 INFO:tasks.workunit.client.1.smithi190.stdout:4/678: dread d2/d11/d16/d8e/d80/f97 [0,4194304] 0 2024-05-06T12:14:57.124 INFO:tasks.workunit.client.1.smithi190.stdout:9/838: write db/dd/df/dfe/d87/d22/d49/d88/d9f/df2/f102 [1467577,63627] 0 2024-05-06T12:14:57.128 INFO:tasks.workunit.client.1.smithi190.stdout:0/852: rename d9/dcf/d21/d31 to d9/dc/d28/d12e 0 2024-05-06T12:14:57.128 INFO:tasks.workunit.client.1.smithi190.stdout:0/853: chown d9/d91/f10f 51764590 1 2024-05-06T12:14:57.129 INFO:tasks.workunit.client.1.smithi190.stdout:4/679: stat d2/d11/d16/d61/d9e/d6e/d78/c87 0 2024-05-06T12:14:57.135 INFO:tasks.workunit.client.1.smithi190.stdout:1/690: dwrite d5/d4f/d65/d24/d3a/f9c [0,4194304] 0 2024-05-06T12:14:57.135 INFO:tasks.workunit.client.1.smithi190.stdout:4/680: chown d2/d11/d16/d41/d81/lc9 301697078 1 2024-05-06T12:14:57.139 INFO:tasks.workunit.client.1.smithi190.stdout:3/926: getdents d5/d14/d11/d21/deb 0 2024-05-06T12:14:57.140 INFO:tasks.workunit.client.1.smithi190.stdout:9/839: mkdir db/dd/df/dfe/ddc/d47/d10c/d11f 0 2024-05-06T12:14:57.148 INFO:tasks.workunit.client.1.smithi190.stdout:9/840: dread db/f77 [0,4194304] 0 2024-05-06T12:14:57.149 INFO:tasks.workunit.client.1.smithi190.stdout:4/681: rename d2/d11/d16/d61 to d2/d11/d16/d8e/d7b/de8 0 2024-05-06T12:14:57.151 INFO:tasks.workunit.client.1.smithi190.stdout:8/865: fdatasync d3/d8/d25/d3b/d6b/d7a/fe3 0 2024-05-06T12:14:57.154 INFO:tasks.workunit.client.1.smithi190.stdout:0/854: link d9/dc/d28/d4c/d80/da7/l122 d9/dc/db6/l12f 0 2024-05-06T12:14:57.156 INFO:tasks.workunit.client.1.smithi190.stdout:9/841: creat db/dd/d5c/f120 x:0 0 0 2024-05-06T12:14:57.158 INFO:tasks.workunit.client.1.smithi190.stdout:4/682: creat d2/d11/d16/d8e/d7b/de8/d9e/fe9 x:0 0 0 2024-05-06T12:14:57.159 INFO:tasks.workunit.client.1.smithi190.stdout:4/683: chown d2/d11/d16/d8e/d7b/l91 6 1 2024-05-06T12:14:57.162 INFO:tasks.workunit.client.1.smithi190.stdout:3/927: creat d5/d14/d11/d21/d44/d80/d82/dfc/f141 x:0 0 0 2024-05-06T12:14:57.164 INFO:tasks.workunit.client.1.smithi190.stdout:1/691: creat d5/d4f/d65/ff0 x:0 0 0 2024-05-06T12:14:57.165 INFO:tasks.workunit.client.1.smithi190.stdout:1/692: write d5/d6b/d8f/fcf [408609,118132] 0 2024-05-06T12:14:57.167 INFO:tasks.workunit.client.1.smithi190.stdout:3/928: symlink d5/d7/d106/l142 0 2024-05-06T12:14:57.168 INFO:tasks.workunit.client.1.smithi190.stdout:3/929: stat d5/d7/f3a 0 2024-05-06T12:14:57.168 INFO:tasks.workunit.client.1.smithi190.stdout:0/855: symlink d9/dc/d28/l130 0 2024-05-06T12:14:57.171 INFO:tasks.workunit.client.1.smithi190.stdout:8/866: creat d3/d8/d25/f12a x:0 0 0 2024-05-06T12:14:57.173 INFO:tasks.workunit.client.1.smithi190.stdout:8/867: readlink d3/d8/d25/d3b/d90/lb6 0 2024-05-06T12:14:57.175 INFO:tasks.workunit.client.1.smithi190.stdout:3/930: unlink d5/d14/d11/d21/d44/d92/d9e/da4/cba 0 2024-05-06T12:14:57.182 INFO:tasks.workunit.client.1.smithi190.stdout:0/856: link d9/dc/d28/d12e/f34 d9/d51/dc9/f131 0 2024-05-06T12:14:57.183 INFO:tasks.workunit.client.1.smithi190.stdout:1/693: getdents d5/d15/d1e/d94 0 2024-05-06T12:14:57.184 INFO:tasks.workunit.client.1.smithi190.stdout:3/931: dwrite d5/d14/d11/d21/d44/d80/d82/d3d/ff4 [0,4194304] 0 2024-05-06T12:14:57.186 INFO:tasks.workunit.client.1.smithi190.stdout:3/932: fsync d5/d14/d11/d21/d44/d92/d9e/da4/d105/db3/f7d 0 2024-05-06T12:14:57.187 INFO:tasks.workunit.client.1.smithi190.stdout:9/842: truncate db/dd/df/dfe/d87/d22/d49/d88/d9f/f11b 842511 0 2024-05-06T12:14:57.188 INFO:tasks.workunit.client.1.smithi190.stdout:0/857: mknod d9/d51/d93/c132 0 2024-05-06T12:14:57.195 INFO:tasks.workunit.client.1.smithi190.stdout:8/868: write d3/d8/d25/f66 [527844,124090] 0 2024-05-06T12:14:57.196 INFO:tasks.workunit.client.1.smithi190.stdout:9/843: dread db/dd/df/dfe/d87/d22/d49/d8c/fcc [0,4194304] 0 2024-05-06T12:14:57.196 INFO:tasks.workunit.client.1.smithi190.stdout:1/694: mknod d5/d4f/d65/d32/d47/de5/cf1 0 2024-05-06T12:14:57.198 INFO:tasks.workunit.client.1.smithi190.stdout:1/695: chown d5/d4f/d65/d32/c74 452598 1 2024-05-06T12:14:57.199 INFO:tasks.workunit.client.1.smithi190.stdout:8/869: truncate d3/d8/d25/d3b/d6b/ffc 125523 0 2024-05-06T12:14:57.201 INFO:tasks.workunit.client.1.smithi190.stdout:1/696: symlink d5/d4f/d65/d62/d9d/dba/lf2 0 2024-05-06T12:14:57.202 INFO:tasks.workunit.client.1.smithi190.stdout:9/844: read db/daa/fab [266644,101675] 0 2024-05-06T12:14:57.204 INFO:tasks.workunit.client.1.smithi190.stdout:9/845: stat db/dd/d16/d4c/d4e/f9d 0 2024-05-06T12:14:57.206 INFO:tasks.workunit.client.1.smithi190.stdout:1/697: mknod d5/d4f/d65/d24/dd4/cf3 0 2024-05-06T12:14:57.210 INFO:tasks.workunit.client.1.smithi190.stdout:8/870: rmdir d3/d35/d3a/db9/d11a 0 2024-05-06T12:14:57.215 INFO:tasks.workunit.client.1.smithi190.stdout:3/933: dwrite d5/d14/d11/d21/d44/d80/d82/d3d/ff4 [4194304,4194304] 0 2024-05-06T12:14:57.217 INFO:tasks.workunit.client.1.smithi190.stdout:8/871: dread - d3/d35/d3a/db9/fd0 zero size 2024-05-06T12:14:57.218 INFO:tasks.workunit.client.1.smithi190.stdout:8/872: write d3/f52 [1293094,6408] 0 2024-05-06T12:14:57.222 INFO:tasks.workunit.client.1.smithi190.stdout:9/846: write db/f13 [2481967,29714] 0 2024-05-06T12:14:57.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:56 smithi092 ceph-mon[26954]: from='client.14692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:14:57.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:56 smithi092 ceph-mon[26954]: from='client.14696 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:14:57.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:56 smithi092 ceph-mon[26954]: pgmap v30: 65 pgs: 65 active+clean; 3.5 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 96 MiB/s rd, 221 MiB/s wr, 506 op/s 2024-05-06T12:14:57.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:56 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:57.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:56 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:14:57.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:56 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/2480165038' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:14:57.225 INFO:tasks.workunit.client.1.smithi190.stdout:1/698: dread d5/d4f/d65/d24/dd4/f79 [0,4194304] 0 2024-05-06T12:14:57.226 INFO:tasks.workunit.client.1.smithi190.stdout:1/699: fsync d5/d4f/d65/d24/d3a/f9c 0 2024-05-06T12:14:57.227 INFO:tasks.workunit.client.1.smithi190.stdout:9/847: creat db/dd/d5c/dac/deb/f121 x:0 0 0 2024-05-06T12:14:57.228 INFO:tasks.workunit.client.1.smithi190.stdout:8/873: mknod d3/d8/d2b/d122/c12b 0 2024-05-06T12:14:57.228 INFO:tasks.workunit.client.1.smithi190.stdout:9/848: stat db/dd/d16 0 2024-05-06T12:14:57.229 INFO:tasks.workunit.client.1.smithi190.stdout:8/874: chown d3/d13/f7d 3027 1 2024-05-06T12:14:57.232 INFO:tasks.workunit.client.1.smithi190.stdout:8/875: symlink d3/d8/d25/d3b/d90/l12c 0 2024-05-06T12:14:57.234 INFO:tasks.workunit.client.1.smithi190.stdout:1/700: creat d5/d4f/d65/d32/d47/d53/d98/da7/ff4 x:0 0 0 2024-05-06T12:14:57.236 INFO:tasks.workunit.client.1.smithi190.stdout:9/849: creat db/dd/df/dfe/d87/d22/d49/d88/d9f/f122 x:0 0 0 2024-05-06T12:14:57.236 INFO:tasks.workunit.client.1.smithi190.stdout:8/876: read - d3/d8/d25/d3b/d90/d6e/f94 zero size 2024-05-06T12:14:57.238 INFO:tasks.workunit.client.1.smithi190.stdout:1/701: mkdir d5/d4f/d65/d32/d47/d53/df5 0 2024-05-06T12:14:57.239 INFO:tasks.workunit.client.1.smithi190.stdout:3/934: dwrite d5/d7/f90 [0,4194304] 0 2024-05-06T12:14:57.246 INFO:tasks.workunit.client.1.smithi190.stdout:1/702: chown d5/d4f/d65/d32/d47/f96 3060 1 2024-05-06T12:14:57.246 INFO:tasks.workunit.client.1.smithi190.stdout:8/877: rename d3/ff3 to d3/d35/f12d 0 2024-05-06T12:14:57.248 INFO:tasks.workunit.client.1.smithi190.stdout:1/703: mknod d5/d6b/d8f/cf6 0 2024-05-06T12:14:57.249 INFO:tasks.workunit.client.1.smithi190.stdout:8/878: truncate d3/d8/d2b/d3e/dad/fbe 4281614 0 2024-05-06T12:14:57.250 INFO:tasks.workunit.client.1.smithi190.stdout:1/704: rmdir d5/d4f/d65/d62/d9d/dba 39 2024-05-06T12:14:57.253 INFO:tasks.workunit.client.1.smithi190.stdout:8/879: write d3/d35/d55/d61/df9/f11f [644542,128469] 0 2024-05-06T12:14:57.255 INFO:tasks.workunit.client.1.smithi190.stdout:1/705: rename d5/d4f/d65/d32/d47/d53/fe0 to d5/d15/d1e/d94/ff7 0 2024-05-06T12:14:57.257 INFO:tasks.workunit.client.1.smithi190.stdout:8/880: fdatasync d3/d8/d25/f7b 0 2024-05-06T12:14:57.263 INFO:tasks.workunit.client.1.smithi190.stdout:8/881: link d3/d8/fd d3/d35/d55/d61/d11e/f12e 0 2024-05-06T12:14:57.265 INFO:tasks.workunit.client.1.smithi190.stdout:1/706: dwrite d5/d4f/d65/d24/dd4/fb5 [0,4194304] 0 2024-05-06T12:14:57.267 INFO:tasks.workunit.client.1.smithi190.stdout:3/935: dwrite d5/d14/d11/d21/f13f [0,4194304] 0 2024-05-06T12:14:57.267 INFO:tasks.workunit.client.1.smithi190.stdout:9/850: dread db/f3d [0,4194304] 0 2024-05-06T12:14:57.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.265+0000 7f6fa2702700 1 -- 172.21.15.92:0/113730957 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6f7c005860 con 0x7f6f9c11cd50 2024-05-06T12:14:57.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.266+0000 7f6f98ff9700 1 -- 172.21.15.92:0/113730957 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1850 (secure 0 0 0) 0x7f6f8c058360 con 0x7f6f9c11cd50 2024-05-06T12:14:57.269 INFO:tasks.workunit.client.1.smithi190.stdout:8/882: mknod d3/d8/d68/d9f/c12f 0 2024-05-06T12:14:57.269 INFO:teuthology.orchestra.run.smithi092.stdout:e19 2024-05-06T12:14:57.269 INFO:teuthology.orchestra.run.smithi092.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-06T12:14:57.270 INFO:teuthology.orchestra.run.smithi092.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:14:57.270 INFO:teuthology.orchestra.run.smithi092.stdout:legacy client fscid: 1 2024-05-06T12:14:57.270 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:14:57.270 INFO:teuthology.orchestra.run.smithi092.stdout:Filesystem 'cephfs' (1) 2024-05-06T12:14:57.270 INFO:teuthology.orchestra.run.smithi092.stdout:fs_name cephfs 2024-05-06T12:14:57.270 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 19 2024-05-06T12:14:57.270 INFO:teuthology.orchestra.run.smithi092.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-06T12:14:57.270 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-05-06T12:11:44.952671+0000 2024-05-06T12:14:57.270 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2024-05-06T12:13:52.244842+0000 2024-05-06T12:14:57.270 INFO:teuthology.orchestra.run.smithi092.stdout:tableserver 0 2024-05-06T12:14:57.270 INFO:teuthology.orchestra.run.smithi092.stdout:root 0 2024-05-06T12:14:57.270 INFO:teuthology.orchestra.run.smithi092.stdout:session_timeout 60 2024-05-06T12:14:57.270 INFO:teuthology.orchestra.run.smithi092.stdout:session_autoclose 300 2024-05-06T12:14:57.271 INFO:teuthology.orchestra.run.smithi092.stdout:max_file_size 1099511627776 2024-05-06T12:14:57.271 INFO:teuthology.orchestra.run.smithi092.stdout:required_client_features {} 2024-05-06T12:14:57.271 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure 0 2024-05-06T12:14:57.271 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure_osd_epoch 46 2024-05-06T12:14:57.271 INFO:teuthology.orchestra.run.smithi092.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:14:57.271 INFO:teuthology.orchestra.run.smithi092.stdout:max_mds 2 2024-05-06T12:14:57.271 INFO:teuthology.orchestra.run.smithi092.stdout:in 0,1 2024-05-06T12:14:57.271 INFO:teuthology.orchestra.run.smithi092.stdout:up {0=14472,1=24267} 2024-05-06T12:14:57.271 INFO:teuthology.orchestra.run.smithi092.stdout:failed 2024-05-06T12:14:57.271 INFO:teuthology.orchestra.run.smithi092.stdout:damaged 2024-05-06T12:14:57.271 INFO:teuthology.orchestra.run.smithi092.stdout:stopped 2024-05-06T12:14:57.271 INFO:teuthology.orchestra.run.smithi092.stdout:data_pools [3] 2024-05-06T12:14:57.271 INFO:teuthology.orchestra.run.smithi092.stdout:metadata_pool 2 2024-05-06T12:14:57.271 INFO:teuthology.orchestra.run.smithi092.stdout:inline_data enabled 2024-05-06T12:14:57.272 INFO:teuthology.orchestra.run.smithi092.stdout:balancer 2024-05-06T12:14:57.272 INFO:teuthology.orchestra.run.smithi092.stdout:standby_count_wanted 1 2024-05-06T12:14:57.272 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.rosbll{0:14472} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.92:6828/1663253598,v1:172.21.15.92:6829/1663253598] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:14:57.272 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.ijsqgp{0:14484} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.92:6826/3613441054,v1:172.21.15.92:6827/3613441054] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:14:57.272 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.byjvgf{1:24267} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.190:6824/1565151699,v1:172.21.15.190:6825/1565151699] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:14:57.272 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.idzoff{1:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.190:6826/2453728380,v1:172.21.15.190:6827/2453728380] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:14:57.272 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:14:57.272 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:14:57.272 INFO:tasks.workunit.client.1.smithi190.stdout:1/707: dread - d5/d4f/d65/f5b zero size 2024-05-06T12:14:57.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.270+0000 7f6fa2702700 1 -- 172.21.15.92:0/113730957 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f6f8806b8a0 msgr2=0x7f6f8806dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:57.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.270+0000 7f6fa2702700 1 --2- 172.21.15.92:0/113730957 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f6f8806b8a0 0x7f6f8806dd50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6f84008fd0 tx=0x7f6f8400b040 comp rx=0 tx=0).stop 2024-05-06T12:14:57.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.270+0000 7f6fa2702700 1 -- 172.21.15.92:0/113730957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f9c11cd50 msgr2=0x7f6f9c121d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:57.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.270+0000 7f6fa2702700 1 --2- 172.21.15.92:0/113730957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f9c11cd50 0x7f6f9c121d80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c008000 tx=0x7f6f8c009e30 comp rx=0 tx=0).stop 2024-05-06T12:14:57.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.270+0000 7f6fa2702700 1 -- 172.21.15.92:0/113730957 shutdown_connections 2024-05-06T12:14:57.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.270+0000 7f6fa2702700 1 --2- 172.21.15.92:0/113730957 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f6f8806b8a0 0x7f6f8806dd50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:57.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.270+0000 7f6fa2702700 1 --2- 172.21.15.92:0/113730957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f9c06e440 0x7f6f9c11c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:57.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.270+0000 7f6fa2702700 1 --2- 172.21.15.92:0/113730957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f9c11cd50 0x7f6f9c121d80 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:57.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.270+0000 7f6fa2702700 1 -- 172.21.15.92:0/113730957 >> 172.21.15.92:0/113730957 conn(0x7f6f9c069dc0 msgr2=0x7f6f9c1070b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:14:57.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.271+0000 7f6fa2702700 1 -- 172.21.15.92:0/113730957 shutdown_connections 2024-05-06T12:14:57.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.271+0000 7f6fa2702700 1 -- 172.21.15.92:0/113730957 wait complete. 2024-05-06T12:14:57.275 INFO:tasks.workunit.client.1.smithi190.stdout:9/851: readlink db/dd/df/l10b 0 2024-05-06T12:14:57.276 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 19 2024-05-06T12:14:57.278 INFO:tasks.workunit.client.1.smithi190.stdout:8/883: creat d3/d35/d3a/f130 x:0 0 0 2024-05-06T12:14:57.280 INFO:tasks.workunit.client.1.smithi190.stdout:8/884: dread - d3/d35/d55/ff7 zero size 2024-05-06T12:14:57.281 INFO:tasks.workunit.client.1.smithi190.stdout:9/852: rmdir db/dd/df/dfe/d87/d22/d49/d88/d9f 39 2024-05-06T12:14:57.282 INFO:tasks.workunit.client.1.smithi190.stdout:3/936: truncate d5/d14/d11/d21/d44/d80/d82/f12f 1208348 0 2024-05-06T12:14:57.284 INFO:tasks.workunit.client.1.smithi190.stdout:8/885: creat d3/d8/d25/d3b/d6b/d7a/f131 x:0 0 0 2024-05-06T12:14:57.285 INFO:tasks.workunit.client.1.smithi190.stdout:1/708: creat d5/d4f/d65/ff8 x:0 0 0 2024-05-06T12:14:57.286 INFO:tasks.workunit.client.1.smithi190.stdout:3/937: unlink d5/d14/d11/d21/d44/d80/d82/d3d/f99 0 2024-05-06T12:14:57.287 INFO:tasks.workunit.client.1.smithi190.stdout:8/886: symlink d3/d8/d25/d3b/d90/dcb/dfb/l132 0 2024-05-06T12:14:57.288 INFO:tasks.workunit.client.1.smithi190.stdout:3/938: chown d5/d14/d11/d21/d44/d80/d82/fd3 623066 1 2024-05-06T12:14:57.288 INFO:tasks.workunit.client.1.smithi190.stdout:1/709: mknod d5/d4f/d65/d32/d47/d53/cf9 0 2024-05-06T12:14:57.290 INFO:tasks.workunit.client.1.smithi190.stdout:8/887: symlink d3/d8/l133 0 2024-05-06T12:14:57.294 INFO:tasks.workunit.client.1.smithi190.stdout:8/888: creat d3/d35/d55/d61/f134 x:0 0 0 2024-05-06T12:14:57.295 INFO:tasks.workunit.client.1.smithi190.stdout:1/710: dread d5/d4f/d65/d24/d43/fe3 [0,4194304] 0 2024-05-06T12:14:57.308 INFO:tasks.workunit.client.1.smithi190.stdout:1/711: dwrite d5/d4f/d65/f7b [0,4194304] 0 2024-05-06T12:14:57.309 INFO:tasks.workunit.client.1.smithi190.stdout:3/939: dwrite d5/f59 [0,4194304] 0 2024-05-06T12:14:57.309 INFO:tasks.workunit.client.1.smithi190.stdout:9/853: dwrite db/d23/fdd [0,4194304] 0 2024-05-06T12:14:57.320 INFO:tasks.workunit.client.1.smithi190.stdout:3/940: symlink d5/d14/d11/d21/deb/d109/l143 0 2024-05-06T12:14:57.320 INFO:tasks.workunit.client.1.smithi190.stdout:9/854: mknod db/dd/d5c/d111/c123 0 2024-05-06T12:14:57.325 INFO:tasks.workunit.client.1.smithi190.stdout:9/855: truncate db/d23/da4/ff7 675015 0 2024-05-06T12:14:57.326 INFO:tasks.workunit.client.1.smithi190.stdout:1/712: dwrite d5/d4f/d65/d32/d47/d53/d98/da7/de7/f9b [0,4194304] 0 2024-05-06T12:14:57.328 INFO:tasks.workunit.client.1.smithi190.stdout:9/856: write db/dd/df/dfe/d87/d99/fc9 [967342,70678] 0 2024-05-06T12:14:57.332 INFO:tasks.workunit.client.1.smithi190.stdout:9/857: mkdir db/dd/df/d124 0 2024-05-06T12:14:57.333 INFO:tasks.workunit.client.1.smithi190.stdout:9/858: chown db/dd/df/dfe/d87/d22/c101 15441 1 2024-05-06T12:14:57.354 INFO:tasks.workunit.client.1.smithi190.stdout:3/941: dwrite d5/d14/d11/d21/d44/fbf [0,4194304] 0 2024-05-06T12:14:57.356 INFO:tasks.workunit.client.1.smithi190.stdout:1/713: dread d5/f34 [0,4194304] 0 2024-05-06T12:14:57.361 INFO:tasks.workunit.client.1.smithi190.stdout:1/714: fdatasync d5/d4f/d65/d62/d9d/dba/fd0 0 2024-05-06T12:14:57.362 INFO:tasks.workunit.client.1.smithi190.stdout:3/942: dwrite d5/d7/d15/f28 [0,4194304] 0 2024-05-06T12:14:57.365 INFO:tasks.workunit.client.1.smithi190.stdout:1/715: creat d5/d6b/ffa x:0 0 0 2024-05-06T12:14:57.367 INFO:tasks.workunit.client.1.smithi190.stdout:1/716: dread - d5/d4f/d65/d62/d9d/fdb zero size 2024-05-06T12:14:57.369 INFO:tasks.workunit.client.1.smithi190.stdout:3/943: mknod d5/c144 0 2024-05-06T12:14:57.370 INFO:tasks.workunit.client.1.smithi190.stdout:3/944: readlink d5/d14/d11/d21/d44/d92/d9e/da4/d105/lad 0 2024-05-06T12:14:57.370 INFO:tasks.workunit.client.1.smithi190.stdout:1/717: write d5/d15/fa2 [135046,94243] 0 2024-05-06T12:14:57.374 INFO:tasks.workunit.client.1.smithi190.stdout:3/945: dread - d5/d14/d11/d21/d44/d80/d82/d3d/fdd zero size 2024-05-06T12:14:57.385 INFO:tasks.workunit.client.1.smithi190.stdout:3/946: dwrite d5/d14/d11/d21/d44/d80/d82/f8b [0,4194304] 0 2024-05-06T12:14:57.388 INFO:tasks.workunit.client.1.smithi190.stdout:3/947: fdatasync d5/d14/d11/d21/d44/d92/d9e/da4/d105/fe5 0 2024-05-06T12:14:57.393 INFO:tasks.workunit.client.1.smithi190.stdout:1/718: dread d5/d15/f3f [0,4194304] 0 2024-05-06T12:14:57.396 INFO:tasks.workunit.client.1.smithi190.stdout:3/948: dwrite d5/d14/d11/d21/d44/d80/d82/d3d/ded/d33/d5c/f129 [0,4194304] 0 2024-05-06T12:14:57.398 INFO:tasks.workunit.client.1.smithi190.stdout:1/719: creat d5/d4f/d65/d24/d43/ffb x:0 0 0 2024-05-06T12:14:57.400 INFO:tasks.workunit.client.1.smithi190.stdout:1/720: creat d5/d4f/d65/d24/d43/ffc x:0 0 0 2024-05-06T12:14:57.401 INFO:tasks.workunit.client.1.smithi190.stdout:1/721: read d5/d4f/d65/d32/f61 [3236864,127540] 0 2024-05-06T12:14:57.403 INFO:tasks.workunit.client.1.smithi190.stdout:1/722: dread - d5/d6b/ffa zero size 2024-05-06T12:14:57.404 INFO:tasks.workunit.client.1.smithi190.stdout:1/723: mkdir d5/d4f/d65/d2b/dfd 0 2024-05-06T12:14:57.405 INFO:tasks.workunit.client.1.smithi190.stdout:1/724: chown d5/d4f/d65/d32/dac/fcb 18140 1 2024-05-06T12:14:57.406 INFO:tasks.workunit.client.1.smithi190.stdout:1/725: fsync d5/d4f/d65/d62/d9d/dba/fd0 0 2024-05-06T12:14:57.409 INFO:tasks.workunit.client.1.smithi190.stdout:1/726: creat d5/d15/d1e/ffe x:0 0 0 2024-05-06T12:14:57.413 INFO:tasks.workunit.client.1.smithi190.stdout:1/727: rename d5/d4f/d65/d2b/dfd to d5/d15/d1e/dff 0 2024-05-06T12:14:57.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.415+0000 7f51d41a4700 1 -- 172.21.15.92:0/3113736923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51cc0fff70 msgr2=0x7f51cc102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:57.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.415+0000 7f51d41a4700 1 --2- 172.21.15.92:0/3113736923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51cc0fff70 0x7f51cc102350 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f51bc004880 tx=0x7f51bc009e30 comp rx=0 tx=0).stop 2024-05-06T12:14:57.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.415+0000 7f51d41a4700 1 -- 172.21.15.92:0/3113736923 shutdown_connections 2024-05-06T12:14:57.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.415+0000 7f51d41a4700 1 --2- 172.21.15.92:0/3113736923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51cc0fff70 0x7f51cc102350 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:57.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.415+0000 7f51d41a4700 1 --2- 172.21.15.92:0/3113736923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51cc066af0 0x7f51cc0ffa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:57.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.415+0000 7f51d41a4700 1 -- 172.21.15.92:0/3113736923 >> 172.21.15.92:0/3113736923 conn(0x7f51cc0f7440 msgr2=0x7f51cc0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:14:57.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.416+0000 7f51d41a4700 1 -- 172.21.15.92:0/3113736923 shutdown_connections 2024-05-06T12:14:57.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.416+0000 7f51d41a4700 1 -- 172.21.15.92:0/3113736923 wait complete. 2024-05-06T12:14:57.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.416+0000 7f51d41a4700 1 Processor -- start 2024-05-06T12:14:57.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.416+0000 7f51d41a4700 1 -- start start 2024-05-06T12:14:57.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.416+0000 7f51d41a4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51cc066af0 0x7f51cc10da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:57.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.416+0000 7f51d41a4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51cc0fff70 0x7f51cc10dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:57.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.416+0000 7f51d41a4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51cc10e5c0 con 0x7f51cc0fff70 2024-05-06T12:14:57.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.416+0000 7f51d41a4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51cc10edc0 con 0x7f51cc066af0 2024-05-06T12:14:57.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.417+0000 7f51d173f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51cc0fff70 0x7f51cc10dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:57.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.417+0000 7f51d173f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51cc0fff70 0x7f51cc10dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36192/0 (socket says 172.21.15.92:36192) 2024-05-06T12:14:57.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.417+0000 7f51d173f700 1 -- 172.21.15.92:0/1861272406 learned_addr learned my addr 172.21.15.92:0/1861272406 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:14:57.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.417+0000 7f51d1f40700 1 --2- 172.21.15.92:0/1861272406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51cc066af0 0x7f51cc10da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:57.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.417+0000 7f51d1f40700 1 -- 172.21.15.92:0/1861272406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51cc0fff70 msgr2=0x7f51cc10dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:57.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.417+0000 7f51d1f40700 1 --2- 172.21.15.92:0/1861272406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51cc0fff70 0x7f51cc10dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:57.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.417+0000 7f51d1f40700 1 -- 172.21.15.92:0/1861272406 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51bc005040 con 0x7f51cc066af0 2024-05-06T12:14:57.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.417+0000 7f51d173f700 1 --2- 172.21.15.92:0/1861272406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51cc0fff70 0x7f51cc10dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-06T12:14:57.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.417+0000 7f51d1f40700 1 --2- 172.21.15.92:0/1861272406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51cc066af0 0x7f51cc10da80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f51cc066830 tx=0x7f51c800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:14:57.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.418+0000 7f51c27fc700 1 -- 172.21.15.92:0/1861272406 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51c8007cc0 con 0x7f51cc066af0 2024-05-06T12:14:57.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.418+0000 7f51c27fc700 1 -- 172.21.15.92:0/1861272406 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f51c800f040 con 0x7f51cc066af0 2024-05-06T12:14:57.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.418+0000 7f51c27fc700 1 -- 172.21.15.92:0/1861272406 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51c800c960 con 0x7f51cc066af0 2024-05-06T12:14:57.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.418+0000 7f51d41a4700 1 -- 172.21.15.92:0/1861272406 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51cc10f040 con 0x7f51cc066af0 2024-05-06T12:14:57.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.418+0000 7f51d41a4700 1 -- 172.21.15.92:0/1861272406 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51cc10f580 con 0x7f51cc066af0 2024-05-06T12:14:57.424 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.419+0000 7f51d41a4700 1 -- 172.21.15.92:0/1861272406 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51cc05f5c0 con 0x7f51cc066af0 2024-05-06T12:14:57.424 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.420+0000 7f51c27fc700 1 -- 172.21.15.92:0/1861272406 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7f51c800cac0 con 0x7f51cc066af0 2024-05-06T12:14:57.424 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.420+0000 7f51c27fc700 1 --2- 172.21.15.92:0/1861272406 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f51b806b790 0x7f51b806dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:57.424 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.420+0000 7f51c27fc700 1 -- 172.21.15.92:0/1861272406 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f51c8089d40 con 0x7f51cc066af0 2024-05-06T12:14:57.424 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.420+0000 7f51d173f700 1 --2- 172.21.15.92:0/1861272406 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f51b806b790 0x7f51b806dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:57.424 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.421+0000 7f51d173f700 1 --2- 172.21.15.92:0/1861272406 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f51b806b790 0x7f51b806dc40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f51bc000c00 tx=0x7f51bc009e30 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:14:57.425 INFO:tasks.workunit.client.1.smithi190.stdout:1/728: truncate d5/d4f/d65/d32/d47/f4c 1518158 0 2024-05-06T12:14:57.426 INFO:tasks.workunit.client.1.smithi190.stdout:1/729: symlink d5/d15/l100 0 2024-05-06T12:14:57.428 INFO:tasks.workunit.client.1.smithi190.stdout:1/730: read d5/d4f/d65/d32/d47/d53/d98/da7/fc7 [32795,119777] 0 2024-05-06T12:14:57.428 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.425+0000 7f51c27fc700 1 -- 172.21.15.92:0/1861272406 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f51c8050100 con 0x7f51cc066af0 2024-05-06T12:14:57.430 INFO:tasks.workunit.client.1.smithi190.stdout:1/731: fsync d5/d4f/d65/d32/d47/d53/d98/da7/fc6 0 2024-05-06T12:14:57.431 INFO:tasks.workunit.client.1.smithi190.stdout:1/732: mkdir d5/d4f/d65/d24/dd4/d101 0 2024-05-06T12:14:57.433 INFO:tasks.workunit.client.1.smithi190.stdout:1/733: mknod d5/d4f/d65/d24/d43/da6/c102 0 2024-05-06T12:14:57.433 INFO:tasks.workunit.client.1.smithi190.stdout:1/734: readlink d5/d4f/d65/l5c 0 2024-05-06T12:14:57.433 INFO:tasks.workunit.client.1.smithi190.stdout:1/735: chown d5/d4f/d65/d32/f61 46063356 1 2024-05-06T12:14:57.436 INFO:tasks.workunit.client.1.smithi190.stdout:1/736: creat d5/d4f/d65/d62/d9d/f103 x:0 0 0 2024-05-06T12:14:57.437 INFO:tasks.workunit.client.1.smithi190.stdout:1/737: write d5/d4f/d65/d24/f7e [1518040,112471] 0 2024-05-06T12:14:57.440 INFO:tasks.workunit.client.1.smithi190.stdout:1/738: rename d5/d6b/d8f/caf to d5/d4f/d65/d24/d43/da6/c104 0 2024-05-06T12:14:57.442 INFO:tasks.workunit.client.1.smithi190.stdout:1/739: mkdir d5/d15/d105 0 2024-05-06T12:14:57.443 INFO:tasks.workunit.client.1.smithi190.stdout:1/740: fdatasync d5/d4f/d65/d32/d47/d53/d98/da7/de7/fda 0 2024-05-06T12:14:57.444 INFO:tasks.workunit.client.1.smithi190.stdout:1/741: chown d5/d15/l1b 2050041 1 2024-05-06T12:14:57.454 INFO:tasks.workunit.client.1.smithi190.stdout:1/742: truncate d5/d4f/d65/d24/f45 2610398 0 2024-05-06T12:14:57.455 INFO:tasks.workunit.client.1.smithi190.stdout:1/743: write d5/d4f/d65/d32/d47/f57 [1143555,64060] 0 2024-05-06T12:14:57.560 INFO:tasks.workunit.client.1.smithi190.stdout:7/814: sync 2024-05-06T12:14:57.560 INFO:tasks.workunit.client.1.smithi190.stdout:7/815: truncate d0/d4/f39 4523480 0 2024-05-06T12:14:57.565 INFO:tasks.workunit.client.1.smithi190.stdout:7/816: dwrite d0/d1/d32/daa/fcb [0,4194304] 0 2024-05-06T12:14:57.572 INFO:tasks.workunit.client.1.smithi190.stdout:7/817: dread d0/da/d36/d53/fb5 [0,4194304] 0 2024-05-06T12:14:57.580 INFO:tasks.workunit.client.1.smithi190.stdout:7/818: truncate d0/d1/d18/d2b/f30 3894161 0 2024-05-06T12:14:57.581 INFO:tasks.workunit.client.1.smithi190.stdout:7/819: chown d0/da/d36/d53/d77 203517 1 2024-05-06T12:14:57.584 INFO:tasks.workunit.client.1.smithi190.stdout:7/820: rename d0/d1/d32/f38 to d0/da/d36/d53/d77/d8c/dd1/d60/f11b 0 2024-05-06T12:14:57.588 INFO:tasks.workunit.client.1.smithi190.stdout:7/821: link d0/d1/d2/d84/dc6/fff d0/da/d36/d53/d77/f11c 0 2024-05-06T12:14:57.588 INFO:tasks.workunit.client.1.smithi190.stdout:7/822: chown d0/d1/d32/d45/d70/d33/df7 976773438 1 2024-05-06T12:14:57.599 INFO:tasks.workunit.client.1.smithi190.stdout:7/823: dread d0/d1/f3e [0,4194304] 0 2024-05-06T12:14:57.599 INFO:tasks.workunit.client.1.smithi190.stdout:7/824: readlink d0/d1/d18/d6c/de1/de7/led 0 2024-05-06T12:14:57.600 INFO:tasks.workunit.client.1.smithi190.stdout:7/825: write d0/d1/d18/d4e/d87/f8b [606293,24425] 0 2024-05-06T12:14:57.602 INFO:tasks.workunit.client.1.smithi190.stdout:7/826: read d0/d1/d18/d2b/f80 [1042055,116934] 0 2024-05-06T12:14:57.604 INFO:tasks.workunit.client.1.smithi190.stdout:7/827: creat d0/d14/d6a/d8f/f11d x:0 0 0 2024-05-06T12:14:57.611 INFO:tasks.workunit.client.1.smithi190.stdout:7/828: write d0/d1/d2/d84/dc6/fff [1743928,29334] 0 2024-05-06T12:14:57.613 INFO:tasks.workunit.client.1.smithi190.stdout:7/829: mkdir d0/d1/d2/d11e 0 2024-05-06T12:14:57.615 INFO:tasks.workunit.client.1.smithi190.stdout:7/830: mkdir d0/d1/d2/d84/dc6/d10c/d11f 0 2024-05-06T12:14:57.616 INFO:tasks.workunit.client.1.smithi190.stdout:7/831: creat d0/d4/f120 x:0 0 0 2024-05-06T12:14:57.618 INFO:tasks.workunit.client.1.smithi190.stdout:7/832: mknod d0/db4/c121 0 2024-05-06T12:14:57.623 INFO:tasks.workunit.client.1.smithi190.stdout:7/833: rmdir d0/d4 39 2024-05-06T12:14:57.624 INFO:tasks.workunit.client.1.smithi190.stdout:7/834: stat d0/d4/f6b 0 2024-05-06T12:14:57.629 INFO:tasks.workunit.client.1.smithi190.stdout:7/835: dwrite d0/d1/f3e [4194304,4194304] 0 2024-05-06T12:14:57.636 INFO:tasks.workunit.client.1.smithi190.stdout:7/836: link d0/da/d36/cc1 d0/d1/d18/c122 0 2024-05-06T12:14:57.638 INFO:tasks.workunit.client.1.smithi190.stdout:7/837: unlink d0/d1/fc 0 2024-05-06T12:14:57.649 INFO:tasks.workunit.client.1.smithi190.stdout:7/838: rename d0/d1/d32/d45/d70/d33/df7/d10e/l10a to d0/d1/d2/d11e/l123 0 2024-05-06T12:14:57.650 INFO:tasks.workunit.client.1.smithi190.stdout:7/839: chown d0/da/f55 20827785 1 2024-05-06T12:14:57.652 INFO:tasks.workunit.client.1.smithi190.stdout:7/840: getdents d0/d1/d32/d45 0 2024-05-06T12:14:57.654 INFO:tasks.workunit.client.1.smithi190.stdout:7/841: dread - d0/d14/d6a/f113 zero size 2024-05-06T12:14:57.654 INFO:tasks.workunit.client.1.smithi190.stdout:7/842: creat d0/d1/d2/d84/f124 x:0 0 0 2024-05-06T12:14:57.660 INFO:tasks.workunit.client.1.smithi190.stdout:7/843: dwrite d0/d1/d18/d4e/d87/f8b [0,4194304] 0 2024-05-06T12:14:57.676 INFO:tasks.workunit.client.1.smithi190.stdout:2/854: sync 2024-05-06T12:14:57.679 INFO:tasks.workunit.client.1.smithi190.stdout:7/844: dread d0/d1/d2/f1d [0,4194304] 0 2024-05-06T12:14:57.679 INFO:tasks.workunit.client.1.smithi190.stdout:7/845: fdatasync d0/d1/d2/d84/f124 0 2024-05-06T12:14:57.680 INFO:tasks.workunit.client.1.smithi190.stdout:7/846: readlink d0/d1/d18/d37/l5f 0 2024-05-06T12:14:57.681 INFO:tasks.workunit.client.1.smithi190.stdout:2/855: write d2/d21/d123/fb3 [368084,99326] 0 2024-05-06T12:14:57.682 INFO:tasks.workunit.client.1.smithi190.stdout:2/856: getdents d2/d112 0 2024-05-06T12:14:57.684 INFO:tasks.workunit.client.1.smithi190.stdout:2/857: read d2/d8/d71/f88 [2457990,85293] 0 2024-05-06T12:14:57.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.709+0000 7f51d41a4700 1 -- 172.21.15.92:0/1861272406 --> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f51cc10f940 con 0x7f51b806b790 2024-05-06T12:14:57.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.710+0000 7f51c27fc700 1 -- 172.21.15.92:0/1861272406 <== mgr.14636 v2:172.21.15.190:6828/3484477933 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f51cc10f940 con 0x7f51b806b790 2024-05-06T12:14:57.713 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:14:57.713 INFO:teuthology.orchestra.run.smithi092.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb", 2024-05-06T12:14:57.713 INFO:teuthology.orchestra.run.smithi092.stdout: "in_progress": true, 2024-05-06T12:14:57.713 INFO:teuthology.orchestra.run.smithi092.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-06T12:14:57.713 INFO:teuthology.orchestra.run.smithi092.stdout: "services_complete": [], 2024-05-06T12:14:57.713 INFO:teuthology.orchestra.run.smithi092.stdout: "progress": "1/21 daemons upgraded", 2024-05-06T12:14:57.714 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "", 2024-05-06T12:14:57.714 INFO:teuthology.orchestra.run.smithi092.stdout: "is_paused": false 2024-05-06T12:14:57.714 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:14:57.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.713+0000 7f51d41a4700 1 -- 172.21.15.92:0/1861272406 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f51b806b790 msgr2=0x7f51b806dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:57.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.713+0000 7f51d41a4700 1 --2- 172.21.15.92:0/1861272406 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f51b806b790 0x7f51b806dc40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f51bc000c00 tx=0x7f51bc009e30 comp rx=0 tx=0).stop 2024-05-06T12:14:57.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.713+0000 7f51d41a4700 1 -- 172.21.15.92:0/1861272406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51cc066af0 msgr2=0x7f51cc10da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:57.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.713+0000 7f51d41a4700 1 --2- 172.21.15.92:0/1861272406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51cc066af0 0x7f51cc10da80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f51cc066830 tx=0x7f51c800b040 comp rx=0 tx=0).stop 2024-05-06T12:14:57.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.713+0000 7f51d41a4700 1 -- 172.21.15.92:0/1861272406 shutdown_connections 2024-05-06T12:14:57.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.713+0000 7f51d41a4700 1 --2- 172.21.15.92:0/1861272406 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f51b806b790 0x7f51b806dc40 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:57.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.713+0000 7f51d41a4700 1 --2- 172.21.15.92:0/1861272406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51cc066af0 0x7f51cc10da80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:57.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.713+0000 7f51d41a4700 1 --2- 172.21.15.92:0/1861272406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51cc0fff70 0x7f51cc10dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:57.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.713+0000 7f51d41a4700 1 -- 172.21.15.92:0/1861272406 >> 172.21.15.92:0/1861272406 conn(0x7f51cc0f7440 msgr2=0x7f51cc0f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:14:57.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.713+0000 7f51d41a4700 1 -- 172.21.15.92:0/1861272406 shutdown_connections 2024-05-06T12:14:57.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.713+0000 7f51d41a4700 1 -- 172.21.15.92:0/1861272406 wait complete. 2024-05-06T12:14:57.745 INFO:tasks.workunit.client.1.smithi190.stdout:7/847: dread d0/da/d36/d42/fc4 [0,4194304] 0 2024-05-06T12:14:57.745 INFO:tasks.workunit.client.1.smithi190.stdout:2/858: dwrite d2/d21/f89 [0,4194304] 0 2024-05-06T12:14:57.748 INFO:tasks.workunit.client.1.smithi190.stdout:7/848: creat d0/da/d36/d53/d77/d8c/dd1/d60/f125 x:0 0 0 2024-05-06T12:14:57.785 INFO:tasks.workunit.client.1.smithi190.stdout:6/632: sync 2024-05-06T12:14:57.785 INFO:tasks.workunit.client.1.smithi190.stdout:5/858: sync 2024-05-06T12:14:57.789 INFO:tasks.workunit.client.1.smithi190.stdout:7/849: creat d0/d1/d2/d84/f126 x:0 0 0 2024-05-06T12:14:57.790 INFO:tasks.workunit.client.1.smithi190.stdout:2/859: dwrite d2/d8/d19/d9b/fef [0,4194304] 0 2024-05-06T12:14:57.792 INFO:tasks.workunit.client.1.smithi190.stdout:6/633: creat df/d17/fda x:0 0 0 2024-05-06T12:14:57.794 INFO:tasks.workunit.client.1.smithi190.stdout:2/860: chown d2/d21/f42 502911 1 2024-05-06T12:14:57.796 INFO:tasks.workunit.client.1.smithi190.stdout:7/850: unlink d0/d4/l89 0 2024-05-06T12:14:57.799 INFO:tasks.workunit.client.1.smithi190.stdout:6/634: rename df/d2c/d1e/f95 to df/d12/dbd/fdb 0 2024-05-06T12:14:57.802 INFO:tasks.workunit.client.1.smithi190.stdout:2/861: dwrite d2/d8/d33/de6/f10e [0,4194304] 0 2024-05-06T12:14:57.803 INFO:tasks.workunit.client.1.smithi190.stdout:7/851: link d0/d4/f120 d0/d1/d18/f127 0 2024-05-06T12:14:57.806 INFO:tasks.workunit.client.1.smithi190.stdout:2/862: symlink d2/d8/d33/d9a/l12d 0 2024-05-06T12:14:57.807 INFO:tasks.workunit.client.1.smithi190.stdout:7/852: fdatasync d0/d1/d18/dd2/fd4 0 2024-05-06T12:14:57.810 INFO:tasks.workunit.client.1.smithi190.stdout:7/853: symlink d0/d1/d32/d45/l128 0 2024-05-06T12:14:57.811 INFO:tasks.workunit.client.1.smithi190.stdout:2/863: symlink d2/d8/d33/d46/dab/d91/l12e 0 2024-05-06T12:14:57.811 INFO:tasks.workunit.client.1.smithi190.stdout:7/854: dread - d0/d1/d32/f44 zero size 2024-05-06T12:14:57.826 INFO:tasks.workunit.client.1.smithi190.stdout:6/635: truncate df/d2c/dae/fb3 1296425 0 2024-05-06T12:14:57.828 INFO:tasks.workunit.client.1.smithi190.stdout:6/636: truncate df/d17/d44/fac 48911 0 2024-05-06T12:14:57.829 INFO:tasks.workunit.client.1.smithi190.stdout:7/855: write d0/d1/d18/d2b/f30 [1796648,88498] 0 2024-05-06T12:14:57.832 INFO:tasks.workunit.client.1.smithi190.stdout:7/856: mkdir d0/d1/d32/daa/d129 0 2024-05-06T12:14:57.837 INFO:tasks.workunit.client.1.smithi190.stdout:7/857: dread d0/d1/d32/d45/d70/d33/d57/fc0 [4194304,4194304] 0 2024-05-06T12:14:57.838 INFO:tasks.workunit.client.1.smithi190.stdout:6/637: getdents df/d17/d44 0 2024-05-06T12:14:57.838 INFO:tasks.workunit.client.1.smithi190.stdout:7/858: chown d0/da/d36/d53/fb5 339930 1 2024-05-06T12:14:57.839 INFO:tasks.workunit.client.1.smithi190.stdout:7/859: dread - d0/d1/d18/d2b/ff5 zero size 2024-05-06T12:14:57.840 INFO:tasks.workunit.client.1.smithi190.stdout:7/860: write d0/d4/f39 [4576586,65537] 0 2024-05-06T12:14:57.843 INFO:tasks.workunit.client.1.smithi190.stdout:7/861: symlink d0/d1/d18/d4e/d87/df1/l12a 0 2024-05-06T12:14:57.845 INFO:tasks.workunit.client.1.smithi190.stdout:7/862: mkdir d0/d1/d32/d45/d70/d33/df7/d12b 0 2024-05-06T12:14:57.847 INFO:tasks.workunit.client.1.smithi190.stdout:7/863: creat d0/da/d36/d119/f12c x:0 0 0 2024-05-06T12:14:57.849 INFO:tasks.workunit.client.1.smithi190.stdout:7/864: symlink d0/d1/d2/d84/l12d 0 2024-05-06T12:14:57.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.846+0000 7f8b71774700 1 -- 172.21.15.92:0/2539849825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b6c0fdc60 msgr2=0x7f8b6c1045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:57.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.846+0000 7f8b71774700 1 --2- 172.21.15.92:0/2539849825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b6c0fdc60 0x7f8b6c1045b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8b60004940 tx=0x7f8b60009e30 comp rx=0 tx=0).stop 2024-05-06T12:14:57.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.846+0000 7f8b71774700 1 -- 172.21.15.92:0/2539849825 shutdown_connections 2024-05-06T12:14:57.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.846+0000 7f8b71774700 1 --2- 172.21.15.92:0/2539849825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b6c0fdc60 0x7f8b6c1045b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:57.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.846+0000 7f8b71774700 1 --2- 172.21.15.92:0/2539849825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b6c0fd310 0x7f8b6c0fd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:57.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.846+0000 7f8b71774700 1 -- 172.21.15.92:0/2539849825 >> 172.21.15.92:0/2539849825 conn(0x7f8b6c0f6b80 msgr2=0x7f8b6c0f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:14:57.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.846+0000 7f8b71774700 1 -- 172.21.15.92:0/2539849825 shutdown_connections 2024-05-06T12:14:57.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.846+0000 7f8b71774700 1 -- 172.21.15.92:0/2539849825 wait complete. 2024-05-06T12:14:57.850 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:57 smithi092 ceph-mon[26954]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:14:57.850 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:57 smithi092 ceph-mon[26954]: Reconfiguring grafana.smithi092 (dependencies changed)... 2024-05-06T12:14:57.850 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:57 smithi092 ceph-mon[26954]: Reconfiguring daemon grafana.smithi092 on smithi092 2024-05-06T12:14:57.850 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:57 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/113730957' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:14:57.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.847+0000 7f8b71774700 1 Processor -- start 2024-05-06T12:14:57.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.847+0000 7f8b71774700 1 -- start start 2024-05-06T12:14:57.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.847+0000 7f8b71774700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b6c0fd310 0x7f8b6c10fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:57.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.847+0000 7f8b71774700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b6c0fdc60 0x7f8b6c110160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:57.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.847+0000 7f8b6a7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b6c0fdc60 0x7f8b6c110160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:57.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.848+0000 7f8b6a7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b6c0fdc60 0x7f8b6c110160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36212/0 (socket says 172.21.15.92:36212) 2024-05-06T12:14:57.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.848+0000 7f8b6a7fc700 1 -- 172.21.15.92:0/1648514215 learned_addr learned my addr 172.21.15.92:0/1648514215 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:14:57.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.848+0000 7f8b71774700 1 -- 172.21.15.92:0/1648514215 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c110760 con 0x7f8b6c0fdc60 2024-05-06T12:14:57.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.848+0000 7f8b71774700 1 -- 172.21.15.92:0/1648514215 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c110fb0 con 0x7f8b6c0fd310 2024-05-06T12:14:57.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.848+0000 7f8b6affd700 1 --2- 172.21.15.92:0/1648514215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b6c0fd310 0x7f8b6c10fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:57.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.848+0000 7f8b6affd700 1 -- 172.21.15.92:0/1648514215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b6c0fdc60 msgr2=0x7f8b6c110160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:57.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.848+0000 7f8b6affd700 1 --2- 172.21.15.92:0/1648514215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b6c0fdc60 0x7f8b6c110160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:57.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.848+0000 7f8b6affd700 1 -- 172.21.15.92:0/1648514215 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b60005040 con 0x7f8b6c0fd310 2024-05-06T12:14:57.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.848+0000 7f8b6a7fc700 1 --2- 172.21.15.92:0/1648514215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b6c0fdc60 0x7f8b6c110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-06T12:14:57.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.848+0000 7f8b6affd700 1 --2- 172.21.15.92:0/1648514215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b6c0fd310 0x7f8b6c10fc20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c002820 tx=0x7f8b5c004570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:14:57.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.849+0000 7f8b537fe700 1 -- 172.21.15.92:0/1648514215 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b5c007cc0 con 0x7f8b6c0fd310 2024-05-06T12:14:57.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.849+0000 7f8b537fe700 1 -- 172.21.15.92:0/1648514215 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b5c00f040 con 0x7f8b6c0fd310 2024-05-06T12:14:57.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.849+0000 7f8b71774700 1 -- 172.21.15.92:0/1648514215 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b6c111290 con 0x7f8b6c0fd310 2024-05-06T12:14:57.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.849+0000 7f8b71774700 1 -- 172.21.15.92:0/1648514215 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b6c111710 con 0x7f8b6c0fd310 2024-05-06T12:14:57.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.849+0000 7f8b537fe700 1 -- 172.21.15.92:0/1648514215 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b5c013920 con 0x7f8b6c0fd310 2024-05-06T12:14:57.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.850+0000 7f8b71774700 1 -- 172.21.15.92:0/1648514215 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b580052e0 con 0x7f8b6c0fd310 2024-05-06T12:14:57.854 INFO:tasks.workunit.client.1.smithi190.stdout:7/865: fdatasync d0/f69 0 2024-05-06T12:14:57.854 INFO:tasks.workunit.client.1.smithi190.stdout:7/866: creat d0/da/d36/d42/f12e x:0 0 0 2024-05-06T12:14:57.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.851+0000 7f8b537fe700 1 -- 172.21.15.92:0/1648514215 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7f8b5c009070 con 0x7f8b6c0fd310 2024-05-06T12:14:57.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.851+0000 7f8b537fe700 1 --2- 172.21.15.92:0/1648514215 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f8b5406b860 0x7f8b5406dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:14:57.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.852+0000 7f8b537fe700 1 -- 172.21.15.92:0/1648514215 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f8b5c089f30 con 0x7f8b6c0fd310 2024-05-06T12:14:57.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.852+0000 7f8b6a7fc700 1 --2- 172.21.15.92:0/1648514215 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f8b5406b860 0x7f8b5406dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:14:57.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.853+0000 7f8b6a7fc700 1 --2- 172.21.15.92:0/1648514215 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f8b5406b860 0x7f8b5406dd10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8b60004f90 tx=0x7f8b60009e30 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:14:57.856 INFO:tasks.workunit.client.1.smithi190.stdout:6/638: dread df/d2c/d61/f79 [4194304,4194304] 0 2024-05-06T12:14:57.856 INFO:tasks.workunit.client.1.smithi190.stdout:7/867: mkdir d0/d1/d2/d84/d12f 0 2024-05-06T12:14:57.857 INFO:tasks.workunit.client.1.smithi190.stdout:7/868: fsync d0/d1/d32/daa/fcb 0 2024-05-06T12:14:57.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:57.859+0000 7f8b537fe700 1 -- 172.21.15.92:0/1648514215 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f8b5c057660 con 0x7f8b6c0fd310 2024-05-06T12:14:57.864 INFO:tasks.workunit.client.1.smithi190.stdout:7/869: dwrite d0/d14/d6a/d8f/fdb [0,4194304] 0 2024-05-06T12:14:57.866 INFO:tasks.workunit.client.1.smithi190.stdout:6/639: write df/d2c/d92/f9a [259709,33410] 0 2024-05-06T12:14:57.868 INFO:tasks.workunit.client.1.smithi190.stdout:7/870: creat d0/d1/d2/d84/f130 x:0 0 0 2024-05-06T12:14:57.869 INFO:tasks.workunit.client.1.smithi190.stdout:6/640: symlink df/d2c/d61/d84/d8f/ldc 0 2024-05-06T12:14:57.873 INFO:tasks.workunit.client.1.smithi190.stdout:6/641: mkdir df/d2c/d61/d87/dc4/ddd 0 2024-05-06T12:14:57.878 INFO:tasks.workunit.client.1.smithi190.stdout:6/642: rename df/d17/d44/f7d to df/d2c/d61/d87/fde 0 2024-05-06T12:14:57.879 INFO:tasks.workunit.client.1.smithi190.stdout:6/643: chown df/d12/f2b 19443198 1 2024-05-06T12:14:57.881 INFO:tasks.workunit.client.1.smithi190.stdout:6/644: truncate df/d2c/d61/d87/fba 172597 0 2024-05-06T12:14:57.885 INFO:tasks.workunit.client.1.smithi190.stdout:7/871: dwrite d0/f69 [0,4194304] 0 2024-05-06T12:14:57.888 INFO:tasks.workunit.client.1.smithi190.stdout:6/645: mknod df/d2c/d61/d87/cdf 0 2024-05-06T12:14:57.891 INFO:tasks.workunit.client.1.smithi190.stdout:6/646: chown df/d17/f38 83164191 1 2024-05-06T12:14:57.905 INFO:tasks.workunit.client.1.smithi190.stdout:6/647: rename df/d17/d44 to df/d2c/d28/de0 0 2024-05-06T12:14:57.906 INFO:tasks.workunit.client.1.smithi190.stdout:6/648: write df/d2c/d61/d84/fb4 [1036148,111187] 0 2024-05-06T12:14:57.907 INFO:tasks.workunit.client.1.smithi190.stdout:7/872: truncate d0/d1/d2/d84/dc6/fff 508025 0 2024-05-06T12:14:57.908 INFO:tasks.workunit.client.1.smithi190.stdout:6/649: rmdir df/d2c/d1e 39 2024-05-06T12:14:57.909 INFO:tasks.workunit.client.1.smithi190.stdout:7/873: mknod d0/d1/d18/d37/c131 0 2024-05-06T12:14:57.912 INFO:tasks.workunit.client.1.smithi190.stdout:7/874: creat d0/da/d36/d53/f132 x:0 0 0 2024-05-06T12:14:57.916 INFO:tasks.workunit.client.1.smithi190.stdout:7/875: rename d0/da/d36/cc1 to d0/d1/d2/d84/dc6/d10c/c133 0 2024-05-06T12:14:57.921 INFO:tasks.workunit.client.1.smithi190.stdout:4/684: sync 2024-05-06T12:14:57.921 INFO:tasks.workunit.client.1.smithi190.stdout:0/858: sync 2024-05-06T12:14:57.922 INFO:tasks.workunit.client.1.smithi190.stdout:0/859: stat d9/dc/d28/d4c/d80 0 2024-05-06T12:14:57.943 INFO:tasks.workunit.client.1.smithi190.stdout:7/876: symlink d0/d1/d32/dd8/l134 0 2024-05-06T12:14:57.945 INFO:tasks.workunit.client.1.smithi190.stdout:0/860: mkdir d9/dcf/d56/d133 0 2024-05-06T12:14:57.947 INFO:tasks.workunit.client.1.smithi190.stdout:0/861: truncate d9/f5f 5480494 0 2024-05-06T12:14:57.947 INFO:tasks.workunit.client.1.smithi190.stdout:7/877: creat d0/d1/d2/f135 x:0 0 0 2024-05-06T12:14:57.949 INFO:tasks.workunit.client.1.smithi190.stdout:4/685: creat d2/d11/d16/d8e/d7b/de8/d66/d6b/fea x:0 0 0 2024-05-06T12:14:57.950 INFO:tasks.workunit.client.1.smithi190.stdout:0/862: mknod d9/d6f/c134 0 2024-05-06T12:14:57.953 INFO:tasks.workunit.client.1.smithi190.stdout:4/686: creat d2/d11/d16/d41/d81/feb x:0 0 0 2024-05-06T12:14:57.953 INFO:tasks.workunit.client.1.smithi190.stdout:0/863: write d9/dc/d28/dc5/dd5/de4/dea/f121 [1024180,14404] 0 2024-05-06T12:14:57.961 INFO:tasks.workunit.client.1.smithi190.stdout:0/864: creat d9/dcf/f135 x:0 0 0 2024-05-06T12:14:57.962 INFO:tasks.workunit.client.1.smithi190.stdout:4/687: mkdir d2/d11/d16/d8e/d7b/de8/d66/d6b/de3/dec 0 2024-05-06T12:14:57.964 INFO:tasks.workunit.client.1.smithi190.stdout:0/865: mkdir d9/dc/d48/d50/d87/d136 0 2024-05-06T12:14:57.965 INFO:tasks.workunit.client.1.smithi190.stdout:0/866: fsync d9/dc/d28/d4c/d96/f128 0 2024-05-06T12:14:57.966 INFO:tasks.workunit.client.1.smithi190.stdout:0/867: chown d9/dc/d28/d4c/f4b 0 1 2024-05-06T12:14:57.966 INFO:tasks.workunit.client.1.smithi190.stdout:4/688: rename d2/d7d/fbe to d2/d11/d16/d8e/d7b/de8/d66/d6b/de3/dec/fed 0 2024-05-06T12:14:57.967 INFO:tasks.workunit.client.1.smithi190.stdout:0/868: write d9/dcf/f71 [1625796,71088] 0 2024-05-06T12:14:57.968 INFO:tasks.workunit.client.1.smithi190.stdout:4/689: write d2/d11/d16/d8e/d7b/de8/d66/fb7 [126603,24746] 0 2024-05-06T12:14:57.970 INFO:tasks.workunit.client.1.smithi190.stdout:4/690: chown d2/d11/d16/d4e/fd6 942072949 1 2024-05-06T12:14:57.976 INFO:tasks.workunit.client.1.smithi190.stdout:4/691: link d2/d10/l8c d2/d11/d16/d8e/d7b/de8/d9e/d6e/lee 0 2024-05-06T12:14:57.977 INFO:tasks.workunit.client.1.smithi190.stdout:7/878: dwrite d0/d1/d18/dd2/fd4 [0,4194304] 0 2024-05-06T12:14:57.982 INFO:tasks.workunit.client.1.smithi190.stdout:7/879: creat d0/d1/d18/dd2/f136 x:0 0 0 2024-05-06T12:14:57.984 INFO:tasks.workunit.client.1.smithi190.stdout:0/869: dread d9/d51/dc9/f102 [0,4194304] 0 2024-05-06T12:14:57.985 INFO:tasks.workunit.client.1.smithi190.stdout:4/692: write d2/d11/d16/f1f [138598,124329] 0 2024-05-06T12:14:57.988 INFO:tasks.workunit.client.1.smithi190.stdout:0/870: read d9/d51/f55 [2652488,82404] 0 2024-05-06T12:14:57.998 INFO:tasks.workunit.client.1.smithi190.stdout:7/880: truncate d0/d1/fbe 1050179 0 2024-05-06T12:14:57.999 INFO:tasks.workunit.client.1.smithi190.stdout:4/693: write d2/d10/f58 [1293511,39279] 0 2024-05-06T12:14:58.000 INFO:tasks.workunit.client.1.smithi190.stdout:7/881: stat d0/d1/d32/db0/c109 0 2024-05-06T12:14:58.002 INFO:tasks.workunit.client.1.smithi190.stdout:4/694: mkdir d2/d11/d16/d8e/d7b/de8/dd1/def 0 2024-05-06T12:14:58.004 INFO:tasks.workunit.client.1.smithi190.stdout:0/871: dwrite d9/dcf/d21/f88 [0,4194304] 0 2024-05-06T12:14:58.007 INFO:tasks.workunit.client.1.smithi190.stdout:7/882: creat d0/d1/d32/d45/d70/d3b/f137 x:0 0 0 2024-05-06T12:14:58.011 INFO:tasks.workunit.client.1.smithi190.stdout:0/872: creat d9/dc/d28/dc5/dd5/de4/dea/dfd/d110/f137 x:0 0 0 2024-05-06T12:14:58.011 INFO:tasks.workunit.client.1.smithi190.stdout:4/695: dwrite d2/d11/d16/f90 [0,4194304] 0 2024-05-06T12:14:58.012 INFO:tasks.workunit.client.1.smithi190.stdout:7/883: rmdir d0/d1/d32/d45/d70/d33/df7/d10e 39 2024-05-06T12:14:58.013 INFO:tasks.workunit.client.1.smithi190.stdout:4/696: write d2/d11/d16/d8e/fd3 [812657,89824] 0 2024-05-06T12:14:58.015 INFO:tasks.workunit.client.1.smithi190.stdout:7/884: mknod d0/da/d36/d53/d77/d8c/c138 0 2024-05-06T12:14:58.017 INFO:tasks.workunit.client.1.smithi190.stdout:4/697: creat d2/d11/d16/d31/ff0 x:0 0 0 2024-05-06T12:14:58.018 INFO:tasks.workunit.client.1.smithi190.stdout:7/885: creat d0/da/d36/d42/f139 x:0 0 0 2024-05-06T12:14:58.020 INFO:tasks.workunit.client.1.smithi190.stdout:0/873: dwrite d9/dc/f6c [0,4194304] 0 2024-05-06T12:14:58.021 INFO:tasks.workunit.client.1.smithi190.stdout:7/886: rmdir d0 39 2024-05-06T12:14:58.021 INFO:tasks.workunit.client.1.smithi190.stdout:0/874: fsync d9/d6f/f11d 0 2024-05-06T12:14:58.023 INFO:tasks.workunit.client.1.smithi190.stdout:4/698: dread d2/d11/d16/f1f [0,4194304] 0 2024-05-06T12:14:58.026 INFO:tasks.workunit.client.1.smithi190.stdout:0/875: rename d9/dc/d48/d50/l5b to d9/dc/d48/l138 0 2024-05-06T12:14:58.028 INFO:tasks.workunit.client.1.smithi190.stdout:0/876: chown d9/dc/cd1 11 1 2024-05-06T12:14:58.033 INFO:tasks.workunit.client.1.smithi190.stdout:0/877: dwrite d9/dc/d28/dc5/f11c [0,4194304] 0 2024-05-06T12:14:58.040 INFO:tasks.workunit.client.1.smithi190.stdout:1/744: sync 2024-05-06T12:14:58.040 INFO:tasks.workunit.client.1.smithi190.stdout:9/859: sync 2024-05-06T12:14:58.040 INFO:tasks.workunit.client.1.smithi190.stdout:5/859: sync 2024-05-06T12:14:58.040 INFO:tasks.workunit.client.1.smithi190.stdout:8/889: sync 2024-05-06T12:14:58.040 INFO:tasks.workunit.client.1.smithi190.stdout:2/864: sync 2024-05-06T12:14:58.040 INFO:tasks.workunit.client.1.smithi190.stdout:3/949: sync 2024-05-06T12:14:58.042 INFO:tasks.workunit.client.1.smithi190.stdout:9/860: stat db/dd/d16 0 2024-05-06T12:14:58.043 INFO:tasks.workunit.client.1.smithi190.stdout:1/745: chown d5/d4f/d65/d62/d9d/dba/fd0 2686 1 2024-05-06T12:14:58.044 INFO:tasks.workunit.client.1.smithi190.stdout:3/950: mknod d5/d94/dd9/c145 0 2024-05-06T12:14:58.045 INFO:tasks.workunit.client.1.smithi190.stdout:2/865: fsync d2/fa5 0 2024-05-06T12:14:58.050 INFO:tasks.workunit.client.1.smithi190.stdout:9/861: rmdir db/dd/df 39 2024-05-06T12:14:58.054 INFO:tasks.workunit.client.1.smithi190.stdout:3/951: unlink d5/d14/d11/d21/d44/d80/d82/c103 0 2024-05-06T12:14:58.054 INFO:tasks.workunit.client.1.smithi190.stdout:4/699: dread d2/f59 [0,4194304] 0 2024-05-06T12:14:58.056 INFO:tasks.workunit.client.1.smithi190.stdout:0/878: dwrite d9/d51/f64 [0,4194304] 0 2024-05-06T12:14:58.057 INFO:tasks.workunit.client.1.smithi190.stdout:5/860: creat d2/d65/d83/df3/f115 x:0 0 0 2024-05-06T12:14:58.059 INFO:tasks.workunit.client.1.smithi190.stdout:8/890: rename d3/d8/d25/lb4 to d3/d8/d25/d65/d11d/l135 0 2024-05-06T12:14:58.059 INFO:tasks.workunit.client.1.smithi190.stdout:9/862: write db/dd/df/dfe/d87/fcb [4066560,104699] 0 2024-05-06T12:14:58.061 INFO:tasks.workunit.client.1.smithi190.stdout:3/952: mkdir d5/d14/d11/d21/d44/d92/d9e/da4/d146 0 2024-05-06T12:14:58.062 INFO:tasks.workunit.client.1.smithi190.stdout:9/863: chown db/dd/df/dfe/d87/d99/fc9 413 1 2024-05-06T12:14:58.068 INFO:tasks.workunit.client.1.smithi190.stdout:1/746: rename d5/d4f/d65/d32/d47/d53/d98/da7/fb6 to d5/d4f/d68/f106 0 2024-05-06T12:14:58.069 INFO:tasks.workunit.client.1.smithi190.stdout:5/861: mknod d2/da/d59/d5f/d2e/d3b/d76/d9a/c116 0 2024-05-06T12:14:58.070 INFO:tasks.workunit.client.1.smithi190.stdout:8/891: mknod d3/d8/d2b/d3e/dad/c136 0 2024-05-06T12:14:58.072 INFO:tasks.workunit.client.1.smithi190.stdout:8/892: truncate d3/d8/d2b/d3e/dad/fe8 968725 0 2024-05-06T12:14:58.072 INFO:tasks.workunit.client.1.smithi190.stdout:4/700: mknod d2/d11/d16/d8e/d7b/de8/d66/d6b/cf1 0 2024-05-06T12:14:58.074 INFO:tasks.workunit.client.1.smithi190.stdout:3/953: rmdir d5/d14/d11/d21/d44/d92/d9e/da4/d105/dfb 39 2024-05-06T12:14:58.076 INFO:tasks.workunit.client.1.smithi190.stdout:9/864: read - db/dd/df/dfe/f91 zero size 2024-05-06T12:14:58.077 INFO:tasks.workunit.client.1.smithi190.stdout:3/954: write d5/d14/d11/d21/d44/d80/d82/d3d/ded/d33/d5c/f129 [2831330,119372] 0 2024-05-06T12:14:58.077 INFO:tasks.workunit.client.1.smithi190.stdout:9/865: chown db/dd/df/dfe/cf0 1112193254 1 2024-05-06T12:14:58.078 INFO:tasks.workunit.client.1.smithi190.stdout:2/866: getdents d2/d8/d33/dc2/dcd 0 2024-05-06T12:14:58.082 INFO:tasks.workunit.client.1.smithi190.stdout:1/747: mkdir d5/d4f/d65/d32/dac/d107 0 2024-05-06T12:14:58.083 INFO:tasks.workunit.client.1.smithi190.stdout:8/893: dread d3/d8/d2b/d3e/dad/f120 [0,4194304] 0 2024-05-06T12:14:58.083 INFO:tasks.workunit.client.1.smithi190.stdout:5/862: creat d2/da/f117 x:0 0 0 2024-05-06T12:14:58.084 INFO:tasks.workunit.client.1.smithi190.stdout:1/748: chown d5/d4f/d65/d24/d43/da6/ded 97591839 1 2024-05-06T12:14:58.086 INFO:tasks.workunit.client.1.smithi190.stdout:0/879: dwrite d9/dc/d28/d4c/d96/fa0 [0,4194304] 0 2024-05-06T12:14:58.086 INFO:tasks.workunit.client.1.smithi190.stdout:1/749: read - d5/d4f/d65/d62/d9d/fdb zero size 2024-05-06T12:14:58.088 INFO:tasks.workunit.client.1.smithi190.stdout:5/863: write d2/da/d59/d5f/d2e/d8b/fd6 [569186,69636] 0 2024-05-06T12:14:58.089 INFO:tasks.workunit.client.1.smithi190.stdout:1/750: write d5/d4f/d65/d32/d47/f57 [4853983,103069] 0 2024-05-06T12:14:58.093 INFO:tasks.workunit.client.1.smithi190.stdout:9/866: write db/dd/f41 [1135772,101896] 0 2024-05-06T12:14:58.095 INFO:tasks.workunit.client.1.smithi190.stdout:9/867: fdatasync db/dd/d16/d4c/d4e/d5e/ff3 0 2024-05-06T12:14:58.095 INFO:tasks.workunit.client.1.smithi190.stdout:1/751: truncate d5/d4f/d65/d62/f8d 1719835 0 2024-05-06T12:14:58.096 INFO:tasks.workunit.client.1.smithi190.stdout:8/894: rmdir d3/d8/d68/d9f 39 2024-05-06T12:14:58.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:57 smithi190 ceph-mon[33798]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:14:58.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:57 smithi190 ceph-mon[33798]: Reconfiguring grafana.smithi092 (dependencies changed)... 2024-05-06T12:14:58.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:57 smithi190 ceph-mon[33798]: Reconfiguring daemon grafana.smithi092 on smithi092 2024-05-06T12:14:58.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:57 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/113730957' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:14:58.100 INFO:tasks.workunit.client.1.smithi190.stdout:0/880: mknod d9/dc/d48/d50/d83/c139 0 2024-05-06T12:14:58.104 INFO:tasks.workunit.client.1.smithi190.stdout:1/752: creat d5/d4f/d65/d32/d47/d53/d98/f108 x:0 0 0 2024-05-06T12:14:58.106 INFO:tasks.workunit.client.1.smithi190.stdout:4/701: dwrite d2/f48 [4194304,4194304] 0 2024-05-06T12:14:58.106 INFO:tasks.workunit.client.1.smithi190.stdout:1/753: readlink d5/d6b/l97 0 2024-05-06T12:14:58.106 INFO:tasks.workunit.client.1.smithi190.stdout:5/864: dread d2/f8 [0,4194304] 0 2024-05-06T12:14:58.109 INFO:tasks.workunit.client.1.smithi190.stdout:3/955: write d5/d14/d11/d21/d44/d92/d9e/da4/d105/fbd [205767,37864] 0 2024-05-06T12:14:58.110 INFO:tasks.workunit.client.1.smithi190.stdout:2/867: getdents d2/d8/d33/d9a 0 2024-05-06T12:14:58.111 INFO:tasks.workunit.client.1.smithi190.stdout:1/754: chown d5/d15/fde 797542 1 2024-05-06T12:14:58.112 INFO:tasks.workunit.client.1.smithi190.stdout:8/895: getdents d3/d8/d25/d3b/d90/dcb/dfb/d116 0 2024-05-06T12:14:58.113 INFO:tasks.workunit.client.1.smithi190.stdout:9/868: dread db/dd/df/dfe/d87/d22/d49/d8c/fa0 [0,4194304] 0 2024-05-06T12:14:58.115 INFO:tasks.workunit.client.1.smithi190.stdout:5/865: creat d2/da/d59/d5f/d2e/d3b/d72/dec/f118 x:0 0 0 2024-05-06T12:14:58.115 INFO:tasks.workunit.client.1.smithi190.stdout:3/956: dread d5/d14/d11/d21/d44/d80/f11f [0,4194304] 0 2024-05-06T12:14:58.116 INFO:tasks.workunit.client.1.smithi190.stdout:5/866: chown d2/d65/l97 0 1 2024-05-06T12:14:58.118 INFO:tasks.workunit.client.1.smithi190.stdout:2/868: dread d2/d21/d123/fb1 [0,4194304] 0 2024-05-06T12:14:58.118 INFO:tasks.workunit.client.1.smithi190.stdout:0/881: truncate d9/dc/db6/d11b/f124 658996 0 2024-05-06T12:14:58.118 INFO:tasks.workunit.client.1.smithi190.stdout:3/957: truncate d5/d7/d106/f134 916350 0 2024-05-06T12:14:58.119 INFO:tasks.workunit.client.1.smithi190.stdout:4/702: rmdir d2/d11/d16/d8e/d7b/de8/d66/d6b/da4/de1 39 2024-05-06T12:14:58.120 INFO:tasks.workunit.client.1.smithi190.stdout:2/869: write d2/f38 [46275,12412] 0 2024-05-06T12:14:58.125 INFO:tasks.workunit.client.1.smithi190.stdout:0/882: dread d9/f5f [4194304,4194304] 0 2024-05-06T12:14:58.127 INFO:tasks.workunit.client.1.smithi190.stdout:5/867: rename d2/da/l105 to d2/da/d59/d5f/d2e/d3b/d76/l119 0 2024-05-06T12:14:58.128 INFO:tasks.workunit.client.1.smithi190.stdout:0/883: truncate d9/dc/d48/d50/d83/f94 4355943 0 2024-05-06T12:14:58.129 INFO:tasks.workunit.client.1.smithi190.stdout:0/884: truncate d9/dcf/f62 4438435 0 2024-05-06T12:14:58.132 INFO:tasks.workunit.client.1.smithi190.stdout:9/869: write db/f77 [1226528,55520] 0 2024-05-06T12:14:58.133 INFO:tasks.workunit.client.1.smithi190.stdout:1/755: creat d5/d4f/d65/d24/f109 x:0 0 0 2024-05-06T12:14:58.136 INFO:tasks.workunit.client.1.smithi190.stdout:8/896: dwrite d3/d8/fa2 [0,4194304] 0 2024-05-06T12:14:58.145 INFO:tasks.workunit.client.1.smithi190.stdout:1/756: dwrite d5/d4f/f50 [0,4194304] 0 2024-05-06T12:14:58.148 INFO:tasks.workunit.client.1.smithi190.stdout:2/870: unlink d2/d8/c53 0 2024-05-06T12:14:58.148 INFO:tasks.workunit.client.1.smithi190.stdout:4/703: mknod d2/d11/d16/d8e/d7b/de8/dd1/def/cf2 0 2024-05-06T12:14:58.149 INFO:tasks.workunit.client.1.smithi190.stdout:3/958: creat d5/d14/f147 x:0 0 0 2024-05-06T12:14:58.150 INFO:tasks.workunit.client.1.smithi190.stdout:1/757: chown d5/fa 163375960 1 2024-05-06T12:14:58.150 INFO:tasks.workunit.client.1.smithi190.stdout:5/868: symlink d2/d65/l11a 0 2024-05-06T12:14:58.152 INFO:tasks.workunit.client.1.smithi190.stdout:8/897: dwrite d3/f119 [0,4194304] 0 2024-05-06T12:14:58.153 INFO:tasks.workunit.client.1.smithi190.stdout:0/885: write d9/dc/d48/f9e [908857,61645] 0 2024-05-06T12:14:58.154 INFO:tasks.workunit.client.1.smithi190.stdout:1/758: write d5/d4f/d65/d2b/fef [586266,62813] 0 2024-05-06T12:14:58.156 INFO:tasks.workunit.client.1.smithi190.stdout:0/886: truncate d9/feb 1127957 0 2024-05-06T12:14:58.157 INFO:tasks.workunit.client.1.smithi190.stdout:2/871: creat d2/d4c/d10a/d12b/f12f x:0 0 0 2024-05-06T12:14:58.158 INFO:tasks.workunit.client.1.smithi190.stdout:4/704: symlink d2/lf3 0 2024-05-06T12:14:58.160 INFO:tasks.workunit.client.1.smithi190.stdout:1/759: chown d5/d4f/d65/d32/dac/fcb 47482873 1 2024-05-06T12:14:58.162 INFO:tasks.workunit.client.1.smithi190.stdout:2/872: symlink d2/d8/d33/dc2/l130 0 2024-05-06T12:14:58.164 INFO:tasks.workunit.client.1.smithi190.stdout:2/873: dread - d2/d8/d19/fda zero size 2024-05-06T12:14:58.165 INFO:tasks.workunit.client.1.smithi190.stdout:3/959: rename d5/d14/d11/d21/d44/d80/d82/d3d/ded/d33/f120 to d5/d14/d11/d21/d44/d80/f148 0 2024-05-06T12:14:58.166 INFO:tasks.workunit.client.1.smithi190.stdout:0/887: mkdir d9/dc/d28/dc5/dd5/d13a 0 2024-05-06T12:14:58.166 INFO:tasks.workunit.client.1.smithi190.stdout:0/888: fdatasync d9/d51/f64 0 2024-05-06T12:14:58.167 INFO:tasks.workunit.client.1.smithi190.stdout:8/898: creat d3/d8/d25/d3b/d90/dcb/dfb/d116/f137 x:0 0 0 2024-05-06T12:14:58.168 INFO:tasks.workunit.client.1.smithi190.stdout:4/705: rmdir d2/d11/d16/d8e/d7b/de8/d66/d6b 39 2024-05-06T12:14:58.170 INFO:tasks.workunit.client.1.smithi190.stdout:1/760: truncate d5/d4f/d65/f5b 479214 0 2024-05-06T12:14:58.173 INFO:tasks.workunit.client.1.smithi190.stdout:0/889: creat d9/dc/d48/d50/d87/dd2/f13b x:0 0 0 2024-05-06T12:14:58.173 INFO:tasks.workunit.client.1.smithi190.stdout:8/899: creat d3/d8/d25/d3b/d6b/d7a/f138 x:0 0 0 2024-05-06T12:14:58.175 INFO:tasks.workunit.client.1.smithi190.stdout:4/706: mknod d2/d11/d16/d8e/d7b/de8/dd1/cf4 0 2024-05-06T12:14:58.177 INFO:tasks.workunit.client.1.smithi190.stdout:0/890: write d9/d51/f55 [4398393,77386] 0 2024-05-06T12:14:58.177 INFO:tasks.workunit.client.1.smithi190.stdout:0/891: readlink d9/dc/d28/d77/l126 0 2024-05-06T12:14:58.178 INFO:tasks.workunit.client.1.smithi190.stdout:0/892: chown d9/dcf/d56/dab/l119 258351421 1 2024-05-06T12:14:58.181 INFO:tasks.workunit.client.1.smithi190.stdout:5/869: dread d2/da/d59/d5f/d50/d6d/db3/fb4 [0,4194304] 0 2024-05-06T12:14:58.182 INFO:tasks.workunit.client.1.smithi190.stdout:5/870: truncate d2/d65/f114 611960 0 2024-05-06T12:14:58.183 INFO:tasks.workunit.client.1.smithi190.stdout:5/871: chown d2/da/d59/lcd 8 1 2024-05-06T12:14:58.186 INFO:tasks.workunit.client.1.smithi190.stdout:8/900: link d3/fa3 d3/d8/d68/f139 0 2024-05-06T12:14:58.187 INFO:tasks.workunit.client.1.smithi190.stdout:0/893: creat d9/dc/d48/d50/d87/d136/f13c x:0 0 0 2024-05-06T12:14:58.188 INFO:tasks.workunit.client.1.smithi190.stdout:2/874: link d2/d8/d33/caf d2/d8/d33/d46/dab/c131 0 2024-05-06T12:14:58.190 INFO:tasks.workunit.client.1.smithi190.stdout:4/707: dwrite d2/f32 [8388608,4194304] 0 2024-05-06T12:14:58.190 INFO:tasks.workunit.client.1.smithi190.stdout:2/875: dread d2/d21/d123/d79/fd6 [0,4194304] 0 2024-05-06T12:14:58.191 INFO:tasks.workunit.client.1.smithi190.stdout:5/872: link d2/da/f4a d2/da/d59/d5f/d2e/da3/f11b 0 2024-05-06T12:14:58.193 INFO:tasks.workunit.client.1.smithi190.stdout:8/901: fdatasync d3/d8/d2b/fb7 0 2024-05-06T12:14:58.201 INFO:tasks.workunit.client.1.smithi190.stdout:2/876: fsync d2/d21/d123/f12a 0 2024-05-06T12:14:58.202 INFO:tasks.workunit.client.1.smithi190.stdout:5/873: dwrite d2/da/d59/d5f/d6e/fb5 [0,4194304] 0 2024-05-06T12:14:58.236 INFO:tasks.workunit.client.1.smithi190.stdout:6/650: sync 2024-05-06T12:14:58.262 INFO:tasks.workunit.client.1.smithi190.stdout:8/902: rename d3/d8/d25/d65/dc6/fea to d3/d35/d3a/dc5/f13a 0 2024-05-06T12:14:58.264 INFO:tasks.workunit.client.1.smithi190.stdout:8/903: truncate d3/d8/d68/f139 5085131 0 2024-05-06T12:14:58.268 INFO:tasks.workunit.client.1.smithi190.stdout:6/651: readlink df/d2c/l34 0 2024-05-06T12:14:58.272 INFO:tasks.workunit.client.1.smithi190.stdout:8/904: rename d3/d8/d25/d65/cdf to d3/d35/d55/c13b 0 2024-05-06T12:14:58.273 INFO:tasks.workunit.client.1.smithi190.stdout:0/894: write d9/dc/db6/d11b/f124 [836835,108981] 0 2024-05-06T12:14:58.277 INFO:tasks.workunit.client.1.smithi190.stdout:6/652: symlink df/d2c/dae/le1 0 2024-05-06T12:14:58.278 INFO:tasks.workunit.client.1.smithi190.stdout:2/877: link d2/d21/d123/d79/l107 d2/d8/d33/d127/l132 0 2024-05-06T12:14:58.280 INFO:tasks.workunit.client.1.smithi190.stdout:8/905: truncate d3/d8/d25/d3b/d90/d6e/f82 386261 0 2024-05-06T12:14:58.282 INFO:tasks.workunit.client.1.smithi190.stdout:8/906: truncate d3/d35/f74 5059383 0 2024-05-06T12:14:58.283 INFO:tasks.workunit.client.1.smithi190.stdout:4/708: dwrite d2/d11/d16/f1a [0,4194304] 0 2024-05-06T12:14:58.286 INFO:tasks.workunit.client.1.smithi190.stdout:4/709: write d2/d11/d16/d4e/f85 [854962,81829] 0 2024-05-06T12:14:58.286 INFO:tasks.workunit.client.1.smithi190.stdout:2/878: dread d2/d21/d123/d79/fd6 [0,4194304] 0 2024-05-06T12:14:58.288 INFO:tasks.workunit.client.1.smithi190.stdout:2/879: write d2/d8/dc/f95 [4803716,49046] 0 2024-05-06T12:14:58.290 INFO:tasks.workunit.client.1.smithi190.stdout:8/907: fdatasync d3/d8/d25/d3b/d90/dcb/dfb/d116/f103 0 2024-05-06T12:14:58.291 INFO:tasks.workunit.client.1.smithi190.stdout:2/880: readlink d2/d8/d33/dc2/dcd/dfe/l1b 0 2024-05-06T12:14:58.292 INFO:tasks.workunit.client.1.smithi190.stdout:6/653: truncate df/d2c/d1e/d57/d53/f67 112305 0 2024-05-06T12:14:58.292 INFO:tasks.workunit.client.1.smithi190.stdout:2/881: readlink d2/lb 0 2024-05-06T12:14:58.294 INFO:tasks.workunit.client.1.smithi190.stdout:4/710: unlink d2/d10/l9b 0 2024-05-06T12:14:58.295 INFO:tasks.workunit.client.1.smithi190.stdout:5/874: dwrite d2/f111 [0,4194304] 0 2024-05-06T12:14:58.296 INFO:tasks.workunit.client.1.smithi190.stdout:4/711: chown d2/d11/d16/d8e/d7b/de8/cbf 1884 1 2024-05-06T12:14:58.297 INFO:tasks.workunit.client.1.smithi190.stdout:5/875: write d2/f111 [4476840,66987] 0 2024-05-06T12:14:58.298 INFO:tasks.workunit.client.1.smithi190.stdout:6/654: truncate df/d17/d37/f42 944589 0 2024-05-06T12:14:58.299 INFO:tasks.workunit.client.1.smithi190.stdout:8/908: creat d3/d8/d2b/d3e/dfd/f13c x:0 0 0 2024-05-06T12:14:58.300 INFO:tasks.workunit.client.1.smithi190.stdout:5/876: fdatasync d2/da/d59/d5f/d6e/fb5 0 2024-05-06T12:14:58.302 INFO:tasks.workunit.client.1.smithi190.stdout:8/909: dread - d3/d35/d55/d61/fee zero size 2024-05-06T12:14:58.305 INFO:tasks.workunit.client.1.smithi190.stdout:8/910: fdatasync d3/d8/d25/d65/dc6/fed 0 2024-05-06T12:14:58.305 INFO:tasks.workunit.client.1.smithi190.stdout:4/712: fdatasync d2/d11/d16/d8e/d7b/de8/d82/fa2 0 2024-05-06T12:14:58.309 INFO:tasks.workunit.client.1.smithi190.stdout:6/655: dwrite df/d2c/d28/de0/f54 [0,4194304] 0 2024-05-06T12:14:58.309 INFO:tasks.workunit.client.1.smithi190.stdout:2/882: getdents d2/d8/d33/d46/db7 0 2024-05-06T12:14:58.311 INFO:tasks.workunit.client.1.smithi190.stdout:5/877: rename d2/da/d59/d5f/d50/ccb to d2/da/d59/d5f/d2e/da7/d9d/c11c 0 2024-05-06T12:14:58.312 INFO:tasks.workunit.client.1.smithi190.stdout:8/911: creat d3/d8/d25/d65/f13d x:0 0 0 2024-05-06T12:14:58.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:58.310+0000 7f8b71774700 1 -- 172.21.15.92:0/1648514215 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8b58005860 con 0x7f8b6c0fd310 2024-05-06T12:14:58.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:58.311+0000 7f8b537fe700 1 -- 172.21.15.92:0/1648514215 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f8b5c0571f0 con 0x7f8b6c0fd310 2024-05-06T12:14:58.314 INFO:teuthology.orchestra.run.smithi092.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-06T12:14:58.314 INFO:teuthology.orchestra.run.smithi092.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-06T12:14:58.314 INFO:teuthology.orchestra.run.smithi092.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-06T12:14:58.315 INFO:tasks.workunit.client.1.smithi190.stdout:6/656: readlink df/d2c/d1e/d57/d53/l5d 0 2024-05-06T12:14:58.316 INFO:tasks.workunit.client.1.smithi190.stdout:6/657: write df/d17/f5c [9230899,74967] 0 2024-05-06T12:14:58.318 INFO:tasks.workunit.client.1.smithi190.stdout:4/713: rename d2/d11/d16/lad to d2/d11/d16/d8e/d80/lf5 0 2024-05-06T12:14:58.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:58.314+0000 7f8b71774700 1 -- 172.21.15.92:0/1648514215 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f8b5406b860 msgr2=0x7f8b5406dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:58.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:58.314+0000 7f8b71774700 1 --2- 172.21.15.92:0/1648514215 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f8b5406b860 0x7f8b5406dd10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8b60004f90 tx=0x7f8b60009e30 comp rx=0 tx=0).stop 2024-05-06T12:14:58.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:58.314+0000 7f8b71774700 1 -- 172.21.15.92:0/1648514215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b6c0fd310 msgr2=0x7f8b6c10fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:14:58.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:58.314+0000 7f8b71774700 1 --2- 172.21.15.92:0/1648514215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b6c0fd310 0x7f8b6c10fc20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c002820 tx=0x7f8b5c004570 comp rx=0 tx=0).stop 2024-05-06T12:14:58.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:58.314+0000 7f8b71774700 1 -- 172.21.15.92:0/1648514215 shutdown_connections 2024-05-06T12:14:58.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:58.314+0000 7f8b71774700 1 --2- 172.21.15.92:0/1648514215 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f8b5406b860 0x7f8b5406dd10 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:58.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:58.314+0000 7f8b71774700 1 --2- 172.21.15.92:0/1648514215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b6c0fd310 0x7f8b6c10fc20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:58.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:58.314+0000 7f8b71774700 1 --2- 172.21.15.92:0/1648514215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b6c0fdc60 0x7f8b6c110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:14:58.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:58.314+0000 7f8b71774700 1 -- 172.21.15.92:0/1648514215 >> 172.21.15.92:0/1648514215 conn(0x7f8b6c0f6b80 msgr2=0x7f8b6c0f6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:14:58.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:58.315+0000 7f8b71774700 1 -- 172.21.15.92:0/1648514215 shutdown_connections 2024-05-06T12:14:58.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:14:58.315+0000 7f8b71774700 1 -- 172.21.15.92:0/1648514215 wait complete. 2024-05-06T12:14:58.319 INFO:tasks.workunit.client.1.smithi190.stdout:8/912: creat d3/d8/d25/d3b/d90/dcb/dfb/f13e x:0 0 0 2024-05-06T12:14:58.323 INFO:tasks.workunit.client.1.smithi190.stdout:6/658: symlink df/d2c/d92/da3/le2 0 2024-05-06T12:14:58.325 INFO:tasks.workunit.client.1.smithi190.stdout:8/913: rename d3/d13/ld4 to d3/d8/d25/d3b/d90/dcb/dfb/l13f 0 2024-05-06T12:14:58.329 INFO:tasks.workunit.client.1.smithi190.stdout:6/659: unlink df/d2c/d28/f2a 0 2024-05-06T12:14:58.331 INFO:tasks.workunit.client.1.smithi190.stdout:4/714: creat d2/d11/ff6 x:0 0 0 2024-05-06T12:14:58.337 INFO:tasks.workunit.client.1.smithi190.stdout:6/660: dwrite f7 [0,4194304] 0 2024-05-06T12:14:58.344 INFO:tasks.workunit.client.1.smithi190.stdout:6/661: dread - df/d2c/d92/dce/fd9 zero size 2024-05-06T12:14:58.344 INFO:tasks.workunit.client.1.smithi190.stdout:8/914: dwrite d3/d8/d2b/d3e/dad/f120 [0,4194304] 0 2024-05-06T12:14:58.349 INFO:tasks.workunit.client.1.smithi190.stdout:5/878: dread d2/da/d59/d5f/d2e/d8b/ff6 [0,4194304] 0 2024-05-06T12:14:58.352 INFO:tasks.workunit.client.1.smithi190.stdout:8/915: readlink d3/d8/dcf/ld2 0 2024-05-06T12:14:58.352 INFO:tasks.workunit.client.1.smithi190.stdout:2/883: dwrite d2/d21/d123/fae [0,4194304] 0 2024-05-06T12:14:58.352 INFO:tasks.workunit.client.1.smithi190.stdout:6/662: unlink df/d2c/d1e/d57/d89/fa1 0 2024-05-06T12:14:58.354 INFO:tasks.workunit.client.1.smithi190.stdout:8/916: write d3/f2e [699848,90122] 0 2024-05-06T12:14:58.354 INFO:tasks.workunit.client.1.smithi190.stdout:2/884: dread - d2/d110/f11d zero size 2024-05-06T12:14:58.356 INFO:tasks.workunit.client.1.smithi190.stdout:4/715: dread d2/d11/d16/d8e/d80/f95 [0,4194304] 0 2024-05-06T12:14:58.357 INFO:tasks.workunit.client.1.smithi190.stdout:8/917: mkdir d3/d35/d3a/db9/d140 0 2024-05-06T12:14:58.358 INFO:tasks.workunit.client.1.smithi190.stdout:4/716: chown d2/d11/d16/d4e/fd6 306 1 2024-05-06T12:14:58.358 INFO:tasks.workunit.client.1.smithi190.stdout:5/879: creat d2/da/d59/d5f/f11d x:0 0 0 2024-05-06T12:14:58.358 INFO:tasks.workunit.client.1.smithi190.stdout:8/918: chown d3/d8/d25/d3b/d111/l114 1650790 1 2024-05-06T12:14:58.359 INFO:tasks.workunit.client.1.smithi190.stdout:5/880: chown d2/db6/cc4 2862 1 2024-05-06T12:14:58.360 INFO:tasks.workunit.client.1.smithi190.stdout:2/885: fsync d2/d8/d33/d46/d5d/f8c 0 2024-05-06T12:14:58.361 INFO:tasks.workunit.client.1.smithi190.stdout:4/717: stat d2/d11/d16/d8e/d7b/de8/d66/d6b/de3/dec 0 2024-05-06T12:14:58.363 INFO:tasks.workunit.client.1.smithi190.stdout:8/919: mknod d3/d35/d3a/dc5/c141 0 2024-05-06T12:14:58.365 INFO:tasks.workunit.client.1.smithi190.stdout:8/920: dread - d3/d8/d2b/d3e/dfd/f13c zero size 2024-05-06T12:14:58.366 INFO:tasks.workunit.client.1.smithi190.stdout:2/886: creat d2/d8/d19/db8/db9/f133 x:0 0 0 2024-05-06T12:14:58.368 INFO:tasks.workunit.client.1.smithi190.stdout:4/718: creat d2/d11/d16/d8e/ff7 x:0 0 0 2024-05-06T12:14:58.371 INFO:tasks.workunit.client.1.smithi190.stdout:5/881: dwrite d2/f3 [0,4194304] 0 2024-05-06T12:14:58.371 INFO:tasks.workunit.client.1.smithi190.stdout:2/887: mknod d2/d8/d33/dc2/dcd/dfe/c134 0 2024-05-06T12:14:58.372 INFO:tasks.workunit.client.1.smithi190.stdout:2/888: chown d2/d21/d123/d79/d7e 49 1 2024-05-06T12:14:58.377 INFO:tasks.workunit.client.1.smithi190.stdout:5/882: creat d2/da/d59/d5f/d6e/f11e x:0 0 0 2024-05-06T12:14:58.380 INFO:tasks.workunit.client.1.smithi190.stdout:5/883: fsync d2/da/d59/d5f/d50/d6d/fd1 0 2024-05-06T12:14:58.382 INFO:tasks.workunit.client.1.smithi190.stdout:4/719: dread d2/d10/f58 [0,4194304] 0 2024-05-06T12:14:58.383 INFO:tasks.workunit.client.1.smithi190.stdout:8/921: write d3/d8/d25/d3b/d90/fb3 [420450,120406] 0 2024-05-06T12:14:58.384 INFO:tasks.workunit.client.1.smithi190.stdout:4/720: write d2/d11/ff6 [46108,76747] 0 2024-05-06T12:14:58.385 INFO:tasks.workunit.client.1.smithi190.stdout:5/884: creat d2/da/d59/d5f/d2e/d3b/d76/d9a/f11f x:0 0 0 2024-05-06T12:14:58.385 INFO:tasks.workunit.client.1.smithi190.stdout:8/922: chown d3/d35/d3a/db9/ff2 42 1 2024-05-06T12:14:58.386 INFO:tasks.workunit.client.1.smithi190.stdout:4/721: dread - d2/d11/d16/d8e/d80/da0/db2/fd5 zero size 2024-05-06T12:14:58.390 INFO:tasks.workunit.client.1.smithi190.stdout:5/885: getdents d2/dea 0 2024-05-06T12:14:58.391 INFO:tasks.workunit.client.1.smithi190.stdout:4/722: creat d2/d11/d16/d8e/d7b/de8/d66/d6b/de3/ff8 x:0 0 0 2024-05-06T12:14:58.393 INFO:tasks.workunit.client.1.smithi190.stdout:4/723: fdatasync d2/f32 0 2024-05-06T12:14:58.396 INFO:tasks.workunit.client.1.smithi190.stdout:8/923: dwrite d3/d8/d68/d9f/fbc [0,4194304] 0 2024-05-06T12:14:58.398 INFO:tasks.workunit.client.1.smithi190.stdout:4/724: rmdir d2/d11/d16/d8e/d80 39 2024-05-06T12:14:58.401 INFO:tasks.workunit.client.1.smithi190.stdout:8/924: dread d3/d35/d55/d61/ff1 [0,4194304] 0 2024-05-06T12:14:58.401 INFO:tasks.workunit.client.1.smithi190.stdout:4/725: fsync d2/d10/fb8 0 2024-05-06T12:14:58.403 INFO:tasks.workunit.client.1.smithi190.stdout:2/889: dwrite d2/d8/d19/d4f/fd8 [0,4194304] 0 2024-05-06T12:14:58.405 INFO:tasks.workunit.client.1.smithi190.stdout:8/925: chown d3/d35/d55/l108 96713462 1 2024-05-06T12:14:58.406 INFO:tasks.workunit.client.1.smithi190.stdout:8/926: write d3/d35/f3c [8491487,56013] 0 2024-05-06T12:14:58.406 INFO:tasks.workunit.client.1.smithi190.stdout:4/726: symlink d2/d11/d16/d8e/d7b/lf9 0 2024-05-06T12:14:58.409 INFO:tasks.workunit.client.1.smithi190.stdout:7/887: sync 2024-05-06T12:14:58.413 INFO:tasks.workunit.client.1.smithi190.stdout:2/890: getdents d2/d8/d71 0 2024-05-06T12:14:58.414 INFO:tasks.workunit.client.1.smithi190.stdout:4/727: mkdir d2/d11/d16/d8e/d7b/de8/d66/d6b/da4/dfa 0 2024-05-06T12:14:58.416 INFO:tasks.workunit.client.1.smithi190.stdout:8/927: dwrite d3/f119 [0,4194304] 0 2024-05-06T12:14:58.416 INFO:tasks.workunit.client.1.smithi190.stdout:5/886: dread d2/da/d59/d5f/d6e/fc8 [0,4194304] 0 2024-05-06T12:14:58.438 INFO:tasks.workunit.client.1.smithi190.stdout:2/891: read d2/d4c/f6f [2565004,122847] 0 2024-05-06T12:14:58.438 INFO:tasks.workunit.client.1.smithi190.stdout:4/728: dread d2/d11/ff6 [0,4194304] 0 2024-05-06T12:14:58.442 INFO:tasks.workunit.client.1.smithi190.stdout:4/729: creat d2/d11/d16/d8e/d7b/dd4/ffb x:0 0 0 2024-05-06T12:14:58.443 INFO:tasks.workunit.client.1.smithi190.stdout:2/892: mknod d2/d8/d33/d46/dab/dac/c135 0 2024-05-06T12:14:58.444 INFO:tasks.workunit.client.1.smithi190.stdout:7/888: dwrite d0/d1/d32/d45/d70/d33/d57/fc0 [0,4194304] 0 2024-05-06T12:14:58.444 INFO:tasks.workunit.client.1.smithi190.stdout:5/887: truncate d2/db6/fc9 1036494 0 2024-05-06T12:14:58.446 INFO:tasks.workunit.client.1.smithi190.stdout:4/730: write d2/d11/d16/d8e/d7b/de8/d66/d6b/fea [835201,78033] 0 2024-05-06T12:14:58.446 INFO:tasks.workunit.client.1.smithi190.stdout:8/928: rename d3/d8/d25/d3b/d90/f7c to d3/d8/d68/f142 0 2024-05-06T12:14:58.449 INFO:tasks.workunit.client.1.smithi190.stdout:7/889: rename d0/d1/d18 to d0/d1/d18/dd2/d13a 22 2024-05-06T12:14:58.450 INFO:tasks.workunit.client.1.smithi190.stdout:2/893: truncate d2/d8/d33/dc2/dcd/dfe/f2c 2195139 0 2024-05-06T12:14:58.451 INFO:tasks.workunit.client.1.smithi190.stdout:8/929: fdatasync d3/d35/d55/f96 0 2024-05-06T12:14:58.451 INFO:tasks.workunit.client.1.smithi190.stdout:5/888: mkdir d2/d120 0 2024-05-06T12:14:58.452 INFO:tasks.workunit.client.1.smithi190.stdout:7/890: mknod d0/d1/d2/c13b 0 2024-05-06T12:14:58.454 INFO:tasks.workunit.client.1.smithi190.stdout:4/731: rename d2/d11/d16/d31/f56 to d2/d11/d16/d8e/d7b/de8/d9e/d6e/ffc 0 2024-05-06T12:14:58.454 INFO:tasks.workunit.client.1.smithi190.stdout:2/894: truncate d2/d8/dc/f1a 396443 0 2024-05-06T12:14:58.455 INFO:tasks.workunit.client.1.smithi190.stdout:2/895: read - d2/d21/f42 zero size 2024-05-06T12:14:58.456 INFO:tasks.workunit.client.1.smithi190.stdout:5/889: symlink d2/da/d59/d5f/d2e/d3b/d76/d9a/l121 0 2024-05-06T12:14:58.458 INFO:tasks.workunit.client.1.smithi190.stdout:8/930: rename d3/d8/d25/fac to d3/d8/d25/d3b/d111/f143 0 2024-05-06T12:14:58.459 INFO:tasks.workunit.client.1.smithi190.stdout:8/931: write d3/d8/fa2 [3033918,71809] 0 2024-05-06T12:14:58.461 INFO:tasks.workunit.client.1.smithi190.stdout:2/896: symlink d2/d8/d19/d7b/da0/l136 0 2024-05-06T12:14:58.463 INFO:tasks.workunit.client.1.smithi190.stdout:8/932: creat d3/d35/d3a/f144 x:0 0 0 2024-05-06T12:14:58.464 INFO:tasks.workunit.client.1.smithi190.stdout:5/890: mknod d2/db6/dfc/c122 0 2024-05-06T12:14:58.465 INFO:tasks.workunit.client.1.smithi190.stdout:7/891: getdents d0/d1/d18/d6c 0 2024-05-06T12:14:58.465 INFO:tasks.workunit.client.1.smithi190.stdout:8/933: chown d3/d8/d25/d3b/d6b/d7a/fb2 857 1 2024-05-06T12:14:58.467 INFO:tasks.workunit.client.1.smithi190.stdout:2/897: mkdir d2/d8/d33/d46/d137 0 2024-05-06T12:14:58.469 INFO:tasks.workunit.client.1.smithi190.stdout:7/892: read d0/da/d36/d53/d77/d8c/dd1/d60/fc2 [2127197,120478] 0 2024-05-06T12:14:58.469 INFO:tasks.workunit.client.1.smithi190.stdout:8/934: unlink d3/d35/d3a/db9/ff2 0 2024-05-06T12:14:58.470 INFO:tasks.workunit.client.1.smithi190.stdout:2/898: chown d2/d8/d33/dc2/dcd/dfe/l3f 4662 1 2024-05-06T12:14:58.473 INFO:tasks.workunit.client.1.smithi190.stdout:7/893: chown d0/d1/d18/d2b/ce4 1887 1 2024-05-06T12:14:58.473 INFO:tasks.workunit.client.1.smithi190.stdout:5/891: dwrite d2/da/d59/d5f/d6e/fb5 [0,4194304] 0 2024-05-06T12:14:58.477 INFO:tasks.workunit.client.1.smithi190.stdout:8/935: write d3/f1c [1539470,8612] 0 2024-05-06T12:14:58.479 INFO:tasks.workunit.client.1.smithi190.stdout:4/732: dread d2/d11/d16/d31/fb4 [4194304,4194304] 0 2024-05-06T12:14:58.479 INFO:tasks.workunit.client.1.smithi190.stdout:7/894: symlink d0/d1/d18/d4e/d87/d107/l13c 0 2024-05-06T12:14:58.480 INFO:tasks.workunit.client.1.smithi190.stdout:2/899: dread d2/d8/d19/d4f/fd8 [0,4194304] 0 2024-05-06T12:14:58.481 INFO:tasks.workunit.client.1.smithi190.stdout:5/892: creat d2/da/d59/d5f/d2e/d3b/d72/dec/f123 x:0 0 0 2024-05-06T12:14:58.481 INFO:tasks.workunit.client.1.smithi190.stdout:7/895: fdatasync d0/d1/d2/f1b 0 2024-05-06T12:14:58.482 INFO:tasks.workunit.client.1.smithi190.stdout:4/733: fsync d2/d11/d16/d8e/d7b/de8/d66/f74 0 2024-05-06T12:14:58.486 INFO:tasks.workunit.client.1.smithi190.stdout:8/936: creat d3/d8/d25/d3b/d90/dcb/f145 x:0 0 0 2024-05-06T12:14:58.494 INFO:tasks.workunit.client.1.smithi190.stdout:8/937: symlink d3/d8/d25/d3b/l146 0 2024-05-06T12:14:58.496 INFO:tasks.workunit.client.1.smithi190.stdout:2/900: creat d2/d8/d19/d4f/f138 x:0 0 0 2024-05-06T12:14:58.497 INFO:tasks.workunit.client.1.smithi190.stdout:4/734: dwrite d2/d11/d16/d4e/fcf [0,4194304] 0 2024-05-06T12:14:58.498 INFO:tasks.workunit.client.1.smithi190.stdout:5/893: creat d2/f124 x:0 0 0 2024-05-06T12:14:58.502 INFO:tasks.workunit.client.1.smithi190.stdout:5/894: read d2/da/d59/d5f/ff0 [66874,63799] 0 2024-05-06T12:14:58.505 INFO:tasks.workunit.client.1.smithi190.stdout:2/901: rename d2/d8/d19/d7b to d2/d8/dc/d116/d139 0 2024-05-06T12:14:58.507 INFO:tasks.workunit.client.1.smithi190.stdout:8/938: getdents d3/d35/d55/d61/df9/d10f 0 2024-05-06T12:14:58.510 INFO:tasks.workunit.client.1.smithi190.stdout:5/895: dread d2/da/d59/d5f/d2e/d8b/fd6 [0,4194304] 0 2024-05-06T12:14:58.510 INFO:tasks.workunit.client.1.smithi190.stdout:4/735: dwrite d2/d11/d16/d8e/d7b/de8/d9e/fe9 [0,4194304] 0 2024-05-06T12:14:58.516 INFO:tasks.workunit.client.1.smithi190.stdout:2/902: dread d2/d8/d33/d46/dab/fee [0,4194304] 0 2024-05-06T12:14:58.518 INFO:tasks.workunit.client.1.smithi190.stdout:8/939: write d3/d8/d68/faf [4795371,27020] 0 2024-05-06T12:14:58.520 INFO:tasks.workunit.client.1.smithi190.stdout:5/896: dwrite d2/fe6 [0,4194304] 0 2024-05-06T12:14:58.524 INFO:tasks.workunit.client.1.smithi190.stdout:5/897: creat d2/d6c/d7d/f125 x:0 0 0 2024-05-06T12:14:58.525 INFO:tasks.workunit.client.1.smithi190.stdout:5/898: chown d2/da/d59/d5f/d2e/d8b/ff6 95379 1 2024-05-06T12:14:58.525 INFO:tasks.workunit.client.1.smithi190.stdout:8/940: rename d3/d8/d68/d9f/l127 to d3/d35/d55/d61/df9/d10f/l147 0 2024-05-06T12:14:58.528 INFO:tasks.workunit.client.1.smithi190.stdout:8/941: fdatasync d3/d8/d25/d3b/d90/fb3 0 2024-05-06T12:14:58.530 INFO:tasks.workunit.client.1.smithi190.stdout:5/899: truncate d2/da/d59/d5f/d2e/d3b/d72/dec/f118 428172 0 2024-05-06T12:14:58.531 INFO:tasks.workunit.client.1.smithi190.stdout:5/900: readlink d2/l2a 0 2024-05-06T12:14:58.534 INFO:tasks.workunit.client.1.smithi190.stdout:2/903: dwrite d2/d8/dc/d116/d139/da0/fb5 [0,4194304] 0 2024-05-06T12:14:58.536 INFO:tasks.workunit.client.1.smithi190.stdout:4/736: dread d2/d11/f14 [0,4194304] 0 2024-05-06T12:14:58.536 INFO:tasks.workunit.client.1.smithi190.stdout:5/901: fdatasync d2/da/d59/d5f/d2e/d3b/fa5 0 2024-05-06T12:14:58.539 INFO:tasks.workunit.client.1.smithi190.stdout:5/902: mkdir d2/da/d59/d5f/d2e/d126 0 2024-05-06T12:14:58.539 INFO:tasks.workunit.client.1.smithi190.stdout:2/904: creat d2/d110/f13a x:0 0 0 2024-05-06T12:14:58.546 INFO:tasks.workunit.client.1.smithi190.stdout:4/737: truncate d2/d11/d16/d8e/d7b/de8/d9e/d6e/fc4 1939151 0 2024-05-06T12:14:58.548 INFO:tasks.workunit.client.1.smithi190.stdout:8/942: dread d3/d8/d2b/d3e/dad/fbe [0,4194304] 0 2024-05-06T12:14:58.548 INFO:tasks.workunit.client.1.smithi190.stdout:2/905: mknod d2/d8/d33/d46/dab/dbb/c13b 0 2024-05-06T12:14:58.550 INFO:tasks.workunit.client.1.smithi190.stdout:2/906: dread - d2/d8/dc/fad zero size 2024-05-06T12:14:58.552 INFO:tasks.workunit.client.1.smithi190.stdout:4/738: rename d2/d11/d16/d41/d81/ld9 to d2/d11/d16/d8e/d80/lfd 0 2024-05-06T12:14:58.554 INFO:tasks.workunit.client.1.smithi190.stdout:2/907: truncate d2/d21/d123/f7c 3905303 0 2024-05-06T12:14:58.554 INFO:tasks.workunit.client.1.smithi190.stdout:4/739: stat d2/d11/d16/d8e/d7b/de8/d66/d6b/cf1 0 2024-05-06T12:14:58.556 INFO:tasks.workunit.client.1.smithi190.stdout:4/740: chown d2/d11/d16/d8e/d80/lf5 53 1 2024-05-06T12:14:58.558 INFO:tasks.workunit.client.1.smithi190.stdout:5/903: truncate d2/da/d59/d5f/d2e/d3b/d76/d9a/f11f 179727 0 2024-05-06T12:14:58.558 INFO:tasks.workunit.client.1.smithi190.stdout:8/943: read d3/d8/d25/d3b/d6b/d7a/f9c [1283961,45304] 0 2024-05-06T12:14:58.559 INFO:tasks.workunit.client.1.smithi190.stdout:2/908: rename d2/d110/f11d to d2/d8/dc/d116/d139/dde/f13c 0 2024-05-06T12:14:58.560 INFO:tasks.workunit.client.1.smithi190.stdout:5/904: chown d2/da/d59/d5f/d50/l7b 737 1 2024-05-06T12:14:58.562 INFO:tasks.workunit.client.1.smithi190.stdout:2/909: creat d2/d8/d33/d46/db7/f13d x:0 0 0 2024-05-06T12:14:58.578 INFO:tasks.workunit.client.1.smithi190.stdout:4/741: dread d2/d11/d16/d8e/d7b/de8/d9e/d6e/fc4 [0,4194304] 0 2024-05-06T12:14:58.578 INFO:tasks.workunit.client.1.smithi190.stdout:8/944: dwrite d3/d35/d3a/fba [0,4194304] 0 2024-05-06T12:14:58.583 INFO:tasks.workunit.client.1.smithi190.stdout:2/910: dwrite d2/d21/d123/f12a [0,4194304] 0 2024-05-06T12:14:58.583 INFO:tasks.workunit.client.1.smithi190.stdout:5/905: dread d2/da/f4a [0,4194304] 0 2024-05-06T12:14:58.586 INFO:tasks.workunit.client.1.smithi190.stdout:4/742: dread d2/d11/d16/d8e/d7b/de8/d9e/d6e/fc4 [0,4194304] 0 2024-05-06T12:14:58.586 INFO:tasks.workunit.client.1.smithi190.stdout:2/911: chown d2/d8/d33/dc2/dcd/ce4 441992970 1 2024-05-06T12:14:58.587 INFO:tasks.workunit.client.1.smithi190.stdout:9/870: sync 2024-05-06T12:14:58.587 INFO:tasks.workunit.client.1.smithi190.stdout:3/960: sync 2024-05-06T12:14:58.592 INFO:tasks.workunit.client.1.smithi190.stdout:2/912: mknod d2/d8/d33/d46/c13e 0 2024-05-06T12:14:58.592 INFO:tasks.workunit.client.1.smithi190.stdout:8/945: dread d3/d13/fc8 [0,4194304] 0 2024-05-06T12:14:58.594 INFO:tasks.workunit.client.1.smithi190.stdout:3/961: unlink d5/d7/cd1 0 2024-05-06T12:14:58.595 INFO:tasks.workunit.client.1.smithi190.stdout:4/743: mknod d2/d11/d16/d31/cfe 0 2024-05-06T12:14:58.596 INFO:tasks.workunit.client.1.smithi190.stdout:3/962: write d5/d14/d11/d21/d44/d80/d82/f102 [1266741,91501] 0 2024-05-06T12:14:58.597 INFO:tasks.workunit.client.1.smithi190.stdout:2/913: rmdir d2/d8/d33/dc2/dcd 39 2024-05-06T12:14:58.599 INFO:tasks.workunit.client.1.smithi190.stdout:8/946: mknod d3/d35/d55/d61/c148 0 2024-05-06T12:14:58.600 INFO:tasks.workunit.client.1.smithi190.stdout:4/744: rmdir d2/d11/d16/d8e/d7b/de8/d66/d6b 39 2024-05-06T12:14:58.601 INFO:tasks.workunit.client.1.smithi190.stdout:5/906: dread d2/da/d59/d5f/d2e/d3b/fa5 [0,4194304] 0 2024-05-06T12:14:58.601 INFO:tasks.workunit.client.1.smithi190.stdout:9/871: getdents db/dd/d5c 0 2024-05-06T12:14:58.602 INFO:tasks.workunit.client.1.smithi190.stdout:3/963: symlink d5/d14/d11/d21/d44/d80/d82/d3d/ded/d33/l149 0 2024-05-06T12:14:58.605 INFO:tasks.workunit.client.1.smithi190.stdout:4/745: dread d2/d11/f14 [0,4194304] 0 2024-05-06T12:14:58.606 INFO:tasks.workunit.client.1.smithi190.stdout:4/746: chown d2/d11/d16/d8e/d7b/de8/d9e/d6e 8803384 1 2024-05-06T12:14:58.697 INFO:tasks.workunit.client.1.smithi190.stdout:8/947: rename d3/d8/d25/d3b/d90/dcb/dfb/d110 to d3/d35/d3a/dc5/d149 0 2024-05-06T12:14:58.698 INFO:tasks.workunit.client.1.smithi190.stdout:8/948: write d3/d8/f2c [2640406,37623] 0 2024-05-06T12:14:58.701 INFO:tasks.workunit.client.1.smithi190.stdout:5/907: rename d2/da/d59/d5f/d50/d6d/la8 to d2/db6/l127 0 2024-05-06T12:14:58.705 INFO:tasks.workunit.client.1.smithi190.stdout:5/908: dread - d2/d65/d83/df3/f115 zero size 2024-05-06T12:14:58.706 INFO:tasks.workunit.client.1.smithi190.stdout:9/872: creat db/dd/df/dfe/d87/d22/d49/d88/f125 x:0 0 0 2024-05-06T12:14:58.707 INFO:tasks.workunit.client.1.smithi190.stdout:3/964: rmdir d5/d14/d11/d21/d44/d92/d9e/da4/d146 0 2024-05-06T12:14:58.707 INFO:tasks.workunit.client.1.smithi190.stdout:8/949: dwrite d3/d35/f87 [4194304,4194304] 0 2024-05-06T12:14:58.708 INFO:tasks.workunit.client.1.smithi190.stdout:4/747: dwrite d2/f59 [4194304,4194304] 0 2024-05-06T12:14:58.710 INFO:tasks.workunit.client.1.smithi190.stdout:8/950: dread - d3/d8/d25/d3b/d90/dcb/dfb/f104 zero size 2024-05-06T12:14:58.748 INFO:tasks.workunit.client.1.smithi190.stdout:1/761: sync 2024-05-06T12:14:58.758 INFO:tasks.workunit.client.1.smithi190.stdout:1/762: dwrite d5/d4f/d65/d32/d47/f57 [0,4194304] 0 2024-05-06T12:14:58.763 INFO:tasks.workunit.client.1.smithi190.stdout:8/951: rmdir d3/d35/d3a/dc5 39 2024-05-06T12:14:58.765 INFO:tasks.workunit.client.1.smithi190.stdout:4/748: readlink d2/d11/d16/d8e/d7b/de8/d66/d6b/da4/lb9 0 2024-05-06T12:14:58.767 INFO:tasks.workunit.client.1.smithi190.stdout:9/873: rename db/dd/d16/d4c/f9b to db/dd/df/dfe/d87/d22/f126 0 2024-05-06T12:14:58.768 INFO:tasks.workunit.client.1.smithi190.stdout:9/874: readlink la 0 2024-05-06T12:14:58.770 INFO:tasks.workunit.client.1.smithi190.stdout:1/763: symlink d5/d4f/d65/d62/d9d/l10a 0 2024-05-06T12:14:58.771 INFO:tasks.workunit.client.1.smithi190.stdout:8/952: dwrite d3/d35/d55/ff7 [0,4194304] 0 2024-05-06T12:14:58.772 INFO:tasks.workunit.client.1.smithi190.stdout:3/965: rename d5/d14/d11/d21/d44/d80/d82/l72 to d5/d7/d6d/d9a/l14a 0 2024-05-06T12:14:58.773 INFO:tasks.workunit.client.1.smithi190.stdout:1/764: write d5/fce [491345,62247] 0 2024-05-06T12:14:58.773 INFO:tasks.workunit.client.1.smithi190.stdout:9/875: rmdir db/daa 39 2024-05-06T12:14:58.776 INFO:tasks.workunit.client.1.smithi190.stdout:8/953: symlink d3/d35/d3a/l14a 0 2024-05-06T12:14:58.778 INFO:tasks.workunit.client.1.smithi190.stdout:9/876: symlink db/dd/df/dfe/d87/d99/l127 0 2024-05-06T12:14:58.781 INFO:tasks.workunit.client.1.smithi190.stdout:3/966: rename d5/d14/d11/dc9 to d5/d14/d11/d21/d44/d92/d9e/da4/d105/db3/da1/d14b 0 2024-05-06T12:14:58.782 INFO:tasks.workunit.client.1.smithi190.stdout:8/954: creat d3/d8/d2b/d3e/f14b x:0 0 0 2024-05-06T12:14:58.783 INFO:tasks.workunit.client.1.smithi190.stdout:3/967: write d5/d14/d11/d21/d44/d80/d82/fea [5191659,31125] 0 2024-05-06T12:14:58.785 INFO:tasks.workunit.client.1.smithi190.stdout:1/765: rename d5/d15/d1e/cb7 to d5/d4f/d65/d32/d47/de5/c10b 0 2024-05-06T12:14:58.787 INFO:tasks.workunit.client.1.smithi190.stdout:8/955: unlink d3/d35/d55/d61/cf4 0 2024-05-06T12:14:58.788 INFO:tasks.workunit.client.1.smithi190.stdout:9/877: truncate db/d23/da4/f10a 268865 0 2024-05-06T12:14:58.789 INFO:tasks.workunit.client.1.smithi190.stdout:9/878: chown db/dd/df/dfe/ddc/d47/d10c/d11f 96 1 2024-05-06T12:14:58.790 INFO:tasks.workunit.client.1.smithi190.stdout:9/879: dread - db/dd/d16/d4c/d4e/f110 zero size 2024-05-06T12:14:58.791 INFO:tasks.workunit.client.1.smithi190.stdout:8/956: symlink d3/d8/d2b/d3e/l14c 0 2024-05-06T12:14:58.796 INFO:tasks.workunit.client.1.smithi190.stdout:8/957: creat d3/d35/d55/d61/df9/d10f/f14d x:0 0 0 2024-05-06T12:14:58.798 INFO:tasks.workunit.client.1.smithi190.stdout:8/958: creat d3/d8/d25/d3b/d90/d6e/f14e x:0 0 0 2024-05-06T12:14:58.800 INFO:tasks.workunit.client.1.smithi190.stdout:8/959: readlink d3/d8/dcf/l86 0 2024-05-06T12:14:58.802 INFO:tasks.workunit.client.1.smithi190.stdout:8/960: mkdir d3/d8/dcf/d14f 0 2024-05-06T12:14:58.802 INFO:tasks.workunit.client.1.smithi190.stdout:4/749: write d2/d11/d16/f38 [1459871,120797] 0 2024-05-06T12:14:58.803 INFO:tasks.workunit.client.1.smithi190.stdout:4/750: chown d2/l8 1 1 2024-05-06T12:14:58.807 INFO:tasks.workunit.client.1.smithi190.stdout:4/751: mkdir d2/d11/d16/d8e/d7b/de8/d66/d6b/de3/dec/dff 0 2024-05-06T12:14:58.808 INFO:tasks.workunit.client.1.smithi190.stdout:4/752: chown d2/d11/d16/d8e/d7b/de8/d9e/d6e 5642534 1 2024-05-06T12:14:58.809 INFO:tasks.workunit.client.1.smithi190.stdout:9/880: dread db/dd/df/dfe/d87/d22/f54 [0,4194304] 0 2024-05-06T12:14:58.817 INFO:tasks.workunit.client.1.smithi190.stdout:3/968: truncate d5/d14/d11/d21/f30 2722695 0 2024-05-06T12:14:58.819 INFO:tasks.workunit.client.1.smithi190.stdout:8/961: write d3/d8/d25/f39 [1392916,86528] 0 2024-05-06T12:14:58.820 INFO:tasks.workunit.client.1.smithi190.stdout:3/969: chown d5/d14/d11/d21/d44/d92/d9e/da4/d105/db3/da1/d14b/d132 474 1 2024-05-06T12:14:58.826 INFO:tasks.workunit.client.1.smithi190.stdout:1/766: dwrite d5/d4f/d65/f5b [0,4194304] 0 2024-05-06T12:14:58.828 INFO:tasks.workunit.client.1.smithi190.stdout:3/970: write d5/d14/d11/d21/d44/fbf [487456,68312] 0 2024-05-06T12:14:58.832 INFO:tasks.workunit.client.1.smithi190.stdout:4/753: dwrite d2/d11/d16/f2f [0,4194304] 0 2024-05-06T12:14:58.833 INFO:tasks.workunit.client.1.smithi190.stdout:6/663: sync 2024-05-06T12:14:58.833 INFO:tasks.workunit.client.1.smithi190.stdout:0/895: sync 2024-05-06T12:14:58.835 INFO:tasks.workunit.client.1.smithi190.stdout:9/881: dwrite db/dd/df/dfe/ddc/d47/d4b/f9e [0,4194304] 0 2024-05-06T12:14:58.839 INFO:tasks.workunit.client.1.smithi190.stdout:9/882: readlink db/d23/lf6 0 2024-05-06T12:14:58.840 INFO:tasks.workunit.client.1.smithi190.stdout:8/962: dwrite d3/d8/d68/d9f/f10d [0,4194304] 0 2024-05-06T12:14:58.842 INFO:tasks.workunit.client.1.smithi190.stdout:3/971: mkdir d5/d7/d106/d14c 0 2024-05-06T12:14:58.843 INFO:tasks.workunit.client.1.smithi190.stdout:1/767: truncate d5/d4f/d68/f8e 1438705 0 2024-05-06T12:14:58.844 INFO:tasks.workunit.client.1.smithi190.stdout:6/664: stat df/d2c/dae/fb3 0 2024-05-06T12:14:58.844 INFO:tasks.workunit.client.1.smithi190.stdout:8/963: mknod d3/d8/d25/d3b/d111/c150 0 2024-05-06T12:14:58.847 INFO:tasks.workunit.client.1.smithi190.stdout:8/964: chown d3/d8/d25/d3b 20118114 1 2024-05-06T12:14:58.848 INFO:tasks.workunit.client.1.smithi190.stdout:1/768: mkdir d5/d4f/d10c 0 2024-05-06T12:14:58.850 INFO:tasks.workunit.client.1.smithi190.stdout:6/665: write df/d2c/d28/de0/fa8 [1186565,42339] 0 2024-05-06T12:14:58.851 INFO:tasks.workunit.client.1.smithi190.stdout:4/754: creat d2/d11/d16/d8e/d7b/de8/d66/d6b/da4/f100 x:0 0 0 2024-05-06T12:14:58.851 INFO:tasks.workunit.client.1.smithi190.stdout:1/769: chown d5/d15/fbf 1237 1 2024-05-06T12:14:58.852 INFO:tasks.workunit.client.1.smithi190.stdout:8/965: creat d3/d8/d25/d3b/d90/d6e/f151 x:0 0 0 2024-05-06T12:14:58.858 INFO:tasks.workunit.client.1.smithi190.stdout:3/972: dwrite d5/fc [0,4194304] 0 2024-05-06T12:14:58.863 INFO:tasks.workunit.client.1.smithi190.stdout:9/883: link db/dd/df/dfe/d87/d22/d49/d88/c7b db/dd/d5c/dac/c128 0 2024-05-06T12:14:58.864 INFO:tasks.workunit.client.1.smithi190.stdout:8/966: creat d3/d8/d25/d3b/d90/dff/f152 x:0 0 0 2024-05-06T12:14:58.864 INFO:tasks.workunit.client.1.smithi190.stdout:6/666: creat df/d2c/d61/d87/fe3 x:0 0 0 2024-05-06T12:14:58.864 INFO:tasks.workunit.client.1.smithi190.stdout:4/755: dread - d2/d11/d16/d8e/d7b/de8/d66/d6b/da4/de1/fe4 zero size 2024-05-06T12:14:58.866 INFO:tasks.workunit.client.1.smithi190.stdout:1/770: link d5/f34 d5/d4f/d65/d24/d43/de2/f10d 0 2024-05-06T12:14:58.869 INFO:tasks.workunit.client.1.smithi190.stdout:6/667: mknod df/d2c/d61/d84/ce4 0 2024-05-06T12:14:58.870 INFO:tasks.workunit.client.1.smithi190.stdout:8/967: dread d3/d8/d68/d9f/f10d [0,4194304] 0 2024-05-06T12:14:58.871 INFO:tasks.workunit.client.1.smithi190.stdout:6/668: chown df/d2c/d61/d84/ce4 2292 1 2024-05-06T12:14:58.872 INFO:tasks.workunit.client.1.smithi190.stdout:4/756: chown d2/d11/d16/d8e/d7b/de8/d9e/d6e/d78/cab 2 1 2024-05-06T12:14:58.873 INFO:tasks.workunit.client.1.smithi190.stdout:3/973: creat d5/f14d x:0 0 0 2024-05-06T12:14:58.875 INFO:tasks.workunit.client.1.smithi190.stdout:3/974: fdatasync d5/d14/d11/d21/d44/d80/d82/f8b 0 2024-05-06T12:14:58.877 INFO:tasks.workunit.client.1.smithi190.stdout:8/968: truncate d3/d8/d68/fe5 677727 0 2024-05-06T12:14:58.878 INFO:tasks.workunit.client.1.smithi190.stdout:3/975: stat d5/d14/d11/d21/d44/d92/d9e/da4/ca6 0 2024-05-06T12:14:58.878 INFO:tasks.workunit.client.1.smithi190.stdout:6/669: mkdir df/d2c/d28/dd1/de5 0 2024-05-06T12:14:58.879 INFO:tasks.workunit.client.1.smithi190.stdout:2/914: sync 2024-05-06T12:14:58.879 INFO:tasks.workunit.client.1.smithi190.stdout:5/909: sync 2024-05-06T12:14:58.879 INFO:tasks.workunit.client.1.smithi190.stdout:7/896: sync 2024-05-06T12:14:58.885 INFO:tasks.workunit.client.1.smithi190.stdout:4/757: creat d2/d11/d16/f101 x:0 0 0 2024-05-06T12:14:58.885 INFO:tasks.workunit.client.1.smithi190.stdout:0/896: dwrite d9/dcf/f75 [0,4194304] 0 2024-05-06T12:14:58.886 INFO:tasks.workunit.client.1.smithi190.stdout:3/976: fsync d5/d14/d11/d21/d44/d92/f101 0 2024-05-06T12:14:58.887 INFO:tasks.workunit.client.1.smithi190.stdout:5/910: write d2/da/d59/d5f/d6e/fb5 [2392592,50684] 0 2024-05-06T12:14:58.890 INFO:tasks.workunit.client.1.smithi190.stdout:0/897: dread - d9/d51/dc9/f118 zero size 2024-05-06T12:14:58.898 INFO:tasks.workunit.client.1.smithi190.stdout:5/911: dread d2/da/d59/d5f/d2e/d3b/d72/dec/f118 [0,4194304] 0 2024-05-06T12:14:58.898 INFO:tasks.workunit.client.1.smithi190.stdout:5/912: write d2/f44 [755917,31994] 0 2024-05-06T12:14:58.898 INFO:tasks.workunit.client.1.smithi190.stdout:5/913: dread - d2/ff7 zero size 2024-05-06T12:14:58.900 INFO:tasks.workunit.client.1.smithi190.stdout:2/915: mkdir d2/d8/d33/d46/dab/dac/d13f 0 2024-05-06T12:14:58.901 INFO:tasks.workunit.client.1.smithi190.stdout:7/897: fsync d0/d4/f7 0 2024-05-06T12:14:58.902 INFO:tasks.workunit.client.1.smithi190.stdout:7/898: fdatasync d0/f10b 0 2024-05-06T12:14:58.903 INFO:tasks.workunit.client.1.smithi190.stdout:4/758: symlink d2/d11/d16/d8e/d7b/de8/dd1/def/l102 0 2024-05-06T12:14:58.906 INFO:tasks.workunit.client.1.smithi190.stdout:0/898: read d9/dc/dd0/fd8 [495432,43650] 0 2024-05-06T12:14:58.907 INFO:tasks.workunit.client.1.smithi190.stdout:6/670: symlink df/d2c/d61/d84/d8f/db6/le6 0 2024-05-06T12:14:58.913 INFO:tasks.workunit.client.1.smithi190.stdout:9/884: dwrite db/dd/d16/fae [0,4194304] 0 2024-05-06T12:14:58.920 INFO:tasks.workunit.client.1.smithi190.stdout:2/916: mkdir d2/d8/dc/d116/d139/da0/d140 0 2024-05-06T12:14:58.925 INFO:tasks.workunit.client.1.smithi190.stdout:4/759: rename d2/d11/d16/d8e/d7b/de8/d9e/d6e/c70 to d2/d11/d16/d8e/d80/c103 0 2024-05-06T12:14:58.926 INFO:tasks.workunit.client.1.smithi190.stdout:0/899: dwrite d9/dc/d28/d12e/f10d [0,4194304] 0 2024-05-06T12:14:58.926 INFO:tasks.workunit.client.1.smithi190.stdout:7/899: creat d0/d1/d2/d84/dc6/d10c/d11f/f13d x:0 0 0 2024-05-06T12:14:58.926 INFO:tasks.workunit.client.1.smithi190.stdout:8/969: write d3/fa3 [4936256,79422] 0 2024-05-06T12:14:58.931 INFO:tasks.workunit.client.1.smithi190.stdout:2/917: unlink d2/fa5 0 2024-05-06T12:14:58.931 INFO:tasks.workunit.client.1.smithi190.stdout:6/671: truncate df/d2c/d1e/d57/d53/f67 718028 0 2024-05-06T12:14:58.933 INFO:tasks.workunit.client.1.smithi190.stdout:8/970: mknod d3/d8/d68/d9f/c153 0 2024-05-06T12:14:58.935 INFO:tasks.workunit.client.1.smithi190.stdout:6/672: dread - df/d2c/d61/f78 zero size 2024-05-06T12:14:58.937 INFO:tasks.workunit.client.1.smithi190.stdout:2/918: creat d2/d8/d33/dc2/dcd/f141 x:0 0 0 2024-05-06T12:14:58.939 INFO:tasks.workunit.client.1.smithi190.stdout:2/919: chown d2/c80 102 1 2024-05-06T12:14:58.940 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:58 smithi092 ceph-mon[26954]: pgmap v31: 65 pgs: 65 active+clean; 3.4 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 109 MiB/s rd, 251 MiB/s wr, 587 op/s 2024-05-06T12:14:58.940 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:58 smithi092 ceph-mon[26954]: from='client.24471 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:14:58.940 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:14:58 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/1648514215' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:14:58.941 INFO:tasks.workunit.client.1.smithi190.stdout:6/673: creat df/d2c/d61/d87/fe7 x:0 0 0 2024-05-06T12:14:58.941 INFO:tasks.workunit.client.1.smithi190.stdout:8/971: creat d3/d35/d3a/db9/d140/f154 x:0 0 0 2024-05-06T12:14:58.943 INFO:tasks.workunit.client.1.smithi190.stdout:6/674: write df/d2c/d28/de0/fac [856592,129248] 0 2024-05-06T12:14:58.945 INFO:tasks.workunit.client.1.smithi190.stdout:7/900: rename d0/da/d36/d53/d77/f11c to d0/d1/d32/f13e 0 2024-05-06T12:14:58.948 INFO:tasks.workunit.client.1.smithi190.stdout:3/977: truncate d5/d14/d11/d21/d44/d80/d82/fbb 1147207 0 2024-05-06T12:14:58.950 INFO:tasks.workunit.client.1.smithi190.stdout:6/675: fsync df/d2c/dae/fd8 0 2024-05-06T12:14:58.950 INFO:tasks.workunit.client.1.smithi190.stdout:5/914: dwrite d2/da/d59/d5f/d50/d6d/fd1 [0,4194304] 0 2024-05-06T12:14:58.953 INFO:tasks.workunit.client.1.smithi190.stdout:3/978: fsync d5/d14/d11/d21/d44/d92/d9e/da4/ff7 0 2024-05-06T12:14:58.959 INFO:tasks.workunit.client.1.smithi190.stdout:5/915: mkdir d2/d65/d83/df3/d128 0 2024-05-06T12:14:58.961 INFO:tasks.workunit.client.1.smithi190.stdout:5/916: write d2/da/d59/d5f/d2e/da7/d9d/fa4 [394707,89555] 0 2024-05-06T12:14:58.965 INFO:tasks.workunit.client.1.smithi190.stdout:9/885: truncate db/dd/df/dfe/d87/d22/d49/d8c/fcc 56220 0 2024-05-06T12:14:58.970 INFO:tasks.workunit.client.1.smithi190.stdout:5/917: fdatasync d2/da/d59/d5f/d50/d6d/fd1 0 2024-05-06T12:14:58.972 INFO:tasks.workunit.client.1.smithi190.stdout:5/918: write d2/d65/fb0 [2117892,28462] 0 2024-05-06T12:14:58.974 INFO:tasks.workunit.client.1.smithi190.stdout:4/760: rename d2/d11/d16/d8e/d7b/de8/d66/d6b/f88 to d2/d11/d16/d8e/d7b/de8/d66/d6b/da4/f104 0 2024-05-06T12:14:58.977 INFO:tasks.workunit.client.1.smithi190.stdout:9/886: mknod db/d23/d29/c129 0 2024-05-06T12:14:58.980 INFO:tasks.workunit.client.1.smithi190.stdout:8/972: write d3/d8/d68/fd8 [799075,41261] 0 2024-05-06T12:14:58.981 INFO:tasks.workunit.client.1.smithi190.stdout:5/919: fsync d2/d6c/feb 0 2024-05-06T12:14:58.983 INFO:tasks.workunit.client.1.smithi190.stdout:4/761: creat d2/d11/d16/d31/f105 x:0 0 0 2024-05-06T12:14:58.983 INFO:tasks.workunit.client.1.smithi190.stdout:2/920: dread d2/d8/d33/de6/f10e [0,4194304] 0 2024-05-06T12:14:58.984 INFO:tasks.workunit.client.1.smithi190.stdout:6/676: write df/d2c/d28/de0/faf [909736,54086] 0 2024-05-06T12:14:58.987 INFO:tasks.workunit.client.1.smithi190.stdout:2/921: stat d2/d8/d33/dc2/dcb/cfa 0 2024-05-06T12:14:58.990 INFO:tasks.workunit.client.1.smithi190.stdout:3/979: write d5/d14/d11/d21/d44/d80/ddb/d124/ff9 [863912,82639] 0 2024-05-06T12:14:58.990 INFO:tasks.workunit.client.1.smithi190.stdout:6/677: read df/d2c/d28/de0/fa8 [5649388,119264] 0 2024-05-06T12:14:58.994 INFO:tasks.workunit.client.1.smithi190.stdout:2/922: rmdir d2/d8/d33/d46/dab/dac 39 2024-05-06T12:14:58.996 INFO:tasks.workunit.client.1.smithi190.stdout:2/923: dread - d2/d8/d19/da9/f124 zero size 2024-05-06T12:14:58.997 INFO:tasks.workunit.client.1.smithi190.stdout:8/973: creat d3/d8/d68/f155 x:0 0 0 2024-05-06T12:14:58.997 INFO:tasks.workunit.client.1.smithi190.stdout:5/920: link d2/d65/c6b d2/da/d59/d5f/d6e/daa/c129 0 2024-05-06T12:14:58.999 INFO:tasks.workunit.client.1.smithi190.stdout:3/980: fdatasync d5/d14/d11/d21/d44/d80/d82/d3d/ded/d33/f76 0 2024-05-06T12:14:59.000 INFO:tasks.workunit.client.1.smithi190.stdout:9/887: write db/dd/df/dfe/d87/d22/fba [720861,105895] 0 2024-05-06T12:14:59.000 INFO:tasks.workunit.client.1.smithi190.stdout:7/901: dread d0/f20 [4194304,4194304] 0 2024-05-06T12:14:59.007 INFO:tasks.workunit.client.1.smithi190.stdout:5/921: creat d2/da/f12a x:0 0 0 2024-05-06T12:14:59.007 INFO:tasks.workunit.client.1.smithi190.stdout:4/762: dread d2/d11/d16/d31/f43 [0,4194304] 0 2024-05-06T12:14:59.008 INFO:tasks.workunit.client.1.smithi190.stdout:4/763: dread - d2/d11/d16/d31/f105 zero size 2024-05-06T12:14:59.009 INFO:tasks.workunit.client.1.smithi190.stdout:7/902: mkdir d0/db4/d13f 0 2024-05-06T12:14:59.010 INFO:tasks.workunit.client.1.smithi190.stdout:9/888: fsync db/dd/df/dfe/d87/d22/d49/d8c/fa0 0 2024-05-06T12:14:59.012 INFO:tasks.workunit.client.1.smithi190.stdout:3/981: mknod d5/d14/d11/d21/d44/d92/d9e/da4/c14e 0 2024-05-06T12:14:59.015 INFO:tasks.workunit.client.1.smithi190.stdout:6/678: rmdir df/d2c/d61/d87/dc4/ddd 0 2024-05-06T12:14:59.016 INFO:tasks.workunit.client.1.smithi190.stdout:4/764: mkdir d2/d11/d16/d8e/d7b/de8/d66/d6b/de3/dec/d106 0 2024-05-06T12:14:59.017 INFO:tasks.workunit.client.1.smithi190.stdout:7/903: symlink d0/d1/d32/dd8/l140 0 2024-05-06T12:14:59.020 INFO:tasks.workunit.client.1.smithi190.stdout:9/889: truncate db/dd/df/dfe/ddc/f32 2203200 0 2024-05-06T12:14:59.022 INFO:tasks.workunit.client.1.smithi190.stdout:5/922: creat d2/da/d59/d5f/d2e/d8b/dbe/f12b x:0 0 0 2024-05-06T12:14:59.024 INFO:tasks.workunit.client.1.smithi190.stdout:4/765: creat d2/d11/d16/d4e/f107 x:0 0 0 2024-05-06T12:14:59.028 INFO:tasks.workunit.client.1.smithi190.stdout:2/924: getdents d2/d8/dc/d116/d139 0 2024-05-06T12:14:59.029 INFO:tasks.workunit.client.1.smithi190.stdout:8/974: dwrite d3/f2f [0,4194304] 0 2024-05-06T12:14:59.031 INFO:tasks.workunit.client.1.smithi190.stdout:2/925: dread - d2/d8/d33/dc2/dcd/f141 zero size 2024-05-06T12:14:59.039 INFO:tasks.workunit.client.1.smithi190.stdout:8/975: symlink d3/d8/dcf/l156 0 2024-05-06T12:14:59.044 INFO:tasks.workunit.client.1.smithi190.stdout:8/976: stat d3/d8/d25/d3b/d6b/d7a/f9d 0 2024-05-06T12:14:59.046 INFO:tasks.workunit.client.1.smithi190.stdout:4/766: write d2/d10/f35 [2171847,67198] 0 2024-05-06T12:14:59.051 INFO:tasks.workunit.client.1.smithi190.stdout:2/926: dread d2/d21/f70 [0,4194304] 0 2024-05-06T12:14:59.051 INFO:tasks.workunit.client.1.smithi190.stdout:6/679: dwrite df/d2c/d1e/d57/d53/f6f [0,4194304] 0 2024-05-06T12:14:59.057 INFO:tasks.workunit.client.1.smithi190.stdout:7/904: dwrite d0/d1/d2/d84/fd3 [0,4194304] 0 2024-05-06T12:14:59.062 INFO:tasks.workunit.client.1.smithi190.stdout:9/890: dwrite db/dd/df/dfe/d87/d22/d49/d88/d76/fc8 [0,4194304] 0 2024-05-06T12:14:59.062 INFO:tasks.workunit.client.1.smithi190.stdout:4/767: read d2/f64 [3254695,9314] 0 2024-05-06T12:14:59.064 INFO:tasks.workunit.client.1.smithi190.stdout:6/680: dread df/d2c/d61/f70 [0,4194304] 0 2024-05-06T12:14:59.066 INFO:tasks.workunit.client.1.smithi190.stdout:3/982: dwrite d5/d14/f56 [0,4194304] 0 2024-05-06T12:14:59.068 INFO:tasks.workunit.client.1.smithi190.stdout:3/983: chown d5/d14/d11/fc7 153447584 1 2024-05-06T12:14:59.069 INFO:tasks.workunit.client.1.smithi190.stdout:5/923: dwrite d2/d6c/fc5 [0,4194304] 0 2024-05-06T12:14:59.070 INFO:tasks.workunit.client.1.smithi190.stdout:8/977: link d3/d35/d55/d61/f63 d3/d35/d3a/f157 0 2024-05-06T12:14:59.076 INFO:tasks.workunit.client.1.smithi190.stdout:8/978: read d3/d35/d3a/db9/f10c [216467,47630] 0 2024-05-06T12:14:59.076 INFO:tasks.workunit.client.1.smithi190.stdout:4/768: unlink d2/d10/l5f 0 2024-05-06T12:14:59.077 INFO:tasks.workunit.client.1.smithi190.stdout:8/979: chown d3/d35/f74 234 1 2024-05-06T12:14:59.077 INFO:tasks.workunit.client.1.smithi190.stdout:3/984: rmdir d5/d14/d11/d21/deb 39 2024-05-06T12:14:59.079 INFO:tasks.workunit.client.1.smithi190.stdout:4/769: read d2/d11/d16/d8e/d7b/de8/d82/fa2 [1844043,71293] 0 2024-05-06T12:14:59.080 INFO:tasks.workunit.client.1.smithi190.stdout:3/985: write d5/d14/d11/d21/d44/d92/d9e/da4/ff7 [187013,51590] 0 2024-05-06T12:14:59.081 INFO:tasks.workunit.client.1.smithi190.stdout:9/891: rename db/dd/df/dfe/d87/d22/d49/d88/f125 to db/d23/d7e/f12a 0 2024-05-06T12:14:59.083 INFO:tasks.workunit.client.1.smithi190.stdout:2/927: getdents d2/d21/d123/d79 0 2024-05-06T12:14:59.085 INFO:tasks.workunit.client.1.smithi190.stdout:5/924: fsync d2/da/d59/d5f/d2e/d3b/d5a/f91 0 2024-05-06T12:14:59.087 INFO:tasks.workunit.client.1.smithi190.stdout:7/905: dread d0/d4/f7 [0,4194304] 0 2024-05-06T12:14:59.091 INFO:tasks.workunit.client.1.smithi190.stdout:8/980: dwrite d3/d8/d2b/d3e/dfd/f13c [0,4194304] 0 2024-05-06T12:14:59.091 INFO:tasks.workunit.client.1.smithi190.stdout:5/925: fdatasync d2/f60 0 2024-05-06T12:14:59.095 INFO:tasks.workunit.client.1.smithi190.stdout:9/892: dread db/d23/d29/f61 [0,4194304] 0 2024-05-06T12:14:59.097 INFO:tasks.workunit.client.1.smithi190.stdout:8/981: chown d3/c8f 117339 1 2024-05-06T12:14:59.098 INFO:tasks.workunit.client.1.smithi190.stdout:7/906: symlink d0/d1/d2/d11e/l141 0 2024-05-06T12:14:59.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:58 smithi190 ceph-mon[33798]: pgmap v31: 65 pgs: 65 active+clean; 3.4 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 109 MiB/s rd, 251 MiB/s wr, 587 op/s 2024-05-06T12:14:59.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:58 smithi190 ceph-mon[33798]: from='client.24471 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:14:59.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:14:58 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/1648514215' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:14:59.100 INFO:tasks.workunit.client.1.smithi190.stdout:5/926: rmdir d2/da/d59/d5f/d50/d6d 39 2024-05-06T12:14:59.100 INFO:tasks.workunit.client.1.smithi190.stdout:4/770: dwrite d2/d11/d16/d8e/d7b/de8/d66/fb7 [0,4194304] 0 2024-05-06T12:14:59.106 INFO:tasks.workunit.client.1.smithi190.stdout:6/681: dread df/d17/f26 [0,4194304] 0 2024-05-06T12:14:59.107 INFO:tasks.workunit.client.1.smithi190.stdout:8/982: mknod d3/d8/d68/dde/c158 0 2024-05-06T12:14:59.112 INFO:tasks.workunit.client.1.smithi190.stdout:4/771: dwrite d2/d10/ff [4194304,4194304] 0 2024-05-06T12:14:59.112 INFO:tasks.workunit.client.1.smithi190.stdout:6/682: read df/d17/d37/f49 [2627248,123910] 0 2024-05-06T12:14:59.115 INFO:tasks.workunit.client.1.smithi190.stdout:7/907: dwrite d0/d1/d18/d4e/db3/fd5 [0,4194304] 0 2024-05-06T12:14:59.116 INFO:tasks.workunit.client.1.smithi190.stdout:9/893: symlink db/dd/df/dfe/d87/d99/d11c/l12b 0 2024-05-06T12:14:59.118 INFO:tasks.workunit.client.1.smithi190.stdout:6/683: mkdir df/d2c/d1e/d57/d53/de8 0 2024-05-06T12:14:59.120 INFO:tasks.workunit.client.1.smithi190.stdout:1/771: sync 2024-05-06T12:14:59.122 INFO:tasks.workunit.client.1.smithi190.stdout:8/983: dread d3/f2f [0,4194304] 0 2024-05-06T12:14:59.124 INFO:tasks.workunit.client.1.smithi190.stdout:9/894: write db/dd/df/dfe/d87/d22/d49/d8c/fde [869289,29966] 0 2024-05-06T12:14:59.124 INFO:tasks.workunit.client.1.smithi190.stdout:7/908: creat d0/d14/d6a/f142 x:0 0 0 2024-05-06T12:14:59.126 INFO:tasks.workunit.client.1.smithi190.stdout:8/984: chown d3/d35/d3a/l14a 63068 1 2024-05-06T12:14:59.127 INFO:tasks.workunit.client.1.smithi190.stdout:7/909: stat d0/d1/d32/d45/d70/d35/c79 0 2024-05-06T12:14:59.127 INFO:tasks.workunit.client.1.smithi190.stdout:1/772: unlink d5/d15/fd5 0 2024-05-06T12:14:59.131 INFO:tasks.workunit.client.1.smithi190.stdout:2/928: truncate d2/d8/d33/d46/d5d/fe1 1024370 0 2024-05-06T12:14:59.132 INFO:tasks.workunit.client.1.smithi190.stdout:9/895: mknod db/dd/c12c 0 2024-05-06T12:14:59.132 INFO:tasks.workunit.client.1.smithi190.stdout:6/684: dwrite df/f99 [0,4194304] 0 2024-05-06T12:14:59.132 INFO:tasks.workunit.client.1.smithi190.stdout:8/985: chown d3/d35/d55/d61/ff1 687109129 1 2024-05-06T12:14:59.136 INFO:tasks.workunit.client.1.smithi190.stdout:2/929: creat d2/d8/d19/db8/f142 x:0 0 0 2024-05-06T12:14:59.138 INFO:tasks.workunit.client.1.smithi190.stdout:6/685: creat df/d2c/d28/fe9 x:0 0 0 2024-05-06T12:14:59.139 INFO:tasks.workunit.client.1.smithi190.stdout:5/927: dread d2/da/d59/f77 [0,4194304] 0 2024-05-06T12:14:59.140 INFO:tasks.workunit.client.1.smithi190.stdout:8/986: unlink d3/d8/f27 0 2024-05-06T12:14:59.142 INFO:tasks.workunit.client.1.smithi190.stdout:2/930: creat d2/d8/d19/da9/f143 x:0 0 0 2024-05-06T12:14:59.143 INFO:tasks.workunit.client.1.smithi190.stdout:6/686: unlink df/d2c/d61/d73/dad/cd3 0 2024-05-06T12:14:59.144 INFO:tasks.workunit.client.1.smithi190.stdout:5/928: mknod d2/da/d59/d5f/d2e/d3b/d5a/c12c 0 2024-05-06T12:14:59.145 INFO:tasks.workunit.client.1.smithi190.stdout:5/929: dread - d2/d65/d83/fe0 zero size 2024-05-06T12:14:59.147 INFO:tasks.workunit.client.1.smithi190.stdout:8/987: rename d3/d8/d2b/d3e/dad/cc4 to d3/d8/d25/d3b/d90/dcb/dfb/c159 0 2024-05-06T12:14:59.148 INFO:tasks.workunit.client.1.smithi190.stdout:4/772: write d2/d11/d16/d31/ff0 [3715,32535] 0 2024-05-06T12:14:59.148 INFO:tasks.workunit.client.1.smithi190.stdout:3/986: dread d5/d14/d11/d21/d44/d92/d9e/da4/d105/db3/dfa/f11c [4194304,4194304] 0 2024-05-06T12:14:59.150 INFO:tasks.workunit.client.1.smithi190.stdout:6/687: creat df/d2c/d1e/d57/d53/de8/fea x:0 0 0 2024-05-06T12:14:59.152 INFO:tasks.workunit.client.1.smithi190.stdout:5/930: dread d2/da/d59/f77 [0,4194304] 0 2024-05-06T12:14:59.157 INFO:tasks.workunit.client.1.smithi190.stdout:2/931: dwrite d2/d8/d33/f37 [0,4194304] 0 2024-05-06T12:14:59.158 INFO:tasks.workunit.client.1.smithi190.stdout:3/987: creat d5/d7/f14f x:0 0 0 2024-05-06T12:14:59.158 INFO:tasks.workunit.client.1.smithi190.stdout:1/773: truncate d5/d15/d1e/f27 2159513 0 2024-05-06T12:14:59.160 INFO:tasks.workunit.client.1.smithi190.stdout:6/688: mkdir df/d2c/d28/deb 0 2024-05-06T12:14:59.161 INFO:tasks.workunit.client.1.smithi190.stdout:5/931: creat d2/dea/f12d x:0 0 0 2024-05-06T12:14:59.163 INFO:tasks.workunit.client.1.smithi190.stdout:8/988: link d3/d8/d25/d3b/d6b/d7a/f9d d3/d35/f15a 0 2024-05-06T12:14:59.163 INFO:tasks.workunit.client.1.smithi190.stdout:9/896: write db/dd/df/dfe/ddc/f95 [10511,9603] 0 2024-05-06T12:14:59.165 INFO:tasks.workunit.client.1.smithi190.stdout:2/932: rename d2/d8/d33/f37 to d2/d8/d33/de6/f144 0 2024-05-06T12:14:59.167 INFO:tasks.workunit.client.1.smithi190.stdout:3/988: symlink d5/d7/d6d/l150 0 2024-05-06T12:14:59.170 INFO:tasks.workunit.client.1.smithi190.stdout:4/773: mkdir d2/d11/d16/d8e/d7b/de8/d82/dd7/d108 0 2024-05-06T12:14:59.172 INFO:tasks.workunit.client.1.smithi190.stdout:6/689: mkdir df/d2c/d28/dd1/dec 0 2024-05-06T12:14:59.172 INFO:tasks.workunit.client.1.smithi190.stdout:4/774: dread - d2/d11/d16/d8e/d7b/dd4/ffb zero size 2024-05-06T12:14:59.173 INFO:tasks.workunit.client.1.smithi190.stdout:8/989: dread d3/f98 [0,4194304] 0 2024-05-06T12:14:59.176 INFO:tasks.workunit.client.1.smithi190.stdout:9/897: truncate db/dd/f8d 623019 0 2024-05-06T12:14:59.177 INFO:tasks.workunit.client.1.smithi190.stdout:2/933: fdatasync d2/d8/d33/d46/dab/dbb/f108 0 2024-05-06T12:14:59.179 INFO:tasks.workunit.client.1.smithi190.stdout:6/690: creat df/d2c/d61/d84/d8f/fed x:0 0 0 2024-05-06T12:14:59.181 INFO:tasks.workunit.client.1.smithi190.stdout:1/774: dread d5/d15/d1e/f5a [0,4194304] 0 2024-05-06T12:14:59.181 INFO:tasks.workunit.client.1.smithi190.stdout:4/775: unlink d2/d11/d16/d41/d81/l8b 0 2024-05-06T12:14:59.184 INFO:tasks.workunit.client.1.smithi190.stdout:8/990: symlink d3/d8/dcf/l15b 0 2024-05-06T12:14:59.185 INFO:tasks.workunit.client.1.smithi190.stdout:8/991: chown d3/d8/d2b/d3e/d8a/cfe 123930 1 2024-05-06T12:14:59.187 INFO:tasks.workunit.client.1.smithi190.stdout:9/898: unlink db/d23/d7e/f12a 0 2024-05-06T12:14:59.190 INFO:tasks.workunit.client.1.smithi190.stdout:2/934: dread d2/d21/d123/d79/f82 [0,4194304] 0 2024-05-06T12:14:59.196 INFO:tasks.workunit.client.1.smithi190.stdout:8/992: read d3/d8/d2b/f34 [1612026,117354] 0 2024-05-06T12:14:59.197 INFO:tasks.workunit.client.1.smithi190.stdout:8/993: read - d3/d35/d55/d61/f134 zero size 2024-05-06T12:14:59.198 INFO:tasks.workunit.client.1.smithi190.stdout:2/935: truncate d2/d21/f42 250177 0 2024-05-06T12:14:59.199 INFO:tasks.workunit.client.1.smithi190.stdout:2/936: chown d2/d21/d123/d79 0 1 2024-05-06T12:14:59.200 INFO:tasks.workunit.client.1.smithi190.stdout:6/691: mkdir df/dee 0 2024-05-06T12:14:59.205 INFO:tasks.workunit.client.1.smithi190.stdout:9/899: dread db/dd/df/dfe/d87/d22/d49/d88/d9f/f11e [0,4194304] 0 2024-05-06T12:14:59.205 INFO:tasks.workunit.client.1.smithi190.stdout:3/989: link d5/d14/d11/d21/d44/d92/d9e/da4/c14e d5/d14/d11/d21/d44/d92/d9e/da4/d105/c151 0 2024-05-06T12:14:59.207 INFO:tasks.workunit.client.1.smithi190.stdout:8/994: creat d3/d8/dcf/f15c x:0 0 0 2024-05-06T12:14:59.207 INFO:tasks.workunit.client.1.smithi190.stdout:3/990: chown d5/d14/d11/d21/d44/d80/d82/d3d/ded/l115 6 1 2024-05-06T12:14:59.208 INFO:tasks.workunit.client.1.smithi190.stdout:8/995: readlink d3/d8/dcf/l6f 0 2024-05-06T12:14:59.209 INFO:tasks.workunit.client.1.smithi190.stdout:2/937: creat d2/d8/d33/dc2/dcd/f145 x:0 0 0 2024-05-06T12:14:59.219 INFO:tasks.workunit.client.1.smithi190.stdout:1/775: truncate d5/d4f/d65/d24/dd4/f79 958411 0 2024-05-06T12:14:59.222 INFO:tasks.workunit.client.1.smithi190.stdout:6/692: write df/d2c/d28/de0/fa8 [9038315,72316] 0 2024-05-06T12:14:59.223 INFO:tasks.workunit.client.1.smithi190.stdout:4/776: link d2/d10/l7f d2/d11/d16/d8e/d7b/de8/d82/dd7/l109 0 2024-05-06T12:14:59.223 INFO:tasks.workunit.client.1.smithi190.stdout:2/938: dread d2/d8/d33/d46/f5f [4194304,4194304] 0 2024-05-06T12:14:59.223 INFO:tasks.workunit.client.1.smithi190.stdout:6/693: readlink df/d2c/d28/l4f 0 2024-05-06T12:14:59.225 INFO:tasks.workunit.client.1.smithi190.stdout:6/694: truncate df/d2c/d61/d87/fba 1211522 0 2024-05-06T12:14:59.227 INFO:tasks.workunit.client.1.smithi190.stdout:3/991: mknod d5/d7/c152 0 2024-05-06T12:14:59.227 INFO:tasks.workunit.client.1.smithi190.stdout:9/900: write db/dd/d16/d4c/d4e/f62 [3120982,19457] 0 2024-05-06T12:14:59.228 INFO:tasks.workunit.client.1.smithi190.stdout:1/776: mkdir d5/d15/d1e/dff/d10e 0 2024-05-06T12:14:59.228 INFO:tasks.workunit.client.1.smithi190.stdout:9/901: stat db/dd/df/dfe/d87/l11d 0 2024-05-06T12:14:59.230 INFO:tasks.workunit.client.1.smithi190.stdout:8/996: write d3/d8/d25/d3b/d90/d6e/f94 [654013,74509] 0 2024-05-06T12:14:59.231 INFO:tasks.workunit.client.1.smithi190.stdout:9/902: write db/dd/d16/f1e [3639311,27016] 0 2024-05-06T12:14:59.232 INFO:tasks.workunit.client.1.smithi190.stdout:1/777: write d5/d4f/d65/d32/f4a [60135,63949] 0 2024-05-06T12:14:59.233 INFO:tasks.workunit.client.1.smithi190.stdout:4/777: chown d2/d11/d16/d8e/d80/c103 96 1 2024-05-06T12:14:59.235 INFO:tasks.workunit.client.1.smithi190.stdout:6/695: rename df/d2c/d1e/d57/d89/fa2 to df/dee/fef 0 2024-05-06T12:14:59.235 INFO:tasks.workunit.client.1.smithi190.stdout:4/778: write d2/d11/d16/d31/f43 [491950,16424] 0 2024-05-06T12:14:59.238 INFO:tasks.workunit.client.1.smithi190.stdout:1/778: dread d5/d4f/d65/d32/d47/d53/d98/da7/de7/fda [0,4194304] 0 2024-05-06T12:14:59.239 INFO:tasks.workunit.client.1.smithi190.stdout:2/939: mkdir d2/d146 0 2024-05-06T12:14:59.239 INFO:tasks.workunit.client.1.smithi190.stdout:8/997: creat d3/d13/f15d x:0 0 0 2024-05-06T12:14:59.241 INFO:tasks.workunit.client.1.smithi190.stdout:4/779: unlink d2/d11/d16/d8e/d7b/de8/d82/fa2 0 2024-05-06T12:14:59.321 INFO:tasks.workunit.client.1.smithi190.stdout:0/900: sync 2024-05-06T12:14:59.323 INFO:tasks.workunit.client.1.smithi190.stdout:0/901: truncate d9/dc/d48/d50/d83/f94 5278303 0 2024-05-06T12:14:59.324 INFO:tasks.workunit.client.1.smithi190.stdout:3/992: rename d5/d14/d11/d21/l48 to d5/d14/d11/d21/d44/d80/ddb/l153 0 2024-05-06T12:14:59.326 INFO:tasks.workunit.client.1.smithi190.stdout:8/998: unlink d3/d8/d25/d3b/d90/dcb/dfb/l13f 0 2024-05-06T12:14:59.327 INFO:tasks.workunit.client.1.smithi190.stdout:1/779: creat d5/d4f/d65/d24/f10f x:0 0 0 2024-05-06T12:14:59.329 INFO:tasks.workunit.client.1.smithi190.stdout:0/902: truncate d9/d51/f9d 58728 0 2024-05-06T12:14:59.331 INFO:tasks.workunit.client.1.smithi190.stdout:2/940: rename d2/d8/dc/d116/d139/da0/fb5 to d2/d8/d19/dbd/f147 0 2024-05-06T12:14:59.333 INFO:tasks.workunit.client.1.smithi190.stdout:8/999: stat d3/d8/d2b/c4d 0 2024-05-06T12:14:59.335 INFO:tasks.workunit.client.1.smithi190.stdout:4/780: dwrite d2/d11/d16/d8e/d7b/de8/d66/fae [0,4194304] 0 2024-05-06T12:14:59.342 INFO:tasks.workunit.client.1.smithi190.stdout:1/780: mknod d5/d15/d105/c110 0 2024-05-06T12:14:59.343 INFO:tasks.workunit.client.1.smithi190.stdout:2/941: mkdir d2/d4c/d10a/d12b/d148 0 2024-05-06T12:14:59.344 INFO:tasks.workunit.client.1.smithi190.stdout:0/903: creat d9/dc/d28/d4c/d80/da7/d120/f13d x:0 0 0 2024-05-06T12:14:59.346 INFO:tasks.workunit.client.1.smithi190.stdout:3/993: creat d5/d14/d11/d21/d44/d92/d9e/da4/d105/db3/f154 x:0 0 0 2024-05-06T12:14:59.348 INFO:tasks.workunit.client.1.smithi190.stdout:1/781: mkdir d5/d4f/d65/d24/d3a/d111 0 2024-05-06T12:14:59.357 INFO:tasks.workunit.client.1.smithi190.stdout:4/781: link d2/d11/d16/d31/l54 d2/d11/d16/d8e/d7b/de8/d82/dd7/l10a 0 2024-05-06T12:14:59.362 INFO:tasks.workunit.client.1.smithi190.stdout:4/782: readlink d2/d10/l36 0 2024-05-06T12:14:59.363 INFO:tasks.workunit.client.1.smithi190.stdout:1/782: dread d5/d15/d1e/f27 [0,4194304] 0 2024-05-06T12:14:59.366 INFO:tasks.workunit.client.1.smithi190.stdout:4/783: fdatasync d2/d11/d16/d8e/d7b/de8/d9e/d6e/f92 0 2024-05-06T12:14:59.367 INFO:tasks.workunit.client.1.smithi190.stdout:4/784: fsync d2/d11/d16/d31/f43 0 2024-05-06T12:14:59.371 INFO:tasks.workunit.client.1.smithi190.stdout:4/785: creat d2/d11/d16/d8e/d80/da0/f10b x:0 0 0 2024-05-06T12:14:59.372 INFO:tasks.workunit.client.1.smithi190.stdout:1/783: dwrite d5/f6d [0,4194304] 0 2024-05-06T12:14:59.372 INFO:tasks.workunit.client.1.smithi190.stdout:4/786: chown d2/d11/d16/d4e/l6a 4 1 2024-05-06T12:14:59.376 INFO:tasks.workunit.client.1.smithi190.stdout:1/784: mknod d5/d4f/d65/d2b/c112 0 2024-05-06T12:14:59.376 INFO:tasks.workunit.client.1.smithi190.stdout:3/994: write d5/d14/d11/d21/d44/d80/d82/d3d/ded/f96 [436952,62960] 0 2024-05-06T12:14:59.378 INFO:tasks.workunit.client.1.smithi190.stdout:1/785: fdatasync d5/d4f/d65/d62/fcc 0 2024-05-06T12:14:59.384 INFO:tasks.workunit.client.1.smithi190.stdout:0/904: dwrite d9/dcf/d21/fd9 [0,4194304] 0 2024-05-06T12:14:59.385 INFO:tasks.workunit.client.1.smithi190.stdout:3/995: mknod d5/d14/d11/d21/d44/d80/d82/d3d/ded/d33/d5c/c155 0 2024-05-06T12:14:59.391 INFO:tasks.workunit.client.1.smithi190.stdout:4/787: fsync d2/d11/d16/d8e/d7b/de8/d66/d6b/f8f 0 2024-05-06T12:14:59.396 INFO:tasks.workunit.client.1.smithi190.stdout:1/786: getdents d5/d6b/d8f 0 2024-05-06T12:14:59.397 INFO:tasks.workunit.client.1.smithi190.stdout:0/905: write d9/f1b [3573012,76953] 0 2024-05-06T12:14:59.399 INFO:tasks.workunit.client.1.smithi190.stdout:4/788: truncate d2/f64 3602370 0 2024-05-06T12:14:59.405 INFO:tasks.workunit.client.1.smithi190.stdout:4/789: write d2/d11/d16/d8e/d7b/de8/d66/fb7 [1058024,67841] 0 2024-05-06T12:14:59.405 INFO:tasks.workunit.client.1.smithi190.stdout:1/787: dwrite d5/d4f/fe4 [0,4194304] 0 2024-05-06T12:14:59.409 INFO:tasks.workunit.client.1.smithi190.stdout:4/790: symlink d2/d11/d16/d8e/d7b/l10c 0 2024-05-06T12:14:59.410 INFO:tasks.workunit.client.1.smithi190.stdout:3/996: dread d5/d7/f86 [0,4194304] 0 2024-05-06T12:14:59.410 INFO:tasks.workunit.client.1.smithi190.stdout:0/906: write d9/dc/d48/fe1 [1484652,113056] 0 2024-05-06T12:14:59.414 INFO:tasks.workunit.client.1.smithi190.stdout:3/997: readlink d5/d14/d11/d21/d44/d92/d9e/da4/d105/db3/l140 0 2024-05-06T12:14:59.416 INFO:tasks.workunit.client.1.smithi190.stdout:1/788: dwrite d5/fa [0,4194304] 0 2024-05-06T12:14:59.417 INFO:tasks.workunit.client.1.smithi190.stdout:0/907: truncate d9/dc/d28/d4c/d80/da7/f103 692826 0 2024-05-06T12:14:59.419 INFO:tasks.workunit.client.1.smithi190.stdout:3/998: symlink d5/d94/dd9/l156 0 2024-05-06T12:14:59.422 INFO:tasks.workunit.client.1.smithi190.stdout:1/789: chown d5/d4f/d65/d32/c71 95217550 1 2024-05-06T12:14:59.422 INFO:tasks.workunit.client.1.smithi190.stdout:4/791: dwrite d2/d11/d16/d8e/d7b/de8/d66/fb7 [0,4194304] 0 2024-05-06T12:14:59.422 INFO:tasks.workunit.client.1.smithi190.stdout:0/908: mkdir d9/dc/d48/d50/d87/dd2/d13e 0 2024-05-06T12:14:59.422 INFO:tasks.workunit.client.1.smithi190.stdout:3/999: dread - d5/d14/d11/d21/d44/d92/d9e/da4/df3/f138 zero size 2024-05-06T12:14:59.423 INFO:tasks.workunit.client.1.smithi190.stdout:1/790: chown d5/d4f/d65/ff8 1693704 1 2024-05-06T12:14:59.424 INFO:tasks.workunit.client.1.smithi190.stdout:1/791: readlink d5/d4f/d65/l5c 0 2024-05-06T12:14:59.427 INFO:tasks.workunit.client.1.smithi190.stdout:0/909: symlink d9/dc/d48/d50/d87/l13f 0 2024-05-06T12:14:59.431 INFO:tasks.workunit.client.1.smithi190.stdout:1/792: creat d5/d4f/d65/d2b/f113 x:0 0 0 2024-05-06T12:14:59.432 INFO:tasks.workunit.client.1.smithi190.stdout:0/910: mkdir d9/dc/d140 0 2024-05-06T12:14:59.433 INFO:tasks.workunit.client.1.smithi190.stdout:4/792: getdents d2/d11/d16/d8e/d7b/de8/d66 0 2024-05-06T12:14:59.435 INFO:tasks.workunit.client.1.smithi190.stdout:1/793: read d5/d4f/d65/d2b/fbc [4653344,2778] 0 2024-05-06T12:14:59.435 INFO:tasks.workunit.client.1.smithi190.stdout:0/911: rmdir d9/dc/d28/dc5/dd5/de4/dea 39 2024-05-06T12:14:59.437 INFO:tasks.workunit.client.1.smithi190.stdout:4/793: unlink d2/d11/d16/d8e/d7b/de8/la5 0 2024-05-06T12:14:59.439 INFO:tasks.workunit.client.1.smithi190.stdout:0/912: write d9/dcf/d21/f2a [2182607,79203] 0 2024-05-06T12:14:59.439 INFO:tasks.workunit.client.1.smithi190.stdout:4/794: symlink d2/d11/d16/d31/l10d 0 2024-05-06T12:14:59.441 INFO:tasks.workunit.client.1.smithi190.stdout:1/794: truncate d5/d15/f82 826972 0 2024-05-06T12:14:59.442 INFO:tasks.workunit.client.1.smithi190.stdout:0/913: mkdir d9/d91/d141 0 2024-05-06T12:14:59.443 INFO:tasks.workunit.client.1.smithi190.stdout:0/914: readlink d9/dc/d48/lb5 0 2024-05-06T12:14:59.445 INFO:tasks.workunit.client.1.smithi190.stdout:4/795: symlink d2/d11/d16/d8e/d7b/de8/d9e/d6e/d78/l10e 0 2024-05-06T12:14:59.446 INFO:tasks.workunit.client.1.smithi190.stdout:0/915: dread - d9/d51/dc9/f8d zero size 2024-05-06T12:14:59.446 INFO:tasks.workunit.client.1.smithi190.stdout:4/796: stat d2/d11/d16/d41/f65 0 2024-05-06T12:14:59.448 INFO:tasks.workunit.client.1.smithi190.stdout:1/795: dwrite d5/d4f/d65/d24/d43/ffc [0,4194304] 0 2024-05-06T12:14:59.449 INFO:tasks.workunit.client.1.smithi190.stdout:1/796: stat d5/d15/d1e/l21 0 2024-05-06T12:14:59.450 INFO:tasks.workunit.client.1.smithi190.stdout:4/797: readlink d2/d10/l7f 0 2024-05-06T12:14:59.457 INFO:tasks.workunit.client.1.smithi190.stdout:1/797: rename d5/d4f/d65/d24/dd4 to d5/d4f/d65/d24/d43/de2/d114 0 2024-05-06T12:14:59.458 INFO:tasks.workunit.client.1.smithi190.stdout:1/798: stat d5/d4f/d65/d62/f8d 0 2024-05-06T12:14:59.459 INFO:tasks.workunit.client.1.smithi190.stdout:1/799: stat d5/d4f/d65/d32/d47/de5 0 2024-05-06T12:14:59.460 INFO:tasks.workunit.client.1.smithi190.stdout:4/798: mkdir d2/d11/d16/d8e/d7b/de8/d66/d6b/da4/dfa/d10f 0 2024-05-06T12:14:59.463 INFO:tasks.workunit.client.1.smithi190.stdout:4/799: write d2/d11/d16/d8e/d7b/de8/d9e/d6e/d78/fc3 [535000,41268] 0 2024-05-06T12:14:59.466 INFO:tasks.workunit.client.1.smithi190.stdout:4/800: write d2/d11/d16/d8e/d7b/de8/d9e/d6e/d78/fdf [603079,17996] 0 2024-05-06T12:14:59.467 INFO:tasks.workunit.client.1.smithi190.stdout:0/916: dwrite d9/dc/d28/d12e/f34 [0,4194304] 0 2024-05-06T12:14:59.467 INFO:tasks.workunit.client.1.smithi190.stdout:0/917: chown d9/d51 28571 1 2024-05-06T12:14:59.469 INFO:tasks.workunit.client.1.smithi190.stdout:4/801: creat d2/d10/f110 x:0 0 0 2024-05-06T12:14:59.472 INFO:tasks.workunit.client.1.smithi190.stdout:0/918: mkdir d9/dc/d48/d50/d87/dd2/d142 0 2024-05-06T12:14:59.475 INFO:tasks.workunit.client.1.smithi190.stdout:0/919: rmdir d9/dc/d28/d4c/d96 39 2024-05-06T12:14:59.479 INFO:tasks.workunit.client.1.smithi190.stdout:0/920: symlink d9/dc/d48/d50/d87/dd2/db3/l143 0 2024-05-06T12:14:59.481 INFO:tasks.workunit.client.1.smithi190.stdout:1/800: dwrite d5/d4f/d65/d32/d47/f4c [0,4194304] 0 2024-05-06T12:14:59.483 INFO:tasks.workunit.client.1.smithi190.stdout:4/802: dwrite d2/d11/d16/d8e/d7b/de8/d66/d6b/fea [0,4194304] 0 2024-05-06T12:14:59.484 INFO:tasks.workunit.client.1.smithi190.stdout:0/921: write d9/dcf/d21/f7e [5146874,109772] 0 2024-05-06T12:14:59.486 INFO:tasks.workunit.client.1.smithi190.stdout:1/801: write d5/d4f/d65/d24/d43/de2/d114/fae [60407,71705] 0 2024-05-06T12:14:59.488 INFO:tasks.workunit.client.1.smithi190.stdout:4/803: creat d2/d11/d16/d8e/d80/da0/db2/f111 x:0 0 0 2024-05-06T12:14:59.488 INFO:tasks.workunit.client.1.smithi190.stdout:0/922: symlink d9/d91/d141/l144 0 2024-05-06T12:14:59.495 INFO:tasks.workunit.client.1.smithi190.stdout:0/923: link d9/dc/l46 d9/l145 0 2024-05-06T12:14:59.499 INFO:tasks.workunit.client.1.smithi190.stdout:4/804: dwrite d2/d11/d16/d8e/d7b/de8/d9e/d6e/daa/fdd [0,4194304] 0 2024-05-06T12:14:59.500 INFO:tasks.workunit.client.1.smithi190.stdout:0/924: creat d9/dc/d48/d50/d83/dc6/d109/f146 x:0 0 0 2024-05-06T12:14:59.502 INFO:tasks.workunit.client.1.smithi190.stdout:1/802: dread d5/d4f/d65/d32/d47/f77 [0,4194304] 0 2024-05-06T12:14:59.504 INFO:tasks.workunit.client.1.smithi190.stdout:7/910: sync 2024-05-06T12:14:59.504 INFO:tasks.workunit.client.1.smithi190.stdout:0/925: write d9/dc/d28/dc5/dd5/de4/dea/f121 [2000780,68637] 0 2024-05-06T12:14:59.508 INFO:tasks.workunit.client.1.smithi190.stdout:1/803: creat d5/d4f/d65/d62/d9d/dba/f115 x:0 0 0 2024-05-06T12:14:59.510 INFO:tasks.workunit.client.1.smithi190.stdout:1/804: fsync d5/d4f/d65/d32/d47/d53/d98/f108 0 2024-05-06T12:14:59.513 INFO:tasks.workunit.client.1.smithi190.stdout:7/911: dread - d0/d1/d32/d45/d70/d3b/f96 zero size 2024-05-06T12:14:59.513 INFO:tasks.workunit.client.1.smithi190.stdout:1/805: chown d5/d4f/d65/d32/d47/f77 0 1 2024-05-06T12:14:59.513 INFO:tasks.workunit.client.1.smithi190.stdout:0/926: link d9/d51/dc9/c90 d9/dc/d48/d50/d83/d12c/c147 0 2024-05-06T12:14:59.516 INFO:tasks.workunit.client.1.smithi190.stdout:7/912: creat d0/d1/d18/d2b/f143 x:0 0 0 2024-05-06T12:14:59.517 INFO:tasks.workunit.client.1.smithi190.stdout:4/805: truncate d2/fb0 3636736 0 2024-05-06T12:14:59.519 INFO:tasks.workunit.client.1.smithi190.stdout:0/927: rename d9/dc/d28/l3c to d9/dc/d28/d4c/d80/da7/l148 0 2024-05-06T12:14:59.520 INFO:tasks.workunit.client.1.smithi190.stdout:1/806: dwrite d5/d4f/d65/d24/d43/ffc [4194304,4194304] 0 2024-05-06T12:14:59.520 INFO:tasks.workunit.client.1.smithi190.stdout:4/806: chown d2/d11/d16/d8e/d7b/dd4 0 1 2024-05-06T12:14:59.521 INFO:tasks.workunit.client.1.smithi190.stdout:7/913: symlink d0/d1/d18/dfa/l144 0 2024-05-06T12:14:59.524 INFO:tasks.workunit.client.1.smithi190.stdout:0/928: creat d9/dcf/d56/f149 x:0 0 0 2024-05-06T12:14:59.526 INFO:tasks.workunit.client.1.smithi190.stdout:0/929: write d9/dcf/d21/fd9 [4745746,120843] 0 2024-05-06T12:14:59.528 INFO:tasks.workunit.client.1.smithi190.stdout:0/930: stat d9/dc/d48/d50/d87/d129 0 2024-05-06T12:14:59.531 INFO:tasks.workunit.client.1.smithi190.stdout:7/914: creat d0/d1/d32/d45/d70/d3b/f145 x:0 0 0 2024-05-06T12:14:59.534 INFO:tasks.workunit.client.1.smithi190.stdout:4/807: dwrite d2/d11/ff6 [0,4194304] 0 2024-05-06T12:14:59.537 INFO:tasks.workunit.client.1.smithi190.stdout:7/915: fdatasync d0/d1/d2/d84/fbf 0 2024-05-06T12:14:59.537 INFO:tasks.workunit.client.1.smithi190.stdout:7/916: fdatasync d0/f10b 0 2024-05-06T12:14:59.539 INFO:tasks.workunit.client.1.smithi190.stdout:4/808: fdatasync d2/d11/d16/d8e/d7b/de8/d66/d6b/f9f 0 2024-05-06T12:14:59.539 INFO:tasks.workunit.client.1.smithi190.stdout:7/917: write d0/d1/d2/f1b [4811119,3095] 0 2024-05-06T12:14:59.542 INFO:tasks.workunit.client.1.smithi190.stdout:4/809: unlink d2/d10/l28 0 2024-05-06T12:14:59.544 INFO:tasks.workunit.client.1.smithi190.stdout:7/918: rename d0/d1/d2/d84/l12d to d0/d1/d32/db0/l146 0 2024-05-06T12:14:59.545 INFO:tasks.workunit.client.1.smithi190.stdout:1/807: dwrite d5/d15/f82 [0,4194304] 0 2024-05-06T12:14:59.551 INFO:tasks.workunit.client.1.smithi190.stdout:7/919: truncate d0/d1/d32/d45/f7d 641395 0 2024-05-06T12:14:59.555 INFO:tasks.workunit.client.1.smithi190.stdout:1/808: dwrite d5/d4f/d65/d24/f7e [4194304,4194304] 0 2024-05-06T12:14:59.556 INFO:tasks.workunit.client.1.smithi190.stdout:7/920: unlink d0/d1/d32/d45/d70/d33/df7/d10e/l64 0 2024-05-06T12:14:59.558 INFO:tasks.workunit.client.1.smithi190.stdout:1/809: creat d5/d4f/d65/d24/d43/da6/ded/f116 x:0 0 0 2024-05-06T12:14:59.561 INFO:tasks.workunit.client.1.smithi190.stdout:7/921: getdents d0/d1/d32/d116 0 2024-05-06T12:14:59.561 INFO:tasks.workunit.client.1.smithi190.stdout:7/922: stat d0/d1/d18/d2b/f143 0 2024-05-06T12:14:59.562 INFO:tasks.workunit.client.1.smithi190.stdout:4/810: write d2/d11/d16/d8e/d7b/de8/d9e/d6e/ffc [3708447,16094] 0 2024-05-06T12:14:59.564 INFO:tasks.workunit.client.1.smithi190.stdout:7/923: truncate d0/da/d36/d42/f139 635044 0 2024-05-06T12:14:59.565 INFO:tasks.workunit.client.1.smithi190.stdout:1/810: dwrite d5/d4f/d65/d32/f4a [0,4194304] 0 2024-05-06T12:14:59.568 INFO:tasks.workunit.client.1.smithi190.stdout:4/811: creat d2/d11/d16/d8e/d7b/de8/d66/d6b/de3/dec/f112 x:0 0 0 2024-05-06T12:14:59.570 INFO:tasks.workunit.client.1.smithi190.stdout:1/811: unlink d5/d4f/d65/d32/c71 0 2024-05-06T12:14:59.570 INFO:tasks.workunit.client.1.smithi190.stdout:7/924: truncate d0/d1/d2/f5 419922 0 2024-05-06T12:14:59.572 INFO:tasks.workunit.client.1.smithi190.stdout:4/812: mkdir d2/d11/d16/d8e/d80/d113 0 2024-05-06T12:14:59.573 INFO:tasks.workunit.client.1.smithi190.stdout:1/812: read - d5/d4f/d65/d2b/fa8 zero size 2024-05-06T12:14:59.573 INFO:tasks.workunit.client.1.smithi190.stdout:7/925: write d0/d1/d18/d4e/d87/df1/f101 [1029162,82786] 0 2024-05-06T12:14:59.574 INFO:tasks.workunit.client.1.smithi190.stdout:4/813: truncate d2/d11/d16/d31/ff0 668587 0 2024-05-06T12:14:59.578 INFO:tasks.workunit.client.1.smithi190.stdout:7/926: link d0/d1/d18/d2b/f143 d0/d1/d32/d45/f147 0 2024-05-06T12:14:59.579 INFO:tasks.workunit.client.1.smithi190.stdout:7/927: chown d0/d1/d2/d84/dc6/dfb 62220155 1 2024-05-06T12:14:59.580 INFO:tasks.workunit.client.1.smithi190.stdout:4/814: link d2/d11/d16/d8e/c84 d2/d11/d16/d31/c114 0 2024-05-06T12:14:59.581 INFO:tasks.workunit.client.1.smithi190.stdout:4/815: dread - d2/d11/d16/f101 zero size 2024-05-06T12:14:59.582 INFO:tasks.workunit.client.1.smithi190.stdout:7/928: chown d0/d1/d32/d45/d70/d33/df7/d10e/f91 54798 1 2024-05-06T12:14:59.582 INFO:tasks.workunit.client.1.smithi190.stdout:4/816: read - d2/d11/d16/d8e/d80/da0/db2/fc5 zero size 2024-05-06T12:14:59.583 INFO:tasks.workunit.client.1.smithi190.stdout:4/817: readlink d2/d10/l76 0 2024-05-06T12:14:59.584 INFO:tasks.workunit.client.1.smithi190.stdout:4/818: write d2/d11/d16/d41/f65 [178683,50476] 0 2024-05-06T12:14:59.586 INFO:tasks.workunit.client.1.smithi190.stdout:7/929: getdents d0/da/d36/d53/d77 0 2024-05-06T12:14:59.588 INFO:tasks.workunit.client.1.smithi190.stdout:4/819: symlink d2/d11/d16/d8e/d7b/de8/d9e/d6e/daa/ddc/l115 0 2024-05-06T12:14:59.591 INFO:tasks.workunit.client.1.smithi190.stdout:4/820: fdatasync d2/d11/d16/d8e/d80/f95 0 2024-05-06T12:14:59.594 INFO:tasks.workunit.client.1.smithi190.stdout:7/930: getdents d0/da/d36/d53/d77/d8c/dd1/d60 0 2024-05-06T12:14:59.598 INFO:tasks.workunit.client.1.smithi190.stdout:4/821: creat d2/d11/d16/d8e/d7b/de8/f116 x:0 0 0 2024-05-06T12:14:59.599 INFO:tasks.workunit.client.1.smithi190.stdout:4/822: fdatasync d2/d11/d16/d8e/d7b/de8/d9e/d6e/d78/fc3 0 2024-05-06T12:14:59.599 INFO:tasks.workunit.client.1.smithi190.stdout:7/931: rename d0/d1/d32/d45/d70/d33/df7/d12b to d0/d1/d32/d45/d70/d35/d148 0 2024-05-06T12:14:59.600 INFO:tasks.workunit.client.1.smithi190.stdout:4/823: write d2/d11/d16/f38 [2459439,113155] 0 2024-05-06T12:14:59.601 INFO:tasks.workunit.client.1.smithi190.stdout:4/824: chown d2/d11/d16/d4e 637 1 2024-05-06T12:14:59.605 INFO:tasks.workunit.client.1.smithi190.stdout:7/932: mkdir d0/d1/d32/d45/d70/d33/d149 0 2024-05-06T12:14:59.606 INFO:tasks.workunit.client.1.smithi190.stdout:4/825: creat d2/d11/d16/d8e/d80/da0/db2/f117 x:0 0 0 2024-05-06T12:14:59.608 INFO:tasks.workunit.client.1.smithi190.stdout:7/933: creat d0/d1/d18/d6c/de1/f14a x:0 0 0 2024-05-06T12:14:59.610 INFO:tasks.workunit.client.1.smithi190.stdout:4/826: link d2/d11/d16/d4e/fd6 d2/d11/d16/d8e/d80/da0/db2/f118 0 2024-05-06T12:14:59.611 INFO:tasks.workunit.client.1.smithi190.stdout:7/934: symlink d0/da/d36/d53/l14b 0 2024-05-06T12:14:59.612 INFO:tasks.workunit.client.1.smithi190.stdout:4/827: fsync d2/d11/d16/d8e/d7b/de8/d9e/d6e/fc4 0 2024-05-06T12:14:59.613 INFO:tasks.workunit.client.1.smithi190.stdout:7/935: creat d0/d1/d2/d84/dc6/d10c/d11f/f14c x:0 0 0 2024-05-06T12:14:59.615 INFO:tasks.workunit.client.1.smithi190.stdout:4/828: creat d2/f119 x:0 0 0 2024-05-06T12:14:59.616 INFO:tasks.workunit.client.1.smithi190.stdout:7/936: mkdir d0/d1/d32/d45/d70/d33/d57/d14d 0 2024-05-06T12:14:59.617 INFO:tasks.workunit.client.1.smithi190.stdout:4/829: mkdir d2/d10/d11a 0 2024-05-06T12:14:59.617 INFO:tasks.workunit.client.1.smithi190.stdout:7/937: chown d0/d1/d2/d84/f130 365933 1 2024-05-06T12:14:59.619 INFO:tasks.workunit.client.1.smithi190.stdout:7/938: mkdir d0/d1/d32/db0/d14e 0 2024-05-06T12:14:59.620 INFO:tasks.workunit.client.1.smithi190.stdout:7/939: chown d0/f40 192 1 2024-05-06T12:14:59.622 INFO:tasks.workunit.client.1.smithi190.stdout:4/830: rmdir d2/d11/d16/d8e/d7b/de8/d66/d6b/de3/dec/d106 0 2024-05-06T12:14:59.624 INFO:tasks.workunit.client.1.smithi190.stdout:4/831: write d2/d10/fc0 [1772885,101055] 0 2024-05-06T12:14:59.628 INFO:tasks.workunit.client.1.smithi190.stdout:4/832: truncate d2/d11/d16/d8e/d7b/de8/d9e/d6e/f92 1011820 0 2024-05-06T12:14:59.635 INFO:tasks.workunit.client.1.smithi190.stdout:7/940: truncate d0/d1/d2/f88 840198 0 2024-05-06T12:14:59.636 INFO:tasks.workunit.client.1.smithi190.stdout:4/833: truncate d2/d11/d16/d8e/d7b/de8/d9e/d6e/f92 1917624 0 2024-05-06T12:14:59.637 INFO:tasks.workunit.client.1.smithi190.stdout:7/941: symlink d0/d14/d6a/l14f 0 2024-05-06T12:14:59.639 INFO:tasks.workunit.client.1.smithi190.stdout:4/834: creat d2/d11/d16/d8e/d7b/de8/d66/d6b/da4/dfa/d10f/f11b x:0 0 0 2024-05-06T12:14:59.640 INFO:tasks.workunit.client.1.smithi190.stdout:7/942: getdents d0/da/d36/d53/d77/d8c/dd1/d60 0 2024-05-06T12:14:59.642 INFO:tasks.workunit.client.1.smithi190.stdout:4/835: creat d2/d11/d16/d8e/d7b/de8/d9e/d6e/daa/f11c x:0 0 0 2024-05-06T12:14:59.647 INFO:tasks.workunit.client.1.smithi190.stdout:4/836: dwrite d2/d11/d16/f38 [0,4194304] 0 2024-05-06T12:14:59.649 INFO:tasks.workunit.client.1.smithi190.stdout:7/943: dread d0/d1/fb [0,4194304] 0 2024-05-06T12:14:59.649 INFO:tasks.workunit.client.1.smithi190.stdout:4/837: mknod d2/d11/d16/d8e/d80/da0/db2/c11d 0 2024-05-06T12:14:59.651 INFO:tasks.workunit.client.1.smithi190.stdout:7/944: chown d0/d1/d32/d45/d70/d33/df7/d10e/fa1 2408 1 2024-05-06T12:14:59.652 INFO:tasks.workunit.client.1.smithi190.stdout:4/838: mkdir d2/d10/d11a/d11e 0 2024-05-06T12:14:59.653 INFO:tasks.workunit.client.1.smithi190.stdout:4/839: chown d2/d11/d16/d31/f105 13050 1 2024-05-06T12:14:59.654 INFO:tasks.workunit.client.1.smithi190.stdout:7/945: creat d0/d1/d32/daa/f150 x:0 0 0 2024-05-06T12:14:59.654 INFO:tasks.workunit.client.1.smithi190.stdout:4/840: readlink d2/lc 0 2024-05-06T12:14:59.655 INFO:tasks.workunit.client.1.smithi190.stdout:4/841: chown d2/d11 48671853 1 2024-05-06T12:14:59.655 INFO:tasks.workunit.client.1.smithi190.stdout:7/946: fsync d0/d1/d18/d4e/d87/df1/f101 0 2024-05-06T12:14:59.657 INFO:tasks.workunit.client.1.smithi190.stdout:7/947: mknod d0/d1/d2/d11e/c151 0 2024-05-06T12:14:59.658 INFO:tasks.workunit.client.1.smithi190.stdout:4/842: truncate d2/d11/d16/d8e/d80/da0/db2/fc5 971980 0 2024-05-06T12:14:59.661 INFO:tasks.workunit.client.1.smithi190.stdout:4/843: creat d2/d11/d16/d8e/d7b/de8/dd1/f11f x:0 0 0 2024-05-06T12:14:59.661 INFO:tasks.workunit.client.1.smithi190.stdout:4/844: readlink d2/d10/l22 0 2024-05-06T12:14:59.670 INFO:tasks.workunit.client.1.smithi190.stdout:4/845: truncate d2/d11/d16/d8e/d80/da0/db2/fc5 600044 0 2024-05-06T12:14:59.675 INFO:tasks.workunit.client.1.smithi190.stdout:4/846: mkdir d2/d11/d16/d8e/d7b/de8/d66/d6b/da4/dfa/d10f/d120 0 2024-05-06T12:14:59.675 INFO:tasks.workunit.client.1.smithi190.stdout:7/948: dwrite d0/da/d36/fe5 [0,4194304] 0 2024-05-06T12:14:59.683 INFO:tasks.workunit.client.1.smithi190.stdout:5/932: sync 2024-05-06T12:14:59.683 INFO:tasks.workunit.client.1.smithi190.stdout:7/949: creat d0/da/d36/d53/d77/d8c/f152 x:0 0 0 2024-05-06T12:14:59.683 INFO:tasks.workunit.client.1.smithi190.stdout:4/847: truncate d2/f64 4200195 0 2024-05-06T12:14:59.683 INFO:tasks.workunit.client.1.smithi190.stdout:7/950: symlink d0/d1/d2/d84/dc6/d10c/l153 0 2024-05-06T12:14:59.684 INFO:tasks.workunit.client.1.smithi190.stdout:5/933: read d2/da/d59/d5f/d6e/daa/fe4 [181247,76580] 0 2024-05-06T12:14:59.684 INFO:tasks.workunit.client.1.smithi190.stdout:4/848: write d2/d11/d16/d8e/f57 [2427700,29369] 0 2024-05-06T12:14:59.685 INFO:tasks.workunit.client.1.smithi190.stdout:5/934: creat d2/d10e/f12e x:0 0 0 2024-05-06T12:14:59.686 INFO:tasks.workunit.client.1.smithi190.stdout:7/951: chown d0/d1/d18/d2b/la9 109 1 2024-05-06T12:14:59.687 INFO:tasks.workunit.client.1.smithi190.stdout:4/849: creat d2/d11/d16/d8e/d7b/de8/d82/f121 x:0 0 0 2024-05-06T12:14:59.687 INFO:tasks.workunit.client.1.smithi190.stdout:5/935: chown d2/da/d59/d5f/d2e/d3b/d5a/c12c 772027 1 2024-05-06T12:14:59.688 INFO:tasks.workunit.client.1.smithi190.stdout:7/952: chown d0/d1/d32/d45/d70/d3b/f94 45511 1 2024-05-06T12:14:59.691 INFO:tasks.workunit.client.1.smithi190.stdout:5/936: creat d2/da/d59/d5f/d2e/d3b/d76/f12f x:0 0 0 2024-05-06T12:14:59.691 INFO:tasks.workunit.client.1.smithi190.stdout:7/953: read d0/da/d36/d53/d77/d8c/dd1/d60/fa8 [2983274,27143] 0 2024-05-06T12:14:59.692 INFO:tasks.workunit.client.1.smithi190.stdout:4/850: dread d2/d11/d16/d8e/d80/f9a [0,4194304] 0 2024-05-06T12:14:59.694 INFO:tasks.workunit.client.1.smithi190.stdout:7/954: mknod d0/d1/d32/d45/d70/d33/c154 0 2024-05-06T12:14:59.695 INFO:tasks.workunit.client.1.smithi190.stdout:5/937: rename d2/da/d59/d5f/d6e/lfa to d2/d120/l130 0 2024-05-06T12:14:59.696 INFO:tasks.workunit.client.1.smithi190.stdout:7/955: mknod d0/d1/d18/d4e/d87/df1/c155 0 2024-05-06T12:14:59.698 INFO:tasks.workunit.client.1.smithi190.stdout:5/938: write d2/da/d59/d5f/d2e/d3b/d76/f12f [365803,85374] 0 2024-05-06T12:14:59.699 INFO:tasks.workunit.client.1.smithi190.stdout:7/956: dread - d0/d1/d32/d45/d70/d3b/f137 zero size 2024-05-06T12:14:59.699 INFO:tasks.workunit.client.1.smithi190.stdout:4/851: dwrite d2/f32 [0,4194304] 0 2024-05-06T12:14:59.701 INFO:tasks.workunit.client.1.smithi190.stdout:7/957: write d0/d1/d18/d37/fee [522057,124685] 0 2024-05-06T12:14:59.704 INFO:tasks.workunit.client.1.smithi190.stdout:4/852: mkdir d2/d11/d16/d8e/d7b/dd4/d122 0 2024-05-06T12:14:59.706 INFO:tasks.workunit.client.1.smithi190.stdout:5/939: creat d2/da/d59/d5f/d50/f131 x:0 0 0 2024-05-06T12:14:59.708 INFO:tasks.workunit.client.1.smithi190.stdout:7/958: link d0/d1/d32/d45/d70/d3b/f94 d0/d14/d6a/d8f/f156 0 2024-05-06T12:14:59.710 INFO:tasks.workunit.client.1.smithi190.stdout:4/853: rename d2/d11/d16/d8e/d7b/l91 to d2/d11/d16/d8e/d7b/dd4/d122/l123 0 2024-05-06T12:14:59.711 INFO:tasks.workunit.client.1.smithi190.stdout:5/940: dread d2/da/d59/d5f/d2e/d3b/fa5 [0,4194304] 0 2024-05-06T12:14:59.712 INFO:tasks.workunit.client.1.smithi190.stdout:5/941: readlink d2/da/d59/d5f/d2e/da7/d9d/lfd 0 2024-05-06T12:14:59.714 INFO:tasks.workunit.client.1.smithi190.stdout:4/854: creat d2/d11/d16/d8e/d80/d113/f124 x:0 0 0 2024-05-06T12:14:59.715 INFO:tasks.workunit.client.1.smithi190.stdout:4/855: dread - d2/d11/d16/f101 zero size 2024-05-06T12:14:59.717 INFO:tasks.workunit.client.1.smithi190.stdout:5/942: read d2/da/d59/d5f/d2e/da7/d9d/fda [361744,68106] 0 2024-05-06T12:14:59.718 INFO:tasks.workunit.client.1.smithi190.stdout:4/856: fsync d2/d11/d16/d8e/d7b/de8/d82/f121 0 2024-05-06T12:14:59.723 INFO:tasks.workunit.client.1.smithi190.stdout:5/943: mkdir d2/da/d59/d5f/d50/d6d/db3/d132 0 2024-05-06T12:14:59.723 INFO:tasks.workunit.client.1.smithi190.stdout:4/857: dwrite d2/d10/f63 [8388608,4194304] 0 2024-05-06T12:14:59.726 INFO:tasks.workunit.client.1.smithi190.stdout:5/944: fsync d2/da/d59/f94 0 2024-05-06T12:14:59.728 INFO:tasks.workunit.client.1.smithi190.stdout:4/858: mkdir d2/d11/d16/d8e/d7b/de8/d9e/d6e/daa/d125 0 2024-05-06T12:14:59.729 INFO:tasks.workunit.client.1.smithi190.stdout:4/859: fdatasync d2/d10/fb8 0 2024-05-06T12:14:59.733 INFO:tasks.workunit.client.1.smithi190.stdout:4/860: read d2/d11/d16/d8e/d7b/de8/fca [157406,7436] 0 2024-05-06T12:14:59.734 INFO:tasks.workunit.client.1.smithi190.stdout:5/945: mknod d2/da/d59/d5f/d2e/c133 0 2024-05-06T12:14:59.735 INFO:tasks.workunit.client.1.smithi190.stdout:5/946: write d2/f3 [1308158,95664] 0 2024-05-06T12:14:59.737 INFO:tasks.workunit.client.1.smithi190.stdout:5/947: chown d2/d27/c8c 763897566 1 2024-05-06T12:14:59.737 INFO:tasks.workunit.client.1.smithi190.stdout:7/959: dwrite d0/d1/d32/d45/f7d [0,4194304] 0 2024-05-06T12:14:59.746 INFO:tasks.workunit.client.1.smithi190.stdout:7/960: mknod d0/d1/d2/d84/d12f/c157 0 2024-05-06T12:14:59.747 INFO:tasks.workunit.client.1.smithi190.stdout:4/861: creat d2/d11/d16/d8e/d7b/de8/d66/d6b/f126 x:0 0 0 2024-05-06T12:14:59.748 INFO:tasks.workunit.client.1.smithi190.stdout:4/862: chown d2/d11/d16/d4e 27 1 2024-05-06T12:14:59.749 INFO:tasks.workunit.client.1.smithi190.stdout:4/863: chown d2/d11/d16/fe7 114652 1 2024-05-06T12:14:59.750 INFO:tasks.workunit.client.1.smithi190.stdout:7/961: getdents d0/d1/d18/d2b 0 2024-05-06T12:14:59.756 INFO:tasks.workunit.client.1.smithi190.stdout:9/903: sync 2024-05-06T12:14:59.756 INFO:tasks.workunit.client.1.smithi190.stdout:6/696: sync 2024-05-06T12:14:59.756 INFO:tasks.workunit.client.1.smithi190.stdout:2/942: sync 2024-05-06T12:14:59.760 INFO:tasks.workunit.client.1.smithi190.stdout:4/864: write d2/d11/d16/d31/f5c [1541285,121537] 0 2024-05-06T12:14:59.762 INFO:tasks.workunit.client.1.smithi190.stdout:9/904: stat db/dd/d16/l20 0 2024-05-06T12:14:59.764 INFO:tasks.workunit.client.1.smithi190.stdout:5/948: dwrite d2/da/d59/d5f/d2e/d3b/f42 [0,4194304] 0 2024-05-06T12:14:59.765 INFO:tasks.workunit.client.1.smithi190.stdout:4/865: write d2/d11/d16/d8e/d7b/de8/f116 [889749,58092] 0 2024-05-06T12:14:59.784 INFO:tasks.workunit.client.1.smithi190.stdout:2/943: rename d2/d21/f70 to d2/d8/dc/f149 0 2024-05-06T12:14:59.785 INFO:tasks.workunit.client.1.smithi190.stdout:6/697: truncate df/d17/d37/f3c 3446302 0 2024-05-06T12:14:59.785 INFO:tasks.workunit.client.1.smithi190.stdout:2/944: fsync d2/d21/d123/d79/d7e/fff 0 2024-05-06T12:14:59.785 INFO:tasks.workunit.client.1.smithi190.stdout:5/949: mknod d2/db6/c134 0 2024-05-06T12:14:59.787 INFO:tasks.workunit.client.1.smithi190.stdout:2/945: readlink d2/d8/d33/d46/dab/dbb/lfb 0 2024-05-06T12:14:59.788 INFO:tasks.workunit.client.1.smithi190.stdout:0/931: sync 2024-05-06T12:14:59.788 INFO:tasks.workunit.client.1.smithi190.stdout:1/813: sync 2024-05-06T12:14:59.789 INFO:tasks.workunit.client.1.smithi190.stdout:6/698: mknod df/d12/cf0 0 2024-05-06T12:14:59.790 INFO:tasks.workunit.client.1.smithi190.stdout:4/866: mknod d2/d11/d16/d41/c127 0 2024-05-06T12:14:59.791 INFO:tasks.workunit.client.1.smithi190.stdout:4/867: write d2/f32 [8071592,8736] 0 2024-05-06T12:14:59.791 INFO:tasks.workunit.client.1.smithi190.stdout:5/950: symlink d2/d10e/l135 0 2024-05-06T12:14:59.793 INFO:tasks.workunit.client.1.smithi190.stdout:4/868: chown d2/d11/d16/d4e/f107 23098150 1 2024-05-06T12:14:59.794 INFO:tasks.workunit.client.1.smithi190.stdout:0/932: dread - d9/d51/d93/ffe zero size 2024-05-06T12:14:59.796 INFO:tasks.workunit.client.1.smithi190.stdout:7/962: getdents d0/d1/d18/d4e/d87/df1 0 2024-05-06T12:14:59.801 INFO:tasks.workunit.client.1.smithi190.stdout:6/699: mkdir df/d2c/d28/deb/df1 0 2024-05-06T12:14:59.803 INFO:tasks.workunit.client.1.smithi190.stdout:2/946: creat d2/d8/f14a x:0 0 0 2024-05-06T12:14:59.803 INFO:tasks.workunit.client.1.smithi190.stdout:6/700: truncate df/d2c/d61/d87/fe3 278545 0 2024-05-06T12:14:59.805 INFO:tasks.workunit.client.1.smithi190.stdout:7/963: rmdir d0/d1/d2/d84/dc6/d10c/d11f 39 2024-05-06T12:14:59.806 INFO:tasks.workunit.client.1.smithi190.stdout:2/947: truncate d2/d8/d33/d46/db7/f13d 1038335 0 2024-05-06T12:14:59.809 INFO:tasks.workunit.client.1.smithi190.stdout:4/869: link d2/f59 d2/d11/d16/d8e/d7b/f128 0 2024-05-06T12:14:59.811 INFO:tasks.workunit.client.1.smithi190.stdout:6/701: symlink df/d2c/d1e/d57/d89/lf2 0 2024-05-06T12:14:59.811 INFO:tasks.workunit.client.1.smithi190.stdout:0/933: dread d9/dc/d28/d4c/f4b [0,4194304] 0 2024-05-06T12:14:59.814 INFO:tasks.workunit.client.1.smithi190.stdout:7/964: dwrite d0/d14/d6a/f142 [0,4194304] 0 2024-05-06T12:14:59.814 INFO:tasks.workunit.client.1.smithi190.stdout:4/870: chown d2/d11/d16/d8e/d7b/dd4/lda 22 1 2024-05-06T12:14:59.816 INFO:tasks.workunit.client.1.smithi190.stdout:5/951: dread d2/da/d59/d5f/d2e/d3b/f52 [0,4194304] 0 2024-05-06T12:14:59.819 INFO:tasks.workunit.client.1.smithi190.stdout:2/948: truncate d2/d21/d123/d79/d7e/fd4 980795 0 2024-05-06T12:14:59.820 INFO:tasks.workunit.client.1.smithi190.stdout:5/952: readlink d2/da/d59/d5f/d50/l7b 0 2024-05-06T12:14:59.820 INFO:tasks.workunit.client.1.smithi190.stdout:5/953: write d2/f124 [880482,100860] 0 2024-05-06T12:14:59.821 INFO:tasks.workunit.client.1.smithi190.stdout:5/954: chown d2/d6c 0 1 2024-05-06T12:14:59.822 INFO:tasks.workunit.client.1.smithi190.stdout:5/955: readlink d2/da/d59/d5f/d2e/da7/d9d/lfd 0 2024-05-06T12:14:59.825 INFO:tasks.workunit.client.1.smithi190.stdout:6/702: rmdir df/d2c/d1e/d57/d53/dd6 0 2024-05-06T12:14:59.828 INFO:tasks.workunit.client.1.smithi190.stdout:4/871: truncate d2/d11/d16/d31/f43 4882420 0 2024-05-06T12:14:59.832 INFO:tasks.workunit.client.1.smithi190.stdout:5/956: unlink d2/da/d59/d5f/d2e/d3b/d76/d9a/l121 0 2024-05-06T12:14:59.832 INFO:tasks.workunit.client.1.smithi190.stdout:5/957: chown d2/da/d59/d5f/d2e/c133 487 1 2024-05-06T12:14:59.833 INFO:tasks.workunit.client.1.smithi190.stdout:4/872: symlink d2/d11/d16/d8e/d7b/de8/dd1/l129 0 2024-05-06T12:14:59.833 INFO:tasks.workunit.client.1.smithi190.stdout:4/873: chown d2/f5e 22954 1 2024-05-06T12:14:59.836 INFO:tasks.workunit.client.1.smithi190.stdout:7/965: dread d0/d1/d18/d2b/f80 [0,4194304] 0 2024-05-06T12:14:59.836 INFO:tasks.workunit.client.1.smithi190.stdout:4/874: creat d2/d11/d16/d8e/d7b/de8/d9e/f12a x:0 0 0 2024-05-06T12:14:59.837 INFO:tasks.workunit.client.1.smithi190.stdout:5/958: creat d2/da/d59/d5f/f136 x:0 0 0 2024-05-06T12:14:59.839 INFO:tasks.workunit.client.1.smithi190.stdout:9/905: write db/d23/d29/fc6 [655686,106705] 0 2024-05-06T12:14:59.839 INFO:tasks.workunit.client.1.smithi190.stdout:7/966: mknod d0/d1/d32/daa/d129/c158 0 2024-05-06T12:14:59.839 INFO:tasks.workunit.client.1.smithi190.stdout:4/875: write d2/d11/d16/d8e/d7b/de8/d66/d6b/da4/dfa/d10f/f11b [193081,106319] 0 2024-05-06T12:14:59.843 INFO:tasks.workunit.client.1.smithi190.stdout:7/967: read d0/d1/d2/d84/fbf [352142,78180] 0 2024-05-06T12:14:59.845 INFO:tasks.workunit.client.1.smithi190.stdout:7/968: write d0/d1/d32/daa/f150 [65660,100031] 0 2024-05-06T12:14:59.847 INFO:tasks.workunit.client.1.smithi190.stdout:7/969: dread - d0/d1/d32/d45/d70/d3b/f145 zero size 2024-05-06T12:14:59.847 INFO:tasks.workunit.client.1.smithi190.stdout:9/906: truncate db/daa/fab 101725 0 2024-05-06T12:14:59.849 INFO:tasks.workunit.client.1.smithi190.stdout:5/959: rename d2/da/d59/f6a to d2/da/d59/d5f/d2e/d8b/f137 0 2024-05-06T12:14:59.851 INFO:tasks.workunit.client.1.smithi190.stdout:7/970: chown d0/d1/d18/d4e/db3/fe2 2407236 1 2024-05-06T12:14:59.853 INFO:tasks.workunit.client.1.smithi190.stdout:5/960: creat d2/d10e/f138 x:0 0 0 2024-05-06T12:14:59.853 INFO:tasks.workunit.client.1.smithi190.stdout:5/961: chown d2/da/d59 3 1 2024-05-06T12:14:59.854 INFO:tasks.workunit.client.1.smithi190.stdout:9/907: symlink db/dd/d5c/dac/deb/d10e/l12d 0 2024-05-06T12:14:59.858 INFO:tasks.workunit.client.1.smithi190.stdout:7/971: creat d0/d1/d32/d45/d70/d33/d57/d14d/f159 x:0 0 0 2024-05-06T12:14:59.858 INFO:tasks.workunit.client.1.smithi190.stdout:0/934: truncate d9/dcf/d21/f7e 2691801 0 2024-05-06T12:14:59.859 INFO:tasks.workunit.client.1.smithi190.stdout:7/972: dread - d0/da/d36/d53/d77/d8c/dd1/d60/dac/f115 zero size 2024-05-06T12:14:59.860 INFO:tasks.workunit.client.1.smithi190.stdout:9/908: mkdir db/d12e 0 2024-05-06T12:14:59.861 INFO:tasks.workunit.client.1.smithi190.stdout:9/909: fdatasync db/dd/df/f107 0 2024-05-06T12:14:59.863 INFO:tasks.workunit.client.1.smithi190.stdout:5/962: creat d2/da/d59/d5f/d50/f139 x:0 0 0 2024-05-06T12:14:59.863 INFO:tasks.workunit.client.1.smithi190.stdout:2/949: write d2/d4c/fd2 [3432798,120240] 0 2024-05-06T12:14:59.865 INFO:tasks.workunit.client.1.smithi190.stdout:7/973: creat d0/d1/d2/f15a x:0 0 0 2024-05-06T12:14:59.869 INFO:tasks.workunit.client.1.smithi190.stdout:2/950: symlink d2/d4c/l14b 0 2024-05-06T12:14:59.873 INFO:tasks.workunit.client.1.smithi190.stdout:9/910: mkdir db/dd/df/dfe/d87/d22/d49/d88/d12f 0 2024-05-06T12:14:59.882 INFO:tasks.workunit.client.1.smithi190.stdout:4/876: dwrite d2/d11/d16/d8e/d80/da0/db2/fc5 [0,4194304] 0 2024-05-06T12:14:59.882 INFO:tasks.workunit.client.1.smithi190.stdout:9/911: creat db/d23/d7e/f130 x:0 0 0 2024-05-06T12:14:59.883 INFO:tasks.workunit.client.1.smithi190.stdout:0/935: link d9/dc/d48/l61 d9/dc/d28/d4c/d96/l14a 0 2024-05-06T12:14:59.883 INFO:tasks.workunit.client.1.smithi190.stdout:5/963: dwrite d2/da/d59/d5f/d2e/da3/f11b [4194304,4194304] 0 2024-05-06T12:14:59.895 INFO:tasks.workunit.client.1.smithi190.stdout:0/936: rename d9/dcf/d21/f4f to d9/dc/db6/d11b/f14b 0 2024-05-06T12:14:59.899 INFO:tasks.workunit.client.1.smithi190.stdout:4/877: dwrite d2/d11/d16/d8e/d7b/de8/d9e/d6e/daa/fc8 [0,4194304] 0 2024-05-06T12:14:59.903 INFO:tasks.workunit.client.1.smithi190.stdout:5/964: rename d2/da/d59/d5f/d2e to d2/da/d59/d5f/dfb/d13a 0 2024-05-06T12:14:59.904 INFO:tasks.workunit.client.1.smithi190.stdout:4/878: fdatasync d2/d11/d16/d8e/d7b/de8/d9e/f12a 0 2024-05-06T12:14:59.905 INFO:tasks.workunit.client.1.smithi190.stdout:0/937: dwrite d9/fb [8388608,4194304] 0 2024-05-06T12:14:59.906 INFO:tasks.workunit.client.1.smithi190.stdout:7/974: write d0/d1/d32/d45/d70/d3b/f94 [1287114,114226] 0 2024-05-06T12:14:59.908 INFO:tasks.workunit.client.1.smithi190.stdout:5/965: dread - d2/da/d59/d5f/dfb/d13a/d3b/d72/dec/f123 zero size 2024-05-06T12:14:59.909 INFO:tasks.workunit.client.1.smithi190.stdout:0/938: write d9/dcf/d56/f149 [1008997,86800] 0 2024-05-06T12:14:59.915 INFO:tasks.workunit.client.1.smithi190.stdout:4/879: mknod d2/d11/d16/d8e/d7b/de8/d66/c12b 0 2024-05-06T12:14:59.917 INFO:tasks.workunit.client.1.smithi190.stdout:5/966: mknod d2/d65/d83/df3/c13b 0 2024-05-06T12:14:59.919 INFO:tasks.workunit.client.1.smithi190.stdout:2/951: write d2/d8/d19/f8a [1566304,59361] 0 2024-05-06T12:14:59.922 INFO:tasks.workunit.client.1.smithi190.stdout:5/967: read d2/f3 [4072475,28330] 0 2024-05-06T12:14:59.926 INFO:tasks.workunit.client.1.smithi190.stdout:2/952: mknod d2/d8/d33/d46/dab/dac/c14c 0 2024-05-06T12:14:59.927 INFO:tasks.workunit.client.1.smithi190.stdout:5/968: symlink d2/da/d59/d5f/dfb/d13a/l13c 0 2024-05-06T12:14:59.928 INFO:tasks.workunit.client.1.smithi190.stdout:2/953: chown d2/d8/d33/d46/db7/d126 796 1 2024-05-06T12:14:59.928 INFO:tasks.workunit.client.1.smithi190.stdout:5/969: fsync d2/d10e/f138 0 2024-05-06T12:14:59.929 INFO:tasks.workunit.client.1.smithi190.stdout:2/954: readlink d2/d8/d33/dc2/dcd/dfe/l3f 0 2024-05-06T12:14:59.934 INFO:tasks.workunit.client.1.smithi190.stdout:2/955: dread d2/d8/d19/d4f/fe3 [0,4194304] 0 2024-05-06T12:14:59.940 INFO:tasks.workunit.client.1.smithi190.stdout:5/970: link d2/da/d59/d5f/dfb/d13a/d3b/d5a/fdc d2/da/f13d 0 2024-05-06T12:14:59.942 INFO:tasks.workunit.client.1.smithi190.stdout:5/971: dread d2/da/d59/d5f/dfb/d13a/d3b/d76/d9a/f11f [0,4194304] 0 2024-05-06T12:14:59.943 INFO:tasks.workunit.client.1.smithi190.stdout:2/956: rename d2/d8/d33/dc2/dcd/f145 to d2/d21/d123/d79/db2/f14d 0 2024-05-06T12:14:59.944 INFO:tasks.workunit.client.1.smithi190.stdout:5/972: truncate d2/d65/f114 1561920 0 2024-05-06T12:14:59.945 INFO:tasks.workunit.client.1.smithi190.stdout:2/957: write d2/d4c/fd2 [1164021,78093] 0 2024-05-06T12:14:59.948 INFO:tasks.workunit.client.1.smithi190.stdout:2/958: chown d2/d4c/d10a/d12b/f12f 12366 1 2024-05-06T12:14:59.950 INFO:tasks.workunit.client.1.smithi190.stdout:4/880: truncate d2/d11/d16/f38 1707293 0 2024-05-06T12:14:59.952 INFO:tasks.workunit.client.1.smithi190.stdout:0/939: dread d9/d51/f55 [0,4194304] 0 2024-05-06T12:14:59.952 INFO:tasks.workunit.client.1.smithi190.stdout:2/959: write d2/d8/dc/f95 [2999564,15239] 0 2024-05-06T12:14:59.952 INFO:tasks.workunit.client.1.smithi190.stdout:5/973: mknod d2/da/d59/d5f/dfb/d13a/d3b/d76/c13e 0 2024-05-06T12:14:59.955 INFO:tasks.workunit.client.1.smithi190.stdout:7/975: dwrite d0/d4/f6b [0,4194304] 0 2024-05-06T12:14:59.958 INFO:tasks.workunit.client.1.smithi190.stdout:7/976: write d0/da/d36/d119/f12c [638971,108741] 0 2024-05-06T12:14:59.959 INFO:tasks.workunit.client.1.smithi190.stdout:5/974: write d2/d6c/d7d/f125 [911886,18173] 0 2024-05-06T12:14:59.960 INFO:tasks.workunit.client.1.smithi190.stdout:2/960: symlink d2/d21/d123/d79/db2/l14e 0 2024-05-06T12:14:59.964 INFO:tasks.workunit.client.1.smithi190.stdout:0/940: dwrite d9/fb [8388608,4194304] 0 2024-05-06T12:14:59.965 INFO:tasks.workunit.client.1.smithi190.stdout:7/977: truncate d0/da/d36/d53/f98 819673 0 2024-05-06T12:14:59.966 INFO:tasks.workunit.client.1.smithi190.stdout:2/961: symlink d2/d8/d33/l14f 0 2024-05-06T12:14:59.968 INFO:tasks.workunit.client.1.smithi190.stdout:5/975: symlink d2/da/d59/d5f/l13f 0 2024-05-06T12:14:59.968 INFO:tasks.workunit.client.1.smithi190.stdout:5/976: stat d2/d6c/fc5 0 2024-05-06T12:14:59.969 INFO:tasks.workunit.client.1.smithi190.stdout:0/941: mkdir d9/dc/d48/d50/d14c 0 2024-05-06T12:14:59.970 INFO:tasks.workunit.client.1.smithi190.stdout:7/978: fdatasync d0/d1/d18/f127 0 2024-05-06T12:14:59.971 INFO:tasks.workunit.client.1.smithi190.stdout:2/962: mkdir d2/d8/d33/d46/dab/dbb/d150 0 2024-05-06T12:14:59.973 INFO:tasks.workunit.client.1.smithi190.stdout:5/977: mkdir d2/da/d59/d5f/d50/d6d/d140 0 2024-05-06T12:14:59.975 INFO:tasks.workunit.client.1.smithi190.stdout:2/963: unlink d2/d8/f14a 0 2024-05-06T12:14:59.979 INFO:tasks.workunit.client.1.smithi190.stdout:5/978: unlink d2/da/d59/d5f/d50/fe1 0 2024-05-06T12:14:59.980 INFO:tasks.workunit.client.1.smithi190.stdout:5/979: readlink d2/da/d59/la6 0 2024-05-06T12:14:59.981 INFO:tasks.workunit.client.1.smithi190.stdout:0/942: rename d9/dcf/l32 to d9/dc/d48/d50/d83/l14d 0 2024-05-06T12:14:59.982 INFO:tasks.workunit.client.1.smithi190.stdout:7/979: symlink d0/d1/d32/d45/d70/d33/d149/l15b 0 2024-05-06T12:14:59.983 INFO:tasks.workunit.client.1.smithi190.stdout:2/964: dwrite d2/d8/d33/d46/dab/f6d [0,4194304] 0 2024-05-06T12:14:59.987 INFO:tasks.workunit.client.1.smithi190.stdout:0/943: creat d9/dc/db6/d11b/f14e x:0 0 0 2024-05-06T12:14:59.987 INFO:tasks.workunit.client.1.smithi190.stdout:0/944: read - d9/d6f/f11d zero size 2024-05-06T12:14:59.989 INFO:tasks.workunit.client.1.smithi190.stdout:5/980: getdents d2/d6c/d7d 0 2024-05-06T12:14:59.989 INFO:tasks.workunit.client.1.smithi190.stdout:0/945: fdatasync d9/dc/d48/d50/d83/f94 0 2024-05-06T12:14:59.992 INFO:tasks.workunit.client.1.smithi190.stdout:2/965: readlink d2/d8/d19/d4f/l58 0 2024-05-06T12:14:59.997 INFO:tasks.workunit.client.1.smithi190.stdout:5/981: dwrite d2/da/f117 [0,4194304] 0 2024-05-06T12:15:00.000 INFO:tasks.workunit.client.1.smithi190.stdout:2/966: mkdir d2/d8/d33/d46/db7/d151 0 2024-05-06T12:15:00.000 INFO:tasks.workunit.client.1.smithi190.stdout:5/982: write d2/da/d59/d5f/dfb/d13a/d3b/f42 [396135,83828] 0 2024-05-06T12:15:00.003 INFO:tasks.workunit.client.1.smithi190.stdout:5/983: chown d2/da/d59/d5f/dfb/d13a/d3b/d5a/c8a 1530 1 2024-05-06T12:15:00.003 INFO:tasks.workunit.client.1.smithi190.stdout:7/980: dread d0/da/d36/d53/f98 [0,4194304] 0 2024-05-06T12:15:00.007 INFO:tasks.workunit.client.1.smithi190.stdout:5/984: creat d2/da/d59/d5f/dfb/d13a/da3/f141 x:0 0 0 2024-05-06T12:15:00.008 INFO:tasks.workunit.client.1.smithi190.stdout:7/981: write d0/d1/d18/d2b/f80 [3664722,108692] 0 2024-05-06T12:15:00.010 INFO:tasks.workunit.client.1.smithi190.stdout:7/982: chown d0/d1/d32/db0/l10f 1562560 1 2024-05-06T12:15:00.010 INFO:tasks.workunit.client.1.smithi190.stdout:2/967: creat d2/d146/f152 x:0 0 0 2024-05-06T12:15:00.011 INFO:tasks.workunit.client.1.smithi190.stdout:2/968: write d2/f38 [2057801,33724] 0 2024-05-06T12:15:00.017 INFO:tasks.workunit.client.1.smithi190.stdout:7/983: symlink d0/d1/d32/db0/d14e/l15c 0 2024-05-06T12:15:00.027 INFO:tasks.workunit.client.1.smithi190.stdout:7/984: rename d0/da/d36/d53 to d0/d1/d32/d15d 0 2024-05-06T12:15:00.033 INFO:tasks.workunit.client.1.smithi190.stdout:7/985: rename d0/d1/d32/d45/d70/d33/df7 to d0/d1/d2/d84/dc6/dfb/d15e 0 2024-05-06T12:15:00.034 INFO:tasks.workunit.client.1.smithi190.stdout:5/985: write d2/da/d59/d5f/dfb/d13a/d3b/d72/dec/f118 [135539,121031] 0 2024-05-06T12:15:00.036 INFO:tasks.workunit.client.1.smithi190.stdout:5/986: fdatasync d2/dea/ff1 0 2024-05-06T12:15:00.036 INFO:tasks.workunit.client.1.smithi190.stdout:7/986: symlink d0/d1/d32/d45/d70/d33/d149/l15f 0 2024-05-06T12:15:00.040 INFO:tasks.workunit.client.1.smithi190.stdout:7/987: truncate d0/d1/d18/d4e/db3/fe2 797206 0 2024-05-06T12:15:00.043 INFO:tasks.workunit.client.1.smithi190.stdout:7/988: rename d0/d14/lad to d0/d1/d18/d6c/de1/l160 0 2024-05-06T12:15:00.045 INFO:tasks.workunit.client.1.smithi190.stdout:7/989: mkdir d0/d1/d18/d4e/d161 0 2024-05-06T12:15:00.045 INFO:tasks.workunit.client.1.smithi190.stdout:7/990: fdatasync d0/d1/d2/d84/f126 0 2024-05-06T12:15:00.047 INFO:tasks.workunit.client.1.smithi190.stdout:7/991: chown d0/d1/d2/d84/dc6/d10c 168 1 2024-05-06T12:15:00.050 INFO:tasks.workunit.client.1.smithi190.stdout:7/992: link d0/d1/d18/d2b/f30 d0/d1/d18/d6c/de1/de7/f162 0 2024-05-06T12:15:00.051 INFO:tasks.workunit.client.1.smithi190.stdout:7/993: write d0/d1/d32/daa/fcb [2574168,56613] 0 2024-05-06T12:15:00.052 INFO:tasks.workunit.client.1.smithi190.stdout:5/987: dread d2/fe6 [0,4194304] 0 2024-05-06T12:15:00.053 INFO:tasks.workunit.client.1.smithi190.stdout:7/994: mknod d0/da/c163 0 2024-05-06T12:15:00.055 INFO:tasks.workunit.client.1.smithi190.stdout:5/988: dread d2/fe6 [0,4194304] 0 2024-05-06T12:15:00.056 INFO:tasks.workunit.client.1.smithi190.stdout:7/995: truncate d0/d1/d32/d15d/d77/d8c/dd1/d60/dac/f11a 818703 0 2024-05-06T12:15:00.056 INFO:tasks.workunit.client.1.smithi190.stdout:5/989: read d2/da/f4a [4212757,39262] 0 2024-05-06T12:15:00.058 INFO:tasks.workunit.client.1.smithi190.stdout:5/990: rmdir d2/dea 39 2024-05-06T12:15:00.060 INFO:tasks.workunit.client.1.smithi190.stdout:7/996: truncate d0/d1/d18/d2b/f83 1443266 0 2024-05-06T12:15:00.063 INFO:tasks.workunit.client.1.smithi190.stdout:7/997: stat d0/d14/d5d/ce0 0 2024-05-06T12:15:00.071 INFO:tasks.workunit.client.1.smithi190.stdout:5/991: dwrite d2/d27/f20 [4194304,4194304] 0 2024-05-06T12:15:00.076 INFO:tasks.workunit.client.1.smithi190.stdout:7/998: dread d0/d1/d32/d45/d70/d3b/f48 [0,4194304] 0 2024-05-06T12:15:00.077 INFO:tasks.workunit.client.1.smithi190.stdout:7/999: dread - d0/d1/d18/f127 zero size 2024-05-06T12:15:00.078 INFO:tasks.workunit.client.1.smithi190.stdout:5/992: rename d2/da/d59/d5f/dfb/d13a/c79 to d2/da/d59/d5f/dfb/d13a/d3b/d76/c142 0 2024-05-06T12:15:00.081 INFO:tasks.workunit.client.1.smithi190.stdout:5/993: getdents d2/d65/d83/df3 0 2024-05-06T12:15:00.083 INFO:tasks.workunit.client.1.smithi190.stdout:5/994: creat d2/da/d59/d5f/dfb/f143 x:0 0 0 2024-05-06T12:15:00.088 INFO:tasks.workunit.client.1.smithi190.stdout:5/995: dwrite d2/f44 [0,4194304] 0 2024-05-06T12:15:00.089 INFO:tasks.workunit.client.1.smithi190.stdout:5/996: chown d2/c4d 3 1 2024-05-06T12:15:00.094 INFO:tasks.workunit.client.1.smithi190.stdout:5/997: rename d2/db6/c134 to d2/da/d59/d5f/dfb/d13a/da7/d9d/c144 0 2024-05-06T12:15:00.095 INFO:tasks.workunit.client.1.smithi190.stdout:5/998: write d2/da/d59/d5f/d6e/daa/f103 [62962,17290] 0 2024-05-06T12:15:00.098 INFO:tasks.workunit.client.1.smithi190.stdout:5/999: creat d2/d65/d83/df3/d128/f145 x:0 0 0 2024-05-06T12:15:00.200 INFO:tasks.workunit.client.1.smithi190.stdout:4/881: sync 2024-05-06T12:15:00.200 INFO:tasks.workunit.client.1.smithi190.stdout:1/814: sync 2024-05-06T12:15:00.200 INFO:tasks.workunit.client.1.smithi190.stdout:6/703: sync 2024-05-06T12:15:00.200 INFO:tasks.workunit.client.1.smithi190.stdout:9/912: sync 2024-05-06T12:15:00.201 INFO:tasks.workunit.client.1.smithi190.stdout:1/815: chown d5/d15/d1e/f27 175840152 1 2024-05-06T12:15:00.201 INFO:tasks.workunit.client.1.smithi190.stdout:6/704: read - df/d2c/d28/de0/f9d zero size 2024-05-06T12:15:00.202 INFO:tasks.workunit.client.1.smithi190.stdout:9/913: mknod db/dd/df/c131 0 2024-05-06T12:15:00.203 INFO:tasks.workunit.client.1.smithi190.stdout:1/816: dread - d5/d4f/d65/d32/d47/d53/fd7 zero size 2024-05-06T12:15:00.205 INFO:tasks.workunit.client.1.smithi190.stdout:6/705: rename df/d2c/d1e/c7b to df/dee/cf3 0 2024-05-06T12:15:00.207 INFO:tasks.workunit.client.1.smithi190.stdout:1/817: fdatasync d5/d4f/d65/d24/d43/de2/d114/f86 0 2024-05-06T12:15:00.208 INFO:tasks.workunit.client.1.smithi190.stdout:4/882: dwrite d2/d11/d16/d8e/fd3 [0,4194304] 0 2024-05-06T12:15:00.210 INFO:tasks.workunit.client.1.smithi190.stdout:6/706: unlink df/d2c/d61/f70 0 2024-05-06T12:15:00.212 INFO:tasks.workunit.client.1.smithi190.stdout:6/707: write df/d2c/d61/d84/d8f/fed [21781,128486] 0 2024-05-06T12:15:00.215 INFO:tasks.workunit.client.1.smithi190.stdout:1/818: rename d5/d6b/d8f/fcf to d5/f117 0 2024-05-06T12:15:00.217 INFO:tasks.workunit.client.1.smithi190.stdout:6/708: stat df/f82 0 2024-05-06T12:15:00.217 INFO:tasks.workunit.client.1.smithi190.stdout:1/819: write d5/d4f/d65/d32/dac/fcb [101287,62572] 0 2024-05-06T12:15:00.218 INFO:tasks.workunit.client.1.smithi190.stdout:1/820: chown d5/d4f/d65/d2b/fef 13 1 2024-05-06T12:15:00.219 INFO:tasks.workunit.client.1.smithi190.stdout:4/883: symlink d2/d11/d16/d8e/d7b/de8/d66/d6b/da4/l12c 0 2024-05-06T12:15:00.223 INFO:tasks.workunit.client.1.smithi190.stdout:4/884: symlink d2/d11/d16/d8e/d7b/de8/d9e/d6e/daa/ddc/l12d 0 2024-05-06T12:15:00.223 INFO:tasks.workunit.client.1.smithi190.stdout:4/885: chown d2/l2c 16529763 1 2024-05-06T12:15:00.226 INFO:tasks.workunit.client.1.smithi190.stdout:4/886: mkdir d2/d11/d16/d12e 0 2024-05-06T12:15:00.229 INFO:tasks.workunit.client.1.smithi190.stdout:9/914: dread db/dd/df/dfe/d87/d22/d49/f98 [0,4194304] 0 2024-05-06T12:15:00.231 INFO:tasks.workunit.client.1.smithi190.stdout:4/887: getdents d2/d11/d16/d8e/d7b/de8/d82 0 2024-05-06T12:15:00.233 INFO:tasks.workunit.client.1.smithi190.stdout:9/915: mknod db/dd/df/dfe/ddc/d47/d4b/c132 0 2024-05-06T12:15:00.234 INFO:tasks.workunit.client.1.smithi190.stdout:4/888: creat d2/d11/d16/d41/d81/f12f x:0 0 0 2024-05-06T12:15:00.236 INFO:tasks.workunit.client.1.smithi190.stdout:9/916: creat db/dd/d16/d4c/d4e/d5e/dce/f133 x:0 0 0 2024-05-06T12:15:00.238 INFO:tasks.workunit.client.1.smithi190.stdout:9/917: chown db/d23/f8a 41 1 2024-05-06T12:15:00.239 INFO:tasks.workunit.client.1.smithi190.stdout:2/969: sync 2024-05-06T12:15:00.240 INFO:tasks.workunit.client.1.smithi190.stdout:0/946: sync 2024-05-06T12:15:00.241 INFO:tasks.workunit.client.1.smithi190.stdout:9/918: chown db/dd/df/dfe/d87/d22/d49/d88/d76/cc5 35 1 2024-05-06T12:15:00.241 INFO:tasks.workunit.client.1.smithi190.stdout:6/709: write df/d12/f21 [304982,107687] 0 2024-05-06T12:15:00.242 INFO:tasks.workunit.client.1.smithi190.stdout:4/889: dwrite d2/d11/d16/d41/d81/f12f [0,4194304] 0 2024-05-06T12:15:00.242 INFO:tasks.workunit.client.1.smithi190.stdout:2/970: chown d2/l113 14813327 1 2024-05-06T12:15:00.242 INFO:tasks.workunit.client.1.smithi190.stdout:0/947: stat d9/dc/d48/fcb 0 2024-05-06T12:15:00.243 INFO:tasks.workunit.client.1.smithi190.stdout:1/821: write d5/d4f/d65/f3c [1640787,23153] 0 2024-05-06T12:15:00.246 INFO:tasks.workunit.client.1.smithi190.stdout:2/971: stat d2/d8/d19/da9/le0 0 2024-05-06T12:15:00.246 INFO:tasks.workunit.client.1.smithi190.stdout:0/948: write d9/dc/db6/d11b/f124 [555752,24259] 0 2024-05-06T12:15:00.247 INFO:tasks.workunit.client.1.smithi190.stdout:1/822: chown d5/d4f/d65/d2b/c7c 1387023 1 2024-05-06T12:15:00.247 INFO:tasks.workunit.client.1.smithi190.stdout:6/710: dread - df/d12/f72 zero size 2024-05-06T12:15:00.247 INFO:tasks.workunit.client.1.smithi190.stdout:4/890: dread - d2/d11/d16/d8e/d80/d113/f124 zero size 2024-05-06T12:15:00.250 INFO:tasks.workunit.client.1.smithi190.stdout:2/972: dread - d2/d21/d123/d79/db2/f14d zero size 2024-05-06T12:15:00.255 INFO:tasks.workunit.client.1.smithi190.stdout:6/711: symlink df/d2c/dae/dbb/lf4 0 2024-05-06T12:15:00.256 INFO:tasks.workunit.client.1.smithi190.stdout:1/823: dwrite d5/d4f/d65/ff0 [0,4194304] 0 2024-05-06T12:15:00.260 INFO:tasks.workunit.client.1.smithi190.stdout:4/891: creat d2/d11/f130 x:0 0 0 2024-05-06T12:15:00.265 INFO:tasks.workunit.client.1.smithi190.stdout:4/892: symlink d2/d11/d16/d8e/d7b/de8/d66/d6b/de3/l131 0 2024-05-06T12:15:00.266 INFO:tasks.workunit.client.1.smithi190.stdout:0/949: write d9/d51/d93/fe6 [124087,21543] 0 2024-05-06T12:15:00.266 INFO:tasks.workunit.client.1.smithi190.stdout:2/973: write d2/d21/f42 [189641,46556] 0 2024-05-06T12:15:00.267 INFO:tasks.workunit.client.1.smithi190.stdout:2/974: chown d2/d4c/d10a/f10c 2616 1 2024-05-06T12:15:00.270 INFO:tasks.workunit.client.1.smithi190.stdout:0/950: creat d9/d6f/f14f x:0 0 0 2024-05-06T12:15:00.272 INFO:tasks.workunit.client.1.smithi190.stdout:4/893: fsync d2/d11/d16/f38 0 2024-05-06T12:15:00.273 INFO:tasks.workunit.client.1.smithi190.stdout:9/919: dread db/d23/f83 [0,4194304] 0 2024-05-06T12:15:00.274 INFO:tasks.workunit.client.1.smithi190.stdout:2/975: truncate d2/d21/d123/f7c 3168729 0 2024-05-06T12:15:00.275 INFO:tasks.workunit.client.1.smithi190.stdout:0/951: truncate d9/dc/d28/d4c/f53 1721902 0 2024-05-06T12:15:00.278 INFO:tasks.workunit.client.1.smithi190.stdout:4/894: mkdir d2/d11/d16/d8e/d7b/de8/d9e/d6e/daa/ddc/d132 0 2024-05-06T12:15:00.278 INFO:tasks.workunit.client.1.smithi190.stdout:1/824: dread d5/d4f/d68/f106 [0,4194304] 0 2024-05-06T12:15:00.279 INFO:tasks.workunit.client.1.smithi190.stdout:0/952: mkdir d9/dcf/d56/dae/d150 0 2024-05-06T12:15:00.280 INFO:tasks.workunit.client.1.smithi190.stdout:2/976: creat d2/d112/f153 x:0 0 0 2024-05-06T12:15:00.280 INFO:tasks.workunit.client.1.smithi190.stdout:0/953: stat d9/dc/d28/d4c 0 2024-05-06T12:15:00.283 INFO:tasks.workunit.client.1.smithi190.stdout:4/895: symlink d2/d11/d16/d4e/l133 0 2024-05-06T12:15:00.283 INFO:tasks.workunit.client.1.smithi190.stdout:6/712: dread df/d2c/d28/f7c [0,4194304] 0 2024-05-06T12:15:00.285 INFO:tasks.workunit.client.1.smithi190.stdout:1/825: creat d5/d4f/d68/f118 x:0 0 0 2024-05-06T12:15:00.287 INFO:tasks.workunit.client.1.smithi190.stdout:9/920: link db/f1f db/dd/df/dfe/f134 0 2024-05-06T12:15:00.288 INFO:tasks.workunit.client.1.smithi190.stdout:6/713: truncate df/d2c/d61/d73/dad/fb0 6593093 0 2024-05-06T12:15:00.289 INFO:tasks.workunit.client.1.smithi190.stdout:1/826: creat d5/d4f/d65/d2b/f119 x:0 0 0 2024-05-06T12:15:00.290 INFO:tasks.workunit.client.1.smithi190.stdout:0/954: dwrite d9/dc/d28/f8c [0,4194304] 0 2024-05-06T12:15:00.290 INFO:tasks.workunit.client.1.smithi190.stdout:6/714: read - df/d2c/d1e/d57/d53/de8/fea zero size 2024-05-06T12:15:00.294 INFO:tasks.workunit.client.1.smithi190.stdout:6/715: write df/d2c/d61/f79 [3222268,114349] 0 2024-05-06T12:15:00.296 INFO:tasks.workunit.client.1.smithi190.stdout:6/716: write df/fc1 [698868,2233] 0 2024-05-06T12:15:00.300 INFO:tasks.workunit.client.1.smithi190.stdout:0/955: dwrite d9/dc/d28/dc5/f11c [4194304,4194304] 0 2024-05-06T12:15:00.300 INFO:tasks.workunit.client.1.smithi190.stdout:4/896: creat d2/d11/d16/d8e/d7b/de8/f134 x:0 0 0 2024-05-06T12:15:00.303 INFO:tasks.workunit.client.1.smithi190.stdout:9/921: mknod db/dd/df/dfe/d87/d22/d49/c135 0 2024-05-06T12:15:00.303 INFO:tasks.workunit.client.1.smithi190.stdout:2/977: write d2/d21/d123/fc8 [648411,24269] 0 2024-05-06T12:15:00.305 INFO:tasks.workunit.client.1.smithi190.stdout:1/827: symlink d5/d4f/d65/d24/d43/da6/ded/l11a 0 2024-05-06T12:15:00.305 INFO:tasks.workunit.client.1.smithi190.stdout:1/828: readlink d5/l89 0 2024-05-06T12:15:00.307 INFO:tasks.workunit.client.1.smithi190.stdout:0/956: mknod d9/dc/d48/d50/d87/dd2/dcc/c151 0 2024-05-06T12:15:00.308 INFO:tasks.workunit.client.1.smithi190.stdout:1/829: write d5/d4f/d65/d2b/f35 [1864426,54671] 0 2024-05-06T12:15:00.311 INFO:tasks.workunit.client.1.smithi190.stdout:1/830: chown d5/d4f/d65/d2b/c112 9777150 1 2024-05-06T12:15:00.314 INFO:tasks.workunit.client.1.smithi190.stdout:9/922: mkdir db/dd/df/dfe/d136 0 2024-05-06T12:15:00.316 INFO:tasks.workunit.client.1.smithi190.stdout:9/923: fdatasync db/dd/df/dfe/d87/d22/d49/d88/d9f/df2/fdf 0 2024-05-06T12:15:00.318 INFO:tasks.workunit.client.1.smithi190.stdout:9/924: write db/dd/d16/f1e [1389177,37109] 0 2024-05-06T12:15:00.320 INFO:tasks.workunit.client.1.smithi190.stdout:1/831: chown d5/d15/la5 210542 1 2024-05-06T12:15:00.323 INFO:tasks.workunit.client.1.smithi190.stdout:6/717: dwrite df/d2c/dae/fb3 [0,4194304] 0 2024-05-06T12:15:00.324 INFO:tasks.workunit.client.1.smithi190.stdout:2/978: rename d2/d21/la1 to d2/d8/d33/d46/dab/l154 0 2024-05-06T12:15:00.325 INFO:tasks.workunit.client.1.smithi190.stdout:6/718: write df/d17/fda [620850,106068] 0 2024-05-06T12:15:00.331 INFO:tasks.workunit.client.1.smithi190.stdout:4/897: getdents d2/d11/d16/d8e/d7b/de8/d66/d6b/da4/dfa 0 2024-05-06T12:15:00.414 INFO:tasks.workunit.client.1.smithi190.stdout:1/832: symlink d5/d4f/d65/d2b/l11b 0 2024-05-06T12:15:00.414 INFO:tasks.workunit.client.1.smithi190.stdout:2/979: rmdir d2/d8/d71 39 2024-05-06T12:15:00.414 INFO:tasks.workunit.client.1.smithi190.stdout:6/719: rmdir df/d2c/d28/dd1 39 2024-05-06T12:15:00.414 INFO:tasks.workunit.client.1.smithi190.stdout:0/957: link d9/dc/d48/d50/d83/dc6/d109/c10e d9/dc/d28/dc5/dd5/de4/dea/dfd/c152 0 2024-05-06T12:15:00.415 INFO:tasks.workunit.client.1.smithi190.stdout:1/833: chown d5/d4f/d65/d32/d47/d53/d98/da7/de7/f9b 1083968 1 2024-05-06T12:15:00.417 INFO:tasks.workunit.client.1.smithi190.stdout:2/980: creat d2/d8/d33/de6/f155 x:0 0 0 2024-05-06T12:15:00.418 INFO:tasks.workunit.client.1.smithi190.stdout:0/958: mknod d9/dc/c153 0 2024-05-06T12:15:00.419 INFO:tasks.workunit.client.1.smithi190.stdout:6/720: symlink df/d2c/d28/dd1/lf5 0 2024-05-06T12:15:00.420 INFO:tasks.workunit.client.1.smithi190.stdout:1/834: fsync d5/d4f/d65/d62/d9d/fe6 0 2024-05-06T12:15:00.422 INFO:tasks.workunit.client.1.smithi190.stdout:4/898: dwrite d2/d11/d16/d8e/d7b/de8/d9e/d6e/d78/fdf [0,4194304] 0 2024-05-06T12:15:00.423 INFO:tasks.workunit.client.1.smithi190.stdout:1/835: read d5/d15/f82 [2141243,16888] 0 2024-05-06T12:15:00.424 INFO:tasks.workunit.client.1.smithi190.stdout:6/721: read df/d17/f38 [444704,20264] 0 2024-05-06T12:15:00.426 INFO:tasks.workunit.client.1.smithi190.stdout:0/959: truncate d9/d51/dc9/f8d 724461 0 2024-05-06T12:15:00.428 INFO:tasks.workunit.client.1.smithi190.stdout:1/836: truncate d5/d4f/d65/d62/d9d/dba/f115 627829 0 2024-05-06T12:15:00.429 INFO:tasks.workunit.client.1.smithi190.stdout:4/899: mknod d2/d7d/c135 0 2024-05-06T12:15:00.429 INFO:tasks.workunit.client.1.smithi190.stdout:0/960: dread - d9/dc/d48/d50/d87/d136/f13c zero size 2024-05-06T12:15:00.430 INFO:tasks.workunit.client.1.smithi190.stdout:0/961: dread - d9/dcf/f135 zero size 2024-05-06T12:15:00.434 INFO:tasks.workunit.client.1.smithi190.stdout:0/962: read d9/dcf/d21/f4a [2851854,66139] 0 2024-05-06T12:15:00.437 INFO:tasks.workunit.client.1.smithi190.stdout:1/837: write d5/f34 [4418558,9877] 0 2024-05-06T12:15:00.438 INFO:tasks.workunit.client.1.smithi190.stdout:0/963: mkdir d9/dc/d28/d4c/d96/d154 0 2024-05-06T12:15:00.439 INFO:tasks.workunit.client.1.smithi190.stdout:4/900: write d2/d11/d16/f38 [1025549,76177] 0 2024-05-06T12:15:00.439 INFO:tasks.workunit.client.1.smithi190.stdout:2/981: dread d2/d8/d33/d46/d5d/f93 [0,4194304] 0 2024-05-06T12:15:00.441 INFO:tasks.workunit.client.1.smithi190.stdout:6/722: dwrite df/d2c/d28/de0/fac [0,4194304] 0 2024-05-06T12:15:00.442 INFO:tasks.workunit.client.1.smithi190.stdout:0/964: read d9/dc/d48/ff0 [4254,69645] 0 2024-05-06T12:15:00.444 INFO:tasks.workunit.client.1.smithi190.stdout:6/723: write df/d2c/d1e/d57/d89/fd5 [194723,4643] 0 2024-05-06T12:15:00.445 INFO:tasks.workunit.client.1.smithi190.stdout:2/982: creat d2/d8/d33/f156 x:0 0 0 2024-05-06T12:15:00.447 INFO:tasks.workunit.client.1.smithi190.stdout:6/724: dread - df/d2c/d61/d73/dad/fc2 zero size 2024-05-06T12:15:00.448 INFO:tasks.workunit.client.1.smithi190.stdout:4/901: rename d2/f50 to d2/d11/d16/d8e/d7b/f136 0 2024-05-06T12:15:00.449 INFO:tasks.workunit.client.1.smithi190.stdout:0/965: mknod d9/dc/d28/dc5/dd5/c155 0 2024-05-06T12:15:00.451 INFO:tasks.workunit.client.1.smithi190.stdout:2/983: fsync d2/d8/d33/d46/fe9 0 2024-05-06T12:15:00.452 INFO:tasks.workunit.client.1.smithi190.stdout:4/902: symlink d2/d11/d16/d8e/d7b/de8/d82/l137 0 2024-05-06T12:15:00.452 INFO:tasks.workunit.client.1.smithi190.stdout:0/966: dread - d9/dcf/d56/fe9 zero size 2024-05-06T12:15:00.454 INFO:tasks.workunit.client.1.smithi190.stdout:2/984: rename d2/d4c/c103 to d2/d8/d33/de6/c157 0 2024-05-06T12:15:00.454 INFO:tasks.workunit.client.1.smithi190.stdout:6/725: truncate df/d2c/d61/d87/fde 626672 0 2024-05-06T12:15:00.455 INFO:tasks.workunit.client.1.smithi190.stdout:2/985: truncate d2/d21/d123/fae 5210410 0 2024-05-06T12:15:00.456 INFO:tasks.workunit.client.1.smithi190.stdout:4/903: unlink d2/d11/d16/d8e/d7b/de8/d9e/d6e/d78/l10e 0 2024-05-06T12:15:00.456 INFO:tasks.workunit.client.1.smithi190.stdout:2/986: chown d2/d8/d33/dc2/dcd/l11b 79 1 2024-05-06T12:15:00.457 INFO:tasks.workunit.client.1.smithi190.stdout:0/967: mkdir d9/dc/d28/dc5/dd5/de4/dea/dfd/d156 0 2024-05-06T12:15:00.458 INFO:tasks.workunit.client.1.smithi190.stdout:6/726: mkdir df/d2c/d61/df6 0 2024-05-06T12:15:00.459 INFO:tasks.workunit.client.1.smithi190.stdout:6/727: chown df/d2c/d1e/d57/d53/de8 222405885 1 2024-05-06T12:15:00.460 INFO:tasks.workunit.client.1.smithi190.stdout:2/987: mknod d2/d8/dc/c158 0 2024-05-06T12:15:00.462 INFO:tasks.workunit.client.1.smithi190.stdout:6/728: write df/d2c/d61/d73/dad/fc2 [565502,1294] 0 2024-05-06T12:15:00.462 INFO:tasks.workunit.client.1.smithi190.stdout:4/904: getdents d2/d7d 0 2024-05-06T12:15:00.464 INFO:tasks.workunit.client.1.smithi190.stdout:2/988: symlink d2/d8/dc/d116/d139/d10f/l159 0 2024-05-06T12:15:00.466 INFO:tasks.workunit.client.1.smithi190.stdout:0/968: link d9/dc/d28/dc5/f11c d9/dc/d28/dc5/dd5/de4/dea/dfd/d156/f157 0 2024-05-06T12:15:00.469 INFO:tasks.workunit.client.1.smithi190.stdout:0/969: chown d9/dc/d48/d50/d83/dc6/d109 1163151 1 2024-05-06T12:15:00.469 INFO:tasks.workunit.client.1.smithi190.stdout:6/729: read df/d2c/d28/de0/f54 [1653056,5291] 0 2024-05-06T12:15:00.469 INFO:tasks.workunit.client.1.smithi190.stdout:4/905: creat d2/d11/d16/d41/d81/f138 x:0 0 0 2024-05-06T12:15:00.474 INFO:tasks.workunit.client.1.smithi190.stdout:2/989: dwrite d2/d21/d123/d79/db2/f11c [0,4194304] 0 2024-05-06T12:15:00.475 INFO:tasks.workunit.client.1.smithi190.stdout:6/730: chown df/d2c/d1e/d57/f8a 187298 1 2024-05-06T12:15:00.476 INFO:tasks.workunit.client.1.smithi190.stdout:1/838: dread d5/d4f/d65/d32/d47/f57 [0,4194304] 0 2024-05-06T12:15:00.483 INFO:tasks.workunit.client.1.smithi190.stdout:2/990: dwrite d2/d8/d19/dbd/f10d [0,4194304] 0 2024-05-06T12:15:00.484 INFO:tasks.workunit.client.1.smithi190.stdout:4/906: unlink d2/d11/d16/d31/c114 0 2024-05-06T12:15:00.486 INFO:tasks.workunit.client.1.smithi190.stdout:6/731: symlink df/d2c/d1e/d57/lf7 0 2024-05-06T12:15:00.492 INFO:tasks.workunit.client.1.smithi190.stdout:6/732: truncate df/d2c/d61/d84/faa 966625 0 2024-05-06T12:15:00.494 INFO:tasks.workunit.client.1.smithi190.stdout:1/839: rename d5/d4f/d65/d2b/f119 to d5/d4f/d65/d32/f11c 0 2024-05-06T12:15:00.495 INFO:tasks.workunit.client.1.smithi190.stdout:0/970: write d9/dc/d48/fcb [1183157,87738] 0 2024-05-06T12:15:00.497 INFO:tasks.workunit.client.1.smithi190.stdout:6/733: dread df/d17/d37/f42 [0,4194304] 0 2024-05-06T12:15:00.497 INFO:tasks.workunit.client.1.smithi190.stdout:9/925: sync 2024-05-06T12:15:00.502 INFO:tasks.workunit.client.1.smithi190.stdout:9/926: mknod db/daa/c137 0 2024-05-06T12:15:00.503 INFO:tasks.workunit.client.1.smithi190.stdout:6/734: truncate df/d2c/d1e/d57/f6a 1302743 0 2024-05-06T12:15:00.504 INFO:tasks.workunit.client.1.smithi190.stdout:9/927: readlink db/dd/df/dfe/d87/l11d 0 2024-05-06T12:15:00.504 INFO:tasks.workunit.client.1.smithi190.stdout:4/907: getdents d2/d11/d16/d8e/d80 0 2024-05-06T12:15:00.505 INFO:tasks.workunit.client.1.smithi190.stdout:2/991: dwrite d2/d21/d123/d79/d7e/f63 [0,4194304] 0 2024-05-06T12:15:00.505 INFO:tasks.workunit.client.1.smithi190.stdout:1/840: creat d5/d4f/d65/f11d x:0 0 0 2024-05-06T12:15:00.507 INFO:tasks.workunit.client.1.smithi190.stdout:0/971: creat d9/dcf/d21/f158 x:0 0 0 2024-05-06T12:15:00.509 INFO:tasks.workunit.client.1.smithi190.stdout:2/992: rmdir d2/d8/dc/d116/d139/d10f 39 2024-05-06T12:15:00.511 INFO:tasks.workunit.client.1.smithi190.stdout:4/908: truncate d2/d11/d16/d8e/d80/da0/f10b 862324 0 2024-05-06T12:15:00.511 INFO:tasks.workunit.client.1.smithi190.stdout:0/972: truncate d9/dc/d48/d50/d83/faf 1626496 0 2024-05-06T12:15:00.511 INFO:tasks.workunit.client.1.smithi190.stdout:1/841: mkdir d5/d4f/d65/d32/d47/d53/d11e 0 2024-05-06T12:15:00.513 INFO:tasks.workunit.client.1.smithi190.stdout:2/993: dread - d2/d8/d71/f8e zero size 2024-05-06T12:15:00.514 INFO:tasks.workunit.client.1.smithi190.stdout:0/973: write d9/dcf/f62 [3744370,97847] 0 2024-05-06T12:15:00.516 INFO:tasks.workunit.client.1.smithi190.stdout:1/842: chown d5/d15/f52 170840 1 2024-05-06T12:15:00.518 INFO:tasks.workunit.client.1.smithi190.stdout:4/909: creat d2/d11/d16/d8e/d7b/de8/d66/d6b/da4/de1/f139 x:0 0 0 2024-05-06T12:15:00.518 INFO:tasks.workunit.client.1.smithi190.stdout:9/928: dwrite db/dd/df/dfe/d87/fcb [0,4194304] 0 2024-05-06T12:15:00.522 INFO:tasks.workunit.client.1.smithi190.stdout:4/910: readlink d2/d11/d16/d8e/d7b/dd4/lda 0 2024-05-06T12:15:00.522 INFO:tasks.workunit.client.1.smithi190.stdout:0/974: unlink d9/dc/d28/d77/l126 0 2024-05-06T12:15:00.523 INFO:tasks.workunit.client.1.smithi190.stdout:2/994: dread d2/d21/d123/d79/db2/f11c [0,4194304] 0 2024-05-06T12:15:00.523 INFO:tasks.workunit.client.1.smithi190.stdout:9/929: rmdir db/dd/df/dfe/ddc/d47/d4b 39 2024-05-06T12:15:00.525 INFO:tasks.workunit.client.1.smithi190.stdout:2/995: dread - d2/d8/d33/d46/dab/dac/f104 zero size 2024-05-06T12:15:00.531 INFO:tasks.workunit.client.1.smithi190.stdout:6/735: dread df/d2c/d28/f40 [0,4194304] 0 2024-05-06T12:15:00.532 INFO:tasks.workunit.client.1.smithi190.stdout:1/843: creat d5/d15/f11f x:0 0 0 2024-05-06T12:15:00.533 INFO:tasks.workunit.client.1.smithi190.stdout:2/996: unlink d2/d21/d123/c54 0 2024-05-06T12:15:00.535 INFO:tasks.workunit.client.1.smithi190.stdout:2/997: fsync d2/d8/d19/d9b/f11f 0 2024-05-06T12:15:00.537 INFO:tasks.workunit.client.1.smithi190.stdout:0/975: creat d9/dc/d48/d50/d87/dd2/d142/f159 x:0 0 0 2024-05-06T12:15:00.540 INFO:tasks.workunit.client.1.smithi190.stdout:4/911: rename d2/d11/l2d to d2/d11/d16/d8e/d7b/de8/d9e/l13a 0 2024-05-06T12:15:00.540 INFO:tasks.workunit.client.1.smithi190.stdout:1/844: dwrite d5/d4f/d65/d24/d43/da6/ded/f116 [0,4194304] 0 2024-05-06T12:15:00.541 INFO:tasks.workunit.client.1.smithi190.stdout:4/912: dread - d2/d11/d16/d4e/f107 zero size 2024-05-06T12:15:00.542 INFO:tasks.workunit.client.1.smithi190.stdout:6/736: mkdir df/d2c/d1e/d57/d89/df8 0 2024-05-06T12:15:00.544 INFO:tasks.workunit.client.1.smithi190.stdout:2/998: unlink d2/d8/d33/dc2/ffd 0 2024-05-06T12:15:00.547 INFO:tasks.workunit.client.1.smithi190.stdout:1/845: creat d5/d6b/f120 x:0 0 0 2024-05-06T12:15:00.554 INFO:tasks.workunit.client.1.smithi190.stdout:6/737: mkdir df/d2c/d61/d87/df9 0 2024-05-06T12:15:00.555 INFO:tasks.workunit.client.1.smithi190.stdout:1/846: symlink d5/d15/d105/l121 0 2024-05-06T12:15:00.556 INFO:tasks.workunit.client.1.smithi190.stdout:4/913: rmdir d2/d11/d16/d8e/d7b/de8/d9e 39 2024-05-06T12:15:00.557 INFO:tasks.workunit.client.1.smithi190.stdout:6/738: truncate df/f82 1601215 0 2024-05-06T12:15:00.558 INFO:tasks.workunit.client.1.smithi190.stdout:9/930: link db/f1f db/dd/df/dfe/ddc/d47/f138 0 2024-05-06T12:15:00.561 INFO:tasks.workunit.client.1.smithi190.stdout:0/976: write d9/dc/d28/d4c/f4b [3452567,63283] 0 2024-05-06T12:15:00.562 INFO:tasks.workunit.client.1.smithi190.stdout:0/977: stat d9/dc/d28/c73 0 2024-05-06T12:15:00.564 INFO:tasks.workunit.client.1.smithi190.stdout:9/931: creat db/d23/d29/dc4/f139 x:0 0 0 2024-05-06T12:15:00.569 INFO:tasks.workunit.client.1.smithi190.stdout:0/978: fsync d9/dc/dd0/fd8 0 2024-05-06T12:15:00.571 INFO:tasks.workunit.client.1.smithi190.stdout:4/914: link d2/d11/lcb d2/d11/d16/d8e/d7b/de8/d82/dd7/l13b 0 2024-05-06T12:15:00.572 INFO:tasks.workunit.client.1.smithi190.stdout:4/915: write d2/d11/d16/f38 [2344776,124401] 0 2024-05-06T12:15:00.573 INFO:tasks.workunit.client.1.smithi190.stdout:4/916: stat d2/d11/d16/d31/fb4 0 2024-05-06T12:15:00.577 INFO:tasks.workunit.client.1.smithi190.stdout:0/979: unlink d9/dc/d48/lb5 0 2024-05-06T12:15:00.579 INFO:tasks.workunit.client.1.smithi190.stdout:1/847: write d5/d4f/d65/d24/f45 [2039425,96511] 0 2024-05-06T12:15:00.586 INFO:tasks.workunit.client.1.smithi190.stdout:6/739: link df/d17/c4b df/d17/cfa 0 2024-05-06T12:15:00.587 INFO:tasks.workunit.client.1.smithi190.stdout:4/917: mknod d2/d11/d16/d8e/d7b/de8/d9e/d6e/daa/ddc/c13c 0 2024-05-06T12:15:00.588 INFO:tasks.workunit.client.1.smithi190.stdout:0/980: creat d9/dc/d48/d50/d87/d129/f15a x:0 0 0 2024-05-06T12:15:00.589 INFO:tasks.workunit.client.1.smithi190.stdout:1/848: mknod d5/d4f/d65/d32/d47/d53/d11e/c122 0 2024-05-06T12:15:00.592 INFO:tasks.workunit.client.1.smithi190.stdout:0/981: stat d9/dc/d48/d50/d87/dd2/d142 0 2024-05-06T12:15:00.593 INFO:tasks.workunit.client.1.smithi190.stdout:9/932: dwrite db/dd/df/dfe/d87/d22/d49/d88/d9f/f11b [0,4194304] 0 2024-05-06T12:15:00.595 INFO:tasks.workunit.client.1.smithi190.stdout:6/740: truncate df/d2c/d92/f93 1473334 0 2024-05-06T12:15:00.596 INFO:tasks.workunit.client.1.smithi190.stdout:1/849: mknod d5/d4f/d65/d24/d43/de2/d114/c123 0 2024-05-06T12:15:00.602 INFO:tasks.workunit.client.1.smithi190.stdout:1/850: unlink d5/d4f/d65/d24/f45 0 2024-05-06T12:15:00.606 INFO:tasks.workunit.client.1.smithi190.stdout:1/851: creat d5/d4f/d65/d24/d43/da6/ded/f124 x:0 0 0 2024-05-06T12:15:00.607 INFO:tasks.workunit.client.1.smithi190.stdout:4/918: link d2/d11/d16/d8e/d7b/l10c d2/d11/d16/d8e/d7b/de8/d66/d6b/de3/l13d 0 2024-05-06T12:15:00.609 INFO:tasks.workunit.client.1.smithi190.stdout:0/982: rename d9/dcf/d56/dab to d9/d51/d93/d15b 0 2024-05-06T12:15:00.612 INFO:tasks.workunit.client.1.smithi190.stdout:0/983: mkdir d9/dc/d48/d50/d87/d15c 0 2024-05-06T12:15:00.615 INFO:tasks.workunit.client.1.smithi190.stdout:9/933: dread db/dd/df/dfe/ddc/d47/d4b/f43 [0,4194304] 0 2024-05-06T12:15:00.616 INFO:tasks.workunit.client.1.smithi190.stdout:1/852: dwrite d5/d4f/d65/d32/d47/d53/fdc [0,4194304] 0 2024-05-06T12:15:00.616 INFO:tasks.workunit.client.1.smithi190.stdout:0/984: mknod d9/dcf/d56/dae/c15d 0 2024-05-06T12:15:00.617 INFO:tasks.workunit.client.1.smithi190.stdout:9/934: dread - db/d23/d7e/f130 zero size 2024-05-06T12:15:00.619 INFO:tasks.workunit.client.1.smithi190.stdout:0/985: write d9/dc/d28/d4c/d96/f128 [142435,55955] 0 2024-05-06T12:15:00.620 INFO:tasks.workunit.client.1.smithi190.stdout:0/986: dread - d9/d6f/f14f zero size 2024-05-06T12:15:00.621 INFO:tasks.workunit.client.1.smithi190.stdout:1/853: unlink d5/d4f/d65/d24/d43/de2/d114/cf3 0 2024-05-06T12:15:00.621 INFO:tasks.workunit.client.1.smithi190.stdout:0/987: fsync d9/f2c 0 2024-05-06T12:15:00.623 INFO:tasks.workunit.client.1.smithi190.stdout:1/854: dread - d5/d4f/d65/ff8 zero size 2024-05-06T12:15:00.626 INFO:tasks.workunit.client.1.smithi190.stdout:0/988: creat d9/dc/d28/dc5/f15e x:0 0 0 2024-05-06T12:15:00.626 INFO:tasks.workunit.client.1.smithi190.stdout:1/855: truncate d5/d4f/d65/f11d 350316 0 2024-05-06T12:15:00.628 INFO:tasks.workunit.client.1.smithi190.stdout:0/989: readlink d9/l145 0 2024-05-06T12:15:00.630 INFO:tasks.workunit.client.1.smithi190.stdout:1/856: dread - d5/d15/d1e/fb8 zero size 2024-05-06T12:15:00.630 INFO:tasks.workunit.client.1.smithi190.stdout:9/935: write db/dd/df/dfe/d87/d22/d49/f79 [1054376,79546] 0 2024-05-06T12:15:00.631 INFO:tasks.workunit.client.1.smithi190.stdout:0/990: write d9/dcf/d21/f97 [180489,76404] 0 2024-05-06T12:15:00.631 INFO:tasks.workunit.client.1.smithi190.stdout:4/919: dread d2/f59 [4194304,4194304] 0 2024-05-06T12:15:00.634 INFO:tasks.workunit.client.1.smithi190.stdout:1/857: creat d5/d15/f125 x:0 0 0 2024-05-06T12:15:00.636 INFO:tasks.workunit.client.1.smithi190.stdout:4/920: rmdir d2/d11/d16/d41/d81 39 2024-05-06T12:15:00.639 INFO:tasks.workunit.client.1.smithi190.stdout:9/936: symlink db/dd/df/dfe/d87/d22/d49/d88/l13a 0 2024-05-06T12:15:00.642 INFO:tasks.workunit.client.1.smithi190.stdout:9/937: rmdir db/dd/d5c/dac 39 2024-05-06T12:15:00.643 INFO:tasks.workunit.client.1.smithi190.stdout:1/858: write d5/d4f/d65/d62/f93 [472153,110240] 0 2024-05-06T12:15:00.644 INFO:tasks.workunit.client.1.smithi190.stdout:0/991: creat d9/dc/d28/d4c/d80/f15f x:0 0 0 2024-05-06T12:15:00.645 INFO:tasks.workunit.client.1.smithi190.stdout:4/921: getdents d2/d11/d16/d8e/d7b/de8/d66/d6b/da4/dfa/d10f/d120 0 2024-05-06T12:15:00.646 INFO:tasks.workunit.client.1.smithi190.stdout:0/992: chown d9/dc/d28/d4c/l54 960 1 2024-05-06T12:15:00.649 INFO:tasks.workunit.client.1.smithi190.stdout:1/859: rename d5/d6b/d8f to d5/d4f/d65/d32/d47/d53/d98/da7/de7/d126 0 2024-05-06T12:15:00.650 INFO:tasks.workunit.client.1.smithi190.stdout:4/922: symlink d2/d11/l13e 0 2024-05-06T12:15:00.652 INFO:tasks.workunit.client.1.smithi190.stdout:0/993: rename d9/dc/d28/dc5/dd5/d13a to d9/d51/dc9/d160 0 2024-05-06T12:15:00.653 INFO:tasks.workunit.client.1.smithi190.stdout:9/938: link db/dd/d16/f64 db/dd/df/dfe/d87/d22/d49/f13b 0 2024-05-06T12:15:00.654 INFO:tasks.workunit.client.1.smithi190.stdout:4/923: rmdir d2/d11/d16/d8e/d7b/de8/d9e/d6e/daa 39 2024-05-06T12:15:00.656 INFO:tasks.workunit.client.1.smithi190.stdout:9/939: truncate db/d23/d7e/fa8 4258372 0 2024-05-06T12:15:00.657 INFO:tasks.workunit.client.1.smithi190.stdout:0/994: dread d9/dc/d48/fcb [0,4194304] 0 2024-05-06T12:15:00.658 INFO:tasks.workunit.client.1.smithi190.stdout:4/924: rmdir d2/d11/d16/d8e/d80/da0 39 2024-05-06T12:15:00.659 INFO:tasks.workunit.client.1.smithi190.stdout:9/940: dread - db/dd/df/dfe/d87/d22/d49/d8c/f103 zero size 2024-05-06T12:15:00.660 INFO:tasks.workunit.client.1.smithi190.stdout:0/995: creat d9/dc/dd0/f161 x:0 0 0 2024-05-06T12:15:00.660 INFO:tasks.workunit.client.1.smithi190.stdout:4/925: fdatasync d2/d11/d16/d8e/d7b/de8/d66/d6b/f9f 0 2024-05-06T12:15:00.663 INFO:tasks.workunit.client.1.smithi190.stdout:1/860: dread d5/d4f/d65/d2b/fbc [0,4194304] 0 2024-05-06T12:15:00.664 INFO:tasks.workunit.client.1.smithi190.stdout:0/996: creat d9/dc/d48/d50/d87/d129/f162 x:0 0 0 2024-05-06T12:15:00.671 INFO:tasks.workunit.client.1.smithi190.stdout:1/861: dwrite d5/d4f/d65/d62/d9d/dba/f115 [0,4194304] 0 2024-05-06T12:15:00.676 INFO:tasks.workunit.client.1.smithi190.stdout:4/926: rename d2/d11/d16/d8e/d7b/de8/d66/d6b to d2/d11/d16/d13f 0 2024-05-06T12:15:00.677 INFO:tasks.workunit.client.1.smithi190.stdout:9/941: dwrite db/f52 [4194304,4194304] 0 2024-05-06T12:15:00.681 INFO:tasks.workunit.client.1.smithi190.stdout:0/997: rmdir d9/dc/d28/dc5/dd5/de4/d12d 0 2024-05-06T12:15:00.684 INFO:tasks.workunit.client.1.smithi190.stdout:4/927: write d2/d11/d16/d8e/d7b/de8/d9e/d6e/daa/fe6 [415822,35576] 0 2024-05-06T12:15:00.687 INFO:tasks.workunit.client.1.smithi190.stdout:1/862: dwrite d5/d4f/d65/d32/d47/d53/d98/da7/de7/fd8 [0,4194304] 0 2024-05-06T12:15:00.690 INFO:tasks.workunit.client.1.smithi190.stdout:1/863: stat d5/d4f/d65/d62/d9d/dba/f115 0 2024-05-06T12:15:00.690 INFO:tasks.workunit.client.1.smithi190.stdout:0/998: symlink d9/dc/d28/dc5/dd5/de4/dea/l163 0 2024-05-06T12:15:00.691 INFO:tasks.workunit.client.1.smithi190.stdout:4/928: truncate d2/d11/d16/d31/f5c 2019300 0 2024-05-06T12:15:00.699 INFO:tasks.workunit.client.1.smithi190.stdout:1/864: dwrite d5/d4f/d65/d32/d47/d53/d98/da7/fc8 [0,4194304] 0 2024-05-06T12:15:00.702 INFO:tasks.workunit.client.1.smithi190.stdout:1/865: truncate d5/d4f/d68/f8e 1648294 0 2024-05-06T12:15:00.718 INFO:tasks.workunit.client.1.smithi190.stdout:9/942: dread db/dd/df/dfe/f134 [0,4194304] 0 2024-05-06T12:15:00.719 INFO:tasks.workunit.client.1.smithi190.stdout:4/929: truncate d2/d10/f35 2896184 0 2024-05-06T12:15:00.721 INFO:tasks.workunit.client.1.smithi190.stdout:1/866: truncate d5/d4f/d65/d32/d47/f4c 1224286 0 2024-05-06T12:15:00.724 INFO:tasks.workunit.client.1.smithi190.stdout:4/930: mknod d2/d11/d16/d8e/d7b/de8/d82/dd7/c140 0 2024-05-06T12:15:00.724 INFO:tasks.workunit.client.1.smithi190.stdout:0/999: dwrite d9/dc/dd0/fd8 [0,4194304] 0 2024-05-06T12:15:00.729 INFO:tasks.workunit.client.1.smithi190.stdout:9/943: dread db/d23/f3a [0,4194304] 0 2024-05-06T12:15:00.733 INFO:tasks.workunit.client.1.smithi190.stdout:1/867: dwrite d5/d4f/d65/d24/d3a/f3e [4194304,4194304] 0 2024-05-06T12:15:00.738 INFO:tasks.workunit.client.1.smithi190.stdout:9/944: creat db/f13c x:0 0 0 2024-05-06T12:15:00.741 INFO:tasks.workunit.client.1.smithi190.stdout:9/945: stat db/dd/df/dfe/d87/d99/lb6 0 2024-05-06T12:15:00.741 INFO:tasks.workunit.client.1.smithi190.stdout:1/868: dwrite d5/d4f/d65/d24/d3a/f3e [0,4194304] 0 2024-05-06T12:15:00.743 INFO:tasks.workunit.client.1.smithi190.stdout:4/931: truncate d2/d11/d16/d8e/d7b/f128 1709058 0 2024-05-06T12:15:00.747 INFO:tasks.workunit.client.1.smithi190.stdout:1/869: mknod d5/c127 0 2024-05-06T12:15:00.749 INFO:tasks.workunit.client.1.smithi190.stdout:9/946: dwrite db/f77 [0,4194304] 0 2024-05-06T12:15:00.752 INFO:tasks.workunit.client.1.smithi190.stdout:1/870: creat d5/d4f/d65/d32/d47/d53/d11e/f128 x:0 0 0 2024-05-06T12:15:00.754 INFO:tasks.workunit.client.1.smithi190.stdout:9/947: mkdir db/dd/df/dfe/ddc/d47/d13d 0 2024-05-06T12:15:00.758 INFO:tasks.workunit.client.1.smithi190.stdout:9/948: write db/dd/d16/fae [1317430,90620] 0 2024-05-06T12:15:00.775 INFO:tasks.workunit.client.1.smithi190.stdout:2/999: sync 2024-05-06T12:15:00.781 INFO:tasks.workunit.client.1.smithi190.stdout:4/932: dread d2/d11/d16/d8e/d7b/de8/d9e/d6e/daa/fdd [0,4194304] 0 2024-05-06T12:15:00.783 INFO:tasks.workunit.client.1.smithi190.stdout:1/871: dwrite d5/d4f/d65/d32/f61 [0,4194304] 0 2024-05-06T12:15:00.788 INFO:tasks.workunit.client.1.smithi190.stdout:4/933: dread d2/d11/d16/d13f/da4/dfa/d10f/f11b [0,4194304] 0 2024-05-06T12:15:00.790 INFO:tasks.workunit.client.1.smithi190.stdout:9/949: dwrite db/dd/df/dfe/d87/d22/d49/d88/f9c [0,4194304] 0 2024-05-06T12:15:00.795 INFO:tasks.workunit.client.1.smithi190.stdout:4/934: dwrite d2/d10/ff [0,4194304] 0 2024-05-06T12:15:00.796 INFO:tasks.workunit.client.1.smithi190.stdout:4/935: chown d2/d11/d16/d8e/d7b/cbd 822 1 2024-05-06T12:15:00.805 INFO:tasks.workunit.client.1.smithi190.stdout:9/950: truncate db/dd/df/f2c 1256157 0 2024-05-06T12:15:00.809 INFO:tasks.workunit.client.1.smithi190.stdout:1/872: rename d5/d4f/d65/d32/c7d to d5/d4f/d65/d32/dac/d107/c129 0 2024-05-06T12:15:00.809 INFO:tasks.workunit.client.1.smithi190.stdout:4/936: fdatasync d2/d11/d16/d31/f43 0 2024-05-06T12:15:00.812 INFO:tasks.workunit.client.1.smithi190.stdout:1/873: mkdir d5/d4f/d65/d32/d47/d12a 0 2024-05-06T12:15:00.815 INFO:tasks.workunit.client.1.smithi190.stdout:4/937: symlink d2/d11/l141 0 2024-05-06T12:15:00.818 INFO:tasks.workunit.client.1.smithi190.stdout:4/938: write d2/d11/d16/d8e/d80/da0/db2/f111 [213858,119257] 0 2024-05-06T12:15:00.823 INFO:tasks.workunit.client.1.smithi190.stdout:4/939: dread d2/d11/d16/d8e/d80/f9a [0,4194304] 0 2024-05-06T12:15:00.823 INFO:tasks.workunit.client.1.smithi190.stdout:6/741: sync 2024-05-06T12:15:00.825 INFO:tasks.workunit.client.1.smithi190.stdout:9/951: dread db/dd/df/dfe/ddc/d47/fbf [0,4194304] 0 2024-05-06T12:15:00.826 INFO:tasks.workunit.client.1.smithi190.stdout:6/742: fdatasync df/d2c/d61/fab 0 2024-05-06T12:15:00.828 INFO:tasks.workunit.client.1.smithi190.stdout:4/940: mkdir d2/d10/d11a/d11e/d142 0 2024-05-06T12:15:00.830 INFO:tasks.workunit.client.1.smithi190.stdout:4/941: write d2/d11/d16/d13f/de3/dec/f112 [290358,83746] 0 2024-05-06T12:15:00.832 INFO:tasks.workunit.client.1.smithi190.stdout:9/952: stat db/dd/df/dfe/ddc/f32 0 2024-05-06T12:15:00.834 INFO:tasks.workunit.client.1.smithi190.stdout:9/953: write db/dd/d16/d4c/d4e/d5e/fbe [1059699,46039] 0 2024-05-06T12:15:00.837 INFO:tasks.workunit.client.1.smithi190.stdout:6/743: getdents df/d2c/d61/d73/dad 0 2024-05-06T12:15:00.837 INFO:tasks.workunit.client.1.smithi190.stdout:1/874: write d5/d15/d1e/fb8 [155058,80775] 0 2024-05-06T12:15:00.839 INFO:tasks.workunit.client.1.smithi190.stdout:9/954: rename db/dd/df/f106 to db/daa/f13e 0 2024-05-06T12:15:00.843 INFO:tasks.workunit.client.1.smithi190.stdout:9/955: creat db/dd/d16/d4c/d4e/d5e/dce/f13f x:0 0 0 2024-05-06T12:15:00.844 INFO:tasks.workunit.client.1.smithi190.stdout:1/875: rename d5/d15/c5f to d5/d4f/c12b 0 2024-05-06T12:15:00.845 INFO:tasks.workunit.client.1.smithi190.stdout:1/876: chown d5/d4f/d65/d2b/fef 13384797 1 2024-05-06T12:15:00.846 INFO:tasks.workunit.client.1.smithi190.stdout:1/877: rename d5/d4f/d65/d32 to d5/d4f/d65/d32/d47/d12a/d12c 22 2024-05-06T12:15:00.852 INFO:tasks.workunit.client.1.smithi190.stdout:1/878: dwrite d5/d4f/d65/d62/fcc [0,4194304] 0 2024-05-06T12:15:00.856 INFO:tasks.workunit.client.1.smithi190.stdout:1/879: symlink d5/d4f/d65/d32/d47/d53/d98/l12d 0 2024-05-06T12:15:00.857 INFO:tasks.workunit.client.1.smithi190.stdout:6/744: dread df/d2c/d28/de0/fa8 [0,4194304] 0 2024-05-06T12:15:00.858 INFO:tasks.workunit.client.1.smithi190.stdout:1/880: fdatasync d5/d4f/d65/d32/d47/d53/d98/f108 0 2024-05-06T12:15:00.859 INFO:tasks.workunit.client.1.smithi190.stdout:9/956: dwrite db/d23/f68 [0,4194304] 0 2024-05-06T12:15:00.860 INFO:tasks.workunit.client.1.smithi190.stdout:6/745: fdatasync fd 0 2024-05-06T12:15:00.861 INFO:tasks.workunit.client.1.smithi190.stdout:1/881: chown d5/d4f/d65/d62/c90 4 1 2024-05-06T12:15:00.863 INFO:tasks.workunit.client.1.smithi190.stdout:6/746: mknod df/d2c/d1e/cfb 0 2024-05-06T12:15:00.866 INFO:tasks.workunit.client.1.smithi190.stdout:6/747: read df/d2c/d28/f9f [3823998,104567] 0 2024-05-06T12:15:00.943 INFO:tasks.workunit.client.1.smithi190.stdout:1/882: dwrite d5/d4f/d65/d24/d3a/f76 [0,4194304] 0 2024-05-06T12:15:00.944 INFO:tasks.workunit.client.1.smithi190.stdout:6/748: dwrite df/d2c/d28/f7f [0,4194304] 0 2024-05-06T12:15:00.947 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:00 smithi092 ceph-mon[26954]: pgmap v32: 65 pgs: 65 active+clean; 3.3 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 109 MiB/s rd, 240 MiB/s wr, 600 op/s 2024-05-06T12:15:00.951 INFO:tasks.workunit.client.1.smithi190.stdout:9/957: dread db/f5a [0,4194304] 0 2024-05-06T12:15:00.952 INFO:tasks.workunit.client.1.smithi190.stdout:6/749: creat df/ffc x:0 0 0 2024-05-06T12:15:00.953 INFO:tasks.workunit.client.1.smithi190.stdout:1/883: creat d5/d4f/d65/d32/d47/d53/d98/da7/f12e x:0 0 0 2024-05-06T12:15:00.962 INFO:tasks.workunit.client.1.smithi190.stdout:9/958: symlink db/dd/d5c/dac/deb/l140 0 2024-05-06T12:15:00.975 INFO:tasks.workunit.client.1.smithi190.stdout:6/750: dwrite df/d2c/d61/f78 [0,4194304] 0 2024-05-06T12:15:00.983 INFO:tasks.workunit.client.1.smithi190.stdout:6/751: dread df/d2c/d61/f78 [0,4194304] 0 2024-05-06T12:15:00.985 INFO:tasks.workunit.client.1.smithi190.stdout:1/884: dread d5/d4f/d65/d24/f2c [0,4194304] 0 2024-05-06T12:15:00.986 INFO:tasks.workunit.client.1.smithi190.stdout:1/885: write d5/d4f/d65/d32/f4a [1240183,5856] 0 2024-05-06T12:15:00.986 INFO:tasks.workunit.client.1.smithi190.stdout:6/752: mkdir df/d2c/d61/d73/dfd 0 2024-05-06T12:15:00.990 INFO:tasks.workunit.client.1.smithi190.stdout:6/753: fdatasync df/d2c/d1e/d57/f6b 0 2024-05-06T12:15:00.992 INFO:tasks.workunit.client.1.smithi190.stdout:9/959: dread db/dd/d16/fae [0,4194304] 0 2024-05-06T12:15:00.993 INFO:tasks.workunit.client.1.smithi190.stdout:6/754: mknod df/d12/dbd/cfe 0 2024-05-06T12:15:00.995 INFO:tasks.workunit.client.1.smithi190.stdout:9/960: symlink db/dd/l141 0 2024-05-06T12:15:00.997 INFO:tasks.workunit.client.1.smithi190.stdout:1/886: dread d5/d4f/d65/d62/f8d [0,4194304] 0 2024-05-06T12:15:00.998 INFO:tasks.workunit.client.1.smithi190.stdout:9/961: fsync db/dd/f34 0 2024-05-06T12:15:01.000 INFO:tasks.workunit.client.1.smithi190.stdout:9/962: rmdir db/dd/df/dfe 39 2024-05-06T12:15:01.003 INFO:tasks.workunit.client.1.smithi190.stdout:9/963: read db/dd/d16/d4c/d4e/fe1 [1042771,16518] 0 2024-05-06T12:15:01.004 INFO:tasks.workunit.client.1.smithi190.stdout:9/964: read - db/d23/d29/dc4/f139 zero size 2024-05-06T12:15:01.005 INFO:tasks.workunit.client.1.smithi190.stdout:9/965: write db/d23/da4/f108 [522564,54600] 0 2024-05-06T12:15:01.007 INFO:tasks.workunit.client.1.smithi190.stdout:6/755: dread df/d2c/d92/f93 [0,4194304] 0 2024-05-06T12:15:01.007 INFO:tasks.workunit.client.1.smithi190.stdout:9/966: write db/d23/d29/fc6 [303068,14928] 0 2024-05-06T12:15:01.008 INFO:tasks.workunit.client.1.smithi190.stdout:1/887: write d5/d15/f7a [2655053,129237] 0 2024-05-06T12:15:01.010 INFO:tasks.workunit.client.1.smithi190.stdout:1/888: chown d5/d4f/d65/fa9 40 1 2024-05-06T12:15:01.010 INFO:tasks.workunit.client.1.smithi190.stdout:9/967: dread - db/dd/df/dfe/d87/d99/fca zero size 2024-05-06T12:15:01.011 INFO:tasks.workunit.client.1.smithi190.stdout:6/756: truncate df/d2c/d61/d84/f8e 1833901 0 2024-05-06T12:15:01.015 INFO:tasks.workunit.client.1.smithi190.stdout:6/757: rename df/ca6 to df/d2c/dae/dbb/cff 0 2024-05-06T12:15:01.018 INFO:tasks.workunit.client.1.smithi190.stdout:1/889: dwrite d5/d4f/d65/d32/d47/d53/d11e/f128 [0,4194304] 0 2024-05-06T12:15:01.019 INFO:tasks.workunit.client.1.smithi190.stdout:6/758: unlink df/d2c/d61/d84/d8f/f94 0 2024-05-06T12:15:01.020 INFO:tasks.workunit.client.1.smithi190.stdout:9/968: truncate db/f5a 1150867 0 2024-05-06T12:15:01.022 INFO:tasks.workunit.client.1.smithi190.stdout:1/890: dread d5/d4f/d65/d62/f93 [0,4194304] 0 2024-05-06T12:15:01.023 INFO:tasks.workunit.client.1.smithi190.stdout:6/759: chown df/d17/cfa 0 1 2024-05-06T12:15:01.024 INFO:tasks.workunit.client.1.smithi190.stdout:4/942: sync 2024-05-06T12:15:01.025 INFO:tasks.workunit.client.1.smithi190.stdout:1/891: write d5/d4f/d65/d32/d47/d53/d98/da7/de7/fda [436237,88132] 0 2024-05-06T12:15:01.028 INFO:tasks.workunit.client.1.smithi190.stdout:1/892: chown d5/d4f/d65/d62/d9d/dba/f115 11991653 1 2024-05-06T12:15:01.029 INFO:tasks.workunit.client.1.smithi190.stdout:1/893: fsync d5/d6b/ffa 0 2024-05-06T12:15:01.030 INFO:tasks.workunit.client.1.smithi190.stdout:9/969: dwrite db/dd/df/dfe/d87/d22/d49/d88/d9f/df2/f102 [0,4194304] 0 2024-05-06T12:15:01.031 INFO:tasks.workunit.client.1.smithi190.stdout:1/894: chown d5/d4f/d65/d32/d47/d53/d98/da7/de7/c9f 62 1 2024-05-06T12:15:01.033 INFO:tasks.workunit.client.1.smithi190.stdout:6/760: dread df/d17/d37/f9c [0,4194304] 0 2024-05-06T12:15:01.040 INFO:tasks.workunit.client.1.smithi190.stdout:1/895: dwrite d5/d4f/d65/f11d [0,4194304] 0 2024-05-06T12:15:01.040 INFO:tasks.workunit.client.1.smithi190.stdout:6/761: rename df/d2c/d1e/d57/d89/lf2 to df/d2c/d1e/d57/d89/df8/l100 0 2024-05-06T12:15:01.041 INFO:tasks.workunit.client.1.smithi190.stdout:6/762: chown df/d2c/d1e/d57/d53/l5d 0 1 2024-05-06T12:15:01.043 INFO:tasks.workunit.client.1.smithi190.stdout:1/896: mkdir d5/d15/d1e/dff/d10e/d12f 0 2024-05-06T12:15:01.044 INFO:tasks.workunit.client.1.smithi190.stdout:6/763: creat df/d2c/d1e/f101 x:0 0 0 2024-05-06T12:15:01.047 INFO:tasks.workunit.client.1.smithi190.stdout:6/764: mkdir df/d17/d37/d102 0 2024-05-06T12:15:01.049 INFO:tasks.workunit.client.1.smithi190.stdout:6/765: mknod df/d2c/d92/c103 0 2024-05-06T12:15:01.053 INFO:tasks.workunit.client.1.smithi190.stdout:9/970: write db/dd/d16/d4c/d4e/f9d [5368405,93381] 0 2024-05-06T12:15:01.057 INFO:tasks.workunit.client.1.smithi190.stdout:9/971: symlink db/dd/df/dfe/d87/d22/d49/d8c/l142 0 2024-05-06T12:15:01.057 INFO:tasks.workunit.client.1.smithi190.stdout:6/766: dwrite df/d17/fda [0,4194304] 0 2024-05-06T12:15:01.058 INFO:tasks.workunit.client.1.smithi190.stdout:6/767: chown df/d12/c85 806 1 2024-05-06T12:15:01.059 INFO:tasks.workunit.client.1.smithi190.stdout:9/972: fdatasync db/dd/df/dfe/d87/d22/d49/d88/f9c 0 2024-05-06T12:15:01.064 INFO:tasks.workunit.client.1.smithi190.stdout:6/768: link df/d2c/d61/d87/fba df/d2c/d61/d87/df9/f104 0 2024-05-06T12:15:01.067 INFO:tasks.workunit.client.1.smithi190.stdout:9/973: rename db/dd/df/dfe/ddc/d47/d13d to db/dd/df/d124/d143 0 2024-05-06T12:15:01.071 INFO:tasks.workunit.client.1.smithi190.stdout:6/769: mknod df/d2c/d28/c105 0 2024-05-06T12:15:01.072 INFO:tasks.workunit.client.1.smithi190.stdout:1/897: dwrite d5/d4f/d65/d32/d47/f77 [0,4194304] 0 2024-05-06T12:15:01.073 INFO:tasks.workunit.client.1.smithi190.stdout:9/974: rename db/d23/c71 to db/daa/c144 0 2024-05-06T12:15:01.075 INFO:tasks.workunit.client.1.smithi190.stdout:1/898: symlink d5/d4f/d65/d32/d47/de5/l130 0 2024-05-06T12:15:01.077 INFO:tasks.workunit.client.1.smithi190.stdout:1/899: write d5/d4f/d65/d24/d43/de2/d114/f86 [1274374,27028] 0 2024-05-06T12:15:01.078 INFO:tasks.workunit.client.1.smithi190.stdout:9/975: mknod db/dd/d16/d4c/d4e/d5e/dce/c145 0 2024-05-06T12:15:01.079 INFO:tasks.workunit.client.1.smithi190.stdout:1/900: symlink d5/d4f/d65/d24/d43/da6/ded/l131 0 2024-05-06T12:15:01.080 INFO:tasks.workunit.client.1.smithi190.stdout:6/770: getdents df/d12/dbd 0 2024-05-06T12:15:01.082 INFO:tasks.workunit.client.1.smithi190.stdout:9/976: fdatasync db/dd/f8d 0 2024-05-06T12:15:01.085 INFO:tasks.workunit.client.1.smithi190.stdout:6/771: symlink df/d2c/d1e/d57/l106 0 2024-05-06T12:15:01.086 INFO:tasks.workunit.client.1.smithi190.stdout:1/901: creat d5/d4f/d65/d32/f132 x:0 0 0 2024-05-06T12:15:01.087 INFO:tasks.workunit.client.1.smithi190.stdout:9/977: creat db/dd/df/dfe/d87/d99/f146 x:0 0 0 2024-05-06T12:15:01.095 INFO:tasks.workunit.client.1.smithi190.stdout:9/978: symlink db/dd/d16/d4c/d4e/l147 0 2024-05-06T12:15:01.096 INFO:tasks.workunit.client.1.smithi190.stdout:9/979: chown db/dd/d5c/dac/le9 6239 1 2024-05-06T12:15:01.097 INFO:tasks.workunit.client.1.smithi190.stdout:1/902: getdents d5/d4f/d65 0 2024-05-06T12:15:01.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:00 smithi190 ceph-mon[33798]: pgmap v32: 65 pgs: 65 active+clean; 3.3 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 109 MiB/s rd, 240 MiB/s wr, 600 op/s 2024-05-06T12:15:01.098 INFO:tasks.workunit.client.1.smithi190.stdout:9/980: fdatasync db/dd/df/dfe/d87/d22/d49/d88/d9f/df2/fe3 0 2024-05-06T12:15:01.100 INFO:tasks.workunit.client.1.smithi190.stdout:9/981: chown db/dd/df/d124 4874 1 2024-05-06T12:15:01.100 INFO:tasks.workunit.client.1.smithi190.stdout:1/903: write d5/d4f/d65/d32/dac/fcb [353207,78335] 0 2024-05-06T12:15:01.102 INFO:tasks.workunit.client.1.smithi190.stdout:9/982: write db/d23/d29/dc4/f139 [890187,125458] 0 2024-05-06T12:15:01.105 INFO:tasks.workunit.client.1.smithi190.stdout:1/904: mknod d5/d4f/d65/d32/d47/d53/d98/da7/de7/d126/c133 0 2024-05-06T12:15:01.105 INFO:tasks.workunit.client.1.smithi190.stdout:6/772: dwrite df/d2c/d61/d84/d8f/fd2 [0,4194304] 0 2024-05-06T12:15:01.109 INFO:tasks.workunit.client.1.smithi190.stdout:1/905: creat d5/d15/d1e/dff/f134 x:0 0 0 2024-05-06T12:15:01.110 INFO:tasks.workunit.client.1.smithi190.stdout:9/983: creat db/dd/d16/d4c/f148 x:0 0 0 2024-05-06T12:15:01.112 INFO:tasks.workunit.client.1.smithi190.stdout:6/773: rename df/d2c/d61/d73/dad/fb0 to df/d2c/dae/f107 0 2024-05-06T12:15:01.116 INFO:tasks.workunit.client.1.smithi190.stdout:9/984: rename db/d23/d29/l5d to db/dd/df/dfe/d87/d99/d11c/l149 0 2024-05-06T12:15:01.117 INFO:tasks.workunit.client.1.smithi190.stdout:6/774: unlink df/d2c/d28/dd1/lf5 0 2024-05-06T12:15:01.118 INFO:tasks.workunit.client.1.smithi190.stdout:6/775: read df/d17/f38 [21400,59374] 0 2024-05-06T12:15:01.125 INFO:tasks.workunit.client.1.smithi190.stdout:1/906: rmdir d5/d4f/d65/d32 39 2024-05-06T12:15:01.129 INFO:tasks.workunit.client.1.smithi190.stdout:1/907: write d5/d4f/d65/d32/d47/d53/fdc [4706895,40561] 0 2024-05-06T12:15:01.133 INFO:tasks.workunit.client.1.smithi190.stdout:9/985: write db/dd/df/dfe/f134 [1997977,13135] 0 2024-05-06T12:15:01.135 INFO:tasks.workunit.client.1.smithi190.stdout:1/908: creat d5/d4f/d65/d32/d47/d53/d98/da7/f135 x:0 0 0 2024-05-06T12:15:01.138 INFO:tasks.workunit.client.1.smithi190.stdout:1/909: write d5/d4f/d65/d62/d9d/dba/f115 [4918311,42917] 0 2024-05-06T12:15:01.138 INFO:tasks.workunit.client.1.smithi190.stdout:6/776: dread df/d12/f2b [0,4194304] 0 2024-05-06T12:15:01.138 INFO:tasks.workunit.client.1.smithi190.stdout:9/986: fsync db/dd/df/dfe/f134 0 2024-05-06T12:15:01.140 INFO:tasks.workunit.client.1.smithi190.stdout:6/777: chown df/d2c/d1e/d57/d89/lbe 198676750 1 2024-05-06T12:15:01.143 INFO:tasks.workunit.client.1.smithi190.stdout:6/778: rmdir df/d2c/dae/dbb 39 2024-05-06T12:15:01.145 INFO:tasks.workunit.client.1.smithi190.stdout:9/987: write db/dd/df/dfe/ddc/d47/fbf [4570999,117107] 0 2024-05-06T12:15:01.145 INFO:tasks.workunit.client.1.smithi190.stdout:6/779: dread - df/d12/f8b zero size 2024-05-06T12:15:01.147 INFO:tasks.workunit.client.1.smithi190.stdout:6/780: creat df/d2c/d61/d87/f108 x:0 0 0 2024-05-06T12:15:01.148 INFO:tasks.workunit.client.1.smithi190.stdout:1/910: dread d5/d4f/d65/d24/d43/de2/d114/f79 [0,4194304] 0 2024-05-06T12:15:01.150 INFO:tasks.workunit.client.1.smithi190.stdout:1/911: rmdir d5/d4f/d65/d24/d3a 39 2024-05-06T12:15:01.151 INFO:tasks.workunit.client.1.smithi190.stdout:9/988: write db/dd/df/dfe/d87/d22/d49/d88/d9f/fef [512455,73442] 0 2024-05-06T12:15:01.152 INFO:tasks.workunit.client.1.smithi190.stdout:6/781: write df/d2c/d1e/d57/f6a [1307272,49831] 0 2024-05-06T12:15:01.152 INFO:tasks.workunit.client.1.smithi190.stdout:1/912: chown d5/d4f/d68/f106 966 1 2024-05-06T12:15:01.152 INFO:tasks.workunit.client.1.smithi190.stdout:9/989: fsync db/dd/df/dfe/f134 0 2024-05-06T12:15:01.154 INFO:tasks.workunit.client.1.smithi190.stdout:1/913: write d5/d4f/d65/d62/d9d/dba/fd0 [1674651,57407] 0 2024-05-06T12:15:01.156 INFO:tasks.workunit.client.1.smithi190.stdout:6/782: symlink df/d2c/d28/dd1/de5/l109 0 2024-05-06T12:15:01.157 INFO:tasks.workunit.client.1.smithi190.stdout:9/990: creat db/d23/f14a x:0 0 0 2024-05-06T12:15:01.158 INFO:tasks.workunit.client.1.smithi190.stdout:1/914: creat d5/d4f/d65/d32/dac/d107/f136 x:0 0 0 2024-05-06T12:15:01.159 INFO:tasks.workunit.client.1.smithi190.stdout:9/991: chown db/dd/ce7 101371 1 2024-05-06T12:15:01.161 INFO:tasks.workunit.client.1.smithi190.stdout:1/915: chown d5/d4f/d65/d2b/c112 12940 1 2024-05-06T12:15:01.162 INFO:tasks.workunit.client.1.smithi190.stdout:1/916: stat d5/d4f/d65/d62/f8d 0 2024-05-06T12:15:01.163 INFO:tasks.workunit.client.1.smithi190.stdout:6/783: truncate df/fc1 334608 0 2024-05-06T12:15:01.165 INFO:tasks.workunit.client.1.smithi190.stdout:9/992: dwrite db/d23/da4/f108 [0,4194304] 0 2024-05-06T12:15:01.169 INFO:tasks.workunit.client.1.smithi190.stdout:9/993: creat db/dd/d5c/dac/deb/d10e/f14b x:0 0 0 2024-05-06T12:15:01.170 INFO:tasks.workunit.client.1.smithi190.stdout:9/994: chown db/dd/d16/d4c/d4e/d5e/dce/f133 2 1 2024-05-06T12:15:01.171 INFO:tasks.workunit.client.1.smithi190.stdout:1/917: rename d5/d4f/d65/d62/d9d/dba/f115 to d5/d4f/d65/d24/f137 0 2024-05-06T12:15:01.174 INFO:tasks.workunit.client.1.smithi190.stdout:9/995: symlink db/dd/df/dfe/d136/l14c 0 2024-05-06T12:15:01.175 INFO:tasks.workunit.client.1.smithi190.stdout:6/784: rename df/d2c/d28/c50 to df/d12/dbd/c10a 0 2024-05-06T12:15:01.176 INFO:tasks.workunit.client.1.smithi190.stdout:1/918: symlink d5/d4f/d65/d24/d43/de2/d114/d101/l138 0 2024-05-06T12:15:01.177 INFO:tasks.workunit.client.1.smithi190.stdout:6/785: mknod df/d17/d37/c10b 0 2024-05-06T12:15:01.178 INFO:tasks.workunit.client.1.smithi190.stdout:9/996: fsync db/d23/f5b 0 2024-05-06T12:15:01.184 INFO:tasks.workunit.client.1.smithi190.stdout:9/997: dread db/dd/df/dfe/d87/d22/d49/d88/d9f/df2/f102 [0,4194304] 0 2024-05-06T12:15:01.185 INFO:tasks.workunit.client.1.smithi190.stdout:1/919: getdents d5/d4f/d65/d24/d43/de2 0 2024-05-06T12:15:01.186 INFO:tasks.workunit.client.1.smithi190.stdout:1/920: fdatasync d5/d4f/d65/d2b/f35 0 2024-05-06T12:15:01.188 INFO:tasks.workunit.client.1.smithi190.stdout:1/921: chown d5/d15/d1e/d94/fb2 1964049 1 2024-05-06T12:15:01.190 INFO:tasks.workunit.client.1.smithi190.stdout:9/998: dwrite db/d23/fdd [0,4194304] 0 2024-05-06T12:15:01.191 INFO:tasks.workunit.client.1.smithi190.stdout:9/999: chown db/dd/df/dfe/d87/d22/c40 75 1 2024-05-06T12:15:01.192 INFO:tasks.workunit.client.1.smithi190.stdout:6/786: write df/d12/dbd/fdb [3515019,94475] 0 2024-05-06T12:15:01.193 INFO:tasks.workunit.client.1.smithi190.stdout:6/787: chown df/d2c/dae/le1 234169031 1 2024-05-06T12:15:01.194 INFO:tasks.workunit.client.1.smithi190.stdout:6/788: dread - df/d2c/d61/fab zero size 2024-05-06T12:15:01.197 INFO:tasks.workunit.client.1.smithi190.stdout:1/922: getdents d5/d4f/d65/d32 0 2024-05-06T12:15:01.198 INFO:tasks.workunit.client.1.smithi190.stdout:6/789: rmdir df/d2c/d28 39 2024-05-06T12:15:01.198 INFO:tasks.workunit.client.1.smithi190.stdout:4/943: sync 2024-05-06T12:15:01.200 INFO:tasks.workunit.client.1.smithi190.stdout:4/944: stat d2/d11/d16/d31/fc7 0 2024-05-06T12:15:01.203 INFO:tasks.workunit.client.1.smithi190.stdout:1/923: rename d5/d4f/d65/d32/d47/d53/d98/f108 to d5/d4f/d68/f139 0 2024-05-06T12:15:01.215 INFO:tasks.workunit.client.1.smithi190.stdout:6/790: dwrite df/d2c/fa9 [0,4194304] 0 2024-05-06T12:15:01.215 INFO:tasks.workunit.client.1.smithi190.stdout:1/924: dwrite d5/f16 [0,4194304] 0 2024-05-06T12:15:01.223 INFO:tasks.workunit.client.1.smithi190.stdout:6/791: getdents df/d2c/d92 0 2024-05-06T12:15:01.225 INFO:tasks.workunit.client.1.smithi190.stdout:1/925: dwrite d5/d4f/d65/f5b [0,4194304] 0 2024-05-06T12:15:01.225 INFO:tasks.workunit.client.1.smithi190.stdout:4/945: dread d2/d11/d16/d8e/d7b/de8/fca [0,4194304] 0 2024-05-06T12:15:01.226 INFO:tasks.workunit.client.1.smithi190.stdout:1/926: chown d5/d4f/c58 61120097 1 2024-05-06T12:15:01.230 INFO:tasks.workunit.client.1.smithi190.stdout:1/927: fdatasync d5/d4f/d65/d62/f93 0 2024-05-06T12:15:01.232 INFO:tasks.workunit.client.1.smithi190.stdout:1/928: fdatasync d5/d4f/d65/d32/d47/d53/fc0 0 2024-05-06T12:15:01.234 INFO:tasks.workunit.client.1.smithi190.stdout:4/946: write d2/d11/d16/d31/fc7 [1308018,10981] 0 2024-05-06T12:15:01.238 INFO:tasks.workunit.client.1.smithi190.stdout:4/947: symlink d2/d11/l143 0 2024-05-06T12:15:01.239 INFO:tasks.workunit.client.1.smithi190.stdout:6/792: dwrite df/d2c/f75 [0,4194304] 0 2024-05-06T12:15:01.241 INFO:tasks.workunit.client.1.smithi190.stdout:4/948: write d2/d11/d16/d8e/d7b/de8/f116 [1559219,68023] 0 2024-05-06T12:15:01.242 INFO:tasks.workunit.client.1.smithi190.stdout:6/793: chown df/d2c/d1e/d57/d89/cbf 656328359 1 2024-05-06T12:15:01.245 INFO:tasks.workunit.client.1.smithi190.stdout:4/949: chown d2/d11/d16/d13f/da4/dfa 94 1 2024-05-06T12:15:01.246 INFO:tasks.workunit.client.1.smithi190.stdout:6/794: unlink df/d17/lb9 0 2024-05-06T12:15:01.247 INFO:tasks.workunit.client.1.smithi190.stdout:4/950: symlink d2/d11/d16/d13f/de3/dec/dff/l144 0 2024-05-06T12:15:01.249 INFO:tasks.workunit.client.1.smithi190.stdout:1/929: write d5/d4f/d65/d32/d47/f96 [438281,64641] 0 2024-05-06T12:15:01.250 INFO:tasks.workunit.client.1.smithi190.stdout:6/795: read df/d2c/d61/d84/fb4 [1120608,45990] 0 2024-05-06T12:15:01.254 INFO:tasks.workunit.client.1.smithi190.stdout:1/930: rename d5/d4f/d65/cc9 to d5/d4f/c13a 0 2024-05-06T12:15:01.254 INFO:tasks.workunit.client.1.smithi190.stdout:4/951: dwrite d2/d11/d16/d8e/d7b/de8/d9e/f12a [0,4194304] 0 2024-05-06T12:15:01.256 INFO:tasks.workunit.client.1.smithi190.stdout:6/796: rmdir df/d2c/d1e/d57/d53 39 2024-05-06T12:15:01.259 INFO:tasks.workunit.client.1.smithi190.stdout:1/931: truncate d5/d4f/d65/d2b/f95 588185 0 2024-05-06T12:15:01.260 INFO:tasks.workunit.client.1.smithi190.stdout:4/952: symlink d2/d11/d16/d13f/da4/dfa/d10f/d120/l145 0 2024-05-06T12:15:01.261 INFO:tasks.workunit.client.1.smithi190.stdout:4/953: write d2/f32 [8368525,39098] 0 2024-05-06T12:15:01.264 INFO:tasks.workunit.client.1.smithi190.stdout:1/932: mkdir d5/d4f/d65/d24/d13b 0 2024-05-06T12:15:01.270 INFO:tasks.workunit.client.1.smithi190.stdout:4/954: dwrite d2/d11/d16/fe7 [0,4194304] 0 2024-05-06T12:15:01.271 INFO:tasks.workunit.client.1.smithi190.stdout:1/933: rename d5/d4f/d65/d62/d9d/cd2 to d5/d4f/d65/d62/d9d/dba/c13c 0 2024-05-06T12:15:01.278 INFO:tasks.workunit.client.1.smithi190.stdout:6/797: dwrite df/d2c/d1e/f52 [0,4194304] 0 2024-05-06T12:15:01.279 INFO:tasks.workunit.client.1.smithi190.stdout:4/955: mknod d2/d11/d16/d8e/d7b/de8/d9e/d6e/daa/d125/c146 0 2024-05-06T12:15:01.279 INFO:tasks.workunit.client.1.smithi190.stdout:6/798: chown df/d2c/dae 102010445 1 2024-05-06T12:15:01.280 INFO:tasks.workunit.client.1.smithi190.stdout:6/799: dread - df/d2c/d1e/f101 zero size 2024-05-06T12:15:01.280 INFO:tasks.workunit.client.1.smithi190.stdout:4/956: readlink d2/d11/d16/d8e/d7b/de8/dd1/def/l102 0 2024-05-06T12:15:01.284 INFO:tasks.workunit.client.1.smithi190.stdout:6/800: mkdir df/d2c/d61/d84/d8f/d10c 0 2024-05-06T12:15:01.285 INFO:tasks.workunit.client.1.smithi190.stdout:4/957: mkdir d2/d11/d16/d8e/d7b/de8/d9e/d6e/daa/d147 0 2024-05-06T12:15:01.285 INFO:tasks.workunit.client.1.smithi190.stdout:6/801: chown df/d17/l3a 258692684 1 2024-05-06T12:15:01.289 INFO:tasks.workunit.client.1.smithi190.stdout:4/958: creat d2/d11/d16/d8e/d7b/de8/d9e/d6e/f148 x:0 0 0 2024-05-06T12:15:01.292 INFO:tasks.workunit.client.1.smithi190.stdout:4/959: creat d2/d11/d16/d8e/d7b/de8/d9e/d6e/f149 x:0 0 0 2024-05-06T12:15:01.295 INFO:tasks.workunit.client.1.smithi190.stdout:1/934: write d5/d15/d1e/f27 [1411581,107123] 0 2024-05-06T12:15:01.300 INFO:tasks.workunit.client.1.smithi190.stdout:6/802: dwrite fd [0,4194304] 0 2024-05-06T12:15:01.301 INFO:tasks.workunit.client.1.smithi190.stdout:4/960: dwrite d2/d11/f4c [4194304,4194304] 0 2024-05-06T12:15:01.356 INFO:tasks.workunit.client.1.smithi190.stdout:4/961: rename d2/d11/d16/d41/l47 to d2/d11/d16/d8e/d7b/de8/d66/l14a 0 2024-05-06T12:15:01.359 INFO:tasks.workunit.client.1.smithi190.stdout:4/962: creat d2/d11/d16/d8e/d7b/de8/d9e/d6e/d78/f14b x:0 0 0 2024-05-06T12:15:01.365 INFO:tasks.workunit.client.1.smithi190.stdout:4/963: dwrite d2/d11/d16/d8e/d7b/de8/d82/f98 [4194304,4194304] 0 2024-05-06T12:15:01.365 INFO:tasks.workunit.client.1.smithi190.stdout:6/803: dread df/f99 [0,4194304] 0 2024-05-06T12:15:01.366 INFO:tasks.workunit.client.1.smithi190.stdout:4/964: readlink d2/d11/d16/d8e/d80/le5 0 2024-05-06T12:15:01.379 INFO:tasks.workunit.client.1.smithi190.stdout:6/804: truncate df/d12/dbd/fdb 3225786 0 2024-05-06T12:15:01.382 INFO:tasks.workunit.client.1.smithi190.stdout:6/805: unlink df/d2c/d92/ccd 0 2024-05-06T12:15:01.382 INFO:tasks.workunit.client.1.smithi190.stdout:4/965: dwrite d2/fb0 [0,4194304] 0 2024-05-06T12:15:01.386 INFO:tasks.workunit.client.1.smithi190.stdout:6/806: symlink df/d2c/d61/d73/dfd/l10d 0 2024-05-06T12:15:01.387 INFO:tasks.workunit.client.1.smithi190.stdout:6/807: mkdir df/d2c/d61/d84/d8f/d10c/d10e 0 2024-05-06T12:15:01.390 INFO:tasks.workunit.client.1.smithi190.stdout:6/808: dread - df/d2c/d1e/d57/d53/de8/fea zero size 2024-05-06T12:15:01.391 INFO:tasks.workunit.client.1.smithi190.stdout:6/809: fdatasync df/d2c/d61/d87/fe7 0 2024-05-06T12:15:01.395 INFO:tasks.workunit.client.1.smithi190.stdout:4/966: dread d2/f32 [4194304,4194304] 0 2024-05-06T12:15:01.396 INFO:tasks.workunit.client.1.smithi190.stdout:4/967: fdatasync d2/d11/d16/d31/f5c 0 2024-05-06T12:15:01.399 INFO:tasks.workunit.client.1.smithi190.stdout:4/968: rmdir d2/d11/d16/d8e/d7b/de8/d82/dd7 39 2024-05-06T12:15:01.400 INFO:tasks.workunit.client.1.smithi190.stdout:4/969: fdatasync d2/d11/d16/d8e/d7b/de8/fca 0 2024-05-06T12:15:01.404 INFO:tasks.workunit.client.1.smithi190.stdout:4/970: creat d2/d11/d16/d13f/da4/dfa/f14c x:0 0 0 2024-05-06T12:15:01.420 INFO:tasks.workunit.client.1.smithi190.stdout:4/971: dread d2/d11/d16/d8e/d80/f97 [0,4194304] 0 2024-05-06T12:15:01.424 INFO:tasks.workunit.client.1.smithi190.stdout:4/972: creat d2/d11/d16/d8e/d80/da0/db2/f14d x:0 0 0 2024-05-06T12:15:01.426 INFO:tasks.workunit.client.1.smithi190.stdout:4/973: symlink d2/d10/d11a/d11e/l14e 0 2024-05-06T12:15:01.427 INFO:tasks.workunit.client.1.smithi190.stdout:4/974: chown d2/d11/d16/f38 2899 1 2024-05-06T12:15:01.428 INFO:tasks.workunit.client.1.smithi190.stdout:4/975: rmdir d2/d11/d16/d13f/da4/dfa/d10f/d120 39 2024-05-06T12:15:01.552 INFO:tasks.workunit.client.1.smithi190.stdout:1/935: sync 2024-05-06T12:15:01.552 INFO:tasks.workunit.client.1.smithi190.stdout:1/936: fdatasync d5/d4f/d65/d24/d43/da6/fb9 0 2024-05-06T12:15:01.556 INFO:tasks.workunit.client.1.smithi190.stdout:1/937: dread d5/d4f/d65/d32/d47/f77 [0,4194304] 0 2024-05-06T12:15:01.556 INFO:tasks.workunit.client.1.smithi190.stdout:1/938: stat d5/d4f/d65/d32/dac/ddd 0 2024-05-06T12:15:01.557 INFO:tasks.workunit.client.1.smithi190.stdout:1/939: chown d5/d4f/d65/d24/f10f 21966 1 2024-05-06T12:15:01.566 INFO:tasks.workunit.client.1.smithi190.stdout:1/940: write d5/d4f/d65/d2b/fa8 [411072,69475] 0 2024-05-06T12:15:01.568 INFO:tasks.workunit.client.1.smithi190.stdout:1/941: dread d5/d4f/d65/d62/f8d [0,4194304] 0 2024-05-06T12:15:01.570 INFO:tasks.workunit.client.1.smithi190.stdout:1/942: creat d5/d15/d1e/dff/f13d x:0 0 0 2024-05-06T12:15:01.573 INFO:tasks.workunit.client.1.smithi190.stdout:1/943: rename d5/d4f/d65/d32/f61 to d5/d15/d1e/f13e 0 2024-05-06T12:15:01.576 INFO:tasks.workunit.client.1.smithi190.stdout:1/944: getdents d5/d4f/d65/d24/d43 0 2024-05-06T12:15:01.580 INFO:tasks.workunit.client.1.smithi190.stdout:1/945: link d5/d4f/f50 d5/d15/d1e/f13f 0 2024-05-06T12:15:01.581 INFO:tasks.workunit.client.1.smithi190.stdout:1/946: fdatasync d5/d4f/d65/f11d 0 2024-05-06T12:15:01.584 INFO:tasks.workunit.client.1.smithi190.stdout:1/947: read - d5/d4f/d65/d32/d47/d53/d98/da7/f12e zero size 2024-05-06T12:15:01.586 INFO:tasks.workunit.client.1.smithi190.stdout:1/948: fdatasync d5/d4f/d65/fa9 0 2024-05-06T12:15:01.588 INFO:tasks.workunit.client.1.smithi190.stdout:1/949: mkdir d5/d4f/d65/d24/d13b/d140 0 2024-05-06T12:15:01.591 INFO:tasks.workunit.client.1.smithi190.stdout:1/950: rename d5/d15/cbd to d5/d15/d1e/d94/c141 0 2024-05-06T12:15:01.592 INFO:tasks.workunit.client.1.smithi190.stdout:1/951: mknod d5/d4f/d65/d24/d43/c142 0 2024-05-06T12:15:01.594 INFO:tasks.workunit.client.1.smithi190.stdout:1/952: mknod d5/d4f/d65/d32/d47/d53/d98/da7/de7/c143 0 2024-05-06T12:15:01.595 INFO:tasks.workunit.client.1.smithi190.stdout:1/953: truncate d5/d4f/d65/d24/f10f 591812 0 2024-05-06T12:15:01.596 INFO:tasks.workunit.client.1.smithi190.stdout:1/954: mkdir d5/d4f/d65/d32/dac/d144 0 2024-05-06T12:15:01.598 INFO:tasks.workunit.client.1.smithi190.stdout:1/955: mkdir d5/d15/d1e/dff/d10e/d12f/d145 0 2024-05-06T12:15:01.599 INFO:tasks.workunit.client.1.smithi190.stdout:1/956: fdatasync d5/d4f/d65/d32/f132 0 2024-05-06T12:15:01.600 INFO:tasks.workunit.client.1.smithi190.stdout:6/810: sync 2024-05-06T12:15:01.603 INFO:tasks.workunit.client.1.smithi190.stdout:6/811: mknod df/d2c/d61/d84/d8f/c10f 0 2024-05-06T12:15:01.604 INFO:tasks.workunit.client.1.smithi190.stdout:6/812: mknod df/d2c/d61/c110 0 2024-05-06T12:15:01.606 INFO:tasks.workunit.client.1.smithi190.stdout:6/813: stat df/d2c/d28/f40 0 2024-05-06T12:15:01.656 INFO:tasks.workunit.client.1.smithi190.stdout:4/976: sync 2024-05-06T12:15:01.679 INFO:tasks.workunit.client.1.smithi190.stdout:1/957: dwrite d5/d4f/d65/d24/d43/de2/d114/f84 [0,4194304] 0 2024-05-06T12:15:01.685 INFO:tasks.workunit.client.1.smithi190.stdout:1/958: mknod d5/d4f/d65/d24/d13b/d140/c146 0 2024-05-06T12:15:01.706 INFO:tasks.workunit.client.1.smithi190.stdout:6/814: mknod df/d2c/d92/c111 0 2024-05-06T12:15:01.709 INFO:tasks.workunit.client.1.smithi190.stdout:6/815: dread df/d2c/d61/f78 [0,4194304] 0 2024-05-06T12:15:01.727 INFO:tasks.workunit.client.1.smithi190.stdout:6/816: rename df/d2c/d1e/d57/d53 to df/d2c/d61/d73/dad/d112 0 2024-05-06T12:15:01.729 INFO:tasks.workunit.client.1.smithi190.stdout:6/817: mknod df/d2c/d61/d73/dad/d112/de8/c113 0 2024-05-06T12:15:01.730 INFO:tasks.workunit.client.1.smithi190.stdout:6/818: chown df/d2c/d61/ld7 1370066 1 2024-05-06T12:15:01.742 INFO:tasks.workunit.client.1.smithi190.stdout:1/959: creat d5/d4f/d65/d32/d47/f147 x:0 0 0 2024-05-06T12:15:01.744 INFO:tasks.workunit.client.1.smithi190.stdout:1/960: mknod d5/d4f/d65/d32/d47/d53/de8/c148 0 2024-05-06T12:15:01.745 INFO:tasks.workunit.client.1.smithi190.stdout:1/961: dread d5/d4f/d65/d62/f93 [0,4194304] 0 2024-05-06T12:15:01.746 INFO:tasks.workunit.client.1.smithi190.stdout:1/962: write d5/d4f/d65/d2b/fa8 [1238613,46861] 0 2024-05-06T12:15:01.750 INFO:tasks.workunit.client.1.smithi190.stdout:1/963: write d5/d4f/d65/d32/f4a [4810640,44014] 0 2024-05-06T12:15:01.750 INFO:tasks.workunit.client.1.smithi190.stdout:1/964: read d5/d15/f30 [257504,2401] 0 2024-05-06T12:15:01.754 INFO:tasks.workunit.client.1.smithi190.stdout:4/977: link d2/f55 d2/d11/d16/f14f 0 2024-05-06T12:15:01.755 INFO:tasks.workunit.client.1.smithi190.stdout:1/965: creat d5/d4f/d65/d32/d47/d12a/f149 x:0 0 0 2024-05-06T12:15:01.757 INFO:tasks.workunit.client.1.smithi190.stdout:1/966: dread d5/d15/fbf [0,4194304] 0 2024-05-06T12:15:01.761 INFO:tasks.workunit.client.1.smithi190.stdout:6/819: truncate df/f18 671398 0 2024-05-06T12:15:01.762 INFO:tasks.workunit.client.1.smithi190.stdout:6/820: write df/d2c/d1e/f101 [927069,19305] 0 2024-05-06T12:15:01.770 INFO:tasks.workunit.client.1.smithi190.stdout:1/967: rmdir d5/d4f/d65/d32/d47/d12a 39 2024-05-06T12:15:01.771 INFO:tasks.workunit.client.1.smithi190.stdout:4/978: write d2/f59 [2470714,62321] 0 2024-05-06T12:15:01.773 INFO:tasks.workunit.client.1.smithi190.stdout:6/821: write df/fc1 [998131,28187] 0 2024-05-06T12:15:01.777 INFO:tasks.workunit.client.1.smithi190.stdout:4/979: dwrite d2/d11/d16/fe7 [0,4194304] 0 2024-05-06T12:15:01.816 INFO:tasks.workunit.client.1.smithi190.stdout:1/968: creat d5/d4f/d65/d32/d47/d53/de8/f14a x:0 0 0 2024-05-06T12:15:01.816 INFO:tasks.workunit.client.1.smithi190.stdout:6/822: rmdir df/d2c/d92/da3 39 2024-05-06T12:15:01.819 INFO:tasks.workunit.client.1.smithi190.stdout:1/969: readlink d5/d4f/d65/l78 0 2024-05-06T12:15:01.821 INFO:tasks.workunit.client.1.smithi190.stdout:6/823: symlink df/d2c/d28/dd1/l114 0 2024-05-06T12:15:01.822 INFO:tasks.workunit.client.1.smithi190.stdout:4/980: truncate d2/d11/d16/d8e/d7b/de8/fca 2956865 0 2024-05-06T12:15:01.824 INFO:tasks.workunit.client.1.smithi190.stdout:6/824: write df/d2c/d28/de0/fa8 [7714332,60362] 0 2024-05-06T12:15:01.825 INFO:tasks.workunit.client.1.smithi190.stdout:4/981: symlink d2/d11/d16/d8e/d7b/de8/d9e/d6e/daa/ddc/l150 0 2024-05-06T12:15:01.827 INFO:tasks.workunit.client.1.smithi190.stdout:6/825: chown df/d12/cbc 234 1 2024-05-06T12:15:01.828 INFO:tasks.workunit.client.1.smithi190.stdout:1/970: dwrite d5/d4f/d65/d32/f37 [0,4194304] 0 2024-05-06T12:15:01.830 INFO:tasks.workunit.client.1.smithi190.stdout:1/971: read - d5/d4f/d65/d24/d43/ffb zero size 2024-05-06T12:15:01.831 INFO:tasks.workunit.client.1.smithi190.stdout:6/826: dread df/d2c/d1e/d57/d89/fd5 [0,4194304] 0 2024-05-06T12:15:01.836 INFO:tasks.workunit.client.1.smithi190.stdout:6/827: rename df/d2c/d28/de0/fac to df/d2c/d28/deb/f115 0 2024-05-06T12:15:01.837 INFO:tasks.workunit.client.1.smithi190.stdout:1/972: creat d5/d15/d1e/dff/d10e/d12f/d145/f14b x:0 0 0 2024-05-06T12:15:01.840 INFO:tasks.workunit.client.1.smithi190.stdout:6/828: write df/d2c/f25 [2396358,25810] 0 2024-05-06T12:15:01.842 INFO:tasks.workunit.client.1.smithi190.stdout:1/973: creat d5/d4f/d10c/f14c x:0 0 0 2024-05-06T12:15:01.844 INFO:tasks.workunit.client.1.smithi190.stdout:6/829: creat df/d2c/d61/d84/f116 x:0 0 0 2024-05-06T12:15:01.847 INFO:tasks.workunit.client.1.smithi190.stdout:1/974: dread d5/d4f/d65/d24/d43/de2/d114/f84 [0,4194304] 0 2024-05-06T12:15:01.850 INFO:tasks.workunit.client.1.smithi190.stdout:6/830: rename df/f47 to df/d2c/d61/d73/dad/d112/de8/f117 0 2024-05-06T12:15:01.851 INFO:tasks.workunit.client.1.smithi190.stdout:1/975: symlink d5/d15/d1e/l14d 0 2024-05-06T12:15:01.853 INFO:tasks.workunit.client.1.smithi190.stdout:6/831: creat df/d2c/d28/deb/df1/f118 x:0 0 0 2024-05-06T12:15:01.854 INFO:tasks.workunit.client.1.smithi190.stdout:1/976: mkdir d5/d15/d105/d14e 0 2024-05-06T12:15:01.859 INFO:tasks.workunit.client.1.smithi190.stdout:6/832: dwrite df/d17/f26 [0,4194304] 0 2024-05-06T12:15:01.866 INFO:tasks.workunit.client.1.smithi190.stdout:6/833: rename df/d2c/d92/da3 to df/d2c/d61/d84/d8f/d10c/d10e/d119 0 2024-05-06T12:15:01.871 INFO:tasks.workunit.client.1.smithi190.stdout:4/982: dread d2/d11/d16/d8e/d80/da0/db2/fc5 [0,4194304] 0 2024-05-06T12:15:01.871 INFO:tasks.workunit.client.1.smithi190.stdout:4/983: stat d2/d11/d16/d13f/f126 0 2024-05-06T12:15:01.874 INFO:tasks.workunit.client.1.smithi190.stdout:6/834: dwrite df/d2c/d1e/d57/f6b [4194304,4194304] 0 2024-05-06T12:15:01.876 INFO:tasks.workunit.client.1.smithi190.stdout:4/984: symlink d2/d11/d16/d13f/da4/dfa/l151 0 2024-05-06T12:15:01.876 INFO:tasks.workunit.client.1.smithi190.stdout:6/835: truncate df/d2c/d1e/f2d 5131083 0 2024-05-06T12:15:01.877 INFO:tasks.workunit.client.1.smithi190.stdout:4/985: stat d2/d10/f99 0 2024-05-06T12:15:01.879 INFO:tasks.workunit.client.1.smithi190.stdout:1/977: truncate d5/d4f/d65/d32/f37 5567602 0 2024-05-06T12:15:01.883 INFO:tasks.workunit.client.1.smithi190.stdout:6/836: mknod df/d2c/d61/d87/df9/c11a 0 2024-05-06T12:15:01.884 INFO:tasks.workunit.client.1.smithi190.stdout:1/978: symlink d5/d4f/d65/d24/d43/l14f 0 2024-05-06T12:15:01.885 INFO:tasks.workunit.client.1.smithi190.stdout:4/986: getdents d2/d11/d16/d8e/d7b/de8/d9e/d6e/daa/ddc/d132 0 2024-05-06T12:15:01.885 INFO:tasks.workunit.client.1.smithi190.stdout:4/987: chown d2/lf3 0 1 2024-05-06T12:15:01.889 INFO:tasks.workunit.client.1.smithi190.stdout:4/988: creat d2/d11/d16/d8e/d7b/de8/d9e/f152 x:0 0 0 2024-05-06T12:15:01.891 INFO:tasks.workunit.client.1.smithi190.stdout:1/979: getdents d5/d15/d1e/dff 0 2024-05-06T12:15:01.895 INFO:tasks.workunit.client.1.smithi190.stdout:1/980: creat d5/d4f/d65/d24/d13b/f150 x:0 0 0 2024-05-06T12:15:01.896 INFO:tasks.workunit.client.1.smithi190.stdout:4/989: getdents d2/d11/d16/d41 0 2024-05-06T12:15:01.899 INFO:tasks.workunit.client.1.smithi190.stdout:1/981: mkdir d5/d15/d1e/dff/d151 0 2024-05-06T12:15:01.901 INFO:tasks.workunit.client.1.smithi190.stdout:6/837: write df/d2c/d28/de0/f48 [4302609,16347] 0 2024-05-06T12:15:01.902 INFO:tasks.workunit.client.1.smithi190.stdout:6/838: chown df/d2c/d28 148802222 1 2024-05-06T12:15:01.902 INFO:tasks.workunit.client.1.smithi190.stdout:1/982: mknod d5/d4f/d65/d32/d47/d53/d98/da7/de7/d126/c152 0 2024-05-06T12:15:01.907 INFO:tasks.workunit.client.1.smithi190.stdout:4/990: dwrite d2/d11/d16/d8e/d80/f9a [0,4194304] 0 2024-05-06T12:15:01.908 INFO:tasks.workunit.client.1.smithi190.stdout:1/983: mkdir d5/d4f/d65/d24/d43/de2/d114/d153 0 2024-05-06T12:15:01.908 INFO:tasks.workunit.client.1.smithi190.stdout:4/991: fdatasync d2/d11/d16/d8e/d80/da0/db2/fd5 0 2024-05-06T12:15:01.909 INFO:tasks.workunit.client.1.smithi190.stdout:1/984: chown d5/d4f/d65/d2b/fef 0 1 2024-05-06T12:15:01.911 INFO:tasks.workunit.client.1.smithi190.stdout:4/992: mknod d2/d11/d16/c153 0 2024-05-06T12:15:01.912 INFO:tasks.workunit.client.1.smithi190.stdout:6/839: write df/d17/f38 [1262169,95385] 0 2024-05-06T12:15:01.916 INFO:tasks.workunit.client.1.smithi190.stdout:1/985: rename d5/d15/d1e/dff/d151 to d5/d4f/d65/d32/d47/d53/d98/da7/de7/d126/d154 0 2024-05-06T12:15:01.919 INFO:tasks.workunit.client.1.smithi190.stdout:1/986: truncate d5/d4f/d65/d32/d47/f77 4627003 0 2024-05-06T12:15:01.920 INFO:tasks.workunit.client.1.smithi190.stdout:6/840: dread df/d2c/d61/d73/dad/d112/f67 [0,4194304] 0 2024-05-06T12:15:01.920 INFO:tasks.workunit.client.1.smithi190.stdout:4/993: creat d2/d11/d16/d12e/f154 x:0 0 0 2024-05-06T12:15:01.927 INFO:tasks.workunit.client.1.smithi190.stdout:1/987: dread d5/d4f/d65/d24/d43/fe3 [0,4194304] 0 2024-05-06T12:15:01.928 INFO:tasks.workunit.client.1.smithi190.stdout:6/841: dwrite df/d2c/d1e/f52 [0,4194304] 0 2024-05-06T12:15:01.930 INFO:tasks.workunit.client.1.smithi190.stdout:6/842: readlink df/d2c/d61/d73/dfd/l10d 0 2024-05-06T12:15:01.930 INFO:tasks.workunit.client.1.smithi190.stdout:1/988: readlink d5/d4f/d65/d62/d9d/dba/lf2 0 2024-05-06T12:15:01.931 INFO:tasks.workunit.client.1.smithi190.stdout:6/843: stat df/d2c/f30 0 2024-05-06T12:15:01.937 INFO:tasks.workunit.client.1.smithi190.stdout:1/989: creat d5/f155 x:0 0 0 2024-05-06T12:15:01.937 INFO:tasks.workunit.client.1.smithi190.stdout:1/990: fsync d5/d15/f125 0 2024-05-06T12:15:01.938 INFO:tasks.workunit.client.1.smithi190.stdout:6/844: truncate df/d12/f6d 621496 0 2024-05-06T12:15:01.939 INFO:tasks.workunit.client.1.smithi190.stdout:1/991: chown d5/d4f/d65/d24/f7e 8385590 1 2024-05-06T12:15:01.943 INFO:tasks.workunit.client.1.smithi190.stdout:1/992: creat d5/d15/d105/d14e/f156 x:0 0 0 2024-05-06T12:15:01.944 INFO:tasks.workunit.client.1.smithi190.stdout:6/845: getdents df/d2c/d61/d73/dad/d112/de8 0 2024-05-06T12:15:01.944 INFO:tasks.workunit.client.1.smithi190.stdout:6/846: stat df/d2c/d61/f79 0 2024-05-06T12:15:01.945 INFO:tasks.workunit.client.1.smithi190.stdout:6/847: stat df/d2c/d28/de0/f48 0 2024-05-06T12:15:01.945 INFO:tasks.workunit.client.1.smithi190.stdout:1/993: fdatasync d5/d15/fbf 0 2024-05-06T12:15:01.946 INFO:tasks.workunit.client.1.smithi190.stdout:6/848: chown df/d2c/d1e/d57/l106 27441 1 2024-05-06T12:15:01.948 INFO:tasks.workunit.client.1.smithi190.stdout:6/849: mkdir df/d2c/d28/deb/d11b 0 2024-05-06T12:15:01.951 INFO:tasks.workunit.client.1.smithi190.stdout:6/850: mknod df/d2c/d61/d73/dad/c11c 0 2024-05-06T12:15:01.956 INFO:tasks.workunit.client.1.smithi190.stdout:6/851: truncate df/d17/d37/f49 282889 0 2024-05-06T12:15:01.960 INFO:tasks.workunit.client.1.smithi190.stdout:1/994: write d5/d4f/f50 [1186211,103902] 0 2024-05-06T12:15:01.962 INFO:tasks.workunit.client.1.smithi190.stdout:1/995: fdatasync d5/d4f/d68/f139 0 2024-05-06T12:15:01.963 INFO:tasks.workunit.client.1.smithi190.stdout:1/996: stat d5/d4f/d65/d2b/c56 0 2024-05-06T12:15:01.966 INFO:tasks.workunit.client.1.smithi190.stdout:1/997: creat d5/d4f/d65/d24/d43/de2/f157 x:0 0 0 2024-05-06T12:15:01.967 INFO:tasks.workunit.client.1.smithi190.stdout:1/998: dread - d5/d4f/d65/d62/d9d/fdb zero size 2024-05-06T12:15:01.968 INFO:tasks.workunit.client.1.smithi190.stdout:1/999: readlink d5/d15/d1e/l41 0 2024-05-06T12:15:01.971 INFO:tasks.workunit.client.1.smithi190.stdout:6/852: rename df/d17/d37/f49 to df/dee/f11d 0 2024-05-06T12:15:01.975 INFO:tasks.workunit.client.1.smithi190.stdout:6/853: symlink df/l11e 0 2024-05-06T12:15:01.976 INFO:tasks.workunit.client.1.smithi190.stdout:6/854: fdatasync df/d2c/d92/f9a 0 2024-05-06T12:15:01.976 INFO:tasks.workunit.client.1.smithi190.stdout:6/855: chown df/l68 29 1 2024-05-06T12:15:01.977 INFO:tasks.workunit.client.1.smithi190.stdout:6/856: chown df/d2c/d61/d87/fe3 614 1 2024-05-06T12:15:01.982 INFO:tasks.workunit.client.1.smithi190.stdout:6/857: rename df/d2c/f30 to df/d2c/d28/deb/f11f 0 2024-05-06T12:15:01.986 INFO:tasks.workunit.client.1.smithi190.stdout:6/858: rename df/d2c/d92/c103 to df/d17/d37/c120 0 2024-05-06T12:15:01.989 INFO:tasks.workunit.client.1.smithi190.stdout:6/859: creat df/d2c/d61/d84/d8f/db6/f121 x:0 0 0 2024-05-06T12:15:01.992 INFO:tasks.workunit.client.1.smithi190.stdout:6/860: unlink df/d2c/d1e/d57/f6b 0 2024-05-06T12:15:01.994 INFO:tasks.workunit.client.1.smithi190.stdout:6/861: unlink df/d2c/d28/dd1/l114 0 2024-05-06T12:15:01.995 INFO:tasks.workunit.client.1.smithi190.stdout:6/862: write fd [4642564,3143] 0 2024-05-06T12:15:01.999 INFO:tasks.workunit.client.1.smithi190.stdout:6/863: truncate df/d2c/d61/d73/dad/d112/f67 782704 0 2024-05-06T12:15:02.003 INFO:tasks.workunit.client.1.smithi190.stdout:6/864: mkdir df/d2c/d61/d84/d8f/db6/d122 0 2024-05-06T12:15:02.006 INFO:tasks.workunit.client.1.smithi190.stdout:6/865: creat df/d2c/d61/d84/d8f/d10c/f123 x:0 0 0 2024-05-06T12:15:02.012 INFO:tasks.workunit.client.1.smithi190.stdout:6/866: link df/d2c/d61/f78 df/d2c/d28/de0/f124 0 2024-05-06T12:15:02.014 INFO:tasks.workunit.client.1.smithi190.stdout:6/867: symlink df/d2c/d61/d73/dad/d112/l125 0 2024-05-06T12:15:02.018 INFO:tasks.workunit.client.1.smithi190.stdout:6/868: creat df/d2c/d28/f126 x:0 0 0 2024-05-06T12:15:02.045 INFO:tasks.workunit.client.1.smithi190.stdout:6/869: dwrite df/f82 [0,4194304] 0 2024-05-06T12:15:02.051 INFO:tasks.workunit.client.1.smithi190.stdout:6/870: fsync df/d2c/d28/de0/fc3 0 2024-05-06T12:15:02.055 INFO:tasks.workunit.client.1.smithi190.stdout:6/871: rename df/d17/fda to df/d2c/d61/d84/d8f/d10c/f127 0 2024-05-06T12:15:02.065 INFO:tasks.workunit.client.1.smithi190.stdout:6/872: dwrite df/d12/f6d [0,4194304] 0 2024-05-06T12:15:02.066 INFO:tasks.workunit.client.1.smithi190.stdout:6/873: read df/d12/f6d [2406600,114038] 0 2024-05-06T12:15:02.067 INFO:tasks.workunit.client.1.smithi190.stdout:6/874: stat df/d2c/d28/deb/df1/f118 0 2024-05-06T12:15:02.069 INFO:tasks.workunit.client.1.smithi190.stdout:6/875: chown df/dee/fef 30474811 1 2024-05-06T12:15:02.069 INFO:tasks.workunit.client.1.smithi190.stdout:6/876: fdatasync df/d17/f26 0 2024-05-06T12:15:02.072 INFO:tasks.workunit.client.1.smithi190.stdout:6/877: creat df/d12/f128 x:0 0 0 2024-05-06T12:15:02.074 INFO:tasks.workunit.client.1.smithi190.stdout:6/878: creat df/f129 x:0 0 0 2024-05-06T12:15:02.075 INFO:tasks.workunit.client.1.smithi190.stdout:6/879: chown df/d2c/d61/d84/d8f/db6 1714355 1 2024-05-06T12:15:02.089 INFO:tasks.workunit.client.1.smithi190.stdout:6/880: dread df/d17/f5c [0,4194304] 0 2024-05-06T12:15:02.091 INFO:tasks.workunit.client.1.smithi190.stdout:6/881: chown df/d2c/d61/d87/fde 35897055 1 2024-05-06T12:15:02.092 INFO:tasks.workunit.client.1.smithi190.stdout:6/882: fsync df/d2c/d28/de0/f48 0 2024-05-06T12:15:02.093 INFO:tasks.workunit.client.1.smithi190.stdout:6/883: mkdir df/dee/d12a 0 2024-05-06T12:15:02.094 INFO:tasks.workunit.client.1.smithi190.stdout:6/884: read df/d17/f26 [233671,18459] 0 2024-05-06T12:15:02.100 INFO:tasks.workunit.client.1.smithi190.stdout:6/885: unlink df/d2c/d61/d84/d8f/fed 0 2024-05-06T12:15:02.102 INFO:tasks.workunit.client.1.smithi190.stdout:6/886: symlink df/d2c/d61/d87/df9/l12b 0 2024-05-06T12:15:02.106 INFO:tasks.workunit.client.1.smithi190.stdout:6/887: unlink df/d2c/d28/de0/l5f 0 2024-05-06T12:15:02.106 INFO:tasks.workunit.client.1.smithi190.stdout:6/888: fdatasync df/d2c/d61/d87/f108 0 2024-05-06T12:15:02.110 INFO:tasks.workunit.client.1.smithi190.stdout:6/889: dread df/d17/d37/f42 [0,4194304] 0 2024-05-06T12:15:02.118 INFO:tasks.workunit.client.1.smithi190.stdout:6/890: write df/d17/d37/f8c [4272492,31368] 0 2024-05-06T12:15:02.121 INFO:tasks.workunit.client.1.smithi190.stdout:6/891: creat df/d2c/d61/d84/f12c x:0 0 0 2024-05-06T12:15:02.124 INFO:tasks.workunit.client.1.smithi190.stdout:6/892: rename df/d12/dbd/cfe to df/d2c/d28/dd1/c12d 0 2024-05-06T12:15:02.126 INFO:tasks.workunit.client.1.smithi190.stdout:6/893: creat df/d2c/d61/d84/d8f/d10c/d10e/f12e x:0 0 0 2024-05-06T12:15:02.141 INFO:tasks.workunit.client.1.smithi190.stdout:6/894: dread df/d17/d37/f8c [0,4194304] 0 2024-05-06T12:15:02.148 INFO:tasks.workunit.client.1.smithi190.stdout:6/895: truncate df/d2c/d1e/d57/f8a 4719402 0 2024-05-06T12:15:02.149 INFO:tasks.workunit.client.1.smithi190.stdout:6/896: chown df/d2c/d28/de0/f9d 1114985 1 2024-05-06T12:15:02.151 INFO:tasks.workunit.client.1.smithi190.stdout:6/897: mknod df/d2c/d28/deb/d11b/c12f 0 2024-05-06T12:15:02.153 INFO:tasks.workunit.client.1.smithi190.stdout:6/898: rename df/d2c/d1e/c11 to df/d2c/d61/d84/d8f/d10c/d10e/d119/c130 0 2024-05-06T12:15:02.156 INFO:tasks.workunit.client.1.smithi190.stdout:6/899: unlink df/d12/f128 0 2024-05-06T12:15:02.157 INFO:tasks.workunit.client.1.smithi190.stdout:6/900: dread - df/d2c/d61/d73/dad/d112/fd4 zero size 2024-05-06T12:15:02.162 INFO:tasks.workunit.client.1.smithi190.stdout:6/901: dwrite df/d2c/d61/d87/fe7 [0,4194304] 0 2024-05-06T12:15:02.166 INFO:tasks.workunit.client.1.smithi190.stdout:6/902: creat df/d2c/d61/d84/d8f/d10c/d10e/f131 x:0 0 0 2024-05-06T12:15:02.168 INFO:tasks.workunit.client.1.smithi190.stdout:6/903: mkdir df/d2c/d61/d84/d8f/db6/d132 0 2024-05-06T12:15:02.172 INFO:tasks.workunit.client.1.smithi190.stdout:6/904: getdents df/d2c/d28/deb/d11b 0 2024-05-06T12:15:02.174 INFO:tasks.workunit.client.1.smithi190.stdout:6/905: mkdir df/d133 0 2024-05-06T12:15:02.177 INFO:tasks.workunit.client.1.smithi190.stdout:6/906: creat df/d2c/d61/d84/d8f/d10c/d10e/f134 x:0 0 0 2024-05-06T12:15:02.184 INFO:tasks.workunit.client.1.smithi190.stdout:6/907: dwrite df/d2c/d61/d84/d8f/d10c/d10e/f12e [0,4194304] 0 2024-05-06T12:15:02.185 INFO:tasks.workunit.client.1.smithi190.stdout:6/908: read df/d2c/d61/d87/fe3 [252797,77176] 0 2024-05-06T12:15:02.289 INFO:tasks.workunit.client.1.smithi190.stdout:4/994: sync 2024-05-06T12:15:02.291 INFO:tasks.workunit.client.1.smithi190.stdout:4/995: rmdir d2/d11/d16/d8e/d7b/de8/dd1/def 39 2024-05-06T12:15:02.294 INFO:tasks.workunit.client.1.smithi190.stdout:4/996: dread d2/d11/d16/d31/ff0 [0,4194304] 0 2024-05-06T12:15:02.296 INFO:tasks.workunit.client.1.smithi190.stdout:4/997: stat d2/d11/d16/d8e/d7b/de8/d9e/d6e/f92 0 2024-05-06T12:15:02.297 INFO:tasks.workunit.client.1.smithi190.stdout:4/998: write d2/d11/d16/d8e/d80/f97 [3116627,4887] 0 2024-05-06T12:15:02.298 INFO:tasks.workunit.client.1.smithi190.stdout:4/999: write d2/d11/ff6 [568498,15481] 0 2024-05-06T12:15:02.400 INFO:tasks.workunit.client.1.smithi190.stdout:6/909: sync 2024-05-06T12:15:02.400 INFO:tasks.workunit.client.1.smithi190.stdout:6/910: fsync df/d2c/d92/f9a 0 2024-05-06T12:15:02.401 INFO:tasks.workunit.client.1.smithi190.stdout:6/911: chown df/d2c/d1e/cfb 156 1 2024-05-06T12:15:02.402 INFO:tasks.workunit.client.1.smithi190.stdout:6/912: chown df/d2c/d1e/d57/d89/la0 388286 1 2024-05-06T12:15:02.402 INFO:tasks.workunit.client.1.smithi190.stdout:6/913: stat df/d2c/d1e/c1a 0 2024-05-06T12:15:02.405 INFO:tasks.workunit.client.1.smithi190.stdout:6/914: getdents df/d2c/d1e/d57/d89/df8 0 2024-05-06T12:15:02.409 INFO:tasks.workunit.client.1.smithi190.stdout:6/915: write df/d12/f2b [176666,57498] 0 2024-05-06T12:15:02.410 INFO:tasks.workunit.client.1.smithi190.stdout:6/916: stat df/d2c/d61/d87/dc4 0 2024-05-06T12:15:02.411 INFO:tasks.workunit.client.1.smithi190.stdout:6/917: read - df/d2c/d28/de0/f9d zero size 2024-05-06T12:15:02.413 INFO:tasks.workunit.client.1.smithi190.stdout:6/918: symlink df/d2c/d28/dd1/l135 0 2024-05-06T12:15:02.416 INFO:tasks.workunit.client.1.smithi190.stdout:6/919: rename df/d2c/d28/deb to df/d2c/d61/d73/d136 0 2024-05-06T12:15:02.419 INFO:tasks.workunit.client.1.smithi190.stdout:6/920: getdents df/d2c/d61/d87/dc4 0 2024-05-06T12:15:02.422 INFO:tasks.workunit.client.1.smithi190.stdout:6/921: getdents df/d2c/d61/d84/d8f/d10c/d10e 0 2024-05-06T12:15:02.423 INFO:tasks.workunit.client.1.smithi190.stdout:6/922: read df/fc1 [556036,41361] 0 2024-05-06T12:15:02.426 INFO:tasks.workunit.client.1.smithi190.stdout:6/923: chown df/d2c/d1e/d57/lf7 164 1 2024-05-06T12:15:02.428 INFO:tasks.workunit.client.1.smithi190.stdout:6/924: truncate df/d2c/d61/d84/d8f/d10c/d10e/d119/fcf 753360 0 2024-05-06T12:15:02.429 INFO:tasks.workunit.client.1.smithi190.stdout:6/925: creat df/d2c/d61/d87/f137 x:0 0 0 2024-05-06T12:15:02.433 INFO:tasks.workunit.client.1.smithi190.stdout:6/926: dread df/d2c/d61/d84/fb4 [0,4194304] 0 2024-05-06T12:15:02.441 INFO:tasks.workunit.client.1.smithi190.stdout:6/927: dwrite df/d2c/d61/d87/fba [0,4194304] 0 2024-05-06T12:15:02.444 INFO:tasks.workunit.client.1.smithi190.stdout:6/928: unlink df/d2c/d61/d73/dad/d112/fd4 0 2024-05-06T12:15:02.448 INFO:tasks.workunit.client.1.smithi190.stdout:6/929: dread df/f99 [0,4194304] 0 2024-05-06T12:15:02.449 INFO:tasks.workunit.client.1.smithi190.stdout:6/930: chown df/d2c/d61/f79 4333 1 2024-05-06T12:15:02.452 INFO:tasks.workunit.client.1.smithi190.stdout:6/931: rename df/d2c/f36 to df/dee/d12a/f138 0 2024-05-06T12:15:02.454 INFO:tasks.workunit.client.1.smithi190.stdout:6/932: creat df/d2c/d61/d73/dad/d112/de8/f139 x:0 0 0 2024-05-06T12:15:02.457 INFO:tasks.workunit.client.1.smithi190.stdout:6/933: fsync df/d2c/d92/f93 0 2024-05-06T12:15:02.460 INFO:tasks.workunit.client.1.smithi190.stdout:6/934: dread df/d2c/d61/d84/d8f/d10c/d10e/d119/fcf [0,4194304] 0 2024-05-06T12:15:02.465 INFO:tasks.workunit.client.1.smithi190.stdout:6/935: dwrite f7 [4194304,4194304] 0 2024-05-06T12:15:02.471 INFO:tasks.workunit.client.1.smithi190.stdout:6/936: rename df/c74 to df/d2c/d92/c13a 0 2024-05-06T12:15:02.474 INFO:tasks.workunit.client.1.smithi190.stdout:6/937: mknod df/d2c/c13b 0 2024-05-06T12:15:02.476 INFO:tasks.workunit.client.1.smithi190.stdout:6/938: fdatasync df/d2c/d28/de0/faf 0 2024-05-06T12:15:02.478 INFO:tasks.workunit.client.1.smithi190.stdout:6/939: mkdir df/d2c/d1e/d57/d13c 0 2024-05-06T12:15:02.480 INFO:tasks.workunit.client.1.smithi190.stdout:6/940: symlink df/d2c/d61/d73/dad/d112/l13d 0 2024-05-06T12:15:02.482 INFO:tasks.workunit.client.1.smithi190.stdout:6/941: creat df/d2c/d61/f13e x:0 0 0 2024-05-06T12:15:02.484 INFO:tasks.workunit.client.1.smithi190.stdout:6/942: creat df/d2c/d1e/d57/d89/f13f x:0 0 0 2024-05-06T12:15:02.485 INFO:tasks.workunit.client.1.smithi190.stdout:6/943: chown df/d2c/d61/d84/d8f/d10c/d10e/d119/fcf 29410 1 2024-05-06T12:15:02.486 INFO:tasks.workunit.client.1.smithi190.stdout:6/944: chown df/d2c/d28/de0/f48 47562152 1 2024-05-06T12:15:02.488 INFO:tasks.workunit.client.1.smithi190.stdout:6/945: mknod df/dee/c140 0 2024-05-06T12:15:02.489 INFO:tasks.workunit.client.1.smithi190.stdout:6/946: write df/d2c/d61/d87/fe7 [2882837,51853] 0 2024-05-06T12:15:02.491 INFO:tasks.workunit.client.1.smithi190.stdout:6/947: symlink df/d2c/d61/d73/dfd/l141 0 2024-05-06T12:15:02.499 INFO:tasks.workunit.client.1.smithi190.stdout:6/948: dwrite df/d2c/d61/d84/d8f/db6/fc5 [4194304,4194304] 0 2024-05-06T12:15:02.504 INFO:tasks.workunit.client.1.smithi190.stdout:6/949: rename df/lcc to df/d2c/l142 0 2024-05-06T12:15:02.507 INFO:tasks.workunit.client.1.smithi190.stdout:6/950: creat df/d2c/d28/de0/f143 x:0 0 0 2024-05-06T12:15:02.509 INFO:tasks.workunit.client.1.smithi190.stdout:6/951: mknod df/c144 0 2024-05-06T12:15:02.510 INFO:tasks.workunit.client.1.smithi190.stdout:6/952: creat df/d2c/d61/d73/dad/d112/f145 x:0 0 0 2024-05-06T12:15:02.513 INFO:tasks.workunit.client.1.smithi190.stdout:6/953: link df/d2c/d28/c105 df/dee/d12a/c146 0 2024-05-06T12:15:02.519 INFO:tasks.workunit.client.1.smithi190.stdout:6/954: dread df/d2c/f25 [0,4194304] 0 2024-05-06T12:15:02.523 INFO:tasks.workunit.client.1.smithi190.stdout:6/955: rename df/d2c/d61/d84/d8f/db6/f121 to df/d2c/f147 0 2024-05-06T12:15:02.524 INFO:tasks.workunit.client.1.smithi190.stdout:6/956: chown df/d2c/d28/de0/fa7 31228777 1 2024-05-06T12:15:02.527 INFO:tasks.workunit.client.1.smithi190.stdout:6/957: link df/d17/l86 df/d2c/d1e/d57/l148 0 2024-05-06T12:15:02.529 INFO:tasks.workunit.client.1.smithi190.stdout:6/958: creat df/d2c/d1e/d57/f149 x:0 0 0 2024-05-06T12:15:02.533 INFO:tasks.workunit.client.1.smithi190.stdout:6/959: rename df/d2c/d61/d73/dad/lc6 to df/d2c/d61/d84/d8f/db6/d122/l14a 0 2024-05-06T12:15:02.543 INFO:tasks.workunit.client.1.smithi190.stdout:6/960: dread df/f15 [0,4194304] 0 2024-05-06T12:15:02.550 INFO:tasks.workunit.client.1.smithi190.stdout:6/961: write df/d2c/d1e/f2d [2838603,98559] 0 2024-05-06T12:15:02.552 INFO:tasks.workunit.client.1.smithi190.stdout:6/962: symlink df/l14b 0 2024-05-06T12:15:02.553 INFO:tasks.workunit.client.1.smithi190.stdout:6/963: write df/d2c/d1e/d57/f149 [457668,63815] 0 2024-05-06T12:15:02.559 INFO:tasks.workunit.client.1.smithi190.stdout:6/964: dwrite df/d2c/d92/f9a [0,4194304] 0 2024-05-06T12:15:02.568 INFO:tasks.workunit.client.1.smithi190.stdout:6/965: dwrite df/d2c/d28/de0/fa7 [0,4194304] 0 2024-05-06T12:15:02.569 INFO:tasks.workunit.client.1.smithi190.stdout:6/966: read - df/d2c/d28/fe9 zero size 2024-05-06T12:15:02.570 INFO:tasks.workunit.client.1.smithi190.stdout:6/967: truncate df/f15 4940725 0 2024-05-06T12:15:02.572 INFO:tasks.workunit.client.1.smithi190.stdout:6/968: chown df/d2c/d61/d73/d136/df1/f118 0 1 2024-05-06T12:15:02.574 INFO:tasks.workunit.client.1.smithi190.stdout:6/969: fdatasync df/d2c/d61/fab 0 2024-05-06T12:15:02.577 INFO:tasks.workunit.client.1.smithi190.stdout:6/970: rename df/d2c/d61/d84/d8f/d10c/f123 to df/d133/f14c 0 2024-05-06T12:15:02.582 INFO:tasks.workunit.client.1.smithi190.stdout:6/971: dwrite df/d2c/d28/f126 [0,4194304] 0 2024-05-06T12:15:02.583 INFO:tasks.workunit.client.1.smithi190.stdout:6/972: readlink df/d2c/d61/d73/dfd/l10d 0 2024-05-06T12:15:02.587 INFO:tasks.workunit.client.1.smithi190.stdout:6/973: unlink df/d2c/d61/d73/c81 0 2024-05-06T12:15:02.590 INFO:tasks.workunit.client.1.smithi190.stdout:6/974: dread df/d2c/d61/f78 [0,4194304] 0 2024-05-06T12:15:02.596 INFO:tasks.workunit.client.1.smithi190.stdout:6/975: write df/d12/f72 [46683,1004] 0 2024-05-06T12:15:02.809 INFO:tasks.workunit.client.1.smithi190.stdout:6/976: sync 2024-05-06T12:15:02.810 INFO:tasks.workunit.client.1.smithi190.stdout:6/977: write df/d2c/fa9 [1493089,1923] 0 2024-05-06T12:15:02.813 INFO:tasks.workunit.client.1.smithi190.stdout:6/978: mknod df/d2c/dae/dbb/c14d 0 2024-05-06T12:15:02.819 INFO:tasks.workunit.client.1.smithi190.stdout:6/979: dwrite df/d2c/d61/d84/d8f/d10c/d10e/f131 [0,4194304] 0 2024-05-06T12:15:02.822 INFO:tasks.workunit.client.1.smithi190.stdout:6/980: truncate df/d2c/d61/d87/f108 716595 0 2024-05-06T12:15:02.825 INFO:tasks.workunit.client.1.smithi190.stdout:6/981: read df/d2c/d61/d73/d136/f11f [464835,130503] 0 2024-05-06T12:15:02.829 INFO:tasks.workunit.client.1.smithi190.stdout:6/982: getdents df/d2c/d61/d84 0 2024-05-06T12:15:02.830 INFO:tasks.workunit.client.1.smithi190.stdout:6/983: mkdir df/dee/d12a/d14e 0 2024-05-06T12:15:02.831 INFO:tasks.workunit.client.1.smithi190.stdout:6/984: stat df/d2c/d61/d73/d136/f11f 0 2024-05-06T12:15:02.842 INFO:tasks.workunit.client.1.smithi190.stdout:6/985: dread df/d17/d37/f58 [0,4194304] 0 2024-05-06T12:15:02.844 INFO:tasks.workunit.client.1.smithi190.stdout:6/986: read df/d2c/d61/d87/fde [255985,27364] 0 2024-05-06T12:15:02.846 INFO:tasks.workunit.client.1.smithi190.stdout:6/987: rename df/d2c/d61/d73/dad/fc7 to df/d2c/d92/dce/f14f 0 2024-05-06T12:15:02.960 INFO:tasks.workunit.client.1.smithi190.stdout:6/988: sync 2024-05-06T12:15:02.973 INFO:tasks.workunit.client.1.smithi190.stdout:6/989: dwrite df/f18 [0,4194304] 0 2024-05-06T12:15:02.974 INFO:tasks.workunit.client.1.smithi190.stdout:6/990: chown df/d17/d37/d102 97 1 2024-05-06T12:15:02.978 INFO:tasks.workunit.client.1.smithi190.stdout:6/991: dread df/d2c/f25 [0,4194304] 0 2024-05-06T12:15:02.981 INFO:tasks.workunit.client.1.smithi190.stdout:6/992: mkdir df/d2c/d61/d73/dad/d112/d150 0 2024-05-06T12:15:02.983 INFO:tasks.workunit.client.1.smithi190.stdout:6/993: fdatasync df/d2c/d28/f40 0 2024-05-06T12:15:02.986 INFO:tasks.workunit.client.1.smithi190.stdout:6/994: creat df/d2c/d61/d84/d8f/d10c/d10e/d119/f151 x:0 0 0 2024-05-06T12:15:02.989 INFO:tasks.workunit.client.1.smithi190.stdout:6/995: creat df/d2c/d61/d84/d8f/db6/d132/f152 x:0 0 0 2024-05-06T12:15:02.992 INFO:tasks.workunit.client.1.smithi190.stdout:6/996: symlink df/d2c/d61/d84/d8f/d10c/d10e/l153 0 2024-05-06T12:15:02.993 INFO:tasks.workunit.client.1.smithi190.stdout:6/997: chown df/d2c/d61/d84/d8f/db6 46 1 2024-05-06T12:15:02.995 INFO:tasks.workunit.client.1.smithi190.stdout:6/998: mknod df/c154 0 2024-05-06T12:15:02.996 INFO:tasks.workunit.client.1.smithi190.stdout:6/999: mknod df/d2c/d61/d87/dc4/c155 0 2024-05-06T12:15:02.999 INFO:tasks.workunit.client.1.smithi190.stderr:+ rm -rf -- ./tmp.n2MO3EjBDS 2024-05-06T12:15:03.056 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:02 smithi092 ceph-mon[26954]: pgmap v33: 65 pgs: 65 active+clean; 3.3 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 71 MiB/s rd, 162 MiB/s wr, 410 op/s 2024-05-06T12:15:03.056 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:02 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:15:03.056 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:02 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:03.056 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:02 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:03.056 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:02 smithi092 ceph-mon[26954]: Reconfiguring prometheus.smithi092 (dependencies changed)... 2024-05-06T12:15:03.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:02 smithi190 ceph-mon[33798]: pgmap v33: 65 pgs: 65 active+clean; 3.3 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 71 MiB/s rd, 162 MiB/s wr, 410 op/s 2024-05-06T12:15:03.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:02 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:15:03.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:02 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:03.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:02 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:03.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:02 smithi190 ceph-mon[33798]: Reconfiguring prometheus.smithi092 (dependencies changed)... 2024-05-06T12:15:04.081 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:03 smithi092 ceph-mon[26954]: Reconfiguring daemon prometheus.smithi092 on smithi092 2024-05-06T12:15:04.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:03 smithi190 ceph-mon[33798]: Reconfiguring daemon prometheus.smithi092 on smithi092 2024-05-06T12:15:04.200 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-06T12:15:04.200 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-06T12:15:04.998 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:04 smithi092 ceph-mon[26954]: pgmap v34: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 96 MiB/s rd, 219 MiB/s wr, 597 op/s 2024-05-06T12:15:05.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:04 smithi190 ceph-mon[33798]: pgmap v34: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 96 MiB/s rd, 219 MiB/s wr, 597 op/s 2024-05-06T12:15:06.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:06 smithi092 ceph-mon[26954]: pgmap v35: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 60 MiB/s rd, 129 MiB/s wr, 392 op/s 2024-05-06T12:15:07.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:06 smithi190 ceph-mon[33798]: pgmap v35: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 60 MiB/s rd, 129 MiB/s wr, 392 op/s 2024-05-06T12:15:08.838 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:08 smithi190 ceph-mon[33798]: fsmap cephfs:2 {0=cephfs.smithi092.rosbll=up:active,1=cephfs.smithi190.byjvgf=up:active} 2 up:standby-replay 2024-05-06T12:15:08.838 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:08 smithi190 ceph-mon[33798]: pgmap v36: 65 pgs: 65 active+clean; 2.5 GiB data, 9.2 GiB used, 527 GiB / 536 GiB avail; 61 MiB/s rd, 129 MiB/s wr, 413 op/s 2024-05-06T12:15:08.839 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:08 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:08.839 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:08 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:08.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:08 smithi092 ceph-mon[26954]: fsmap cephfs:2 {0=cephfs.smithi092.rosbll=up:active,1=cephfs.smithi190.byjvgf=up:active} 2 up:standby-replay 2024-05-06T12:15:08.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:08 smithi092 ceph-mon[26954]: pgmap v36: 65 pgs: 65 active+clean; 2.5 GiB data, 9.2 GiB used, 527 GiB / 536 GiB avail; 61 MiB/s rd, 129 MiB/s wr, 413 op/s 2024-05-06T12:15:08.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:08 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:08.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:08 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:09.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:09 smithi190 ceph-mon[33798]: Reconfiguring node-exporter.smithi190 (dependencies changed)... 2024-05-06T12:15:09.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:09 smithi190 ceph-mon[33798]: Reconfiguring daemon node-exporter.smithi190 on smithi190 2024-05-06T12:15:09.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:09 smithi092 ceph-mon[26954]: Reconfiguring node-exporter.smithi190 (dependencies changed)... 2024-05-06T12:15:09.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:09 smithi092 ceph-mon[26954]: Reconfiguring daemon node-exporter.smithi190 on smithi190 2024-05-06T12:15:10.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:10 smithi190 ceph-mon[33798]: pgmap v37: 65 pgs: 65 active+clean; 1.7 GiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 49 MiB/s rd, 101 MiB/s wr, 425 op/s 2024-05-06T12:15:10.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:10 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:10.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:10 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:10.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:10 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-06T12:15:10.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:10 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi092.front.sepia.ceph.com:9093"}]: dispatch 2024-05-06T12:15:10.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:10 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:10.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:10 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-06T12:15:10.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:10 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi092.front.sepia.ceph.com:3000"}]: dispatch 2024-05-06T12:15:10.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:10 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:10.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:10 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-06T12:15:10.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:10 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi092.front.sepia.ceph.com:9095"}]: dispatch 2024-05-06T12:15:10.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:10 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:10.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:10 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:15:10.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:10 smithi092 ceph-mon[26954]: pgmap v37: 65 pgs: 65 active+clean; 1.7 GiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 49 MiB/s rd, 101 MiB/s wr, 425 op/s 2024-05-06T12:15:10.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:10 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:10.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:10 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:10.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:10 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-06T12:15:10.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:10 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi092.front.sepia.ceph.com:9093"}]: dispatch 2024-05-06T12:15:10.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:10 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:10.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:10 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-06T12:15:10.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:10 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi092.front.sepia.ceph.com:3000"}]: dispatch 2024-05-06T12:15:10.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:10 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:10.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:10 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-06T12:15:10.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:10 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi092.front.sepia.ceph.com:9095"}]: dispatch 2024-05-06T12:15:10.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:10 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:10.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:10 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:15:11.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:11 smithi190 ceph-mon[33798]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-06T12:15:11.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:11 smithi190 ceph-mon[33798]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi092.front.sepia.ceph.com:9093"}]: dispatch 2024-05-06T12:15:11.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:11 smithi190 ceph-mon[33798]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-06T12:15:11.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:11 smithi190 ceph-mon[33798]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi092.front.sepia.ceph.com:3000"}]: dispatch 2024-05-06T12:15:11.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:11 smithi190 ceph-mon[33798]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-06T12:15:11.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:11 smithi190 ceph-mon[33798]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi092.front.sepia.ceph.com:9095"}]: dispatch 2024-05-06T12:15:11.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:11 smithi190 ceph-mon[33798]: Upgrade: Need to upgrade myself (mgr.smithi190.kczxkh) 2024-05-06T12:15:11.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:11 smithi190 ceph-mon[33798]: Upgrade: Need to upgrade myself (mgr.smithi190.kczxkh) 2024-05-06T12:15:11.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:11 smithi092 ceph-mon[26954]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-06T12:15:11.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:11 smithi092 ceph-mon[26954]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi092.front.sepia.ceph.com:9093"}]: dispatch 2024-05-06T12:15:11.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:11 smithi092 ceph-mon[26954]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-06T12:15:11.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:11 smithi092 ceph-mon[26954]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi092.front.sepia.ceph.com:3000"}]: dispatch 2024-05-06T12:15:11.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:11 smithi092 ceph-mon[26954]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-06T12:15:11.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:11 smithi092 ceph-mon[26954]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi092.front.sepia.ceph.com:9095"}]: dispatch 2024-05-06T12:15:11.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:11 smithi092 ceph-mon[26954]: Upgrade: Need to upgrade myself (mgr.smithi190.kczxkh) 2024-05-06T12:15:11.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:11 smithi092 ceph-mon[26954]: Upgrade: Need to upgrade myself (mgr.smithi190.kczxkh) 2024-05-06T12:15:12.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:12 smithi190 ceph-mon[33798]: pgmap v38: 65 pgs: 65 active+clean; 1.7 GiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 26 MiB/s rd, 58 MiB/s wr, 300 op/s 2024-05-06T12:15:12.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:12 smithi092 ceph-mon[26954]: pgmap v38: 65 pgs: 65 active+clean; 1.7 GiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 26 MiB/s rd, 58 MiB/s wr, 300 op/s 2024-05-06T12:15:14.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:14 smithi092 ceph-mon[26954]: pgmap v39: 65 pgs: 65 active+clean; 1.5 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 27 MiB/s rd, 59 MiB/s wr, 358 op/s 2024-05-06T12:15:15.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:14 smithi190 ceph-mon[33798]: pgmap v39: 65 pgs: 65 active+clean; 1.5 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 27 MiB/s rd, 59 MiB/s wr, 358 op/s 2024-05-06T12:15:16.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:16 smithi092 ceph-mon[26954]: pgmap v40: 65 pgs: 65 active+clean; 1.5 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 171 op/s 2024-05-06T12:15:17.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:16 smithi190 ceph-mon[33798]: pgmap v40: 65 pgs: 65 active+clean; 1.5 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 171 op/s 2024-05-06T12:15:17.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:17 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:15:18.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:17 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:15:18.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:18 smithi092 ceph-mon[26954]: pgmap v41: 65 pgs: 65 active+clean; 1.3 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 184 op/s 2024-05-06T12:15:19.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:18 smithi190 ceph-mon[33798]: pgmap v41: 65 pgs: 65 active+clean; 1.3 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 184 op/s 2024-05-06T12:15:20.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:20 smithi092 ceph-mon[26954]: pgmap v42: 65 pgs: 65 active+clean; 1011 MiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 2.7 MiB/s rd, 3.5 MiB/s wr, 220 op/s 2024-05-06T12:15:21.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:20 smithi190 ceph-mon[33798]: pgmap v42: 65 pgs: 65 active+clean; 1011 MiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 2.7 MiB/s rd, 3.5 MiB/s wr, 220 op/s 2024-05-06T12:15:21.627 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-06T12:15:21.627 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-05-06T12:15:22.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:22 smithi092 ceph-mon[26954]: pgmap v43: 65 pgs: 65 active+clean; 1011 MiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 127 op/s 2024-05-06T12:15:23.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:22 smithi190 ceph-mon[33798]: pgmap v43: 65 pgs: 65 active+clean; 1011 MiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 127 op/s 2024-05-06T12:15:24.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:23 smithi190 ceph-mon[33798]: Upgrade: Updating mgr.smithi092.gyzeqb 2024-05-06T12:15:24.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:23 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:24.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:23 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi092.gyzeqb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-06T12:15:24.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:23 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi092.gyzeqb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-06T12:15:24.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:23 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-06T12:15:24.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:23 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:15:24.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:23 smithi190 ceph-mon[33798]: Deploying daemon mgr.smithi092.gyzeqb on smithi092 2024-05-06T12:15:24.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:23 smithi092 ceph-mon[26954]: Upgrade: Updating mgr.smithi092.gyzeqb 2024-05-06T12:15:24.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:23 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:24.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:23 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi092.gyzeqb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-06T12:15:24.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:23 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi092.gyzeqb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-06T12:15:24.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:23 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-06T12:15:24.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:23 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:15:24.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:23 smithi092 ceph-mon[26954]: Deploying daemon mgr.smithi092.gyzeqb on smithi092 2024-05-06T12:15:24.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:24 smithi092 ceph-mon[26954]: pgmap v44: 65 pgs: 65 active+clean; 440 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 192 op/s 2024-05-06T12:15:25.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:24 smithi190 ceph-mon[33798]: pgmap v44: 65 pgs: 65 active+clean; 440 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 192 op/s 2024-05-06T12:15:27.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:26 smithi190 ceph-mon[33798]: pgmap v45: 65 pgs: 65 active+clean; 440 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-05-06T12:15:27.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:26 smithi092 ceph-mon[26954]: pgmap v45: 65 pgs: 65 active+clean; 440 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-05-06T12:15:28.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.433+0000 7fa54e2f4700 1 -- 172.21.15.92:0/3902776267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5480fbcb0 msgr2=0x7fa5480fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:15:28.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.433+0000 7fa54e2f4700 1 --2- 172.21.15.92:0/3902776267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5480fbcb0 0x7fa5480fc0c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa538004940 tx=0x7fa538009e30 comp rx=0 tx=0).stop 2024-05-06T12:15:28.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.433+0000 7fa54e2f4700 1 -- 172.21.15.92:0/3902776267 shutdown_connections 2024-05-06T12:15:28.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.433+0000 7fa54e2f4700 1 --2- 172.21.15.92:0/3902776267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5480fcf60 0x7fa5480fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:28.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.433+0000 7fa54e2f4700 1 --2- 172.21.15.92:0/3902776267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5480fbcb0 0x7fa5480fc0c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:28.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.433+0000 7fa54e2f4700 1 -- 172.21.15.92:0/3902776267 >> 172.21.15.92:0/3902776267 conn(0x7fa5480f7430 msgr2=0x7fa5480f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:15:28.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.433+0000 7fa54e2f4700 1 -- 172.21.15.92:0/3902776267 shutdown_connections 2024-05-06T12:15:28.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.433+0000 7fa54e2f4700 1 -- 172.21.15.92:0/3902776267 wait complete. 2024-05-06T12:15:28.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.434+0000 7fa54e2f4700 1 Processor -- start 2024-05-06T12:15:28.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.434+0000 7fa54e2f4700 1 -- start start 2024-05-06T12:15:28.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.434+0000 7fa54e2f4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5480fbcb0 0x7fa54810d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:15:28.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.434+0000 7fa54e2f4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5480fcf60 0x7fa54810deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:15:28.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.434+0000 7fa54e2f4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54810e4b0 con 0x7fa5480fbcb0 2024-05-06T12:15:28.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.434+0000 7fa54e2f4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54810ed00 con 0x7fa5480fcf60 2024-05-06T12:15:28.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.434+0000 7fa547fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5480fbcb0 0x7fa54810d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:15:28.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.434+0000 7fa547fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5480fbcb0 0x7fa54810d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44362/0 (socket says 172.21.15.92:44362) 2024-05-06T12:15:28.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.434+0000 7fa547fff700 1 -- 172.21.15.92:0/1415712996 learned_addr learned my addr 172.21.15.92:0/1415712996 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:15:28.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.434+0000 7fa5477fe700 1 --2- 172.21.15.92:0/1415712996 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5480fcf60 0x7fa54810deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:15:28.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.435+0000 7fa547fff700 1 -- 172.21.15.92:0/1415712996 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5480fcf60 msgr2=0x7fa54810deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:15:28.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.435+0000 7fa547fff700 1 --2- 172.21.15.92:0/1415712996 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5480fcf60 0x7fa54810deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:28.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.435+0000 7fa547fff700 1 -- 172.21.15.92:0/1415712996 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa538005040 con 0x7fa5480fbcb0 2024-05-06T12:15:28.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.435+0000 7fa547fff700 1 --2- 172.21.15.92:0/1415712996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5480fbcb0 0x7fa54810d970 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa538013870 tx=0x7fa538009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:15:28.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.435+0000 7fa544ff9700 1 -- 172.21.15.92:0/1415712996 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa538021070 con 0x7fa5480fbcb0 2024-05-06T12:15:28.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.435+0000 7fa544ff9700 1 -- 172.21.15.92:0/1415712996 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa538009430 con 0x7fa5480fbcb0 2024-05-06T12:15:28.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.435+0000 7fa54e2f4700 1 -- 172.21.15.92:0/1415712996 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa54810ef80 con 0x7fa5480fbcb0 2024-05-06T12:15:28.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.435+0000 7fa544ff9700 1 -- 172.21.15.92:0/1415712996 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa53800e070 con 0x7fa5480fbcb0 2024-05-06T12:15:28.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.435+0000 7fa54e2f4700 1 -- 172.21.15.92:0/1415712996 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa54810f4c0 con 0x7fa5480fbcb0 2024-05-06T12:15:28.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.436+0000 7fa54e2f4700 1 -- 172.21.15.92:0/1415712996 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa548107340 con 0x7fa5480fbcb0 2024-05-06T12:15:28.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.437+0000 7fa544ff9700 1 -- 172.21.15.92:0/1415712996 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7fa53801c750 con 0x7fa5480fbcb0 2024-05-06T12:15:28.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.437+0000 7fa544ff9700 1 --2- 172.21.15.92:0/1415712996 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fa53006b730 0x7fa53006dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:15:28.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.437+0000 7fa544ff9700 1 -- 172.21.15.92:0/1415712996 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fa53808b560 con 0x7fa5480fbcb0 2024-05-06T12:15:28.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.437+0000 7fa5477fe700 1 --2- 172.21.15.92:0/1415712996 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fa53006b730 0x7fa53006dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:15:28.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.438+0000 7fa5477fe700 1 --2- 172.21.15.92:0/1415712996 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fa53006b730 0x7fa53006dbe0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa53c009b80 tx=0x7fa53c00b040 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:15:28.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.443+0000 7fa544ff9700 1 -- 172.21.15.92:0/1415712996 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fa538058cf0 con 0x7fa5480fbcb0 2024-05-06T12:15:28.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.791+0000 7fa54e2f4700 1 -- 172.21.15.92:0/1415712996 --> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa548104910 con 0x7fa53006b730 2024-05-06T12:15:28.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.792+0000 7fa544ff9700 1 -- 172.21.15.92:0/1415712996 <== mgr.14636 v2:172.21.15.190:6828/3484477933 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7fa548104910 con 0x7fa53006b730 2024-05-06T12:15:28.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.794+0000 7fa54e2f4700 1 -- 172.21.15.92:0/1415712996 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fa53006b730 msgr2=0x7fa53006dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:15:28.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.794+0000 7fa54e2f4700 1 --2- 172.21.15.92:0/1415712996 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fa53006b730 0x7fa53006dbe0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa53c009b80 tx=0x7fa53c00b040 comp rx=0 tx=0).stop 2024-05-06T12:15:28.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.794+0000 7fa54e2f4700 1 -- 172.21.15.92:0/1415712996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5480fbcb0 msgr2=0x7fa54810d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:15:28.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.794+0000 7fa54e2f4700 1 --2- 172.21.15.92:0/1415712996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5480fbcb0 0x7fa54810d970 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa538013870 tx=0x7fa538009e30 comp rx=0 tx=0).stop 2024-05-06T12:15:28.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.794+0000 7fa54e2f4700 1 -- 172.21.15.92:0/1415712996 shutdown_connections 2024-05-06T12:15:28.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.794+0000 7fa54e2f4700 1 --2- 172.21.15.92:0/1415712996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5480fbcb0 0x7fa54810d970 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:28.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.794+0000 7fa54e2f4700 1 --2- 172.21.15.92:0/1415712996 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fa53006b730 0x7fa53006dbe0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:28.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.794+0000 7fa54e2f4700 1 --2- 172.21.15.92:0/1415712996 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5480fcf60 0x7fa54810deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:28.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.794+0000 7fa54e2f4700 1 -- 172.21.15.92:0/1415712996 >> 172.21.15.92:0/1415712996 conn(0x7fa5480f7430 msgr2=0x7fa5480f9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:15:28.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.795+0000 7fa54e2f4700 1 -- 172.21.15.92:0/1415712996 shutdown_connections 2024-05-06T12:15:28.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.795+0000 7fa54e2f4700 1 -- 172.21.15.92:0/1415712996 wait complete. 2024-05-06T12:15:28.810 INFO:teuthology.orchestra.run.smithi092.stdout:true 2024-05-06T12:15:28.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.932+0000 7f29f34d8700 1 -- 172.21.15.92:0/3964286460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29ec066260 msgr2=0x7f29ec0ffa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:15:28.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.932+0000 7f29f34d8700 1 --2- 172.21.15.92:0/3964286460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29ec066260 0x7f29ec0ffa00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f29e8004940 tx=0x7f29e8009e30 comp rx=0 tx=0).stop 2024-05-06T12:15:28.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.932+0000 7f29f34d8700 1 -- 172.21.15.92:0/3964286460 shutdown_connections 2024-05-06T12:15:28.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.932+0000 7f29f34d8700 1 --2- 172.21.15.92:0/3964286460 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29ec0fff40 0x7f29ec102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:28.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.932+0000 7f29f34d8700 1 --2- 172.21.15.92:0/3964286460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29ec066260 0x7f29ec0ffa00 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:28.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.932+0000 7f29f34d8700 1 -- 172.21.15.92:0/3964286460 >> 172.21.15.92:0/3964286460 conn(0x7f29ec0f7430 msgr2=0x7f29ec0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:15:28.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.932+0000 7f29f34d8700 1 -- 172.21.15.92:0/3964286460 shutdown_connections 2024-05-06T12:15:28.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.932+0000 7f29f34d8700 1 -- 172.21.15.92:0/3964286460 wait complete. 2024-05-06T12:15:28.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.932+0000 7f29f34d8700 1 Processor -- start 2024-05-06T12:15:28.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.933+0000 7f29f34d8700 1 -- start start 2024-05-06T12:15:28.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.933+0000 7f29f34d8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29ec066260 0x7f29ec0fd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:15:28.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.933+0000 7f29f34d8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29ec0fff40 0x7f29ec0fb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:15:28.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.933+0000 7f29f34d8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec0fbeb0 con 0x7f29ec0fff40 2024-05-06T12:15:28.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.933+0000 7f29f34d8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec0fc020 con 0x7f29ec066260 2024-05-06T12:15:28.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.933+0000 7f29f0a73700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29ec0fff40 0x7f29ec0fb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:15:28.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.933+0000 7f29f1274700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29ec066260 0x7f29ec0fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:15:28.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.933+0000 7f29f1274700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29ec066260 0x7f29ec0fd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.92:36146/0 (socket says 172.21.15.92:36146) 2024-05-06T12:15:28.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.933+0000 7f29f0a73700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29ec0fff40 0x7f29ec0fb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44390/0 (socket says 172.21.15.92:44390) 2024-05-06T12:15:28.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.933+0000 7f29f1274700 1 -- 172.21.15.92:0/3164529608 learned_addr learned my addr 172.21.15.92:0/3164529608 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:15:28.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.933+0000 7f29f1274700 1 -- 172.21.15.92:0/3164529608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29ec0fff40 msgr2=0x7f29ec0fb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:15:28.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.933+0000 7f29f1274700 1 --2- 172.21.15.92:0/3164529608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29ec0fff40 0x7f29ec0fb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:28.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.933+0000 7f29f1274700 1 -- 172.21.15.92:0/3164529608 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e8005040 con 0x7f29ec066260 2024-05-06T12:15:28.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.933+0000 7f29f0a73700 1 --2- 172.21.15.92:0/3164529608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29ec0fff40 0x7f29ec0fb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-06T12:15:28.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.934+0000 7f29f1274700 1 --2- 172.21.15.92:0/3164529608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29ec066260 0x7f29ec0fd430 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f29e800b3b0 tx=0x7f29e8009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:15:28.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.934+0000 7f29e1ffb700 1 -- 172.21.15.92:0/3164529608 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29e800e040 con 0x7f29ec066260 2024-05-06T12:15:28.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.934+0000 7f29f34d8700 1 -- 172.21.15.92:0/3164529608 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29ec0fc2a0 con 0x7f29ec066260 2024-05-06T12:15:28.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.934+0000 7f29e1ffb700 1 -- 172.21.15.92:0/3164529608 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f29e800bbf0 con 0x7f29ec066260 2024-05-06T12:15:28.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.934+0000 7f29e1ffb700 1 -- 172.21.15.92:0/3164529608 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29e801c630 con 0x7f29ec066260 2024-05-06T12:15:28.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.934+0000 7f29f34d8700 1 -- 172.21.15.92:0/3164529608 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29ec115a60 con 0x7f29ec066260 2024-05-06T12:15:28.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.936+0000 7f29f34d8700 1 -- 172.21.15.92:0/3164529608 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29ec05f5c0 con 0x7f29ec066260 2024-05-06T12:15:28.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.937+0000 7f29e1ffb700 1 -- 172.21.15.92:0/3164529608 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7f29e800bd60 con 0x7f29ec066260 2024-05-06T12:15:28.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.937+0000 7f29e1ffb700 1 --2- 172.21.15.92:0/3164529608 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f29d806b790 0x7f29d806dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:15:28.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.937+0000 7f29e1ffb700 1 -- 172.21.15.92:0/3164529608 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f29e808afd0 con 0x7f29ec066260 2024-05-06T12:15:28.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.938+0000 7f29f0a73700 1 --2- 172.21.15.92:0/3164529608 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f29d806b790 0x7f29d806dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:15:28.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.938+0000 7f29f0a73700 1 --2- 172.21.15.92:0/3164529608 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f29d806b790 0x7f29d806dc40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f29dc004fd0 tx=0x7f29dc00b040 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:15:28.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:28.945+0000 7f29e1ffb700 1 -- 172.21.15.92:0/3164529608 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f29e8054c30 con 0x7f29ec066260 2024-05-06T12:15:29.051 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:28 smithi092 ceph-mon[26954]: pgmap v46: 65 pgs: 65 active+clean; 393 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 185 op/s 2024-05-06T12:15:29.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:28 smithi190 ceph-mon[33798]: pgmap v46: 65 pgs: 65 active+clean; 393 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 185 op/s 2024-05-06T12:15:29.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.267+0000 7f29f34d8700 1 -- 172.21.15.92:0/3164529608 --> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f29ec0f8f20 con 0x7f29d806b790 2024-05-06T12:15:29.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.269+0000 7f29e1ffb700 1 -- 172.21.15.92:0/3164529608 <== mgr.14636 v2:172.21.15.190:6828/3484477933 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f29ec0f8f20 con 0x7f29d806b790 2024-05-06T12:15:29.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.272+0000 7f29f34d8700 1 -- 172.21.15.92:0/3164529608 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f29d806b790 msgr2=0x7f29d806dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:15:29.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.272+0000 7f29f34d8700 1 --2- 172.21.15.92:0/3164529608 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f29d806b790 0x7f29d806dc40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f29dc004fd0 tx=0x7f29dc00b040 comp rx=0 tx=0).stop 2024-05-06T12:15:29.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.272+0000 7f29f34d8700 1 -- 172.21.15.92:0/3164529608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29ec066260 msgr2=0x7f29ec0fd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:15:29.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.272+0000 7f29f34d8700 1 --2- 172.21.15.92:0/3164529608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29ec066260 0x7f29ec0fd430 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f29e800b3b0 tx=0x7f29e8009e30 comp rx=0 tx=0).stop 2024-05-06T12:15:29.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.272+0000 7f29f34d8700 1 -- 172.21.15.92:0/3164529608 shutdown_connections 2024-05-06T12:15:29.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.272+0000 7f29f34d8700 1 --2- 172.21.15.92:0/3164529608 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f29d806b790 0x7f29d806dc40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:29.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.272+0000 7f29f34d8700 1 --2- 172.21.15.92:0/3164529608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29ec066260 0x7f29ec0fd430 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:29.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.272+0000 7f29f34d8700 1 --2- 172.21.15.92:0/3164529608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29ec0fff40 0x7f29ec0fb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:29.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.272+0000 7f29f34d8700 1 -- 172.21.15.92:0/3164529608 >> 172.21.15.92:0/3164529608 conn(0x7f29ec0f7430 msgr2=0x7f29ec0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:15:29.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.272+0000 7f29f34d8700 1 -- 172.21.15.92:0/3164529608 shutdown_connections 2024-05-06T12:15:29.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.272+0000 7f29f34d8700 1 -- 172.21.15.92:0/3164529608 wait complete. 2024-05-06T12:15:29.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.395+0000 7efdfe9ea700 1 -- 172.21.15.92:0/3034169250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdf80fcf60 msgr2=0x7efdf80fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:15:29.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.395+0000 7efdfe9ea700 1 --2- 172.21.15.92:0/3034169250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdf80fcf60 0x7efdf80fd3d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efdec004940 tx=0x7efdec009e30 comp rx=0 tx=0).stop 2024-05-06T12:15:29.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.395+0000 7efdfe9ea700 1 -- 172.21.15.92:0/3034169250 shutdown_connections 2024-05-06T12:15:29.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.395+0000 7efdfe9ea700 1 --2- 172.21.15.92:0/3034169250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdf80fcf60 0x7efdf80fd3d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:29.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.395+0000 7efdfe9ea700 1 --2- 172.21.15.92:0/3034169250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdf80fbcb0 0x7efdf80fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:29.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.395+0000 7efdfe9ea700 1 -- 172.21.15.92:0/3034169250 >> 172.21.15.92:0/3034169250 conn(0x7efdf80f7430 msgr2=0x7efdf80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:15:29.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.396+0000 7efdfe9ea700 1 -- 172.21.15.92:0/3034169250 shutdown_connections 2024-05-06T12:15:29.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.396+0000 7efdfe9ea700 1 -- 172.21.15.92:0/3034169250 wait complete. 2024-05-06T12:15:29.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.397+0000 7efdfe9ea700 1 Processor -- start 2024-05-06T12:15:29.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.397+0000 7efdfe9ea700 1 -- start start 2024-05-06T12:15:29.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.397+0000 7efdfe9ea700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdf80fbcb0 0x7efdf810b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:15:29.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.397+0000 7efdfe9ea700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdf80fcf60 0x7efdf810bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:15:29.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.397+0000 7efdfe9ea700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf810c2f0 con 0x7efdf80fcf60 2024-05-06T12:15:29.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.397+0000 7efdfe9ea700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf810cb40 con 0x7efdf80fbcb0 2024-05-06T12:15:29.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.398+0000 7efdf7fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdf80fbcb0 0x7efdf810b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:15:29.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.398+0000 7efdf7fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdf80fbcb0 0x7efdf810b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.92:36170/0 (socket says 172.21.15.92:36170) 2024-05-06T12:15:29.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.398+0000 7efdf7fff700 1 -- 172.21.15.92:0/2960423110 learned_addr learned my addr 172.21.15.92:0/2960423110 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:15:29.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.398+0000 7efdf77fe700 1 --2- 172.21.15.92:0/2960423110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdf80fcf60 0x7efdf810bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:15:29.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.398+0000 7efdf7fff700 1 -- 172.21.15.92:0/2960423110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdf80fcf60 msgr2=0x7efdf810bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:15:29.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.398+0000 7efdf7fff700 1 --2- 172.21.15.92:0/2960423110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdf80fcf60 0x7efdf810bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:29.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.398+0000 7efdf7fff700 1 -- 172.21.15.92:0/2960423110 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdec005040 con 0x7efdf80fbcb0 2024-05-06T12:15:29.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.398+0000 7efdf7fff700 1 --2- 172.21.15.92:0/2960423110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdf80fbcb0 0x7efdf810b760 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efde800ba50 tx=0x7efde800d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:15:29.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.398+0000 7efdf4ff9700 1 -- 172.21.15.92:0/2960423110 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efde8010040 con 0x7efdf80fbcb0 2024-05-06T12:15:29.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.399+0000 7efdf4ff9700 1 -- 172.21.15.92:0/2960423110 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efde800ee10 con 0x7efdf80fbcb0 2024-05-06T12:15:29.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.399+0000 7efdf4ff9700 1 -- 172.21.15.92:0/2960423110 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efde8014400 con 0x7efdf80fbcb0 2024-05-06T12:15:29.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.399+0000 7efdfe9ea700 1 -- 172.21.15.92:0/2960423110 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdf810cdc0 con 0x7efdf80fbcb0 2024-05-06T12:15:29.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.399+0000 7efdfe9ea700 1 -- 172.21.15.92:0/2960423110 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdf810d300 con 0x7efdf80fbcb0 2024-05-06T12:15:29.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.400+0000 7efdfe9ea700 1 -- 172.21.15.92:0/2960423110 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efde40052b0 con 0x7efdf80fbcb0 2024-05-06T12:15:29.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.400+0000 7efdf4ff9700 1 -- 172.21.15.92:0/2960423110 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7efde8014560 con 0x7efdf80fbcb0 2024-05-06T12:15:29.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.401+0000 7efdf4ff9700 1 --2- 172.21.15.92:0/2960423110 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7efde006b790 0x7efde006dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:15:29.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.401+0000 7efdf4ff9700 1 -- 172.21.15.92:0/2960423110 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7efde8089fe0 con 0x7efdf80fbcb0 2024-05-06T12:15:29.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.401+0000 7efdf77fe700 1 --2- 172.21.15.92:0/2960423110 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7efde006b790 0x7efde006dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:15:29.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.402+0000 7efdf77fe700 1 --2- 172.21.15.92:0/2960423110 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7efde006b790 0x7efde006dc40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efdec013740 tx=0x7efdec009e30 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:15:29.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.409+0000 7efdf4ff9700 1 -- 172.21.15.92:0/2960423110 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7efde8050100 con 0x7efdf80fbcb0 2024-05-06T12:15:29.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.729+0000 7efdfe9ea700 1 -- 172.21.15.92:0/2960423110 --> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7efde4000bc0 con 0x7efde006b790 2024-05-06T12:15:29.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.739+0000 7efdf4ff9700 1 -- 172.21.15.92:0/2960423110 <== mgr.14636 v2:172.21.15.190:6828/3484477933 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7efde4000bc0 con 0x7efde006b790 2024-05-06T12:15:29.743 INFO:teuthology.orchestra.run.smithi092.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-06T12:15:29.743 INFO:teuthology.orchestra.run.smithi092.stdout:alertmanager.smithi092 smithi092 *:9093,9094 starting - - - - 2024-05-06T12:15:29.743 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi092 smithi092 running (9m) 69s ago 9m 6660k - 17.2.7-925-ge81df420 96d5ae7fd4b4 95dc7e14bd9a 2024-05-06T12:15:29.743 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi190 smithi190 running (8m) 60s ago 8m 6665k - 17.2.7-925-ge81df420 96d5ae7fd4b4 6cf97785406f 2024-05-06T12:15:29.743 INFO:teuthology.orchestra.run.smithi092.stdout:grafana.smithi092 smithi092 *:3000 starting - - - - 2024-05-06T12:15:29.743 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.ijsqgp smithi092 running (3m) 69s ago 3m 79.3M - 17.2.7-925-ge81df420 96d5ae7fd4b4 b4c2ed6d2c82 2024-05-06T12:15:29.743 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.rosbll smithi092 running (3m) 69s ago 3m 101M - 17.2.7-925-ge81df420 96d5ae7fd4b4 5f5426cd1bf5 2024-05-06T12:15:29.743 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.byjvgf smithi190 running (3m) 60s ago 3m 112M - 17.2.7-925-ge81df420 96d5ae7fd4b4 24581d2e1c5d 2024-05-06T12:15:29.743 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.idzoff smithi190 running (3m) 60s ago 3m 69.7M - 17.2.7-925-ge81df420 96d5ae7fd4b4 20f2641696ef 2024-05-06T12:15:29.743 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi092.gyzeqb smithi092 *:9283 running (10m) 69s ago 10m 419M - 17.2.7-925-ge81df420 96d5ae7fd4b4 820ba9ba8a82 2024-05-06T12:15:29.743 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi190.kczxkh smithi190 *:8443,9283 running (97s) 60s ago 8m 491M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 efed230a6048 2024-05-06T12:15:29.744 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi092 smithi092 running (10m) 69s ago 10m 46.6M 2048M 17.2.7-925-ge81df420 96d5ae7fd4b4 df5b322e9719 2024-05-06T12:15:29.744 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi190 smithi190 running (8m) 60s ago 8m 37.7M 2048M 17.2.7-925-ge81df420 96d5ae7fd4b4 912ded072601 2024-05-06T12:15:29.744 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi092 smithi092 *:9100 starting - - - - 2024-05-06T12:15:29.744 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi190 smithi190 *:9100 starting - - - - 2024-05-06T12:15:29.744 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 smithi092 running (7m) 69s ago 7m 77.5M 1557M 17.2.7-925-ge81df420 96d5ae7fd4b4 ce84e7af9084 2024-05-06T12:15:29.744 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 smithi092 running (6m) 69s ago 6m 81.5M 1557M 17.2.7-925-ge81df420 96d5ae7fd4b4 8ab870f57eb2 2024-05-06T12:15:29.744 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 smithi092 running (6m) 69s ago 6m 67.4M 1557M 17.2.7-925-ge81df420 96d5ae7fd4b4 1e6ed04a0688 2024-05-06T12:15:29.744 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 smithi190 running (6m) 60s ago 6m 87.2M 2581M 17.2.7-925-ge81df420 96d5ae7fd4b4 52f9521286d5 2024-05-06T12:15:29.744 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 smithi190 running (5m) 60s ago 5m 72.8M 2581M 17.2.7-925-ge81df420 96d5ae7fd4b4 3a7da09e4c48 2024-05-06T12:15:29.744 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 smithi190 running (5m) 60s ago 5m 73.6M 2581M 17.2.7-925-ge81df420 96d5ae7fd4b4 e7452c6b5cec 2024-05-06T12:15:29.744 INFO:teuthology.orchestra.run.smithi092.stdout:prometheus.smithi092 smithi092 *:9095 starting - - - - 2024-05-06T12:15:29.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.742+0000 7efdfe9ea700 1 -- 172.21.15.92:0/2960423110 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7efde006b790 msgr2=0x7efde006dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:15:29.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.742+0000 7efdfe9ea700 1 --2- 172.21.15.92:0/2960423110 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7efde006b790 0x7efde006dc40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efdec013740 tx=0x7efdec009e30 comp rx=0 tx=0).stop 2024-05-06T12:15:29.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.742+0000 7efdfe9ea700 1 -- 172.21.15.92:0/2960423110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdf80fbcb0 msgr2=0x7efdf810b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:15:29.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.742+0000 7efdfe9ea700 1 --2- 172.21.15.92:0/2960423110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdf80fbcb0 0x7efdf810b760 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efde800ba50 tx=0x7efde800d040 comp rx=0 tx=0).stop 2024-05-06T12:15:29.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.742+0000 7efdfe9ea700 1 -- 172.21.15.92:0/2960423110 shutdown_connections 2024-05-06T12:15:29.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.742+0000 7efdfe9ea700 1 --2- 172.21.15.92:0/2960423110 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7efde006b790 0x7efde006dc40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:29.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.742+0000 7efdfe9ea700 1 --2- 172.21.15.92:0/2960423110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdf80fbcb0 0x7efdf810b760 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:29.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.742+0000 7efdfe9ea700 1 --2- 172.21.15.92:0/2960423110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdf80fcf60 0x7efdf810bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:29.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.742+0000 7efdfe9ea700 1 -- 172.21.15.92:0/2960423110 >> 172.21.15.92:0/2960423110 conn(0x7efdf80f7430 msgr2=0x7efdf80f8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:15:29.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.743+0000 7efdfe9ea700 1 -- 172.21.15.92:0/2960423110 shutdown_connections 2024-05-06T12:15:29.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.743+0000 7efdfe9ea700 1 -- 172.21.15.92:0/2960423110 wait complete. 2024-05-06T12:15:29.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.887+0000 7fa06da8d700 1 -- 172.21.15.92:0/1841495810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0680fcf90 msgr2=0x7fa0680fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:15:29.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.887+0000 7fa06da8d700 1 --2- 172.21.15.92:0/1841495810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0680fcf90 0x7fa0680fd400 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa05c004940 tx=0x7fa05c009e30 comp rx=0 tx=0).stop 2024-05-06T12:15:29.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.887+0000 7fa06da8d700 1 -- 172.21.15.92:0/1841495810 shutdown_connections 2024-05-06T12:15:29.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.887+0000 7fa06da8d700 1 --2- 172.21.15.92:0/1841495810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0680fcf90 0x7fa0680fd400 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:29.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.887+0000 7fa06da8d700 1 --2- 172.21.15.92:0/1841495810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0680fbce0 0x7fa0680fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:29.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.887+0000 7fa06da8d700 1 -- 172.21.15.92:0/1841495810 >> 172.21.15.92:0/1841495810 conn(0x7fa0680f7440 msgr2=0x7fa0680f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:15:29.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.887+0000 7fa06da8d700 1 -- 172.21.15.92:0/1841495810 shutdown_connections 2024-05-06T12:15:29.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.887+0000 7fa06da8d700 1 -- 172.21.15.92:0/1841495810 wait complete. 2024-05-06T12:15:29.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.888+0000 7fa06da8d700 1 Processor -- start 2024-05-06T12:15:29.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.888+0000 7fa06da8d700 1 -- start start 2024-05-06T12:15:29.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.888+0000 7fa06da8d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0680fbce0 0x7fa068110660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:15:29.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.888+0000 7fa06da8d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0680fcf90 0x7fa068112bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:15:29.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.888+0000 7fa06da8d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa068066c20 con 0x7fa0680fbce0 2024-05-06T12:15:29.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.888+0000 7fa06da8d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa068066d60 con 0x7fa0680fcf90 2024-05-06T12:15:29.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.888+0000 7fa0677fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0680fbce0 0x7fa068110660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:15:29.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.888+0000 7fa0677fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0680fbce0 0x7fa068110660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44426/0 (socket says 172.21.15.92:44426) 2024-05-06T12:15:29.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.888+0000 7fa0677fe700 1 -- 172.21.15.92:0/3369482869 learned_addr learned my addr 172.21.15.92:0/3369482869 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:15:29.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.889+0000 7fa066ffd700 1 --2- 172.21.15.92:0/3369482869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0680fcf90 0x7fa068112bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:15:29.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.889+0000 7fa0677fe700 1 -- 172.21.15.92:0/3369482869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0680fcf90 msgr2=0x7fa068112bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:15:29.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.889+0000 7fa0677fe700 1 --2- 172.21.15.92:0/3369482869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0680fcf90 0x7fa068112bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:29.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.889+0000 7fa0677fe700 1 -- 172.21.15.92:0/3369482869 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa05c005040 con 0x7fa0680fbce0 2024-05-06T12:15:29.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.889+0000 7fa066ffd700 1 --2- 172.21.15.92:0/3369482869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0680fcf90 0x7fa068112bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-06T12:15:29.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.889+0000 7fa0677fe700 1 --2- 172.21.15.92:0/3369482869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0680fbce0 0x7fa068110660 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa05800eb40 tx=0x7fa05800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:15:29.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.889+0000 7fa04ffff700 1 -- 172.21.15.92:0/3369482869 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa058007cc0 con 0x7fa0680fbce0 2024-05-06T12:15:29.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.889+0000 7fa06da8d700 1 -- 172.21.15.92:0/3369482869 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa068113150 con 0x7fa0680fbce0 2024-05-06T12:15:29.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.889+0000 7fa04ffff700 1 -- 172.21.15.92:0/3369482869 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa05800f040 con 0x7fa0680fbce0 2024-05-06T12:15:29.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.889+0000 7fa04ffff700 1 -- 172.21.15.92:0/3369482869 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa05800ca10 con 0x7fa0680fbce0 2024-05-06T12:15:29.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.889+0000 7fa06da8d700 1 -- 172.21.15.92:0/3369482869 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa068113690 con 0x7fa0680fbce0 2024-05-06T12:15:29.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.891+0000 7fa06da8d700 1 -- 172.21.15.92:0/3369482869 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa06805f5c0 con 0x7fa0680fbce0 2024-05-06T12:15:29.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.891+0000 7fa04ffff700 1 -- 172.21.15.92:0/3369482869 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7fa05800cc80 con 0x7fa0680fbce0 2024-05-06T12:15:29.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.891+0000 7fa04ffff700 1 --2- 172.21.15.92:0/3369482869 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fa05006b790 0x7fa05006dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:15:29.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.891+0000 7fa04ffff700 1 -- 172.21.15.92:0/3369482869 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fa05808a040 con 0x7fa0680fbce0 2024-05-06T12:15:29.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.892+0000 7fa066ffd700 1 --2- 172.21.15.92:0/3369482869 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fa05006b790 0x7fa05006dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:15:29.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.892+0000 7fa066ffd700 1 --2- 172.21.15.92:0/3369482869 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fa05006b790 0x7fa05006dc40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009b30 tx=0x7fa05c009e30 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:15:29.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:29.900+0000 7fa04ffff700 1 -- 172.21.15.92:0/3369482869 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fa058050100 con 0x7fa0680fbce0 2024-05-06T12:15:30.029 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:29 smithi092 ceph-mon[26954]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:15:30.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:29 smithi190 ceph-mon[33798]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:15:30.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.249+0000 7fa06da8d700 1 -- 172.21.15.92:0/3369482869 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa0681047d0 con 0x7fa0680fbce0 2024-05-06T12:15:30.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.250+0000 7fa04ffff700 1 -- 172.21.15.92:0/3369482869 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7fa058053bb0 con 0x7fa0680fbce0 2024-05-06T12:15:30.253 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:15:30.253 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": { 2024-05-06T12:15:30.253 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 2 2024-05-06T12:15:30.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:15:30.253 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": { 2024-05-06T12:15:30.254 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 1, 2024-05-06T12:15:30.254 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 1 2024-05-06T12:15:30.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:15:30.254 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": { 2024-05-06T12:15:30.254 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 6 2024-05-06T12:15:30.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:15:30.254 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-05-06T12:15:30.254 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 4 2024-05-06T12:15:30.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:15:30.254 INFO:teuthology.orchestra.run.smithi092.stdout: "overall": { 2024-05-06T12:15:30.254 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 13, 2024-05-06T12:15:30.254 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 1 2024-05-06T12:15:30.254 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-06T12:15:30.254 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:15:30.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.252+0000 7fa06da8d700 1 -- 172.21.15.92:0/3369482869 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fa05006b790 msgr2=0x7fa05006dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:15:30.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.252+0000 7fa06da8d700 1 --2- 172.21.15.92:0/3369482869 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fa05006b790 0x7fa05006dc40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009b30 tx=0x7fa05c009e30 comp rx=0 tx=0).stop 2024-05-06T12:15:30.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.252+0000 7fa06da8d700 1 -- 172.21.15.92:0/3369482869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0680fbce0 msgr2=0x7fa068110660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:15:30.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.252+0000 7fa06da8d700 1 --2- 172.21.15.92:0/3369482869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0680fbce0 0x7fa068110660 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa05800eb40 tx=0x7fa05800b040 comp rx=0 tx=0).stop 2024-05-06T12:15:30.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.252+0000 7fa06da8d700 1 -- 172.21.15.92:0/3369482869 shutdown_connections 2024-05-06T12:15:30.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.252+0000 7fa06da8d700 1 --2- 172.21.15.92:0/3369482869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0680fbce0 0x7fa068110660 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:30.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.252+0000 7fa06da8d700 1 --2- 172.21.15.92:0/3369482869 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fa05006b790 0x7fa05006dc40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:30.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.252+0000 7fa06da8d700 1 --2- 172.21.15.92:0/3369482869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0680fcf90 0x7fa068112bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:30.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.252+0000 7fa06da8d700 1 -- 172.21.15.92:0/3369482869 >> 172.21.15.92:0/3369482869 conn(0x7fa0680f7440 msgr2=0x7fa0680f8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:15:30.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.252+0000 7fa06da8d700 1 -- 172.21.15.92:0/3369482869 shutdown_connections 2024-05-06T12:15:30.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.253+0000 7fa06da8d700 1 -- 172.21.15.92:0/3369482869 wait complete. 2024-05-06T12:15:30.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.369+0000 7f74ef60f700 1 -- 172.21.15.92:0/3852656317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74e8066260 msgr2=0x7f74e80ffa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:15:30.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.369+0000 7f74ef60f700 1 --2- 172.21.15.92:0/3852656317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74e8066260 0x7f74e80ffa00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f74e4004940 tx=0x7f74e4009e30 comp rx=0 tx=0).stop 2024-05-06T12:15:30.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.370+0000 7f74ef60f700 1 -- 172.21.15.92:0/3852656317 shutdown_connections 2024-05-06T12:15:30.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.370+0000 7f74ef60f700 1 --2- 172.21.15.92:0/3852656317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74e80fff40 0x7f74e8102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:30.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.370+0000 7f74ef60f700 1 --2- 172.21.15.92:0/3852656317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74e8066260 0x7f74e80ffa00 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:30.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.370+0000 7f74ef60f700 1 -- 172.21.15.92:0/3852656317 >> 172.21.15.92:0/3852656317 conn(0x7f74e80f7430 msgr2=0x7f74e80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:15:30.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.370+0000 7f74ef60f700 1 -- 172.21.15.92:0/3852656317 shutdown_connections 2024-05-06T12:15:30.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.370+0000 7f74ef60f700 1 -- 172.21.15.92:0/3852656317 wait complete. 2024-05-06T12:15:30.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.370+0000 7f74ef60f700 1 Processor -- start 2024-05-06T12:15:30.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.370+0000 7f74ef60f700 1 -- start start 2024-05-06T12:15:30.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.371+0000 7f74ef60f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74e8066260 0x7f74e810da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:15:30.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.371+0000 7f74ef60f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74e80fff40 0x7f74e810df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:15:30.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.371+0000 7f74ef60f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e810e570 con 0x7f74e80fff40 2024-05-06T12:15:30.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.371+0000 7f74ef60f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e810edc0 con 0x7f74e8066260 2024-05-06T12:15:30.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.371+0000 7f74ecbaa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74e80fff40 0x7f74e810df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:15:30.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.371+0000 7f74ed3ab700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74e8066260 0x7f74e810da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:15:30.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.371+0000 7f74ecbaa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74e80fff40 0x7f74e810df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44442/0 (socket says 172.21.15.92:44442) 2024-05-06T12:15:30.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.371+0000 7f74ecbaa700 1 -- 172.21.15.92:0/1339664548 learned_addr learned my addr 172.21.15.92:0/1339664548 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:15:30.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.371+0000 7f74ecbaa700 1 -- 172.21.15.92:0/1339664548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74e8066260 msgr2=0x7f74e810da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:15:30.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.371+0000 7f74ecbaa700 1 --2- 172.21.15.92:0/1339664548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74e8066260 0x7f74e810da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:30.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.371+0000 7f74ecbaa700 1 -- 172.21.15.92:0/1339664548 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e4005040 con 0x7f74e80fff40 2024-05-06T12:15:30.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.371+0000 7f74ecbaa700 1 --2- 172.21.15.92:0/1339664548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74e80fff40 0x7f74e810df70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f74d8002a50 tx=0x7f74d80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:15:30.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.371+0000 7f74ddffb700 1 -- 172.21.15.92:0/1339664548 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74d8007cc0 con 0x7f74e80fff40 2024-05-06T12:15:30.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.371+0000 7f74ef60f700 1 -- 172.21.15.92:0/1339664548 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e810f0a0 con 0x7f74e80fff40 2024-05-06T12:15:30.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.372+0000 7f74ddffb700 1 -- 172.21.15.92:0/1339664548 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74d800f040 con 0x7f74e80fff40 2024-05-06T12:15:30.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.372+0000 7f74ddffb700 1 -- 172.21.15.92:0/1339664548 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74d800da30 con 0x7f74e80fff40 2024-05-06T12:15:30.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.372+0000 7f74ef60f700 1 -- 172.21.15.92:0/1339664548 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e810f5e0 con 0x7f74e80fff40 2024-05-06T12:15:30.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.373+0000 7f74ef60f700 1 -- 172.21.15.92:0/1339664548 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74e805f5c0 con 0x7f74e80fff40 2024-05-06T12:15:30.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.373+0000 7f74ddffb700 1 -- 172.21.15.92:0/1339664548 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7f74d800db90 con 0x7f74e80fff40 2024-05-06T12:15:30.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.373+0000 7f74ddffb700 1 --2- 172.21.15.92:0/1339664548 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f74d406b790 0x7f74d406dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:15:30.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.374+0000 7f74ddffb700 1 -- 172.21.15.92:0/1339664548 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f74d808a120 con 0x7f74e80fff40 2024-05-06T12:15:30.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.374+0000 7f74ed3ab700 1 --2- 172.21.15.92:0/1339664548 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f74d406b790 0x7f74d406dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:15:30.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.374+0000 7f74ed3ab700 1 --2- 172.21.15.92:0/1339664548 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f74d406b790 0x7f74d406dc40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f74e4004890 tx=0x7f74e4009e30 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:15:30.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.380+0000 7f74ddffb700 1 -- 172.21.15.92:0/1339664548 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f74d8057790 con 0x7f74e80fff40 2024-05-06T12:15:30.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.783+0000 7f74ef60f700 1 -- 172.21.15.92:0/1339664548 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f74e810f850 con 0x7f74e80fff40 2024-05-06T12:15:30.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.783+0000 7f74ddffb700 1 -- 172.21.15.92:0/1339664548 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1867 (secure 0 0 0) 0x7f74d8057320 con 0x7f74e80fff40 2024-05-06T12:15:30.787 INFO:teuthology.orchestra.run.smithi092.stdout:e20 2024-05-06T12:15:30.787 INFO:teuthology.orchestra.run.smithi092.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-06T12:15:30.787 INFO:teuthology.orchestra.run.smithi092.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:15:30.787 INFO:teuthology.orchestra.run.smithi092.stdout:legacy client fscid: 1 2024-05-06T12:15:30.787 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:15:30.787 INFO:teuthology.orchestra.run.smithi092.stdout:Filesystem 'cephfs' (1) 2024-05-06T12:15:30.787 INFO:teuthology.orchestra.run.smithi092.stdout:fs_name cephfs 2024-05-06T12:15:30.787 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 20 2024-05-06T12:15:30.787 INFO:teuthology.orchestra.run.smithi092.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-06T12:15:30.787 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-05-06T12:11:44.952671+0000 2024-05-06T12:15:30.787 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2024-05-06T12:15:07.568167+0000 2024-05-06T12:15:30.787 INFO:teuthology.orchestra.run.smithi092.stdout:tableserver 0 2024-05-06T12:15:30.787 INFO:teuthology.orchestra.run.smithi092.stdout:root 0 2024-05-06T12:15:30.788 INFO:teuthology.orchestra.run.smithi092.stdout:session_timeout 60 2024-05-06T12:15:30.788 INFO:teuthology.orchestra.run.smithi092.stdout:session_autoclose 300 2024-05-06T12:15:30.788 INFO:teuthology.orchestra.run.smithi092.stdout:max_file_size 1099511627776 2024-05-06T12:15:30.788 INFO:teuthology.orchestra.run.smithi092.stdout:required_client_features {} 2024-05-06T12:15:30.788 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure 0 2024-05-06T12:15:30.788 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure_osd_epoch 46 2024-05-06T12:15:30.788 INFO:teuthology.orchestra.run.smithi092.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:15:30.788 INFO:teuthology.orchestra.run.smithi092.stdout:max_mds 2 2024-05-06T12:15:30.788 INFO:teuthology.orchestra.run.smithi092.stdout:in 0,1 2024-05-06T12:15:30.788 INFO:teuthology.orchestra.run.smithi092.stdout:up {0=14472,1=24267} 2024-05-06T12:15:30.788 INFO:teuthology.orchestra.run.smithi092.stdout:failed 2024-05-06T12:15:30.788 INFO:teuthology.orchestra.run.smithi092.stdout:damaged 2024-05-06T12:15:30.788 INFO:teuthology.orchestra.run.smithi092.stdout:stopped 2024-05-06T12:15:30.788 INFO:teuthology.orchestra.run.smithi092.stdout:data_pools [3] 2024-05-06T12:15:30.789 INFO:teuthology.orchestra.run.smithi092.stdout:metadata_pool 2 2024-05-06T12:15:30.789 INFO:teuthology.orchestra.run.smithi092.stdout:inline_data enabled 2024-05-06T12:15:30.789 INFO:teuthology.orchestra.run.smithi092.stdout:balancer 2024-05-06T12:15:30.789 INFO:teuthology.orchestra.run.smithi092.stdout:standby_count_wanted 1 2024-05-06T12:15:30.789 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.rosbll{0:14472} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.92:6828/1663253598,v1:172.21.15.92:6829/1663253598] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:15:30.789 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.ijsqgp{0:14484} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.92:6826/3613441054,v1:172.21.15.92:6827/3613441054] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:15:30.789 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.byjvgf{1:24267} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.190:6824/1565151699,v1:172.21.15.190:6825/1565151699] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:15:30.789 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.idzoff{1:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.190:6826/2453728380,v1:172.21.15.190:6827/2453728380] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:15:30.789 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:15:30.789 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:15:30.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.785+0000 7f74ef60f700 1 -- 172.21.15.92:0/1339664548 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f74d406b790 msgr2=0x7f74d406dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:15:30.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.785+0000 7f74ef60f700 1 --2- 172.21.15.92:0/1339664548 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f74d406b790 0x7f74d406dc40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f74e4004890 tx=0x7f74e4009e30 comp rx=0 tx=0).stop 2024-05-06T12:15:30.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.785+0000 7f74ef60f700 1 -- 172.21.15.92:0/1339664548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74e80fff40 msgr2=0x7f74e810df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:15:30.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.785+0000 7f74ef60f700 1 --2- 172.21.15.92:0/1339664548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74e80fff40 0x7f74e810df70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f74d8002a50 tx=0x7f74d80046b0 comp rx=0 tx=0).stop 2024-05-06T12:15:30.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.785+0000 7f74ef60f700 1 -- 172.21.15.92:0/1339664548 shutdown_connections 2024-05-06T12:15:30.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.785+0000 7f74ef60f700 1 --2- 172.21.15.92:0/1339664548 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f74d406b790 0x7f74d406dc40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:30.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.785+0000 7f74ef60f700 1 --2- 172.21.15.92:0/1339664548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74e8066260 0x7f74e810da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:30.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.785+0000 7f74ef60f700 1 --2- 172.21.15.92:0/1339664548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74e80fff40 0x7f74e810df70 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:30.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.785+0000 7f74ef60f700 1 -- 172.21.15.92:0/1339664548 >> 172.21.15.92:0/1339664548 conn(0x7f74e80f7430 msgr2=0x7f74e80f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:15:30.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.786+0000 7f74ef60f700 1 -- 172.21.15.92:0/1339664548 shutdown_connections 2024-05-06T12:15:30.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.786+0000 7f74ef60f700 1 -- 172.21.15.92:0/1339664548 wait complete. 2024-05-06T12:15:30.791 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 20 2024-05-06T12:15:30.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.919+0000 7fa8045ab700 1 -- 172.21.15.92:0/120204024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7fc066af0 msgr2=0x7fa7fc0ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:15:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.919+0000 7fa8045ab700 1 --2- 172.21.15.92:0/120204024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7fc066af0 0x7fa7fc0ffa30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8004940 tx=0x7fa7f8009e30 comp rx=0 tx=0).stop 2024-05-06T12:15:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.919+0000 7fa8045ab700 1 -- 172.21.15.92:0/120204024 shutdown_connections 2024-05-06T12:15:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.919+0000 7fa8045ab700 1 --2- 172.21.15.92:0/120204024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7fc0fff70 0x7fa7fc102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.919+0000 7fa8045ab700 1 --2- 172.21.15.92:0/120204024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7fc066af0 0x7fa7fc0ffa30 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.919+0000 7fa8045ab700 1 -- 172.21.15.92:0/120204024 >> 172.21.15.92:0/120204024 conn(0x7fa7fc0f7440 msgr2=0x7fa7fc0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:15:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.919+0000 7fa8045ab700 1 -- 172.21.15.92:0/120204024 shutdown_connections 2024-05-06T12:15:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.919+0000 7fa8045ab700 1 -- 172.21.15.92:0/120204024 wait complete. 2024-05-06T12:15:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.920+0000 7fa8045ab700 1 Processor -- start 2024-05-06T12:15:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.920+0000 7fa8045ab700 1 -- start start 2024-05-06T12:15:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.920+0000 7fa8045ab700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7fc066af0 0x7fa7fc10da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:15:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.920+0000 7fa8045ab700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7fc0fff70 0x7fa7fc10df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:15:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.920+0000 7fa8045ab700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc10e550 con 0x7fa7fc066af0 2024-05-06T12:15:30.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.920+0000 7fa8045ab700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc10ed50 con 0x7fa7fc0fff70 2024-05-06T12:15:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.920+0000 7fa802347700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7fc066af0 0x7fa7fc10da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:15:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.920+0000 7fa802347700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7fc066af0 0x7fa7fc10da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44456/0 (socket says 172.21.15.92:44456) 2024-05-06T12:15:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.920+0000 7fa802347700 1 -- 172.21.15.92:0/1658821307 learned_addr learned my addr 172.21.15.92:0/1658821307 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:15:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.920+0000 7fa801b46700 1 --2- 172.21.15.92:0/1658821307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7fc0fff70 0x7fa7fc10df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:15:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.921+0000 7fa802347700 1 -- 172.21.15.92:0/1658821307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7fc0fff70 msgr2=0x7fa7fc10df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:15:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.921+0000 7fa802347700 1 --2- 172.21.15.92:0/1658821307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7fc0fff70 0x7fa7fc10df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.921+0000 7fa802347700 1 -- 172.21.15.92:0/1658821307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f8005040 con 0x7fa7fc066af0 2024-05-06T12:15:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.921+0000 7fa802347700 1 --2- 172.21.15.92:0/1658821307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7fc066af0 0x7fa7fc10da10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8000c00 tx=0x7fa7f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:15:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.921+0000 7fa7f2ffd700 1 -- 172.21.15.92:0/1658821307 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7f800e030 con 0x7fa7fc066af0 2024-05-06T12:15:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.921+0000 7fa7f2ffd700 1 -- 172.21.15.92:0/1658821307 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa7f8013de0 con 0x7fa7fc066af0 2024-05-06T12:15:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.921+0000 7fa7f2ffd700 1 -- 172.21.15.92:0/1658821307 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7f801ce50 con 0x7fa7fc066af0 2024-05-06T12:15:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.921+0000 7fa8045ab700 1 -- 172.21.15.92:0/1658821307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7fc10efd0 con 0x7fa7fc066af0 2024-05-06T12:15:30.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.922+0000 7fa8045ab700 1 -- 172.21.15.92:0/1658821307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7fc10f4c0 con 0x7fa7fc066af0 2024-05-06T12:15:30.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.923+0000 7fa8045ab700 1 -- 172.21.15.92:0/1658821307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7fc05f5c0 con 0x7fa7fc066af0 2024-05-06T12:15:30.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.923+0000 7fa7f2ffd700 1 -- 172.21.15.92:0/1658821307 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7fa7f801c400 con 0x7fa7fc066af0 2024-05-06T12:15:30.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.924+0000 7fa7f2ffd700 1 --2- 172.21.15.92:0/1658821307 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fa7e806b730 0x7fa7e806dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:15:30.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.924+0000 7fa7f2ffd700 1 -- 172.21.15.92:0/1658821307 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fa7f808be00 con 0x7fa7fc066af0 2024-05-06T12:15:30.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.924+0000 7fa801b46700 1 --2- 172.21.15.92:0/1658821307 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fa7e806b730 0x7fa7e806dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:15:30.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.924+0000 7fa801b46700 1 --2- 172.21.15.92:0/1658821307 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fa7e806b730 0x7fa7e806dbe0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec009b80 tx=0x7fa7ec00b040 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:15:30.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:30.929+0000 7fa7f2ffd700 1 -- 172.21.15.92:0/1658821307 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fa7f8059510 con 0x7fa7fc066af0 2024-05-06T12:15:31.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:30 smithi190 ceph-mon[33798]: from='client.24483 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:15:31.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:30 smithi190 ceph-mon[33798]: pgmap v47: 65 pgs: 65 active+clean; 301 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 209 op/s 2024-05-06T12:15:31.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:30 smithi190 ceph-mon[33798]: from='client.24487 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:15:31.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:30 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/3369482869' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:15:31.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:30 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/1339664548' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:15:31.155 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:30 smithi092 ceph-mon[26954]: from='client.24483 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:15:31.155 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:30 smithi092 ceph-mon[26954]: pgmap v47: 65 pgs: 65 active+clean; 301 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 209 op/s 2024-05-06T12:15:31.155 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:30 smithi092 ceph-mon[26954]: from='client.24487 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:15:31.156 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:30 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/3369482869' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:15:31.156 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:30 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/1339664548' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:15:31.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.250+0000 7fa8045ab700 1 -- 172.21.15.92:0/1658821307 --> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7fc10f7a0 con 0x7fa7e806b730 2024-05-06T12:15:31.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.251+0000 7fa7f2ffd700 1 -- 172.21.15.92:0/1658821307 <== mgr.14636 v2:172.21.15.190:6828/3484477933 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7fa7fc10f7a0 con 0x7fa7e806b730 2024-05-06T12:15:31.255 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:15:31.255 INFO:teuthology.orchestra.run.smithi092.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb", 2024-05-06T12:15:31.255 INFO:teuthology.orchestra.run.smithi092.stdout: "in_progress": true, 2024-05-06T12:15:31.255 INFO:teuthology.orchestra.run.smithi092.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-06T12:15:31.255 INFO:teuthology.orchestra.run.smithi092.stdout: "services_complete": [], 2024-05-06T12:15:31.255 INFO:teuthology.orchestra.run.smithi092.stdout: "progress": "1/21 daemons upgraded", 2024-05-06T12:15:31.255 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "Currently upgrading mgr daemons", 2024-05-06T12:15:31.255 INFO:teuthology.orchestra.run.smithi092.stdout: "is_paused": false 2024-05-06T12:15:31.255 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:15:31.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.254+0000 7fa8045ab700 1 -- 172.21.15.92:0/1658821307 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fa7e806b730 msgr2=0x7fa7e806dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:15:31.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.254+0000 7fa8045ab700 1 --2- 172.21.15.92:0/1658821307 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fa7e806b730 0x7fa7e806dbe0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec009b80 tx=0x7fa7ec00b040 comp rx=0 tx=0).stop 2024-05-06T12:15:31.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.254+0000 7fa8045ab700 1 -- 172.21.15.92:0/1658821307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7fc066af0 msgr2=0x7fa7fc10da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:15:31.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.254+0000 7fa8045ab700 1 --2- 172.21.15.92:0/1658821307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7fc066af0 0x7fa7fc10da10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8000c00 tx=0x7fa7f8009e30 comp rx=0 tx=0).stop 2024-05-06T12:15:31.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.254+0000 7fa8045ab700 1 -- 172.21.15.92:0/1658821307 shutdown_connections 2024-05-06T12:15:31.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.254+0000 7fa8045ab700 1 --2- 172.21.15.92:0/1658821307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7fc066af0 0x7fa7fc10da10 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:31.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.254+0000 7fa8045ab700 1 --2- 172.21.15.92:0/1658821307 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7fa7e806b730 0x7fa7e806dbe0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:31.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.254+0000 7fa8045ab700 1 --2- 172.21.15.92:0/1658821307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7fc0fff70 0x7fa7fc10df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:31.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.254+0000 7fa8045ab700 1 -- 172.21.15.92:0/1658821307 >> 172.21.15.92:0/1658821307 conn(0x7fa7fc0f7440 msgr2=0x7fa7fc0f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:15:31.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.254+0000 7fa8045ab700 1 -- 172.21.15.92:0/1658821307 shutdown_connections 2024-05-06T12:15:31.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.254+0000 7fa8045ab700 1 -- 172.21.15.92:0/1658821307 wait complete. 2024-05-06T12:15:31.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.387+0000 7f0bb2375700 1 -- 172.21.15.92:0/2361510297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bac0fc290 msgr2=0x7f0bac0fc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:15:31.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.387+0000 7f0bb2375700 1 --2- 172.21.15.92:0/2361510297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bac0fc290 0x7f0bac0fc6a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c004940 tx=0x7f0b9c009e30 comp rx=0 tx=0).stop 2024-05-06T12:15:31.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.387+0000 7f0bb2375700 1 -- 172.21.15.92:0/2361510297 shutdown_connections 2024-05-06T12:15:31.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.387+0000 7f0bb2375700 1 --2- 172.21.15.92:0/2361510297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bac0fcbe0 0x7f0bac0fd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:31.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.387+0000 7f0bb2375700 1 --2- 172.21.15.92:0/2361510297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bac0fc290 0x7f0bac0fc6a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:31.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.387+0000 7f0bb2375700 1 -- 172.21.15.92:0/2361510297 >> 172.21.15.92:0/2361510297 conn(0x7f0bac0f6b80 msgr2=0x7f0bac0f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:15:31.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.387+0000 7f0bb2375700 1 -- 172.21.15.92:0/2361510297 shutdown_connections 2024-05-06T12:15:31.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.387+0000 7f0bb2375700 1 -- 172.21.15.92:0/2361510297 wait complete. 2024-05-06T12:15:31.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.388+0000 7f0bb2375700 1 Processor -- start 2024-05-06T12:15:31.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.388+0000 7f0bb2375700 1 -- start start 2024-05-06T12:15:31.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.388+0000 7f0bb2375700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bac0fc290 0x7f0bac10c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:15:31.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.388+0000 7f0bb2375700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bac0fcbe0 0x7f0bac10cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:15:31.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.388+0000 7f0bb2375700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac10d3f0 con 0x7f0bac0fcbe0 2024-05-06T12:15:31.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.388+0000 7f0bb2375700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac10dc40 con 0x7f0bac0fc290 2024-05-06T12:15:31.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.388+0000 7f0bab7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bac0fcbe0 0x7f0bac10cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:15:31.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.388+0000 7f0bab7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bac0fcbe0 0x7f0bac10cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44484/0 (socket says 172.21.15.92:44484) 2024-05-06T12:15:31.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.388+0000 7f0bab7fe700 1 -- 172.21.15.92:0/3312142104 learned_addr learned my addr 172.21.15.92:0/3312142104 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:15:31.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.388+0000 7f0babfff700 1 --2- 172.21.15.92:0/3312142104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bac0fc290 0x7f0bac10c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:15:31.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.389+0000 7f0bab7fe700 1 -- 172.21.15.92:0/3312142104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bac0fc290 msgr2=0x7f0bac10c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:15:31.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.389+0000 7f0bab7fe700 1 --2- 172.21.15.92:0/3312142104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bac0fc290 0x7f0bac10c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:31.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.389+0000 7f0bab7fe700 1 -- 172.21.15.92:0/3312142104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b9c005040 con 0x7f0bac0fcbe0 2024-05-06T12:15:31.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.389+0000 7f0bab7fe700 1 --2- 172.21.15.92:0/3312142104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bac0fcbe0 0x7f0bac10cdf0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0010d50 tx=0x7f0ba000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:15:31.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.389+0000 7f0ba8ff9700 1 -- 172.21.15.92:0/3312142104 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ba000ed60 con 0x7f0bac0fcbe0 2024-05-06T12:15:31.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.389+0000 7f0ba8ff9700 1 -- 172.21.15.92:0/3312142104 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ba00090d0 con 0x7f0bac0fcbe0 2024-05-06T12:15:31.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.389+0000 7f0bb2375700 1 -- 172.21.15.92:0/3312142104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bac10dec0 con 0x7f0bac0fcbe0 2024-05-06T12:15:31.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.389+0000 7f0ba8ff9700 1 -- 172.21.15.92:0/3312142104 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ba0012670 con 0x7f0bac0fcbe0 2024-05-06T12:15:31.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.389+0000 7f0bb2375700 1 -- 172.21.15.92:0/3312142104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bac10e400 con 0x7f0bac0fcbe0 2024-05-06T12:15:31.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.391+0000 7f0bb2375700 1 -- 172.21.15.92:0/3312142104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bac05f5c0 con 0x7f0bac0fcbe0 2024-05-06T12:15:31.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.391+0000 7f0ba8ff9700 1 -- 172.21.15.92:0/3312142104 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7f0ba0018030 con 0x7f0bac0fcbe0 2024-05-06T12:15:31.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.391+0000 7f0ba8ff9700 1 --2- 172.21.15.92:0/3312142104 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f0b9406b790 0x7f0b9406dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:15:31.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.392+0000 7f0ba8ff9700 1 -- 172.21.15.92:0/3312142104 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f0ba0016070 con 0x7f0bac0fcbe0 2024-05-06T12:15:31.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.392+0000 7f0babfff700 1 --2- 172.21.15.92:0/3312142104 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f0b9406b790 0x7f0b9406dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:15:31.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.392+0000 7f0babfff700 1 --2- 172.21.15.92:0/3312142104 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f0b9406b790 0x7f0b9406dc40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c004890 tx=0x7f0b9c009e30 comp rx=0 tx=0).ready entity=mgr.14636 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:15:31.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.398+0000 7f0ba8ff9700 1 -- 172.21.15.92:0/3312142104 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f0ba005a450 con 0x7f0bac0fcbe0 2024-05-06T12:15:31.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.781+0000 7f0bb2375700 1 -- 172.21.15.92:0/3312142104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0bac10e670 con 0x7f0bac0fcbe0 2024-05-06T12:15:31.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.782+0000 7f0ba8ff9700 1 -- 172.21.15.92:0/3312142104 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f0ba0059fe0 con 0x7f0bac0fcbe0 2024-05-06T12:15:31.785 INFO:teuthology.orchestra.run.smithi092.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-06T12:15:31.785 INFO:teuthology.orchestra.run.smithi092.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-06T12:15:31.785 INFO:teuthology.orchestra.run.smithi092.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-06T12:15:31.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.785+0000 7f0bb2375700 1 -- 172.21.15.92:0/3312142104 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f0b9406b790 msgr2=0x7f0b9406dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:15:31.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.785+0000 7f0bb2375700 1 --2- 172.21.15.92:0/3312142104 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f0b9406b790 0x7f0b9406dc40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c004890 tx=0x7f0b9c009e30 comp rx=0 tx=0).stop 2024-05-06T12:15:31.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.785+0000 7f0bb2375700 1 -- 172.21.15.92:0/3312142104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bac0fcbe0 msgr2=0x7f0bac10cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:15:31.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.785+0000 7f0bb2375700 1 --2- 172.21.15.92:0/3312142104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bac0fcbe0 0x7f0bac10cdf0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0010d50 tx=0x7f0ba000b040 comp rx=0 tx=0).stop 2024-05-06T12:15:31.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.785+0000 7f0bb2375700 1 -- 172.21.15.92:0/3312142104 shutdown_connections 2024-05-06T12:15:31.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.785+0000 7f0bb2375700 1 --2- 172.21.15.92:0/3312142104 >> [v2:172.21.15.190:6828/3484477933,v1:172.21.15.190:6829/3484477933] conn(0x7f0b9406b790 0x7f0b9406dc40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:31.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.785+0000 7f0bb2375700 1 --2- 172.21.15.92:0/3312142104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bac0fc290 0x7f0bac10c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:31.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.785+0000 7f0bb2375700 1 --2- 172.21.15.92:0/3312142104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bac0fcbe0 0x7f0bac10cdf0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:15:31.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.785+0000 7f0bb2375700 1 -- 172.21.15.92:0/3312142104 >> 172.21.15.92:0/3312142104 conn(0x7f0bac0f6b80 msgr2=0x7f0bac0f8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:15:31.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.785+0000 7f0bb2375700 1 -- 172.21.15.92:0/3312142104 shutdown_connections 2024-05-06T12:15:31.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:15:31.785+0000 7f0bb2375700 1 -- 172.21.15.92:0/3312142104 wait complete. 2024-05-06T12:15:31.963 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:31 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/3312142104' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:15:32.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:31 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/3312142104' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:15:33.075 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:32 smithi092 ceph-mon[26954]: from='client.14740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:15:33.075 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:32 smithi092 ceph-mon[26954]: pgmap v48: 65 pgs: 65 active+clean; 301 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 152 op/s 2024-05-06T12:15:33.075 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:32 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:15:33.075 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:32 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:33.075 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:32 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:33.075 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:32 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:15:33.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:32 smithi190 ceph-mon[33798]: from='client.14740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:15:33.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:32 smithi190 ceph-mon[33798]: pgmap v48: 65 pgs: 65 active+clean; 301 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 152 op/s 2024-05-06T12:15:33.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:32 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:15:33.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:32 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:33.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:32 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:33.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:32 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:15:34.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:34 smithi092 ceph-mon[26954]: pgmap v49: 65 pgs: 65 active+clean; 316 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 207 op/s 2024-05-06T12:15:34.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:34 smithi092 ceph-mon[26954]: mgrmap e26: smithi190.kczxkh(active, since 92s), standbys: smithi092.gyzeqb 2024-05-06T12:15:35.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:34 smithi190 ceph-mon[33798]: pgmap v49: 65 pgs: 65 active+clean; 316 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 207 op/s 2024-05-06T12:15:35.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:34 smithi190 ceph-mon[33798]: mgrmap e26: smithi190.kczxkh(active, since 92s), standbys: smithi092.gyzeqb 2024-05-06T12:15:37.071 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:36 smithi092 ceph-mon[26954]: pgmap v50: 65 pgs: 65 active+clean; 316 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 143 op/s 2024-05-06T12:15:37.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:36 smithi190 ceph-mon[33798]: pgmap v50: 65 pgs: 65 active+clean; 316 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 143 op/s 2024-05-06T12:15:38.942 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:38 smithi092 ceph-mon[26954]: pgmap v51: 65 pgs: 65 active+clean; 319 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 168 op/s 2024-05-06T12:15:39.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:38 smithi190 ceph-mon[33798]: pgmap v51: 65 pgs: 65 active+clean; 319 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 168 op/s 2024-05-06T12:15:40.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:40 smithi092 ceph-mon[26954]: pgmap v52: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 165 op/s 2024-05-06T12:15:40.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:40 smithi092 ceph-mon[26954]: Standby manager daemon smithi092.gyzeqb restarted 2024-05-06T12:15:40.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:40 smithi092 ceph-mon[26954]: Standby manager daemon smithi092.gyzeqb started 2024-05-06T12:15:40.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:40 smithi092 ceph-mon[26954]: from='mgr.? 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi092.gyzeqb/crt"}]: dispatch 2024-05-06T12:15:40.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:40 smithi092 ceph-mon[26954]: from='mgr.? 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-06T12:15:40.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:40 smithi092 ceph-mon[26954]: from='mgr.? 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi092.gyzeqb/key"}]: dispatch 2024-05-06T12:15:40.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:40 smithi092 ceph-mon[26954]: from='mgr.? 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-06T12:15:41.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:40 smithi190 ceph-mon[33798]: pgmap v52: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 165 op/s 2024-05-06T12:15:41.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:40 smithi190 ceph-mon[33798]: Standby manager daemon smithi092.gyzeqb restarted 2024-05-06T12:15:41.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:40 smithi190 ceph-mon[33798]: Standby manager daemon smithi092.gyzeqb started 2024-05-06T12:15:41.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:40 smithi190 ceph-mon[33798]: from='mgr.? 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi092.gyzeqb/crt"}]: dispatch 2024-05-06T12:15:41.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:40 smithi190 ceph-mon[33798]: from='mgr.? 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-06T12:15:41.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:40 smithi190 ceph-mon[33798]: from='mgr.? 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi092.gyzeqb/key"}]: dispatch 2024-05-06T12:15:41.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:40 smithi190 ceph-mon[33798]: from='mgr.? 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-06T12:15:42.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:41 smithi190 ceph-mon[33798]: mgrmap e27: smithi190.kczxkh(active, since 99s), standbys: smithi092.gyzeqb 2024-05-06T12:15:42.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:41 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:42.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:41 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:42.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:41 smithi092 ceph-mon[26954]: mgrmap e27: smithi190.kczxkh(active, since 99s), standbys: smithi092.gyzeqb 2024-05-06T12:15:42.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:41 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:42.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:41 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:43.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:42 smithi190 ceph-mon[33798]: pgmap v53: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 128 op/s 2024-05-06T12:15:43.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:42 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:43.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:42 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:43.209 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:42 smithi092 ceph-mon[26954]: pgmap v53: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 128 op/s 2024-05-06T12:15:43.209 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:42 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:43.209 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:42 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:44.860 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:44 smithi092 ceph-mon[26954]: pgmap v54: 65 pgs: 65 active+clean; 317 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 198 op/s 2024-05-06T12:15:44.860 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:44 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:44.860 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:44 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:45.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:44 smithi190 ceph-mon[33798]: pgmap v54: 65 pgs: 65 active+clean; 317 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 198 op/s 2024-05-06T12:15:45.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:44 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:45.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:44 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:47.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:46 smithi190 ceph-mon[33798]: pgmap v55: 65 pgs: 65 active+clean; 317 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 142 op/s 2024-05-06T12:15:47.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:46 smithi092 ceph-mon[26954]: pgmap v55: 65 pgs: 65 active+clean; 317 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 142 op/s 2024-05-06T12:15:48.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:48.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:48.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:15:48.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:15:48.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:15:48.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:48.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:15:48.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "mgr fail", "who": "smithi190.kczxkh"}]: dispatch 2024-05-06T12:15:48.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "mgr fail", "who": "smithi190.kczxkh"}]: dispatch 2024-05-06T12:15:48.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: Activating manager daemon smithi092.gyzeqb 2024-05-06T12:15:48.101 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: osdmap e49: 6 total, 6 up, 6 in 2024-05-06T12:15:48.101 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' cmd='[{"prefix": "mgr fail", "who": "smithi190.kczxkh"}]': finished 2024-05-06T12:15:48.101 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: mgrmap e28: smithi092.gyzeqb(active, starting, since 0.00553842s) 2024-05-06T12:15:48.101 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi092"}]: dispatch 2024-05-06T12:15:48.101 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi190"}]: dispatch 2024-05-06T12:15:48.101 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi092.rosbll"}]: dispatch 2024-05-06T12:15:48.101 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi092.ijsqgp"}]: dispatch 2024-05-06T12:15:48.101 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi190.byjvgf"}]: dispatch 2024-05-06T12:15:48.102 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi190.idzoff"}]: dispatch 2024-05-06T12:15:48.102 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mgr metadata", "who": "smithi092.gyzeqb", "id": "smithi092.gyzeqb"}]: dispatch 2024-05-06T12:15:48.102 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-06T12:15:48.102 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-06T12:15:48.102 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-06T12:15:48.102 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-06T12:15:48.102 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-06T12:15:48.102 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-06T12:15:48.103 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-06T12:15:48.103 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-06T12:15:48.103 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-06T12:15:48.103 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: Manager daemon smithi092.gyzeqb is now available 2024-05-06T12:15:48.103 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:15:48.103 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:15:48.103 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi092.gyzeqb/mirror_snapshot_schedule"}]: dispatch 2024-05-06T12:15:48.103 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:47 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi092.gyzeqb/trash_purge_schedule"}]: dispatch 2024-05-06T12:15:48.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:48.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:48.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:15:48.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:15:48.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:15:48.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' 2024-05-06T12:15:48.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:15:48.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: from='mgr.14636 172.21.15.190:0/473447146' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "mgr fail", "who": "smithi190.kczxkh"}]: dispatch 2024-05-06T12:15:48.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "mgr fail", "who": "smithi190.kczxkh"}]: dispatch 2024-05-06T12:15:48.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: Activating manager daemon smithi092.gyzeqb 2024-05-06T12:15:48.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: osdmap e49: 6 total, 6 up, 6 in 2024-05-06T12:15:48.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: from='mgr.14636 ' entity='mgr.smithi190.kczxkh' cmd='[{"prefix": "mgr fail", "who": "smithi190.kczxkh"}]': finished 2024-05-06T12:15:48.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: mgrmap e28: smithi092.gyzeqb(active, starting, since 0.00553842s) 2024-05-06T12:15:48.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi092"}]: dispatch 2024-05-06T12:15:48.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi190"}]: dispatch 2024-05-06T12:15:48.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi092.rosbll"}]: dispatch 2024-05-06T12:15:48.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi092.ijsqgp"}]: dispatch 2024-05-06T12:15:48.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi190.byjvgf"}]: dispatch 2024-05-06T12:15:48.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi190.idzoff"}]: dispatch 2024-05-06T12:15:48.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mgr metadata", "who": "smithi092.gyzeqb", "id": "smithi092.gyzeqb"}]: dispatch 2024-05-06T12:15:48.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-06T12:15:48.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-06T12:15:48.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-06T12:15:48.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-06T12:15:48.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-06T12:15:48.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-06T12:15:48.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-06T12:15:48.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-06T12:15:48.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-06T12:15:48.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: Manager daemon smithi092.gyzeqb is now available 2024-05-06T12:15:48.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:15:48.226 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:15:48.226 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi092.gyzeqb/mirror_snapshot_schedule"}]: dispatch 2024-05-06T12:15:48.226 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:47 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi092.gyzeqb/trash_purge_schedule"}]: dispatch 2024-05-06T12:15:49.097 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:48 smithi092 ceph-mon[26954]: [06/May/2024:12:15:47] ENGINE Bus STARTING 2024-05-06T12:15:49.097 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:48 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:15:49.097 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:48 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:15:49.097 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:48 smithi092 ceph-mon[26954]: mgrmap e29: smithi092.gyzeqb(active, since 1.0095s) 2024-05-06T12:15:49.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:48 smithi190 ceph-mon[33798]: [06/May/2024:12:15:47] ENGINE Bus STARTING 2024-05-06T12:15:49.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:48 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:15:49.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:48 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:15:49.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:48 smithi190 ceph-mon[33798]: mgrmap e29: smithi092.gyzeqb(active, since 1.0095s) 2024-05-06T12:15:50.060 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:49 smithi092 ceph-mon[26954]: [06/May/2024:12:15:48] ENGINE Serving on https://172.21.15.92:7150 2024-05-06T12:15:50.060 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:49 smithi092 ceph-mon[26954]: [06/May/2024:12:15:48] ENGINE Serving on http://172.21.15.92:8765 2024-05-06T12:15:50.060 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:49 smithi092 ceph-mon[26954]: [06/May/2024:12:15:48] ENGINE Bus STARTED 2024-05-06T12:15:50.060 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:49 smithi092 ceph-mon[26954]: pgmap v3: 65 pgs: 65 active+clean; 321 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-05-06T12:15:50.060 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:49 smithi092 ceph-mon[26954]: mgrmap e30: smithi092.gyzeqb(active, since 2s) 2024-05-06T12:15:50.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:49 smithi190 ceph-mon[33798]: [06/May/2024:12:15:48] ENGINE Serving on https://172.21.15.92:7150 2024-05-06T12:15:50.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:49 smithi190 ceph-mon[33798]: [06/May/2024:12:15:48] ENGINE Serving on http://172.21.15.92:8765 2024-05-06T12:15:50.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:49 smithi190 ceph-mon[33798]: [06/May/2024:12:15:48] ENGINE Bus STARTED 2024-05-06T12:15:50.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:49 smithi190 ceph-mon[33798]: pgmap v3: 65 pgs: 65 active+clean; 321 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-05-06T12:15:50.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:49 smithi190 ceph-mon[33798]: mgrmap e30: smithi092.gyzeqb(active, since 2s) 2024-05-06T12:15:51.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:50 smithi190 ceph-mon[33798]: pgmap v4: 65 pgs: 65 active+clean; 321 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-05-06T12:15:51.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:50 smithi092 ceph-mon[26954]: pgmap v4: 65 pgs: 65 active+clean; 321 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-05-06T12:15:52.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:52 smithi092 ceph-mon[26954]: pgmap v5: 65 pgs: 65 active+clean; 321 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-05-06T12:15:52.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:52 smithi092 ceph-mon[26954]: mgrmap e31: smithi092.gyzeqb(active, since 4s) 2024-05-06T12:15:52.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:52 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:15:52.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:52 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:15:52.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:52 smithi190 ceph-mon[33798]: pgmap v5: 65 pgs: 65 active+clean; 321 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-05-06T12:15:52.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:52 smithi190 ceph-mon[33798]: mgrmap e31: smithi092.gyzeqb(active, since 4s) 2024-05-06T12:15:52.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:52 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:15:52.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:52 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:15:54.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:53 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:15:54.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:53 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:15:54.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:53 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:15:54.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:53 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:15:54.321 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:53 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:15:54.321 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:53 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:15:54.321 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:53 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:15:54.321 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:53 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:15:55.031 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:54 smithi190 ceph-mon[33798]: pgmap v6: 65 pgs: 65 active+clean; 321 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-05-06T12:15:55.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:54 smithi092 ceph-mon[26954]: pgmap v6: 65 pgs: 65 active+clean; 321 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-05-06T12:15:56.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:55 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:15:56.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:55 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:15:56.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:55 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:15:56.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:55 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:15:57.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:57 smithi092 ceph-mon[26954]: pgmap v7: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 97 op/s 2024-05-06T12:15:57.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:57 smithi092 ceph-mon[26954]: Standby manager daemon smithi190.kczxkh started 2024-05-06T12:15:57.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:57 smithi092 ceph-mon[26954]: from='mgr.? 172.21.15.190:0/1010967832' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi190.kczxkh/crt"}]: dispatch 2024-05-06T12:15:57.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:57 smithi092 ceph-mon[26954]: from='mgr.? 172.21.15.190:0/1010967832' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-06T12:15:57.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:57 smithi092 ceph-mon[26954]: from='mgr.? 172.21.15.190:0/1010967832' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi190.kczxkh/key"}]: dispatch 2024-05-06T12:15:57.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:57 smithi092 ceph-mon[26954]: from='mgr.? 172.21.15.190:0/1010967832' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-06T12:15:57.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:57 smithi190 ceph-mon[33798]: pgmap v7: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 97 op/s 2024-05-06T12:15:57.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:57 smithi190 ceph-mon[33798]: Standby manager daemon smithi190.kczxkh started 2024-05-06T12:15:57.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:57 smithi190 ceph-mon[33798]: from='mgr.? 172.21.15.190:0/1010967832' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi190.kczxkh/crt"}]: dispatch 2024-05-06T12:15:57.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:57 smithi190 ceph-mon[33798]: from='mgr.? 172.21.15.190:0/1010967832' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-06T12:15:57.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:57 smithi190 ceph-mon[33798]: from='mgr.? 172.21.15.190:0/1010967832' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi190.kczxkh/key"}]: dispatch 2024-05-06T12:15:57.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:57 smithi190 ceph-mon[33798]: from='mgr.? 172.21.15.190:0/1010967832' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-06T12:15:58.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:58 smithi190 ceph-mon[33798]: mgrmap e32: smithi092.gyzeqb(active, since 9s), standbys: smithi190.kczxkh 2024-05-06T12:15:58.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:58 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mgr metadata", "who": "smithi190.kczxkh", "id": "smithi190.kczxkh"}]: dispatch 2024-05-06T12:15:58.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:15:58 smithi190 ceph-mon[33798]: pgmap v8: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 933 KiB/s rd, 837 KiB/s wr, 76 op/s 2024-05-06T12:15:58.369 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:58 smithi092 ceph-mon[26954]: mgrmap e32: smithi092.gyzeqb(active, since 9s), standbys: smithi190.kczxkh 2024-05-06T12:15:58.369 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:58 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mgr metadata", "who": "smithi190.kczxkh", "id": "smithi190.kczxkh"}]: dispatch 2024-05-06T12:15:58.369 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:15:58 smithi092 ceph-mon[26954]: pgmap v8: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 933 KiB/s rd, 837 KiB/s wr, 76 op/s 2024-05-06T12:16:00.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:00 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:00.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:00 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:00.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:00 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:16:00.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:00 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:16:00.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:00 smithi190 ceph-mon[33798]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-06T12:16:00.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:00 smithi190 ceph-mon[33798]: Updating smithi190:/etc/ceph/ceph.conf 2024-05-06T12:16:00.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:00 smithi190 ceph-mon[33798]: pgmap v9: 65 pgs: 65 active+clean; 329 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-05-06T12:16:00.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:00 smithi190 ceph-mon[33798]: Updating smithi190:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-06T12:16:00.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:00 smithi190 ceph-mon[33798]: Updating smithi092:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-06T12:16:00.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:00 smithi190 ceph-mon[33798]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-05-06T12:16:00.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:00 smithi190 ceph-mon[33798]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-05-06T12:16:00.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:00 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:00.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:00 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:00.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:00 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:16:00.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:00 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:16:00.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:00 smithi092 ceph-mon[26954]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-06T12:16:00.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:00 smithi092 ceph-mon[26954]: Updating smithi190:/etc/ceph/ceph.conf 2024-05-06T12:16:00.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:00 smithi092 ceph-mon[26954]: pgmap v9: 65 pgs: 65 active+clean; 329 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-05-06T12:16:00.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:00 smithi092 ceph-mon[26954]: Updating smithi190:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-06T12:16:00.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:00 smithi092 ceph-mon[26954]: Updating smithi092:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-06T12:16:00.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:00 smithi092 ceph-mon[26954]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-05-06T12:16:00.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:00 smithi092 ceph-mon[26954]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-05-06T12:16:01.679 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:01 smithi092 ceph-mon[26954]: Updating smithi190:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-06T12:16:01.679 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:01 smithi092 ceph-mon[26954]: Updating smithi092:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-06T12:16:01.680 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:01 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:01.680 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:01 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:01.680 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:01 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:01.680 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:01 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:01.680 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:01 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:01.680 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:01 smithi092 ceph-mon[26954]: Reconfiguring prometheus.smithi092 (dependencies changed)... 2024-05-06T12:16:01.680 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:01 smithi092 ceph-mon[26954]: Reconfiguring daemon prometheus.smithi092 on smithi092 2024-05-06T12:16:01.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:01 smithi190 ceph-mon[33798]: Updating smithi190:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-06T12:16:01.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:01 smithi190 ceph-mon[33798]: Updating smithi092:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-06T12:16:01.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:01 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:01.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:01 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:01.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:01 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:01.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:01 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:01.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:01 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:01.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:01 smithi190 ceph-mon[33798]: Reconfiguring prometheus.smithi092 (dependencies changed)... 2024-05-06T12:16:01.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:01 smithi190 ceph-mon[33798]: Reconfiguring daemon prometheus.smithi092 on smithi092 2024-05-06T12:16:01.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.918+0000 7f6f47474700 1 -- 172.21.15.92:0/1211208594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f400fed30 msgr2=0x7f6f400fae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:01.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.918+0000 7f6f47474700 1 --2- 172.21.15.92:0/1211208594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f400fed30 0x7f6f400fae70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6f30004940 tx=0x7f6f30009e30 comp rx=0 tx=0).stop 2024-05-06T12:16:01.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.918+0000 7f6f47474700 1 -- 172.21.15.92:0/1211208594 shutdown_connections 2024-05-06T12:16:01.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.918+0000 7f6f47474700 1 --2- 172.21.15.92:0/1211208594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f400fed30 0x7f6f400fae70 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:01.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.918+0000 7f6f47474700 1 --2- 172.21.15.92:0/1211208594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f400fe3e0 0x7f6f400fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:01.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.918+0000 7f6f47474700 1 -- 172.21.15.92:0/1211208594 >> 172.21.15.92:0/1211208594 conn(0x7f6f400f6b80 msgr2=0x7f6f400f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:16:01.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.918+0000 7f6f47474700 1 -- 172.21.15.92:0/1211208594 shutdown_connections 2024-05-06T12:16:01.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.918+0000 7f6f47474700 1 -- 172.21.15.92:0/1211208594 wait complete. 2024-05-06T12:16:01.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.918+0000 7f6f47474700 1 Processor -- start 2024-05-06T12:16:01.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.919+0000 7f6f47474700 1 -- start start 2024-05-06T12:16:01.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.919+0000 7f6f47474700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f400fe3e0 0x7f6f400fdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:01.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.919+0000 7f6f47474700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f400fed30 0x7f6f400fc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:01.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.919+0000 7f6f47474700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f400fc650 con 0x7f6f400fe3e0 2024-05-06T12:16:01.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.919+0000 7f6f47474700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f400fc7c0 con 0x7f6f400fed30 2024-05-06T12:16:01.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.919+0000 7f6f45210700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f400fe3e0 0x7f6f400fdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:01.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.919+0000 7f6f45210700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f400fe3e0 0x7f6f400fdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46272/0 (socket says 172.21.15.92:46272) 2024-05-06T12:16:01.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.919+0000 7f6f45210700 1 -- 172.21.15.92:0/770323790 learned_addr learned my addr 172.21.15.92:0/770323790 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:16:01.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.919+0000 7f6f44a0f700 1 --2- 172.21.15.92:0/770323790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f400fed30 0x7f6f400fc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:01.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.919+0000 7f6f45210700 1 -- 172.21.15.92:0/770323790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f400fed30 msgr2=0x7f6f400fc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:01.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.919+0000 7f6f45210700 1 --2- 172.21.15.92:0/770323790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f400fed30 0x7f6f400fc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:01.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.919+0000 7f6f45210700 1 -- 172.21.15.92:0/770323790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f30005040 con 0x7f6f400fe3e0 2024-05-06T12:16:01.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.919+0000 7f6f45210700 1 --2- 172.21.15.92:0/770323790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f400fe3e0 0x7f6f400fdbd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c002a50 tx=0x7f6f3c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:16:01.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.920+0000 7f6f35ffb700 1 -- 172.21.15.92:0/770323790 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f3c007cc0 con 0x7f6f400fe3e0 2024-05-06T12:16:01.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.920+0000 7f6f47474700 1 -- 172.21.15.92:0/770323790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f400fcaa0 con 0x7f6f400fe3e0 2024-05-06T12:16:01.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.920+0000 7f6f35ffb700 1 -- 172.21.15.92:0/770323790 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6f3c00f040 con 0x7f6f400fe3e0 2024-05-06T12:16:01.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.920+0000 7f6f35ffb700 1 -- 172.21.15.92:0/770323790 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f3c00da30 con 0x7f6f400fe3e0 2024-05-06T12:16:01.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.920+0000 7f6f47474700 1 -- 172.21.15.92:0/770323790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f401159f0 con 0x7f6f400fe3e0 2024-05-06T12:16:01.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.921+0000 7f6f47474700 1 -- 172.21.15.92:0/770323790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f4005f5c0 con 0x7f6f400fe3e0 2024-05-06T12:16:01.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.921+0000 7f6f35ffb700 1 -- 172.21.15.92:0/770323790 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 32) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f6f3c00db90 con 0x7f6f400fe3e0 2024-05-06T12:16:01.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.922+0000 7f6f35ffb700 1 --2- 172.21.15.92:0/770323790 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f6f2c06dc30 0x7f6f2c0700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:01.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.922+0000 7f6f35ffb700 1 -- 172.21.15.92:0/770323790 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f6f3c08d1b0 con 0x7f6f400fe3e0 2024-05-06T12:16:01.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.922+0000 7f6f44a0f700 1 --2- 172.21.15.92:0/770323790 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f6f2c06dc30 0x7f6f2c0700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:01.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.923+0000 7f6f44a0f700 1 --2- 172.21.15.92:0/770323790 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f6f2c06dc30 0x7f6f2c0700e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000b970 tx=0x7f6f30009e30 comp rx=0 tx=0).ready entity=mgr.14748 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:16:01.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:01.929+0000 7f6f35ffb700 1 -- 172.21.15.92:0/770323790 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f6f3c056b40 con 0x7f6f400fe3e0 2024-05-06T12:16:02.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.220+0000 7f6f47474700 1 -- 172.21.15.92:0/770323790 --> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f40102500 con 0x7f6f2c06dc30 2024-05-06T12:16:02.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.221+0000 7f6f35ffb700 1 -- 172.21.15.92:0/770323790 <== mgr.14748 v2:172.21.15.92:6800/648200129 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f6f40102500 con 0x7f6f2c06dc30 2024-05-06T12:16:02.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.224+0000 7f6f47474700 1 -- 172.21.15.92:0/770323790 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f6f2c06dc30 msgr2=0x7f6f2c0700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:02.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.224+0000 7f6f47474700 1 --2- 172.21.15.92:0/770323790 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f6f2c06dc30 0x7f6f2c0700e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000b970 tx=0x7f6f30009e30 comp rx=0 tx=0).stop 2024-05-06T12:16:02.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.224+0000 7f6f47474700 1 -- 172.21.15.92:0/770323790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f400fe3e0 msgr2=0x7f6f400fdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:02.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.224+0000 7f6f47474700 1 --2- 172.21.15.92:0/770323790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f400fe3e0 0x7f6f400fdbd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c002a50 tx=0x7f6f3c0046b0 comp rx=0 tx=0).stop 2024-05-06T12:16:02.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.224+0000 7f6f47474700 1 -- 172.21.15.92:0/770323790 shutdown_connections 2024-05-06T12:16:02.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.224+0000 7f6f47474700 1 --2- 172.21.15.92:0/770323790 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f6f2c06dc30 0x7f6f2c0700e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:02.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.224+0000 7f6f47474700 1 --2- 172.21.15.92:0/770323790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f400fe3e0 0x7f6f400fdbd0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:02.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.224+0000 7f6f47474700 1 --2- 172.21.15.92:0/770323790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f400fed30 0x7f6f400fc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:02.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.224+0000 7f6f47474700 1 -- 172.21.15.92:0/770323790 >> 172.21.15.92:0/770323790 conn(0x7f6f400f6b80 msgr2=0x7f6f400f8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:16:02.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.224+0000 7f6f47474700 1 -- 172.21.15.92:0/770323790 shutdown_connections 2024-05-06T12:16:02.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.225+0000 7f6f47474700 1 -- 172.21.15.92:0/770323790 wait complete. 2024-05-06T12:16:02.240 INFO:teuthology.orchestra.run.smithi092.stdout:true 2024-05-06T12:16:02.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.371+0000 7fa3685b9700 1 -- 172.21.15.92:0/4269914819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3600fff40 msgr2=0x7fa360102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:02.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.371+0000 7fa3685b9700 1 --2- 172.21.15.92:0/4269914819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3600fff40 0x7fa360102320 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa350004940 tx=0x7fa350009e30 comp rx=0 tx=0).stop 2024-05-06T12:16:02.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.371+0000 7fa3685b9700 1 -- 172.21.15.92:0/4269914819 shutdown_connections 2024-05-06T12:16:02.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.371+0000 7fa3685b9700 1 --2- 172.21.15.92:0/4269914819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3600fff40 0x7fa360102320 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:02.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.371+0000 7fa3685b9700 1 --2- 172.21.15.92:0/4269914819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa360066260 0x7fa3600ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:02.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.371+0000 7fa3685b9700 1 -- 172.21.15.92:0/4269914819 >> 172.21.15.92:0/4269914819 conn(0x7fa3600f7430 msgr2=0x7fa3600f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:16:02.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.371+0000 7fa3685b9700 1 -- 172.21.15.92:0/4269914819 shutdown_connections 2024-05-06T12:16:02.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.371+0000 7fa3685b9700 1 -- 172.21.15.92:0/4269914819 wait complete. 2024-05-06T12:16:02.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.372+0000 7fa3685b9700 1 Processor -- start 2024-05-06T12:16:02.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.372+0000 7fa3685b9700 1 -- start start 2024-05-06T12:16:02.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.372+0000 7fa3685b9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa360066260 0x7fa360113fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:02.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.372+0000 7fa3685b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3600fff40 0x7fa360114510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:02.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.372+0000 7fa3685b9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa360114b10 con 0x7fa360066260 2024-05-06T12:16:02.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.372+0000 7fa3685b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa360115310 con 0x7fa3600fff40 2024-05-06T12:16:02.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.372+0000 7fa366355700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa360066260 0x7fa360113fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:02.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.372+0000 7fa366355700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa360066260 0x7fa360113fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46302/0 (socket says 172.21.15.92:46302) 2024-05-06T12:16:02.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.372+0000 7fa366355700 1 -- 172.21.15.92:0/2670337837 learned_addr learned my addr 172.21.15.92:0/2670337837 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:16:02.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.372+0000 7fa365b54700 1 --2- 172.21.15.92:0/2670337837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3600fff40 0x7fa360114510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:02.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.373+0000 7fa366355700 1 -- 172.21.15.92:0/2670337837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3600fff40 msgr2=0x7fa360114510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:02.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.373+0000 7fa366355700 1 --2- 172.21.15.92:0/2670337837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3600fff40 0x7fa360114510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:02.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.373+0000 7fa366355700 1 -- 172.21.15.92:0/2670337837 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35c005040 con 0x7fa360066260 2024-05-06T12:16:02.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.373+0000 7fa366355700 1 --2- 172.21.15.92:0/2670337837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa360066260 0x7fa360113fd0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa35c0096d0 tx=0x7fa35c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:16:02.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.373+0000 7fa356ffd700 1 -- 172.21.15.92:0/2670337837 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa35c00e7e0 con 0x7fa360066260 2024-05-06T12:16:02.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.373+0000 7fa356ffd700 1 -- 172.21.15.92:0/2670337837 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa35c00f040 con 0x7fa360066260 2024-05-06T12:16:02.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.373+0000 7fa356ffd700 1 -- 172.21.15.92:0/2670337837 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa35c013d40 con 0x7fa360066260 2024-05-06T12:16:02.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.373+0000 7fa3685b9700 1 -- 172.21.15.92:0/2670337837 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa350005040 con 0x7fa360066260 2024-05-06T12:16:02.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.373+0000 7fa3685b9700 1 -- 172.21.15.92:0/2670337837 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa360115940 con 0x7fa360066260 2024-05-06T12:16:02.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.374+0000 7fa3685b9700 1 -- 172.21.15.92:0/2670337837 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa36005f5c0 con 0x7fa360066260 2024-05-06T12:16:02.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.376+0000 7fa356ffd700 1 -- 172.21.15.92:0/2670337837 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 32) v1 ==== 92027+0+0 (secure 0 0 0) 0x7fa35c00e940 con 0x7fa360066260 2024-05-06T12:16:02.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.376+0000 7fa356ffd700 1 --2- 172.21.15.92:0/2670337837 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7fa34c06db60 0x7fa34c070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:02.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.376+0000 7fa356ffd700 1 -- 172.21.15.92:0/2670337837 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fa35c08dd40 con 0x7fa360066260 2024-05-06T12:16:02.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.377+0000 7fa365b54700 1 --2- 172.21.15.92:0/2670337837 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7fa34c06db60 0x7fa34c070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:02.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.377+0000 7fa365b54700 1 --2- 172.21.15.92:0/2670337837 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7fa34c06db60 0x7fa34c070010 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa35000b0d0 tx=0x7fa350009e30 comp rx=0 tx=0).ready entity=mgr.14748 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:16:02.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.383+0000 7fa356ffd700 1 -- 172.21.15.92:0/2670337837 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fa35c05b340 con 0x7fa360066260 2024-05-06T12:16:02.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.708+0000 7fa3685b9700 1 -- 172.21.15.92:0/2670337837 --> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa360115d80 con 0x7fa34c06db60 2024-05-06T12:16:02.711 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:02 smithi092 ceph-mon[26954]: pgmap v10: 65 pgs: 65 active+clean; 329 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 123 op/s 2024-05-06T12:16:02.711 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:02 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:16:02.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.709+0000 7fa356ffd700 1 -- 172.21.15.92:0/2670337837 <== mgr.14748 v2:172.21.15.92:6800/648200129 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fa360115d80 con 0x7fa34c06db60 2024-05-06T12:16:02.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.712+0000 7fa3685b9700 1 -- 172.21.15.92:0/2670337837 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7fa34c06db60 msgr2=0x7fa34c070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:02.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.712+0000 7fa3685b9700 1 --2- 172.21.15.92:0/2670337837 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7fa34c06db60 0x7fa34c070010 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa35000b0d0 tx=0x7fa350009e30 comp rx=0 tx=0).stop 2024-05-06T12:16:02.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.712+0000 7fa3685b9700 1 -- 172.21.15.92:0/2670337837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa360066260 msgr2=0x7fa360113fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:02.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.712+0000 7fa3685b9700 1 --2- 172.21.15.92:0/2670337837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa360066260 0x7fa360113fd0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa35c0096d0 tx=0x7fa35c009e30 comp rx=0 tx=0).stop 2024-05-06T12:16:02.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.712+0000 7fa3685b9700 1 -- 172.21.15.92:0/2670337837 shutdown_connections 2024-05-06T12:16:02.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.712+0000 7fa3685b9700 1 --2- 172.21.15.92:0/2670337837 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7fa34c06db60 0x7fa34c070010 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:02.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.712+0000 7fa3685b9700 1 --2- 172.21.15.92:0/2670337837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa360066260 0x7fa360113fd0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:02.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.712+0000 7fa3685b9700 1 --2- 172.21.15.92:0/2670337837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3600fff40 0x7fa360114510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:02.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.712+0000 7fa3685b9700 1 -- 172.21.15.92:0/2670337837 >> 172.21.15.92:0/2670337837 conn(0x7fa3600f7430 msgr2=0x7fa3600f9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:16:02.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.712+0000 7fa3685b9700 1 -- 172.21.15.92:0/2670337837 shutdown_connections 2024-05-06T12:16:02.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.712+0000 7fa3685b9700 1 -- 172.21.15.92:0/2670337837 wait complete. 2024-05-06T12:16:02.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.843+0000 7f5f63516700 1 -- 172.21.15.92:0/4046358579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f5c0662c0 msgr2=0x7f5f5c0ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:02.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.843+0000 7f5f63516700 1 --2- 172.21.15.92:0/4046358579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f5c0662c0 0x7f5f5c0ff9a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5f58004940 tx=0x7f5f58009e30 comp rx=0 tx=0).stop 2024-05-06T12:16:02.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.843+0000 7f5f63516700 1 -- 172.21.15.92:0/4046358579 shutdown_connections 2024-05-06T12:16:02.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.843+0000 7f5f63516700 1 --2- 172.21.15.92:0/4046358579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f5c0ffee0 0x7f5f5c102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:02.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.843+0000 7f5f63516700 1 --2- 172.21.15.92:0/4046358579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f5c0662c0 0x7f5f5c0ff9a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:02.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.843+0000 7f5f63516700 1 -- 172.21.15.92:0/4046358579 >> 172.21.15.92:0/4046358579 conn(0x7f5f5c0f7430 msgr2=0x7f5f5c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:16:02.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.843+0000 7f5f63516700 1 -- 172.21.15.92:0/4046358579 shutdown_connections 2024-05-06T12:16:02.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.843+0000 7f5f63516700 1 -- 172.21.15.92:0/4046358579 wait complete. 2024-05-06T12:16:02.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.844+0000 7f5f63516700 1 Processor -- start 2024-05-06T12:16:02.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.844+0000 7f5f63516700 1 -- start start 2024-05-06T12:16:02.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.844+0000 7f5f63516700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f5c0662c0 0x7f5f5c10da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:02.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.844+0000 7f5f63516700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f5c0ffee0 0x7f5f5c10df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:02.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.844+0000 7f5f63516700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c10e560 con 0x7f5f5c0662c0 2024-05-06T12:16:02.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.844+0000 7f5f63516700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c10edb0 con 0x7f5f5c0ffee0 2024-05-06T12:16:02.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.844+0000 7f5f612b2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f5c0662c0 0x7f5f5c10da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:02.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.844+0000 7f5f60ab1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f5c0ffee0 0x7f5f5c10df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:02.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.844+0000 7f5f612b2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f5c0662c0 0x7f5f5c10da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47714/0 (socket says 172.21.15.92:47714) 2024-05-06T12:16:02.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.844+0000 7f5f612b2700 1 -- 172.21.15.92:0/3821469756 learned_addr learned my addr 172.21.15.92:0/3821469756 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:16:02.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.844+0000 7f5f612b2700 1 -- 172.21.15.92:0/3821469756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f5c0ffee0 msgr2=0x7f5f5c10df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:02.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.844+0000 7f5f612b2700 1 --2- 172.21.15.92:0/3821469756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f5c0ffee0 0x7f5f5c10df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:02.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.844+0000 7f5f612b2700 1 -- 172.21.15.92:0/3821469756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f58005040 con 0x7f5f5c0662c0 2024-05-06T12:16:02.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.845+0000 7f5f612b2700 1 --2- 172.21.15.92:0/3821469756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f5c0662c0 0x7f5f5c10da20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800b3b0 tx=0x7f5f58009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:16:02.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.845+0000 7f5f51ffb700 1 -- 172.21.15.92:0/3821469756 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f5800e040 con 0x7f5f5c0662c0 2024-05-06T12:16:02.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.845+0000 7f5f63516700 1 -- 172.21.15.92:0/3821469756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f5c10f030 con 0x7f5f5c0662c0 2024-05-06T12:16:02.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:02 smithi190 ceph-mon[33798]: pgmap v10: 65 pgs: 65 active+clean; 329 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 123 op/s 2024-05-06T12:16:02.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:02 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:16:02.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.845+0000 7f5f51ffb700 1 -- 172.21.15.92:0/3821469756 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f5800bbf0 con 0x7f5f5c0662c0 2024-05-06T12:16:02.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.845+0000 7f5f51ffb700 1 -- 172.21.15.92:0/3821469756 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f5801c630 con 0x7f5f5c0662c0 2024-05-06T12:16:02.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.845+0000 7f5f63516700 1 -- 172.21.15.92:0/3821469756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f5c10f570 con 0x7f5f5c0662c0 2024-05-06T12:16:02.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.846+0000 7f5f63516700 1 -- 172.21.15.92:0/3821469756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f5c05f5c0 con 0x7f5f5c0662c0 2024-05-06T12:16:02.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.847+0000 7f5f51ffb700 1 -- 172.21.15.92:0/3821469756 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 32) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f5f5800bd60 con 0x7f5f5c0662c0 2024-05-06T12:16:02.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.848+0000 7f5f51ffb700 1 --2- 172.21.15.92:0/3821469756 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f5f4806db60 0x7f5f48070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:02.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.848+0000 7f5f51ffb700 1 -- 172.21.15.92:0/3821469756 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f5f5808ded0 con 0x7f5f5c0662c0 2024-05-06T12:16:02.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.848+0000 7f5f60ab1700 1 --2- 172.21.15.92:0/3821469756 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f5f4806db60 0x7f5f48070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:02.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.848+0000 7f5f60ab1700 1 --2- 172.21.15.92:0/3821469756 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f5f4806db60 0x7f5f48070010 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c004f40 tx=0x7f5f4c00b040 comp rx=0 tx=0).ready entity=mgr.14748 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:16:02.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:02.854+0000 7f5f51ffb700 1 -- 172.21.15.92:0/3821469756 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f5f5805b4d0 con 0x7f5f5c0662c0 2024-05-06T12:16:03.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.154+0000 7f5f63516700 1 -- 172.21.15.92:0/3821469756 --> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f5c10f910 con 0x7f5f4806db60 2024-05-06T12:16:03.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.165+0000 7f5f51ffb700 1 -- 172.21.15.92:0/3821469756 <== mgr.14748 v2:172.21.15.92:6800/648200129 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f5f5c10f910 con 0x7f5f4806db60 2024-05-06T12:16:03.169 INFO:teuthology.orchestra.run.smithi092.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-06T12:16:03.169 INFO:teuthology.orchestra.run.smithi092.stdout:alertmanager.smithi092 smithi092 *:9093,9094 running (67s) 9s ago 9m 17.0M - 0.25.0 c8568f914cd2 be5637d7d25a 2024-05-06T12:16:03.169 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi092 smithi092 running (9m) 9s ago 9m 6660k - 17.2.7-925-ge81df420 96d5ae7fd4b4 95dc7e14bd9a 2024-05-06T12:16:03.169 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi190 smithi190 running (8m) 11s ago 8m 6665k - 17.2.7-925-ge81df420 96d5ae7fd4b4 6cf97785406f 2024-05-06T12:16:03.169 INFO:teuthology.orchestra.run.smithi092.stdout:grafana.smithi092 smithi092 *:3000 running (61s) 9s ago 9m 76.2M - 9.4.7 954c08fa6188 9180cb3e5258 2024-05-06T12:16:03.169 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.ijsqgp smithi092 running (4m) 9s ago 4m 68.2M - 17.2.7-925-ge81df420 96d5ae7fd4b4 b4c2ed6d2c82 2024-05-06T12:16:03.169 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.rosbll smithi092 running (4m) 9s ago 4m 137M - 17.2.7-925-ge81df420 96d5ae7fd4b4 5f5426cd1bf5 2024-05-06T12:16:03.169 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.byjvgf smithi190 running (4m) 11s ago 4m 167M - 17.2.7-925-ge81df420 96d5ae7fd4b4 24581d2e1c5d 2024-05-06T12:16:03.169 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.idzoff smithi190 running (4m) 11s ago 4m 142M - 17.2.7-925-ge81df420 96d5ae7fd4b4 20f2641696ef 2024-05-06T12:16:03.169 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi092.gyzeqb smithi092 *:8443,9283,8765 running (31s) 9s ago 11m 479M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 de9d2198327a 2024-05-06T12:16:03.170 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi190.kczxkh smithi190 *:8443,9283 running (2m) 11s ago 8m 68.9M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 efed230a6048 2024-05-06T12:16:03.170 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi092 smithi092 running (11m) 9s ago 11m 53.8M 2048M 17.2.7-925-ge81df420 96d5ae7fd4b4 df5b322e9719 2024-05-06T12:16:03.170 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi190 smithi190 running (8m) 11s ago 8m 43.9M 2048M 17.2.7-925-ge81df420 96d5ae7fd4b4 912ded072601 2024-05-06T12:16:03.170 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi092 smithi092 *:9100 running (74s) 9s ago 9m 8048k - 1.5.0 0da6a335fe13 c796446b206f 2024-05-06T12:16:03.170 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi190 smithi190 *:9100 running (53s) 11s ago 8m 8539k - 1.5.0 0da6a335fe13 647e28111a76 2024-05-06T12:16:03.170 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 smithi092 running (7m) 9s ago 7m 185M 1557M 17.2.7-925-ge81df420 96d5ae7fd4b4 ce84e7af9084 2024-05-06T12:16:03.170 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 smithi092 running (7m) 9s ago 7m 190M 1557M 17.2.7-925-ge81df420 96d5ae7fd4b4 8ab870f57eb2 2024-05-06T12:16:03.170 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 smithi092 running (6m) 9s ago 6m 154M 1557M 17.2.7-925-ge81df420 96d5ae7fd4b4 1e6ed04a0688 2024-05-06T12:16:03.170 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 smithi190 running (6m) 11s ago 6m 207M 2581M 17.2.7-925-ge81df420 96d5ae7fd4b4 52f9521286d5 2024-05-06T12:16:03.170 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 smithi190 running (6m) 11s ago 6m 189M 2581M 17.2.7-925-ge81df420 96d5ae7fd4b4 3a7da09e4c48 2024-05-06T12:16:03.170 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 smithi190 running (5m) 11s ago 5m 179M 2581M 17.2.7-925-ge81df420 96d5ae7fd4b4 e7452c6b5cec 2024-05-06T12:16:03.170 INFO:teuthology.orchestra.run.smithi092.stdout:prometheus.smithi092 smithi092 *:9095 running (55s) 9s ago 9m 44.8M - 2.43.0 a07b618ecd1d 330febe55f5c 2024-05-06T12:16:03.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.168+0000 7f5f63516700 1 -- 172.21.15.92:0/3821469756 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f5f4806db60 msgr2=0x7f5f48070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:03.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.168+0000 7f5f63516700 1 --2- 172.21.15.92:0/3821469756 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f5f4806db60 0x7f5f48070010 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c004f40 tx=0x7f5f4c00b040 comp rx=0 tx=0).stop 2024-05-06T12:16:03.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.168+0000 7f5f63516700 1 -- 172.21.15.92:0/3821469756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f5c0662c0 msgr2=0x7f5f5c10da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:03.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.168+0000 7f5f63516700 1 --2- 172.21.15.92:0/3821469756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f5c0662c0 0x7f5f5c10da20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800b3b0 tx=0x7f5f58009e30 comp rx=0 tx=0).stop 2024-05-06T12:16:03.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.168+0000 7f5f63516700 1 -- 172.21.15.92:0/3821469756 shutdown_connections 2024-05-06T12:16:03.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.168+0000 7f5f63516700 1 --2- 172.21.15.92:0/3821469756 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f5f4806db60 0x7f5f48070010 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:03.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.168+0000 7f5f63516700 1 --2- 172.21.15.92:0/3821469756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f5c0662c0 0x7f5f5c10da20 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:03.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.168+0000 7f5f63516700 1 --2- 172.21.15.92:0/3821469756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f5c0ffee0 0x7f5f5c10df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:03.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.168+0000 7f5f63516700 1 -- 172.21.15.92:0/3821469756 >> 172.21.15.92:0/3821469756 conn(0x7f5f5c0f7430 msgr2=0x7f5f5c0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:16:03.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.168+0000 7f5f63516700 1 -- 172.21.15.92:0/3821469756 shutdown_connections 2024-05-06T12:16:03.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.168+0000 7f5f63516700 1 -- 172.21.15.92:0/3821469756 wait complete. 2024-05-06T12:16:03.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.312+0000 7fe49e7b5700 1 -- 172.21.15.92:0/1155505351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe498070810 msgr2=0x7fe498070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:03.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.312+0000 7fe49e7b5700 1 --2- 172.21.15.92:0/1155505351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe498070810 0x7fe498070c80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe48c004940 tx=0x7fe48c009e30 comp rx=0 tx=0).stop 2024-05-06T12:16:03.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.312+0000 7fe49e7b5700 1 -- 172.21.15.92:0/1155505351 shutdown_connections 2024-05-06T12:16:03.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.312+0000 7fe49e7b5700 1 --2- 172.21.15.92:0/1155505351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe498070810 0x7fe498070c80 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:03.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.312+0000 7fe49e7b5700 1 --2- 172.21.15.92:0/1155505351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe498071dc0 0x7fe4980702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:03.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.312+0000 7fe49e7b5700 1 -- 172.21.15.92:0/1155505351 >> 172.21.15.92:0/1155505351 conn(0x7fe4980f85c0 msgr2=0x7fe4980fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:16:03.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.312+0000 7fe49e7b5700 1 -- 172.21.15.92:0/1155505351 shutdown_connections 2024-05-06T12:16:03.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.312+0000 7fe49e7b5700 1 -- 172.21.15.92:0/1155505351 wait complete. 2024-05-06T12:16:03.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.312+0000 7fe49e7b5700 1 Processor -- start 2024-05-06T12:16:03.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.312+0000 7fe49e7b5700 1 -- start start 2024-05-06T12:16:03.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.313+0000 7fe49e7b5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe498070810 0x7fe49810fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:03.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.313+0000 7fe49e7b5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe498071dc0 0x7fe4981100e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:03.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.313+0000 7fe497fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe498070810 0x7fe49810fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:03.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.313+0000 7fe497fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe498070810 0x7fe49810fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47720/0 (socket says 172.21.15.92:47720) 2024-05-06T12:16:03.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.313+0000 7fe497fff700 1 -- 172.21.15.92:0/2614713335 learned_addr learned my addr 172.21.15.92:0/2614713335 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:16:03.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.313+0000 7fe49e7b5700 1 -- 172.21.15.92:0/2614713335 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4981106e0 con 0x7fe498070810 2024-05-06T12:16:03.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.313+0000 7fe49e7b5700 1 -- 172.21.15.92:0/2614713335 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe498110f30 con 0x7fe498071dc0 2024-05-06T12:16:03.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.313+0000 7fe4977fe700 1 --2- 172.21.15.92:0/2614713335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe498071dc0 0x7fe4981100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:03.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.313+0000 7fe497fff700 1 -- 172.21.15.92:0/2614713335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe498071dc0 msgr2=0x7fe4981100e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:03.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.313+0000 7fe497fff700 1 --2- 172.21.15.92:0/2614713335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe498071dc0 0x7fe4981100e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:03.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.313+0000 7fe497fff700 1 -- 172.21.15.92:0/2614713335 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe48c005040 con 0x7fe498070810 2024-05-06T12:16:03.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.314+0000 7fe497fff700 1 --2- 172.21.15.92:0/2614713335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe498070810 0x7fe49810fba0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe48800eb70 tx=0x7fe48800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:16:03.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.314+0000 7fe494ff9700 1 -- 172.21.15.92:0/2614713335 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe488007cc0 con 0x7fe498070810 2024-05-06T12:16:03.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.314+0000 7fe49e7b5700 1 -- 172.21.15.92:0/2614713335 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4981111b0 con 0x7fe498070810 2024-05-06T12:16:03.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.314+0000 7fe494ff9700 1 -- 172.21.15.92:0/2614713335 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe48800f040 con 0x7fe498070810 2024-05-06T12:16:03.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.314+0000 7fe494ff9700 1 -- 172.21.15.92:0/2614713335 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe48800cb20 con 0x7fe498070810 2024-05-06T12:16:03.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.314+0000 7fe49e7b5700 1 -- 172.21.15.92:0/2614713335 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4981116f0 con 0x7fe498070810 2024-05-06T12:16:03.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.315+0000 7fe49e7b5700 1 -- 172.21.15.92:0/2614713335 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe49805f5c0 con 0x7fe498070810 2024-05-06T12:16:03.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.316+0000 7fe494ff9700 1 -- 172.21.15.92:0/2614713335 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 32) v1 ==== 92027+0+0 (secure 0 0 0) 0x7fe48800cc80 con 0x7fe498070810 2024-05-06T12:16:03.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.316+0000 7fe494ff9700 1 --2- 172.21.15.92:0/2614713335 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7fe48006dc30 0x7fe4800700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:03.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.316+0000 7fe494ff9700 1 -- 172.21.15.92:0/2614713335 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fe48808ce60 con 0x7fe498070810 2024-05-06T12:16:03.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.317+0000 7fe4977fe700 1 --2- 172.21.15.92:0/2614713335 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7fe48006dc30 0x7fe4800700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:03.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.317+0000 7fe4977fe700 1 --2- 172.21.15.92:0/2614713335 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7fe48006dc30 0x7fe4800700e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00b0d0 tx=0x7fe48c009e30 comp rx=0 tx=0).ready entity=mgr.14748 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:16:03.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.323+0000 7fe494ff9700 1 -- 172.21.15.92:0/2614713335 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fe48805a3d0 con 0x7fe498070810 2024-05-06T12:16:03.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:03 smithi092 ceph-mon[26954]: from='client.14780 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:16:03.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:03 smithi092 ceph-mon[26954]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:16:03.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.741+0000 7fe49e7b5700 1 -- 172.21.15.92:0/2614713335 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe498107ad0 con 0x7fe498070810 2024-05-06T12:16:03.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.742+0000 7fe494ff9700 1 -- 172.21.15.92:0/2614713335 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7fe488056070 con 0x7fe498070810 2024-05-06T12:16:03.745 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:16:03.745 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": { 2024-05-06T12:16:03.745 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 2 2024-05-06T12:16:03.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:16:03.745 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": { 2024-05-06T12:16:03.745 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:16:03.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:16:03.745 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": { 2024-05-06T12:16:03.746 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 6 2024-05-06T12:16:03.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:16:03.746 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-05-06T12:16:03.746 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 4 2024-05-06T12:16:03.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:16:03.746 INFO:teuthology.orchestra.run.smithi092.stdout: "overall": { 2024-05-06T12:16:03.746 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 12, 2024-05-06T12:16:03.746 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:16:03.746 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-06T12:16:03.746 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:16:03.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.745+0000 7fe49e7b5700 1 -- 172.21.15.92:0/2614713335 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7fe48006dc30 msgr2=0x7fe4800700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:03.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.745+0000 7fe49e7b5700 1 --2- 172.21.15.92:0/2614713335 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7fe48006dc30 0x7fe4800700e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00b0d0 tx=0x7fe48c009e30 comp rx=0 tx=0).stop 2024-05-06T12:16:03.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.745+0000 7fe49e7b5700 1 -- 172.21.15.92:0/2614713335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe498070810 msgr2=0x7fe49810fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:03.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.745+0000 7fe49e7b5700 1 --2- 172.21.15.92:0/2614713335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe498070810 0x7fe49810fba0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe48800eb70 tx=0x7fe48800b040 comp rx=0 tx=0).stop 2024-05-06T12:16:03.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.745+0000 7fe49e7b5700 1 -- 172.21.15.92:0/2614713335 shutdown_connections 2024-05-06T12:16:03.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.745+0000 7fe49e7b5700 1 --2- 172.21.15.92:0/2614713335 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7fe48006dc30 0x7fe4800700e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:03.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.745+0000 7fe49e7b5700 1 --2- 172.21.15.92:0/2614713335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe498070810 0x7fe49810fba0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:03.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.746+0000 7fe49e7b5700 1 --2- 172.21.15.92:0/2614713335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe498071dc0 0x7fe4981100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:03.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.746+0000 7fe49e7b5700 1 -- 172.21.15.92:0/2614713335 >> 172.21.15.92:0/2614713335 conn(0x7fe4980f85c0 msgr2=0x7fe4980f9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:16:03.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.746+0000 7fe49e7b5700 1 -- 172.21.15.92:0/2614713335 shutdown_connections 2024-05-06T12:16:03.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.746+0000 7fe49e7b5700 1 -- 172.21.15.92:0/2614713335 wait complete. 2024-05-06T12:16:03.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:03 smithi190 ceph-mon[33798]: from='client.14780 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:16:03.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:03 smithi190 ceph-mon[33798]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:16:03.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.886+0000 7f4da1247700 1 -- 172.21.15.92:0/3715514168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d9c0662c0 msgr2=0x7f4d9c0ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:03.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.886+0000 7f4da1247700 1 --2- 172.21.15.92:0/3715514168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d9c0662c0 0x7f4d9c0ff9a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c004940 tx=0x7f4d8c009e30 comp rx=0 tx=0).stop 2024-05-06T12:16:03.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.886+0000 7f4da1247700 1 -- 172.21.15.92:0/3715514168 shutdown_connections 2024-05-06T12:16:03.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.886+0000 7f4da1247700 1 --2- 172.21.15.92:0/3715514168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d9c0ffee0 0x7f4d9c102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:03.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.886+0000 7f4da1247700 1 --2- 172.21.15.92:0/3715514168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d9c0662c0 0x7f4d9c0ff9a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:03.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.886+0000 7f4da1247700 1 -- 172.21.15.92:0/3715514168 >> 172.21.15.92:0/3715514168 conn(0x7f4d9c0f7430 msgr2=0x7f4d9c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:16:03.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.886+0000 7f4da1247700 1 -- 172.21.15.92:0/3715514168 shutdown_connections 2024-05-06T12:16:03.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.886+0000 7f4da1247700 1 -- 172.21.15.92:0/3715514168 wait complete. 2024-05-06T12:16:03.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.887+0000 7f4da1247700 1 Processor -- start 2024-05-06T12:16:03.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.887+0000 7f4da1247700 1 -- start start 2024-05-06T12:16:03.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.887+0000 7f4da1247700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d9c0662c0 0x7f4d9c113fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:03.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.887+0000 7f4da1247700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d9c0ffee0 0x7f4d9c1144f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:03.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.887+0000 7f4da1247700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c114af0 con 0x7f4d9c0662c0 2024-05-06T12:16:03.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.887+0000 7f4d9ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d9c0662c0 0x7f4d9c113fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:03.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.887+0000 7f4d9ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d9c0662c0 0x7f4d9c113fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47734/0 (socket says 172.21.15.92:47734) 2024-05-06T12:16:03.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.887+0000 7f4d9ad9d700 1 -- 172.21.15.92:0/3742234125 learned_addr learned my addr 172.21.15.92:0/3742234125 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:16:03.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.887+0000 7f4d9a59c700 1 --2- 172.21.15.92:0/3742234125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d9c0ffee0 0x7f4d9c1144f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:03.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.887+0000 7f4da1247700 1 -- 172.21.15.92:0/3742234125 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c1152f0 con 0x7f4d9c0ffee0 2024-05-06T12:16:03.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.888+0000 7f4d9a59c700 1 -- 172.21.15.92:0/3742234125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d9c0662c0 msgr2=0x7f4d9c113fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:03.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.888+0000 7f4d9a59c700 1 --2- 172.21.15.92:0/3742234125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d9c0662c0 0x7f4d9c113fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:03.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.888+0000 7f4d9a59c700 1 -- 172.21.15.92:0/3742234125 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d8c005040 con 0x7f4d9c0ffee0 2024-05-06T12:16:03.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.888+0000 7f4d9a59c700 1 --2- 172.21.15.92:0/3742234125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d9c0ffee0 0x7f4d9c1144f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4d90009fd0 tx=0x7f4d9000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:16:03.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.888+0000 7f4d837fe700 1 -- 172.21.15.92:0/3742234125 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d90004e90 con 0x7f4d9c0ffee0 2024-05-06T12:16:03.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.888+0000 7f4d837fe700 1 -- 172.21.15.92:0/3742234125 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d9000f040 con 0x7f4d9c0ffee0 2024-05-06T12:16:03.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.888+0000 7f4da1247700 1 -- 172.21.15.92:0/3742234125 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d9c115570 con 0x7f4d9c0ffee0 2024-05-06T12:16:03.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.889+0000 7f4d837fe700 1 -- 172.21.15.92:0/3742234125 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d9000ca50 con 0x7f4d9c0ffee0 2024-05-06T12:16:03.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.889+0000 7f4da1247700 1 -- 172.21.15.92:0/3742234125 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d9c1159f0 con 0x7f4d9c0ffee0 2024-05-06T12:16:03.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.890+0000 7f4da1247700 1 -- 172.21.15.92:0/3742234125 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d9c05f5c0 con 0x7f4d9c0ffee0 2024-05-06T12:16:03.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.890+0000 7f4d837fe700 1 -- 172.21.15.92:0/3742234125 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 32) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f4d9000cbb0 con 0x7f4d9c0ffee0 2024-05-06T12:16:03.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.891+0000 7f4d837fe700 1 --2- 172.21.15.92:0/3742234125 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f4d8406dc30 0x7f4d840700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:03.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.891+0000 7f4d837fe700 1 -- 172.21.15.92:0/3742234125 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f4d9008ce50 con 0x7f4d9c0ffee0 2024-05-06T12:16:03.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.891+0000 7f4d9ad9d700 1 --2- 172.21.15.92:0/3742234125 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f4d8406dc30 0x7f4d840700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:03.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.892+0000 7f4d9ad9d700 1 --2- 172.21.15.92:0/3742234125 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f4d8406dc30 0x7f4d840700e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c009fd0 tx=0x7f4d8c009e30 comp rx=0 tx=0).ready entity=mgr.14748 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:16:03.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:03.897+0000 7f4d837fe700 1 -- 172.21.15.92:0/3742234125 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f4d9005a3c0 con 0x7f4d9c0ffee0 2024-05-06T12:16:04.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.244+0000 7f4da1247700 1 -- 172.21.15.92:0/3742234125 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4d9c107a90 con 0x7f4d9c0ffee0 2024-05-06T12:16:04.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.245+0000 7f4d837fe700 1 -- 172.21.15.92:0/3742234125 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1867 (secure 0 0 0) 0x7f4d90019760 con 0x7f4d9c0ffee0 2024-05-06T12:16:04.249 INFO:teuthology.orchestra.run.smithi092.stdout:e20 2024-05-06T12:16:04.249 INFO:teuthology.orchestra.run.smithi092.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-06T12:16:04.249 INFO:teuthology.orchestra.run.smithi092.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:16:04.250 INFO:teuthology.orchestra.run.smithi092.stdout:legacy client fscid: 1 2024-05-06T12:16:04.250 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:16:04.250 INFO:teuthology.orchestra.run.smithi092.stdout:Filesystem 'cephfs' (1) 2024-05-06T12:16:04.250 INFO:teuthology.orchestra.run.smithi092.stdout:fs_name cephfs 2024-05-06T12:16:04.250 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 20 2024-05-06T12:16:04.250 INFO:teuthology.orchestra.run.smithi092.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-06T12:16:04.251 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-05-06T12:11:44.952671+0000 2024-05-06T12:16:04.251 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2024-05-06T12:15:07.568167+0000 2024-05-06T12:16:04.251 INFO:teuthology.orchestra.run.smithi092.stdout:tableserver 0 2024-05-06T12:16:04.251 INFO:teuthology.orchestra.run.smithi092.stdout:root 0 2024-05-06T12:16:04.251 INFO:teuthology.orchestra.run.smithi092.stdout:session_timeout 60 2024-05-06T12:16:04.252 INFO:teuthology.orchestra.run.smithi092.stdout:session_autoclose 300 2024-05-06T12:16:04.252 INFO:teuthology.orchestra.run.smithi092.stdout:max_file_size 1099511627776 2024-05-06T12:16:04.252 INFO:teuthology.orchestra.run.smithi092.stdout:required_client_features {} 2024-05-06T12:16:04.252 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure 0 2024-05-06T12:16:04.252 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure_osd_epoch 46 2024-05-06T12:16:04.252 INFO:teuthology.orchestra.run.smithi092.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:16:04.253 INFO:teuthology.orchestra.run.smithi092.stdout:max_mds 2 2024-05-06T12:16:04.253 INFO:teuthology.orchestra.run.smithi092.stdout:in 0,1 2024-05-06T12:16:04.253 INFO:teuthology.orchestra.run.smithi092.stdout:up {0=14472,1=24267} 2024-05-06T12:16:04.253 INFO:teuthology.orchestra.run.smithi092.stdout:failed 2024-05-06T12:16:04.253 INFO:teuthology.orchestra.run.smithi092.stdout:damaged 2024-05-06T12:16:04.253 INFO:teuthology.orchestra.run.smithi092.stdout:stopped 2024-05-06T12:16:04.254 INFO:teuthology.orchestra.run.smithi092.stdout:data_pools [3] 2024-05-06T12:16:04.254 INFO:teuthology.orchestra.run.smithi092.stdout:metadata_pool 2 2024-05-06T12:16:04.254 INFO:teuthology.orchestra.run.smithi092.stdout:inline_data enabled 2024-05-06T12:16:04.254 INFO:teuthology.orchestra.run.smithi092.stdout:balancer 2024-05-06T12:16:04.254 INFO:teuthology.orchestra.run.smithi092.stdout:standby_count_wanted 1 2024-05-06T12:16:04.255 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.rosbll{0:14472} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.92:6828/1663253598,v1:172.21.15.92:6829/1663253598] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:16:04.255 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.ijsqgp{0:14484} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.92:6826/3613441054,v1:172.21.15.92:6827/3613441054] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:16:04.255 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.byjvgf{1:24267} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.190:6824/1565151699,v1:172.21.15.190:6825/1565151699] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:16:04.255 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.idzoff{1:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.190:6826/2453728380,v1:172.21.15.190:6827/2453728380] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:16:04.255 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:16:04.255 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:16:04.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.249+0000 7f4da1247700 1 -- 172.21.15.92:0/3742234125 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f4d8406dc30 msgr2=0x7f4d840700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:04.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.249+0000 7f4da1247700 1 --2- 172.21.15.92:0/3742234125 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f4d8406dc30 0x7f4d840700e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c009fd0 tx=0x7f4d8c009e30 comp rx=0 tx=0).stop 2024-05-06T12:16:04.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.249+0000 7f4da1247700 1 -- 172.21.15.92:0/3742234125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d9c0ffee0 msgr2=0x7f4d9c1144f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.249+0000 7f4da1247700 1 --2- 172.21.15.92:0/3742234125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d9c0ffee0 0x7f4d9c1144f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4d90009fd0 tx=0x7f4d9000b040 comp rx=0 tx=0).stop 2024-05-06T12:16:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.249+0000 7f4da1247700 1 -- 172.21.15.92:0/3742234125 shutdown_connections 2024-05-06T12:16:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.249+0000 7f4da1247700 1 --2- 172.21.15.92:0/3742234125 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f4d8406dc30 0x7f4d840700e0 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c009fd0 tx=0x7f4d8c009e30 comp rx=0 tx=0).stop 2024-05-06T12:16:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.249+0000 7f4da1247700 1 --2- 172.21.15.92:0/3742234125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d9c0662c0 0x7f4d9c113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.249+0000 7f4da1247700 1 --2- 172.21.15.92:0/3742234125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d9c0ffee0 0x7f4d9c1144f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.249+0000 7f4da1247700 1 -- 172.21.15.92:0/3742234125 >> 172.21.15.92:0/3742234125 conn(0x7f4d9c0f7430 msgr2=0x7f4d9c0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:16:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.250+0000 7f4da1247700 1 -- 172.21.15.92:0/3742234125 shutdown_connections 2024-05-06T12:16:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.250+0000 7f4da1247700 1 -- 172.21.15.92:0/3742234125 wait complete. 2024-05-06T12:16:04.260 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 20 2024-05-06T12:16:04.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.381+0000 7f9e2a5bd700 1 -- 172.21.15.92:0/767292077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e24066ed0 msgr2=0x7f9e240ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:04.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.381+0000 7f9e2a5bd700 1 --2- 172.21.15.92:0/767292077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e24066ed0 0x7f9e240ffa30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9e14004940 tx=0x7f9e14009e30 comp rx=0 tx=0).stop 2024-05-06T12:16:04.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.382+0000 7f9e2a5bd700 1 -- 172.21.15.92:0/767292077 shutdown_connections 2024-05-06T12:16:04.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.382+0000 7f9e2a5bd700 1 --2- 172.21.15.92:0/767292077 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e240fff70 0x7f9e24102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:04.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.382+0000 7f9e2a5bd700 1 --2- 172.21.15.92:0/767292077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e24066ed0 0x7f9e240ffa30 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:04.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.382+0000 7f9e2a5bd700 1 -- 172.21.15.92:0/767292077 >> 172.21.15.92:0/767292077 conn(0x7f9e240f7440 msgr2=0x7f9e240f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:16:04.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.382+0000 7f9e2a5bd700 1 -- 172.21.15.92:0/767292077 shutdown_connections 2024-05-06T12:16:04.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.382+0000 7f9e2a5bd700 1 -- 172.21.15.92:0/767292077 wait complete. 2024-05-06T12:16:04.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.382+0000 7f9e2a5bd700 1 Processor -- start 2024-05-06T12:16:04.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.383+0000 7f9e2a5bd700 1 -- start start 2024-05-06T12:16:04.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.383+0000 7f9e2a5bd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e24066ed0 0x7f9e240fd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:04.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.383+0000 7f9e2a5bd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e240fff70 0x7f9e240fb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:04.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.383+0000 7f9e2a5bd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e240fd880 con 0x7f9e240fff70 2024-05-06T12:16:04.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.383+0000 7f9e2a5bd700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e240fbe70 con 0x7f9e24066ed0 2024-05-06T12:16:04.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.383+0000 7f9e237fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e240fff70 0x7f9e240fb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:04.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.383+0000 7f9e237fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e240fff70 0x7f9e240fb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47754/0 (socket says 172.21.15.92:47754) 2024-05-06T12:16:04.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.383+0000 7f9e237fe700 1 -- 172.21.15.92:0/652139091 learned_addr learned my addr 172.21.15.92:0/652139091 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:16:04.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.383+0000 7f9e23fff700 1 --2- 172.21.15.92:0/652139091 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e24066ed0 0x7f9e240fd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:04.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.383+0000 7f9e237fe700 1 -- 172.21.15.92:0/652139091 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e24066ed0 msgr2=0x7f9e240fd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:04.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.383+0000 7f9e237fe700 1 --2- 172.21.15.92:0/652139091 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e24066ed0 0x7f9e240fd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:04.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.383+0000 7f9e237fe700 1 -- 172.21.15.92:0/652139091 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e14005040 con 0x7f9e240fff70 2024-05-06T12:16:04.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.383+0000 7f9e237fe700 1 --2- 172.21.15.92:0/652139091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e240fff70 0x7f9e240fb930 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9e1800eb40 tx=0x7f9e1800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:16:04.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.384+0000 7f9e20ff9700 1 -- 172.21.15.92:0/652139091 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e18007cc0 con 0x7f9e240fff70 2024-05-06T12:16:04.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.384+0000 7f9e2a5bd700 1 -- 172.21.15.92:0/652139091 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e240fc0c0 con 0x7f9e240fff70 2024-05-06T12:16:04.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.384+0000 7f9e20ff9700 1 -- 172.21.15.92:0/652139091 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e1800f040 con 0x7f9e240fff70 2024-05-06T12:16:04.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.384+0000 7f9e2a5bd700 1 -- 172.21.15.92:0/652139091 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e240fc600 con 0x7f9e240fff70 2024-05-06T12:16:04.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.384+0000 7f9e20ff9700 1 -- 172.21.15.92:0/652139091 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e1800ca10 con 0x7f9e240fff70 2024-05-06T12:16:04.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.385+0000 7f9e2a5bd700 1 -- 172.21.15.92:0/652139091 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e100052e0 con 0x7f9e240fff70 2024-05-06T12:16:04.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.386+0000 7f9e20ff9700 1 -- 172.21.15.92:0/652139091 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 32) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f9e1800cb70 con 0x7f9e240fff70 2024-05-06T12:16:04.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.386+0000 7f9e20ff9700 1 --2- 172.21.15.92:0/652139091 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f9e0c06db70 0x7f9e0c070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:04.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.386+0000 7f9e20ff9700 1 -- 172.21.15.92:0/652139091 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f9e1808ce70 con 0x7f9e240fff70 2024-05-06T12:16:04.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.387+0000 7f9e23fff700 1 --2- 172.21.15.92:0/652139091 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f9e0c06db70 0x7f9e0c070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:04.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.387+0000 7f9e23fff700 1 --2- 172.21.15.92:0/652139091 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f9e0c06db70 0x7f9e0c070020 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9e1400b0d0 tx=0x7f9e14009e30 comp rx=0 tx=0).ready entity=mgr.14748 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:16:04.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.393+0000 7f9e20ff9700 1 -- 172.21.15.92:0/652139091 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f9e1805a1f0 con 0x7f9e240fff70 2024-05-06T12:16:04.702 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:04 smithi092 ceph-mon[26954]: from='client.14788 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:16:04.702 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:04 smithi092 ceph-mon[26954]: pgmap v11: 65 pgs: 65 active+clean; 329 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 123 op/s 2024-05-06T12:16:04.702 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:04 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/2614713335' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:16:04.702 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:04 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/3742234125' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:16:04.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.699+0000 7f9e2a5bd700 1 -- 172.21.15.92:0/652139091 --> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e10000bf0 con 0x7f9e0c06db70 2024-05-06T12:16:04.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.701+0000 7f9e20ff9700 1 -- 172.21.15.92:0/652139091 <== mgr.14748 v2:172.21.15.92:6800/648200129 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f9e10000bf0 con 0x7f9e0c06db70 2024-05-06T12:16:04.705 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:16:04.705 INFO:teuthology.orchestra.run.smithi092.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb", 2024-05-06T12:16:04.705 INFO:teuthology.orchestra.run.smithi092.stdout: "in_progress": true, 2024-05-06T12:16:04.705 INFO:teuthology.orchestra.run.smithi092.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-06T12:16:04.705 INFO:teuthology.orchestra.run.smithi092.stdout: "services_complete": [ 2024-05-06T12:16:04.705 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr" 2024-05-06T12:16:04.706 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-05-06T12:16:04.706 INFO:teuthology.orchestra.run.smithi092.stdout: "progress": "2/21 daemons upgraded", 2024-05-06T12:16:04.706 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "", 2024-05-06T12:16:04.706 INFO:teuthology.orchestra.run.smithi092.stdout: "is_paused": false 2024-05-06T12:16:04.706 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:16:04.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.705+0000 7f9e2a5bd700 1 -- 172.21.15.92:0/652139091 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f9e0c06db70 msgr2=0x7f9e0c070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:04.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.705+0000 7f9e2a5bd700 1 --2- 172.21.15.92:0/652139091 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f9e0c06db70 0x7f9e0c070020 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9e1400b0d0 tx=0x7f9e14009e30 comp rx=0 tx=0).stop 2024-05-06T12:16:04.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.705+0000 7f9e2a5bd700 1 -- 172.21.15.92:0/652139091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e240fff70 msgr2=0x7f9e240fb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:04.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.705+0000 7f9e2a5bd700 1 --2- 172.21.15.92:0/652139091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e240fff70 0x7f9e240fb930 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9e1800eb40 tx=0x7f9e1800b040 comp rx=0 tx=0).stop 2024-05-06T12:16:04.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.705+0000 7f9e2a5bd700 1 -- 172.21.15.92:0/652139091 shutdown_connections 2024-05-06T12:16:04.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.705+0000 7f9e2a5bd700 1 --2- 172.21.15.92:0/652139091 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e24066ed0 0x7f9e240fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:04.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.705+0000 7f9e2a5bd700 1 --2- 172.21.15.92:0/652139091 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f9e0c06db70 0x7f9e0c070020 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:04.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.705+0000 7f9e2a5bd700 1 --2- 172.21.15.92:0/652139091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e240fff70 0x7f9e240fb930 secure :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9e1800eb40 tx=0x7f9e1800b040 comp rx=0 tx=0).stop 2024-05-06T12:16:04.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.705+0000 7f9e2a5bd700 1 -- 172.21.15.92:0/652139091 >> 172.21.15.92:0/652139091 conn(0x7f9e240f7440 msgr2=0x7f9e240f7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:16:04.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.705+0000 7f9e2a5bd700 1 -- 172.21.15.92:0/652139091 shutdown_connections 2024-05-06T12:16:04.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.705+0000 7f9e2a5bd700 1 -- 172.21.15.92:0/652139091 wait complete. 2024-05-06T12:16:04.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.842+0000 7f5a55b38700 1 -- 172.21.15.92:0/1700761745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a500fdbc0 msgr2=0x7f5a500fbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:04.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.842+0000 7f5a55b38700 1 --2- 172.21.15.92:0/1700761745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a500fdbc0 0x7f5a500fbfe0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5a44004940 tx=0x7f5a44009e30 comp rx=0 tx=0).stop 2024-05-06T12:16:04.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.842+0000 7f5a55b38700 1 -- 172.21.15.92:0/1700761745 shutdown_connections 2024-05-06T12:16:04.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.842+0000 7f5a55b38700 1 --2- 172.21.15.92:0/1700761745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a500fdbc0 0x7f5a500fbfe0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:04.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.842+0000 7f5a55b38700 1 --2- 172.21.15.92:0/1700761745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a500fd270 0x7f5a500fd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:04.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.842+0000 7f5a55b38700 1 -- 172.21.15.92:0/1700761745 >> 172.21.15.92:0/1700761745 conn(0x7f5a500f6b80 msgr2=0x7f5a500f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:16:04.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.842+0000 7f5a55b38700 1 -- 172.21.15.92:0/1700761745 shutdown_connections 2024-05-06T12:16:04.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.842+0000 7f5a55b38700 1 -- 172.21.15.92:0/1700761745 wait complete. 2024-05-06T12:16:04.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.842+0000 7f5a55b38700 1 Processor -- start 2024-05-06T12:16:04.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.843+0000 7f5a55b38700 1 -- start start 2024-05-06T12:16:04.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.843+0000 7f5a55b38700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a500fd270 0x7f5a501106c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:04.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.843+0000 7f5a55b38700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a500fbab0 0x7f5a50112c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:04.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.843+0000 7f5a55b38700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a50113150 con 0x7f5a500fbab0 2024-05-06T12:16:04.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.843+0000 7f5a55b38700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a501132c0 con 0x7f5a500fd270 2024-05-06T12:16:04.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.843+0000 7f5a4effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a500fbab0 0x7f5a50112c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:04.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.843+0000 7f5a4effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a500fbab0 0x7f5a50112c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47772/0 (socket says 172.21.15.92:47772) 2024-05-06T12:16:04.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.843+0000 7f5a4effd700 1 -- 172.21.15.92:0/87823452 learned_addr learned my addr 172.21.15.92:0/87823452 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:16:04.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.843+0000 7f5a4f7fe700 1 --2- 172.21.15.92:0/87823452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a500fd270 0x7f5a501106c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:04.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.843+0000 7f5a4f7fe700 1 -- 172.21.15.92:0/87823452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a500fbab0 msgr2=0x7f5a50112c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:04.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.843+0000 7f5a4f7fe700 1 --2- 172.21.15.92:0/87823452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a500fbab0 0x7f5a50112c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:04.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.843+0000 7f5a4f7fe700 1 -- 172.21.15.92:0/87823452 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a44005040 con 0x7f5a500fd270 2024-05-06T12:16:04.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.843+0000 7f5a4effd700 1 --2- 172.21.15.92:0/87823452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a500fbab0 0x7f5a50112c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-06T12:16:04.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.844+0000 7f5a4f7fe700 1 --2- 172.21.15.92:0/87823452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a500fd270 0x7f5a501106c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5a500650b0 tx=0x7f5a4000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:16:04.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.844+0000 7f5a37fff700 1 -- 172.21.15.92:0/87823452 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a40004e90 con 0x7f5a500fd270 2024-05-06T12:16:04.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.844+0000 7f5a37fff700 1 -- 172.21.15.92:0/87823452 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a4000f040 con 0x7f5a500fd270 2024-05-06T12:16:04.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.844+0000 7f5a37fff700 1 -- 172.21.15.92:0/87823452 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a4000c980 con 0x7f5a500fd270 2024-05-06T12:16:04.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.844+0000 7f5a55b38700 1 -- 172.21.15.92:0/87823452 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a50113540 con 0x7f5a500fd270 2024-05-06T12:16:04.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.844+0000 7f5a55b38700 1 -- 172.21.15.92:0/87823452 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a50113a80 con 0x7f5a500fd270 2024-05-06T12:16:04.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:04 smithi190 ceph-mon[33798]: from='client.14788 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:16:04.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:04 smithi190 ceph-mon[33798]: pgmap v11: 65 pgs: 65 active+clean; 329 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 123 op/s 2024-05-06T12:16:04.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:04 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/2614713335' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:16:04.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:04 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/3742234125' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:16:04.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.845+0000 7f5a55b38700 1 -- 172.21.15.92:0/87823452 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a5005f5c0 con 0x7f5a500fd270 2024-05-06T12:16:04.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.846+0000 7f5a37fff700 1 -- 172.21.15.92:0/87823452 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 32) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f5a400090d0 con 0x7f5a500fd270 2024-05-06T12:16:04.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.846+0000 7f5a37fff700 1 --2- 172.21.15.92:0/87823452 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f5a3806db60 0x7f5a38070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:04.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.846+0000 7f5a37fff700 1 -- 172.21.15.92:0/87823452 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f5a4008cd10 con 0x7f5a500fd270 2024-05-06T12:16:04.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.846+0000 7f5a4effd700 1 --2- 172.21.15.92:0/87823452 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f5a3806db60 0x7f5a38070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:04.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.847+0000 7f5a4effd700 1 --2- 172.21.15.92:0/87823452 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f5a3806db60 0x7f5a38070010 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5a44009b30 tx=0x7f5a44009e30 comp rx=0 tx=0).ready entity=mgr.14748 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:16:04.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:04.851+0000 7f5a37fff700 1 -- 172.21.15.92:0/87823452 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f5a400567f0 con 0x7f5a500fd270 2024-05-06T12:16:05.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:05.221+0000 7f5a55b38700 1 -- 172.21.15.92:0/87823452 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5a5004bed0 con 0x7f5a500fd270 2024-05-06T12:16:05.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:05.222+0000 7f5a37fff700 1 -- 172.21.15.92:0/87823452 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f5a40059eb0 con 0x7f5a500fd270 2024-05-06T12:16:05.225 INFO:teuthology.orchestra.run.smithi092.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-06T12:16:05.225 INFO:teuthology.orchestra.run.smithi092.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-06T12:16:05.225 INFO:teuthology.orchestra.run.smithi092.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-06T12:16:05.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:05.224+0000 7f5a55b38700 1 -- 172.21.15.92:0/87823452 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f5a3806db60 msgr2=0x7f5a38070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:05.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:05.224+0000 7f5a55b38700 1 --2- 172.21.15.92:0/87823452 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f5a3806db60 0x7f5a38070010 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5a44009b30 tx=0x7f5a44009e30 comp rx=0 tx=0).stop 2024-05-06T12:16:05.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:05.224+0000 7f5a55b38700 1 -- 172.21.15.92:0/87823452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a500fd270 msgr2=0x7f5a501106c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:05.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:05.224+0000 7f5a55b38700 1 --2- 172.21.15.92:0/87823452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a500fd270 0x7f5a501106c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5a500650b0 tx=0x7f5a4000b040 comp rx=0 tx=0).stop 2024-05-06T12:16:05.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:05.224+0000 7f5a55b38700 1 -- 172.21.15.92:0/87823452 shutdown_connections 2024-05-06T12:16:05.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:05.224+0000 7f5a55b38700 1 --2- 172.21.15.92:0/87823452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a500fd270 0x7f5a501106c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:05.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:05.224+0000 7f5a55b38700 1 --2- 172.21.15.92:0/87823452 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f5a3806db60 0x7f5a38070010 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:05.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:05.224+0000 7f5a55b38700 1 --2- 172.21.15.92:0/87823452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a500fbab0 0x7f5a50112c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:05.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:05.224+0000 7f5a55b38700 1 -- 172.21.15.92:0/87823452 >> 172.21.15.92:0/87823452 conn(0x7f5a500f6b80 msgr2=0x7f5a500f8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:16:05.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:05.224+0000 7f5a55b38700 1 -- 172.21.15.92:0/87823452 shutdown_connections 2024-05-06T12:16:05.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:05.224+0000 7f5a55b38700 1 -- 172.21.15.92:0/87823452 wait complete. 2024-05-06T12:16:05.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:05 smithi092 ceph-mon[26954]: from='client.14800 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:16:05.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:05 smithi092 ceph-mon[26954]: from='client.? 172.21.15.92:0/87823452' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:16:05.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:05 smithi190 ceph-mon[33798]: from='client.14800 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:16:05.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:05 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/87823452' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:16:06.715 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:06 smithi092 ceph-mon[26954]: pgmap v12: 65 pgs: 65 active+clean; 332 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 215 op/s 2024-05-06T12:16:06.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:06 smithi190 ceph-mon[33798]: pgmap v12: 65 pgs: 65 active+clean; 332 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 215 op/s 2024-05-06T12:16:09.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:08 smithi190 ceph-mon[33798]: pgmap v13: 65 pgs: 65 active+clean; 332 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 158 op/s 2024-05-06T12:16:09.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:08 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:09.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:08 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:09.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:08 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-06T12:16:09.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:08 smithi190 ceph-mon[33798]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-06T12:16:09.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:08 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi092:9095"}]: dispatch 2024-05-06T12:16:09.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:08 smithi190 ceph-mon[33798]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi092:9095"}]: dispatch 2024-05-06T12:16:09.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:08 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:09.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:08 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:16:09.121 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:08 smithi092 ceph-mon[26954]: pgmap v13: 65 pgs: 65 active+clean; 332 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 158 op/s 2024-05-06T12:16:09.121 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:08 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:09.121 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:08 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:09.121 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:08 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-06T12:16:09.122 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:08 smithi092 ceph-mon[26954]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-06T12:16:09.122 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:08 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi092:9095"}]: dispatch 2024-05-06T12:16:09.122 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:08 smithi092 ceph-mon[26954]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi092:9095"}]: dispatch 2024-05-06T12:16:09.122 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:08 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:09.122 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:08 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:16:09.396 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-05-06T12:16:09.396 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-06T12:16:09.782 DEBUG:teuthology.parallel:result is None 2024-05-06T12:16:11.068 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:10 smithi190 ceph-mon[33798]: pgmap v14: 65 pgs: 65 active+clean; 331 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 257 op/s 2024-05-06T12:16:11.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:10 smithi092 ceph-mon[26954]: pgmap v14: 65 pgs: 65 active+clean; 331 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 257 op/s 2024-05-06T12:16:13.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:12 smithi190 ceph-mon[33798]: pgmap v15: 65 pgs: 65 active+clean; 331 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 191 op/s 2024-05-06T12:16:13.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:12 smithi190 ceph-mon[33798]: Upgrade: Updating mgr.smithi190.kczxkh 2024-05-06T12:16:13.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:12 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi190.kczxkh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-06T12:16:13.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:12 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-06T12:16:13.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:12 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:16:13.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:12 smithi190 ceph-mon[33798]: Deploying daemon mgr.smithi190.kczxkh on smithi190 2024-05-06T12:16:13.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:12 smithi092 ceph-mon[26954]: pgmap v15: 65 pgs: 65 active+clean; 331 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 191 op/s 2024-05-06T12:16:13.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:12 smithi092 ceph-mon[26954]: Upgrade: Updating mgr.smithi190.kczxkh 2024-05-06T12:16:13.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:12 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi190.kczxkh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-06T12:16:13.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:12 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-06T12:16:13.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:12 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:16:13.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:12 smithi092 ceph-mon[26954]: Deploying daemon mgr.smithi190.kczxkh on smithi190 2024-05-06T12:16:14.910 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:14 smithi190 ceph-mon[33798]: pgmap v16: 65 pgs: 65 active+clean; 331 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 191 op/s 2024-05-06T12:16:15.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:14 smithi092 ceph-mon[26954]: pgmap v16: 65 pgs: 65 active+clean; 331 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 191 op/s 2024-05-06T12:16:16.242 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-05-06T12:16:16.242 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-05-06T12:16:16.559 DEBUG:teuthology.parallel:result is None 2024-05-06T12:16:16.560 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-06T12:16:16.595 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-06T12:16:16.596 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-05-06T12:16:16.627 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-05-06T12:16:16.627 DEBUG:teuthology.parallel:result is None 2024-05-06T12:16:16.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:16 smithi092 ceph-mon[26954]: pgmap v17: 65 pgs: 65 active+clean; 328 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 264 op/s 2024-05-06T12:16:17.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:16 smithi190 ceph-mon[33798]: pgmap v17: 65 pgs: 65 active+clean; 328 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 264 op/s 2024-05-06T12:16:17.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:17 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:17.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:17 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:17.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:17 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:16:17.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:17 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:16:18.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:17 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:18.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:17 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:18.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:17 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:16:18.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:17 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:16:19.107 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:18 smithi190 ceph-mon[33798]: pgmap v18: 65 pgs: 65 active+clean; 328 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 171 op/s 2024-05-06T12:16:19.125 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:18 smithi092 ceph-mon[26954]: pgmap v18: 65 pgs: 65 active+clean; 328 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 171 op/s 2024-05-06T12:16:21.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:20 smithi190 ceph-mon[33798]: pgmap v19: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 237 op/s 2024-05-06T12:16:21.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:20 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:21.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:20 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:21.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:20 smithi092 ceph-mon[26954]: pgmap v19: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 237 op/s 2024-05-06T12:16:21.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:20 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:21.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:20 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:22.152 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:22 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:22.152 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:22 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:22.152 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:22 smithi190 ceph-mon[33798]: pgmap v20: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 796 KiB/s rd, 780 KiB/s wr, 138 op/s 2024-05-06T12:16:22.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:22 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:22.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:22 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:22.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:22 smithi092 ceph-mon[26954]: pgmap v20: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 796 KiB/s rd, 780 KiB/s wr, 138 op/s 2024-05-06T12:16:24.070 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:23 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:24.070 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:23 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:24.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:23 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:24.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:23 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:25.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:24 smithi092 ceph-mon[26954]: pgmap v21: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 796 KiB/s rd, 780 KiB/s wr, 138 op/s 2024-05-06T12:16:25.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:24 smithi092 ceph-mon[26954]: Standby manager daemon smithi190.kczxkh restarted 2024-05-06T12:16:25.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:24 smithi092 ceph-mon[26954]: Standby manager daemon smithi190.kczxkh started 2024-05-06T12:16:25.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:24 smithi092 ceph-mon[26954]: from='mgr.? 172.21.15.190:0/2733961667' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi190.kczxkh/crt"}]: dispatch 2024-05-06T12:16:25.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:24 smithi092 ceph-mon[26954]: from='mgr.? 172.21.15.190:0/2733961667' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-06T12:16:25.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:24 smithi092 ceph-mon[26954]: from='mgr.? 172.21.15.190:0/2733961667' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi190.kczxkh/key"}]: dispatch 2024-05-06T12:16:25.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:24 smithi092 ceph-mon[26954]: from='mgr.? 172.21.15.190:0/2733961667' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-06T12:16:25.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:24 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:25.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:24 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:25.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:24 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:16:25.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:24 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:16:25.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:24 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:25.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:24 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:16:25.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:24 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:16:25.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:24 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:25.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:24 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi092.gyzeqb"}]: dispatch 2024-05-06T12:16:25.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:24 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi092.gyzeqb"}]': finished 2024-05-06T12:16:25.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:24 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi190.kczxkh"}]: dispatch 2024-05-06T12:16:25.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:24 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi190.kczxkh"}]': finished 2024-05-06T12:16:25.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:24 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-05-06T12:16:25.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:24 smithi190 ceph-mon[33798]: pgmap v21: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 796 KiB/s rd, 780 KiB/s wr, 138 op/s 2024-05-06T12:16:25.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:24 smithi190 ceph-mon[33798]: Standby manager daemon smithi190.kczxkh restarted 2024-05-06T12:16:25.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:24 smithi190 ceph-mon[33798]: Standby manager daemon smithi190.kczxkh started 2024-05-06T12:16:25.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:24 smithi190 ceph-mon[33798]: from='mgr.? 172.21.15.190:0/2733961667' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi190.kczxkh/crt"}]: dispatch 2024-05-06T12:16:25.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:24 smithi190 ceph-mon[33798]: from='mgr.? 172.21.15.190:0/2733961667' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-06T12:16:25.351 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:24 smithi190 ceph-mon[33798]: from='mgr.? 172.21.15.190:0/2733961667' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi190.kczxkh/key"}]: dispatch 2024-05-06T12:16:25.351 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:24 smithi190 ceph-mon[33798]: from='mgr.? 172.21.15.190:0/2733961667' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-06T12:16:25.351 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:24 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:25.351 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:24 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:25.351 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:24 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:16:25.351 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:24 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:16:25.351 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:24 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:25.351 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:24 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:16:25.351 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:24 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:16:25.351 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:24 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:25.351 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:24 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi092.gyzeqb"}]: dispatch 2024-05-06T12:16:25.351 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:24 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi092.gyzeqb"}]': finished 2024-05-06T12:16:25.351 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:24 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi190.kczxkh"}]: dispatch 2024-05-06T12:16:25.351 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:24 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi190.kczxkh"}]': finished 2024-05-06T12:16:25.352 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:24 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-05-06T12:16:26.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:25 smithi092 ceph-mon[26954]: Upgrade: Setting container_image for all mgr 2024-05-06T12:16:26.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:25 smithi092 ceph-mon[26954]: mgrmap e33: smithi092.gyzeqb(active, since 37s), standbys: smithi190.kczxkh 2024-05-06T12:16:26.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:25 smithi190 ceph-mon[33798]: Upgrade: Setting container_image for all mgr 2024-05-06T12:16:26.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:25 smithi190 ceph-mon[33798]: mgrmap e33: smithi092.gyzeqb(active, since 37s), standbys: smithi190.kczxkh 2024-05-06T12:16:27.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:26 smithi092 ceph-mon[26954]: pgmap v22: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 828 KiB/s rd, 811 KiB/s wr, 143 op/s 2024-05-06T12:16:27.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:26 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:27.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:26 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-06T12:16:27.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:26 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-06T12:16:27.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:26 smithi092 ceph-mon[26954]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:16:27.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:26 smithi190 ceph-mon[33798]: pgmap v22: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 828 KiB/s rd, 811 KiB/s wr, 143 op/s 2024-05-06T12:16:27.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:26 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:27.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:26 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-06T12:16:27.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:26 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-06T12:16:27.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:26 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:16:28.196 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:27 smithi092 ceph-mon[26954]: Upgrade: Updating mon.smithi092 2024-05-06T12:16:28.197 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:27 smithi092 ceph-mon[26954]: Deploying daemon mon.smithi092 on smithi092 2024-05-06T12:16:28.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:27 smithi190 ceph-mon[33798]: Upgrade: Updating mon.smithi092 2024-05-06T12:16:28.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:27 smithi190 ceph-mon[33798]: Deploying daemon mon.smithi092 on smithi092 2024-05-06T12:16:28.743 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:28 smithi092 systemd[1]: Stopping Ceph mon.smithi092 for aee1d912-0ba0-11ef-bc97-c7b262605968... 2024-05-06T12:16:29.185 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:28 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi092[26950]: 2024-05-06T12:16:28.740+0000 7fae55374700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi092 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-06T12:16:29.185 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:28 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi092[26950]: 2024-05-06T12:16:28.740+0000 7fae55374700 -1 mon.smithi092@0(leader) e2 *** Got Signal Terminated *** 2024-05-06T12:16:29.185 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:28 smithi092 podman[91759]: 2024-05-06 12:16:28.893773985 +0000 UTC m=+0.268176499 container died df5b322e971909a283945f048d107c3733ddc2ad3fefc6ae30f7b0fafadef4c6 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi092, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=quincy-e81df42, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-06T12:16:29.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:29 smithi092 podman[91759]: 2024-05-06 12:16:29.357780521 +0000 UTC m=+0.732183031 container cleanup df5b322e971909a283945f048d107c3733ddc2ad3fefc6ae30f7b0fafadef4c6 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi092, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, RELEASE=quincy-e81df42, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-05-06T12:16:29.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:29 smithi092 bash[91759]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi092 2024-05-06T12:16:29.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:29 smithi092 podman[91784]: 2024-05-06 12:16:29.61765072 +0000 UTC m=+0.726092633 container remove df5b322e971909a283945f048d107c3733ddc2ad3fefc6ae30f7b0fafadef4c6 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi092, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=quincy-e81df42, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-06T12:16:29.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:29 smithi092 systemd[1]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@mon.smithi092.service: Deactivated successfully. 2024-05-06T12:16:29.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:29 smithi092 systemd[1]: Stopped Ceph mon.smithi092 for aee1d912-0ba0-11ef-bc97-c7b262605968. 2024-05-06T12:16:29.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:29 smithi092 systemd[1]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@mon.smithi092.service: Consumed 10.944s CPU time. 2024-05-06T12:16:30.352 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:29 smithi092 systemd[1]: Starting Ceph mon.smithi092 for aee1d912-0ba0-11ef-bc97-c7b262605968... 2024-05-06T12:16:30.352 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 podman[91914]: 2024-05-06 12:16:30.107614877 +0000 UTC m=+0.018098164 image pull e4ce8ad5d0c4b9ea60bc5f6b8b0fdb5efbe59bf7771fb0920fc2200e6ae546b0 quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb 2024-05-06T12:16:30.352 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 podman[91914]: 2024-05-06 12:16:30.235740951 +0000 UTC m=+0.146224231 container create ac07f7158087a05c9a6a634101382e16d869dfd83d110786151d67cee252b2f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi092, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-70ce953, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-06T12:16:30.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 podman[91914]: 2024-05-06 12:16:30.463202971 +0000 UTC m=+0.373686251 container init ac07f7158087a05c9a6a634101382e16d869dfd83d110786151d67cee252b2f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi092, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-70ce953, org.label-schema.build-date=20240430) 2024-05-06T12:16:30.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 podman[91914]: 2024-05-06 12:16:30.466491789 +0000 UTC m=+0.376975073 container start ac07f7158087a05c9a6a634101382e16d869dfd83d110786151d67cee252b2f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi092, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-70ce953, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=) 2024-05-06T12:16:30.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: set uid:gid to 167:167 (ceph:ceph) 2024-05-06T12:16:30.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev), process ceph-mon, pid 2 2024-05-06T12:16:30.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: pidfile_write: ignore empty --pid-file 2024-05-06T12:16:30.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: load: jerasure load: lrc 2024-05-06T12:16:30.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: RocksDB version: 7.9.2 2024-05-06T12:16:30.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Git sha 0 2024-05-06T12:16:30.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Compile date 2024-05-03 15:08:47 2024-05-06T12:16:30.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: DB SUMMARY 2024-05-06T12:16:30.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: DB Session ID: 83UA9T6F0TL1XIXKP9M4 2024-05-06T12:16:30.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: CURRENT file: CURRENT 2024-05-06T12:16:30.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: IDENTITY file: IDENTITY 2024-05-06T12:16:30.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: MANIFEST file: MANIFEST-000015 size: 972 Bytes 2024-05-06T12:16:30.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi092/store.db dir, Total Num: 1, files: 000030.sst 2024-05-06T12:16:30.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi092/store.db: 000028.log size: 2537579 ; 2024-05-06T12:16:30.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.error_if_exists: 0 2024-05-06T12:16:30.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.create_if_missing: 0 2024-05-06T12:16:30.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.paranoid_checks: 1 2024-05-06T12:16:30.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-06T12:16:30.725 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-06T12:16:30.725 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-06T12:16:30.725 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.env: 0x563c24466c60 2024-05-06T12:16:30.725 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.fs: PosixFileSystem 2024-05-06T12:16:30.725 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.info_log: 0x563c2548d980 2024-05-06T12:16:30.725 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.max_file_opening_threads: 16 2024-05-06T12:16:30.725 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.statistics: (nil) 2024-05-06T12:16:30.725 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.use_fsync: 0 2024-05-06T12:16:30.725 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.max_log_file_size: 0 2024-05-06T12:16:30.725 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-06T12:16:30.725 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-06T12:16:30.725 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.keep_log_file_num: 1000 2024-05-06T12:16:30.725 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.recycle_log_file_num: 0 2024-05-06T12:16:30.725 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.allow_fallocate: 1 2024-05-06T12:16:30.726 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.allow_mmap_reads: 0 2024-05-06T12:16:30.726 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.allow_mmap_writes: 0 2024-05-06T12:16:30.726 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.use_direct_reads: 0 2024-05-06T12:16:30.726 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-06T12:16:30.726 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.create_missing_column_families: 0 2024-05-06T12:16:30.726 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.db_log_dir: 2024-05-06T12:16:30.726 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.wal_dir: 2024-05-06T12:16:30.726 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-06T12:16:30.726 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-06T12:16:30.726 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-06T12:16:30.726 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-06T12:16:30.726 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-06T12:16:30.726 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-06T12:16:30.726 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.advise_random_on_open: 1 2024-05-06T12:16:30.727 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.db_write_buffer_size: 0 2024-05-06T12:16:30.727 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.write_buffer_manager: 0x563c254601e0 2024-05-06T12:16:30.727 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-06T12:16:30.727 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-06T12:16:30.727 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-06T12:16:30.727 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.rate_limiter: (nil) 2024-05-06T12:16:30.727 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-06T12:16:30.727 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.wal_recovery_mode: 2 2024-05-06T12:16:30.727 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.enable_thread_tracking: 0 2024-05-06T12:16:30.727 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.enable_pipelined_write: 0 2024-05-06T12:16:30.727 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.unordered_write: 0 2024-05-06T12:16:30.727 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-06T12:16:30.727 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-06T12:16:30.727 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-06T12:16:30.727 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-06T12:16:30.727 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.row_cache: None 2024-05-06T12:16:30.728 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.wal_filter: None 2024-05-06T12:16:30.729 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-06T12:16:30.729 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.allow_ingest_behind: 0 2024-05-06T12:16:30.729 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.two_write_queues: 0 2024-05-06T12:16:30.729 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.manual_wal_flush: 0 2024-05-06T12:16:30.729 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.wal_compression: 0 2024-05-06T12:16:30.729 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.atomic_flush: 0 2024-05-06T12:16:30.729 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-06T12:16:30.729 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-06T12:16:30.730 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-06T12:16:30.730 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.log_readahead_size: 0 2024-05-06T12:16:30.730 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-06T12:16:30.730 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.best_efforts_recovery: 0 2024-05-06T12:16:30.730 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-06T12:16:30.730 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-06T12:16:30.730 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.allow_data_in_errors: 0 2024-05-06T12:16:30.730 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.db_host_id: __hostname__ 2024-05-06T12:16:30.730 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-06T12:16:30.730 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.max_background_jobs: 2 2024-05-06T12:16:30.730 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.max_background_compactions: -1 2024-05-06T12:16:30.730 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.max_subcompactions: 1 2024-05-06T12:16:30.730 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-06T12:16:30.731 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-06T12:16:30.731 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-06T12:16:30.731 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.max_total_wal_size: 0 2024-05-06T12:16:30.731 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-06T12:16:30.731 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-06T12:16:30.731 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-06T12:16:30.731 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-06T12:16:30.731 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.max_open_files: -1 2024-05-06T12:16:30.731 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.bytes_per_sync: 0 2024-05-06T12:16:30.731 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-06T12:16:30.731 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-06T12:16:30.731 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.compaction_readahead_size: 0 2024-05-06T12:16:30.731 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.max_background_flushes: -1 2024-05-06T12:16:30.731 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Compression algorithms supported: 2024-05-06T12:16:30.732 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-06T12:16:30.732 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: kZSTD supported: 0 2024-05-06T12:16:30.732 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: kXpressCompression supported: 0 2024-05-06T12:16:30.732 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: kLZ4HCCompression supported: 1 2024-05-06T12:16:30.732 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: kZlibCompression supported: 1 2024-05-06T12:16:30.732 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: kSnappyCompression supported: 1 2024-05-06T12:16:30.732 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: kLZ4Compression supported: 1 2024-05-06T12:16:30.732 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: kBZip2Compression supported: 0 2024-05-06T12:16:30.732 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-06T12:16:30.732 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-06T12:16:30.732 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi092/store.db/MANIFEST-000015 2024-05-06T12:16:30.732 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-06T12:16:30.732 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-06T12:16:30.732 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.merge_operator: 2024-05-06T12:16:30.732 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.compaction_filter: None 2024-05-06T12:16:30.732 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.compaction_filter_factory: None 2024-05-06T12:16:30.733 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.sst_partitioner_factory: None 2024-05-06T12:16:30.733 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-06T12:16:30.733 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-06T12:16:30.733 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563c2548da80) 2024-05-06T12:16:30.733 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: cache_index_and_filter_blocks: 1 2024-05-06T12:16:30.733 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-06T12:16:30.733 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-06T12:16:30.733 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: pin_top_level_index_and_filter: 1 2024-05-06T12:16:30.733 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: index_type: 0 2024-05-06T12:16:30.733 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: data_block_index_type: 0 2024-05-06T12:16:30.733 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: index_shortening: 1 2024-05-06T12:16:30.733 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-06T12:16:30.733 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: checksum: 4 2024-05-06T12:16:30.733 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: no_block_cache: 0 2024-05-06T12:16:30.733 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: block_cache: 0x563c25491090 2024-05-06T12:16:30.734 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: block_cache_name: BinnedLRUCache 2024-05-06T12:16:30.734 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: block_cache_options: 2024-05-06T12:16:30.734 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: capacity : 536870912 2024-05-06T12:16:30.734 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: num_shard_bits : 4 2024-05-06T12:16:30.734 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: strict_capacity_limit : 0 2024-05-06T12:16:30.734 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: high_pri_pool_ratio: 0.000 2024-05-06T12:16:30.734 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: block_cache_compressed: (nil) 2024-05-06T12:16:30.734 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: persistent_cache: (nil) 2024-05-06T12:16:30.734 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: block_size: 4096 2024-05-06T12:16:30.734 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: block_size_deviation: 10 2024-05-06T12:16:30.734 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: block_restart_interval: 16 2024-05-06T12:16:30.734 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: index_block_restart_interval: 1 2024-05-06T12:16:30.734 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: metadata_block_size: 4096 2024-05-06T12:16:30.735 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: partition_filters: 0 2024-05-06T12:16:30.735 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: use_delta_encoding: 1 2024-05-06T12:16:30.735 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: filter_policy: bloomfilter 2024-05-06T12:16:30.735 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: whole_key_filtering: 1 2024-05-06T12:16:30.735 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: verify_compression: 0 2024-05-06T12:16:30.735 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: read_amp_bytes_per_bit: 0 2024-05-06T12:16:30.735 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: format_version: 5 2024-05-06T12:16:30.735 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: enable_index_compression: 1 2024-05-06T12:16:30.735 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: block_align: 0 2024-05-06T12:16:30.735 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: max_auto_readahead_size: 262144 2024-05-06T12:16:30.735 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: prepopulate_block_cache: 0 2024-05-06T12:16:30.735 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: initial_auto_readahead_size: 8192 2024-05-06T12:16:30.735 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: num_file_reads_for_auto_readahead: 2 2024-05-06T12:16:30.735 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.write_buffer_size: 33554432 2024-05-06T12:16:30.735 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.max_write_buffer_number: 2 2024-05-06T12:16:30.736 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.compression: NoCompression 2024-05-06T12:16:30.736 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.bottommost_compression: Disabled 2024-05-06T12:16:30.736 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.prefix_extractor: nullptr 2024-05-06T12:16:30.736 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-06T12:16:30.736 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.num_levels: 7 2024-05-06T12:16:30.736 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-06T12:16:30.736 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-06T12:16:30.736 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-06T12:16:30.736 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-06T12:16:30.736 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-06T12:16:30.736 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-06T12:16:30.736 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-06T12:16:30.736 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-06T12:16:30.736 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-06T12:16:30.736 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-06T12:16:30.737 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-06T12:16:30.737 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-06T12:16:30.737 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-06T12:16:30.737 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.compression_opts.level: 32767 2024-05-06T12:16:30.737 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.compression_opts.strategy: 0 2024-05-06T12:16:30.737 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-06T12:16:30.737 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-06T12:16:30.737 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-06T12:16:30.737 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-06T12:16:30.737 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.compression_opts.enabled: false 2024-05-06T12:16:30.737 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-06T12:16:30.737 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-06T12:16:30.737 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-06T12:16:30.737 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-06T12:16:30.738 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.target_file_size_base: 67108864 2024-05-06T12:16:30.738 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-06T12:16:30.738 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-06T12:16:30.738 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-06T12:16:30.738 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-06T12:16:30.738 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-06T12:16:30.738 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-06T12:16:30.738 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-06T12:16:30.738 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-06T12:16:30.738 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-06T12:16:30.738 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-06T12:16:30.738 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-06T12:16:30.738 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-06T12:16:30.738 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-06T12:16:30.738 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-06T12:16:30.739 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.arena_block_size: 1048576 2024-05-06T12:16:30.739 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-06T12:16:30.739 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-06T12:16:30.739 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.disable_auto_compactions: 0 2024-05-06T12:16:30.739 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-06T12:16:30.739 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-06T12:16:30.739 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-06T12:16:30.739 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-06T12:16:30.739 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-06T12:16:30.739 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-06T12:16:30.739 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-06T12:16:30.739 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-06T12:16:30.739 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-06T12:16:30.739 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-06T12:16:30.740 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.table_properties_collectors: 2024-05-06T12:16:30.740 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.inplace_update_support: 0 2024-05-06T12:16:30.740 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-06T12:16:30.740 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-06T12:16:30.740 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-06T12:16:30.740 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-06T12:16:30.740 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.bloom_locality: 0 2024-05-06T12:16:30.740 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.max_successive_merges: 0 2024-05-06T12:16:30.740 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-06T12:16:30.740 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.paranoid_file_checks: 0 2024-05-06T12:16:30.740 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.force_consistency_checks: 1 2024-05-06T12:16:30.740 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.report_bg_io_stats: 0 2024-05-06T12:16:30.740 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.ttl: 2592000 2024-05-06T12:16:30.740 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-06T12:16:30.741 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-06T12:16:30.741 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-06T12:16:30.741 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.enable_blob_files: false 2024-05-06T12:16:30.741 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.min_blob_size: 0 2024-05-06T12:16:30.741 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.blob_file_size: 268435456 2024-05-06T12:16:30.741 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-06T12:16:30.741 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-06T12:16:30.741 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-06T12:16:30.741 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-06T12:16:30.741 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-06T12:16:30.741 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.blob_file_starting_level: 0 2024-05-06T12:16:30.741 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-06T12:16:30.741 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 30.sst 2024-05-06T12:16:30.741 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-05-06T12:16:30.742 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi092/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 32, last_sequence is 10354, log_number is 28,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-06T12:16:30.742 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 28 2024-05-06T12:16:30.742 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: bfe465a0-d978-484c-93f8-8fdf204d8d29 2024-05-06T12:16:30.742 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714997790495950, "job": 1, "event": "recovery_started", "wal_files": [28]} 2024-05-06T12:16:30.742 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #28 mode 2 2024-05-06T12:16:30.742 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714997790502959, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 33, "file_size": 2314364, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 10355, "largest_seqno": 11336, "table_properties": {"data_size": 2309197, "index_size": 2996, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1157, "raw_key_size": 10539, "raw_average_key_size": 23, "raw_value_size": 2299411, "raw_average_value_size": 5132, "num_data_blocks": 137, "num_entries": 448, "num_filter_entries": 448, "num_deletions": 2, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714997790, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "bfe465a0-d978-484c-93f8-8fdf204d8d29", "db_session_id": "83UA9T6F0TL1XIXKP9M4", "orig_file_number": 33, "seqno_to_time_mapping": "N/A"}} 2024-05-06T12:16:30.742 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714997790503032, "job": 1, "event": "recovery_finished"} 2024-05-06T12:16:30.742 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: [db/version_set.cc:5047] Creating manifest 35 2024-05-06T12:16:30.742 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-05-06T12:16:30.742 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi092/store.db/000028.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-06T12:16:30.742 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x563c25582000 2024-05-06T12:16:30.742 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: DB pointer 0x563c2556c000 2024-05-06T12:16:30.742 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-06T12:16:30.743 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-06T12:16:30.743 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: ** DB Stats ** 2024-05-06T12:16:30.743 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-06T12:16:30.743 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-06T12:16:30.743 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-06T12:16:30.743 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-06T12:16:30.743 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-06T12:16:30.743 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-06T12:16:30.743 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-06T12:16:30.743 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: 2024-05-06T12:16:30.743 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: ** Compaction Stats [default] ** 2024-05-06T12:16:30.743 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-06T12:16:30.743 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-06T12:16:30.743 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: L0 1/0 2.21 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 615.5 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-05-06T12:16:30.743 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: L6 1/0 5.24 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-05-06T12:16:30.744 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Sum 2/0 7.45 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 615.5 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-05-06T12:16:30.744 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 615.5 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-05-06T12:16:30.744 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: 2024-05-06T12:16:30.744 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: ** Compaction Stats [default] ** 2024-05-06T12:16:30.744 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-06T12:16:30.744 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-06T12:16:30.744 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 615.5 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-05-06T12:16:30.744 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: 2024-05-06T12:16:30.744 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-06T12:16:30.744 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: 2024-05-06T12:16:30.744 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-06T12:16:30.744 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-05-06T12:16:30.744 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-06T12:16:30.744 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-06T12:16:30.744 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-06T12:16:30.745 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-06T12:16:30.745 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Cumulative compaction: 0.00 GB write, 159.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-06T12:16:30.745 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Interval compaction: 0.00 GB write, 159.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-06T12:16:30.745 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-06T12:16:30.745 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Block cache BinnedLRUCache@0x563c25491090#2 capacity: 512.00 MB usage: 4.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-05-06T12:16:30.745 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.28 KB,0.000244379%) IndexBlock(1,3.08 KB,0.000587106%) Misc(1,0.00 KB,0%) 2024-05-06T12:16:30.745 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: 2024-05-06T12:16:30.745 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-06T12:16:30.745 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: starting mon.smithi092 rank 0 at public addrs [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] at bind addrs [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi092 fsid aee1d912-0ba0-11ef-bc97-c7b262605968 2024-05-06T12:16:30.745 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: mon.smithi092@-1(???) e2 preinit fsid aee1d912-0ba0-11ef-bc97-c7b262605968 2024-05-06T12:16:30.745 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: mon.smithi092@-1(???).mds e20 new map 2024-05-06T12:16:30.745 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: mon.smithi092@-1(???).mds e20 print_map 2024-05-06T12:16:30.745 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: e20 2024-05-06T12:16:30.745 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-06T12:16:30.745 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:16:30.746 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: legacy client fscid: 1 2024-05-06T12:16:30.746 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: 2024-05-06T12:16:30.746 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: Filesystem 'cephfs' (1) 2024-05-06T12:16:30.746 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: fs_name cephfs 2024-05-06T12:16:30.746 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: epoch 20 2024-05-06T12:16:30.746 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-06T12:16:30.746 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: created 2024-05-06T12:11:44.952671+0000 2024-05-06T12:16:30.746 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: modified 2024-05-06T12:15:07.568167+0000 2024-05-06T12:16:30.746 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: tableserver 0 2024-05-06T12:16:30.746 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: root 0 2024-05-06T12:16:30.746 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: session_timeout 60 2024-05-06T12:16:30.746 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: session_autoclose 300 2024-05-06T12:16:30.746 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: max_file_size 1099511627776 2024-05-06T12:16:30.746 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: max_xattr_size 65536 2024-05-06T12:16:30.747 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: required_client_features {} 2024-05-06T12:16:30.747 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: last_failure 0 2024-05-06T12:16:30.747 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: last_failure_osd_epoch 46 2024-05-06T12:16:30.747 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:16:30.747 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: max_mds 2 2024-05-06T12:16:30.747 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: in 0,1 2024-05-06T12:16:30.747 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: up {0=14472,1=24267} 2024-05-06T12:16:30.747 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: failed 2024-05-06T12:16:30.747 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: damaged 2024-05-06T12:16:30.747 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: stopped 2024-05-06T12:16:30.747 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: data_pools [3] 2024-05-06T12:16:30.747 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: metadata_pool 2 2024-05-06T12:16:30.747 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: inline_data enabled 2024-05-06T12:16:30.747 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: balancer 2024-05-06T12:16:30.748 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: bal_rank_mask -1 2024-05-06T12:16:30.748 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: standby_count_wanted 1 2024-05-06T12:16:30.748 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: qdb_cluster leader: 0 members: 2024-05-06T12:16:30.748 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: [mds.cephfs.smithi092.rosbll{0:14472} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.92:6828/1663253598,v1:172.21.15.92:6829/1663253598] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:16:30.748 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: [mds.cephfs.smithi092.ijsqgp{0:14484} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.92:6826/3613441054,v1:172.21.15.92:6827/3613441054] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:16:30.748 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: [mds.cephfs.smithi190.byjvgf{1:24267} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.190:6824/1565151699,v1:172.21.15.190:6825/1565151699] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:16:30.748 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: [mds.cephfs.smithi190.idzoff{1:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.190:6826/2453728380,v1:172.21.15.190:6827/2453728380] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:16:30.748 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: 2024-05-06T12:16:30.748 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout: 2024-05-06T12:16:30.748 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: mon.smithi092@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-05-06T12:16:30.748 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: mon.smithi092@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-05-06T12:16:30.748 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: mon.smithi092@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-05-06T12:16:30.748 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: mon.smithi092@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-05-06T12:16:30.748 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 ceph-mon[91933]: mon.smithi092@-1(???).paxosservice(auth 1..20) refresh upgraded, format 0 -> 3 2024-05-06T12:16:30.748 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 bash[91914]: ac07f7158087a05c9a6a634101382e16d869dfd83d110786151d67cee252b2f9 2024-05-06T12:16:30.748 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:30 smithi092 systemd[1]: Started Ceph mon.smithi092 for aee1d912-0ba0-11ef-bc97-c7b262605968. 2024-05-06T12:16:32.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:31 smithi092 ceph-mon[91933]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi092"}]: dispatch 2024-05-06T12:16:32.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:31 smithi092 ceph-mon[91933]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi190"}]: dispatch 2024-05-06T12:16:32.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:31 smithi092 ceph-mon[91933]: pgmap v24: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 413 KiB/s rd, 344 KiB/s wr, 73 op/s 2024-05-06T12:16:32.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:31 smithi092 ceph-mon[91933]: mon.smithi092 calling monitor election 2024-05-06T12:16:32.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:31 smithi092 ceph-mon[91933]: mon.smithi092 is new leader, mons smithi092,smithi190 in quorum (ranks 0,1) 2024-05-06T12:16:32.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:31 smithi092 ceph-mon[91933]: monmap epoch 2 2024-05-06T12:16:32.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:31 smithi092 ceph-mon[91933]: fsid aee1d912-0ba0-11ef-bc97-c7b262605968 2024-05-06T12:16:32.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:31 smithi092 ceph-mon[91933]: last_changed 2024-05-06T12:07:26.387383+0000 2024-05-06T12:16:32.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:31 smithi092 ceph-mon[91933]: created 2024-05-06T12:04:48.255830+0000 2024-05-06T12:16:32.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:31 smithi092 ceph-mon[91933]: min_mon_release 17 (quincy) 2024-05-06T12:16:32.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:31 smithi092 ceph-mon[91933]: election_strategy: 1 2024-05-06T12:16:32.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:31 smithi092 ceph-mon[91933]: 0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.smithi092 2024-05-06T12:16:32.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:31 smithi092 ceph-mon[91933]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.smithi190 2024-05-06T12:16:32.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:31 smithi092 ceph-mon[91933]: fsmap cephfs:2 {0=cephfs.smithi092.rosbll=up:active,1=cephfs.smithi190.byjvgf=up:active} 2 up:standby-replay 2024-05-06T12:16:32.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:31 smithi092 ceph-mon[91933]: osdmap e49: 6 total, 6 up, 6 in 2024-05-06T12:16:32.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:31 smithi092 ceph-mon[91933]: mgrmap e33: smithi092.gyzeqb(active, since 44s), standbys: smithi190.kczxkh 2024-05-06T12:16:32.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:31 smithi092 ceph-mon[91933]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-06T12:16:32.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:31 smithi092 ceph-mon[91933]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-06T12:16:32.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:31 smithi092 ceph-mon[91933]: fs cephfs has deprecated feature inline_data enabled. 2024-05-06T12:16:32.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:31 smithi092 ceph-mon[91933]: mgrmap e34: smithi092.gyzeqb(active, since 44s), standbys: smithi190.kczxkh 2024-05-06T12:16:32.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:31 smithi092 ceph-mon[91933]: from='mgr.14748 ' entity='' 2024-05-06T12:16:32.299 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:31 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi092"}]: dispatch 2024-05-06T12:16:32.299 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:31 smithi190 ceph-mon[33798]: from='mgr.14748 172.21.15.92:0/2011426605' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi190"}]: dispatch 2024-05-06T12:16:32.299 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:31 smithi190 ceph-mon[33798]: pgmap v24: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 413 KiB/s rd, 344 KiB/s wr, 73 op/s 2024-05-06T12:16:32.299 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:31 smithi190 ceph-mon[33798]: mon.smithi092 calling monitor election 2024-05-06T12:16:32.299 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:31 smithi190 ceph-mon[33798]: mon.smithi092 is new leader, mons smithi092,smithi190 in quorum (ranks 0,1) 2024-05-06T12:16:32.299 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:31 smithi190 ceph-mon[33798]: monmap epoch 2 2024-05-06T12:16:32.299 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:31 smithi190 ceph-mon[33798]: fsid aee1d912-0ba0-11ef-bc97-c7b262605968 2024-05-06T12:16:32.299 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:31 smithi190 ceph-mon[33798]: last_changed 2024-05-06T12:07:26.387383+0000 2024-05-06T12:16:32.299 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:31 smithi190 ceph-mon[33798]: created 2024-05-06T12:04:48.255830+0000 2024-05-06T12:16:32.301 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:31 smithi190 ceph-mon[33798]: min_mon_release 17 (quincy) 2024-05-06T12:16:32.301 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:31 smithi190 ceph-mon[33798]: election_strategy: 1 2024-05-06T12:16:32.301 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:31 smithi190 ceph-mon[33798]: 0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.smithi092 2024-05-06T12:16:32.302 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:31 smithi190 ceph-mon[33798]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.smithi190 2024-05-06T12:16:32.302 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:31 smithi190 ceph-mon[33798]: fsmap cephfs:2 {0=cephfs.smithi092.rosbll=up:active,1=cephfs.smithi190.byjvgf=up:active} 2 up:standby-replay 2024-05-06T12:16:32.302 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:31 smithi190 ceph-mon[33798]: osdmap e49: 6 total, 6 up, 6 in 2024-05-06T12:16:32.302 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:31 smithi190 ceph-mon[33798]: mgrmap e33: smithi092.gyzeqb(active, since 44s), standbys: smithi190.kczxkh 2024-05-06T12:16:32.302 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:31 smithi190 ceph-mon[33798]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-06T12:16:32.302 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:31 smithi190 ceph-mon[33798]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-06T12:16:32.302 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:31 smithi190 ceph-mon[33798]: fs cephfs has deprecated feature inline_data enabled. 2024-05-06T12:16:32.302 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:31 smithi190 ceph-mon[33798]: mgrmap e34: smithi092.gyzeqb(active, since 44s), standbys: smithi190.kczxkh 2024-05-06T12:16:32.302 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:31 smithi190 ceph-mon[33798]: from='mgr.14748 ' entity='' 2024-05-06T12:16:35.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.332+0000 7f2618adb700 1 -- 172.21.15.92:0/1426141007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26140fbcb0 msgr2=0x7f26140fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:35.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.332+0000 7f2618adb700 1 --2- 172.21.15.92:0/1426141007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26140fbcb0 0x7f26140fc0c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2608004940 tx=0x7f2608009e30 comp rx=0 tx=0).stop 2024-05-06T12:16:35.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.332+0000 7f2618adb700 1 -- 172.21.15.92:0/1426141007 shutdown_connections 2024-05-06T12:16:35.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.332+0000 7f2618adb700 1 --2- 172.21.15.92:0/1426141007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26140fcf60 0x7f26140fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:35.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.332+0000 7f2618adb700 1 --2- 172.21.15.92:0/1426141007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26140fbcb0 0x7f26140fc0c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:35.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.332+0000 7f2618adb700 1 -- 172.21.15.92:0/1426141007 >> 172.21.15.92:0/1426141007 conn(0x7f26140f7430 msgr2=0x7f26140f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:16:35.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.332+0000 7f2618adb700 1 -- 172.21.15.92:0/1426141007 shutdown_connections 2024-05-06T12:16:35.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.332+0000 7f2618adb700 1 -- 172.21.15.92:0/1426141007 wait complete. 2024-05-06T12:16:35.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.333+0000 7f2618adb700 1 Processor -- start 2024-05-06T12:16:35.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.333+0000 7f2618adb700 1 -- start start 2024-05-06T12:16:35.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.333+0000 7f2618adb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26140fbcb0 0x7f261410da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:35.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.333+0000 7f2618adb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26140fcf60 0x7f261410df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:35.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.333+0000 7f2618adb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261410e590 con 0x7f26140fbcb0 2024-05-06T12:16:35.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.333+0000 7f2618adb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261410ed90 con 0x7f26140fcf60 2024-05-06T12:16:35.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.333+0000 7f261259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26140fbcb0 0x7f261410da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:35.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.333+0000 7f261259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26140fbcb0 0x7f261410da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46664/0 (socket says 172.21.15.92:46664) 2024-05-06T12:16:35.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.333+0000 7f261259c700 1 -- 172.21.15.92:0/3245512712 learned_addr learned my addr 172.21.15.92:0/3245512712 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:16:35.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.333+0000 7f2611d9b700 1 --2- 172.21.15.92:0/3245512712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26140fcf60 0x7f261410df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:35.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.333+0000 7f261259c700 1 -- 172.21.15.92:0/3245512712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26140fcf60 msgr2=0x7f261410df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:35.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.333+0000 7f261259c700 1 --2- 172.21.15.92:0/3245512712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26140fcf60 0x7f261410df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:35.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.333+0000 7f261259c700 1 -- 172.21.15.92:0/3245512712 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2608005040 con 0x7f26140fbcb0 2024-05-06T12:16:35.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.334+0000 7f261259c700 1 --2- 172.21.15.92:0/3245512712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26140fbcb0 0x7f261410da50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f260800b3b0 tx=0x7f2608009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:16:35.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.334+0000 7f2602ffd700 1 -- 172.21.15.92:0/3245512712 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f260800e040 con 0x7f26140fbcb0 2024-05-06T12:16:35.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.334+0000 7f2618adb700 1 -- 172.21.15.92:0/3245512712 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f261410f010 con 0x7f26140fbcb0 2024-05-06T12:16:35.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.334+0000 7f2602ffd700 1 -- 172.21.15.92:0/3245512712 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2608004290 con 0x7f26140fbcb0 2024-05-06T12:16:35.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.334+0000 7f2602ffd700 1 -- 172.21.15.92:0/3245512712 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f260801c3f0 con 0x7f26140fbcb0 2024-05-06T12:16:35.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.334+0000 7f2618adb700 1 -- 172.21.15.92:0/3245512712 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f261410f550 con 0x7f26140fbcb0 2024-05-06T12:16:35.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.335+0000 7f2618adb700 1 -- 172.21.15.92:0/3245512712 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2614101af0 con 0x7f26140fbcb0 2024-05-06T12:16:35.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.336+0000 7f2602ffd700 1 -- 172.21.15.92:0/3245512712 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 34) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f260800bbf0 con 0x7f26140fbcb0 2024-05-06T12:16:35.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.336+0000 7f2602ffd700 1 --2- 172.21.15.92:0/3245512712 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f25fc06e260 0x7f25fc070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:35.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.336+0000 7f2611d9b700 1 -- 172.21.15.92:0/3245512712 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f25fc06e260 msgr2=0x7f25fc070710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/648200129 2024-05-06T12:16:35.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.336+0000 7f2602ffd700 1 -- 172.21.15.92:0/3245512712 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f260808ef80 con 0x7f26140fbcb0 2024-05-06T12:16:35.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.336+0000 7f2611d9b700 1 --2- 172.21.15.92:0/3245512712 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f25fc06e260 0x7f25fc070710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-06T12:16:35.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.341+0000 7f2602ffd700 1 -- 172.21.15.92:0/3245512712 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f260805bde0 con 0x7f26140fbcb0 2024-05-06T12:16:35.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.537+0000 7f2611d9b700 1 -- 172.21.15.92:0/3245512712 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f25fc06e260 msgr2=0x7f25fc070710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/648200129 2024-05-06T12:16:35.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.537+0000 7f2611d9b700 1 --2- 172.21.15.92:0/3245512712 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f25fc06e260 0x7f25fc070710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-06T12:16:35.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.563+0000 7f2618adb700 1 -- 172.21.15.92:0/3245512712 --> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f261410f770 con 0x7f25fc06e260 2024-05-06T12:16:35.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.938+0000 7f2611d9b700 1 -- 172.21.15.92:0/3245512712 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f25fc06e260 msgr2=0x7f25fc070710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/648200129 2024-05-06T12:16:35.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:35.938+0000 7f2611d9b700 1 --2- 172.21.15.92:0/3245512712 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f25fc06e260 0x7f25fc070710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-06T12:16:36.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:36.739+0000 7f2611d9b700 1 -- 172.21.15.92:0/3245512712 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f25fc06e260 msgr2=0x7f25fc070710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/648200129 2024-05-06T12:16:36.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:36.739+0000 7f2611d9b700 1 --2- 172.21.15.92:0/3245512712 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f25fc06e260 0x7f25fc070710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-06T12:16:38.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:38.340+0000 7f2611d9b700 1 -- 172.21.15.92:0/3245512712 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f25fc06e260 msgr2=0x7f25fc070710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/648200129 2024-05-06T12:16:38.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:38.340+0000 7f2611d9b700 1 --2- 172.21.15.92:0/3245512712 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f25fc06e260 0x7f25fc070710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-06T12:16:38.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:38.781+0000 7f2602ffd700 1 -- 172.21.15.92:0/3245512712 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 35) v1 ==== 91828+0+0 (secure 0 0 0) 0x7f260808d320 con 0x7f26140fbcb0 2024-05-06T12:16:38.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:38.781+0000 7f2602ffd700 1 -- 172.21.15.92:0/3245512712 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f25fc06e260 msgr2=0x7f25fc070710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-06T12:16:38.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:38.781+0000 7f2602ffd700 1 --2- 172.21.15.92:0/3245512712 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f25fc06e260 0x7f25fc070710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:38.951 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:38 smithi092 ceph-mon[91933]: Active manager daemon smithi092.gyzeqb restarted 2024-05-06T12:16:38.951 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:38 smithi092 ceph-mon[91933]: Activating manager daemon smithi092.gyzeqb 2024-05-06T12:16:38.951 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:38 smithi092 ceph-mon[91933]: osdmap e50: 6 total, 6 up, 6 in 2024-05-06T12:16:38.952 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:38 smithi092 ceph-mon[91933]: mgrmap e35: smithi092.gyzeqb(active, starting, since 0.00546952s), standbys: smithi190.kczxkh 2024-05-06T12:16:38.952 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:38 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi092"}]: dispatch 2024-05-06T12:16:38.952 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:38 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi190"}]: dispatch 2024-05-06T12:16:38.952 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:38 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi092.rosbll"}]: dispatch 2024-05-06T12:16:38.952 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:38 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi092.ijsqgp"}]: dispatch 2024-05-06T12:16:38.953 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:38 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi190.byjvgf"}]: dispatch 2024-05-06T12:16:38.953 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:38 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi190.idzoff"}]: dispatch 2024-05-06T12:16:38.953 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:38 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mgr metadata", "who": "smithi092.gyzeqb", "id": "smithi092.gyzeqb"}]: dispatch 2024-05-06T12:16:38.953 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:38 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mgr metadata", "who": "smithi190.kczxkh", "id": "smithi190.kczxkh"}]: dispatch 2024-05-06T12:16:38.953 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:38 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-06T12:16:38.953 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:38 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-06T12:16:38.954 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:38 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-06T12:16:38.954 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:38 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-06T12:16:38.954 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:38 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-06T12:16:38.954 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:38 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-06T12:16:38.954 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:38 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-06T12:16:38.954 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:38 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-06T12:16:38.955 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:38 smithi092 ceph-mon[91933]: Standby manager daemon smithi190.kczxkh restarted 2024-05-06T12:16:38.955 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:38 smithi092 ceph-mon[91933]: Standby manager daemon smithi190.kczxkh started 2024-05-06T12:16:38.955 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:38 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-06T12:16:38.955 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:38 smithi092 ceph-mon[91933]: from='mgr.? 172.21.15.190:0/1432540833' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi190.kczxkh/crt"}]: dispatch 2024-05-06T12:16:38.955 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:38 smithi092 ceph-mon[91933]: from='mgr.? 172.21.15.190:0/1432540833' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-06T12:16:38.955 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:38 smithi092 ceph-mon[91933]: from='mgr.? 172.21.15.190:0/1432540833' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi190.kczxkh/key"}]: dispatch 2024-05-06T12:16:38.956 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:38 smithi092 ceph-mon[91933]: from='mgr.? 172.21.15.190:0/1432540833' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-06T12:16:38.956 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:38 smithi092 ceph-mon[91933]: Manager daemon smithi092.gyzeqb is now available 2024-05-06T12:16:39.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:38 smithi190 ceph-mon[33798]: Active manager daemon smithi092.gyzeqb restarted 2024-05-06T12:16:39.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:38 smithi190 ceph-mon[33798]: Activating manager daemon smithi092.gyzeqb 2024-05-06T12:16:39.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:38 smithi190 ceph-mon[33798]: osdmap e50: 6 total, 6 up, 6 in 2024-05-06T12:16:39.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:38 smithi190 ceph-mon[33798]: mgrmap e35: smithi092.gyzeqb(active, starting, since 0.00546952s), standbys: smithi190.kczxkh 2024-05-06T12:16:39.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:38 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi092"}]: dispatch 2024-05-06T12:16:39.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:38 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi190"}]: dispatch 2024-05-06T12:16:39.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:38 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi092.rosbll"}]: dispatch 2024-05-06T12:16:39.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:38 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi092.ijsqgp"}]: dispatch 2024-05-06T12:16:39.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:38 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi190.byjvgf"}]: dispatch 2024-05-06T12:16:39.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:38 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi190.idzoff"}]: dispatch 2024-05-06T12:16:39.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:38 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mgr metadata", "who": "smithi092.gyzeqb", "id": "smithi092.gyzeqb"}]: dispatch 2024-05-06T12:16:39.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:38 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mgr metadata", "who": "smithi190.kczxkh", "id": "smithi190.kczxkh"}]: dispatch 2024-05-06T12:16:39.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:38 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-06T12:16:39.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:38 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-06T12:16:39.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:38 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-06T12:16:39.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:38 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-06T12:16:39.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:38 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-06T12:16:39.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:38 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-06T12:16:39.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:38 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-06T12:16:39.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:38 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-06T12:16:39.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:38 smithi190 ceph-mon[33798]: Standby manager daemon smithi190.kczxkh restarted 2024-05-06T12:16:39.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:38 smithi190 ceph-mon[33798]: Standby manager daemon smithi190.kczxkh started 2024-05-06T12:16:39.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:38 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-06T12:16:39.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:38 smithi190 ceph-mon[33798]: from='mgr.? 172.21.15.190:0/1432540833' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi190.kczxkh/crt"}]: dispatch 2024-05-06T12:16:39.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:38 smithi190 ceph-mon[33798]: from='mgr.? 172.21.15.190:0/1432540833' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-06T12:16:39.101 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:38 smithi190 ceph-mon[33798]: from='mgr.? 172.21.15.190:0/1432540833' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi190.kczxkh/key"}]: dispatch 2024-05-06T12:16:39.101 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:38 smithi190 ceph-mon[33798]: from='mgr.? 172.21.15.190:0/1432540833' entity='mgr.smithi190.kczxkh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-06T12:16:39.101 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:38 smithi190 ceph-mon[33798]: Manager daemon smithi092.gyzeqb is now available 2024-05-06T12:16:39.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.783+0000 7f2602ffd700 1 -- 172.21.15.92:0/3245512712 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mgrmap(e 36) v1 ==== 91955+0+0 (secure 0 0 0) 0x7f2608058680 con 0x7f26140fbcb0 2024-05-06T12:16:39.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.784+0000 7f2602ffd700 1 --2- 172.21.15.92:0/3245512712 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f25fc036b50 0x7f25fc077ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:39.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.784+0000 7f2602ffd700 1 -- 172.21.15.92:0/3245512712 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f261410f770 con 0x7f25fc036b50 2024-05-06T12:16:39.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.785+0000 7f2611d9b700 1 --2- 172.21.15.92:0/3245512712 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f25fc036b50 0x7f25fc077ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:39.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.785+0000 7f2611d9b700 1 --2- 172.21.15.92:0/3245512712 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f25fc036b50 0x7f25fc077ec0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2604004840 tx=0x7f2604004570 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:16:39.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.788+0000 7f2602ffd700 1 -- 172.21.15.92:0/3245512712 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f261410f770 con 0x7f25fc036b50 2024-05-06T12:16:39.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.791+0000 7f2618adb700 1 -- 172.21.15.92:0/3245512712 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f25fc036b50 msgr2=0x7f25fc077ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:39.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.791+0000 7f2618adb700 1 --2- 172.21.15.92:0/3245512712 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f25fc036b50 0x7f25fc077ec0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2604004840 tx=0x7f2604004570 comp rx=0 tx=0).stop 2024-05-06T12:16:39.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.791+0000 7f2618adb700 1 -- 172.21.15.92:0/3245512712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26140fbcb0 msgr2=0x7f261410da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:39.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.791+0000 7f2618adb700 1 --2- 172.21.15.92:0/3245512712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26140fbcb0 0x7f261410da50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f260800b3b0 tx=0x7f2608009e30 comp rx=0 tx=0).stop 2024-05-06T12:16:39.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.792+0000 7f2618adb700 1 -- 172.21.15.92:0/3245512712 shutdown_connections 2024-05-06T12:16:39.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.792+0000 7f2618adb700 1 --2- 172.21.15.92:0/3245512712 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f25fc036b50 0x7f25fc077ec0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:39.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.792+0000 7f2618adb700 1 --2- 172.21.15.92:0/3245512712 >> [v2:172.21.15.92:6800/648200129,v1:172.21.15.92:6801/648200129] conn(0x7f25fc06e260 0x7f25fc070710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:39.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.792+0000 7f2618adb700 1 --2- 172.21.15.92:0/3245512712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26140fbcb0 0x7f261410da50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:39.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.792+0000 7f2618adb700 1 --2- 172.21.15.92:0/3245512712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26140fcf60 0x7f261410df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:39.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.792+0000 7f2618adb700 1 -- 172.21.15.92:0/3245512712 >> 172.21.15.92:0/3245512712 conn(0x7f26140f7430 msgr2=0x7f2614065c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:16:39.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.792+0000 7f2618adb700 1 -- 172.21.15.92:0/3245512712 shutdown_connections 2024-05-06T12:16:39.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.792+0000 7f2618adb700 1 -- 172.21.15.92:0/3245512712 wait complete. 2024-05-06T12:16:39.805 INFO:teuthology.orchestra.run.smithi092.stdout:true 2024-05-06T12:16:39.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.896+0000 7f5cb836a700 1 -- 172.21.15.92:0/4202029743 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cb00fdfd0 msgr2=0x7f5cb00fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:39.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.896+0000 7f5cb836a700 1 --2- 172.21.15.92:0/4202029743 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cb00fdfd0 0x7f5cb00fe440 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0004940 tx=0x7f5ca0009e30 comp rx=0 tx=0).stop 2024-05-06T12:16:39.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.896+0000 7f5cb836a700 1 -- 172.21.15.92:0/4202029743 shutdown_connections 2024-05-06T12:16:39.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.896+0000 7f5cb836a700 1 --2- 172.21.15.92:0/4202029743 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cb00fdfd0 0x7f5cb00fe440 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:39.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.896+0000 7f5cb836a700 1 --2- 172.21.15.92:0/4202029743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cb00fce60 0x7f5cb00fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:39.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.896+0000 7f5cb836a700 1 -- 172.21.15.92:0/4202029743 >> 172.21.15.92:0/4202029743 conn(0x7f5cb00f8540 msgr2=0x7f5cb00fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:16:39.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.896+0000 7f5cb836a700 1 -- 172.21.15.92:0/4202029743 shutdown_connections 2024-05-06T12:16:39.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.896+0000 7f5cb836a700 1 -- 172.21.15.92:0/4202029743 wait complete. 2024-05-06T12:16:39.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.897+0000 7f5cb836a700 1 Processor -- start 2024-05-06T12:16:39.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.897+0000 7f5cb836a700 1 -- start start 2024-05-06T12:16:39.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.897+0000 7f5cb836a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cb00fce60 0x7f5cb010d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:39.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.897+0000 7f5cb836a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cb00fdfd0 0x7f5cb010de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:39.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.897+0000 7f5cb836a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb010e400 con 0x7f5cb00fce60 2024-05-06T12:16:39.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.897+0000 7f5cb836a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb010e570 con 0x7f5cb00fdfd0 2024-05-06T12:16:39.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.897+0000 7f5cb6106700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cb00fce60 0x7f5cb010d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:39.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.897+0000 7f5cb6106700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cb00fce60 0x7f5cb010d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46782/0 (socket says 172.21.15.92:46782) 2024-05-06T12:16:39.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.897+0000 7f5cb6106700 1 -- 172.21.15.92:0/1595456910 learned_addr learned my addr 172.21.15.92:0/1595456910 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:16:39.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.897+0000 7f5cb5905700 1 --2- 172.21.15.92:0/1595456910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cb00fdfd0 0x7f5cb010de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:39.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.897+0000 7f5cb6106700 1 -- 172.21.15.92:0/1595456910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cb00fdfd0 msgr2=0x7f5cb010de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:39.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.897+0000 7f5cb6106700 1 --2- 172.21.15.92:0/1595456910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cb00fdfd0 0x7f5cb010de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:39.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.897+0000 7f5cb6106700 1 -- 172.21.15.92:0/1595456910 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca0005040 con 0x7f5cb00fce60 2024-05-06T12:16:39.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.897+0000 7f5cb6106700 1 --2- 172.21.15.92:0/1595456910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cb00fce60 0x7f5cb010d8c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5cac00eb40 tx=0x7f5cac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:16:39.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.898+0000 7f5ca6ffd700 1 -- 172.21.15.92:0/1595456910 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cac007cc0 con 0x7f5cb00fce60 2024-05-06T12:16:39.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.898+0000 7f5ca6ffd700 1 -- 172.21.15.92:0/1595456910 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5cac00f040 con 0x7f5cb00fce60 2024-05-06T12:16:39.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.898+0000 7f5cb836a700 1 -- 172.21.15.92:0/1595456910 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cb0112f70 con 0x7f5cb00fce60 2024-05-06T12:16:39.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.898+0000 7f5ca6ffd700 1 -- 172.21.15.92:0/1595456910 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cac00ca10 con 0x7f5cb00fce60 2024-05-06T12:16:39.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.898+0000 7f5cb836a700 1 -- 172.21.15.92:0/1595456910 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cb0113420 con 0x7f5cb00fce60 2024-05-06T12:16:39.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.899+0000 7f5cb836a700 1 -- 172.21.15.92:0/1595456910 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cb004c710 con 0x7f5cb00fce60 2024-05-06T12:16:39.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.899+0000 7f5ca6ffd700 1 -- 172.21.15.92:0/1595456910 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 36) v1 ==== 91955+0+0 (secure 0 0 0) 0x7f5cac00cc80 con 0x7f5cb00fce60 2024-05-06T12:16:39.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.900+0000 7f5ca6ffd700 1 --2- 172.21.15.92:0/1595456910 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f5c9c06de40 0x7f5c9c0702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:39.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.900+0000 7f5cb5905700 1 --2- 172.21.15.92:0/1595456910 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f5c9c06de40 0x7f5c9c0702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:39.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.900+0000 7f5ca6ffd700 1 -- 172.21.15.92:0/1595456910 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f5cac08d210 con 0x7f5cb00fce60 2024-05-06T12:16:39.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.900+0000 7f5cb5905700 1 --2- 172.21.15.92:0/1595456910 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f5c9c06de40 0x7f5c9c0702f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000b0d0 tx=0x7f5ca0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:16:39.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:39.905+0000 7f5ca6ffd700 1 -- 172.21.15.92:0/1595456910 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5cac0568a0 con 0x7f5cb00fce60 2024-05-06T12:16:40.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:39 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:16:40.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:39 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:16:40.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:39 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi092.gyzeqb/mirror_snapshot_schedule"}]: dispatch 2024-05-06T12:16:40.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:39 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi092.gyzeqb/trash_purge_schedule"}]: dispatch 2024-05-06T12:16:40.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:39 smithi190 ceph-mon[33798]: mgrmap e36: smithi092.gyzeqb(active, since 1.00978s), standbys: smithi190.kczxkh 2024-05-06T12:16:40.124 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:39 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:16:40.124 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:39 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:16:40.124 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:39 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi092.gyzeqb/mirror_snapshot_schedule"}]: dispatch 2024-05-06T12:16:40.124 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:39 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi092.gyzeqb/trash_purge_schedule"}]: dispatch 2024-05-06T12:16:40.124 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:39 smithi092 ceph-mon[91933]: mgrmap e36: smithi092.gyzeqb(active, since 1.00978s), standbys: smithi190.kczxkh 2024-05-06T12:16:40.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.132+0000 7f5cb836a700 1 -- 172.21.15.92:0/1595456910 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5cb0113610 con 0x7f5c9c06de40 2024-05-06T12:16:40.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.134+0000 7f5ca6ffd700 1 -- 172.21.15.92:0/1595456910 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f5cb0113610 con 0x7f5c9c06de40 2024-05-06T12:16:40.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.136+0000 7f5cb836a700 1 -- 172.21.15.92:0/1595456910 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f5c9c06de40 msgr2=0x7f5c9c0702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:40.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.136+0000 7f5cb836a700 1 --2- 172.21.15.92:0/1595456910 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f5c9c06de40 0x7f5c9c0702f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000b0d0 tx=0x7f5ca0009e30 comp rx=0 tx=0).stop 2024-05-06T12:16:40.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.136+0000 7f5cb836a700 1 -- 172.21.15.92:0/1595456910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cb00fce60 msgr2=0x7f5cb010d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:40.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.136+0000 7f5cb836a700 1 --2- 172.21.15.92:0/1595456910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cb00fce60 0x7f5cb010d8c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5cac00eb40 tx=0x7f5cac00b040 comp rx=0 tx=0).stop 2024-05-06T12:16:40.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.136+0000 7f5cb836a700 1 -- 172.21.15.92:0/1595456910 shutdown_connections 2024-05-06T12:16:40.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.136+0000 7f5cb836a700 1 --2- 172.21.15.92:0/1595456910 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f5c9c06de40 0x7f5c9c0702f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:40.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.136+0000 7f5cb836a700 1 --2- 172.21.15.92:0/1595456910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cb00fce60 0x7f5cb010d8c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:40.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.136+0000 7f5cb836a700 1 --2- 172.21.15.92:0/1595456910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cb00fdfd0 0x7f5cb010de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:40.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.136+0000 7f5cb836a700 1 -- 172.21.15.92:0/1595456910 >> 172.21.15.92:0/1595456910 conn(0x7f5cb00f8540 msgr2=0x7f5cb00f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:16:40.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.136+0000 7f5cb836a700 1 -- 172.21.15.92:0/1595456910 shutdown_connections 2024-05-06T12:16:40.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.136+0000 7f5cb836a700 1 -- 172.21.15.92:0/1595456910 wait complete. 2024-05-06T12:16:40.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.237+0000 7fbfa4b35700 1 -- 172.21.15.92:0/3382859157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfa00fbcc0 msgr2=0x7fbfa00fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:40.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.237+0000 7fbfa4b35700 1 --2- 172.21.15.92:0/3382859157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfa00fbcc0 0x7fbfa00fc0d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbf90004940 tx=0x7fbf90009e30 comp rx=0 tx=0).stop 2024-05-06T12:16:40.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.237+0000 7fbfa4b35700 1 -- 172.21.15.92:0/3382859157 shutdown_connections 2024-05-06T12:16:40.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.238+0000 7fbfa4b35700 1 --2- 172.21.15.92:0/3382859157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfa00fcf70 0x7fbfa00fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:40.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.238+0000 7fbfa4b35700 1 --2- 172.21.15.92:0/3382859157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfa00fbcc0 0x7fbfa00fc0d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:40.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.238+0000 7fbfa4b35700 1 -- 172.21.15.92:0/3382859157 >> 172.21.15.92:0/3382859157 conn(0x7fbfa00f7420 msgr2=0x7fbfa00f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:16:40.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.238+0000 7fbfa4b35700 1 -- 172.21.15.92:0/3382859157 shutdown_connections 2024-05-06T12:16:40.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.238+0000 7fbfa4b35700 1 -- 172.21.15.92:0/3382859157 wait complete. 2024-05-06T12:16:40.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.238+0000 7fbfa4b35700 1 Processor -- start 2024-05-06T12:16:40.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.238+0000 7fbfa4b35700 1 -- start start 2024-05-06T12:16:40.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.238+0000 7fbfa4b35700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfa00fbcc0 0x7fbfa0066d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:40.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.238+0000 7fbfa4b35700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfa00fcf70 0x7fbfa0112740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:40.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.238+0000 7fbfa4b35700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa0112c80 con 0x7fbfa00fbcc0 2024-05-06T12:16:40.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.238+0000 7fbfa4b35700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa0112dc0 con 0x7fbfa00fcf70 2024-05-06T12:16:40.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.238+0000 7fbf9e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfa00fbcc0 0x7fbfa0066d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:40.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.238+0000 7fbf9e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfa00fbcc0 0x7fbfa0066d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46798/0 (socket says 172.21.15.92:46798) 2024-05-06T12:16:40.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.238+0000 7fbf9e59c700 1 -- 172.21.15.92:0/2000002339 learned_addr learned my addr 172.21.15.92:0/2000002339 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:16:40.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.238+0000 7fbf9dd9b700 1 --2- 172.21.15.92:0/2000002339 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfa00fcf70 0x7fbfa0112740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:40.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.239+0000 7fbf9e59c700 1 -- 172.21.15.92:0/2000002339 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfa00fcf70 msgr2=0x7fbfa0112740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:40.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.239+0000 7fbf9e59c700 1 --2- 172.21.15.92:0/2000002339 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfa00fcf70 0x7fbfa0112740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:40.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.239+0000 7fbf9e59c700 1 -- 172.21.15.92:0/2000002339 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf90005040 con 0x7fbfa00fbcc0 2024-05-06T12:16:40.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.239+0000 7fbf9e59c700 1 --2- 172.21.15.92:0/2000002339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfa00fbcc0 0x7fbfa0066d30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbf9000b3b0 tx=0x7fbf90009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:16:40.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.239+0000 7fbf8effd700 1 -- 172.21.15.92:0/2000002339 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf9000e040 con 0x7fbfa00fbcc0 2024-05-06T12:16:40.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.239+0000 7fbf8effd700 1 -- 172.21.15.92:0/2000002339 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf90004290 con 0x7fbfa00fbcc0 2024-05-06T12:16:40.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.239+0000 7fbfa4b35700 1 -- 172.21.15.92:0/2000002339 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa0112fc0 con 0x7fbfa00fbcc0 2024-05-06T12:16:40.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.239+0000 7fbf8effd700 1 -- 172.21.15.92:0/2000002339 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf9001c3f0 con 0x7fbfa00fbcc0 2024-05-06T12:16:40.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.239+0000 7fbfa4b35700 1 -- 172.21.15.92:0/2000002339 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa01134f0 con 0x7fbfa00fbcc0 2024-05-06T12:16:40.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.240+0000 7fbfa4b35700 1 -- 172.21.15.92:0/2000002339 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfa004c710 con 0x7fbfa00fbcc0 2024-05-06T12:16:40.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.242+0000 7fbf8effd700 1 -- 172.21.15.92:0/2000002339 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 36) v1 ==== 91955+0+0 (secure 0 0 0) 0x7fbf9000bbf0 con 0x7fbfa00fbcc0 2024-05-06T12:16:40.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.242+0000 7fbf8effd700 1 --2- 172.21.15.92:0/2000002339 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fbf8806de40 0x7fbf880702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:40.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.242+0000 7fbf8effd700 1 -- 172.21.15.92:0/2000002339 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fbf9008e880 con 0x7fbfa00fbcc0 2024-05-06T12:16:40.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.242+0000 7fbf9dd9b700 1 --2- 172.21.15.92:0/2000002339 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fbf8806de40 0x7fbf880702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:40.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.243+0000 7fbf9dd9b700 1 --2- 172.21.15.92:0/2000002339 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fbf8806de40 0x7fbf880702f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbf94007fd0 tx=0x7fbf94009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:16:40.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.247+0000 7fbf8effd700 1 -- 172.21.15.92:0/2000002339 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fbf90057f60 con 0x7fbfa00fbcc0 2024-05-06T12:16:40.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.459+0000 7fbfa4b35700 1 -- 172.21.15.92:0/2000002339 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbfa0101330 con 0x7fbf8806de40 2024-05-06T12:16:40.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.475+0000 7fbf8effd700 1 -- 172.21.15.92:0/2000002339 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fbfa0101330 con 0x7fbf8806de40 2024-05-06T12:16:40.479 INFO:teuthology.orchestra.run.smithi092.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-06T12:16:40.479 INFO:teuthology.orchestra.run.smithi092.stdout:alertmanager.smithi092 smithi092 *:9093,9094 running (104s) 17s ago 10m 17.0M - 0.25.0 c8568f914cd2 be5637d7d25a 2024-05-06T12:16:40.479 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi092 smithi092 running (10m) 17s ago 10m 6665k - 17.2.7-925-ge81df420 96d5ae7fd4b4 95dc7e14bd9a 2024-05-06T12:16:40.479 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi190 smithi190 running (9m) 20s ago 9m 6665k - 17.2.7-925-ge81df420 96d5ae7fd4b4 6cf97785406f 2024-05-06T12:16:40.479 INFO:teuthology.orchestra.run.smithi092.stdout:grafana.smithi092 smithi092 *:3000 running (99s) 17s ago 9m 76.3M - 9.4.7 954c08fa6188 9180cb3e5258 2024-05-06T12:16:40.479 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.ijsqgp smithi092 running (4m) 17s ago 4m 64.5M - 17.2.7-925-ge81df420 96d5ae7fd4b4 b4c2ed6d2c82 2024-05-06T12:16:40.479 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.rosbll smithi092 running (4m) 17s ago 4m 137M - 17.2.7-925-ge81df420 96d5ae7fd4b4 5f5426cd1bf5 2024-05-06T12:16:40.479 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.byjvgf smithi190 running (4m) 20s ago 4m 127M - 17.2.7-925-ge81df420 96d5ae7fd4b4 24581d2e1c5d 2024-05-06T12:16:40.479 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.idzoff smithi190 running (4m) 20s ago 4m 64.5M - 17.2.7-925-ge81df420 96d5ae7fd4b4 20f2641696ef 2024-05-06T12:16:40.479 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi092.gyzeqb smithi092 *:8443,9283,8765 running (68s) 17s ago 11m 494M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 de9d2198327a 2024-05-06T12:16:40.479 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi190.kczxkh smithi190 *:8443,9283,8765 running (24s) 20s ago 9m 63.2M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 e84ab3758ac7 2024-05-06T12:16:40.479 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi092 smithi092 running (11m) 17s ago 11m 54.0M 2048M 17.2.7-925-ge81df420 96d5ae7fd4b4 df5b322e9719 2024-05-06T12:16:40.480 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi190 smithi190 running (9m) 20s ago 9m 42.9M 2048M 17.2.7-925-ge81df420 96d5ae7fd4b4 912ded072601 2024-05-06T12:16:40.480 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi092 smithi092 *:9100 running (111s) 17s ago 10m 8270k - 1.5.0 0da6a335fe13 c796446b206f 2024-05-06T12:16:40.480 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi190 smithi190 *:9100 running (90s) 20s ago 9m 8464k - 1.5.0 0da6a335fe13 647e28111a76 2024-05-06T12:16:40.480 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 smithi092 running (8m) 17s ago 8m 198M 1557M 17.2.7-925-ge81df420 96d5ae7fd4b4 ce84e7af9084 2024-05-06T12:16:40.480 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 smithi092 running (8m) 17s ago 8m 198M 1557M 17.2.7-925-ge81df420 96d5ae7fd4b4 8ab870f57eb2 2024-05-06T12:16:40.480 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 smithi092 running (7m) 17s ago 7m 164M 1557M 17.2.7-925-ge81df420 96d5ae7fd4b4 1e6ed04a0688 2024-05-06T12:16:40.480 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 smithi190 running (7m) 20s ago 7m 220M 2581M 17.2.7-925-ge81df420 96d5ae7fd4b4 52f9521286d5 2024-05-06T12:16:40.480 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 smithi190 running (6m) 20s ago 6m 203M 2581M 17.2.7-925-ge81df420 96d5ae7fd4b4 3a7da09e4c48 2024-05-06T12:16:40.480 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 smithi190 running (6m) 20s ago 6m 186M 2581M 17.2.7-925-ge81df420 96d5ae7fd4b4 e7452c6b5cec 2024-05-06T12:16:40.480 INFO:teuthology.orchestra.run.smithi092.stdout:prometheus.smithi092 smithi092 *:9095 running (33s) 17s ago 9m 42.5M - 2.43.0 a07b618ecd1d bc5445691527 2024-05-06T12:16:40.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.478+0000 7fbfa4b35700 1 -- 172.21.15.92:0/2000002339 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fbf8806de40 msgr2=0x7fbf880702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:40.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.478+0000 7fbfa4b35700 1 --2- 172.21.15.92:0/2000002339 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fbf8806de40 0x7fbf880702f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbf94007fd0 tx=0x7fbf94009040 comp rx=0 tx=0).stop 2024-05-06T12:16:40.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.478+0000 7fbfa4b35700 1 -- 172.21.15.92:0/2000002339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfa00fbcc0 msgr2=0x7fbfa0066d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:40.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.478+0000 7fbfa4b35700 1 --2- 172.21.15.92:0/2000002339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfa00fbcc0 0x7fbfa0066d30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbf9000b3b0 tx=0x7fbf90009e30 comp rx=0 tx=0).stop 2024-05-06T12:16:40.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.478+0000 7fbfa4b35700 1 -- 172.21.15.92:0/2000002339 shutdown_connections 2024-05-06T12:16:40.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.478+0000 7fbfa4b35700 1 --2- 172.21.15.92:0/2000002339 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fbf8806de40 0x7fbf880702f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:40.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.478+0000 7fbfa4b35700 1 --2- 172.21.15.92:0/2000002339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfa00fbcc0 0x7fbfa0066d30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:40.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.478+0000 7fbfa4b35700 1 --2- 172.21.15.92:0/2000002339 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfa00fcf70 0x7fbfa0112740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:40.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.478+0000 7fbfa4b35700 1 -- 172.21.15.92:0/2000002339 >> 172.21.15.92:0/2000002339 conn(0x7fbfa00f7420 msgr2=0x7fbfa00f8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:16:40.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.478+0000 7fbfa4b35700 1 -- 172.21.15.92:0/2000002339 shutdown_connections 2024-05-06T12:16:40.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.478+0000 7fbfa4b35700 1 -- 172.21.15.92:0/2000002339 wait complete. 2024-05-06T12:16:40.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.586+0000 7f4be116c700 1 -- 172.21.15.92:0/3489260347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bdc0fee10 msgr2=0x7f4bdc0faf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:40.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.586+0000 7f4be116c700 1 --2- 172.21.15.92:0/3489260347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bdc0fee10 0x7f4bdc0faf50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0004940 tx=0x7f4bd0009e30 comp rx=0 tx=0).stop 2024-05-06T12:16:40.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.586+0000 7f4be116c700 1 -- 172.21.15.92:0/3489260347 shutdown_connections 2024-05-06T12:16:40.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.586+0000 7f4be116c700 1 --2- 172.21.15.92:0/3489260347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bdc0fee10 0x7f4bdc0faf50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:40.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.586+0000 7f4be116c700 1 --2- 172.21.15.92:0/3489260347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bdc0fe4c0 0x7f4bdc0fe8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:40.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.586+0000 7f4be116c700 1 -- 172.21.15.92:0/3489260347 >> 172.21.15.92:0/3489260347 conn(0x7f4bdc0f6be0 msgr2=0x7f4bdc0f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:16:40.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.586+0000 7f4be116c700 1 -- 172.21.15.92:0/3489260347 shutdown_connections 2024-05-06T12:16:40.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.586+0000 7f4be116c700 1 -- 172.21.15.92:0/3489260347 wait complete. 2024-05-06T12:16:40.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.587+0000 7f4be116c700 1 Processor -- start 2024-05-06T12:16:40.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.587+0000 7f4be116c700 1 -- start start 2024-05-06T12:16:40.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.587+0000 7f4be116c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bdc0fe4c0 0x7f4bdc10d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:40.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.587+0000 7f4be116c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bdc0fee10 0x7f4bdc10de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:40.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.587+0000 7f4be116c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc10e440 con 0x7f4bdc0fee10 2024-05-06T12:16:40.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.587+0000 7f4be116c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc10e5b0 con 0x7f4bdc0fe4c0 2024-05-06T12:16:40.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.587+0000 7f4bda59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bdc0fee10 0x7f4bdc10de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:40.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.587+0000 7f4bda59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bdc0fee10 0x7f4bdc10de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46804/0 (socket says 172.21.15.92:46804) 2024-05-06T12:16:40.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.587+0000 7f4bda59c700 1 -- 172.21.15.92:0/2590032353 learned_addr learned my addr 172.21.15.92:0/2590032353 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:16:40.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.588+0000 7f4bdad9d700 1 --2- 172.21.15.92:0/2590032353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bdc0fe4c0 0x7f4bdc10d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:40.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.588+0000 7f4bda59c700 1 -- 172.21.15.92:0/2590032353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bdc0fe4c0 msgr2=0x7f4bdc10d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:40.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.588+0000 7f4bda59c700 1 --2- 172.21.15.92:0/2590032353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bdc0fe4c0 0x7f4bdc10d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:40.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.588+0000 7f4bda59c700 1 -- 172.21.15.92:0/2590032353 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd0005040 con 0x7f4bdc0fee10 2024-05-06T12:16:40.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.588+0000 7f4bdad9d700 1 --2- 172.21.15.92:0/2590032353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bdc0fe4c0 0x7f4bdc10d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-06T12:16:40.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.588+0000 7f4bda59c700 1 --2- 172.21.15.92:0/2590032353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bdc0fee10 0x7f4bdc10de40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0009150 tx=0x7f4bd0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:16:40.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.588+0000 7f4bc37fe700 1 -- 172.21.15.92:0/2590032353 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bd000e040 con 0x7f4bdc0fee10 2024-05-06T12:16:40.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.588+0000 7f4bc37fe700 1 -- 172.21.15.92:0/2590032353 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4bd000bcc0 con 0x7f4bdc0fee10 2024-05-06T12:16:40.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.588+0000 7f4be116c700 1 -- 172.21.15.92:0/2590032353 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bdc112fb0 con 0x7f4bdc0fee10 2024-05-06T12:16:40.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.588+0000 7f4bc37fe700 1 -- 172.21.15.92:0/2590032353 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bd001c630 con 0x7f4bdc0fee10 2024-05-06T12:16:40.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.588+0000 7f4be116c700 1 -- 172.21.15.92:0/2590032353 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bdc113540 con 0x7f4bdc0fee10 2024-05-06T12:16:40.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.589+0000 7f4be116c700 1 -- 172.21.15.92:0/2590032353 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bdc04c710 con 0x7f4bdc0fee10 2024-05-06T12:16:40.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.590+0000 7f4bc37fe700 1 -- 172.21.15.92:0/2590032353 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 36) v1 ==== 91955+0+0 (secure 0 0 0) 0x7f4bd0013420 con 0x7f4bdc0fee10 2024-05-06T12:16:40.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.590+0000 7f4bc37fe700 1 --2- 172.21.15.92:0/2590032353 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4bc406de50 0x7f4bc4070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:40.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.590+0000 7f4bc37fe700 1 -- 172.21.15.92:0/2590032353 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f4bd008e330 con 0x7f4bdc0fee10 2024-05-06T12:16:40.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.590+0000 7f4bdad9d700 1 --2- 172.21.15.92:0/2590032353 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4bc406de50 0x7f4bc4070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:40.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.591+0000 7f4bdad9d700 1 --2- 172.21.15.92:0/2590032353 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4bc406de50 0x7f4bc4070300 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc0041d0 tx=0x7f4bcc009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:16:40.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.595+0000 7f4bc37fe700 1 -- 172.21.15.92:0/2590032353 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4bd0057990 con 0x7f4bdc0fee10 2024-05-06T12:16:40.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.840+0000 7f4bc37fe700 1 -- 172.21.15.92:0/2590032353 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 37) v1 ==== 92249+0+0 (secure 0 0 0) 0x7f4bd0057d60 con 0x7f4bdc0fee10 2024-05-06T12:16:40.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.899+0000 7f4be116c700 1 -- 172.21.15.92:0/2590032353 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4bdc113730 con 0x7f4bdc0fee10 2024-05-06T12:16:40.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.899+0000 7f4bc37fe700 1 -- 172.21.15.92:0/2590032353 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+843 (secure 0 0 0) 0x7f4bdc113730 con 0x7f4bdc0fee10 2024-05-06T12:16:40.902 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:16:40.902 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": { 2024-05-06T12:16:40.903 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 1, 2024-05-06T12:16:40.903 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 1 2024-05-06T12:16:40.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:16:40.903 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": { 2024-05-06T12:16:40.903 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:16:40.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:16:40.903 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": { 2024-05-06T12:16:40.903 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 6 2024-05-06T12:16:40.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:16:40.903 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-05-06T12:16:40.903 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 4 2024-05-06T12:16:40.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:16:40.903 INFO:teuthology.orchestra.run.smithi092.stdout: "overall": { 2024-05-06T12:16:40.903 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 11, 2024-05-06T12:16:40.903 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 3 2024-05-06T12:16:40.903 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-06T12:16:40.904 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:16:40.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.902+0000 7f4be116c700 1 -- 172.21.15.92:0/2590032353 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4bc406de50 msgr2=0x7f4bc4070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:40.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.902+0000 7f4be116c700 1 --2- 172.21.15.92:0/2590032353 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4bc406de50 0x7f4bc4070300 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc0041d0 tx=0x7f4bcc009040 comp rx=0 tx=0).stop 2024-05-06T12:16:40.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.902+0000 7f4be116c700 1 -- 172.21.15.92:0/2590032353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bdc0fee10 msgr2=0x7f4bdc10de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:40.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.902+0000 7f4be116c700 1 --2- 172.21.15.92:0/2590032353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bdc0fee10 0x7f4bdc10de40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0009150 tx=0x7f4bd0009e30 comp rx=0 tx=0).stop 2024-05-06T12:16:40.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.902+0000 7f4be116c700 1 -- 172.21.15.92:0/2590032353 shutdown_connections 2024-05-06T12:16:40.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.902+0000 7f4be116c700 1 --2- 172.21.15.92:0/2590032353 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4bc406de50 0x7f4bc4070300 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:40.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.902+0000 7f4be116c700 1 --2- 172.21.15.92:0/2590032353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bdc0fe4c0 0x7f4bdc10d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:40.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.902+0000 7f4be116c700 1 --2- 172.21.15.92:0/2590032353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bdc0fee10 0x7f4bdc10de40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:40.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.902+0000 7f4be116c700 1 -- 172.21.15.92:0/2590032353 >> 172.21.15.92:0/2590032353 conn(0x7f4bdc0f6be0 msgr2=0x7f4bdc0f8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:16:40.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.902+0000 7f4be116c700 1 -- 172.21.15.92:0/2590032353 shutdown_connections 2024-05-06T12:16:40.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:40.902+0000 7f4be116c700 1 -- 172.21.15.92:0/2590032353 wait complete. 2024-05-06T12:16:41.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.006+0000 7f8d284cc700 1 -- 172.21.15.92:0/1547339864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d200fed70 msgr2=0x7f8d200faeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:41.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.006+0000 7f8d284cc700 1 --2- 172.21.15.92:0/1547339864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d200fed70 0x7f8d200faeb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8d10004940 tx=0x7f8d10009e30 comp rx=0 tx=0).stop 2024-05-06T12:16:41.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.006+0000 7f8d284cc700 1 -- 172.21.15.92:0/1547339864 shutdown_connections 2024-05-06T12:16:41.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.006+0000 7f8d284cc700 1 --2- 172.21.15.92:0/1547339864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d200fed70 0x7f8d200faeb0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:41.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.006+0000 7f8d284cc700 1 --2- 172.21.15.92:0/1547339864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d200fe420 0x7f8d200fe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:41.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.006+0000 7f8d284cc700 1 -- 172.21.15.92:0/1547339864 >> 172.21.15.92:0/1547339864 conn(0x7f8d200f6be0 msgr2=0x7f8d200f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:16:41.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.006+0000 7f8d284cc700 1 -- 172.21.15.92:0/1547339864 shutdown_connections 2024-05-06T12:16:41.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.006+0000 7f8d284cc700 1 -- 172.21.15.92:0/1547339864 wait complete. 2024-05-06T12:16:41.011 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:40 smithi092 ceph-mon[91933]: [06/May/2024:12:16:39] ENGINE Bus STARTING 2024-05-06T12:16:41.011 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:40 smithi092 ceph-mon[91933]: [06/May/2024:12:16:39] ENGINE Serving on https://172.21.15.92:7150 2024-05-06T12:16:41.011 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:40 smithi092 ceph-mon[91933]: [06/May/2024:12:16:39] ENGINE Serving on http://172.21.15.92:8765 2024-05-06T12:16:41.011 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:40 smithi092 ceph-mon[91933]: [06/May/2024:12:16:39] ENGINE Bus STARTED 2024-05-06T12:16:41.011 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:40 smithi092 ceph-mon[91933]: from='client.34108 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:16:41.011 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:40 smithi092 ceph-mon[91933]: pgmap v3: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-05-06T12:16:41.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.007+0000 7f8d284cc700 1 Processor -- start 2024-05-06T12:16:41.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.007+0000 7f8d284cc700 1 -- start start 2024-05-06T12:16:41.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.007+0000 7f8d284cc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d200fe420 0x7f8d200fda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:41.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.007+0000 7f8d284cc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d200fed70 0x7f8d200fc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:41.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.007+0000 7f8d284cc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d200fc620 con 0x7f8d200fe420 2024-05-06T12:16:41.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.007+0000 7f8d284cc700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d200fc790 con 0x7f8d200fed70 2024-05-06T12:16:41.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.007+0000 7f8d26268700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d200fe420 0x7f8d200fda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:41.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.007+0000 7f8d26268700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d200fe420 0x7f8d200fda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46852/0 (socket says 172.21.15.92:46852) 2024-05-06T12:16:41.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.007+0000 7f8d26268700 1 -- 172.21.15.92:0/3709420616 learned_addr learned my addr 172.21.15.92:0/3709420616 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:16:41.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.007+0000 7f8d25a67700 1 --2- 172.21.15.92:0/3709420616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d200fed70 0x7f8d200fc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:41.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.008+0000 7f8d26268700 1 -- 172.21.15.92:0/3709420616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d200fed70 msgr2=0x7f8d200fc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:41.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.008+0000 7f8d26268700 1 --2- 172.21.15.92:0/3709420616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d200fed70 0x7f8d200fc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:41.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.008+0000 7f8d26268700 1 -- 172.21.15.92:0/3709420616 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d10005040 con 0x7f8d200fe420 2024-05-06T12:16:41.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.008+0000 7f8d26268700 1 --2- 172.21.15.92:0/3709420616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d200fe420 0x7f8d200fda30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00b790 tx=0x7f8d1c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:16:41.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.008+0000 7f8d16ffd700 1 -- 172.21.15.92:0/3709420616 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d1c00f770 con 0x7f8d200fe420 2024-05-06T12:16:41.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.008+0000 7f8d284cc700 1 -- 172.21.15.92:0/3709420616 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d200fca70 con 0x7f8d200fe420 2024-05-06T12:16:41.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.008+0000 7f8d16ffd700 1 -- 172.21.15.92:0/3709420616 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d1c00fda0 con 0x7f8d200fe420 2024-05-06T12:16:41.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.008+0000 7f8d16ffd700 1 -- 172.21.15.92:0/3709420616 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d1c00d310 con 0x7f8d200fe420 2024-05-06T12:16:41.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.008+0000 7f8d284cc700 1 -- 172.21.15.92:0/3709420616 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d20115a00 con 0x7f8d200fe420 2024-05-06T12:16:41.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.009+0000 7f8d284cc700 1 -- 172.21.15.92:0/3709420616 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d2004c710 con 0x7f8d200fe420 2024-05-06T12:16:41.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.009+0000 7f8d16ffd700 1 -- 172.21.15.92:0/3709420616 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 37) v1 ==== 92249+0+0 (secure 0 0 0) 0x7f8d1c010420 con 0x7f8d200fe420 2024-05-06T12:16:41.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.010+0000 7f8d16ffd700 1 --2- 172.21.15.92:0/3709420616 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f8d0c06e1d0 0x7f8d0c070680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:41.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.010+0000 7f8d16ffd700 1 -- 172.21.15.92:0/3709420616 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f8d1c08dba0 con 0x7f8d200fe420 2024-05-06T12:16:41.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.010+0000 7f8d25a67700 1 --2- 172.21.15.92:0/3709420616 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f8d0c06e1d0 0x7f8d0c070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:41.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.010+0000 7f8d25a67700 1 --2- 172.21.15.92:0/3709420616 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f8d0c06e1d0 0x7f8d0c070680 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8d1000b3b0 tx=0x7f8d10009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:16:41.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.015+0000 7f8d16ffd700 1 -- 172.21.15.92:0/3709420616 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f8d1c05aaa0 con 0x7f8d200fe420 2024-05-06T12:16:41.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:40 smithi190 ceph-mon[33798]: [06/May/2024:12:16:39] ENGINE Bus STARTING 2024-05-06T12:16:41.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:40 smithi190 ceph-mon[33798]: [06/May/2024:12:16:39] ENGINE Serving on https://172.21.15.92:7150 2024-05-06T12:16:41.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:40 smithi190 ceph-mon[33798]: [06/May/2024:12:16:39] ENGINE Serving on http://172.21.15.92:8765 2024-05-06T12:16:41.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:40 smithi190 ceph-mon[33798]: [06/May/2024:12:16:39] ENGINE Bus STARTED 2024-05-06T12:16:41.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:40 smithi190 ceph-mon[33798]: from='client.34108 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:16:41.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:40 smithi190 ceph-mon[33798]: pgmap v3: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-05-06T12:16:41.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.276+0000 7f8d284cc700 1 -- 172.21.15.92:0/3709420616 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8d20115ca0 con 0x7f8d200fe420 2024-05-06T12:16:41.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.277+0000 7f8d16ffd700 1 -- 172.21.15.92:0/3709420616 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1937 (secure 0 0 0) 0x7f8d1c05a630 con 0x7f8d200fe420 2024-05-06T12:16:41.280 INFO:teuthology.orchestra.run.smithi092.stdout:e20 2024-05-06T12:16:41.280 INFO:teuthology.orchestra.run.smithi092.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-06T12:16:41.280 INFO:teuthology.orchestra.run.smithi092.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:16:41.280 INFO:teuthology.orchestra.run.smithi092.stdout:legacy client fscid: 1 2024-05-06T12:16:41.280 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:16:41.280 INFO:teuthology.orchestra.run.smithi092.stdout:Filesystem 'cephfs' (1) 2024-05-06T12:16:41.280 INFO:teuthology.orchestra.run.smithi092.stdout:fs_name cephfs 2024-05-06T12:16:41.281 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 20 2024-05-06T12:16:41.281 INFO:teuthology.orchestra.run.smithi092.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-06T12:16:41.281 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-05-06T12:11:44.952671+0000 2024-05-06T12:16:41.281 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2024-05-06T12:15:07.568167+0000 2024-05-06T12:16:41.281 INFO:teuthology.orchestra.run.smithi092.stdout:tableserver 0 2024-05-06T12:16:41.281 INFO:teuthology.orchestra.run.smithi092.stdout:root 0 2024-05-06T12:16:41.281 INFO:teuthology.orchestra.run.smithi092.stdout:session_timeout 60 2024-05-06T12:16:41.281 INFO:teuthology.orchestra.run.smithi092.stdout:session_autoclose 300 2024-05-06T12:16:41.281 INFO:teuthology.orchestra.run.smithi092.stdout:max_file_size 1099511627776 2024-05-06T12:16:41.281 INFO:teuthology.orchestra.run.smithi092.stdout:max_xattr_size 65536 2024-05-06T12:16:41.281 INFO:teuthology.orchestra.run.smithi092.stdout:required_client_features {} 2024-05-06T12:16:41.281 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure 0 2024-05-06T12:16:41.281 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure_osd_epoch 46 2024-05-06T12:16:41.281 INFO:teuthology.orchestra.run.smithi092.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:16:41.282 INFO:teuthology.orchestra.run.smithi092.stdout:max_mds 2 2024-05-06T12:16:41.282 INFO:teuthology.orchestra.run.smithi092.stdout:in 0,1 2024-05-06T12:16:41.282 INFO:teuthology.orchestra.run.smithi092.stdout:up {0=14472,1=24267} 2024-05-06T12:16:41.282 INFO:teuthology.orchestra.run.smithi092.stdout:failed 2024-05-06T12:16:41.282 INFO:teuthology.orchestra.run.smithi092.stdout:damaged 2024-05-06T12:16:41.282 INFO:teuthology.orchestra.run.smithi092.stdout:stopped 2024-05-06T12:16:41.282 INFO:teuthology.orchestra.run.smithi092.stdout:data_pools [3] 2024-05-06T12:16:41.282 INFO:teuthology.orchestra.run.smithi092.stdout:metadata_pool 2 2024-05-06T12:16:41.282 INFO:teuthology.orchestra.run.smithi092.stdout:inline_data enabled 2024-05-06T12:16:41.282 INFO:teuthology.orchestra.run.smithi092.stdout:balancer 2024-05-06T12:16:41.282 INFO:teuthology.orchestra.run.smithi092.stdout:bal_rank_mask -1 2024-05-06T12:16:41.282 INFO:teuthology.orchestra.run.smithi092.stdout:standby_count_wanted 1 2024-05-06T12:16:41.282 INFO:teuthology.orchestra.run.smithi092.stdout:qdb_cluster leader: 0 members: 2024-05-06T12:16:41.282 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.rosbll{0:14472} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.92:6828/1663253598,v1:172.21.15.92:6829/1663253598] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:16:41.282 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.ijsqgp{0:14484} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.92:6826/3613441054,v1:172.21.15.92:6827/3613441054] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:16:41.283 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.byjvgf{1:24267} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.190:6824/1565151699,v1:172.21.15.190:6825/1565151699] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:16:41.283 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.idzoff{1:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.190:6826/2453728380,v1:172.21.15.190:6827/2453728380] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:16:41.283 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:16:41.283 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:16:41.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.279+0000 7f8d284cc700 1 -- 172.21.15.92:0/3709420616 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f8d0c06e1d0 msgr2=0x7f8d0c070680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:41.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.279+0000 7f8d284cc700 1 --2- 172.21.15.92:0/3709420616 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f8d0c06e1d0 0x7f8d0c070680 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8d1000b3b0 tx=0x7f8d10009e30 comp rx=0 tx=0).stop 2024-05-06T12:16:41.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.279+0000 7f8d284cc700 1 -- 172.21.15.92:0/3709420616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d200fe420 msgr2=0x7f8d200fda30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:41.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.279+0000 7f8d284cc700 1 --2- 172.21.15.92:0/3709420616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d200fe420 0x7f8d200fda30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00b790 tx=0x7f8d1c009250 comp rx=0 tx=0).stop 2024-05-06T12:16:41.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.279+0000 7f8d284cc700 1 -- 172.21.15.92:0/3709420616 shutdown_connections 2024-05-06T12:16:41.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.279+0000 7f8d284cc700 1 --2- 172.21.15.92:0/3709420616 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f8d0c06e1d0 0x7f8d0c070680 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:41.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.279+0000 7f8d284cc700 1 --2- 172.21.15.92:0/3709420616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d200fe420 0x7f8d200fda30 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:41.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.279+0000 7f8d284cc700 1 --2- 172.21.15.92:0/3709420616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d200fed70 0x7f8d200fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:41.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.279+0000 7f8d284cc700 1 -- 172.21.15.92:0/3709420616 >> 172.21.15.92:0/3709420616 conn(0x7f8d200f6be0 msgr2=0x7f8d200f8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:16:41.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.279+0000 7f8d284cc700 1 -- 172.21.15.92:0/3709420616 shutdown_connections 2024-05-06T12:16:41.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.279+0000 7f8d284cc700 1 -- 172.21.15.92:0/3709420616 wait complete. 2024-05-06T12:16:41.284 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 20 2024-05-06T12:16:41.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.380+0000 7f4f236c8700 1 -- 172.21.15.92:0/3377042557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f1c0fff70 msgr2=0x7f4f1c102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:41.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.380+0000 7f4f236c8700 1 --2- 172.21.15.92:0/3377042557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f1c0fff70 0x7f4f1c102390 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c004940 tx=0x7f4f0c009e30 comp rx=0 tx=0).stop 2024-05-06T12:16:41.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.380+0000 7f4f236c8700 1 -- 172.21.15.92:0/3377042557 shutdown_connections 2024-05-06T12:16:41.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.380+0000 7f4f236c8700 1 --2- 172.21.15.92:0/3377042557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f1c0fff70 0x7f4f1c102390 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:41.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.380+0000 7f4f236c8700 1 --2- 172.21.15.92:0/3377042557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f1c066b50 0x7f4f1c0ffa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:41.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.380+0000 7f4f236c8700 1 -- 172.21.15.92:0/3377042557 >> 172.21.15.92:0/3377042557 conn(0x7f4f1c0f74c0 msgr2=0x7f4f1c0f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:16:41.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.380+0000 7f4f236c8700 1 -- 172.21.15.92:0/3377042557 shutdown_connections 2024-05-06T12:16:41.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.380+0000 7f4f236c8700 1 -- 172.21.15.92:0/3377042557 wait complete. 2024-05-06T12:16:41.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.380+0000 7f4f236c8700 1 Processor -- start 2024-05-06T12:16:41.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.380+0000 7f4f236c8700 1 -- start start 2024-05-06T12:16:41.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.380+0000 7f4f236c8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f1c066b50 0x7f4f1c10da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:41.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.380+0000 7f4f236c8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f1c0fff70 0x7f4f1c10df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:41.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.380+0000 7f4f236c8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c10e590 con 0x7f4f1c066b50 2024-05-06T12:16:41.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.380+0000 7f4f236c8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c10ede0 con 0x7f4f1c0fff70 2024-05-06T12:16:41.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.381+0000 7f4f21464700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f1c066b50 0x7f4f1c10da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:41.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.381+0000 7f4f21464700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f1c066b50 0x7f4f1c10da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46874/0 (socket says 172.21.15.92:46874) 2024-05-06T12:16:41.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.381+0000 7f4f21464700 1 -- 172.21.15.92:0/2322520522 learned_addr learned my addr 172.21.15.92:0/2322520522 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:16:41.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.381+0000 7f4f20c63700 1 --2- 172.21.15.92:0/2322520522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f1c0fff70 0x7f4f1c10df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:41.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.381+0000 7f4f21464700 1 -- 172.21.15.92:0/2322520522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f1c0fff70 msgr2=0x7f4f1c10df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:41.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.381+0000 7f4f21464700 1 --2- 172.21.15.92:0/2322520522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f1c0fff70 0x7f4f1c10df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:41.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.381+0000 7f4f21464700 1 -- 172.21.15.92:0/2322520522 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f0c005040 con 0x7f4f1c066b50 2024-05-06T12:16:41.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.381+0000 7f4f21464700 1 --2- 172.21.15.92:0/2322520522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f1c066b50 0x7f4f1c10da50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4f1800cae0 tx=0x7f4f18007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:16:41.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.381+0000 7f4f11ffb700 1 -- 172.21.15.92:0/2322520522 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f1800a980 con 0x7f4f1c066b50 2024-05-06T12:16:41.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.381+0000 7f4f236c8700 1 -- 172.21.15.92:0/2322520522 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f1c10f0c0 con 0x7f4f1c066b50 2024-05-06T12:16:41.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.381+0000 7f4f11ffb700 1 -- 172.21.15.92:0/2322520522 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f18012440 con 0x7f4f1c066b50 2024-05-06T12:16:41.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.381+0000 7f4f11ffb700 1 -- 172.21.15.92:0/2322520522 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f180115c0 con 0x7f4f1c066b50 2024-05-06T12:16:41.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.381+0000 7f4f236c8700 1 -- 172.21.15.92:0/2322520522 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f1c10f600 con 0x7f4f1c066b50 2024-05-06T12:16:41.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.382+0000 7f4f236c8700 1 -- 172.21.15.92:0/2322520522 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f1c05f5c0 con 0x7f4f1c066b50 2024-05-06T12:16:41.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.383+0000 7f4f11ffb700 1 -- 172.21.15.92:0/2322520522 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 37) v1 ==== 92249+0+0 (secure 0 0 0) 0x7f4f18012a50 con 0x7f4f1c066b50 2024-05-06T12:16:41.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.383+0000 7f4f11ffb700 1 --2- 172.21.15.92:0/2322520522 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4f0806e1e0 0x7f4f08070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:41.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.383+0000 7f4f11ffb700 1 -- 172.21.15.92:0/2322520522 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f4f1808e190 con 0x7f4f1c066b50 2024-05-06T12:16:41.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.383+0000 7f4f20c63700 1 --2- 172.21.15.92:0/2322520522 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4f0806e1e0 0x7f4f08070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:41.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.384+0000 7f4f20c63700 1 --2- 172.21.15.92:0/2322520522 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4f0806e1e0 0x7f4f08070690 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00b970 tx=0x7f4f0c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:16:41.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.388+0000 7f4f11ffb700 1 -- 172.21.15.92:0/2322520522 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4f1805b010 con 0x7f4f1c066b50 2024-05-06T12:16:41.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.609+0000 7f4f236c8700 1 -- 172.21.15.92:0/2322520522 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f1c10f980 con 0x7f4f0806e1e0 2024-05-06T12:16:41.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.610+0000 7f4f11ffb700 1 -- 172.21.15.92:0/2322520522 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f4f1c10f980 con 0x7f4f0806e1e0 2024-05-06T12:16:41.613 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:16:41.613 INFO:teuthology.orchestra.run.smithi092.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb", 2024-05-06T12:16:41.613 INFO:teuthology.orchestra.run.smithi092.stdout: "in_progress": true, 2024-05-06T12:16:41.613 INFO:teuthology.orchestra.run.smithi092.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-06T12:16:41.613 INFO:teuthology.orchestra.run.smithi092.stdout: "services_complete": [ 2024-05-06T12:16:41.613 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr" 2024-05-06T12:16:41.613 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-05-06T12:16:41.613 INFO:teuthology.orchestra.run.smithi092.stdout: "progress": "2/21 daemons upgraded", 2024-05-06T12:16:41.613 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "", 2024-05-06T12:16:41.614 INFO:teuthology.orchestra.run.smithi092.stdout: "is_paused": false 2024-05-06T12:16:41.614 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:16:41.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.612+0000 7f4f236c8700 1 -- 172.21.15.92:0/2322520522 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4f0806e1e0 msgr2=0x7f4f08070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:41.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.612+0000 7f4f236c8700 1 --2- 172.21.15.92:0/2322520522 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4f0806e1e0 0x7f4f08070690 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00b970 tx=0x7f4f0c009e30 comp rx=0 tx=0).stop 2024-05-06T12:16:41.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.612+0000 7f4f236c8700 1 -- 172.21.15.92:0/2322520522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f1c066b50 msgr2=0x7f4f1c10da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:41.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.612+0000 7f4f236c8700 1 --2- 172.21.15.92:0/2322520522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f1c066b50 0x7f4f1c10da50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4f1800cae0 tx=0x7f4f18007f60 comp rx=0 tx=0).stop 2024-05-06T12:16:41.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.612+0000 7f4f236c8700 1 -- 172.21.15.92:0/2322520522 shutdown_connections 2024-05-06T12:16:41.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.612+0000 7f4f236c8700 1 --2- 172.21.15.92:0/2322520522 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4f0806e1e0 0x7f4f08070690 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:41.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.612+0000 7f4f236c8700 1 --2- 172.21.15.92:0/2322520522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f1c066b50 0x7f4f1c10da50 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:41.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.612+0000 7f4f236c8700 1 --2- 172.21.15.92:0/2322520522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f1c0fff70 0x7f4f1c10df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:41.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.612+0000 7f4f236c8700 1 -- 172.21.15.92:0/2322520522 >> 172.21.15.92:0/2322520522 conn(0x7f4f1c0f74c0 msgr2=0x7f4f1c0f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:16:41.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.612+0000 7f4f236c8700 1 -- 172.21.15.92:0/2322520522 shutdown_connections 2024-05-06T12:16:41.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.612+0000 7f4f236c8700 1 -- 172.21.15.92:0/2322520522 wait complete. 2024-05-06T12:16:41.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.713+0000 7f67481bf700 1 -- 172.21.15.92:0/3168366087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6740071dc0 msgr2=0x7f67400702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:41.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.713+0000 7f67481bf700 1 --2- 172.21.15.92:0/3168366087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6740071dc0 0x7f67400702d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f673c004940 tx=0x7f673c009e30 comp rx=0 tx=0).stop 2024-05-06T12:16:41.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.713+0000 7f67481bf700 1 -- 172.21.15.92:0/3168366087 shutdown_connections 2024-05-06T12:16:41.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.713+0000 7f67481bf700 1 --2- 172.21.15.92:0/3168366087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6740070810 0x7f6740070c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:41.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.713+0000 7f67481bf700 1 --2- 172.21.15.92:0/3168366087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6740071dc0 0x7f67400702d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:41.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.713+0000 7f67481bf700 1 -- 172.21.15.92:0/3168366087 >> 172.21.15.92:0/3168366087 conn(0x7f67400f8540 msgr2=0x7f67400fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:16:41.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.713+0000 7f67481bf700 1 -- 172.21.15.92:0/3168366087 shutdown_connections 2024-05-06T12:16:41.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.713+0000 7f67481bf700 1 -- 172.21.15.92:0/3168366087 wait complete. 2024-05-06T12:16:41.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.713+0000 7f67481bf700 1 Processor -- start 2024-05-06T12:16:41.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.713+0000 7f67481bf700 1 -- start start 2024-05-06T12:16:41.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.713+0000 7f67481bf700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6740070810 0x7f674010fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:41.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.713+0000 7f67481bf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6740071dc0 0x7f67401100b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:41.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.713+0000 7f67481bf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67401106b0 con 0x7f6740071dc0 2024-05-06T12:16:41.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.713+0000 7f67481bf700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6740110820 con 0x7f6740070810 2024-05-06T12:16:41.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.714+0000 7f674575a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6740071dc0 0x7f67401100b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:41.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.714+0000 7f674575a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6740071dc0 0x7f67401100b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46894/0 (socket says 172.21.15.92:46894) 2024-05-06T12:16:41.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.714+0000 7f674575a700 1 -- 172.21.15.92:0/3624508164 learned_addr learned my addr 172.21.15.92:0/3624508164 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:16:41.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.714+0000 7f6745f5b700 1 --2- 172.21.15.92:0/3624508164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6740070810 0x7f674010fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:41.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.714+0000 7f674575a700 1 -- 172.21.15.92:0/3624508164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6740070810 msgr2=0x7f674010fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:41.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.714+0000 7f674575a700 1 --2- 172.21.15.92:0/3624508164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6740070810 0x7f674010fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:41.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.714+0000 7f674575a700 1 -- 172.21.15.92:0/3624508164 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f673c005040 con 0x7f6740071dc0 2024-05-06T12:16:41.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.714+0000 7f6745f5b700 1 --2- 172.21.15.92:0/3624508164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6740070810 0x7f674010fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-06T12:16:41.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.714+0000 7f674575a700 1 --2- 172.21.15.92:0/3624508164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6740071dc0 0x7f67401100b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6730002a50 tx=0x7f67300046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:16:41.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.714+0000 7f67367fc700 1 -- 172.21.15.92:0/3624508164 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6730007cc0 con 0x7f6740071dc0 2024-05-06T12:16:41.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.714+0000 7f67367fc700 1 -- 172.21.15.92:0/3624508164 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f673000f040 con 0x7f6740071dc0 2024-05-06T12:16:41.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.714+0000 7f67367fc700 1 -- 172.21.15.92:0/3624508164 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6730009640 con 0x7f6740071dc0 2024-05-06T12:16:41.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.714+0000 7f67481bf700 1 -- 172.21.15.92:0/3624508164 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6740115280 con 0x7f6740071dc0 2024-05-06T12:16:41.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.714+0000 7f67481bf700 1 -- 172.21.15.92:0/3624508164 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6740115840 con 0x7f6740071dc0 2024-05-06T12:16:41.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.715+0000 7f67481bf700 1 -- 172.21.15.92:0/3624508164 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f674004c710 con 0x7f6740071dc0 2024-05-06T12:16:41.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.716+0000 7f67367fc700 1 -- 172.21.15.92:0/3624508164 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 37) v1 ==== 92249+0+0 (secure 0 0 0) 0x7f67300097a0 con 0x7f6740071dc0 2024-05-06T12:16:41.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.716+0000 7f67367fc700 1 --2- 172.21.15.92:0/3624508164 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f672c06e1d0 0x7f672c070680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:16:41.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.716+0000 7f67367fc700 1 -- 172.21.15.92:0/3624508164 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f673008d9c0 con 0x7f6740071dc0 2024-05-06T12:16:41.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.716+0000 7f6745f5b700 1 --2- 172.21.15.92:0/3624508164 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f672c06e1d0 0x7f672c070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:16:41.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.717+0000 7f6745f5b700 1 --2- 172.21.15.92:0/3624508164 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f672c06e1d0 0x7f672c070680 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f673c00b970 tx=0x7f673c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:16:41.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:41.721+0000 7f67367fc700 1 -- 172.21.15.92:0/3624508164 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f673005a7b0 con 0x7f6740071dc0 2024-05-06T12:16:41.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:41 smithi092 ceph-mon[91933]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:16:41.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:41 smithi092 ceph-mon[91933]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:16:41.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:41 smithi092 ceph-mon[91933]: pgmap v4: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-05-06T12:16:41.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:41 smithi092 ceph-mon[91933]: mgrmap e37: smithi092.gyzeqb(active, since 2s), standbys: smithi190.kczxkh 2024-05-06T12:16:41.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:41 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/2590032353' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:16:41.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:41 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/3709420616' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:16:42.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:42.009+0000 7f67481bf700 1 -- 172.21.15.92:0/3624508164 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6740115ce0 con 0x7f6740071dc0 2024-05-06T12:16:42.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:42.009+0000 7f67367fc700 1 -- 172.21.15.92:0/3624508164 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f673005a340 con 0x7f6740071dc0 2024-05-06T12:16:42.013 INFO:teuthology.orchestra.run.smithi092.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-06T12:16:42.013 INFO:teuthology.orchestra.run.smithi092.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-06T12:16:42.013 INFO:teuthology.orchestra.run.smithi092.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-06T12:16:42.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:42.011+0000 7f67481bf700 1 -- 172.21.15.92:0/3624508164 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f672c06e1d0 msgr2=0x7f672c070680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:42.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:42.011+0000 7f67481bf700 1 --2- 172.21.15.92:0/3624508164 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f672c06e1d0 0x7f672c070680 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f673c00b970 tx=0x7f673c009e30 comp rx=0 tx=0).stop 2024-05-06T12:16:42.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:42.011+0000 7f67481bf700 1 -- 172.21.15.92:0/3624508164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6740071dc0 msgr2=0x7f67401100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:16:42.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:42.011+0000 7f67481bf700 1 --2- 172.21.15.92:0/3624508164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6740071dc0 0x7f67401100b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6730002a50 tx=0x7f67300046b0 comp rx=0 tx=0).stop 2024-05-06T12:16:42.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:42.011+0000 7f67481bf700 1 -- 172.21.15.92:0/3624508164 shutdown_connections 2024-05-06T12:16:42.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:42.011+0000 7f67481bf700 1 --2- 172.21.15.92:0/3624508164 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f672c06e1d0 0x7f672c070680 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:42.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:42.011+0000 7f67481bf700 1 --2- 172.21.15.92:0/3624508164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6740070810 0x7f674010fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:42.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:42.011+0000 7f67481bf700 1 --2- 172.21.15.92:0/3624508164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6740071dc0 0x7f67401100b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:16:42.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:42.011+0000 7f67481bf700 1 -- 172.21.15.92:0/3624508164 >> 172.21.15.92:0/3624508164 conn(0x7f67400f8540 msgr2=0x7f67400f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:16:42.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:42.012+0000 7f67481bf700 1 -- 172.21.15.92:0/3624508164 shutdown_connections 2024-05-06T12:16:42.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:16:42.012+0000 7f67481bf700 1 -- 172.21.15.92:0/3624508164 wait complete. 2024-05-06T12:16:42.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:41 smithi190 ceph-mon[33798]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:16:42.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:41 smithi190 ceph-mon[33798]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:16:42.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:41 smithi190 ceph-mon[33798]: pgmap v4: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-05-06T12:16:42.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:41 smithi190 ceph-mon[33798]: mgrmap e37: smithi092.gyzeqb(active, since 2s), standbys: smithi190.kczxkh 2024-05-06T12:16:42.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:41 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/2590032353' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:16:42.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:41 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/3709420616' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:16:43.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:42 smithi190 ceph-mon[33798]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:16:43.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:42 smithi190 ceph-mon[33798]: from='client.? 172.21.15.92:0/3624508164' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:16:43.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:42 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:43.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:42 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:43.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:42 smithi092 ceph-mon[91933]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:16:43.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:42 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/3624508164' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:16:43.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:42 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:43.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:42 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:43.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:43 smithi092 ceph-mon[91933]: pgmap v5: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-05-06T12:16:43.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:43 smithi092 ceph-mon[91933]: mgrmap e38: smithi092.gyzeqb(active, since 4s), standbys: smithi190.kczxkh 2024-05-06T12:16:43.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:43 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:43.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:43 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:44.081 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:43 smithi190 ceph-mon[33798]: pgmap v5: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-05-06T12:16:44.081 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:43 smithi190 ceph-mon[33798]: mgrmap e38: smithi092.gyzeqb(active, since 4s), standbys: smithi190.kczxkh 2024-05-06T12:16:44.081 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:43 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:44.081 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:43 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:45.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:45 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:45.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:45 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:45.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:45 smithi190 ceph-mon[33798]: pgmap v6: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-05-06T12:16:45.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:45 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:45.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:45 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:45.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:45 smithi092 ceph-mon[91933]: pgmap v6: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-05-06T12:16:46.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:46 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:46.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:46 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:46.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:46 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:46.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:46 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:47.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:47 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:47.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:47 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:47.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:47 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:16:47.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:47 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:16:47.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:47 smithi092 ceph-mon[91933]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-06T12:16:47.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:47 smithi092 ceph-mon[91933]: Updating smithi190:/etc/ceph/ceph.conf 2024-05-06T12:16:47.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:47 smithi092 ceph-mon[91933]: pgmap v7: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-05-06T12:16:47.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:47 smithi092 ceph-mon[91933]: Updating smithi092:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-06T12:16:47.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:47 smithi092 ceph-mon[91933]: Updating smithi190:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-06T12:16:47.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:47 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:47.974 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:47 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:47.974 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:47 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:47.974 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:47 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:47.974 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:47 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:48.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:47 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:48.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:47 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:48.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:47 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:16:48.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:47 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:16:48.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:47 smithi190 ceph-mon[33798]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-06T12:16:48.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:47 smithi190 ceph-mon[33798]: Updating smithi190:/etc/ceph/ceph.conf 2024-05-06T12:16:48.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:47 smithi190 ceph-mon[33798]: pgmap v7: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-05-06T12:16:48.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:47 smithi190 ceph-mon[33798]: Updating smithi092:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-06T12:16:48.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:47 smithi190 ceph-mon[33798]: Updating smithi190:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-06T12:16:48.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:47 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:48.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:47 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:48.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:47 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:48.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:47 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:48.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:47 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:48.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:48 smithi190 ceph-mon[33798]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-05-06T12:16:48.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:48 smithi190 ceph-mon[33798]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-05-06T12:16:48.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:48 smithi190 ceph-mon[33798]: Updating smithi092:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-06T12:16:48.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:48 smithi190 ceph-mon[33798]: Updating smithi190:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-06T12:16:48.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:48 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:16:48.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:48 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:16:48.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:48 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-05-06T12:16:48.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:48 smithi092 ceph-mon[91933]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-05-06T12:16:48.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:48 smithi092 ceph-mon[91933]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-05-06T12:16:48.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:48 smithi092 ceph-mon[91933]: Updating smithi092:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-06T12:16:48.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:48 smithi092 ceph-mon[91933]: Updating smithi190:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-06T12:16:48.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:48 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:16:48.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:48 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:16:48.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:48 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-05-06T12:16:49.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:49 smithi190 ceph-mon[33798]: pgmap v8: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-05-06T12:16:49.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:49 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:49.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:49 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-06T12:16:49.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:49 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-06T12:16:49.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:49 smithi190 ceph-mon[33798]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:16:49.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:49 smithi092 ceph-mon[91933]: pgmap v8: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-05-06T12:16:49.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:49.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-06T12:16:49.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-06T12:16:49.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:16:50.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:50 smithi092 ceph-mon[91933]: Upgrade: Updating mon.smithi190 2024-05-06T12:16:50.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:50 smithi092 ceph-mon[91933]: Deploying daemon mon.smithi190 on smithi190 2024-05-06T12:16:51.036 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:50 smithi190 ceph-mon[33798]: Upgrade: Updating mon.smithi190 2024-05-06T12:16:51.036 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:50 smithi190 ceph-mon[33798]: Deploying daemon mon.smithi190 on smithi190 2024-05-06T12:16:51.593 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:51 smithi190 systemd[1]: Stopping Ceph mon.smithi190 for aee1d912-0ba0-11ef-bc97-c7b262605968... 2024-05-06T12:16:51.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:51 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi190[33779]: 2024-05-06T12:16:51.589+0000 7f43563ce700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi190 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-06T12:16:51.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:51 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi190[33779]: 2024-05-06T12:16:51.589+0000 7f43563ce700 -1 mon.smithi190@1(peon) e2 *** Got Signal Terminated *** 2024-05-06T12:16:51.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:51 smithi190 podman[79579]: 2024-05-06 12:16:51.804306628 +0000 UTC m=+0.348601234 container died 912ded0726019a4b1ad55443d47ff960f64cb1edaea616a1ae92039695091e69 (image=quay.ceph.io/ceph-ci/ceph@sha256:1a0b7fd3cefd2c5baddf682493f92b4e27596f3c0ff27b02fdccd2891f56859f, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi190, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=quincy-e81df42, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-06T12:16:52.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:52 smithi190 podman[79579]: 2024-05-06 12:16:52.36507705 +0000 UTC m=+0.909371706 container cleanup 912ded0726019a4b1ad55443d47ff960f64cb1edaea616a1ae92039695091e69 (image=quay.ceph.io/ceph-ci/ceph@sha256:1a0b7fd3cefd2c5baddf682493f92b4e27596f3c0ff27b02fdccd2891f56859f, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi190, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, RELEASE=quincy-e81df42, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430) 2024-05-06T12:16:52.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:52 smithi190 bash[79579]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi190 2024-05-06T12:16:52.986 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:52 smithi190 podman[79593]: 2024-05-06 12:16:52.632789167 +0000 UTC m=+0.826969807 container remove 912ded0726019a4b1ad55443d47ff960f64cb1edaea616a1ae92039695091e69 (image=quay.ceph.io/ceph-ci/ceph@sha256:1a0b7fd3cefd2c5baddf682493f92b4e27596f3c0ff27b02fdccd2891f56859f, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi190, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-e81df42, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-06T12:16:52.986 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:52 smithi190 systemd[1]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@mon.smithi190.service: Deactivated successfully. 2024-05-06T12:16:52.986 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:52 smithi190 systemd[1]: Stopped Ceph mon.smithi190 for aee1d912-0ba0-11ef-bc97-c7b262605968. 2024-05-06T12:16:52.986 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:52 smithi190 systemd[1]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@mon.smithi190.service: Consumed 5.945s CPU time. 2024-05-06T12:16:53.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:52 smithi190 systemd[1]: Starting Ceph mon.smithi190 for aee1d912-0ba0-11ef-bc97-c7b262605968... 2024-05-06T12:16:53.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 podman[79725]: 2024-05-06 12:16:53.102024073 +0000 UTC m=+0.018190304 image pull e4ce8ad5d0c4b9ea60bc5f6b8b0fdb5efbe59bf7771fb0920fc2200e6ae546b0 quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb 2024-05-06T12:16:53.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 podman[79725]: 2024-05-06 12:16:53.233750296 +0000 UTC m=+0.149916513 container create 7a3c08eb716fcaa9b3412d4694100f785857fdb65f0f0c02037f77fa2ad1a43a (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi190, org.label-schema.schema-version=1.0, RELEASE=squid-70ce953, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430) 2024-05-06T12:16:53.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 podman[79725]: 2024-05-06 12:16:53.444910581 +0000 UTC m=+0.361076801 container init 7a3c08eb716fcaa9b3412d4694100f785857fdb65f0f0c02037f77fa2ad1a43a (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi190, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-70ce953, CEPH_POINT_RELEASE=) 2024-05-06T12:16:53.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 podman[79725]: 2024-05-06 12:16:53.447936934 +0000 UTC m=+0.364103151 container start 7a3c08eb716fcaa9b3412d4694100f785857fdb65f0f0c02037f77fa2ad1a43a (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi190, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-70ce953, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=) 2024-05-06T12:16:53.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: set uid:gid to 167:167 (ceph:ceph) 2024-05-06T12:16:53.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev), process ceph-mon, pid 2 2024-05-06T12:16:53.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: pidfile_write: ignore empty --pid-file 2024-05-06T12:16:53.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: load: jerasure load: lrc 2024-05-06T12:16:53.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: RocksDB version: 7.9.2 2024-05-06T12:16:53.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Git sha 0 2024-05-06T12:16:53.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Compile date 2024-05-03 15:08:47 2024-05-06T12:16:53.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: DB SUMMARY 2024-05-06T12:16:53.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: DB Session ID: 3PXV0AKZHMSA7G3VB5P7 2024-05-06T12:16:53.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: CURRENT file: CURRENT 2024-05-06T12:16:53.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: IDENTITY file: IDENTITY 2024-05-06T12:16:53.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: MANIFEST file: MANIFEST-000009 size: 897 Bytes 2024-05-06T12:16:53.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi190/store.db dir, Total Num: 1, files: 000024.sst 2024-05-06T12:16:53.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi190/store.db: 000022.log size: 6339379 ; 2024-05-06T12:16:53.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.error_if_exists: 0 2024-05-06T12:16:53.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.create_if_missing: 0 2024-05-06T12:16:53.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.paranoid_checks: 1 2024-05-06T12:16:53.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-06T12:16:53.851 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-06T12:16:53.851 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-06T12:16:53.851 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.env: 0x562493a85c60 2024-05-06T12:16:53.851 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.fs: PosixFileSystem 2024-05-06T12:16:53.851 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.info_log: 0x5624945ab980 2024-05-06T12:16:53.851 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.max_file_opening_threads: 16 2024-05-06T12:16:53.853 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.statistics: (nil) 2024-05-06T12:16:53.853 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.use_fsync: 0 2024-05-06T12:16:53.853 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.max_log_file_size: 0 2024-05-06T12:16:53.853 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-06T12:16:53.853 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-06T12:16:53.853 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.keep_log_file_num: 1000 2024-05-06T12:16:53.853 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.recycle_log_file_num: 0 2024-05-06T12:16:53.853 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.allow_fallocate: 1 2024-05-06T12:16:53.853 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.allow_mmap_reads: 0 2024-05-06T12:16:53.853 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.allow_mmap_writes: 0 2024-05-06T12:16:53.854 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.use_direct_reads: 0 2024-05-06T12:16:53.854 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-06T12:16:53.854 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.create_missing_column_families: 0 2024-05-06T12:16:53.854 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.db_log_dir: 2024-05-06T12:16:53.854 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.wal_dir: 2024-05-06T12:16:53.854 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-06T12:16:53.854 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-06T12:16:53.854 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-06T12:16:53.854 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-06T12:16:53.854 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-06T12:16:53.854 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-06T12:16:53.854 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.advise_random_on_open: 1 2024-05-06T12:16:53.854 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.db_write_buffer_size: 0 2024-05-06T12:16:53.854 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.write_buffer_manager: 0x56249457e1e0 2024-05-06T12:16:53.855 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-06T12:16:53.855 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-06T12:16:53.855 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-06T12:16:53.855 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.rate_limiter: (nil) 2024-05-06T12:16:53.855 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-06T12:16:53.855 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.wal_recovery_mode: 2 2024-05-06T12:16:53.855 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.enable_thread_tracking: 0 2024-05-06T12:16:53.855 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.enable_pipelined_write: 0 2024-05-06T12:16:53.855 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.unordered_write: 0 2024-05-06T12:16:53.855 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-06T12:16:53.855 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-06T12:16:53.855 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-06T12:16:53.855 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-06T12:16:53.855 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.row_cache: None 2024-05-06T12:16:53.855 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.wal_filter: None 2024-05-06T12:16:53.856 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-06T12:16:53.856 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.allow_ingest_behind: 0 2024-05-06T12:16:53.856 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.two_write_queues: 0 2024-05-06T12:16:53.856 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.manual_wal_flush: 0 2024-05-06T12:16:53.856 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.wal_compression: 0 2024-05-06T12:16:53.856 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.atomic_flush: 0 2024-05-06T12:16:53.856 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-06T12:16:53.856 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-06T12:16:53.856 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-06T12:16:53.856 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.log_readahead_size: 0 2024-05-06T12:16:53.856 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-06T12:16:53.856 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.best_efforts_recovery: 0 2024-05-06T12:16:53.856 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-06T12:16:53.856 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-06T12:16:53.856 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.allow_data_in_errors: 0 2024-05-06T12:16:53.857 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.db_host_id: __hostname__ 2024-05-06T12:16:53.857 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-06T12:16:53.857 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.max_background_jobs: 2 2024-05-06T12:16:53.857 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.max_background_compactions: -1 2024-05-06T12:16:53.857 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.max_subcompactions: 1 2024-05-06T12:16:53.857 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-06T12:16:53.857 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-06T12:16:53.857 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-06T12:16:53.857 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.max_total_wal_size: 0 2024-05-06T12:16:53.857 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-06T12:16:53.857 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-06T12:16:53.857 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-06T12:16:53.857 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-06T12:16:53.858 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.max_open_files: -1 2024-05-06T12:16:53.858 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.bytes_per_sync: 0 2024-05-06T12:16:53.858 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-06T12:16:53.858 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-06T12:16:53.858 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.compaction_readahead_size: 0 2024-05-06T12:16:53.858 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.max_background_flushes: -1 2024-05-06T12:16:53.858 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Compression algorithms supported: 2024-05-06T12:16:53.858 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-06T12:16:53.858 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: kZSTD supported: 0 2024-05-06T12:16:53.858 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: kXpressCompression supported: 0 2024-05-06T12:16:53.858 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: kLZ4HCCompression supported: 1 2024-05-06T12:16:53.858 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: kZlibCompression supported: 1 2024-05-06T12:16:53.858 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: kSnappyCompression supported: 1 2024-05-06T12:16:53.858 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: kLZ4Compression supported: 1 2024-05-06T12:16:53.859 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: kBZip2Compression supported: 0 2024-05-06T12:16:53.859 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-06T12:16:53.859 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-06T12:16:53.859 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi190/store.db/MANIFEST-000009 2024-05-06T12:16:53.859 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-06T12:16:53.859 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-06T12:16:53.859 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.merge_operator: 2024-05-06T12:16:53.859 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.compaction_filter: None 2024-05-06T12:16:53.859 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.compaction_filter_factory: None 2024-05-06T12:16:53.859 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.sst_partitioner_factory: None 2024-05-06T12:16:53.859 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-06T12:16:53.859 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-06T12:16:53.859 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5624945aba80) 2024-05-06T12:16:53.859 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: cache_index_and_filter_blocks: 1 2024-05-06T12:16:53.860 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-06T12:16:53.860 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-06T12:16:53.860 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: pin_top_level_index_and_filter: 1 2024-05-06T12:16:53.860 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: index_type: 0 2024-05-06T12:16:53.860 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: data_block_index_type: 0 2024-05-06T12:16:53.860 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: index_shortening: 1 2024-05-06T12:16:53.860 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-06T12:16:53.860 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: checksum: 4 2024-05-06T12:16:53.860 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: no_block_cache: 0 2024-05-06T12:16:53.860 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: block_cache: 0x5624945af090 2024-05-06T12:16:53.860 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: block_cache_name: BinnedLRUCache 2024-05-06T12:16:53.860 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: block_cache_options: 2024-05-06T12:16:53.860 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: capacity : 536870912 2024-05-06T12:16:53.860 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: num_shard_bits : 4 2024-05-06T12:16:53.860 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: strict_capacity_limit : 0 2024-05-06T12:16:53.861 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: high_pri_pool_ratio: 0.000 2024-05-06T12:16:53.861 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: block_cache_compressed: (nil) 2024-05-06T12:16:53.861 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: persistent_cache: (nil) 2024-05-06T12:16:53.861 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: block_size: 4096 2024-05-06T12:16:53.861 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: block_size_deviation: 10 2024-05-06T12:16:53.861 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: block_restart_interval: 16 2024-05-06T12:16:53.861 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: index_block_restart_interval: 1 2024-05-06T12:16:53.861 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: metadata_block_size: 4096 2024-05-06T12:16:53.861 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: partition_filters: 0 2024-05-06T12:16:53.861 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: use_delta_encoding: 1 2024-05-06T12:16:53.861 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: filter_policy: bloomfilter 2024-05-06T12:16:53.861 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: whole_key_filtering: 1 2024-05-06T12:16:53.861 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: verify_compression: 0 2024-05-06T12:16:53.861 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: read_amp_bytes_per_bit: 0 2024-05-06T12:16:53.861 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: format_version: 5 2024-05-06T12:16:53.862 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: enable_index_compression: 1 2024-05-06T12:16:53.862 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: block_align: 0 2024-05-06T12:16:53.862 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: max_auto_readahead_size: 262144 2024-05-06T12:16:53.862 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: prepopulate_block_cache: 0 2024-05-06T12:16:53.862 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: initial_auto_readahead_size: 8192 2024-05-06T12:16:53.862 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: num_file_reads_for_auto_readahead: 2 2024-05-06T12:16:53.862 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.write_buffer_size: 33554432 2024-05-06T12:16:53.862 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.max_write_buffer_number: 2 2024-05-06T12:16:53.862 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.compression: NoCompression 2024-05-06T12:16:53.862 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.bottommost_compression: Disabled 2024-05-06T12:16:53.862 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.prefix_extractor: nullptr 2024-05-06T12:16:53.862 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-06T12:16:53.862 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.num_levels: 7 2024-05-06T12:16:53.862 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-06T12:16:53.863 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-06T12:16:53.863 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-06T12:16:53.863 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-06T12:16:53.863 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-06T12:16:53.863 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-06T12:16:53.863 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-06T12:16:53.863 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-06T12:16:53.863 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-06T12:16:53.863 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-06T12:16:53.863 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-06T12:16:53.863 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-06T12:16:53.863 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-06T12:16:53.863 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.compression_opts.level: 32767 2024-05-06T12:16:53.863 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.compression_opts.strategy: 0 2024-05-06T12:16:53.864 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-06T12:16:53.864 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-06T12:16:53.864 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-06T12:16:53.864 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-06T12:16:53.864 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.compression_opts.enabled: false 2024-05-06T12:16:53.864 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-06T12:16:53.864 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-06T12:16:53.864 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-06T12:16:53.864 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-06T12:16:53.864 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.target_file_size_base: 67108864 2024-05-06T12:16:53.864 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-06T12:16:53.865 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-06T12:16:53.865 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-06T12:16:53.865 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-06T12:16:53.865 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-06T12:16:53.865 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-06T12:16:53.865 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-06T12:16:53.865 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-06T12:16:53.865 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-06T12:16:53.865 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-06T12:16:53.865 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-06T12:16:53.865 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-06T12:16:53.865 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-06T12:16:53.865 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-06T12:16:53.866 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.arena_block_size: 1048576 2024-05-06T12:16:53.866 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-06T12:16:53.866 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-06T12:16:53.866 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.disable_auto_compactions: 0 2024-05-06T12:16:53.866 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-06T12:16:53.866 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-06T12:16:53.866 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-06T12:16:53.866 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-06T12:16:53.866 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-06T12:16:53.866 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-06T12:16:53.866 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-06T12:16:53.866 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-06T12:16:53.866 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-06T12:16:53.866 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-06T12:16:53.867 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.table_properties_collectors: 2024-05-06T12:16:53.867 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.inplace_update_support: 0 2024-05-06T12:16:53.867 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-06T12:16:53.867 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-06T12:16:53.867 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-06T12:16:53.867 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-06T12:16:53.867 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.bloom_locality: 0 2024-05-06T12:16:53.867 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.max_successive_merges: 0 2024-05-06T12:16:53.867 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-06T12:16:53.867 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.paranoid_file_checks: 0 2024-05-06T12:16:53.867 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.force_consistency_checks: 1 2024-05-06T12:16:53.867 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.report_bg_io_stats: 0 2024-05-06T12:16:53.867 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.ttl: 2592000 2024-05-06T12:16:53.867 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-06T12:16:53.867 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-06T12:16:53.868 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-06T12:16:53.868 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.enable_blob_files: false 2024-05-06T12:16:53.868 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.min_blob_size: 0 2024-05-06T12:16:53.868 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.blob_file_size: 268435456 2024-05-06T12:16:53.868 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-06T12:16:53.868 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-06T12:16:53.868 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-06T12:16:53.868 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-06T12:16:53.868 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-06T12:16:53.868 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.blob_file_starting_level: 0 2024-05-06T12:16:53.868 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-06T12:16:53.868 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 24.sst 2024-05-06T12:16:53.868 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-05-06T12:16:53.868 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi190/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 26, last_sequence is 10446, log_number is 22,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-06T12:16:53.869 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 22 2024-05-06T12:16:53.869 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f9488442-92ab-43ee-bccc-b0dcde0e1353 2024-05-06T12:16:53.869 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714997813476906, "job": 1, "event": "recovery_started", "wal_files": [22]} 2024-05-06T12:16:53.869 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #22 mode 2 2024-05-06T12:16:53.869 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714997813492304, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 27, "file_size": 3805531, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 10451, "largest_seqno": 12007, "table_properties": {"data_size": 3798540, "index_size": 4308, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1669, "raw_key_size": 15335, "raw_average_key_size": 23, "raw_value_size": 3784586, "raw_average_value_size": 5804, "num_data_blocks": 196, "num_entries": 652, "num_filter_entries": 652, "num_deletions": 2, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714997813, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f9488442-92ab-43ee-bccc-b0dcde0e1353", "db_session_id": "3PXV0AKZHMSA7G3VB5P7", "orig_file_number": 27, "seqno_to_time_mapping": "N/A"}} 2024-05-06T12:16:53.869 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714997813492428, "job": 1, "event": "recovery_finished"} 2024-05-06T12:16:53.869 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: [db/version_set.cc:5047] Creating manifest 29 2024-05-06T12:16:53.869 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-05-06T12:16:53.869 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi190/store.db/000022.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-06T12:16:53.869 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5624946a0000 2024-05-06T12:16:53.869 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: DB pointer 0x56249468a000 2024-05-06T12:16:53.869 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-06T12:16:53.869 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-06T12:16:53.869 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: ** DB Stats ** 2024-05-06T12:16:53.869 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-06T12:16:53.870 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-06T12:16:53.870 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-06T12:16:53.870 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-06T12:16:53.870 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-06T12:16:53.870 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-06T12:16:53.870 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-06T12:16:53.870 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: 2024-05-06T12:16:53.870 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: ** Compaction Stats [default] ** 2024-05-06T12:16:53.870 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-06T12:16:53.870 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-06T12:16:53.870 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: L0 1/0 3.63 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 499.7 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-05-06T12:16:53.870 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: L6 1/0 5.24 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-05-06T12:16:53.870 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: Sum 2/0 8.87 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 499.7 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-05-06T12:16:53.870 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 499.7 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-05-06T12:16:53.871 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: 2024-05-06T12:16:53.871 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: ** Compaction Stats [default] ** 2024-05-06T12:16:53.871 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-06T12:16:53.871 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-06T12:16:53.871 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 499.7 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-05-06T12:16:53.871 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: 2024-05-06T12:16:53.871 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-06T12:16:53.871 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: 2024-05-06T12:16:53.871 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-06T12:16:53.871 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: Flush(GB): cumulative 0.004, interval 0.004 2024-05-06T12:16:53.871 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-06T12:16:53.871 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-06T12:16:53.871 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-06T12:16:53.872 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-06T12:16:53.872 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: Cumulative compaction: 0.00 GB write, 174.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-06T12:16:53.872 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: Interval compaction: 0.00 GB write, 174.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-06T12:16:53.872 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-06T12:16:53.872 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: Block cache BinnedLRUCache@0x5624945af090#2 capacity: 512.00 MB usage: 6.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-05-06T12:16:53.872 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.78 KB,0.000339746%) IndexBlock(1,4.58 KB,0.000873208%) Misc(1,0.00 KB,0%) 2024-05-06T12:16:53.872 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: 2024-05-06T12:16:53.872 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-06T12:16:53.872 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: starting mon.smithi190 rank 1 at public addrs [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] at bind addrs [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi190 fsid aee1d912-0ba0-11ef-bc97-c7b262605968 2024-05-06T12:16:53.872 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: mon.smithi190@-1(???) e2 preinit fsid aee1d912-0ba0-11ef-bc97-c7b262605968 2024-05-06T12:16:53.872 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: mon.smithi190@-1(???).mds e20 new map 2024-05-06T12:16:53.872 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: mon.smithi190@-1(???).mds e20 print_map 2024-05-06T12:16:53.872 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: e20 2024-05-06T12:16:53.872 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-06T12:16:53.872 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:16:53.873 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: legacy client fscid: 1 2024-05-06T12:16:53.873 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: 2024-05-06T12:16:53.873 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: Filesystem 'cephfs' (1) 2024-05-06T12:16:53.873 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: fs_name cephfs 2024-05-06T12:16:53.873 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: epoch 20 2024-05-06T12:16:53.873 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-06T12:16:53.873 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: created 2024-05-06T12:11:44.952671+0000 2024-05-06T12:16:53.873 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: modified 2024-05-06T12:15:07.568167+0000 2024-05-06T12:16:53.873 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: tableserver 0 2024-05-06T12:16:53.873 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: root 0 2024-05-06T12:16:53.873 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: session_timeout 60 2024-05-06T12:16:53.873 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: session_autoclose 300 2024-05-06T12:16:53.873 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: max_file_size 1099511627776 2024-05-06T12:16:53.873 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: max_xattr_size 65536 2024-05-06T12:16:53.873 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: required_client_features {} 2024-05-06T12:16:53.873 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: last_failure 0 2024-05-06T12:16:53.874 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: last_failure_osd_epoch 46 2024-05-06T12:16:53.874 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:16:53.874 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: max_mds 2 2024-05-06T12:16:53.874 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: in 0,1 2024-05-06T12:16:53.874 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: up {0=14472,1=24267} 2024-05-06T12:16:53.874 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: failed 2024-05-06T12:16:53.874 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: damaged 2024-05-06T12:16:53.874 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: stopped 2024-05-06T12:16:53.874 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: data_pools [3] 2024-05-06T12:16:53.875 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: metadata_pool 2 2024-05-06T12:16:53.875 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: inline_data enabled 2024-05-06T12:16:53.875 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: balancer 2024-05-06T12:16:53.875 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: bal_rank_mask -1 2024-05-06T12:16:53.875 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: standby_count_wanted 1 2024-05-06T12:16:53.875 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: qdb_cluster leader: 0 members: 2024-05-06T12:16:53.875 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: [mds.cephfs.smithi092.rosbll{0:14472} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.92:6828/1663253598,v1:172.21.15.92:6829/1663253598] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:16:53.875 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: [mds.cephfs.smithi092.ijsqgp{0:14484} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.92:6826/3613441054,v1:172.21.15.92:6827/3613441054] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:16:53.875 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: [mds.cephfs.smithi190.byjvgf{1:24267} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.190:6824/1565151699,v1:172.21.15.190:6825/1565151699] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:16:53.875 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: [mds.cephfs.smithi190.idzoff{1:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.190:6826/2453728380,v1:172.21.15.190:6827/2453728380] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:16:53.875 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: 2024-05-06T12:16:53.875 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout: 2024-05-06T12:16:53.875 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: mon.smithi190@-1(???).osd e50 crush map has features 3314933000852226048, adjusting msgr requires 2024-05-06T12:16:53.875 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: mon.smithi190@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-05-06T12:16:53.875 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: mon.smithi190@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-05-06T12:16:53.876 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: mon.smithi190@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-05-06T12:16:53.876 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 ceph-mon[79744]: mon.smithi190@-1(???).paxosservice(auth 1..21) refresh upgraded, format 0 -> 3 2024-05-06T12:16:53.876 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 bash[79725]: 7a3c08eb716fcaa9b3412d4694100f785857fdb65f0f0c02037f77fa2ad1a43a 2024-05-06T12:16:53.876 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:53 smithi190 systemd[1]: Started Ceph mon.smithi190 for aee1d912-0ba0-11ef-bc97-c7b262605968. 2024-05-06T12:16:56.078 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:55 smithi190 ceph-mon[79744]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-06T12:16:56.078 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:55 smithi190 ceph-mon[79744]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-06T12:16:56.078 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:55 smithi190 ceph-mon[79744]: fs cephfs has deprecated feature inline_data enabled. 2024-05-06T12:16:56.078 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:55 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi092"}]: dispatch 2024-05-06T12:16:56.078 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:55 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi190"}]: dispatch 2024-05-06T12:16:56.078 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:55 smithi190 ceph-mon[79744]: mon.smithi092 calling monitor election 2024-05-06T12:16:56.078 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:55 smithi190 ceph-mon[79744]: mon.smithi190 calling monitor election 2024-05-06T12:16:56.078 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:55 smithi190 ceph-mon[79744]: mon.smithi092 is new leader, mons smithi092,smithi190 in quorum (ranks 0,1) 2024-05-06T12:16:56.079 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:55 smithi190 ceph-mon[79744]: monmap epoch 3 2024-05-06T12:16:56.079 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:55 smithi190 ceph-mon[79744]: fsid aee1d912-0ba0-11ef-bc97-c7b262605968 2024-05-06T12:16:56.079 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:55 smithi190 ceph-mon[79744]: last_changed 2024-05-06T12:16:54.738705+0000 2024-05-06T12:16:56.079 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:55 smithi190 ceph-mon[79744]: created 2024-05-06T12:04:48.255830+0000 2024-05-06T12:16:56.079 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:55 smithi190 ceph-mon[79744]: min_mon_release 19 (squid) 2024-05-06T12:16:56.079 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:55 smithi190 ceph-mon[79744]: election_strategy: 1 2024-05-06T12:16:56.079 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:55 smithi190 ceph-mon[79744]: 0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.smithi092 2024-05-06T12:16:56.079 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:55 smithi190 ceph-mon[79744]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.smithi190 2024-05-06T12:16:56.080 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:55 smithi190 ceph-mon[79744]: fsmap cephfs:2 {0=cephfs.smithi092.rosbll=up:active,1=cephfs.smithi190.byjvgf=up:active} 2 up:standby-replay 2024-05-06T12:16:56.080 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:55 smithi190 ceph-mon[79744]: osdmap e50: 6 total, 6 up, 6 in 2024-05-06T12:16:56.080 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:55 smithi190 ceph-mon[79744]: mgrmap e38: smithi092.gyzeqb(active, since 15s), standbys: smithi190.kczxkh 2024-05-06T12:16:56.080 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:55 smithi190 ceph-mon[79744]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-06T12:16:56.080 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:55 smithi190 ceph-mon[79744]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-06T12:16:56.080 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:55 smithi190 ceph-mon[79744]: fs cephfs has deprecated feature inline_data enabled. 2024-05-06T12:16:56.080 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:55 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:56.080 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:55 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:56.080 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:55 smithi190 ceph-mon[79744]: pgmap v11: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 13 op/s 2024-05-06T12:16:56.080 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:55 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:16:56.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:55 smithi092 ceph-mon[91933]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-06T12:16:56.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:55 smithi092 ceph-mon[91933]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-06T12:16:56.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:55 smithi092 ceph-mon[91933]: fs cephfs has deprecated feature inline_data enabled. 2024-05-06T12:16:56.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:55 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi092"}]: dispatch 2024-05-06T12:16:56.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:55 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mon metadata", "id": "smithi190"}]: dispatch 2024-05-06T12:16:56.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:55 smithi092 ceph-mon[91933]: mon.smithi092 calling monitor election 2024-05-06T12:16:56.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:55 smithi092 ceph-mon[91933]: mon.smithi190 calling monitor election 2024-05-06T12:16:56.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:55 smithi092 ceph-mon[91933]: mon.smithi092 is new leader, mons smithi092,smithi190 in quorum (ranks 0,1) 2024-05-06T12:16:56.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:55 smithi092 ceph-mon[91933]: monmap epoch 3 2024-05-06T12:16:56.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:55 smithi092 ceph-mon[91933]: fsid aee1d912-0ba0-11ef-bc97-c7b262605968 2024-05-06T12:16:56.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:55 smithi092 ceph-mon[91933]: last_changed 2024-05-06T12:16:54.738705+0000 2024-05-06T12:16:56.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:55 smithi092 ceph-mon[91933]: created 2024-05-06T12:04:48.255830+0000 2024-05-06T12:16:56.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:55 smithi092 ceph-mon[91933]: min_mon_release 19 (squid) 2024-05-06T12:16:56.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:55 smithi092 ceph-mon[91933]: election_strategy: 1 2024-05-06T12:16:56.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:55 smithi092 ceph-mon[91933]: 0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.smithi092 2024-05-06T12:16:56.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:55 smithi092 ceph-mon[91933]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.smithi190 2024-05-06T12:16:56.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:55 smithi092 ceph-mon[91933]: fsmap cephfs:2 {0=cephfs.smithi092.rosbll=up:active,1=cephfs.smithi190.byjvgf=up:active} 2 up:standby-replay 2024-05-06T12:16:56.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:55 smithi092 ceph-mon[91933]: osdmap e50: 6 total, 6 up, 6 in 2024-05-06T12:16:56.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:55 smithi092 ceph-mon[91933]: mgrmap e38: smithi092.gyzeqb(active, since 15s), standbys: smithi190.kczxkh 2024-05-06T12:16:56.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:55 smithi092 ceph-mon[91933]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-06T12:16:56.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:55 smithi092 ceph-mon[91933]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-06T12:16:56.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:55 smithi092 ceph-mon[91933]: fs cephfs has deprecated feature inline_data enabled. 2024-05-06T12:16:56.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:55 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:56.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:55 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:56.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:55 smithi092 ceph-mon[91933]: pgmap v11: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 13 op/s 2024-05-06T12:16:56.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:55 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:16:58.101 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:57 smithi190 ceph-mon[79744]: pgmap v12: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 14 op/s 2024-05-06T12:16:58.102 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:57 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:58.102 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:57 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:58.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:57 smithi092 ceph-mon[91933]: pgmap v12: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 14 op/s 2024-05-06T12:16:58.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:57 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:16:58.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:57 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:00.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:59 smithi190 ceph-mon[79744]: pgmap v13: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:17:00.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:59 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:00.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:16:59 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:00.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:59 smithi092 ceph-mon[91933]: pgmap v13: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:17:00.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:59 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:00.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:16:59 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:01.945 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:01 smithi190 ceph-mon[79744]: pgmap v14: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:17:02.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:01 smithi092 ceph-mon[91933]: pgmap v14: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:17:03.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:03 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:03.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:03 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:03.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:03 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:03.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:03 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:17:03.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:03 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:03.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:03 smithi190 ceph-mon[79744]: Reconfiguring mon.smithi092 (monmap changed)... 2024-05-06T12:17:03.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:03 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-06T12:17:03.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:03 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-06T12:17:03.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:03 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:03.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:03 smithi190 ceph-mon[79744]: Reconfiguring daemon mon.smithi092 on smithi092 2024-05-06T12:17:03.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:03 smithi190 ceph-mon[79744]: pgmap v15: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:17:03.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:03 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:03.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:03 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:03.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:03 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:03.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:03 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:17:03.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:03 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:03.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:03 smithi092 ceph-mon[91933]: Reconfiguring mon.smithi092 (monmap changed)... 2024-05-06T12:17:03.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:03 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-06T12:17:03.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:03 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-06T12:17:03.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:03 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:03.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:03 smithi092 ceph-mon[91933]: Reconfiguring daemon mon.smithi092 on smithi092 2024-05-06T12:17:03.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:03 smithi092 ceph-mon[91933]: pgmap v15: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:17:05.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:04 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:05.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:04 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:05.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:04 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi092.gyzeqb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-06T12:17:05.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:04 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-06T12:17:05.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:04 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:05.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:04 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:05.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:04 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:05.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:04 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi092.gyzeqb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-06T12:17:05.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:04 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-06T12:17:05.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:04 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:06.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:05 smithi190 ceph-mon[79744]: Reconfiguring mgr.smithi092.gyzeqb (monmap changed)... 2024-05-06T12:17:06.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:05 smithi190 ceph-mon[79744]: Reconfiguring daemon mgr.smithi092.gyzeqb on smithi092 2024-05-06T12:17:06.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:05 smithi190 ceph-mon[79744]: pgmap v16: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:17:06.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:05 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:06.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:05 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:06.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:05 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi092", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-06T12:17:06.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:05 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:06.366 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:05 smithi092 ceph-mon[91933]: Reconfiguring mgr.smithi092.gyzeqb (monmap changed)... 2024-05-06T12:17:06.366 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:05 smithi092 ceph-mon[91933]: Reconfiguring daemon mgr.smithi092.gyzeqb on smithi092 2024-05-06T12:17:06.366 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:05 smithi092 ceph-mon[91933]: pgmap v16: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:17:06.366 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:05 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:06.366 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:05 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:06.366 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:05 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi092", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-06T12:17:06.366 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:05 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:07.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:07 smithi190 ceph-mon[79744]: Reconfiguring crash.smithi092 (monmap changed)... 2024-05-06T12:17:07.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:07 smithi190 ceph-mon[79744]: Reconfiguring daemon crash.smithi092 on smithi092 2024-05-06T12:17:07.364 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:07 smithi092 ceph-mon[91933]: Reconfiguring crash.smithi092 (monmap changed)... 2024-05-06T12:17:07.364 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:07 smithi092 ceph-mon[91933]: Reconfiguring daemon crash.smithi092 on smithi092 2024-05-06T12:17:08.273 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:08 smithi190 ceph-mon[79744]: pgmap v17: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:17:08.273 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:08 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:08.273 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:08 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:08.273 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:08 smithi190 ceph-mon[79744]: Reconfiguring osd.0 (monmap changed)... 2024-05-06T12:17:08.273 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:08 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-06T12:17:08.273 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:08 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:08.273 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:08 smithi190 ceph-mon[79744]: Reconfiguring daemon osd.0 on smithi092 2024-05-06T12:17:08.301 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:08 smithi092 ceph-mon[91933]: pgmap v17: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:17:08.302 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:08 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:08.302 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:08 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:08.302 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:08 smithi092 ceph-mon[91933]: Reconfiguring osd.0 (monmap changed)... 2024-05-06T12:17:08.302 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:08 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-06T12:17:08.302 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:08 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:08.302 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:08 smithi092 ceph-mon[91933]: Reconfiguring daemon osd.0 on smithi092 2024-05-06T12:17:09.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:09 smithi092 ceph-mon[91933]: pgmap v18: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:17:09.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:17:09.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:09.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:09 smithi190 ceph-mon[79744]: pgmap v18: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:17:09.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:17:09.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:10.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:10 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:10.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:10 smithi092 ceph-mon[91933]: Reconfiguring osd.1 (monmap changed)... 2024-05-06T12:17:10.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:10 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-06T12:17:10.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:10 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:10.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:10 smithi092 ceph-mon[91933]: Reconfiguring daemon osd.1 on smithi092 2024-05-06T12:17:10.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:10 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:10.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:10 smithi190 ceph-mon[79744]: Reconfiguring osd.1 (monmap changed)... 2024-05-06T12:17:10.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:10 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-06T12:17:10.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:10 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:10.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:10 smithi190 ceph-mon[79744]: Reconfiguring daemon osd.1 on smithi092 2024-05-06T12:17:11.944 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:11 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:11.944 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:11 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:11.945 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:11 smithi092 ceph-mon[91933]: Reconfiguring osd.2 (monmap changed)... 2024-05-06T12:17:11.945 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:11 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-06T12:17:11.945 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:11 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:11.945 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:11 smithi092 ceph-mon[91933]: Reconfiguring daemon osd.2 on smithi092 2024-05-06T12:17:11.945 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:11 smithi092 ceph-mon[91933]: pgmap v19: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:17:12.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:11 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:12.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:11 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:12.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:11 smithi190 ceph-mon[79744]: Reconfiguring osd.2 (monmap changed)... 2024-05-06T12:17:12.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:11 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-06T12:17:12.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:11 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:12.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:11 smithi190 ceph-mon[79744]: Reconfiguring daemon osd.2 on smithi092 2024-05-06T12:17:12.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:11 smithi190 ceph-mon[79744]: pgmap v19: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:17:12.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.122+0000 7f3e0ae77700 1 -- 172.21.15.92:0/3967794689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e040fcf60 msgr2=0x7f3e040fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:12.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.122+0000 7f3e0ae77700 1 --2- 172.21.15.92:0/3967794689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e040fcf60 0x7f3e040fd3d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3df4004940 tx=0x7f3df4009e30 comp rx=0 tx=0).stop 2024-05-06T12:17:12.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.122+0000 7f3e0ae77700 1 -- 172.21.15.92:0/3967794689 shutdown_connections 2024-05-06T12:17:12.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.122+0000 7f3e0ae77700 1 --2- 172.21.15.92:0/3967794689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e040fcf60 0x7f3e040fd3d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:12.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.122+0000 7f3e0ae77700 1 --2- 172.21.15.92:0/3967794689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e040fbcb0 0x7f3e040fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:12.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.122+0000 7f3e0ae77700 1 -- 172.21.15.92:0/3967794689 >> 172.21.15.92:0/3967794689 conn(0x7f3e040f7430 msgr2=0x7f3e040f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:17:12.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.122+0000 7f3e0ae77700 1 -- 172.21.15.92:0/3967794689 shutdown_connections 2024-05-06T12:17:12.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.122+0000 7f3e0ae77700 1 -- 172.21.15.92:0/3967794689 wait complete. 2024-05-06T12:17:12.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.122+0000 7f3e0ae77700 1 Processor -- start 2024-05-06T12:17:12.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.122+0000 7f3e0ae77700 1 -- start start 2024-05-06T12:17:12.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.123+0000 7f3e0ae77700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e040fbcb0 0x7f3e04110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:12.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.123+0000 7f3e0ae77700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e040fcf60 0x7f3e04112c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:12.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.123+0000 7f3e0ae77700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e04066330 con 0x7f3e040fcf60 2024-05-06T12:17:12.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.123+0000 7f3e0ae77700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e04066470 con 0x7f3e040fbcb0 2024-05-06T12:17:12.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.123+0000 7f3e03fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e040fcf60 0x7f3e04112c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:12.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.123+0000 7f3e03fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e040fcf60 0x7f3e04112c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36468/0 (socket says 172.21.15.92:36468) 2024-05-06T12:17:12.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.123+0000 7f3e03fff700 1 -- 172.21.15.92:0/4179299945 learned_addr learned my addr 172.21.15.92:0/4179299945 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:17:12.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.123+0000 7f3e08c13700 1 --2- 172.21.15.92:0/4179299945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e040fbcb0 0x7f3e04110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:12.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.123+0000 7f3e03fff700 1 -- 172.21.15.92:0/4179299945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e040fbcb0 msgr2=0x7f3e04110720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:12.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.123+0000 7f3e03fff700 1 --2- 172.21.15.92:0/4179299945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e040fbcb0 0x7f3e04110720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:12.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.123+0000 7f3e03fff700 1 -- 172.21.15.92:0/4179299945 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df4005040 con 0x7f3e040fcf60 2024-05-06T12:17:12.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.123+0000 7f3e03fff700 1 --2- 172.21.15.92:0/4179299945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e040fcf60 0x7f3e04112c70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3df400b3b0 tx=0x7f3df4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:17:12.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.124+0000 7f3e017fa700 1 -- 172.21.15.92:0/4179299945 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3df400e040 con 0x7f3e040fcf60 2024-05-06T12:17:12.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.124+0000 7f3e0ae77700 1 -- 172.21.15.92:0/4179299945 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e041131b0 con 0x7f3e040fcf60 2024-05-06T12:17:12.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.124+0000 7f3e017fa700 1 -- 172.21.15.92:0/4179299945 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3df400bcc0 con 0x7f3e040fcf60 2024-05-06T12:17:12.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.124+0000 7f3e017fa700 1 -- 172.21.15.92:0/4179299945 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3df401c630 con 0x7f3e040fcf60 2024-05-06T12:17:12.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.124+0000 7f3e0ae77700 1 -- 172.21.15.92:0/4179299945 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e04113770 con 0x7f3e040fcf60 2024-05-06T12:17:12.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.125+0000 7f3e0ae77700 1 -- 172.21.15.92:0/4179299945 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e0405f5c0 con 0x7f3e040fcf60 2024-05-06T12:17:12.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.125+0000 7f3e017fa700 1 -- 172.21.15.92:0/4179299945 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f3df4026440 con 0x7f3e040fcf60 2024-05-06T12:17:12.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.125+0000 7f3e017fa700 1 --2- 172.21.15.92:0/4179299945 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f3df006e260 0x7f3df0070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:12.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.126+0000 7f3e017fa700 1 -- 172.21.15.92:0/4179299945 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f3df408eab0 con 0x7f3e040fcf60 2024-05-06T12:17:12.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.126+0000 7f3e08c13700 1 --2- 172.21.15.92:0/4179299945 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f3df006e260 0x7f3df0070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:12.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.126+0000 7f3e08c13700 1 --2- 172.21.15.92:0/4179299945 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f3df006e260 0x7f3df0070710 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3e04065fb0 tx=0x7f3dfc00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:17:12.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.130+0000 7f3e017fa700 1 -- 172.21.15.92:0/4179299945 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3df4057c80 con 0x7f3e040fcf60 2024-05-06T12:17:12.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.357+0000 7f3e0ae77700 1 -- 172.21.15.92:0/4179299945 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e041012c0 con 0x7f3df006e260 2024-05-06T12:17:12.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.358+0000 7f3e017fa700 1 -- 172.21.15.92:0/4179299945 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f3e041012c0 con 0x7f3df006e260 2024-05-06T12:17:12.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.360+0000 7f3e0ae77700 1 -- 172.21.15.92:0/4179299945 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f3df006e260 msgr2=0x7f3df0070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:12.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.360+0000 7f3e0ae77700 1 --2- 172.21.15.92:0/4179299945 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f3df006e260 0x7f3df0070710 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3e04065fb0 tx=0x7f3dfc00b040 comp rx=0 tx=0).stop 2024-05-06T12:17:12.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.360+0000 7f3e0ae77700 1 -- 172.21.15.92:0/4179299945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e040fcf60 msgr2=0x7f3e04112c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:12.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.360+0000 7f3e0ae77700 1 --2- 172.21.15.92:0/4179299945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e040fcf60 0x7f3e04112c70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3df400b3b0 tx=0x7f3df4009e30 comp rx=0 tx=0).stop 2024-05-06T12:17:12.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.360+0000 7f3e0ae77700 1 -- 172.21.15.92:0/4179299945 shutdown_connections 2024-05-06T12:17:12.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.360+0000 7f3e0ae77700 1 --2- 172.21.15.92:0/4179299945 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f3df006e260 0x7f3df0070710 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:12.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.360+0000 7f3e0ae77700 1 --2- 172.21.15.92:0/4179299945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e040fbcb0 0x7f3e04110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:12.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.360+0000 7f3e0ae77700 1 --2- 172.21.15.92:0/4179299945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e040fcf60 0x7f3e04112c70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:12.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.360+0000 7f3e0ae77700 1 -- 172.21.15.92:0/4179299945 >> 172.21.15.92:0/4179299945 conn(0x7f3e040f7430 msgr2=0x7f3e040f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:17:12.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.360+0000 7f3e0ae77700 1 -- 172.21.15.92:0/4179299945 shutdown_connections 2024-05-06T12:17:12.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.360+0000 7f3e0ae77700 1 -- 172.21.15.92:0/4179299945 wait complete. 2024-05-06T12:17:12.374 INFO:teuthology.orchestra.run.smithi092.stdout:true 2024-05-06T12:17:12.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.466+0000 7efd811aa700 1 -- 172.21.15.92:0/3334887770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd7c0fce80 msgr2=0x7efd7c0fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:12.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.466+0000 7efd811aa700 1 --2- 172.21.15.92:0/3334887770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd7c0fce80 0x7efd7c0fd290 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efd6c004940 tx=0x7efd6c009e30 comp rx=0 tx=0).stop 2024-05-06T12:17:12.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.467+0000 7efd811aa700 1 -- 172.21.15.92:0/3334887770 shutdown_connections 2024-05-06T12:17:12.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.467+0000 7efd811aa700 1 --2- 172.21.15.92:0/3334887770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd7c0fe080 0x7efd7c0fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:12.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.467+0000 7efd811aa700 1 --2- 172.21.15.92:0/3334887770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd7c0fce80 0x7efd7c0fd290 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:12.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.467+0000 7efd811aa700 1 -- 172.21.15.92:0/3334887770 >> 172.21.15.92:0/3334887770 conn(0x7efd7c0f8590 msgr2=0x7efd7c0fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:17:12.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.467+0000 7efd811aa700 1 -- 172.21.15.92:0/3334887770 shutdown_connections 2024-05-06T12:17:12.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.467+0000 7efd811aa700 1 -- 172.21.15.92:0/3334887770 wait complete. 2024-05-06T12:17:12.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.467+0000 7efd811aa700 1 Processor -- start 2024-05-06T12:17:12.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.467+0000 7efd811aa700 1 -- start start 2024-05-06T12:17:12.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.467+0000 7efd811aa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd7c0fce80 0x7efd7c10da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:12.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.467+0000 7efd811aa700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd7c0fe080 0x7efd7c10df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:12.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.467+0000 7efd811aa700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c10e580 con 0x7efd7c0fce80 2024-05-06T12:17:12.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.467+0000 7efd811aa700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c10edd0 con 0x7efd7c0fe080 2024-05-06T12:17:12.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.468+0000 7efd7ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd7c0fce80 0x7efd7c10da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:12.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.468+0000 7efd7ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd7c0fce80 0x7efd7c10da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36486/0 (socket says 172.21.15.92:36486) 2024-05-06T12:17:12.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.468+0000 7efd7ad9d700 1 -- 172.21.15.92:0/4107501804 learned_addr learned my addr 172.21.15.92:0/4107501804 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:17:12.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.468+0000 7efd7259c700 1 --2- 172.21.15.92:0/4107501804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd7c0fe080 0x7efd7c10df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:12.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.468+0000 7efd7ad9d700 1 -- 172.21.15.92:0/4107501804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd7c0fe080 msgr2=0x7efd7c10df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:12.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.468+0000 7efd7ad9d700 1 --2- 172.21.15.92:0/4107501804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd7c0fe080 0x7efd7c10df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:12.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.468+0000 7efd7ad9d700 1 -- 172.21.15.92:0/4107501804 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd6c005040 con 0x7efd7c0fce80 2024-05-06T12:17:12.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.468+0000 7efd7ad9d700 1 --2- 172.21.15.92:0/4107501804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd7c0fce80 0x7efd7c10da40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efd6c00b3b0 tx=0x7efd6c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:17:12.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.468+0000 7efd73fff700 1 -- 172.21.15.92:0/4107501804 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd6c00e040 con 0x7efd7c0fce80 2024-05-06T12:17:12.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.468+0000 7efd73fff700 1 -- 172.21.15.92:0/4107501804 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd6c00bcc0 con 0x7efd7c0fce80 2024-05-06T12:17:12.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.468+0000 7efd73fff700 1 -- 172.21.15.92:0/4107501804 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd6c01c630 con 0x7efd7c0fce80 2024-05-06T12:17:12.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.468+0000 7efd811aa700 1 -- 172.21.15.92:0/4107501804 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd7c10f050 con 0x7efd7c0fce80 2024-05-06T12:17:12.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.468+0000 7efd811aa700 1 -- 172.21.15.92:0/4107501804 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd7c10f590 con 0x7efd7c0fce80 2024-05-06T12:17:12.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.469+0000 7efd811aa700 1 -- 172.21.15.92:0/4107501804 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd7c05f5c0 con 0x7efd7c0fce80 2024-05-06T12:17:12.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.470+0000 7efd73fff700 1 -- 172.21.15.92:0/4107501804 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7efd6c0139f0 con 0x7efd7c0fce80 2024-05-06T12:17:12.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.470+0000 7efd73fff700 1 --2- 172.21.15.92:0/4107501804 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7efd6806e260 0x7efd68070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:12.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.471+0000 7efd73fff700 1 -- 172.21.15.92:0/4107501804 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6410+0+0 (secure 0 0 0) 0x7efd6c08ea70 con 0x7efd7c0fce80 2024-05-06T12:17:12.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.471+0000 7efd7259c700 1 --2- 172.21.15.92:0/4107501804 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7efd6806e260 0x7efd68070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:12.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.471+0000 7efd7259c700 1 --2- 172.21.15.92:0/4107501804 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7efd6806e260 0x7efd68070710 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efd64007fd0 tx=0x7efd64009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:17:12.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.475+0000 7efd73fff700 1 -- 172.21.15.92:0/4107501804 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7efd6c05b780 con 0x7efd7c0fce80 2024-05-06T12:17:12.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.707+0000 7efd811aa700 1 -- 172.21.15.92:0/4107501804 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efd7c10f8c0 con 0x7efd6806e260 2024-05-06T12:17:12.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.708+0000 7efd73fff700 1 -- 172.21.15.92:0/4107501804 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7efd7c10f8c0 con 0x7efd6806e260 2024-05-06T12:17:12.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.710+0000 7efd811aa700 1 -- 172.21.15.92:0/4107501804 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7efd6806e260 msgr2=0x7efd68070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:12.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.710+0000 7efd811aa700 1 --2- 172.21.15.92:0/4107501804 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7efd6806e260 0x7efd68070710 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efd64007fd0 tx=0x7efd64009040 comp rx=0 tx=0).stop 2024-05-06T12:17:12.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.710+0000 7efd811aa700 1 -- 172.21.15.92:0/4107501804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd7c0fce80 msgr2=0x7efd7c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:12.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.710+0000 7efd811aa700 1 --2- 172.21.15.92:0/4107501804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd7c0fce80 0x7efd7c10da40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efd6c00b3b0 tx=0x7efd6c009e30 comp rx=0 tx=0).stop 2024-05-06T12:17:12.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.710+0000 7efd811aa700 1 -- 172.21.15.92:0/4107501804 shutdown_connections 2024-05-06T12:17:12.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.710+0000 7efd811aa700 1 --2- 172.21.15.92:0/4107501804 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7efd6806e260 0x7efd68070710 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:12.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.710+0000 7efd811aa700 1 --2- 172.21.15.92:0/4107501804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd7c0fce80 0x7efd7c10da40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:12.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.710+0000 7efd811aa700 1 --2- 172.21.15.92:0/4107501804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd7c0fe080 0x7efd7c10df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:12.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.710+0000 7efd811aa700 1 -- 172.21.15.92:0/4107501804 >> 172.21.15.92:0/4107501804 conn(0x7efd7c0f8590 msgr2=0x7efd7c0f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:17:12.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.710+0000 7efd811aa700 1 -- 172.21.15.92:0/4107501804 shutdown_connections 2024-05-06T12:17:12.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.710+0000 7efd811aa700 1 -- 172.21.15.92:0/4107501804 wait complete. 2024-05-06T12:17:12.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.816+0000 7f8bc3d85700 1 -- 172.21.15.92:0/80366854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bbc0fce60 msgr2=0x7f8bbc0fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:12.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.816+0000 7f8bc3d85700 1 --2- 172.21.15.92:0/80366854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bbc0fce60 0x7f8bbc0fd270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8004940 tx=0x7f8bb8009e30 comp rx=0 tx=0).stop 2024-05-06T12:17:12.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.816+0000 7f8bc3d85700 1 -- 172.21.15.92:0/80366854 shutdown_connections 2024-05-06T12:17:12.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.816+0000 7f8bc3d85700 1 --2- 172.21.15.92:0/80366854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bbc0fdfd0 0x7f8bbc0fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:12.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.816+0000 7f8bc3d85700 1 --2- 172.21.15.92:0/80366854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bbc0fce60 0x7f8bbc0fd270 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:12.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.816+0000 7f8bc3d85700 1 -- 172.21.15.92:0/80366854 >> 172.21.15.92:0/80366854 conn(0x7f8bbc0f8540 msgr2=0x7f8bbc0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:17:12.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.816+0000 7f8bc3d85700 1 -- 172.21.15.92:0/80366854 shutdown_connections 2024-05-06T12:17:12.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.816+0000 7f8bc3d85700 1 -- 172.21.15.92:0/80366854 wait complete. 2024-05-06T12:17:12.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.817+0000 7f8bc3d85700 1 Processor -- start 2024-05-06T12:17:12.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.817+0000 7f8bc3d85700 1 -- start start 2024-05-06T12:17:12.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.817+0000 7f8bc3d85700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bbc0fce60 0x7f8bbc071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:12.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.817+0000 7f8bc3d85700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bbc0fdfd0 0x7f8bbc0702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:12.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.817+0000 7f8bc3d85700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc070860 con 0x7f8bbc0fce60 2024-05-06T12:17:12.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.817+0000 7f8bc3d85700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc0709d0 con 0x7f8bbc0fdfd0 2024-05-06T12:17:12.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.817+0000 7f8bc1b21700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bbc0fce60 0x7f8bbc071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:12.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.817+0000 7f8bc1b21700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bbc0fce60 0x7f8bbc071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54848/0 (socket says 172.21.15.92:54848) 2024-05-06T12:17:12.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.817+0000 7f8bc1b21700 1 -- 172.21.15.92:0/213160422 learned_addr learned my addr 172.21.15.92:0/213160422 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:17:12.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.817+0000 7f8bc1320700 1 --2- 172.21.15.92:0/213160422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bbc0fdfd0 0x7f8bbc0702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:12.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.817+0000 7f8bc1b21700 1 -- 172.21.15.92:0/213160422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bbc0fdfd0 msgr2=0x7f8bbc0702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:12.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.817+0000 7f8bc1b21700 1 --2- 172.21.15.92:0/213160422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bbc0fdfd0 0x7f8bbc0702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:12.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.817+0000 7f8bc1b21700 1 -- 172.21.15.92:0/213160422 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb8005040 con 0x7f8bbc0fce60 2024-05-06T12:17:12.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.817+0000 7f8bc1b21700 1 --2- 172.21.15.92:0/213160422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bbc0fce60 0x7f8bbc071de0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8bb800b3b0 tx=0x7f8bb8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:17:12.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.818+0000 7f8bb27fc700 1 -- 172.21.15.92:0/213160422 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bb800e040 con 0x7f8bbc0fce60 2024-05-06T12:17:12.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.818+0000 7f8bb27fc700 1 -- 172.21.15.92:0/213160422 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8bb800bcc0 con 0x7f8bbc0fce60 2024-05-06T12:17:12.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.818+0000 7f8bc3d85700 1 -- 172.21.15.92:0/213160422 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bbc070c50 con 0x7f8bbc0fce60 2024-05-06T12:17:12.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.818+0000 7f8bb27fc700 1 -- 172.21.15.92:0/213160422 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bb801c630 con 0x7f8bbc0fce60 2024-05-06T12:17:12.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.818+0000 7f8bc3d85700 1 -- 172.21.15.92:0/213160422 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bbc115a20 con 0x7f8bbc0fce60 2024-05-06T12:17:12.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.819+0000 7f8bc3d85700 1 -- 172.21.15.92:0/213160422 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bbc05f5c0 con 0x7f8bbc0fce60 2024-05-06T12:17:12.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.819+0000 7f8bb27fc700 1 -- 172.21.15.92:0/213160422 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f8bb8013980 con 0x7f8bbc0fce60 2024-05-06T12:17:12.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.820+0000 7f8bb27fc700 1 --2- 172.21.15.92:0/213160422 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f8ba806e260 0x7f8ba8070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:12.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.820+0000 7f8bb27fc700 1 -- 172.21.15.92:0/213160422 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f8bb808eac0 con 0x7f8bbc0fce60 2024-05-06T12:17:12.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.820+0000 7f8bc1320700 1 --2- 172.21.15.92:0/213160422 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f8ba806e260 0x7f8ba8070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:12.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.820+0000 7f8bc1320700 1 --2- 172.21.15.92:0/213160422 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f8ba806e260 0x7f8ba8070710 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8bac007fd0 tx=0x7f8bac009f60 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:17:12.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:12.824+0000 7f8bb27fc700 1 -- 172.21.15.92:0/213160422 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f8bb8057c20 con 0x7f8bbc0fce60 2024-05-06T12:17:13.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.044+0000 7f8bc3d85700 1 -- 172.21.15.92:0/213160422 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8bbc102340 con 0x7f8ba806e260 2024-05-06T12:17:13.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.054+0000 7f8bb27fc700 1 -- 172.21.15.92:0/213160422 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f8bbc102340 con 0x7f8ba806e260 2024-05-06T12:17:13.057 INFO:teuthology.orchestra.run.smithi092.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-06T12:17:13.057 INFO:teuthology.orchestra.run.smithi092.stdout:alertmanager.smithi092 smithi092 *:9093,9094 running (2m) 28s ago 10m 17.0M - 0.25.0 c8568f914cd2 be5637d7d25a 2024-05-06T12:17:13.057 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi092 smithi092 running (10m) 28s ago 10m 6665k - 17.2.7-925-ge81df420 96d5ae7fd4b4 95dc7e14bd9a 2024-05-06T12:17:13.057 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi190 smithi190 running (9m) 15s ago 9m 6665k - 17.2.7-925-ge81df420 96d5ae7fd4b4 6cf97785406f 2024-05-06T12:17:13.057 INFO:teuthology.orchestra.run.smithi092.stdout:grafana.smithi092 smithi092 *:3000 running (2m) 28s ago 10m 77.0M - 9.4.7 954c08fa6188 9180cb3e5258 2024-05-06T12:17:13.057 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.ijsqgp smithi092 running (5m) 28s ago 5m 60.4M - 17.2.7-925-ge81df420 96d5ae7fd4b4 b4c2ed6d2c82 2024-05-06T12:17:13.057 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.rosbll smithi092 running (5m) 28s ago 5m 122M - 17.2.7-925-ge81df420 96d5ae7fd4b4 5f5426cd1bf5 2024-05-06T12:17:13.057 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.byjvgf smithi190 running (5m) 15s ago 5m 126M - 17.2.7-925-ge81df420 96d5ae7fd4b4 24581d2e1c5d 2024-05-06T12:17:13.057 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.idzoff smithi190 running (5m) 15s ago 5m 64.3M - 17.2.7-925-ge81df420 96d5ae7fd4b4 20f2641696ef 2024-05-06T12:17:13.057 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi092.gyzeqb smithi092 *:8443,9283,8765 running (101s) 28s ago 12m 473M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 de9d2198327a 2024-05-06T12:17:13.058 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi190.kczxkh smithi190 *:8443,9283,8765 running (56s) 15s ago 9m 448M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 e84ab3758ac7 2024-05-06T12:17:13.058 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi092 smithi092 running (42s) 28s ago 12m 28.2M 2048M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 ac07f7158087 2024-05-06T12:17:13.058 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi190 smithi190 running (19s) 15s ago 9m 23.5M 2048M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 7a3c08eb716f 2024-05-06T12:17:13.058 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi092 smithi092 *:9100 running (2m) 28s ago 10m 7956k - 1.5.0 0da6a335fe13 c796446b206f 2024-05-06T12:17:13.058 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi190 smithi190 *:9100 running (2m) 15s ago 9m 14.9M - 1.5.0 0da6a335fe13 647e28111a76 2024-05-06T12:17:13.058 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 smithi092 running (9m) 28s ago 9m 199M 1557M 17.2.7-925-ge81df420 96d5ae7fd4b4 ce84e7af9084 2024-05-06T12:17:13.058 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 smithi092 running (8m) 28s ago 8m 199M 1557M 17.2.7-925-ge81df420 96d5ae7fd4b4 8ab870f57eb2 2024-05-06T12:17:13.058 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 smithi092 running (8m) 28s ago 8m 165M 1557M 17.2.7-925-ge81df420 96d5ae7fd4b4 1e6ed04a0688 2024-05-06T12:17:13.058 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 smithi190 running (7m) 15s ago 7m 221M 2581M 17.2.7-925-ge81df420 96d5ae7fd4b4 52f9521286d5 2024-05-06T12:17:13.058 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 smithi190 running (7m) 15s ago 7m 203M 2581M 17.2.7-925-ge81df420 96d5ae7fd4b4 3a7da09e4c48 2024-05-06T12:17:13.058 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 smithi190 running (6m) 15s ago 6m 187M 2581M 17.2.7-925-ge81df420 96d5ae7fd4b4 e7452c6b5cec 2024-05-06T12:17:13.058 INFO:teuthology.orchestra.run.smithi092.stdout:prometheus.smithi092 smithi092 *:9095 running (65s) 28s ago 10m 47.1M - 2.43.0 a07b618ecd1d bc5445691527 2024-05-06T12:17:13.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.056+0000 7f8bc3d85700 1 -- 172.21.15.92:0/213160422 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f8ba806e260 msgr2=0x7f8ba8070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:13.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.056+0000 7f8bc3d85700 1 --2- 172.21.15.92:0/213160422 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f8ba806e260 0x7f8ba8070710 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8bac007fd0 tx=0x7f8bac009f60 comp rx=0 tx=0).stop 2024-05-06T12:17:13.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.056+0000 7f8bc3d85700 1 -- 172.21.15.92:0/213160422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bbc0fce60 msgr2=0x7f8bbc071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:13.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.056+0000 7f8bc3d85700 1 --2- 172.21.15.92:0/213160422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bbc0fce60 0x7f8bbc071de0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8bb800b3b0 tx=0x7f8bb8009e30 comp rx=0 tx=0).stop 2024-05-06T12:17:13.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.056+0000 7f8bc3d85700 1 -- 172.21.15.92:0/213160422 shutdown_connections 2024-05-06T12:17:13.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.056+0000 7f8bc3d85700 1 --2- 172.21.15.92:0/213160422 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f8ba806e260 0x7f8ba8070710 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:13.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.056+0000 7f8bc3d85700 1 --2- 172.21.15.92:0/213160422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bbc0fce60 0x7f8bbc071de0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:13.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.056+0000 7f8bc3d85700 1 --2- 172.21.15.92:0/213160422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bbc0fdfd0 0x7f8bbc0702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:13.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.056+0000 7f8bc3d85700 1 -- 172.21.15.92:0/213160422 >> 172.21.15.92:0/213160422 conn(0x7f8bbc0f8540 msgr2=0x7f8bbc0f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:17:13.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.056+0000 7f8bc3d85700 1 -- 172.21.15.92:0/213160422 shutdown_connections 2024-05-06T12:17:13.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.056+0000 7f8bc3d85700 1 -- 172.21.15.92:0/213160422 wait complete. 2024-05-06T12:17:13.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.156+0000 7f57d1c97700 1 -- 172.21.15.92:0/1689839361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57cc0fcff0 msgr2=0x7f57cc0fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:13.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.156+0000 7f57d1c97700 1 --2- 172.21.15.92:0/1689839361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57cc0fcff0 0x7f57cc0fd460 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f57b4004940 tx=0x7f57b4009e30 comp rx=0 tx=0).stop 2024-05-06T12:17:13.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.157+0000 7f57d1c97700 1 -- 172.21.15.92:0/1689839361 shutdown_connections 2024-05-06T12:17:13.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.157+0000 7f57d1c97700 1 --2- 172.21.15.92:0/1689839361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57cc0fcff0 0x7f57cc0fd460 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:13.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.157+0000 7f57d1c97700 1 --2- 172.21.15.92:0/1689839361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57cc0fbd40 0x7f57cc0fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:13.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.157+0000 7f57d1c97700 1 -- 172.21.15.92:0/1689839361 >> 172.21.15.92:0/1689839361 conn(0x7f57cc0f74c0 msgr2=0x7f57cc0f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:17:13.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.157+0000 7f57d1c97700 1 -- 172.21.15.92:0/1689839361 shutdown_connections 2024-05-06T12:17:13.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.157+0000 7f57d1c97700 1 -- 172.21.15.92:0/1689839361 wait complete. 2024-05-06T12:17:13.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.157+0000 7f57d1c97700 1 Processor -- start 2024-05-06T12:17:13.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.157+0000 7f57d1c97700 1 -- start start 2024-05-06T12:17:13.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.157+0000 7f57d1c97700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57cc0fbd40 0x7f57cc110740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:13.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.157+0000 7f57d1c97700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57cc0fcff0 0x7f57cc112c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:13.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.157+0000 7f57d1c97700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc066c50 con 0x7f57cc0fcff0 2024-05-06T12:17:13.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.157+0000 7f57d1c97700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc066d90 con 0x7f57cc0fbd40 2024-05-06T12:17:13.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.157+0000 7f57c3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57cc0fcff0 0x7f57cc112c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:13.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.157+0000 7f57c3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57cc0fcff0 0x7f57cc112c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54856/0 (socket says 172.21.15.92:54856) 2024-05-06T12:17:13.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.157+0000 7f57c3fff700 1 -- 172.21.15.92:0/2824034254 learned_addr learned my addr 172.21.15.92:0/2824034254 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:17:13.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.158+0000 7f57cb7fe700 1 --2- 172.21.15.92:0/2824034254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57cc0fbd40 0x7f57cc110740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:13.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.158+0000 7f57c3fff700 1 -- 172.21.15.92:0/2824034254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57cc0fbd40 msgr2=0x7f57cc110740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:13.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.158+0000 7f57c3fff700 1 --2- 172.21.15.92:0/2824034254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57cc0fbd40 0x7f57cc110740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:13.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.158+0000 7f57c3fff700 1 -- 172.21.15.92:0/2824034254 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b4005040 con 0x7f57cc0fcff0 2024-05-06T12:17:13.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.158+0000 7f57c3fff700 1 --2- 172.21.15.92:0/2824034254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57cc0fcff0 0x7f57cc112c90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f57b400b3b0 tx=0x7f57b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:17:13.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.158+0000 7f57c8ff9700 1 -- 172.21.15.92:0/2824034254 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57b400e040 con 0x7f57cc0fcff0 2024-05-06T12:17:13.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.158+0000 7f57d1c97700 1 -- 172.21.15.92:0/2824034254 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57cc113290 con 0x7f57cc0fcff0 2024-05-06T12:17:13.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.158+0000 7f57c8ff9700 1 -- 172.21.15.92:0/2824034254 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f57b400bcc0 con 0x7f57cc0fcff0 2024-05-06T12:17:13.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.158+0000 7f57c8ff9700 1 -- 172.21.15.92:0/2824034254 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57b401c630 con 0x7f57cc0fcff0 2024-05-06T12:17:13.164 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.158+0000 7f57d1c97700 1 -- 172.21.15.92:0/2824034254 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57cc113820 con 0x7f57cc0fcff0 2024-05-06T12:17:13.164 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.159+0000 7f57d1c97700 1 -- 172.21.15.92:0/2824034254 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57cc05f5c0 con 0x7f57cc0fcff0 2024-05-06T12:17:13.164 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.159+0000 7f57c8ff9700 1 -- 172.21.15.92:0/2824034254 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f57b4013980 con 0x7f57cc0fcff0 2024-05-06T12:17:13.164 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.160+0000 7f57c8ff9700 1 --2- 172.21.15.92:0/2824034254 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f57b806e260 0x7f57b8070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:13.164 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.160+0000 7f57c8ff9700 1 -- 172.21.15.92:0/2824034254 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f57b408eae0 con 0x7f57cc0fcff0 2024-05-06T12:17:13.164 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.160+0000 7f57cb7fe700 1 --2- 172.21.15.92:0/2824034254 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f57b806e260 0x7f57b8070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:13.164 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.160+0000 7f57cb7fe700 1 --2- 172.21.15.92:0/2824034254 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f57b806e260 0x7f57b8070710 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f57bc006fd0 tx=0x7f57bc008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:17:13.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.164+0000 7f57c8ff9700 1 -- 172.21.15.92:0/2824034254 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f57b4057c40 con 0x7f57cc0fcff0 2024-05-06T12:17:13.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.465+0000 7f57d1c97700 1 -- 172.21.15.92:0/2824034254 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f57cc04bed0 con 0x7f57cc0fcff0 2024-05-06T12:17:13.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.465+0000 7f57c8ff9700 1 -- 172.21.15.92:0/2824034254 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7f57b405b300 con 0x7f57cc0fcff0 2024-05-06T12:17:13.469 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:17:13.469 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": { 2024-05-06T12:17:13.469 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:17:13.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:17:13.469 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": { 2024-05-06T12:17:13.469 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:17:13.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:17:13.469 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": { 2024-05-06T12:17:13.469 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 6 2024-05-06T12:17:13.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:17:13.470 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-05-06T12:17:13.470 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 4 2024-05-06T12:17:13.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:17:13.470 INFO:teuthology.orchestra.run.smithi092.stdout: "overall": { 2024-05-06T12:17:13.470 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 10, 2024-05-06T12:17:13.470 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 4 2024-05-06T12:17:13.470 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-06T12:17:13.470 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:17:13.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.468+0000 7f57d1c97700 1 -- 172.21.15.92:0/2824034254 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f57b806e260 msgr2=0x7f57b8070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:13.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.468+0000 7f57d1c97700 1 --2- 172.21.15.92:0/2824034254 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f57b806e260 0x7f57b8070710 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f57bc006fd0 tx=0x7f57bc008040 comp rx=0 tx=0).stop 2024-05-06T12:17:13.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.468+0000 7f57d1c97700 1 -- 172.21.15.92:0/2824034254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57cc0fcff0 msgr2=0x7f57cc112c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:13.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.468+0000 7f57d1c97700 1 --2- 172.21.15.92:0/2824034254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57cc0fcff0 0x7f57cc112c90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f57b400b3b0 tx=0x7f57b4009e30 comp rx=0 tx=0).stop 2024-05-06T12:17:13.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.468+0000 7f57d1c97700 1 -- 172.21.15.92:0/2824034254 shutdown_connections 2024-05-06T12:17:13.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.468+0000 7f57d1c97700 1 --2- 172.21.15.92:0/2824034254 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f57b806e260 0x7f57b8070710 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:13.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.468+0000 7f57d1c97700 1 --2- 172.21.15.92:0/2824034254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57cc0fbd40 0x7f57cc110740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:13.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.468+0000 7f57d1c97700 1 --2- 172.21.15.92:0/2824034254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57cc0fcff0 0x7f57cc112c90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:13.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.468+0000 7f57d1c97700 1 -- 172.21.15.92:0/2824034254 >> 172.21.15.92:0/2824034254 conn(0x7f57cc0f74c0 msgr2=0x7f57cc0f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:17:13.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.468+0000 7f57d1c97700 1 -- 172.21.15.92:0/2824034254 shutdown_connections 2024-05-06T12:17:13.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.468+0000 7f57d1c97700 1 -- 172.21.15.92:0/2824034254 wait complete. 2024-05-06T12:17:13.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.569+0000 7f9530024700 1 -- 172.21.15.92:0/3963118764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95280fed30 msgr2=0x7f95280fae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:13.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.569+0000 7f9530024700 1 --2- 172.21.15.92:0/3963118764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95280fed30 0x7f95280fae70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9524004940 tx=0x7f9524009e30 comp rx=0 tx=0).stop 2024-05-06T12:17:13.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.569+0000 7f9530024700 1 -- 172.21.15.92:0/3963118764 shutdown_connections 2024-05-06T12:17:13.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.569+0000 7f9530024700 1 --2- 172.21.15.92:0/3963118764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95280fed30 0x7f95280fae70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:13.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.569+0000 7f9530024700 1 --2- 172.21.15.92:0/3963118764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95280fe3e0 0x7f95280fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:13.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.569+0000 7f9530024700 1 -- 172.21.15.92:0/3963118764 >> 172.21.15.92:0/3963118764 conn(0x7f95280f6b80 msgr2=0x7f95280f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:17:13.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.569+0000 7f9530024700 1 -- 172.21.15.92:0/3963118764 shutdown_connections 2024-05-06T12:17:13.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.569+0000 7f9530024700 1 -- 172.21.15.92:0/3963118764 wait complete. 2024-05-06T12:17:13.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.570+0000 7f9530024700 1 Processor -- start 2024-05-06T12:17:13.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.570+0000 7f9530024700 1 -- start start 2024-05-06T12:17:13.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.570+0000 7f9530024700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95280fe3e0 0x7f952810fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:13.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.570+0000 7f9530024700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95280fed30 0x7f9528110190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:13.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.570+0000 7f9530024700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9528110820 con 0x7f95280fed30 2024-05-06T12:17:13.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.570+0000 7f9530024700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9528111010 con 0x7f95280fe3e0 2024-05-06T12:17:13.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.570+0000 7f952d5bf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95280fed30 0x7f9528110190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:13.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.570+0000 7f952d5bf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95280fed30 0x7f9528110190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54860/0 (socket says 172.21.15.92:54860) 2024-05-06T12:17:13.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.570+0000 7f952d5bf700 1 -- 172.21.15.92:0/2176149618 learned_addr learned my addr 172.21.15.92:0/2176149618 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:17:13.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.570+0000 7f952ddc0700 1 --2- 172.21.15.92:0/2176149618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95280fe3e0 0x7f952810fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:13.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.570+0000 7f952d5bf700 1 -- 172.21.15.92:0/2176149618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95280fe3e0 msgr2=0x7f952810fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:13.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.570+0000 7f952d5bf700 1 --2- 172.21.15.92:0/2176149618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95280fe3e0 0x7f952810fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:13.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.570+0000 7f952d5bf700 1 -- 172.21.15.92:0/2176149618 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9524005040 con 0x7f95280fed30 2024-05-06T12:17:13.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.570+0000 7f952ddc0700 1 --2- 172.21.15.92:0/2176149618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95280fe3e0 0x7f952810fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-06T12:17:13.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.571+0000 7f952d5bf700 1 --2- 172.21.15.92:0/2176149618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95280fed30 0x7f9528110190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f952400b3b0 tx=0x7f9524009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:17:13.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.571+0000 7f951e7fc700 1 -- 172.21.15.92:0/2176149618 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f952400e040 con 0x7f95280fed30 2024-05-06T12:17:13.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.571+0000 7f9530024700 1 -- 172.21.15.92:0/2176149618 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9528111290 con 0x7f95280fed30 2024-05-06T12:17:13.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.571+0000 7f951e7fc700 1 -- 172.21.15.92:0/2176149618 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f952400bcc0 con 0x7f95280fed30 2024-05-06T12:17:13.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.571+0000 7f951e7fc700 1 -- 172.21.15.92:0/2176149618 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f952401c630 con 0x7f95280fed30 2024-05-06T12:17:13.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.571+0000 7f9530024700 1 -- 172.21.15.92:0/2176149618 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9528111850 con 0x7f95280fed30 2024-05-06T12:17:13.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.572+0000 7f9530024700 1 -- 172.21.15.92:0/2176149618 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9528101b20 con 0x7f95280fed30 2024-05-06T12:17:13.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.573+0000 7f951e7fc700 1 -- 172.21.15.92:0/2176149618 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f9524013980 con 0x7f95280fed30 2024-05-06T12:17:13.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.573+0000 7f951e7fc700 1 --2- 172.21.15.92:0/2176149618 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f951406e260 0x7f9514070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:13.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.573+0000 7f951e7fc700 1 -- 172.21.15.92:0/2176149618 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f952408eac0 con 0x7f95280fed30 2024-05-06T12:17:13.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.573+0000 7f952ddc0700 1 --2- 172.21.15.92:0/2176149618 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f951406e260 0x7f9514070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:13.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.573+0000 7f952ddc0700 1 --2- 172.21.15.92:0/2176149618 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f951406e260 0x7f9514070710 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9518006fd0 tx=0x7f9518008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:17:13.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.578+0000 7f951e7fc700 1 -- 172.21.15.92:0/2176149618 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f952405b750 con 0x7f95280fed30 2024-05-06T12:17:13.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:13 smithi092 ceph-mon[91933]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:17:13.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:13 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:13.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:13 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:13.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:13 smithi092 ceph-mon[91933]: Reconfiguring mds.cephfs.smithi092.ijsqgp (monmap changed)... 2024-05-06T12:17:13.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:13 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi092.ijsqgp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-06T12:17:13.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:13 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:13.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:13 smithi092 ceph-mon[91933]: Reconfiguring daemon mds.cephfs.smithi092.ijsqgp on smithi092 2024-05-06T12:17:13.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:13 smithi092 ceph-mon[91933]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:17:13.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:13 smithi092 ceph-mon[91933]: pgmap v20: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:17:13.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.832+0000 7f9530024700 1 -- 172.21.15.92:0/2176149618 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f952805f5c0 con 0x7f95280fed30 2024-05-06T12:17:13.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.832+0000 7f951e7fc700 1 -- 172.21.15.92:0/2176149618 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1937 (secure 0 0 0) 0x7f952405b2e0 con 0x7f95280fed30 2024-05-06T12:17:13.836 INFO:teuthology.orchestra.run.smithi092.stdout:e20 2024-05-06T12:17:13.836 INFO:teuthology.orchestra.run.smithi092.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-06T12:17:13.836 INFO:teuthology.orchestra.run.smithi092.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:17:13.836 INFO:teuthology.orchestra.run.smithi092.stdout:legacy client fscid: 1 2024-05-06T12:17:13.836 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:17:13.836 INFO:teuthology.orchestra.run.smithi092.stdout:Filesystem 'cephfs' (1) 2024-05-06T12:17:13.836 INFO:teuthology.orchestra.run.smithi092.stdout:fs_name cephfs 2024-05-06T12:17:13.836 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 20 2024-05-06T12:17:13.836 INFO:teuthology.orchestra.run.smithi092.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-06T12:17:13.836 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-05-06T12:11:44.952671+0000 2024-05-06T12:17:13.837 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2024-05-06T12:15:07.568167+0000 2024-05-06T12:17:13.837 INFO:teuthology.orchestra.run.smithi092.stdout:tableserver 0 2024-05-06T12:17:13.837 INFO:teuthology.orchestra.run.smithi092.stdout:root 0 2024-05-06T12:17:13.837 INFO:teuthology.orchestra.run.smithi092.stdout:session_timeout 60 2024-05-06T12:17:13.837 INFO:teuthology.orchestra.run.smithi092.stdout:session_autoclose 300 2024-05-06T12:17:13.837 INFO:teuthology.orchestra.run.smithi092.stdout:max_file_size 1099511627776 2024-05-06T12:17:13.837 INFO:teuthology.orchestra.run.smithi092.stdout:max_xattr_size 65536 2024-05-06T12:17:13.837 INFO:teuthology.orchestra.run.smithi092.stdout:required_client_features {} 2024-05-06T12:17:13.837 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure 0 2024-05-06T12:17:13.837 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure_osd_epoch 46 2024-05-06T12:17:13.837 INFO:teuthology.orchestra.run.smithi092.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:17:13.837 INFO:teuthology.orchestra.run.smithi092.stdout:max_mds 2 2024-05-06T12:17:13.837 INFO:teuthology.orchestra.run.smithi092.stdout:in 0,1 2024-05-06T12:17:13.837 INFO:teuthology.orchestra.run.smithi092.stdout:up {0=14472,1=24267} 2024-05-06T12:17:13.838 INFO:teuthology.orchestra.run.smithi092.stdout:failed 2024-05-06T12:17:13.838 INFO:teuthology.orchestra.run.smithi092.stdout:damaged 2024-05-06T12:17:13.838 INFO:teuthology.orchestra.run.smithi092.stdout:stopped 2024-05-06T12:17:13.838 INFO:teuthology.orchestra.run.smithi092.stdout:data_pools [3] 2024-05-06T12:17:13.838 INFO:teuthology.orchestra.run.smithi092.stdout:metadata_pool 2 2024-05-06T12:17:13.838 INFO:teuthology.orchestra.run.smithi092.stdout:inline_data enabled 2024-05-06T12:17:13.838 INFO:teuthology.orchestra.run.smithi092.stdout:balancer 2024-05-06T12:17:13.838 INFO:teuthology.orchestra.run.smithi092.stdout:bal_rank_mask -1 2024-05-06T12:17:13.838 INFO:teuthology.orchestra.run.smithi092.stdout:standby_count_wanted 1 2024-05-06T12:17:13.838 INFO:teuthology.orchestra.run.smithi092.stdout:qdb_cluster leader: 0 members: 2024-05-06T12:17:13.838 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.rosbll{0:14472} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.92:6828/1663253598,v1:172.21.15.92:6829/1663253598] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:17:13.838 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.ijsqgp{0:14484} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.92:6826/3613441054,v1:172.21.15.92:6827/3613441054] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:17:13.839 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.byjvgf{1:24267} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.190:6824/1565151699,v1:172.21.15.190:6825/1565151699] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:17:13.839 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.idzoff{1:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.190:6826/2453728380,v1:172.21.15.190:6827/2453728380] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:17:13.839 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:17:13.839 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:17:13.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.834+0000 7f9530024700 1 -- 172.21.15.92:0/2176149618 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f951406e260 msgr2=0x7f9514070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:13.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.834+0000 7f9530024700 1 --2- 172.21.15.92:0/2176149618 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f951406e260 0x7f9514070710 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9518006fd0 tx=0x7f9518008040 comp rx=0 tx=0).stop 2024-05-06T12:17:13.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.834+0000 7f9530024700 1 -- 172.21.15.92:0/2176149618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95280fed30 msgr2=0x7f9528110190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:13.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.834+0000 7f9530024700 1 --2- 172.21.15.92:0/2176149618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95280fed30 0x7f9528110190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f952400b3b0 tx=0x7f9524009e30 comp rx=0 tx=0).stop 2024-05-06T12:17:13.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.835+0000 7f9530024700 1 -- 172.21.15.92:0/2176149618 shutdown_connections 2024-05-06T12:17:13.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.835+0000 7f9530024700 1 --2- 172.21.15.92:0/2176149618 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f951406e260 0x7f9514070710 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:13.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.835+0000 7f9530024700 1 --2- 172.21.15.92:0/2176149618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95280fe3e0 0x7f952810fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:13.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.835+0000 7f9530024700 1 --2- 172.21.15.92:0/2176149618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95280fed30 0x7f9528110190 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:13.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.835+0000 7f9530024700 1 -- 172.21.15.92:0/2176149618 >> 172.21.15.92:0/2176149618 conn(0x7f95280f6b80 msgr2=0x7f95280f8130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:17:13.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.835+0000 7f9530024700 1 -- 172.21.15.92:0/2176149618 shutdown_connections 2024-05-06T12:17:13.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.835+0000 7f9530024700 1 -- 172.21.15.92:0/2176149618 wait complete. 2024-05-06T12:17:13.840 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 20 2024-05-06T12:17:13.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:13 smithi190 ceph-mon[79744]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:17:13.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:13 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:13.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:13 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:13.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:13 smithi190 ceph-mon[79744]: Reconfiguring mds.cephfs.smithi092.ijsqgp (monmap changed)... 2024-05-06T12:17:13.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:13 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi092.ijsqgp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-06T12:17:13.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:13 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:13.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:13 smithi190 ceph-mon[79744]: Reconfiguring daemon mds.cephfs.smithi092.ijsqgp on smithi092 2024-05-06T12:17:13.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:13 smithi190 ceph-mon[79744]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:17:13.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:13 smithi190 ceph-mon[79744]: pgmap v20: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:17:13.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.938+0000 7f7bc3af0700 1 -- 172.21.15.92:0/1754911282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bbc066e50 msgr2=0x7f7bbc0ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:13.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.938+0000 7f7bc3af0700 1 --2- 172.21.15.92:0/1754911282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bbc066e50 0x7f7bbc0ffa60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8004940 tx=0x7f7bb8009e30 comp rx=0 tx=0).stop 2024-05-06T12:17:13.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.938+0000 7f7bc3af0700 1 -- 172.21.15.92:0/1754911282 shutdown_connections 2024-05-06T12:17:13.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.938+0000 7f7bc3af0700 1 --2- 172.21.15.92:0/1754911282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bbc0fffa0 0x7f7bbc102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:13.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.938+0000 7f7bc3af0700 1 --2- 172.21.15.92:0/1754911282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bbc066e50 0x7f7bbc0ffa60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:13.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.938+0000 7f7bc3af0700 1 -- 172.21.15.92:0/1754911282 >> 172.21.15.92:0/1754911282 conn(0x7f7bbc0f7430 msgr2=0x7f7bbc0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:17:13.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.939+0000 7f7bc3af0700 1 -- 172.21.15.92:0/1754911282 shutdown_connections 2024-05-06T12:17:13.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.939+0000 7f7bc3af0700 1 -- 172.21.15.92:0/1754911282 wait complete. 2024-05-06T12:17:13.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.939+0000 7f7bc3af0700 1 Processor -- start 2024-05-06T12:17:13.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.939+0000 7f7bc3af0700 1 -- start start 2024-05-06T12:17:13.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.939+0000 7f7bc3af0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bbc066e50 0x7f7bbc10d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:13.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.939+0000 7f7bc3af0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bbc0fffa0 0x7f7bbc10df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:13.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.939+0000 7f7bc3af0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc10e590 con 0x7f7bbc0fffa0 2024-05-06T12:17:13.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.939+0000 7f7bc3af0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc112f30 con 0x7f7bbc066e50 2024-05-06T12:17:13.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.939+0000 7f7bc108b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bbc0fffa0 0x7f7bbc10df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:13.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.939+0000 7f7bc108b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bbc0fffa0 0x7f7bbc10df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54880/0 (socket says 172.21.15.92:54880) 2024-05-06T12:17:13.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.939+0000 7f7bc108b700 1 -- 172.21.15.92:0/173590348 learned_addr learned my addr 172.21.15.92:0/173590348 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:17:13.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.940+0000 7f7bc188c700 1 --2- 172.21.15.92:0/173590348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bbc066e50 0x7f7bbc10d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:13.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.940+0000 7f7bc108b700 1 -- 172.21.15.92:0/173590348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bbc066e50 msgr2=0x7f7bbc10d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:13.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.940+0000 7f7bc108b700 1 --2- 172.21.15.92:0/173590348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bbc066e50 0x7f7bbc10d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:13.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.940+0000 7f7bc108b700 1 -- 172.21.15.92:0/173590348 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb8005040 con 0x7f7bbc0fffa0 2024-05-06T12:17:13.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.940+0000 7f7bc108b700 1 --2- 172.21.15.92:0/173590348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bbc0fffa0 0x7f7bbc10df00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7bac00b790 tx=0x7f7bac009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:17:13.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.940+0000 7f7bb27fc700 1 -- 172.21.15.92:0/173590348 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bac00f6f0 con 0x7f7bbc0fffa0 2024-05-06T12:17:13.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.940+0000 7f7bb27fc700 1 -- 172.21.15.92:0/173590348 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7bac00fd20 con 0x7f7bbc0fffa0 2024-05-06T12:17:13.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.940+0000 7f7bb27fc700 1 -- 172.21.15.92:0/173590348 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bac0103a0 con 0x7f7bbc0fffa0 2024-05-06T12:17:13.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.940+0000 7f7bc3af0700 1 -- 172.21.15.92:0/173590348 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bbc113190 con 0x7f7bbc0fffa0 2024-05-06T12:17:13.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.940+0000 7f7bc3af0700 1 -- 172.21.15.92:0/173590348 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bbc113700 con 0x7f7bbc0fffa0 2024-05-06T12:17:13.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.941+0000 7f7bc3af0700 1 -- 172.21.15.92:0/173590348 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bbc04c710 con 0x7f7bbc0fffa0 2024-05-06T12:17:13.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.942+0000 7f7bb27fc700 1 -- 172.21.15.92:0/173590348 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f7bac011400 con 0x7f7bbc0fffa0 2024-05-06T12:17:13.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.942+0000 7f7bb27fc700 1 --2- 172.21.15.92:0/173590348 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7ba806e270 0x7f7ba8070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:13.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.942+0000 7f7bc188c700 1 --2- 172.21.15.92:0/173590348 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7ba806e270 0x7f7ba8070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:13.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.942+0000 7f7bb27fc700 1 -- 172.21.15.92:0/173590348 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f7bac08da70 con 0x7f7bbc0fffa0 2024-05-06T12:17:13.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.943+0000 7f7bc188c700 1 --2- 172.21.15.92:0/173590348 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7ba806e270 0x7f7ba8070720 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800b970 tx=0x7f7bb8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:17:13.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:13.947+0000 7f7bb27fc700 1 -- 172.21.15.92:0/173590348 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7bac05a890 con 0x7f7bbc0fffa0 2024-05-06T12:17:14.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.168+0000 7f7bc3af0700 1 -- 172.21.15.92:0/173590348 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7bbc101c10 con 0x7f7ba806e270 2024-05-06T12:17:14.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.169+0000 7f7bb27fc700 1 -- 172.21.15.92:0/173590348 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f7bbc101c10 con 0x7f7ba806e270 2024-05-06T12:17:14.173 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:17:14.173 INFO:teuthology.orchestra.run.smithi092.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb", 2024-05-06T12:17:14.173 INFO:teuthology.orchestra.run.smithi092.stdout: "in_progress": true, 2024-05-06T12:17:14.173 INFO:teuthology.orchestra.run.smithi092.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-06T12:17:14.173 INFO:teuthology.orchestra.run.smithi092.stdout: "services_complete": [ 2024-05-06T12:17:14.173 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr", 2024-05-06T12:17:14.173 INFO:teuthology.orchestra.run.smithi092.stdout: "mon" 2024-05-06T12:17:14.173 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-05-06T12:17:14.173 INFO:teuthology.orchestra.run.smithi092.stdout: "progress": "4/21 daemons upgraded", 2024-05-06T12:17:14.173 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "Currently upgrading mon daemons", 2024-05-06T12:17:14.173 INFO:teuthology.orchestra.run.smithi092.stdout: "is_paused": false 2024-05-06T12:17:14.173 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:17:14.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.172+0000 7f7bc3af0700 1 -- 172.21.15.92:0/173590348 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7ba806e270 msgr2=0x7f7ba8070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:14.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.172+0000 7f7bc3af0700 1 --2- 172.21.15.92:0/173590348 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7ba806e270 0x7f7ba8070720 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800b970 tx=0x7f7bb8009e30 comp rx=0 tx=0).stop 2024-05-06T12:17:14.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.172+0000 7f7bc3af0700 1 -- 172.21.15.92:0/173590348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bbc0fffa0 msgr2=0x7f7bbc10df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:14.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.172+0000 7f7bc3af0700 1 --2- 172.21.15.92:0/173590348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bbc0fffa0 0x7f7bbc10df00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7bac00b790 tx=0x7f7bac009220 comp rx=0 tx=0).stop 2024-05-06T12:17:14.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.172+0000 7f7bc3af0700 1 -- 172.21.15.92:0/173590348 shutdown_connections 2024-05-06T12:17:14.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.172+0000 7f7bc3af0700 1 --2- 172.21.15.92:0/173590348 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7ba806e270 0x7f7ba8070720 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:14.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.172+0000 7f7bc3af0700 1 --2- 172.21.15.92:0/173590348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bbc066e50 0x7f7bbc10d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:14.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.172+0000 7f7bc3af0700 1 --2- 172.21.15.92:0/173590348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bbc0fffa0 0x7f7bbc10df00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:14.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.172+0000 7f7bc3af0700 1 -- 172.21.15.92:0/173590348 >> 172.21.15.92:0/173590348 conn(0x7f7bbc0f7430 msgr2=0x7f7bbc0f9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:17:14.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.172+0000 7f7bc3af0700 1 -- 172.21.15.92:0/173590348 shutdown_connections 2024-05-06T12:17:14.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.172+0000 7f7bc3af0700 1 -- 172.21.15.92:0/173590348 wait complete. 2024-05-06T12:17:14.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.280+0000 7f6be4680700 1 -- 172.21.15.92:0/2757997350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bdc0fff40 msgr2=0x7f6bdc102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:14.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.280+0000 7f6be4680700 1 --2- 172.21.15.92:0/2757997350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bdc0fff40 0x7f6bdc102320 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc004940 tx=0x7f6bcc009e30 comp rx=0 tx=0).stop 2024-05-06T12:17:14.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.280+0000 7f6be4680700 1 -- 172.21.15.92:0/2757997350 shutdown_connections 2024-05-06T12:17:14.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.280+0000 7f6be4680700 1 --2- 172.21.15.92:0/2757997350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bdc0fff40 0x7f6bdc102320 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:14.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.280+0000 7f6be4680700 1 --2- 172.21.15.92:0/2757997350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bdc066260 0x7f6bdc0ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:14.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.280+0000 7f6be4680700 1 -- 172.21.15.92:0/2757997350 >> 172.21.15.92:0/2757997350 conn(0x7f6bdc0f7430 msgr2=0x7f6bdc0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:17:14.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.281+0000 7f6be4680700 1 -- 172.21.15.92:0/2757997350 shutdown_connections 2024-05-06T12:17:14.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.281+0000 7f6be4680700 1 -- 172.21.15.92:0/2757997350 wait complete. 2024-05-06T12:17:14.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.281+0000 7f6be4680700 1 Processor -- start 2024-05-06T12:17:14.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.281+0000 7f6be4680700 1 -- start start 2024-05-06T12:17:14.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.281+0000 7f6be4680700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bdc066260 0x7f6bdc10da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:14.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.281+0000 7f6be4680700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bdc0fff40 0x7f6bdc10dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:14.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.281+0000 7f6be4680700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc10e660 con 0x7f6bdc066260 2024-05-06T12:17:14.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.281+0000 7f6be4680700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc10ee00 con 0x7f6bdc0fff40 2024-05-06T12:17:14.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.281+0000 7f6be241c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bdc066260 0x7f6bdc10da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:14.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.281+0000 7f6be241c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bdc066260 0x7f6bdc10da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54892/0 (socket says 172.21.15.92:54892) 2024-05-06T12:17:14.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.281+0000 7f6be241c700 1 -- 172.21.15.92:0/413809664 learned_addr learned my addr 172.21.15.92:0/413809664 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:17:14.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.282+0000 7f6be1c1b700 1 --2- 172.21.15.92:0/413809664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bdc0fff40 0x7f6bdc10dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:14.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.282+0000 7f6be241c700 1 -- 172.21.15.92:0/413809664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bdc0fff40 msgr2=0x7f6bdc10dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:14.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.282+0000 7f6be241c700 1 --2- 172.21.15.92:0/413809664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bdc0fff40 0x7f6bdc10dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:14.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.282+0000 7f6be241c700 1 -- 172.21.15.92:0/413809664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bcc005040 con 0x7f6bdc066260 2024-05-06T12:17:14.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.282+0000 7f6be1c1b700 1 --2- 172.21.15.92:0/413809664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bdc0fff40 0x7f6bdc10dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-06T12:17:14.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.282+0000 7f6be241c700 1 --2- 172.21.15.92:0/413809664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bdc066260 0x7f6bdc10da90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6bd800b790 tx=0x7f6bd8009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:17:14.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.282+0000 7f6bd2ffd700 1 -- 172.21.15.92:0/413809664 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bd800f6f0 con 0x7f6bdc066260 2024-05-06T12:17:14.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.282+0000 7f6bd2ffd700 1 -- 172.21.15.92:0/413809664 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6bd800fd20 con 0x7f6bdc066260 2024-05-06T12:17:14.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.282+0000 7f6be4680700 1 -- 172.21.15.92:0/413809664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bdc10f0e0 con 0x7f6bdc066260 2024-05-06T12:17:14.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.282+0000 7f6bd2ffd700 1 -- 172.21.15.92:0/413809664 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bd80103a0 con 0x7f6bdc066260 2024-05-06T12:17:14.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.282+0000 7f6be4680700 1 -- 172.21.15.92:0/413809664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bdc10f6a0 con 0x7f6bdc066260 2024-05-06T12:17:14.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.283+0000 7f6bd2ffd700 1 -- 172.21.15.92:0/413809664 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f6bd8011400 con 0x7f6bdc066260 2024-05-06T12:17:14.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.283+0000 7f6be4680700 1 -- 172.21.15.92:0/413809664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bdc05f5c0 con 0x7f6bdc066260 2024-05-06T12:17:14.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.284+0000 7f6bd2ffd700 1 --2- 172.21.15.92:0/413809664 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f6bc806e260 0x7f6bc8070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:14.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.284+0000 7f6bd2ffd700 1 -- 172.21.15.92:0/413809664 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f6bd808db30 con 0x7f6bdc066260 2024-05-06T12:17:14.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.284+0000 7f6be1c1b700 1 --2- 172.21.15.92:0/413809664 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f6bc806e260 0x7f6bc8070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:14.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.284+0000 7f6be1c1b700 1 --2- 172.21.15.92:0/413809664 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f6bc806e260 0x7f6bc8070710 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc00b970 tx=0x7f6bcc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:17:14.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.289+0000 7f6bd2ffd700 1 -- 172.21.15.92:0/413809664 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6bd805a8d0 con 0x7f6bdc066260 2024-05-06T12:17:14.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.601+0000 7f6be4680700 1 -- 172.21.15.92:0/413809664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6bdc105ac0 con 0x7f6bdc066260 2024-05-06T12:17:14.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.602+0000 7f6bd2ffd700 1 -- 172.21.15.92:0/413809664 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f6bdc105ac0 con 0x7f6bdc066260 2024-05-06T12:17:14.605 INFO:teuthology.orchestra.run.smithi092.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-06T12:17:14.605 INFO:teuthology.orchestra.run.smithi092.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-06T12:17:14.605 INFO:teuthology.orchestra.run.smithi092.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-06T12:17:14.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.604+0000 7f6be4680700 1 -- 172.21.15.92:0/413809664 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f6bc806e260 msgr2=0x7f6bc8070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:14.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.604+0000 7f6be4680700 1 --2- 172.21.15.92:0/413809664 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f6bc806e260 0x7f6bc8070710 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc00b970 tx=0x7f6bcc009e30 comp rx=0 tx=0).stop 2024-05-06T12:17:14.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.604+0000 7f6be4680700 1 -- 172.21.15.92:0/413809664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bdc066260 msgr2=0x7f6bdc10da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:14.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.604+0000 7f6be4680700 1 --2- 172.21.15.92:0/413809664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bdc066260 0x7f6bdc10da90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6bd800b790 tx=0x7f6bd8009220 comp rx=0 tx=0).stop 2024-05-06T12:17:14.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.604+0000 7f6be4680700 1 -- 172.21.15.92:0/413809664 shutdown_connections 2024-05-06T12:17:14.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.604+0000 7f6be4680700 1 --2- 172.21.15.92:0/413809664 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f6bc806e260 0x7f6bc8070710 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:14.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.604+0000 7f6be4680700 1 --2- 172.21.15.92:0/413809664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bdc066260 0x7f6bdc10da90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:14.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.604+0000 7f6be4680700 1 --2- 172.21.15.92:0/413809664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bdc0fff40 0x7f6bdc10dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:14.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.604+0000 7f6be4680700 1 -- 172.21.15.92:0/413809664 >> 172.21.15.92:0/413809664 conn(0x7f6bdc0f7430 msgr2=0x7f6bdc0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:17:14.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.604+0000 7f6be4680700 1 -- 172.21.15.92:0/413809664 shutdown_connections 2024-05-06T12:17:14.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:14.604+0000 7f6be4680700 1 -- 172.21.15.92:0/413809664 wait complete. 2024-05-06T12:17:14.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:14 smithi092 ceph-mon[91933]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:17:14.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:14 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/2824034254' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:17:14.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:14 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/2176149618' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:17:14.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:14 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:14.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:14 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:14.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:14 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi092.rosbll", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-06T12:17:14.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:14 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:14.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:14 smithi190 ceph-mon[79744]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:17:14.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:14 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/2824034254' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:17:14.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:14 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/2176149618' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:17:14.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:14 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:14.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:14 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:14.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:14 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi092.rosbll", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-06T12:17:14.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:14 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:15.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:15 smithi092 ceph-mon[91933]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:17:15.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:15 smithi092 ceph-mon[91933]: Reconfiguring mds.cephfs.smithi092.rosbll (monmap changed)... 2024-05-06T12:17:15.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:15 smithi092 ceph-mon[91933]: Reconfiguring daemon mds.cephfs.smithi092.rosbll on smithi092 2024-05-06T12:17:15.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:15 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/413809664' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:17:15.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:15 smithi092 ceph-mon[91933]: pgmap v21: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:17:15.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:15 smithi190 ceph-mon[79744]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:17:15.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:15 smithi190 ceph-mon[79744]: Reconfiguring mds.cephfs.smithi092.rosbll (monmap changed)... 2024-05-06T12:17:15.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:15 smithi190 ceph-mon[79744]: Reconfiguring daemon mds.cephfs.smithi092.rosbll on smithi092 2024-05-06T12:17:15.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:15 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/413809664' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:17:15.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:15 smithi190 ceph-mon[79744]: pgmap v21: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:17:17.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:16 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:17.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:16 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:17.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:16 smithi092 ceph-mon[91933]: Reconfiguring crash.smithi190 (monmap changed)... 2024-05-06T12:17:17.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:16 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi190", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-06T12:17:17.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:16 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:17.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:16 smithi092 ceph-mon[91933]: Reconfiguring daemon crash.smithi190 on smithi190 2024-05-06T12:17:17.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:16 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:17.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:16 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:17.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:16 smithi190 ceph-mon[79744]: Reconfiguring crash.smithi190 (monmap changed)... 2024-05-06T12:17:17.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:16 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi190", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-06T12:17:17.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:16 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:17.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:16 smithi190 ceph-mon[79744]: Reconfiguring daemon crash.smithi190 on smithi190 2024-05-06T12:17:18.219 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:17 smithi092 ceph-mon[91933]: pgmap v22: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-06T12:17:18.219 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:17 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:18.219 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:17 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:18.219 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:17 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi190.kczxkh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-06T12:17:18.219 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:17 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-06T12:17:18.220 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:17 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:18.292 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:17 smithi190 ceph-mon[79744]: pgmap v22: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-06T12:17:18.292 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:17 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:18.292 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:17 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:18.292 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:17 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi190.kczxkh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-06T12:17:18.292 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:17 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-06T12:17:18.292 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:17 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:19.142 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:18 smithi190 ceph-mon[79744]: Reconfiguring mgr.smithi190.kczxkh (monmap changed)... 2024-05-06T12:17:19.142 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:18 smithi190 ceph-mon[79744]: Reconfiguring daemon mgr.smithi190.kczxkh on smithi190 2024-05-06T12:17:19.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:18 smithi092 ceph-mon[91933]: Reconfiguring mgr.smithi190.kczxkh (monmap changed)... 2024-05-06T12:17:19.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:18 smithi092 ceph-mon[91933]: Reconfiguring daemon mgr.smithi190.kczxkh on smithi190 2024-05-06T12:17:20.053 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:19 smithi190 ceph-mon[79744]: pgmap v23: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:17:20.054 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:19 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:20.054 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:19 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:20.054 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:19 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-06T12:17:20.054 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:19 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-06T12:17:20.054 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:19 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:20.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:19 smithi092 ceph-mon[91933]: pgmap v23: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:17:20.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:19 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:20.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:19 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:20.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:19 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-06T12:17:20.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:19 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-06T12:17:20.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:19 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:21.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:20 smithi190 ceph-mon[79744]: Reconfiguring mon.smithi190 (monmap changed)... 2024-05-06T12:17:21.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:20 smithi190 ceph-mon[79744]: Reconfiguring daemon mon.smithi190 on smithi190 2024-05-06T12:17:21.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:20 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:21.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:20 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:21.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:20 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-06T12:17:21.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:20 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:21.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:20 smithi092 ceph-mon[91933]: Reconfiguring mon.smithi190 (monmap changed)... 2024-05-06T12:17:21.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:20 smithi092 ceph-mon[91933]: Reconfiguring daemon mon.smithi190 on smithi190 2024-05-06T12:17:21.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:20 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:21.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:20 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:21.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:20 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-06T12:17:21.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:20 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:22.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:21 smithi092 ceph-mon[91933]: pgmap v24: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:17:22.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:21 smithi092 ceph-mon[91933]: Reconfiguring osd.3 (monmap changed)... 2024-05-06T12:17:22.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:21 smithi092 ceph-mon[91933]: Reconfiguring daemon osd.3 on smithi190 2024-05-06T12:17:22.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:21 smithi190 ceph-mon[79744]: pgmap v24: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:17:22.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:21 smithi190 ceph-mon[79744]: Reconfiguring osd.3 (monmap changed)... 2024-05-06T12:17:22.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:21 smithi190 ceph-mon[79744]: Reconfiguring daemon osd.3 on smithi190 2024-05-06T12:17:23.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:23 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:23.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:23 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:23.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:23 smithi190 ceph-mon[79744]: Reconfiguring osd.4 (monmap changed)... 2024-05-06T12:17:23.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:23 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-06T12:17:23.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:23 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:23.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:23 smithi190 ceph-mon[79744]: Reconfiguring daemon osd.4 on smithi190 2024-05-06T12:17:23.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:23 smithi190 ceph-mon[79744]: pgmap v25: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:17:23.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:23 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:23.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:23 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:23.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:23 smithi092 ceph-mon[91933]: Reconfiguring osd.4 (monmap changed)... 2024-05-06T12:17:23.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:23 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-06T12:17:23.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:23 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:23.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:23 smithi092 ceph-mon[91933]: Reconfiguring daemon osd.4 on smithi190 2024-05-06T12:17:23.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:23 smithi092 ceph-mon[91933]: pgmap v25: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:17:24.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:24 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:17:24.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:24 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:24.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:24 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:24.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:24 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-06T12:17:24.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:24 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:24.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:24 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:17:24.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:24 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:24.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:24 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:24.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:24 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-06T12:17:24.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:24 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:25.702 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:25 smithi190 ceph-mon[79744]: Reconfiguring osd.5 (monmap changed)... 2024-05-06T12:17:25.702 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:25 smithi190 ceph-mon[79744]: Reconfiguring daemon osd.5 on smithi190 2024-05-06T12:17:25.702 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:25 smithi190 ceph-mon[79744]: pgmap v26: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:17:25.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:25 smithi092 ceph-mon[91933]: Reconfiguring osd.5 (monmap changed)... 2024-05-06T12:17:25.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:25 smithi092 ceph-mon[91933]: Reconfiguring daemon osd.5 on smithi190 2024-05-06T12:17:25.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:25 smithi092 ceph-mon[91933]: pgmap v26: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:17:27.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:26 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:27.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:26 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:27.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:26 smithi190 ceph-mon[79744]: Reconfiguring mds.cephfs.smithi190.byjvgf (monmap changed)... 2024-05-06T12:17:27.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:26 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi190.byjvgf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-06T12:17:27.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:26 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:27.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:26 smithi190 ceph-mon[79744]: Reconfiguring daemon mds.cephfs.smithi190.byjvgf on smithi190 2024-05-06T12:17:27.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:26 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:27.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:26 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:27.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:26 smithi092 ceph-mon[91933]: Reconfiguring mds.cephfs.smithi190.byjvgf (monmap changed)... 2024-05-06T12:17:27.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:26 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi190.byjvgf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-06T12:17:27.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:26 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:27.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:26 smithi092 ceph-mon[91933]: Reconfiguring daemon mds.cephfs.smithi190.byjvgf on smithi190 2024-05-06T12:17:28.066 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:27 smithi190 ceph-mon[79744]: pgmap v27: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:17:28.066 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:27 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:28.066 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:27 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:28.066 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:27 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi190.idzoff", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-06T12:17:28.066 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:27 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:28.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:27 smithi092 ceph-mon[91933]: pgmap v27: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:17:28.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:27 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:28.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:27 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:28.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:27 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi190.idzoff", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-06T12:17:28.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:27 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:28.940 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:28 smithi190 ceph-mon[79744]: Reconfiguring mds.cephfs.smithi190.idzoff (monmap changed)... 2024-05-06T12:17:28.940 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:28 smithi190 ceph-mon[79744]: Reconfiguring daemon mds.cephfs.smithi190.idzoff on smithi190 2024-05-06T12:17:28.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:28 smithi092 ceph-mon[91933]: Reconfiguring mds.cephfs.smithi190.idzoff (monmap changed)... 2024-05-06T12:17:28.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:28 smithi092 ceph-mon[91933]: Reconfiguring daemon mds.cephfs.smithi190.idzoff on smithi190 2024-05-06T12:17:30.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:29 smithi092 ceph-mon[91933]: pgmap v28: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:17:30.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:29 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:30.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:29 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:30.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:29 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:17:30.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:29 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:17:30.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:29 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:17:30.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:29 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:30.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:29 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi092"}]: dispatch 2024-05-06T12:17:30.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:29 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi092"}]': finished 2024-05-06T12:17:30.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:29 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi190"}]: dispatch 2024-05-06T12:17:30.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:29 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi190"}]': finished 2024-05-06T12:17:30.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:29 smithi190 ceph-mon[79744]: pgmap v28: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:17:30.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:29 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:30.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:29 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:30.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:29 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:17:30.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:29 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:17:30.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:29 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:17:30.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:29 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:30.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:30 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi092"}]: dispatch 2024-05-06T12:17:30.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:30 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi092"}]': finished 2024-05-06T12:17:30.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:30 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi190"}]: dispatch 2024-05-06T12:17:30.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:30 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi190"}]': finished 2024-05-06T12:17:31.304 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:31 smithi092 ceph-mon[91933]: Upgrade: Setting container_image for all mon 2024-05-06T12:17:31.304 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:31 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:31.305 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:31 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi092", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-06T12:17:31.305 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:31 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:31.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:31 smithi190 ceph-mon[79744]: Upgrade: Setting container_image for all mon 2024-05-06T12:17:31.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:31 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:31.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:31 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi092", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-06T12:17:31.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:31 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:32.337 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:32 smithi092 ceph-mon[91933]: Upgrade: Updating crash.smithi092 (1/2) 2024-05-06T12:17:32.338 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:32 smithi092 ceph-mon[91933]: Deploying daemon crash.smithi092 on smithi092 2024-05-06T12:17:32.338 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:32 smithi092 ceph-mon[91933]: pgmap v29: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:17:32.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:32 smithi190 ceph-mon[79744]: Upgrade: Updating crash.smithi092 (1/2) 2024-05-06T12:17:32.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:32 smithi190 ceph-mon[79744]: Deploying daemon crash.smithi092 on smithi092 2024-05-06T12:17:32.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:32 smithi190 ceph-mon[79744]: pgmap v29: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:17:33.101 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:33 smithi092 ceph-mon[91933]: pgmap v30: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:17:33.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:33 smithi190 ceph-mon[79744]: pgmap v30: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:17:35.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:35 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:35.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:35 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:35.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:35 smithi190 ceph-mon[79744]: pgmap v31: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:17:35.634 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:35 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:35.634 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:35 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:35.634 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:35 smithi092 ceph-mon[91933]: pgmap v31: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:17:37.320 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:37 smithi190 ceph-mon[79744]: Upgrade: Updating crash.smithi190 (2/2) 2024-05-06T12:17:37.320 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:37 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:37.320 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:37 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi190", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-06T12:17:37.320 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:37 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:37.320 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:37 smithi190 ceph-mon[79744]: Deploying daemon crash.smithi190 on smithi190 2024-05-06T12:17:37.320 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:37 smithi190 ceph-mon[79744]: pgmap v32: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:17:37.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:37 smithi092 ceph-mon[91933]: Upgrade: Updating crash.smithi190 (2/2) 2024-05-06T12:17:37.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:37 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:37.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:37 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi190", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-06T12:17:37.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:37 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:37.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:37 smithi092 ceph-mon[91933]: Deploying daemon crash.smithi190 on smithi190 2024-05-06T12:17:37.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:37 smithi092 ceph-mon[91933]: pgmap v32: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:17:40.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:39 smithi190 ceph-mon[79744]: pgmap v33: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:17:40.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:39 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:17:40.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:39 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:40.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:39 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:40.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:39 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:17:40.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:39 smithi092 ceph-mon[91933]: pgmap v33: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:17:40.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:39 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:17:40.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:39 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:40.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:39 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:40.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:39 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:17:41.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:41 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:41.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:41 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:41.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:41 smithi190 ceph-mon[79744]: pgmap v34: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:17:41.578 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:41 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:41.578 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:41 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:41.578 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:41 smithi092 ceph-mon[91933]: pgmap v34: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:17:43.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:43 smithi092 ceph-mon[91933]: pgmap v35: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:17:43.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:43 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:43.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:43 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:44.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:43 smithi190 ceph-mon[79744]: pgmap v35: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:17:44.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:43 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:44.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:43 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:44.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.726+0000 7fcf3e68e700 1 -- 172.21.15.92:0/4041227173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf380fbcb0 msgr2=0x7fcf380fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:44.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.726+0000 7fcf3e68e700 1 --2- 172.21.15.92:0/4041227173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf380fbcb0 0x7fcf380fc0c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcf28004940 tx=0x7fcf28009e30 comp rx=0 tx=0).stop 2024-05-06T12:17:44.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.726+0000 7fcf3e68e700 1 -- 172.21.15.92:0/4041227173 shutdown_connections 2024-05-06T12:17:44.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.726+0000 7fcf3e68e700 1 --2- 172.21.15.92:0/4041227173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf380fcf60 0x7fcf380fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:44.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.726+0000 7fcf3e68e700 1 --2- 172.21.15.92:0/4041227173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf380fbcb0 0x7fcf380fc0c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:44.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.726+0000 7fcf3e68e700 1 -- 172.21.15.92:0/4041227173 >> 172.21.15.92:0/4041227173 conn(0x7fcf380f7430 msgr2=0x7fcf380f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:17:44.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.726+0000 7fcf3e68e700 1 -- 172.21.15.92:0/4041227173 shutdown_connections 2024-05-06T12:17:44.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.726+0000 7fcf3e68e700 1 -- 172.21.15.92:0/4041227173 wait complete. 2024-05-06T12:17:44.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.726+0000 7fcf3e68e700 1 Processor -- start 2024-05-06T12:17:44.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.727+0000 7fcf3e68e700 1 -- start start 2024-05-06T12:17:44.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.727+0000 7fcf3e68e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf380fbcb0 0x7fcf380ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:44.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.727+0000 7fcf3e68e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf380fcf60 0x7fcf380fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:44.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.727+0000 7fcf3e68e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf380fe7e0 con 0x7fcf380fbcb0 2024-05-06T12:17:44.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.727+0000 7fcf3e68e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf380fe950 con 0x7fcf380fcf60 2024-05-06T12:17:44.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.727+0000 7fcf37fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf380fbcb0 0x7fcf380ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:44.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.727+0000 7fcf37fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf380fbcb0 0x7fcf380ffbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36028/0 (socket says 172.21.15.92:36028) 2024-05-06T12:17:44.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.727+0000 7fcf37fff700 1 -- 172.21.15.92:0/1827878641 learned_addr learned my addr 172.21.15.92:0/1827878641 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:17:44.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.727+0000 7fcf377fe700 1 --2- 172.21.15.92:0/1827878641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf380fcf60 0x7fcf380fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:44.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.727+0000 7fcf37fff700 1 -- 172.21.15.92:0/1827878641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf380fcf60 msgr2=0x7fcf380fe2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:44.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.727+0000 7fcf37fff700 1 --2- 172.21.15.92:0/1827878641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf380fcf60 0x7fcf380fe2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:44.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.727+0000 7fcf37fff700 1 -- 172.21.15.92:0/1827878641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf28005040 con 0x7fcf380fbcb0 2024-05-06T12:17:44.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.727+0000 7fcf377fe700 1 --2- 172.21.15.92:0/1827878641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf380fcf60 0x7fcf380fe2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-06T12:17:44.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.727+0000 7fcf37fff700 1 --2- 172.21.15.92:0/1827878641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf380fbcb0 0x7fcf380ffbf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcf2800b3b0 tx=0x7fcf28009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:17:44.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.728+0000 7fcf34ff9700 1 -- 172.21.15.92:0/1827878641 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf2800e040 con 0x7fcf380fbcb0 2024-05-06T12:17:44.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.728+0000 7fcf34ff9700 1 -- 172.21.15.92:0/1827878641 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf2800bcc0 con 0x7fcf380fbcb0 2024-05-06T12:17:44.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.728+0000 7fcf3e68e700 1 -- 172.21.15.92:0/1827878641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf380febd0 con 0x7fcf380fbcb0 2024-05-06T12:17:44.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.728+0000 7fcf34ff9700 1 -- 172.21.15.92:0/1827878641 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf2801c630 con 0x7fcf380fbcb0 2024-05-06T12:17:44.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.728+0000 7fcf3e68e700 1 -- 172.21.15.92:0/1827878641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf38115a40 con 0x7fcf380fbcb0 2024-05-06T12:17:44.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.729+0000 7fcf3e68e700 1 -- 172.21.15.92:0/1827878641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf3804c710 con 0x7fcf380fbcb0 2024-05-06T12:17:44.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.730+0000 7fcf34ff9700 1 -- 172.21.15.92:0/1827878641 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fcf28013980 con 0x7fcf380fbcb0 2024-05-06T12:17:44.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.730+0000 7fcf34ff9700 1 --2- 172.21.15.92:0/1827878641 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fcf2006e270 0x7fcf20070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:44.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.730+0000 7fcf34ff9700 1 -- 172.21.15.92:0/1827878641 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fcf2808eae0 con 0x7fcf380fbcb0 2024-05-06T12:17:44.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.730+0000 7fcf377fe700 1 --2- 172.21.15.92:0/1827878641 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fcf2006e270 0x7fcf20070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:44.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.731+0000 7fcf377fe700 1 --2- 172.21.15.92:0/1827878641 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fcf2006e270 0x7fcf20070720 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c009b10 tx=0x7fcf2c008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:17:44.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.736+0000 7fcf34ff9700 1 -- 172.21.15.92:0/1827878641 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fcf2805b770 con 0x7fcf380fbcb0 2024-05-06T12:17:44.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.981+0000 7fcf3e68e700 1 -- 172.21.15.92:0/1827878641 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf3805e3a0 con 0x7fcf2006e270 2024-05-06T12:17:44.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.982+0000 7fcf34ff9700 1 -- 172.21.15.92:0/1827878641 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+402 (secure 0 0 0) 0x7fcf3805e3a0 con 0x7fcf2006e270 2024-05-06T12:17:44.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.984+0000 7fcf3e68e700 1 -- 172.21.15.92:0/1827878641 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fcf2006e270 msgr2=0x7fcf20070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:44.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.984+0000 7fcf3e68e700 1 --2- 172.21.15.92:0/1827878641 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fcf2006e270 0x7fcf20070720 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c009b10 tx=0x7fcf2c008040 comp rx=0 tx=0).stop 2024-05-06T12:17:44.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.984+0000 7fcf3e68e700 1 -- 172.21.15.92:0/1827878641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf380fbcb0 msgr2=0x7fcf380ffbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:44.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.984+0000 7fcf3e68e700 1 --2- 172.21.15.92:0/1827878641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf380fbcb0 0x7fcf380ffbf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcf2800b3b0 tx=0x7fcf28009e30 comp rx=0 tx=0).stop 2024-05-06T12:17:44.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.985+0000 7fcf3e68e700 1 -- 172.21.15.92:0/1827878641 shutdown_connections 2024-05-06T12:17:44.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.985+0000 7fcf3e68e700 1 --2- 172.21.15.92:0/1827878641 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fcf2006e270 0x7fcf20070720 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:44.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.985+0000 7fcf3e68e700 1 --2- 172.21.15.92:0/1827878641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf380fbcb0 0x7fcf380ffbf0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:44.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.985+0000 7fcf3e68e700 1 --2- 172.21.15.92:0/1827878641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf380fcf60 0x7fcf380fe2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:44.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.985+0000 7fcf3e68e700 1 -- 172.21.15.92:0/1827878641 >> 172.21.15.92:0/1827878641 conn(0x7fcf380f7430 msgr2=0x7fcf38066820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:17:44.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.985+0000 7fcf3e68e700 1 -- 172.21.15.92:0/1827878641 shutdown_connections 2024-05-06T12:17:44.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:44.985+0000 7fcf3e68e700 1 -- 172.21.15.92:0/1827878641 wait complete. 2024-05-06T12:17:44.998 INFO:teuthology.orchestra.run.smithi092.stdout:true 2024-05-06T12:17:45.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.091+0000 7f97631b3700 1 -- 172.21.15.92:0/4198753239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f975c0fb1c0 msgr2=0x7f975c0fb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:45.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.091+0000 7f97631b3700 1 --2- 172.21.15.92:0/4198753239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f975c0fb1c0 0x7f975c0fb5f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9750004940 tx=0x7f9750009e30 comp rx=0 tx=0).stop 2024-05-06T12:17:45.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.091+0000 7f97631b3700 1 -- 172.21.15.92:0/4198753239 shutdown_connections 2024-05-06T12:17:45.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.091+0000 7f97631b3700 1 --2- 172.21.15.92:0/4198753239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f975c0fbb30 0x7f975c102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:45.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.091+0000 7f97631b3700 1 --2- 172.21.15.92:0/4198753239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f975c0fb1c0 0x7f975c0fb5f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:45.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.091+0000 7f97631b3700 1 -- 172.21.15.92:0/4198753239 >> 172.21.15.92:0/4198753239 conn(0x7f975c0f6b80 msgr2=0x7f975c0f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:17:45.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.091+0000 7f97631b3700 1 -- 172.21.15.92:0/4198753239 shutdown_connections 2024-05-06T12:17:45.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.091+0000 7f97631b3700 1 -- 172.21.15.92:0/4198753239 wait complete. 2024-05-06T12:17:45.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.092+0000 7f97631b3700 1 Processor -- start 2024-05-06T12:17:45.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.092+0000 7f97631b3700 1 -- start start 2024-05-06T12:17:45.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.092+0000 7f97631b3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f975c0fb1c0 0x7f975c10d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:45.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.092+0000 7f97631b3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f975c0fbb30 0x7f975c10dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:45.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.092+0000 7f97631b3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975c10e4e0 con 0x7f975c0fbb30 2024-05-06T12:17:45.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.092+0000 7f97631b3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975c10ed30 con 0x7f975c0fb1c0 2024-05-06T12:17:45.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.092+0000 7f975bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f975c0fbb30 0x7f975c10dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:45.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.092+0000 7f975bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f975c0fbb30 0x7f975c10dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36038/0 (socket says 172.21.15.92:36038) 2024-05-06T12:17:45.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.092+0000 7f975bfff700 1 -- 172.21.15.92:0/2039278539 learned_addr learned my addr 172.21.15.92:0/2039278539 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:17:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.092+0000 7f9760f4f700 1 --2- 172.21.15.92:0/2039278539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f975c0fb1c0 0x7f975c10d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.093+0000 7f975bfff700 1 -- 172.21.15.92:0/2039278539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f975c0fb1c0 msgr2=0x7f975c10d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.093+0000 7f975bfff700 1 --2- 172.21.15.92:0/2039278539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f975c0fb1c0 0x7f975c10d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.093+0000 7f975bfff700 1 -- 172.21.15.92:0/2039278539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9750005040 con 0x7f975c0fbb30 2024-05-06T12:17:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.093+0000 7f975bfff700 1 --2- 172.21.15.92:0/2039278539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f975c0fbb30 0x7f975c10dee0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9748002a80 tx=0x7f97480046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:17:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.093+0000 7f97597fa700 1 -- 172.21.15.92:0/2039278539 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9748007cc0 con 0x7f975c0fbb30 2024-05-06T12:17:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.093+0000 7f97631b3700 1 -- 172.21.15.92:0/2039278539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f975c10f010 con 0x7f975c0fbb30 2024-05-06T12:17:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.093+0000 7f97631b3700 1 -- 172.21.15.92:0/2039278539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f975c10f550 con 0x7f975c0fbb30 2024-05-06T12:17:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.093+0000 7f97597fa700 1 -- 172.21.15.92:0/2039278539 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f974800f040 con 0x7f975c0fbb30 2024-05-06T12:17:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.093+0000 7f97597fa700 1 -- 172.21.15.92:0/2039278539 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f974800dac0 con 0x7f975c0fbb30 2024-05-06T12:17:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.094+0000 7f97631b3700 1 -- 172.21.15.92:0/2039278539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f975c05f5c0 con 0x7f975c0fbb30 2024-05-06T12:17:45.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.095+0000 7f97597fa700 1 -- 172.21.15.92:0/2039278539 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f974800dcc0 con 0x7f975c0fbb30 2024-05-06T12:17:45.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.095+0000 7f97597fa700 1 --2- 172.21.15.92:0/2039278539 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f974c06e260 0x7f974c070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:45.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.095+0000 7f97597fa700 1 -- 172.21.15.92:0/2039278539 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f974808da70 con 0x7f975c0fbb30 2024-05-06T12:17:45.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.095+0000 7f9760f4f700 1 --2- 172.21.15.92:0/2039278539 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f974c06e260 0x7f974c070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:45.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.096+0000 7f9760f4f700 1 --2- 172.21.15.92:0/2039278539 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f974c06e260 0x7f974c070710 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f975000b3b0 tx=0x7f9750009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:17:45.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.100+0000 7f97597fa700 1 -- 172.21.15.92:0/2039278539 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f974805a780 con 0x7f975c0fbb30 2024-05-06T12:17:45.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.332+0000 7f97631b3700 1 -- 172.21.15.92:0/2039278539 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f975c10f8f0 con 0x7f974c06e260 2024-05-06T12:17:45.335 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:45 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:45.335 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:45 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:45.335 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:45 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:45.335 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:45 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:45.336 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:45 smithi092 ceph-mon[91933]: pgmap v36: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:17:45.336 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:45 smithi092 ceph-mon[91933]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:17:45.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.333+0000 7f97597fa700 1 -- 172.21.15.92:0/2039278539 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+402 (secure 0 0 0) 0x7f975c10f8f0 con 0x7f974c06e260 2024-05-06T12:17:45.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.336+0000 7f97631b3700 1 -- 172.21.15.92:0/2039278539 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f974c06e260 msgr2=0x7f974c070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:45.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.336+0000 7f97631b3700 1 --2- 172.21.15.92:0/2039278539 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f974c06e260 0x7f974c070710 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f975000b3b0 tx=0x7f9750009e30 comp rx=0 tx=0).stop 2024-05-06T12:17:45.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.336+0000 7f97631b3700 1 -- 172.21.15.92:0/2039278539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f975c0fbb30 msgr2=0x7f975c10dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:45.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.336+0000 7f97631b3700 1 --2- 172.21.15.92:0/2039278539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f975c0fbb30 0x7f975c10dee0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9748002a80 tx=0x7f97480046b0 comp rx=0 tx=0).stop 2024-05-06T12:17:45.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.336+0000 7f97631b3700 1 -- 172.21.15.92:0/2039278539 shutdown_connections 2024-05-06T12:17:45.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.336+0000 7f97631b3700 1 --2- 172.21.15.92:0/2039278539 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f974c06e260 0x7f974c070710 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:45.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.336+0000 7f97631b3700 1 --2- 172.21.15.92:0/2039278539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f975c0fb1c0 0x7f975c10d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:45.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.336+0000 7f97631b3700 1 --2- 172.21.15.92:0/2039278539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f975c0fbb30 0x7f975c10dee0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:45.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.336+0000 7f97631b3700 1 -- 172.21.15.92:0/2039278539 >> 172.21.15.92:0/2039278539 conn(0x7f975c0f6b80 msgr2=0x7f975c1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:17:45.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.336+0000 7f97631b3700 1 -- 172.21.15.92:0/2039278539 shutdown_connections 2024-05-06T12:17:45.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.336+0000 7f97631b3700 1 -- 172.21.15.92:0/2039278539 wait complete. 2024-05-06T12:17:45.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:45 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:45.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:45 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:45.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:45 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:45.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:45 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:45.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:45 smithi190 ceph-mon[79744]: pgmap v36: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:17:45.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:45 smithi190 ceph-mon[79744]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:17:45.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.445+0000 7f9cf00c9700 1 -- 172.21.15.92:0/2016244012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ce80fe090 msgr2=0x7f9ce80fe500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:45.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.445+0000 7f9cf00c9700 1 --2- 172.21.15.92:0/2016244012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ce80fe090 0x7f9ce80fe500 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8004940 tx=0x7f9cd8009e30 comp rx=0 tx=0).stop 2024-05-06T12:17:45.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.446+0000 7f9cf00c9700 1 -- 172.21.15.92:0/2016244012 shutdown_connections 2024-05-06T12:17:45.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.446+0000 7f9cf00c9700 1 --2- 172.21.15.92:0/2016244012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ce80fe090 0x7f9ce80fe500 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:45.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.446+0000 7f9cf00c9700 1 --2- 172.21.15.92:0/2016244012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ce80fce90 0x7f9ce80fd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:45.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.446+0000 7f9cf00c9700 1 -- 172.21.15.92:0/2016244012 >> 172.21.15.92:0/2016244012 conn(0x7f9ce80f85c0 msgr2=0x7f9ce80fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:17:45.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.446+0000 7f9cf00c9700 1 -- 172.21.15.92:0/2016244012 shutdown_connections 2024-05-06T12:17:45.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.446+0000 7f9cf00c9700 1 -- 172.21.15.92:0/2016244012 wait complete. 2024-05-06T12:17:45.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.446+0000 7f9cf00c9700 1 Processor -- start 2024-05-06T12:17:45.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.446+0000 7f9cf00c9700 1 -- start start 2024-05-06T12:17:45.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.446+0000 7f9cf00c9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ce80fce90 0x7f9ce810d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:45.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.446+0000 7f9cf00c9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ce80fe090 0x7f9ce810def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:45.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.446+0000 7f9cf00c9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce810e4f0 con 0x7f9ce80fce90 2024-05-06T12:17:45.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.446+0000 7f9cf00c9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce810ed40 con 0x7f9ce80fe090 2024-05-06T12:17:45.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.446+0000 7f9cede65700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ce80fce90 0x7f9ce810d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:45.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.446+0000 7f9cede65700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ce80fce90 0x7f9ce810d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36054/0 (socket says 172.21.15.92:36054) 2024-05-06T12:17:45.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.447+0000 7f9cede65700 1 -- 172.21.15.92:0/95027175 learned_addr learned my addr 172.21.15.92:0/95027175 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:17:45.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.447+0000 7f9ced664700 1 --2- 172.21.15.92:0/95027175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ce80fe090 0x7f9ce810def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:45.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.447+0000 7f9cede65700 1 -- 172.21.15.92:0/95027175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ce80fe090 msgr2=0x7f9ce810def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:45.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.447+0000 7f9cede65700 1 --2- 172.21.15.92:0/95027175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ce80fe090 0x7f9ce810def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:45.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.447+0000 7f9cede65700 1 -- 172.21.15.92:0/95027175 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd8005040 con 0x7f9ce80fce90 2024-05-06T12:17:45.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.447+0000 7f9cede65700 1 --2- 172.21.15.92:0/95027175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ce80fce90 0x7f9ce810d9b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9ce400eb70 tx=0x7f9ce400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:17:45.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.447+0000 7f9cde7fc700 1 -- 172.21.15.92:0/95027175 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ce4007cc0 con 0x7f9ce80fce90 2024-05-06T12:17:45.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.447+0000 7f9cde7fc700 1 -- 172.21.15.92:0/95027175 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ce400f040 con 0x7f9ce80fce90 2024-05-06T12:17:45.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.447+0000 7f9cf00c9700 1 -- 172.21.15.92:0/95027175 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ce810efc0 con 0x7f9ce80fce90 2024-05-06T12:17:45.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.448+0000 7f9cde7fc700 1 -- 172.21.15.92:0/95027175 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ce400cb20 con 0x7f9ce80fce90 2024-05-06T12:17:45.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.448+0000 7f9cf00c9700 1 -- 172.21.15.92:0/95027175 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce810f500 con 0x7f9ce80fce90 2024-05-06T12:17:45.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.449+0000 7f9cde7fc700 1 -- 172.21.15.92:0/95027175 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f9ce400cc80 con 0x7f9ce80fce90 2024-05-06T12:17:45.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.449+0000 7f9cde7fc700 1 --2- 172.21.15.92:0/95027175 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f9cd406e060 0x7f9cd4070510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:45.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.449+0000 7f9cf00c9700 1 -- 172.21.15.92:0/95027175 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ce805f5c0 con 0x7f9ce80fce90 2024-05-06T12:17:45.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.449+0000 7f9ced664700 1 --2- 172.21.15.92:0/95027175 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f9cd406e060 0x7f9cd4070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:45.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.449+0000 7f9cde7fc700 1 -- 172.21.15.92:0/95027175 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f9ce408d700 con 0x7f9ce80fce90 2024-05-06T12:17:45.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.450+0000 7f9ced664700 1 --2- 172.21.15.92:0/95027175 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f9cd406e060 0x7f9cd4070510 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8009b30 tx=0x7f9cd8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:17:45.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.454+0000 7f9cde7fc700 1 -- 172.21.15.92:0/95027175 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9ce405a410 con 0x7f9ce80fce90 2024-05-06T12:17:45.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.668+0000 7f9cf00c9700 1 -- 172.21.15.92:0/95027175 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ce810f740 con 0x7f9cd406e060 2024-05-06T12:17:45.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.686+0000 7f9cde7fc700 1 -- 172.21.15.92:0/95027175 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f9ce810f740 con 0x7f9cd406e060 2024-05-06T12:17:45.690 INFO:teuthology.orchestra.run.smithi092.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-06T12:17:45.690 INFO:teuthology.orchestra.run.smithi092.stdout:alertmanager.smithi092 smithi092 *:9093,9094 running (2m) 0s ago 11m 17.0M - 0.25.0 c8568f914cd2 be5637d7d25a 2024-05-06T12:17:45.690 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi092 smithi092 running (11s) 0s ago 11m 6660k - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 8da0494e1bdc 2024-05-06T12:17:45.690 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi190 smithi190 running (6s) 2s ago 10m 6665k - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 641a7cdaa56e 2024-05-06T12:17:45.690 INFO:teuthology.orchestra.run.smithi092.stdout:grafana.smithi092 smithi092 *:3000 running (2m) 0s ago 11m 77.0M - 9.4.7 954c08fa6188 9180cb3e5258 2024-05-06T12:17:45.690 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.ijsqgp smithi092 running (5m) 0s ago 5m 60.6M - 17.2.7-925-ge81df420 96d5ae7fd4b4 b4c2ed6d2c82 2024-05-06T12:17:45.690 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.rosbll smithi092 running (5m) 0s ago 5m 122M - 17.2.7-925-ge81df420 96d5ae7fd4b4 5f5426cd1bf5 2024-05-06T12:17:45.690 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.byjvgf smithi190 running (5m) 2s ago 5m 126M - 17.2.7-925-ge81df420 96d5ae7fd4b4 24581d2e1c5d 2024-05-06T12:17:45.690 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.idzoff smithi190 running (5m) 2s ago 5m 64.3M - 17.2.7-925-ge81df420 96d5ae7fd4b4 20f2641696ef 2024-05-06T12:17:45.690 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi092.gyzeqb smithi092 *:8443,9283,8765 running (2m) 0s ago 12m 498M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 de9d2198327a 2024-05-06T12:17:45.691 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi190.kczxkh smithi190 *:8443,9283,8765 running (89s) 2s ago 10m 449M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 e84ab3758ac7 2024-05-06T12:17:45.691 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi092 smithi092 running (75s) 0s ago 12m 40.7M 2048M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 ac07f7158087 2024-05-06T12:17:45.691 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi190 smithi190 running (52s) 2s ago 10m 34.0M 2048M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 7a3c08eb716f 2024-05-06T12:17:45.691 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi092 smithi092 *:9100 running (2m) 0s ago 11m 12.4M - 1.5.0 0da6a335fe13 c796446b206f 2024-05-06T12:17:45.691 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi190 smithi190 *:9100 running (2m) 2s ago 10m 16.9M - 1.5.0 0da6a335fe13 647e28111a76 2024-05-06T12:17:45.691 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 smithi092 running (9m) 0s ago 9m 200M 1557M 17.2.7-925-ge81df420 96d5ae7fd4b4 ce84e7af9084 2024-05-06T12:17:45.691 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 smithi092 running (9m) 0s ago 9m 200M 1557M 17.2.7-925-ge81df420 96d5ae7fd4b4 8ab870f57eb2 2024-05-06T12:17:45.691 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 smithi092 running (8m) 0s ago 8m 165M 1557M 17.2.7-925-ge81df420 96d5ae7fd4b4 1e6ed04a0688 2024-05-06T12:17:45.691 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 smithi190 running (8m) 2s ago 8m 221M 2581M 17.2.7-925-ge81df420 96d5ae7fd4b4 52f9521286d5 2024-05-06T12:17:45.691 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 smithi190 running (7m) 2s ago 7m 204M 2581M 17.2.7-925-ge81df420 96d5ae7fd4b4 3a7da09e4c48 2024-05-06T12:17:45.691 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 smithi190 running (7m) 2s ago 7m 188M 2581M 17.2.7-925-ge81df420 96d5ae7fd4b4 e7452c6b5cec 2024-05-06T12:17:45.691 INFO:teuthology.orchestra.run.smithi092.stdout:prometheus.smithi092 smithi092 *:9095 running (98s) 0s ago 10m 48.7M - 2.43.0 a07b618ecd1d bc5445691527 2024-05-06T12:17:45.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.688+0000 7f9cf00c9700 1 -- 172.21.15.92:0/95027175 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f9cd406e060 msgr2=0x7f9cd4070510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:45.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.688+0000 7f9cf00c9700 1 --2- 172.21.15.92:0/95027175 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f9cd406e060 0x7f9cd4070510 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8009b30 tx=0x7f9cd8009e30 comp rx=0 tx=0).stop 2024-05-06T12:17:45.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.688+0000 7f9cf00c9700 1 -- 172.21.15.92:0/95027175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ce80fce90 msgr2=0x7f9ce810d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:45.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.688+0000 7f9cf00c9700 1 --2- 172.21.15.92:0/95027175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ce80fce90 0x7f9ce810d9b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9ce400eb70 tx=0x7f9ce400b040 comp rx=0 tx=0).stop 2024-05-06T12:17:45.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.689+0000 7f9cf00c9700 1 -- 172.21.15.92:0/95027175 shutdown_connections 2024-05-06T12:17:45.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.689+0000 7f9cf00c9700 1 --2- 172.21.15.92:0/95027175 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f9cd406e060 0x7f9cd4070510 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:45.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.689+0000 7f9cf00c9700 1 --2- 172.21.15.92:0/95027175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ce80fce90 0x7f9ce810d9b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:45.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.689+0000 7f9cf00c9700 1 --2- 172.21.15.92:0/95027175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ce80fe090 0x7f9ce810def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:45.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.689+0000 7f9cf00c9700 1 -- 172.21.15.92:0/95027175 >> 172.21.15.92:0/95027175 conn(0x7f9ce80f85c0 msgr2=0x7f9ce80f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:17:45.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.689+0000 7f9cf00c9700 1 -- 172.21.15.92:0/95027175 shutdown_connections 2024-05-06T12:17:45.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.689+0000 7f9cf00c9700 1 -- 172.21.15.92:0/95027175 wait complete. 2024-05-06T12:17:45.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.793+0000 7f7d2d737700 1 -- 172.21.15.92:0/3347943147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d280fcf60 msgr2=0x7f7d280fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:45.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.793+0000 7f7d2d737700 1 --2- 172.21.15.92:0/3347943147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d280fcf60 0x7f7d280fd3d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c004940 tx=0x7f7d1c009e30 comp rx=0 tx=0).stop 2024-05-06T12:17:45.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.793+0000 7f7d2d737700 1 -- 172.21.15.92:0/3347943147 shutdown_connections 2024-05-06T12:17:45.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.793+0000 7f7d2d737700 1 --2- 172.21.15.92:0/3347943147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d280fcf60 0x7f7d280fd3d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:45.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.793+0000 7f7d2d737700 1 --2- 172.21.15.92:0/3347943147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d280fbcb0 0x7f7d280fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:45.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.793+0000 7f7d2d737700 1 -- 172.21.15.92:0/3347943147 >> 172.21.15.92:0/3347943147 conn(0x7f7d280f7430 msgr2=0x7f7d280f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:17:45.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.793+0000 7f7d2d737700 1 -- 172.21.15.92:0/3347943147 shutdown_connections 2024-05-06T12:17:45.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.793+0000 7f7d2d737700 1 -- 172.21.15.92:0/3347943147 wait complete. 2024-05-06T12:17:45.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.793+0000 7f7d2d737700 1 Processor -- start 2024-05-06T12:17:45.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.793+0000 7f7d2d737700 1 -- start start 2024-05-06T12:17:45.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.794+0000 7f7d2d737700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d280fbcb0 0x7f7d2810da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:45.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.794+0000 7f7d2d737700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d280fcf60 0x7f7d2810dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:45.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.794+0000 7f7d2d737700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2810e5c0 con 0x7f7d280fbcb0 2024-05-06T12:17:45.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.794+0000 7f7d2d737700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2810ed70 con 0x7f7d280fcf60 2024-05-06T12:17:45.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.794+0000 7f7d26ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d280fbcb0 0x7f7d2810da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:45.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.794+0000 7f7d26ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d280fbcb0 0x7f7d2810da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36078/0 (socket says 172.21.15.92:36078) 2024-05-06T12:17:45.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.794+0000 7f7d26ffd700 1 -- 172.21.15.92:0/4226097394 learned_addr learned my addr 172.21.15.92:0/4226097394 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:17:45.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.794+0000 7f7d267fc700 1 --2- 172.21.15.92:0/4226097394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d280fcf60 0x7f7d2810dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:45.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.794+0000 7f7d26ffd700 1 -- 172.21.15.92:0/4226097394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d280fcf60 msgr2=0x7f7d2810dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:45.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.794+0000 7f7d26ffd700 1 --2- 172.21.15.92:0/4226097394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d280fcf60 0x7f7d2810dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:45.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.794+0000 7f7d26ffd700 1 -- 172.21.15.92:0/4226097394 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1c005040 con 0x7f7d280fbcb0 2024-05-06T12:17:45.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.794+0000 7f7d267fc700 1 --2- 172.21.15.92:0/4226097394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d280fcf60 0x7f7d2810dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-06T12:17:45.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.794+0000 7f7d26ffd700 1 --2- 172.21.15.92:0/4226097394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d280fbcb0 0x7f7d2810da80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800eb40 tx=0x7f7d1800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:17:45.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.794+0000 7f7d0f7fe700 1 -- 172.21.15.92:0/4226097394 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d18007cc0 con 0x7f7d280fbcb0 2024-05-06T12:17:45.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.794+0000 7f7d2d737700 1 -- 172.21.15.92:0/4226097394 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2810eff0 con 0x7f7d280fbcb0 2024-05-06T12:17:45.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.794+0000 7f7d0f7fe700 1 -- 172.21.15.92:0/4226097394 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d1800f040 con 0x7f7d280fbcb0 2024-05-06T12:17:45.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.794+0000 7f7d0f7fe700 1 -- 172.21.15.92:0/4226097394 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d18009620 con 0x7f7d280fbcb0 2024-05-06T12:17:45.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.794+0000 7f7d2d737700 1 -- 172.21.15.92:0/4226097394 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2810f530 con 0x7f7d280fbcb0 2024-05-06T12:17:45.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.795+0000 7f7d2d737700 1 -- 172.21.15.92:0/4226097394 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d28101af0 con 0x7f7d280fbcb0 2024-05-06T12:17:45.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.796+0000 7f7d0f7fe700 1 -- 172.21.15.92:0/4226097394 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f7d18009780 con 0x7f7d280fbcb0 2024-05-06T12:17:45.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.796+0000 7f7d0f7fe700 1 --2- 172.21.15.92:0/4226097394 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7d1006e260 0x7f7d10070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:45.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.796+0000 7f7d0f7fe700 1 -- 172.21.15.92:0/4226097394 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f7d1808daa0 con 0x7f7d280fbcb0 2024-05-06T12:17:45.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.796+0000 7f7d267fc700 1 --2- 172.21.15.92:0/4226097394 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7d1006e260 0x7f7d10070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:45.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.797+0000 7f7d267fc700 1 --2- 172.21.15.92:0/4226097394 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7d1006e260 0x7f7d10070710 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c004560 tx=0x7f7d1c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:17:45.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:45.801+0000 7f7d0f7fe700 1 -- 172.21.15.92:0/4226097394 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7d1805a7e0 con 0x7f7d280fbcb0 2024-05-06T12:17:46.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.106+0000 7f7d2d737700 1 -- 172.21.15.92:0/4226097394 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7d281048a0 con 0x7f7d280fbcb0 2024-05-06T12:17:46.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.106+0000 7f7d0f7fe700 1 -- 172.21.15.92:0/4226097394 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7f7d1805a370 con 0x7f7d280fbcb0 2024-05-06T12:17:46.110 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:17:46.110 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": { 2024-05-06T12:17:46.110 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:17:46.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:17:46.110 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": { 2024-05-06T12:17:46.110 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:17:46.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:17:46.110 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": { 2024-05-06T12:17:46.110 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 6 2024-05-06T12:17:46.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:17:46.110 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-05-06T12:17:46.111 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 4 2024-05-06T12:17:46.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:17:46.111 INFO:teuthology.orchestra.run.smithi092.stdout: "overall": { 2024-05-06T12:17:46.111 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 10, 2024-05-06T12:17:46.111 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 4 2024-05-06T12:17:46.111 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-06T12:17:46.111 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:17:46.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.108+0000 7f7d2d737700 1 -- 172.21.15.92:0/4226097394 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7d1006e260 msgr2=0x7f7d10070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:46.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.108+0000 7f7d2d737700 1 --2- 172.21.15.92:0/4226097394 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7d1006e260 0x7f7d10070710 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c004560 tx=0x7f7d1c009e30 comp rx=0 tx=0).stop 2024-05-06T12:17:46.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.108+0000 7f7d2d737700 1 -- 172.21.15.92:0/4226097394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d280fbcb0 msgr2=0x7f7d2810da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:46.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.108+0000 7f7d2d737700 1 --2- 172.21.15.92:0/4226097394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d280fbcb0 0x7f7d2810da80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800eb40 tx=0x7f7d1800b040 comp rx=0 tx=0).stop 2024-05-06T12:17:46.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.108+0000 7f7d2d737700 1 -- 172.21.15.92:0/4226097394 shutdown_connections 2024-05-06T12:17:46.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.108+0000 7f7d2d737700 1 --2- 172.21.15.92:0/4226097394 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7d1006e260 0x7f7d10070710 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:46.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.108+0000 7f7d2d737700 1 --2- 172.21.15.92:0/4226097394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d280fbcb0 0x7f7d2810da80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:46.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.108+0000 7f7d2d737700 1 --2- 172.21.15.92:0/4226097394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d280fcf60 0x7f7d2810dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:46.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.108+0000 7f7d2d737700 1 -- 172.21.15.92:0/4226097394 >> 172.21.15.92:0/4226097394 conn(0x7f7d280f7430 msgr2=0x7f7d28065c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:17:46.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.109+0000 7f7d2d737700 1 -- 172.21.15.92:0/4226097394 shutdown_connections 2024-05-06T12:17:46.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.109+0000 7f7d2d737700 1 -- 172.21.15.92:0/4226097394 wait complete. 2024-05-06T12:17:46.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.212+0000 7ff9f8025700 1 -- 172.21.15.92:0/380470477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9f00fb420 msgr2=0x7ff9f00fb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:46.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.212+0000 7ff9f8025700 1 --2- 172.21.15.92:0/380470477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9f00fb420 0x7ff9f00fb830 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec004940 tx=0x7ff9ec009e30 comp rx=0 tx=0).stop 2024-05-06T12:17:46.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.212+0000 7ff9f8025700 1 -- 172.21.15.92:0/380470477 shutdown_connections 2024-05-06T12:17:46.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.212+0000 7ff9f8025700 1 --2- 172.21.15.92:0/380470477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9f00fc6d0 0x7ff9f00fcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:46.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.212+0000 7ff9f8025700 1 --2- 172.21.15.92:0/380470477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9f00fb420 0x7ff9f00fb830 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:46.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.212+0000 7ff9f8025700 1 -- 172.21.15.92:0/380470477 >> 172.21.15.92:0/380470477 conn(0x7ff9f00f6b80 msgr2=0x7ff9f00f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:17:46.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.212+0000 7ff9f8025700 1 -- 172.21.15.92:0/380470477 shutdown_connections 2024-05-06T12:17:46.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.212+0000 7ff9f8025700 1 -- 172.21.15.92:0/380470477 wait complete. 2024-05-06T12:17:46.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.212+0000 7ff9f8025700 1 Processor -- start 2024-05-06T12:17:46.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.212+0000 7ff9f8025700 1 -- start start 2024-05-06T12:17:46.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.213+0000 7ff9f8025700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9f00fb420 0x7ff9f01106b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:46.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.213+0000 7ff9f8025700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9f00fc6d0 0x7ff9f0112c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:46.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.213+0000 7ff9f8025700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f00f88e0 con 0x7ff9f00fc6d0 2024-05-06T12:17:46.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.213+0000 7ff9f8025700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f00f8a20 con 0x7ff9f00fb420 2024-05-06T12:17:46.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.213+0000 7ff9f55c0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9f00fc6d0 0x7ff9f0112c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:46.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.213+0000 7ff9f55c0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9f00fc6d0 0x7ff9f0112c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36092/0 (socket says 172.21.15.92:36092) 2024-05-06T12:17:46.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.213+0000 7ff9f55c0700 1 -- 172.21.15.92:0/3395722207 learned_addr learned my addr 172.21.15.92:0/3395722207 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:17:46.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.213+0000 7ff9f5dc1700 1 --2- 172.21.15.92:0/3395722207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9f00fb420 0x7ff9f01106b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:46.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.213+0000 7ff9f55c0700 1 -- 172.21.15.92:0/3395722207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9f00fb420 msgr2=0x7ff9f01106b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:46.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.213+0000 7ff9f55c0700 1 --2- 172.21.15.92:0/3395722207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9f00fb420 0x7ff9f01106b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:46.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.213+0000 7ff9f55c0700 1 -- 172.21.15.92:0/3395722207 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ec005040 con 0x7ff9f00fc6d0 2024-05-06T12:17:46.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.213+0000 7ff9f5dc1700 1 --2- 172.21.15.92:0/3395722207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9f00fb420 0x7ff9f01106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-06T12:17:46.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.213+0000 7ff9f55c0700 1 --2- 172.21.15.92:0/3395722207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9f00fc6d0 0x7ff9f0112c00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff9e000eb40 tx=0x7ff9e000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:17:46.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.213+0000 7ff9e67fc700 1 -- 172.21.15.92:0/3395722207 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9e0007cc0 con 0x7ff9f00fc6d0 2024-05-06T12:17:46.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.213+0000 7ff9e67fc700 1 -- 172.21.15.92:0/3395722207 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff9e000f040 con 0x7ff9f00fc6d0 2024-05-06T12:17:46.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.213+0000 7ff9e67fc700 1 -- 172.21.15.92:0/3395722207 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9e000ca10 con 0x7ff9f00fc6d0 2024-05-06T12:17:46.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.214+0000 7ff9f8025700 1 -- 172.21.15.92:0/3395722207 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f01131a0 con 0x7ff9f00fc6d0 2024-05-06T12:17:46.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.214+0000 7ff9f8025700 1 -- 172.21.15.92:0/3395722207 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f01136b0 con 0x7ff9f00fc6d0 2024-05-06T12:17:46.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.214+0000 7ff9f8025700 1 -- 172.21.15.92:0/3395722207 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9f005f5c0 con 0x7ff9f00fc6d0 2024-05-06T12:17:46.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.215+0000 7ff9e67fc700 1 -- 172.21.15.92:0/3395722207 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7ff9e000cc80 con 0x7ff9f00fc6d0 2024-05-06T12:17:46.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.215+0000 7ff9e67fc700 1 --2- 172.21.15.92:0/3395722207 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff9dc06e260 0x7ff9dc070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:46.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.215+0000 7ff9e67fc700 1 -- 172.21.15.92:0/3395722207 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff9e008d960 con 0x7ff9f00fc6d0 2024-05-06T12:17:46.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.216+0000 7ff9f5dc1700 1 --2- 172.21.15.92:0/3395722207 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff9dc06e260 0x7ff9dc070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:46.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.216+0000 7ff9f5dc1700 1 --2- 172.21.15.92:0/3395722207 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff9dc06e260 0x7ff9dc070710 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec004890 tx=0x7ff9ec009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:17:46.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.220+0000 7ff9e67fc700 1 -- 172.21.15.92:0/3395722207 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff9e0056b70 con 0x7ff9f00fc6d0 2024-05-06T12:17:46.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.472+0000 7ff9f8025700 1 -- 172.21.15.92:0/3395722207 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff9f0113990 con 0x7ff9f00fc6d0 2024-05-06T12:17:46.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.473+0000 7ff9e67fc700 1 -- 172.21.15.92:0/3395722207 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1937 (secure 0 0 0) 0x7ff9e005a230 con 0x7ff9f00fc6d0 2024-05-06T12:17:46.476 INFO:teuthology.orchestra.run.smithi092.stdout:e20 2024-05-06T12:17:46.476 INFO:teuthology.orchestra.run.smithi092.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-06T12:17:46.477 INFO:teuthology.orchestra.run.smithi092.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:17:46.477 INFO:teuthology.orchestra.run.smithi092.stdout:legacy client fscid: 1 2024-05-06T12:17:46.477 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:17:46.477 INFO:teuthology.orchestra.run.smithi092.stdout:Filesystem 'cephfs' (1) 2024-05-06T12:17:46.477 INFO:teuthology.orchestra.run.smithi092.stdout:fs_name cephfs 2024-05-06T12:17:46.477 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 20 2024-05-06T12:17:46.477 INFO:teuthology.orchestra.run.smithi092.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-06T12:17:46.477 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-05-06T12:11:44.952671+0000 2024-05-06T12:17:46.477 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2024-05-06T12:15:07.568167+0000 2024-05-06T12:17:46.477 INFO:teuthology.orchestra.run.smithi092.stdout:tableserver 0 2024-05-06T12:17:46.477 INFO:teuthology.orchestra.run.smithi092.stdout:root 0 2024-05-06T12:17:46.477 INFO:teuthology.orchestra.run.smithi092.stdout:session_timeout 60 2024-05-06T12:17:46.477 INFO:teuthology.orchestra.run.smithi092.stdout:session_autoclose 300 2024-05-06T12:17:46.477 INFO:teuthology.orchestra.run.smithi092.stdout:max_file_size 1099511627776 2024-05-06T12:17:46.477 INFO:teuthology.orchestra.run.smithi092.stdout:max_xattr_size 65536 2024-05-06T12:17:46.478 INFO:teuthology.orchestra.run.smithi092.stdout:required_client_features {} 2024-05-06T12:17:46.478 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure 0 2024-05-06T12:17:46.478 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure_osd_epoch 46 2024-05-06T12:17:46.478 INFO:teuthology.orchestra.run.smithi092.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:17:46.478 INFO:teuthology.orchestra.run.smithi092.stdout:max_mds 2 2024-05-06T12:17:46.478 INFO:teuthology.orchestra.run.smithi092.stdout:in 0,1 2024-05-06T12:17:46.478 INFO:teuthology.orchestra.run.smithi092.stdout:up {0=14472,1=24267} 2024-05-06T12:17:46.478 INFO:teuthology.orchestra.run.smithi092.stdout:failed 2024-05-06T12:17:46.478 INFO:teuthology.orchestra.run.smithi092.stdout:damaged 2024-05-06T12:17:46.478 INFO:teuthology.orchestra.run.smithi092.stdout:stopped 2024-05-06T12:17:46.478 INFO:teuthology.orchestra.run.smithi092.stdout:data_pools [3] 2024-05-06T12:17:46.478 INFO:teuthology.orchestra.run.smithi092.stdout:metadata_pool 2 2024-05-06T12:17:46.478 INFO:teuthology.orchestra.run.smithi092.stdout:inline_data enabled 2024-05-06T12:17:46.478 INFO:teuthology.orchestra.run.smithi092.stdout:balancer 2024-05-06T12:17:46.479 INFO:teuthology.orchestra.run.smithi092.stdout:bal_rank_mask -1 2024-05-06T12:17:46.479 INFO:teuthology.orchestra.run.smithi092.stdout:standby_count_wanted 1 2024-05-06T12:17:46.479 INFO:teuthology.orchestra.run.smithi092.stdout:qdb_cluster leader: 0 members: 2024-05-06T12:17:46.479 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.rosbll{0:14472} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.92:6828/1663253598,v1:172.21.15.92:6829/1663253598] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:17:46.479 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.ijsqgp{0:14484} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.92:6826/3613441054,v1:172.21.15.92:6827/3613441054] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:17:46.479 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.byjvgf{1:24267} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.190:6824/1565151699,v1:172.21.15.190:6825/1565151699] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:17:46.479 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.idzoff{1:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.190:6826/2453728380,v1:172.21.15.190:6827/2453728380] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:17:46.479 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:17:46.479 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:17:46.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.475+0000 7ff9f8025700 1 -- 172.21.15.92:0/3395722207 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff9dc06e260 msgr2=0x7ff9dc070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:46.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.475+0000 7ff9f8025700 1 --2- 172.21.15.92:0/3395722207 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff9dc06e260 0x7ff9dc070710 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec004890 tx=0x7ff9ec009e30 comp rx=0 tx=0).stop 2024-05-06T12:17:46.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.475+0000 7ff9f8025700 1 -- 172.21.15.92:0/3395722207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9f00fc6d0 msgr2=0x7ff9f0112c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:46.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.475+0000 7ff9f8025700 1 --2- 172.21.15.92:0/3395722207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9f00fc6d0 0x7ff9f0112c00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff9e000eb40 tx=0x7ff9e000b040 comp rx=0 tx=0).stop 2024-05-06T12:17:46.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.475+0000 7ff9f8025700 1 -- 172.21.15.92:0/3395722207 shutdown_connections 2024-05-06T12:17:46.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.475+0000 7ff9f8025700 1 --2- 172.21.15.92:0/3395722207 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff9dc06e260 0x7ff9dc070710 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:46.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.475+0000 7ff9f8025700 1 --2- 172.21.15.92:0/3395722207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9f00fb420 0x7ff9f01106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:46.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.475+0000 7ff9f8025700 1 --2- 172.21.15.92:0/3395722207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9f00fc6d0 0x7ff9f0112c00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:46.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.475+0000 7ff9f8025700 1 -- 172.21.15.92:0/3395722207 >> 172.21.15.92:0/3395722207 conn(0x7ff9f00f6b80 msgr2=0x7ff9f00ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:17:46.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.475+0000 7ff9f8025700 1 -- 172.21.15.92:0/3395722207 shutdown_connections 2024-05-06T12:17:46.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.475+0000 7ff9f8025700 1 -- 172.21.15.92:0/3395722207 wait complete. 2024-05-06T12:17:46.480 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 20 2024-05-06T12:17:46.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.577+0000 7f97a77b4700 1 -- 172.21.15.92:0/1547787162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97a00fc720 msgr2=0x7f97a00fcb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:46.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.577+0000 7f97a77b4700 1 --2- 172.21.15.92:0/1547787162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97a00fc720 0x7f97a00fcb90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9790004940 tx=0x7f9790009e30 comp rx=0 tx=0).stop 2024-05-06T12:17:46.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.577+0000 7f97a77b4700 1 -- 172.21.15.92:0/1547787162 shutdown_connections 2024-05-06T12:17:46.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.577+0000 7f97a77b4700 1 --2- 172.21.15.92:0/1547787162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97a00fc720 0x7f97a00fcb90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:46.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.577+0000 7f97a77b4700 1 --2- 172.21.15.92:0/1547787162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a00fb470 0x7f97a00fb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:46.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.577+0000 7f97a77b4700 1 -- 172.21.15.92:0/1547787162 >> 172.21.15.92:0/1547787162 conn(0x7f97a00f6bd0 msgr2=0x7f97a00f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:17:46.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.577+0000 7f97a77b4700 1 -- 172.21.15.92:0/1547787162 shutdown_connections 2024-05-06T12:17:46.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.577+0000 7f97a77b4700 1 -- 172.21.15.92:0/1547787162 wait complete. 2024-05-06T12:17:46.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.578+0000 7f97a77b4700 1 Processor -- start 2024-05-06T12:17:46.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.578+0000 7f97a77b4700 1 -- start start 2024-05-06T12:17:46.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.578+0000 7f97a77b4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97a00fb470 0x7f97a010d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:46.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.578+0000 7f97a77b4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a00fc720 0x7f97a010dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:46.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.578+0000 7f97a77b4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a010e4a0 con 0x7f97a00fb470 2024-05-06T12:17:46.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.578+0000 7f97a77b4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a010e610 con 0x7f97a00fc720 2024-05-06T12:17:46.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.578+0000 7f97a5550700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97a00fb470 0x7f97a010d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:46.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.578+0000 7f97a5550700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97a00fb470 0x7f97a010d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36104/0 (socket says 172.21.15.92:36104) 2024-05-06T12:17:46.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.578+0000 7f97a5550700 1 -- 172.21.15.92:0/1500477231 learned_addr learned my addr 172.21.15.92:0/1500477231 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:17:46.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.578+0000 7f97a4d4f700 1 --2- 172.21.15.92:0/1500477231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a00fc720 0x7f97a010dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:46.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.578+0000 7f97a5550700 1 -- 172.21.15.92:0/1500477231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a00fc720 msgr2=0x7f97a010dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:46.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.578+0000 7f97a5550700 1 --2- 172.21.15.92:0/1500477231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a00fc720 0x7f97a010dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:46.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.578+0000 7f97a5550700 1 -- 172.21.15.92:0/1500477231 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9790005040 con 0x7f97a00fb470 2024-05-06T12:17:46.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.578+0000 7f97a4d4f700 1 --2- 172.21.15.92:0/1500477231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a00fc720 0x7f97a010dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-06T12:17:46.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.578+0000 7f97a5550700 1 --2- 172.21.15.92:0/1500477231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97a00fb470 0x7f97a010d960 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f979c002a50 tx=0x7f979c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:17:46.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.579+0000 7f9795ffb700 1 -- 172.21.15.92:0/1500477231 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f979c007cc0 con 0x7f97a00fb470 2024-05-06T12:17:46.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.579+0000 7f9795ffb700 1 -- 172.21.15.92:0/1500477231 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f979c00f040 con 0x7f97a00fb470 2024-05-06T12:17:46.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.579+0000 7f9795ffb700 1 -- 172.21.15.92:0/1500477231 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f979c009640 con 0x7f97a00fb470 2024-05-06T12:17:46.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.579+0000 7f97a77b4700 1 -- 172.21.15.92:0/1500477231 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a0113070 con 0x7f97a00fb470 2024-05-06T12:17:46.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.579+0000 7f97a77b4700 1 -- 172.21.15.92:0/1500477231 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a0113630 con 0x7f97a00fb470 2024-05-06T12:17:46.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.580+0000 7f97a77b4700 1 -- 172.21.15.92:0/1500477231 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97a004c710 con 0x7f97a00fb470 2024-05-06T12:17:46.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.580+0000 7f9795ffb700 1 -- 172.21.15.92:0/1500477231 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f979c0097a0 con 0x7f97a00fb470 2024-05-06T12:17:46.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.581+0000 7f9795ffb700 1 --2- 172.21.15.92:0/1500477231 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f978c06e270 0x7f978c070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:46.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.581+0000 7f9795ffb700 1 -- 172.21.15.92:0/1500477231 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f979c08dae0 con 0x7f97a00fb470 2024-05-06T12:17:46.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.581+0000 7f97a4d4f700 1 --2- 172.21.15.92:0/1500477231 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f978c06e270 0x7f978c070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:46.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.581+0000 7f97a4d4f700 1 --2- 172.21.15.92:0/1500477231 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f978c06e270 0x7f978c070720 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f979000b3b0 tx=0x7f9790009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:17:46.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.585+0000 7f9795ffb700 1 -- 172.21.15.92:0/1500477231 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f979c05a7f0 con 0x7f97a00fb470 2024-05-06T12:17:46.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.817+0000 7f97a77b4700 1 -- 172.21.15.92:0/1500477231 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f97a00f83e0 con 0x7f978c06e270 2024-05-06T12:17:46.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.819+0000 7f9795ffb700 1 -- 172.21.15.92:0/1500477231 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+402 (secure 0 0 0) 0x7f97a00f83e0 con 0x7f978c06e270 2024-05-06T12:17:46.822 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:17:46.822 INFO:teuthology.orchestra.run.smithi092.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb", 2024-05-06T12:17:46.822 INFO:teuthology.orchestra.run.smithi092.stdout: "in_progress": true, 2024-05-06T12:17:46.822 INFO:teuthology.orchestra.run.smithi092.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-06T12:17:46.822 INFO:teuthology.orchestra.run.smithi092.stdout: "services_complete": [ 2024-05-06T12:17:46.822 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr", 2024-05-06T12:17:46.823 INFO:teuthology.orchestra.run.smithi092.stdout: "mon", 2024-05-06T12:17:46.823 INFO:teuthology.orchestra.run.smithi092.stdout: "crash" 2024-05-06T12:17:46.823 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-05-06T12:17:46.823 INFO:teuthology.orchestra.run.smithi092.stdout: "progress": "6/21 daemons upgraded", 2024-05-06T12:17:46.823 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "Currently upgrading crash daemons", 2024-05-06T12:17:46.823 INFO:teuthology.orchestra.run.smithi092.stdout: "is_paused": false 2024-05-06T12:17:46.823 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:17:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.821+0000 7f97a77b4700 1 -- 172.21.15.92:0/1500477231 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f978c06e270 msgr2=0x7f978c070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.821+0000 7f97a77b4700 1 --2- 172.21.15.92:0/1500477231 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f978c06e270 0x7f978c070720 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f979000b3b0 tx=0x7f9790009e30 comp rx=0 tx=0).stop 2024-05-06T12:17:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.821+0000 7f97a77b4700 1 -- 172.21.15.92:0/1500477231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97a00fb470 msgr2=0x7f97a010d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.821+0000 7f97a77b4700 1 --2- 172.21.15.92:0/1500477231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97a00fb470 0x7f97a010d960 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f979c002a50 tx=0x7f979c0046b0 comp rx=0 tx=0).stop 2024-05-06T12:17:46.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.821+0000 7f97a77b4700 1 -- 172.21.15.92:0/1500477231 shutdown_connections 2024-05-06T12:17:46.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.821+0000 7f97a77b4700 1 --2- 172.21.15.92:0/1500477231 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f978c06e270 0x7f978c070720 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:46.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.821+0000 7f97a77b4700 1 --2- 172.21.15.92:0/1500477231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97a00fb470 0x7f97a010d960 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:46.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.821+0000 7f97a77b4700 1 --2- 172.21.15.92:0/1500477231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a00fc720 0x7f97a010dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:46.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.821+0000 7f97a77b4700 1 -- 172.21.15.92:0/1500477231 >> 172.21.15.92:0/1500477231 conn(0x7f97a00f6bd0 msgr2=0x7f97a00f7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:17:46.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.821+0000 7f97a77b4700 1 -- 172.21.15.92:0/1500477231 shutdown_connections 2024-05-06T12:17:46.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.821+0000 7f97a77b4700 1 -- 172.21.15.92:0/1500477231 wait complete. 2024-05-06T12:17:46.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.924+0000 7feb0bca4700 1 -- 172.21.15.92:0/1135820932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb040fcf70 msgr2=0x7feb040fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:46.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.924+0000 7feb0bca4700 1 --2- 172.21.15.92:0/1135820932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb040fcf70 0x7feb040fd3e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7feaf4004940 tx=0x7feaf4009e30 comp rx=0 tx=0).stop 2024-05-06T12:17:46.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.924+0000 7feb0bca4700 1 -- 172.21.15.92:0/1135820932 shutdown_connections 2024-05-06T12:17:46.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.924+0000 7feb0bca4700 1 --2- 172.21.15.92:0/1135820932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb040fcf70 0x7feb040fd3e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:46.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.924+0000 7feb0bca4700 1 --2- 172.21.15.92:0/1135820932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb040fbcc0 0x7feb040fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:46.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.924+0000 7feb0bca4700 1 -- 172.21.15.92:0/1135820932 >> 172.21.15.92:0/1135820932 conn(0x7feb040f7440 msgr2=0x7feb040f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:17:46.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.925+0000 7feb0bca4700 1 -- 172.21.15.92:0/1135820932 shutdown_connections 2024-05-06T12:17:46.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.925+0000 7feb0bca4700 1 -- 172.21.15.92:0/1135820932 wait complete. 2024-05-06T12:17:46.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.925+0000 7feb0bca4700 1 Processor -- start 2024-05-06T12:17:46.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.925+0000 7feb0bca4700 1 -- start start 2024-05-06T12:17:46.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.925+0000 7feb0bca4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb040fbcc0 0x7feb0410da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:46.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.925+0000 7feb0bca4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb040fcf70 0x7feb0410df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:46.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.925+0000 7feb0bca4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0410e590 con 0x7feb040fbcc0 2024-05-06T12:17:46.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.925+0000 7feb0bca4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0410e700 con 0x7feb040fcf70 2024-05-06T12:17:46.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.925+0000 7feb09a40700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb040fbcc0 0x7feb0410da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:46.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.925+0000 7feb09a40700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb040fbcc0 0x7feb0410da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36116/0 (socket says 172.21.15.92:36116) 2024-05-06T12:17:46.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.925+0000 7feb09a40700 1 -- 172.21.15.92:0/1602218221 learned_addr learned my addr 172.21.15.92:0/1602218221 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:17:46.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.926+0000 7feb0923f700 1 --2- 172.21.15.92:0/1602218221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb040fcf70 0x7feb0410df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:46.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.926+0000 7feb09a40700 1 -- 172.21.15.92:0/1602218221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb040fcf70 msgr2=0x7feb0410df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:46.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.926+0000 7feb09a40700 1 --2- 172.21.15.92:0/1602218221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb040fcf70 0x7feb0410df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:46.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.926+0000 7feb09a40700 1 -- 172.21.15.92:0/1602218221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf4005040 con 0x7feb040fbcc0 2024-05-06T12:17:46.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.926+0000 7feb09a40700 1 --2- 172.21.15.92:0/1602218221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb040fbcc0 0x7feb0410da50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7feb00002a50 tx=0x7feb000046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:17:46.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.926+0000 7feafa7fc700 1 -- 172.21.15.92:0/1602218221 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb00007cc0 con 0x7feb040fbcc0 2024-05-06T12:17:46.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.926+0000 7feafa7fc700 1 -- 172.21.15.92:0/1602218221 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb0000f040 con 0x7feb040fbcc0 2024-05-06T12:17:46.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.926+0000 7feb0bca4700 1 -- 172.21.15.92:0/1602218221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb041130c0 con 0x7feb040fbcc0 2024-05-06T12:17:46.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.926+0000 7feafa7fc700 1 -- 172.21.15.92:0/1602218221 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb0000da30 con 0x7feb040fbcc0 2024-05-06T12:17:46.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.926+0000 7feb0bca4700 1 -- 172.21.15.92:0/1602218221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb04113680 con 0x7feb040fbcc0 2024-05-06T12:17:46.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.927+0000 7feb0bca4700 1 -- 172.21.15.92:0/1602218221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb04101b00 con 0x7feb040fbcc0 2024-05-06T12:17:46.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.928+0000 7feafa7fc700 1 -- 172.21.15.92:0/1602218221 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7feb0000db90 con 0x7feb040fbcc0 2024-05-06T12:17:46.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.928+0000 7feafa7fc700 1 --2- 172.21.15.92:0/1602218221 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7feaf006e270 0x7feaf0070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:17:46.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.928+0000 7feafa7fc700 1 -- 172.21.15.92:0/1602218221 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6410+0+0 (secure 0 0 0) 0x7feb0008da80 con 0x7feb040fbcc0 2024-05-06T12:17:46.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.928+0000 7feb0923f700 1 --2- 172.21.15.92:0/1602218221 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7feaf006e270 0x7feaf0070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:17:46.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.929+0000 7feb0923f700 1 --2- 172.21.15.92:0/1602218221 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7feaf006e270 0x7feaf0070720 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7feaf400b3b0 tx=0x7feaf4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:17:46.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:46.933+0000 7feafa7fc700 1 -- 172.21.15.92:0/1602218221 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7feb0005a790 con 0x7feb040fbcc0 2024-05-06T12:17:47.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:46 smithi092 ceph-mon[91933]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:17:47.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:46 smithi092 ceph-mon[91933]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:17:47.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:46 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:47.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:46 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:47.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:46 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/4226097394' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:17:47.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:46 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/3395722207' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:17:47.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:47.220+0000 7feb0bca4700 1 -- 172.21.15.92:0/1602218221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7feb041047b0 con 0x7feb040fbcc0 2024-05-06T12:17:47.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:47.221+0000 7feafa7fc700 1 -- 172.21.15.92:0/1602218221 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7feb0005a320 con 0x7feb040fbcc0 2024-05-06T12:17:47.224 INFO:teuthology.orchestra.run.smithi092.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-06T12:17:47.224 INFO:teuthology.orchestra.run.smithi092.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-06T12:17:47.224 INFO:teuthology.orchestra.run.smithi092.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-06T12:17:47.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:47.223+0000 7feb0bca4700 1 -- 172.21.15.92:0/1602218221 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7feaf006e270 msgr2=0x7feaf0070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:47.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:47.223+0000 7feb0bca4700 1 --2- 172.21.15.92:0/1602218221 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7feaf006e270 0x7feaf0070720 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7feaf400b3b0 tx=0x7feaf4009e30 comp rx=0 tx=0).stop 2024-05-06T12:17:47.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:47.223+0000 7feb0bca4700 1 -- 172.21.15.92:0/1602218221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb040fbcc0 msgr2=0x7feb0410da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:17:47.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:47.223+0000 7feb0bca4700 1 --2- 172.21.15.92:0/1602218221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb040fbcc0 0x7feb0410da50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7feb00002a50 tx=0x7feb000046b0 comp rx=0 tx=0).stop 2024-05-06T12:17:47.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:47.223+0000 7feb0bca4700 1 -- 172.21.15.92:0/1602218221 shutdown_connections 2024-05-06T12:17:47.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:47.223+0000 7feb0bca4700 1 --2- 172.21.15.92:0/1602218221 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7feaf006e270 0x7feaf0070720 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:47.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:47.223+0000 7feb0bca4700 1 --2- 172.21.15.92:0/1602218221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb040fbcc0 0x7feb0410da50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:47.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:47.223+0000 7feb0bca4700 1 --2- 172.21.15.92:0/1602218221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb040fcf70 0x7feb0410df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:17:47.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:47.223+0000 7feb0bca4700 1 -- 172.21.15.92:0/1602218221 >> 172.21.15.92:0/1602218221 conn(0x7feb040f7440 msgr2=0x7feb040668c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:17:47.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:47.223+0000 7feb0bca4700 1 -- 172.21.15.92:0/1602218221 shutdown_connections 2024-05-06T12:17:47.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:17:47.223+0000 7feb0bca4700 1 -- 172.21.15.92:0/1602218221 wait complete. 2024-05-06T12:17:47.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:46 smithi190 ceph-mon[79744]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:17:47.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:46 smithi190 ceph-mon[79744]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:17:47.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:46 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:47.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:46 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:47.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:46 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/4226097394' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:17:47.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:46 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/3395722207' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:17:48.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:47 smithi190 ceph-mon[79744]: pgmap v37: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:17:48.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:47 smithi190 ceph-mon[79744]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:17:48.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:47 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/1602218221' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:17:48.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:47 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:48.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:47 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:48.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:47 smithi092 ceph-mon[91933]: pgmap v37: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:17:48.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:47 smithi092 ceph-mon[91933]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:17:48.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:47 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/1602218221' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:17:48.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:47 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:48.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:47 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:50.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:49 smithi190 ceph-mon[79744]: pgmap v38: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:17:50.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:50.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:50.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:50.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:17:50.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:50.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:17:50.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:17:50.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:17:50.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:17:50.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:50.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi092"}]: dispatch 2024-05-06T12:17:50.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi092"}]': finished 2024-05-06T12:17:50.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi190"}]: dispatch 2024-05-06T12:17:50.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi190"}]': finished 2024-05-06T12:17:50.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-05-06T12:17:50.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:49 smithi092 ceph-mon[91933]: pgmap v38: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:17:50.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:50.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:50.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:50.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:17:50.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:50.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:17:50.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:17:50.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:17:50.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:17:50.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:50.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi092"}]: dispatch 2024-05-06T12:17:50.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi092"}]': finished 2024-05-06T12:17:50.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi190"}]: dispatch 2024-05-06T12:17:50.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi190"}]': finished 2024-05-06T12:17:50.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-05-06T12:17:51.049 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:50 smithi092 ceph-mon[91933]: Upgrade: Setting container_image for all crash 2024-05-06T12:17:51.049 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:50 smithi092 ceph-mon[91933]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-05-06T12:17:51.049 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:50 smithi092 ceph-mon[91933]: Upgrade: osd.0 is safe to restart 2024-05-06T12:17:51.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:50 smithi190 ceph-mon[79744]: Upgrade: Setting container_image for all crash 2024-05-06T12:17:51.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:50 smithi190 ceph-mon[79744]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-05-06T12:17:51.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:50 smithi190 ceph-mon[79744]: Upgrade: osd.0 is safe to restart 2024-05-06T12:17:52.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:51 smithi092 ceph-mon[91933]: pgmap v39: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:17:52.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:51 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:52.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:51 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-06T12:17:52.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:51 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:52.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:51 smithi190 ceph-mon[79744]: pgmap v39: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:17:52.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:51 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:17:52.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:51 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-06T12:17:52.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:51 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:17:53.209 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:52 smithi092 ceph-mon[91933]: Upgrade: Updating osd.0 2024-05-06T12:17:53.209 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:52 smithi092 ceph-mon[91933]: Deploying daemon osd.0 on smithi092 2024-05-06T12:17:53.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:52 smithi190 ceph-mon[79744]: Upgrade: Updating osd.0 2024-05-06T12:17:53.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:52 smithi190 ceph-mon[79744]: Deploying daemon osd.0 on smithi092 2024-05-06T12:17:54.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:53 smithi092 ceph-mon[91933]: pgmap v40: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:17:54.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:53 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:17:54.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:53 smithi190 ceph-mon[79744]: pgmap v40: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:17:54.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:53 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:17:54.585 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:54 smithi092 systemd[1]: Stopping Ceph osd.0 for aee1d912-0ba0-11ef-bc97-c7b262605968... 2024-05-06T12:17:54.959 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:54 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0[45502]: 2024-05-06T12:17:54.581+0000 7efde2b08700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-06T12:17:54.959 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:54 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0[45502]: 2024-05-06T12:17:54.581+0000 7efde2b08700 -1 osd.0 50 *** Got signal Terminated *** 2024-05-06T12:17:54.959 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:54 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0[45502]: 2024-05-06T12:17:54.581+0000 7efde2b08700 -1 osd.0 50 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-06T12:17:55.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:54 smithi092 ceph-mon[91933]: osd.0 marked itself down and dead 2024-05-06T12:17:55.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:54 smithi190 ceph-mon[79744]: osd.0 marked itself down and dead 2024-05-06T12:17:55.652 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:55 smithi092 podman[99944]: 2024-05-06 12:17:55.387578561 +0000 UTC m=+0.917144815 container died ce84e7af9084288ff7492fd989940829f96cb9b394a44d3fe155f87581dbc2fe (image=quay.ceph.io/ceph-ci/ceph@sha256:1a0b7fd3cefd2c5baddf682493f92b4e27596f3c0ff27b02fdccd2891f56859f, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-e81df42, io.buildah.version=1.33.5) 2024-05-06T12:17:55.960 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:55 smithi092 podman[99944]: 2024-05-06 12:17:55.748718233 +0000 UTC m=+1.278284480 container cleanup ce84e7af9084288ff7492fd989940829f96cb9b394a44d3fe155f87581dbc2fe (image=quay.ceph.io/ceph-ci/ceph@sha256:1a0b7fd3cefd2c5baddf682493f92b4e27596f3c0ff27b02fdccd2891f56859f, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-e81df42, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240430) 2024-05-06T12:17:55.960 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:55 smithi092 bash[99944]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0 2024-05-06T12:17:56.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:55 smithi092 ceph-mon[91933]: pgmap v41: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:17:56.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:55 smithi092 ceph-mon[91933]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-06T12:17:56.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:55 smithi092 ceph-mon[91933]: osdmap e51: 6 total, 5 up, 6 in 2024-05-06T12:17:56.224 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:56 smithi092 podman[99961]: 2024-05-06 12:17:56.008568897 +0000 UTC m=+0.623235192 container remove ce84e7af9084288ff7492fd989940829f96cb9b394a44d3fe155f87581dbc2fe (image=quay.ceph.io/ceph-ci/ceph@sha256:1a0b7fd3cefd2c5baddf682493f92b4e27596f3c0ff27b02fdccd2891f56859f, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=quincy-e81df42, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-06T12:17:56.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:55 smithi190 ceph-mon[79744]: pgmap v41: 65 pgs: 65 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:17:56.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:55 smithi190 ceph-mon[79744]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-06T12:17:56.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:55 smithi190 ceph-mon[79744]: osdmap e51: 6 total, 5 up, 6 in 2024-05-06T12:17:56.623 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:56 smithi092 podman[100054]: 2024-05-06 12:17:56.351196959 +0000 UTC m=+0.110486833 container create 8dcdfb8cc0eb70bfa0a2acc79eaae1e67cf83f37a4777e8dddacdad14c8f5dea (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-70ce953, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-06T12:17:56.624 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:56 smithi092 podman[100054]: 2024-05-06 12:17:56.258639553 +0000 UTC m=+0.017929424 image pull e4ce8ad5d0c4b9ea60bc5f6b8b0fdb5efbe59bf7771fb0920fc2200e6ae546b0 quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb 2024-05-06T12:17:56.922 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:56 smithi092 podman[100054]: 2024-05-06 12:17:56.620546696 +0000 UTC m=+0.379836560 container init 8dcdfb8cc0eb70bfa0a2acc79eaae1e67cf83f37a4777e8dddacdad14c8f5dea (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-70ce953, org.label-schema.vendor=CentOS) 2024-05-06T12:17:56.923 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:56 smithi092 podman[100054]: 2024-05-06 12:17:56.624893243 +0000 UTC m=+0.384183112 container start 8dcdfb8cc0eb70bfa0a2acc79eaae1e67cf83f37a4777e8dddacdad14c8f5dea (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-70ce953, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=) 2024-05-06T12:17:56.923 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:56 smithi092 podman[100054]: 2024-05-06 12:17:56.728828157 +0000 UTC m=+0.488118033 container attach 8dcdfb8cc0eb70bfa0a2acc79eaae1e67cf83f37a4777e8dddacdad14c8f5dea (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-70ce953, io.buildah.version=1.33.5) 2024-05-06T12:17:56.923 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:56 smithi092 podman[100054]: 2024-05-06 12:17:56.920268879 +0000 UTC m=+0.679558736 container died 8dcdfb8cc0eb70bfa0a2acc79eaae1e67cf83f37a4777e8dddacdad14c8f5dea (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0-deactivate, RELEASE=squid-70ce953, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-06T12:17:57.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:56 smithi092 ceph-mon[91933]: osdmap e52: 6 total, 5 up, 6 in 2024-05-06T12:17:57.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:56 smithi190 ceph-mon[79744]: osdmap e52: 6 total, 5 up, 6 in 2024-05-06T12:17:57.919 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:57 smithi092 podman[100080]: 2024-05-06 12:17:57.482807268 +0000 UTC m=+0.557303927 container remove 8dcdfb8cc0eb70bfa0a2acc79eaae1e67cf83f37a4777e8dddacdad14c8f5dea (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-70ce953) 2024-05-06T12:17:57.919 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:57 smithi092 systemd[1]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@osd.0.service: Deactivated successfully. 2024-05-06T12:17:57.919 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:57 smithi092 systemd[1]: Stopped Ceph osd.0 for aee1d912-0ba0-11ef-bc97-c7b262605968. 2024-05-06T12:17:57.919 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:57 smithi092 systemd[1]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@osd.0.service: Consumed 49.912s CPU time. 2024-05-06T12:17:57.919 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:57 smithi092 systemd[1]: Starting Ceph osd.0 for aee1d912-0ba0-11ef-bc97-c7b262605968... 2024-05-06T12:17:58.180 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:57 smithi092 ceph-mon[91933]: pgmap v44: 65 pgs: 9 stale+active+clean, 56 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:17:58.180 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:57 smithi092 podman[100179]: 2024-05-06 12:17:57.819201303 +0000 UTC m=+0.017896672 image pull e4ce8ad5d0c4b9ea60bc5f6b8b0fdb5efbe59bf7771fb0920fc2200e6ae546b0 quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb 2024-05-06T12:17:58.180 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:57 smithi092 podman[100179]: 2024-05-06 12:17:57.942059904 +0000 UTC m=+0.140755263 container create efc3eca327db9e9736adf0cce28e16b64230f159318cc59efa9e9042c869b5bb (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0-activate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-70ce953, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-06T12:17:58.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:57 smithi190 ceph-mon[79744]: pgmap v44: 65 pgs: 9 stale+active+clean, 56 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:17:58.473 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:58 smithi092 podman[100179]: 2024-05-06 12:17:58.178407807 +0000 UTC m=+0.377103171 container init efc3eca327db9e9736adf0cce28e16b64230f159318cc59efa9e9042c869b5bb (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0-activate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-70ce953, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-06T12:17:58.473 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:58 smithi092 podman[100179]: 2024-05-06 12:17:58.181696159 +0000 UTC m=+0.380391521 container start efc3eca327db9e9736adf0cce28e16b64230f159318cc59efa9e9042c869b5bb (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0-activate, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-70ce953, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-06T12:17:58.473 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:58 smithi092 podman[100179]: 2024-05-06 12:17:58.270068039 +0000 UTC m=+0.468763413 container attach efc3eca327db9e9736adf0cce28e16b64230f159318cc59efa9e9042c869b5bb (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0-activate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-70ce953, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=) 2024-05-06T12:17:58.973 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:58 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0-activate[100194]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-06T12:17:58.973 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:58 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0-activate[100194]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-05-06T12:17:58.973 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:58 smithi092 bash[100179]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-06T12:17:58.973 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:58 smithi092 bash[100179]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-05-06T12:17:58.973 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:58 smithi092 bash[100179]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-05-06T12:17:58.974 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:58 smithi092 bash[100179]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-05-06T12:17:58.974 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:58 smithi092 bash[100179]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-05-06T12:17:58.974 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:58 smithi092 bash[100179]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-06T12:17:58.974 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:58 smithi092 bash[100179]: --> ceph-volume raw activate successful for osd ID: 0 2024-05-06T12:17:58.974 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:58 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0-activate[100194]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-05-06T12:17:58.974 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:58 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0-activate[100194]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-05-06T12:17:58.974 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:58 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0-activate[100194]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-05-06T12:17:58.974 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:58 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0-activate[100194]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-06T12:17:58.974 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:58 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0-activate[100194]: --> ceph-volume raw activate successful for osd ID: 0 2024-05-06T12:17:58.974 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:58 smithi092 podman[100179]: 2024-05-06 12:17:58.648429946 +0000 UTC m=+0.847125299 container died efc3eca327db9e9736adf0cce28e16b64230f159318cc59efa9e9042c869b5bb (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0-activate, GIT_CLEAN=True, org.label-schema.build-date=20240430, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-70ce953, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-06T12:17:59.519 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:59 smithi092 podman[100317]: 2024-05-06 12:17:59.265891208 +0000 UTC m=+0.607956541 container remove efc3eca327db9e9736adf0cce28e16b64230f159318cc59efa9e9042c869b5bb (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0-activate, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-70ce953, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-06T12:17:59.519 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:59 smithi092 podman[100376]: 2024-05-06 12:17:59.382168222 +0000 UTC m=+0.017962422 image pull e4ce8ad5d0c4b9ea60bc5f6b8b0fdb5efbe59bf7771fb0920fc2200e6ae546b0 quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb 2024-05-06T12:17:59.519 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:59 smithi092 podman[100376]: 2024-05-06 12:17:59.516704321 +0000 UTC m=+0.152498490 container create 8c96c75b25f48f162b86e1fee897eff5f25b6f588fef97aeb12e05a1084df8aa (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-70ce953, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux ) 2024-05-06T12:17:59.778 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:59 smithi092 podman[100376]: 2024-05-06 12:17:59.752840223 +0000 UTC m=+0.388634396 container init 8c96c75b25f48f162b86e1fee897eff5f25b6f588fef97aeb12e05a1084df8aa (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-70ce953, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-06T12:17:59.778 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:59 smithi092 podman[100376]: 2024-05-06 12:17:59.75632414 +0000 UTC m=+0.392118318 container start 8c96c75b25f48f162b86e1fee897eff5f25b6f588fef97aeb12e05a1084df8aa (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, RELEASE=squid-70ce953, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-06T12:18:00.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:59 smithi092 ceph-mon[91933]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-06T12:18:00.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:59 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:00.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:59 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:00.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:17:59 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:18:00.223 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:59 smithi092 ceph-osd[100395]: -- 172.21.15.92:0/130563366 <== mon.0 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d6a45c1680 con 0x55d6a458ac00 2024-05-06T12:18:00.223 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:59 smithi092 bash[100376]: 8c96c75b25f48f162b86e1fee897eff5f25b6f588fef97aeb12e05a1084df8aa 2024-05-06T12:18:00.223 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:17:59 smithi092 systemd[1]: Started Ceph osd.0 for aee1d912-0ba0-11ef-bc97-c7b262605968. 2024-05-06T12:18:00.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:59 smithi190 ceph-mon[79744]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-06T12:18:00.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:59 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:00.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:59 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:00.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:17:59 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:18:00.723 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:18:00 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0[100391]: 2024-05-06T12:18:00.297+0000 7f56367697c0 -1 Falling back to public interface 2024-05-06T12:18:01.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:01 smithi190 ceph-mon[79744]: pgmap v46: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 73/372 objects degraded (19.624%) 2024-05-06T12:18:01.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:01 smithi190 ceph-mon[79744]: Health check failed: Degraded data redundancy: 73/372 objects degraded (19.624%), 20 pgs degraded (PG_DEGRADED) 2024-05-06T12:18:01.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:01 smithi092 ceph-mon[91933]: pgmap v46: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 73/372 objects degraded (19.624%) 2024-05-06T12:18:01.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:01 smithi092 ceph-mon[91933]: Health check failed: Degraded data redundancy: 73/372 objects degraded (19.624%), 20 pgs degraded (PG_DEGRADED) 2024-05-06T12:18:04.139 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:03 smithi092 ceph-mon[91933]: pgmap v47: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 73/372 objects degraded (19.624%) 2024-05-06T12:18:04.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:03 smithi190 ceph-mon[79744]: pgmap v47: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 73/372 objects degraded (19.624%) 2024-05-06T12:18:04.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:04 smithi092 ceph-mon[91933]: from='osd.0 [v2:172.21.15.92:6802/3125171617,v1:172.21.15.92:6803/3125171617]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-06T12:18:04.973 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:18:04 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0[100391]: 2024-05-06T12:18:04.749+0000 7f56367697c0 -1 osd.0 50 log_to_monitors true 2024-05-06T12:18:04.973 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:18:04 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0[100391]: 2024-05-06T12:18:04.879+0000 7f5627bc0700 -1 osd.0 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-06T12:18:05.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:04 smithi190 ceph-mon[79744]: from='osd.0 [v2:172.21.15.92:6802/3125171617,v1:172.21.15.92:6803/3125171617]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-06T12:18:06.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:05 smithi092 ceph-mon[91933]: pgmap v48: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 73/372 objects degraded (19.624%) 2024-05-06T12:18:06.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:05 smithi092 ceph-mon[91933]: from='osd.0 [v2:172.21.15.92:6802/3125171617,v1:172.21.15.92:6803/3125171617]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-06T12:18:06.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:05 smithi092 ceph-mon[91933]: osdmap e53: 6 total, 5 up, 6 in 2024-05-06T12:18:06.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:05 smithi092 ceph-mon[91933]: from='osd.0 [v2:172.21.15.92:6802/3125171617,v1:172.21.15.92:6803/3125171617]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-06T12:18:06.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:05 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:06.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:05 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:06.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:05 smithi190 ceph-mon[79744]: pgmap v48: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 323 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 73/372 objects degraded (19.624%) 2024-05-06T12:18:06.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:05 smithi190 ceph-mon[79744]: from='osd.0 [v2:172.21.15.92:6802/3125171617,v1:172.21.15.92:6803/3125171617]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-06T12:18:06.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:05 smithi190 ceph-mon[79744]: osdmap e53: 6 total, 5 up, 6 in 2024-05-06T12:18:06.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:05 smithi190 ceph-mon[79744]: from='osd.0 [v2:172.21.15.92:6802/3125171617,v1:172.21.15.92:6803/3125171617]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-06T12:18:06.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:05 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:06.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:05 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:07.161 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:06 smithi092 ceph-mon[91933]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-06T12:18:07.161 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:06 smithi092 ceph-mon[91933]: osd.0 [v2:172.21.15.92:6802/3125171617,v1:172.21.15.92:6803/3125171617] boot 2024-05-06T12:18:07.161 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:06 smithi092 ceph-mon[91933]: osdmap e54: 6 total, 6 up, 6 in 2024-05-06T12:18:07.162 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:06 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-06T12:18:07.163 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:06 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:07.163 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:06 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:07.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:06 smithi190 ceph-mon[79744]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-06T12:18:07.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:06 smithi190 ceph-mon[79744]: osd.0 [v2:172.21.15.92:6802/3125171617,v1:172.21.15.92:6803/3125171617] boot 2024-05-06T12:18:07.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:06 smithi190 ceph-mon[79744]: osdmap e54: 6 total, 6 up, 6 in 2024-05-06T12:18:07.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:06 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-06T12:18:07.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:06 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:07.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:06 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:08.208 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:07 smithi092 ceph-mon[91933]: pgmap v51: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 73/372 objects degraded (19.624%) 2024-05-06T12:18:08.208 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:07 smithi092 ceph-mon[91933]: osdmap e55: 6 total, 6 up, 6 in 2024-05-06T12:18:08.250 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:07 smithi190 ceph-mon[79744]: pgmap v51: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 73/372 objects degraded (19.624%) 2024-05-06T12:18:08.250 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:07 smithi190 ceph-mon[79744]: osdmap e55: 6 total, 6 up, 6 in 2024-05-06T12:18:10.208 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:09 smithi092 ceph-mon[91933]: pgmap v53: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 73/372 objects degraded (19.624%) 2024-05-06T12:18:10.208 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:10.208 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:18:10.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:09 smithi190 ceph-mon[79744]: pgmap v53: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 73/372 objects degraded (19.624%) 2024-05-06T12:18:10.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:10.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:18:11.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:11 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:11.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:11 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:11.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:11 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:18:11.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:11 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:18:11.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:11 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:11.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:11 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:18:11.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:11 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:18:11.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:11 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:18:11.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:11 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:18:11.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:11 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-06T12:18:11.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:11 smithi190 ceph-mon[79744]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-06T12:18:11.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:11 smithi190 ceph-mon[79744]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-05-06T12:18:11.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:11 smithi190 ceph-mon[79744]: pgmap v54: 65 pgs: 65 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-05-06T12:18:11.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:11 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:11.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:11 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:11.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:11 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:18:11.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:11 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:18:11.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:11 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:11.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:11 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:18:11.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:11 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:18:11.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:11 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:18:11.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:11 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:18:11.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:11 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-06T12:18:11.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:11 smithi092 ceph-mon[91933]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-06T12:18:11.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:11 smithi092 ceph-mon[91933]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-05-06T12:18:11.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:11 smithi092 ceph-mon[91933]: pgmap v54: 65 pgs: 65 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-05-06T12:18:12.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:12 smithi190 ceph-mon[79744]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 73/372 objects degraded (19.624%), 20 pgs degraded) 2024-05-06T12:18:12.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:12 smithi190 ceph-mon[79744]: mgrmap e39: smithi092.gyzeqb(active, since 92s), standbys: smithi190.kczxkh 2024-05-06T12:18:12.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:12 smithi092 ceph-mon[91933]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 73/372 objects degraded (19.624%), 20 pgs degraded) 2024-05-06T12:18:12.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:12 smithi092 ceph-mon[91933]: mgrmap e39: smithi092.gyzeqb(active, since 92s), standbys: smithi190.kczxkh 2024-05-06T12:18:13.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:13 smithi190 ceph-mon[79744]: pgmap v55: 65 pgs: 65 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 773 B/s wr, 1 op/s 2024-05-06T12:18:13.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:13 smithi092 ceph-mon[91933]: pgmap v55: 65 pgs: 65 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 773 B/s wr, 1 op/s 2024-05-06T12:18:16.175 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:15 smithi092 ceph-mon[91933]: pgmap v56: 65 pgs: 65 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.2 KiB/s rd, 687 B/s wr, 3 op/s 2024-05-06T12:18:16.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:15 smithi190 ceph-mon[79744]: pgmap v56: 65 pgs: 65 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.2 KiB/s rd, 687 B/s wr, 3 op/s 2024-05-06T12:18:17.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.335+0000 7fae46e29700 1 -- 172.21.15.92:0/3163563272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae400fcf70 msgr2=0x7fae400fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:17.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.335+0000 7fae46e29700 1 --2- 172.21.15.92:0/3163563272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae400fcf70 0x7fae400fd3e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fae30004940 tx=0x7fae30009e30 comp rx=0 tx=0).stop 2024-05-06T12:18:17.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.335+0000 7fae46e29700 1 -- 172.21.15.92:0/3163563272 shutdown_connections 2024-05-06T12:18:17.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.335+0000 7fae46e29700 1 --2- 172.21.15.92:0/3163563272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae400fcf70 0x7fae400fd3e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:17.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.335+0000 7fae46e29700 1 --2- 172.21.15.92:0/3163563272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae400fbcc0 0x7fae400fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:17.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.335+0000 7fae46e29700 1 -- 172.21.15.92:0/3163563272 >> 172.21.15.92:0/3163563272 conn(0x7fae400f7440 msgr2=0x7fae400f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:18:17.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.335+0000 7fae46e29700 1 -- 172.21.15.92:0/3163563272 shutdown_connections 2024-05-06T12:18:17.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.335+0000 7fae46e29700 1 -- 172.21.15.92:0/3163563272 wait complete. 2024-05-06T12:18:17.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.336+0000 7fae46e29700 1 Processor -- start 2024-05-06T12:18:17.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.336+0000 7fae46e29700 1 -- start start 2024-05-06T12:18:17.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.336+0000 7fae46e29700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae400fbcc0 0x7fae400ffb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:17.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.336+0000 7fae46e29700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae400fcf70 0x7fae400fe2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:17.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.336+0000 7fae46e29700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae40100160 con 0x7fae400fbcc0 2024-05-06T12:18:17.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.336+0000 7fae46e29700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae400fe7f0 con 0x7fae400fcf70 2024-05-06T12:18:17.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.336+0000 7fae44bc5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae400fbcc0 0x7fae400ffb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:17.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.336+0000 7fae44bc5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae400fbcc0 0x7fae400ffb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34924/0 (socket says 172.21.15.92:34924) 2024-05-06T12:18:17.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.336+0000 7fae44bc5700 1 -- 172.21.15.92:0/2388592375 learned_addr learned my addr 172.21.15.92:0/2388592375 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:18:17.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.336+0000 7fae3ffff700 1 --2- 172.21.15.92:0/2388592375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae400fcf70 0x7fae400fe2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:17.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.337+0000 7fae44bc5700 1 -- 172.21.15.92:0/2388592375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae400fcf70 msgr2=0x7fae400fe2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:17.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.337+0000 7fae44bc5700 1 --2- 172.21.15.92:0/2388592375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae400fcf70 0x7fae400fe2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:17.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.337+0000 7fae44bc5700 1 -- 172.21.15.92:0/2388592375 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae30005040 con 0x7fae400fbcc0 2024-05-06T12:18:17.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.337+0000 7fae44bc5700 1 --2- 172.21.15.92:0/2388592375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae400fbcc0 0x7fae400ffb60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fae38001d80 tx=0x7fae38009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:18:17.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.337+0000 7fae3d7fa700 1 -- 172.21.15.92:0/2388592375 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae38007d10 con 0x7fae400fbcc0 2024-05-06T12:18:17.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.337+0000 7fae3d7fa700 1 -- 172.21.15.92:0/2388592375 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fae38001e80 con 0x7fae400fbcc0 2024-05-06T12:18:17.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.337+0000 7fae46e29700 1 -- 172.21.15.92:0/2388592375 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae400fead0 con 0x7fae400fbcc0 2024-05-06T12:18:17.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.337+0000 7fae3d7fa700 1 -- 172.21.15.92:0/2388592375 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae38014640 con 0x7fae400fbcc0 2024-05-06T12:18:17.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.337+0000 7fae46e29700 1 -- 172.21.15.92:0/2388592375 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae400ff090 con 0x7fae400fbcc0 2024-05-06T12:18:17.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.338+0000 7fae46e29700 1 -- 172.21.15.92:0/2388592375 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae4004c710 con 0x7fae400fbcc0 2024-05-06T12:18:17.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.338+0000 7fae3d7fa700 1 -- 172.21.15.92:0/2388592375 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fae3800e960 con 0x7fae400fbcc0 2024-05-06T12:18:17.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.339+0000 7fae3d7fa700 1 --2- 172.21.15.92:0/2388592375 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fae2c06e140 0x7fae2c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:17.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.339+0000 7fae3d7fa700 1 -- 172.21.15.92:0/2388592375 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fae3808d900 con 0x7fae400fbcc0 2024-05-06T12:18:17.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.339+0000 7fae3ffff700 1 --2- 172.21.15.92:0/2388592375 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fae2c06e140 0x7fae2c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:17.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.339+0000 7fae3ffff700 1 --2- 172.21.15.92:0/2388592375 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fae2c06e140 0x7fae2c0705f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fae3000b970 tx=0x7fae30009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:18:17.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.344+0000 7fae3d7fa700 1 -- 172.21.15.92:0/2388592375 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fae3805a800 con 0x7fae400fbcc0 2024-05-06T12:18:17.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.561+0000 7fae46e29700 1 -- 172.21.15.92:0/2388592375 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fae4002b340 con 0x7fae2c06e140 2024-05-06T12:18:17.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.562+0000 7fae3d7fa700 1 -- 172.21.15.92:0/2388592375 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fae4002b340 con 0x7fae2c06e140 2024-05-06T12:18:17.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.564+0000 7fae46e29700 1 -- 172.21.15.92:0/2388592375 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fae2c06e140 msgr2=0x7fae2c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:17.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.564+0000 7fae46e29700 1 --2- 172.21.15.92:0/2388592375 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fae2c06e140 0x7fae2c0705f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fae3000b970 tx=0x7fae30009e30 comp rx=0 tx=0).stop 2024-05-06T12:18:17.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.564+0000 7fae46e29700 1 -- 172.21.15.92:0/2388592375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae400fbcc0 msgr2=0x7fae400ffb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:17.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.564+0000 7fae46e29700 1 --2- 172.21.15.92:0/2388592375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae400fbcc0 0x7fae400ffb60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fae38001d80 tx=0x7fae38009ec0 comp rx=0 tx=0).stop 2024-05-06T12:18:17.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.564+0000 7fae46e29700 1 -- 172.21.15.92:0/2388592375 shutdown_connections 2024-05-06T12:18:17.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.564+0000 7fae46e29700 1 --2- 172.21.15.92:0/2388592375 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fae2c06e140 0x7fae2c0705f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:17.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.564+0000 7fae46e29700 1 --2- 172.21.15.92:0/2388592375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae400fbcc0 0x7fae400ffb60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:17.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.564+0000 7fae46e29700 1 --2- 172.21.15.92:0/2388592375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae400fcf70 0x7fae400fe2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:17.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.564+0000 7fae46e29700 1 -- 172.21.15.92:0/2388592375 >> 172.21.15.92:0/2388592375 conn(0x7fae400f7440 msgr2=0x7fae40066840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:18:17.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.564+0000 7fae46e29700 1 -- 172.21.15.92:0/2388592375 shutdown_connections 2024-05-06T12:18:17.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.564+0000 7fae46e29700 1 -- 172.21.15.92:0/2388592375 wait complete. 2024-05-06T12:18:17.577 INFO:teuthology.orchestra.run.smithi092.stdout:true 2024-05-06T12:18:17.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.666+0000 7efca9b60700 1 -- 172.21.15.92:0/2399078063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efca40fbcb0 msgr2=0x7efca40fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:17.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.666+0000 7efca9b60700 1 --2- 172.21.15.92:0/2399078063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efca40fbcb0 0x7efca40fc0c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efc98004940 tx=0x7efc98009e30 comp rx=0 tx=0).stop 2024-05-06T12:18:17.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.666+0000 7efca9b60700 1 -- 172.21.15.92:0/2399078063 shutdown_connections 2024-05-06T12:18:17.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.666+0000 7efca9b60700 1 --2- 172.21.15.92:0/2399078063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efca40fcf60 0x7efca40fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:17.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.666+0000 7efca9b60700 1 --2- 172.21.15.92:0/2399078063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efca40fbcb0 0x7efca40fc0c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:17.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.666+0000 7efca9b60700 1 -- 172.21.15.92:0/2399078063 >> 172.21.15.92:0/2399078063 conn(0x7efca40f7430 msgr2=0x7efca40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:18:17.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.666+0000 7efca9b60700 1 -- 172.21.15.92:0/2399078063 shutdown_connections 2024-05-06T12:18:17.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.666+0000 7efca9b60700 1 -- 172.21.15.92:0/2399078063 wait complete. 2024-05-06T12:18:17.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.666+0000 7efca9b60700 1 Processor -- start 2024-05-06T12:18:17.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.667+0000 7efca9b60700 1 -- start start 2024-05-06T12:18:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.667+0000 7efca9b60700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efca40fbcb0 0x7efca41106e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.667+0000 7efca9b60700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efca40fcf60 0x7efca4112c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.667+0000 7efca9b60700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca40662e0 con 0x7efca40fbcb0 2024-05-06T12:18:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.667+0000 7efca9b60700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca4066420 con 0x7efca40fcf60 2024-05-06T12:18:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.667+0000 7efca37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efca40fbcb0 0x7efca41106e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.667+0000 7efca37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efca40fbcb0 0x7efca41106e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34934/0 (socket says 172.21.15.92:34934) 2024-05-06T12:18:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.667+0000 7efca37fe700 1 -- 172.21.15.92:0/971720633 learned_addr learned my addr 172.21.15.92:0/971720633 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:18:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.667+0000 7efca2ffd700 1 --2- 172.21.15.92:0/971720633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efca40fcf60 0x7efca4112c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.667+0000 7efca37fe700 1 -- 172.21.15.92:0/971720633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efca40fcf60 msgr2=0x7efca4112c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.667+0000 7efca37fe700 1 --2- 172.21.15.92:0/971720633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efca40fcf60 0x7efca4112c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.667+0000 7efca37fe700 1 -- 172.21.15.92:0/971720633 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc98005040 con 0x7efca40fbcb0 2024-05-06T12:18:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.668+0000 7efca37fe700 1 --2- 172.21.15.92:0/971720633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efca40fbcb0 0x7efca41106e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efc98013870 tx=0x7efc98009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:18:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.668+0000 7efc8bfff700 1 -- 172.21.15.92:0/971720633 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc98021070 con 0x7efca40fbcb0 2024-05-06T12:18:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.668+0000 7efc8bfff700 1 -- 172.21.15.92:0/971720633 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc98009430 con 0x7efca40fbcb0 2024-05-06T12:18:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.668+0000 7efca9b60700 1 -- 172.21.15.92:0/971720633 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efca4113170 con 0x7efca40fbcb0 2024-05-06T12:18:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.668+0000 7efc8bfff700 1 -- 172.21.15.92:0/971720633 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc9800e070 con 0x7efca40fbcb0 2024-05-06T12:18:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.668+0000 7efca9b60700 1 -- 172.21.15.92:0/971720633 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca4113660 con 0x7efca40fbcb0 2024-05-06T12:18:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.669+0000 7efca9b60700 1 -- 172.21.15.92:0/971720633 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efca405f5c0 con 0x7efca40fbcb0 2024-05-06T12:18:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.669+0000 7efc8bfff700 1 -- 172.21.15.92:0/971720633 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7efc9801c750 con 0x7efca40fbcb0 2024-05-06T12:18:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.670+0000 7efc8bfff700 1 --2- 172.21.15.92:0/971720633 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7efc8c06e140 0x7efc8c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:17.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.670+0000 7efca2ffd700 1 --2- 172.21.15.92:0/971720633 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7efc8c06e140 0x7efc8c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:17.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.670+0000 7efc8bfff700 1 -- 172.21.15.92:0/971720633 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6410+0+0 (secure 0 0 0) 0x7efc9808eba0 con 0x7efca40fbcb0 2024-05-06T12:18:17.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.670+0000 7efca2ffd700 1 --2- 172.21.15.92:0/971720633 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7efc8c06e140 0x7efc8c0705f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efc94009b80 tx=0x7efc9400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:18:17.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.674+0000 7efc8bfff700 1 -- 172.21.15.92:0/971720633 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7efc98058040 con 0x7efca40fbcb0 2024-05-06T12:18:17.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.892+0000 7efca9b60700 1 -- 172.21.15.92:0/971720633 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efca405f5c0 con 0x7efc8c06e140 2024-05-06T12:18:17.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.893+0000 7efc8bfff700 1 -- 172.21.15.92:0/971720633 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7efca405f5c0 con 0x7efc8c06e140 2024-05-06T12:18:17.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.895+0000 7efca9b60700 1 -- 172.21.15.92:0/971720633 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7efc8c06e140 msgr2=0x7efc8c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:17.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.895+0000 7efca9b60700 1 --2- 172.21.15.92:0/971720633 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7efc8c06e140 0x7efc8c0705f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efc94009b80 tx=0x7efc9400b040 comp rx=0 tx=0).stop 2024-05-06T12:18:17.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.895+0000 7efca9b60700 1 -- 172.21.15.92:0/971720633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efca40fbcb0 msgr2=0x7efca41106e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:17.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.895+0000 7efca9b60700 1 --2- 172.21.15.92:0/971720633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efca40fbcb0 0x7efca41106e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efc98013870 tx=0x7efc98009e30 comp rx=0 tx=0).stop 2024-05-06T12:18:17.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.895+0000 7efca9b60700 1 -- 172.21.15.92:0/971720633 shutdown_connections 2024-05-06T12:18:17.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.895+0000 7efca9b60700 1 --2- 172.21.15.92:0/971720633 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7efc8c06e140 0x7efc8c0705f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:17.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.895+0000 7efca9b60700 1 --2- 172.21.15.92:0/971720633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efca40fbcb0 0x7efca41106e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:17.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.895+0000 7efca9b60700 1 --2- 172.21.15.92:0/971720633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efca40fcf60 0x7efca4112c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:17.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.895+0000 7efca9b60700 1 -- 172.21.15.92:0/971720633 >> 172.21.15.92:0/971720633 conn(0x7efca40f7430 msgr2=0x7efca40f8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:18:17.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.895+0000 7efca9b60700 1 -- 172.21.15.92:0/971720633 shutdown_connections 2024-05-06T12:18:17.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.895+0000 7efca9b60700 1 -- 172.21.15.92:0/971720633 wait complete. 2024-05-06T12:18:18.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.996+0000 7f02bc88a700 1 -- 172.21.15.92:0/1888277117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02b80fce60 msgr2=0x7f02b80fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:18.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.997+0000 7f02bc88a700 1 --2- 172.21.15.92:0/1888277117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02b80fce60 0x7f02b80fd270 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f02a8004940 tx=0x7f02a8009e30 comp rx=0 tx=0).stop 2024-05-06T12:18:18.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.997+0000 7f02bc88a700 1 -- 172.21.15.92:0/1888277117 shutdown_connections 2024-05-06T12:18:18.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.997+0000 7f02bc88a700 1 --2- 172.21.15.92:0/1888277117 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02b80fdfd0 0x7f02b80fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:18.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.997+0000 7f02bc88a700 1 --2- 172.21.15.92:0/1888277117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02b80fce60 0x7f02b80fd270 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:18.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.997+0000 7f02bc88a700 1 -- 172.21.15.92:0/1888277117 >> 172.21.15.92:0/1888277117 conn(0x7f02b80f8540 msgr2=0x7f02b80fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:18:18.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.997+0000 7f02bc88a700 1 -- 172.21.15.92:0/1888277117 shutdown_connections 2024-05-06T12:18:18.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.997+0000 7f02bc88a700 1 -- 172.21.15.92:0/1888277117 wait complete. 2024-05-06T12:18:18.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.997+0000 7f02bc88a700 1 Processor -- start 2024-05-06T12:18:18.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.997+0000 7f02bc88a700 1 -- start start 2024-05-06T12:18:18.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.997+0000 7f02bc88a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02b80fce60 0x7f02b810d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:18.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.997+0000 7f02bc88a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02b80fdfd0 0x7f02b810ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:18.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.997+0000 7f02bc88a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b810e3f0 con 0x7f02b80fdfd0 2024-05-06T12:18:18.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.997+0000 7f02bc88a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b810e560 con 0x7f02b80fce60 2024-05-06T12:18:18.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.998+0000 7f02b5d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02b80fdfd0 0x7f02b810ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:18.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.998+0000 7f02b5d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02b80fdfd0 0x7f02b810ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34956/0 (socket says 172.21.15.92:34956) 2024-05-06T12:18:18.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.998+0000 7f02b5d9b700 1 -- 172.21.15.92:0/3877966657 learned_addr learned my addr 172.21.15.92:0/3877966657 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:18:18.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.998+0000 7f02b659c700 1 --2- 172.21.15.92:0/3877966657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02b80fce60 0x7f02b810d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:18.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.998+0000 7f02b5d9b700 1 -- 172.21.15.92:0/3877966657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02b80fce60 msgr2=0x7f02b810d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:18.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.998+0000 7f02b5d9b700 1 --2- 172.21.15.92:0/3877966657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02b80fce60 0x7f02b810d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:18.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.998+0000 7f02b5d9b700 1 -- 172.21.15.92:0/3877966657 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a8005040 con 0x7f02b80fdfd0 2024-05-06T12:18:18.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.998+0000 7f02b5d9b700 1 --2- 172.21.15.92:0/3877966657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02b80fdfd0 0x7f02b810ddf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f02ac00eb40 tx=0x7f02ac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:18:18.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.998+0000 7f02a6ffd700 1 -- 172.21.15.92:0/3877966657 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02ac007cc0 con 0x7f02b80fdfd0 2024-05-06T12:18:18.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.998+0000 7f02bc88a700 1 -- 172.21.15.92:0/3877966657 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b8112f60 con 0x7f02b80fdfd0 2024-05-06T12:18:18.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.998+0000 7f02a6ffd700 1 -- 172.21.15.92:0/3877966657 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f02ac00f040 con 0x7f02b80fdfd0 2024-05-06T12:18:18.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.998+0000 7f02a6ffd700 1 -- 172.21.15.92:0/3877966657 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02ac009620 con 0x7f02b80fdfd0 2024-05-06T12:18:18.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.998+0000 7f02bc88a700 1 -- 172.21.15.92:0/3877966657 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b8113470 con 0x7f02b80fdfd0 2024-05-06T12:18:18.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:17.999+0000 7f02bc88a700 1 -- 172.21.15.92:0/3877966657 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02b804c710 con 0x7f02b80fdfd0 2024-05-06T12:18:18.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.000+0000 7f02a6ffd700 1 -- 172.21.15.92:0/3877966657 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f02ac009780 con 0x7f02b80fdfd0 2024-05-06T12:18:18.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.000+0000 7f02a6ffd700 1 --2- 172.21.15.92:0/3877966657 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f02a006e140 0x7f02a00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:18.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.000+0000 7f02b659c700 1 --2- 172.21.15.92:0/3877966657 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f02a006e140 0x7f02a00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:18.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.000+0000 7f02a6ffd700 1 -- 172.21.15.92:0/3877966657 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f02ac08d7c0 con 0x7f02b80fdfd0 2024-05-06T12:18:18.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.001+0000 7f02b659c700 1 --2- 172.21.15.92:0/3877966657 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f02a006e140 0x7f02a00705f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f02a8013870 tx=0x7f02a8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:18:18.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.005+0000 7f02a6ffd700 1 -- 172.21.15.92:0/3877966657 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f02ac05a720 con 0x7f02b80fdfd0 2024-05-06T12:18:18.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.214+0000 7f02bc88a700 1 -- 172.21.15.92:0/3877966657 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f02b81022b0 con 0x7f02a006e140 2024-05-06T12:18:18.217 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:17 smithi092 ceph-mon[91933]: pgmap v57: 65 pgs: 65 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 614 B/s wr, 3 op/s 2024-05-06T12:18:18.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.224+0000 7f02a6ffd700 1 -- 172.21.15.92:0/3877966657 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f02b81022b0 con 0x7f02a006e140 2024-05-06T12:18:18.227 INFO:teuthology.orchestra.run.smithi092.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-06T12:18:18.227 INFO:teuthology.orchestra.run.smithi092.stdout:alertmanager.smithi092 smithi092 *:9093,9094 running (3m) 12s ago 11m 17.0M - 0.25.0 c8568f914cd2 be5637d7d25a 2024-05-06T12:18:18.227 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi092 smithi092 running (44s) 12s ago 12m 6660k - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 8da0494e1bdc 2024-05-06T12:18:18.227 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi190 smithi190 running (38s) 35s ago 11m 6665k - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 641a7cdaa56e 2024-05-06T12:18:18.227 INFO:teuthology.orchestra.run.smithi092.stdout:grafana.smithi092 smithi092 *:3000 running (3m) 12s ago 11m 77.0M - 9.4.7 954c08fa6188 9180cb3e5258 2024-05-06T12:18:18.227 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.ijsqgp smithi092 running (6m) 12s ago 6m 60.6M - 17.2.7-925-ge81df420 96d5ae7fd4b4 b4c2ed6d2c82 2024-05-06T12:18:18.227 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.rosbll smithi092 running (6m) 12s ago 6m 122M - 17.2.7-925-ge81df420 96d5ae7fd4b4 5f5426cd1bf5 2024-05-06T12:18:18.227 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.byjvgf smithi190 running (6m) 35s ago 6m 126M - 17.2.7-925-ge81df420 96d5ae7fd4b4 24581d2e1c5d 2024-05-06T12:18:18.227 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.idzoff smithi190 running (6m) 35s ago 6m 64.3M - 17.2.7-925-ge81df420 96d5ae7fd4b4 20f2641696ef 2024-05-06T12:18:18.227 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi092.gyzeqb smithi092 *:8443,9283,8765 running (2m) 12s ago 13m 500M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 de9d2198327a 2024-05-06T12:18:18.228 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi190.kczxkh smithi190 *:8443,9283,8765 running (2m) 35s ago 10m 449M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 e84ab3758ac7 2024-05-06T12:18:18.228 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi092 smithi092 running (108s) 12s ago 13m 42.1M 2048M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 ac07f7158087 2024-05-06T12:18:18.228 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi190 smithi190 running (85s) 35s ago 10m 34.0M 2048M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 7a3c08eb716f 2024-05-06T12:18:18.228 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi092 smithi092 *:9100 running (3m) 12s ago 11m 15.2M - 1.5.0 0da6a335fe13 c796446b206f 2024-05-06T12:18:18.228 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi190 smithi190 *:9100 running (3m) 35s ago 10m 16.9M - 1.5.0 0da6a335fe13 647e28111a76 2024-05-06T12:18:18.228 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 smithi092 running (18s) 12s ago 10m 14.8M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 8c96c75b25f4 2024-05-06T12:18:18.228 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 smithi092 running (9m) 12s ago 9m 200M 1557M 17.2.7-925-ge81df420 96d5ae7fd4b4 8ab870f57eb2 2024-05-06T12:18:18.228 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 smithi092 running (9m) 12s ago 9m 166M 1557M 17.2.7-925-ge81df420 96d5ae7fd4b4 1e6ed04a0688 2024-05-06T12:18:18.228 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 smithi190 running (8m) 35s ago 8m 221M 2581M 17.2.7-925-ge81df420 96d5ae7fd4b4 52f9521286d5 2024-05-06T12:18:18.228 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 smithi190 running (8m) 35s ago 8m 204M 2581M 17.2.7-925-ge81df420 96d5ae7fd4b4 3a7da09e4c48 2024-05-06T12:18:18.228 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 smithi190 running (7m) 35s ago 7m 188M 2581M 17.2.7-925-ge81df420 96d5ae7fd4b4 e7452c6b5cec 2024-05-06T12:18:18.228 INFO:teuthology.orchestra.run.smithi092.stdout:prometheus.smithi092 smithi092 *:9095 running (2m) 12s ago 11m 51.6M - 2.43.0 a07b618ecd1d bc5445691527 2024-05-06T12:18:18.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.226+0000 7f02bc88a700 1 -- 172.21.15.92:0/3877966657 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f02a006e140 msgr2=0x7f02a00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:18.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.226+0000 7f02bc88a700 1 --2- 172.21.15.92:0/3877966657 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f02a006e140 0x7f02a00705f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f02a8013870 tx=0x7f02a8009e30 comp rx=0 tx=0).stop 2024-05-06T12:18:18.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.226+0000 7f02bc88a700 1 -- 172.21.15.92:0/3877966657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02b80fdfd0 msgr2=0x7f02b810ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:18.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.226+0000 7f02bc88a700 1 --2- 172.21.15.92:0/3877966657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02b80fdfd0 0x7f02b810ddf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f02ac00eb40 tx=0x7f02ac00b040 comp rx=0 tx=0).stop 2024-05-06T12:18:18.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.226+0000 7f02bc88a700 1 -- 172.21.15.92:0/3877966657 shutdown_connections 2024-05-06T12:18:18.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.226+0000 7f02bc88a700 1 --2- 172.21.15.92:0/3877966657 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f02a006e140 0x7f02a00705f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:18.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.226+0000 7f02bc88a700 1 --2- 172.21.15.92:0/3877966657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02b80fce60 0x7f02b810d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:18.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.226+0000 7f02bc88a700 1 --2- 172.21.15.92:0/3877966657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02b80fdfd0 0x7f02b810ddf0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:18.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.226+0000 7f02bc88a700 1 -- 172.21.15.92:0/3877966657 >> 172.21.15.92:0/3877966657 conn(0x7f02b80f8540 msgr2=0x7f02b80f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:18:18.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.227+0000 7f02bc88a700 1 -- 172.21.15.92:0/3877966657 shutdown_connections 2024-05-06T12:18:18.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.227+0000 7f02bc88a700 1 -- 172.21.15.92:0/3877966657 wait complete. 2024-05-06T12:18:18.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.326+0000 7fea935a3700 1 -- 172.21.15.92:0/2061597815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea8c0fcff0 msgr2=0x7fea8c0fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:18.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.326+0000 7fea935a3700 1 --2- 172.21.15.92:0/2061597815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea8c0fcff0 0x7fea8c0fd460 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fea7c004940 tx=0x7fea7c009e30 comp rx=0 tx=0).stop 2024-05-06T12:18:18.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.326+0000 7fea935a3700 1 -- 172.21.15.92:0/2061597815 shutdown_connections 2024-05-06T12:18:18.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.326+0000 7fea935a3700 1 --2- 172.21.15.92:0/2061597815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea8c0fcff0 0x7fea8c0fd460 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:18.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.326+0000 7fea935a3700 1 --2- 172.21.15.92:0/2061597815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea8c0fbd40 0x7fea8c0fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:18.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.326+0000 7fea935a3700 1 -- 172.21.15.92:0/2061597815 >> 172.21.15.92:0/2061597815 conn(0x7fea8c0f74c0 msgr2=0x7fea8c0f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:18:18.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.327+0000 7fea935a3700 1 -- 172.21.15.92:0/2061597815 shutdown_connections 2024-05-06T12:18:18.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.327+0000 7fea935a3700 1 -- 172.21.15.92:0/2061597815 wait complete. 2024-05-06T12:18:18.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.327+0000 7fea935a3700 1 Processor -- start 2024-05-06T12:18:18.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.327+0000 7fea935a3700 1 -- start start 2024-05-06T12:18:18.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.327+0000 7fea935a3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea8c0fbd40 0x7fea8c10d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:18.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.327+0000 7fea935a3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea8c0fcff0 0x7fea8c10df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:18.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.327+0000 7fea935a3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8c10e530 con 0x7fea8c0fcff0 2024-05-06T12:18:18.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.327+0000 7fea935a3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8c10ed80 con 0x7fea8c0fbd40 2024-05-06T12:18:18.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.327+0000 7fea90b3e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea8c0fcff0 0x7fea8c10df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:18.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.327+0000 7fea90b3e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea8c0fcff0 0x7fea8c10df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34972/0 (socket says 172.21.15.92:34972) 2024-05-06T12:18:18.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.327+0000 7fea90b3e700 1 -- 172.21.15.92:0/1419866789 learned_addr learned my addr 172.21.15.92:0/1419866789 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:18:18.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.327+0000 7fea9133f700 1 --2- 172.21.15.92:0/1419866789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea8c0fbd40 0x7fea8c10d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:18.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.328+0000 7fea90b3e700 1 -- 172.21.15.92:0/1419866789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea8c0fbd40 msgr2=0x7fea8c10d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:18.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.328+0000 7fea90b3e700 1 --2- 172.21.15.92:0/1419866789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea8c0fbd40 0x7fea8c10d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:18.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.328+0000 7fea90b3e700 1 -- 172.21.15.92:0/1419866789 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea7c005040 con 0x7fea8c0fcff0 2024-05-06T12:18:18.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.328+0000 7fea90b3e700 1 --2- 172.21.15.92:0/1419866789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea8c0fcff0 0x7fea8c10df30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00b3b0 tx=0x7fea7c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:18:18.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.328+0000 7fea81ffb700 1 -- 172.21.15.92:0/1419866789 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea7c00e040 con 0x7fea8c0fcff0 2024-05-06T12:18:18.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.328+0000 7fea81ffb700 1 -- 172.21.15.92:0/1419866789 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fea7c00bbf0 con 0x7fea8c0fcff0 2024-05-06T12:18:18.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.328+0000 7fea81ffb700 1 -- 172.21.15.92:0/1419866789 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea7c01c630 con 0x7fea8c0fcff0 2024-05-06T12:18:18.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.328+0000 7fea935a3700 1 -- 172.21.15.92:0/1419866789 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea8c10f000 con 0x7fea8c0fcff0 2024-05-06T12:18:18.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.328+0000 7fea935a3700 1 -- 172.21.15.92:0/1419866789 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea8c10f540 con 0x7fea8c0fcff0 2024-05-06T12:18:18.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.329+0000 7fea935a3700 1 -- 172.21.15.92:0/1419866789 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea8c101b80 con 0x7fea8c0fcff0 2024-05-06T12:18:18.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.330+0000 7fea81ffb700 1 -- 172.21.15.92:0/1419866789 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fea7c00bd60 con 0x7fea8c0fcff0 2024-05-06T12:18:18.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.330+0000 7fea81ffb700 1 --2- 172.21.15.92:0/1419866789 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fea7806e140 0x7fea780705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:18.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.330+0000 7fea81ffb700 1 -- 172.21.15.92:0/1419866789 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fea7c08e600 con 0x7fea8c0fcff0 2024-05-06T12:18:18.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.330+0000 7fea9133f700 1 --2- 172.21.15.92:0/1419866789 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fea7806e140 0x7fea780705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:18.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.331+0000 7fea9133f700 1 --2- 172.21.15.92:0/1419866789 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fea7806e140 0x7fea780705f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fea88004f40 tx=0x7fea8800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:18:18.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.335+0000 7fea81ffb700 1 -- 172.21.15.92:0/1419866789 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fea7c05b5d0 con 0x7fea8c0fcff0 2024-05-06T12:18:18.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:17 smithi190 ceph-mon[79744]: pgmap v57: 65 pgs: 65 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 614 B/s wr, 3 op/s 2024-05-06T12:18:18.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.618+0000 7fea935a3700 1 -- 172.21.15.92:0/1419866789 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fea8c05f5c0 con 0x7fea8c0fcff0 2024-05-06T12:18:18.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.619+0000 7fea81ffb700 1 -- 172.21.15.92:0/1419866789 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fea7c05b160 con 0x7fea8c0fcff0 2024-05-06T12:18:18.622 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:18:18.622 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": { 2024-05-06T12:18:18.622 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:18:18.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:18:18.622 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": { 2024-05-06T12:18:18.622 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:18:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:18:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": { 2024-05-06T12:18:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 5, 2024-05-06T12:18:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 1 2024-05-06T12:18:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:18:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-05-06T12:18:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 4 2024-05-06T12:18:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:18:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: "overall": { 2024-05-06T12:18:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 9, 2024-05-06T12:18:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 5 2024-05-06T12:18:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-06T12:18:18.623 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:18:18.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.621+0000 7fea935a3700 1 -- 172.21.15.92:0/1419866789 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fea7806e140 msgr2=0x7fea780705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:18.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.621+0000 7fea935a3700 1 --2- 172.21.15.92:0/1419866789 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fea7806e140 0x7fea780705f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fea88004f40 tx=0x7fea8800b040 comp rx=0 tx=0).stop 2024-05-06T12:18:18.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.621+0000 7fea935a3700 1 -- 172.21.15.92:0/1419866789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea8c0fcff0 msgr2=0x7fea8c10df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:18.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.621+0000 7fea935a3700 1 --2- 172.21.15.92:0/1419866789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea8c0fcff0 0x7fea8c10df30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00b3b0 tx=0x7fea7c009e30 comp rx=0 tx=0).stop 2024-05-06T12:18:18.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.621+0000 7fea935a3700 1 -- 172.21.15.92:0/1419866789 shutdown_connections 2024-05-06T12:18:18.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.621+0000 7fea935a3700 1 --2- 172.21.15.92:0/1419866789 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fea7806e140 0x7fea780705f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:18.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.621+0000 7fea935a3700 1 --2- 172.21.15.92:0/1419866789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea8c0fbd40 0x7fea8c10d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:18.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.621+0000 7fea935a3700 1 --2- 172.21.15.92:0/1419866789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea8c0fcff0 0x7fea8c10df30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:18.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.621+0000 7fea935a3700 1 -- 172.21.15.92:0/1419866789 >> 172.21.15.92:0/1419866789 conn(0x7fea8c0f74c0 msgr2=0x7fea8c066420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:18:18.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.621+0000 7fea935a3700 1 -- 172.21.15.92:0/1419866789 shutdown_connections 2024-05-06T12:18:18.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.621+0000 7fea935a3700 1 -- 172.21.15.92:0/1419866789 wait complete. 2024-05-06T12:18:18.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.723+0000 7f347b672700 1 -- 172.21.15.92:0/4031585667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34740fbcc0 msgr2=0x7f34740fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:18.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.723+0000 7f347b672700 1 --2- 172.21.15.92:0/4031585667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34740fbcc0 0x7f34740fc0d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3470004940 tx=0x7f3470009e30 comp rx=0 tx=0).stop 2024-05-06T12:18:18.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.723+0000 7f347b672700 1 -- 172.21.15.92:0/4031585667 shutdown_connections 2024-05-06T12:18:18.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.723+0000 7f347b672700 1 --2- 172.21.15.92:0/4031585667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34740fcf70 0x7f34740fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:18.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.723+0000 7f347b672700 1 --2- 172.21.15.92:0/4031585667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34740fbcc0 0x7f34740fc0d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:18.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.723+0000 7f347b672700 1 -- 172.21.15.92:0/4031585667 >> 172.21.15.92:0/4031585667 conn(0x7f34740f7440 msgr2=0x7f34740f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:18:18.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.724+0000 7f347b672700 1 -- 172.21.15.92:0/4031585667 shutdown_connections 2024-05-06T12:18:18.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.724+0000 7f347b672700 1 -- 172.21.15.92:0/4031585667 wait complete. 2024-05-06T12:18:18.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.724+0000 7f347b672700 1 Processor -- start 2024-05-06T12:18:18.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.724+0000 7f347b672700 1 -- start start 2024-05-06T12:18:18.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.724+0000 7f347b672700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34740fbcc0 0x7f347410d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:18.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.724+0000 7f347b672700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34740fcf70 0x7f347410ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:18.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.724+0000 7f347b672700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347410e3d0 con 0x7f34740fbcc0 2024-05-06T12:18:18.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.724+0000 7f347b672700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347410e540 con 0x7f34740fcf70 2024-05-06T12:18:18.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.724+0000 7f347940e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34740fbcc0 0x7f347410d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:18.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.724+0000 7f347940e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34740fbcc0 0x7f347410d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34998/0 (socket says 172.21.15.92:34998) 2024-05-06T12:18:18.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.724+0000 7f347940e700 1 -- 172.21.15.92:0/1006561967 learned_addr learned my addr 172.21.15.92:0/1006561967 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:18:18.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.725+0000 7f3478c0d700 1 --2- 172.21.15.92:0/1006561967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34740fcf70 0x7f347410ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:18.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.725+0000 7f347940e700 1 -- 172.21.15.92:0/1006561967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34740fcf70 msgr2=0x7f347410ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:18.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.725+0000 7f347940e700 1 --2- 172.21.15.92:0/1006561967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34740fcf70 0x7f347410ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:18.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.725+0000 7f347940e700 1 -- 172.21.15.92:0/1006561967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3470005040 con 0x7f34740fbcc0 2024-05-06T12:18:18.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.725+0000 7f3478c0d700 1 --2- 172.21.15.92:0/1006561967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34740fcf70 0x7f347410ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-06T12:18:18.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.725+0000 7f347940e700 1 --2- 172.21.15.92:0/1006561967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34740fbcc0 0x7f347410d890 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3470009150 tx=0x7f3470009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:18:18.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.725+0000 7f3469ffb700 1 -- 172.21.15.92:0/1006561967 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f347000e040 con 0x7f34740fbcc0 2024-05-06T12:18:18.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.725+0000 7f3469ffb700 1 -- 172.21.15.92:0/1006561967 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f347000bcc0 con 0x7f34740fbcc0 2024-05-06T12:18:18.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.725+0000 7f347b672700 1 -- 172.21.15.92:0/1006561967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3474112f40 con 0x7f34740fbcc0 2024-05-06T12:18:18.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.725+0000 7f3469ffb700 1 -- 172.21.15.92:0/1006561967 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f347001c630 con 0x7f34740fbcc0 2024-05-06T12:18:18.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.725+0000 7f347b672700 1 -- 172.21.15.92:0/1006561967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3474113470 con 0x7f34740fbcc0 2024-05-06T12:18:18.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.726+0000 7f347b672700 1 -- 172.21.15.92:0/1006561967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3474101b00 con 0x7f34740fbcc0 2024-05-06T12:18:18.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.727+0000 7f3469ffb700 1 -- 172.21.15.92:0/1006561967 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3470013420 con 0x7f34740fbcc0 2024-05-06T12:18:18.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.727+0000 7f3469ffb700 1 --2- 172.21.15.92:0/1006561967 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f346006e140 0x7f34600705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:18.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.727+0000 7f3469ffb700 1 -- 172.21.15.92:0/1006561967 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f347008e7b0 con 0x7f34740fbcc0 2024-05-06T12:18:18.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.727+0000 7f3478c0d700 1 --2- 172.21.15.92:0/1006561967 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f346006e140 0x7f34600705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:18.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.727+0000 7f3478c0d700 1 --2- 172.21.15.92:0/1006561967 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f346006e140 0x7f34600705f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3464009b80 tx=0x7f346400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:18:18.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.732+0000 7f3469ffb700 1 -- 172.21.15.92:0/1006561967 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f347005b6e0 con 0x7f34740fbcc0 2024-05-06T12:18:18.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:18 smithi092 ceph-mon[91933]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:18:18.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:18 smithi092 ceph-mon[91933]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:18:18.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:18 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/1419866789' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:18:18.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.978+0000 7f347b672700 1 -- 172.21.15.92:0/1006561967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f347402b370 con 0x7f34740fbcc0 2024-05-06T12:18:18.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.978+0000 7f3469ffb700 1 -- 172.21.15.92:0/1006561967 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1937 (secure 0 0 0) 0x7f347005b270 con 0x7f34740fbcc0 2024-05-06T12:18:18.982 INFO:teuthology.orchestra.run.smithi092.stdout:e20 2024-05-06T12:18:18.982 INFO:teuthology.orchestra.run.smithi092.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-06T12:18:18.982 INFO:teuthology.orchestra.run.smithi092.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:18:18.982 INFO:teuthology.orchestra.run.smithi092.stdout:legacy client fscid: 1 2024-05-06T12:18:18.982 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:18:18.982 INFO:teuthology.orchestra.run.smithi092.stdout:Filesystem 'cephfs' (1) 2024-05-06T12:18:18.982 INFO:teuthology.orchestra.run.smithi092.stdout:fs_name cephfs 2024-05-06T12:18:18.982 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 20 2024-05-06T12:18:18.982 INFO:teuthology.orchestra.run.smithi092.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-06T12:18:18.982 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-05-06T12:11:44.952671+0000 2024-05-06T12:18:18.983 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2024-05-06T12:15:07.568167+0000 2024-05-06T12:18:18.983 INFO:teuthology.orchestra.run.smithi092.stdout:tableserver 0 2024-05-06T12:18:18.983 INFO:teuthology.orchestra.run.smithi092.stdout:root 0 2024-05-06T12:18:18.983 INFO:teuthology.orchestra.run.smithi092.stdout:session_timeout 60 2024-05-06T12:18:18.983 INFO:teuthology.orchestra.run.smithi092.stdout:session_autoclose 300 2024-05-06T12:18:18.983 INFO:teuthology.orchestra.run.smithi092.stdout:max_file_size 1099511627776 2024-05-06T12:18:18.983 INFO:teuthology.orchestra.run.smithi092.stdout:max_xattr_size 65536 2024-05-06T12:18:18.983 INFO:teuthology.orchestra.run.smithi092.stdout:required_client_features {} 2024-05-06T12:18:18.983 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure 0 2024-05-06T12:18:18.983 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure_osd_epoch 46 2024-05-06T12:18:18.983 INFO:teuthology.orchestra.run.smithi092.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:18:18.983 INFO:teuthology.orchestra.run.smithi092.stdout:max_mds 2 2024-05-06T12:18:18.983 INFO:teuthology.orchestra.run.smithi092.stdout:in 0,1 2024-05-06T12:18:18.983 INFO:teuthology.orchestra.run.smithi092.stdout:up {0=14472,1=24267} 2024-05-06T12:18:18.983 INFO:teuthology.orchestra.run.smithi092.stdout:failed 2024-05-06T12:18:18.984 INFO:teuthology.orchestra.run.smithi092.stdout:damaged 2024-05-06T12:18:18.984 INFO:teuthology.orchestra.run.smithi092.stdout:stopped 2024-05-06T12:18:18.984 INFO:teuthology.orchestra.run.smithi092.stdout:data_pools [3] 2024-05-06T12:18:18.984 INFO:teuthology.orchestra.run.smithi092.stdout:metadata_pool 2 2024-05-06T12:18:18.984 INFO:teuthology.orchestra.run.smithi092.stdout:inline_data enabled 2024-05-06T12:18:18.984 INFO:teuthology.orchestra.run.smithi092.stdout:balancer 2024-05-06T12:18:18.984 INFO:teuthology.orchestra.run.smithi092.stdout:bal_rank_mask -1 2024-05-06T12:18:18.984 INFO:teuthology.orchestra.run.smithi092.stdout:standby_count_wanted 1 2024-05-06T12:18:18.984 INFO:teuthology.orchestra.run.smithi092.stdout:qdb_cluster leader: 0 members: 2024-05-06T12:18:18.984 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.rosbll{0:14472} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.92:6828/1663253598,v1:172.21.15.92:6829/1663253598] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:18:18.984 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.ijsqgp{0:14484} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.92:6826/3613441054,v1:172.21.15.92:6827/3613441054] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:18:18.984 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.byjvgf{1:24267} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.190:6824/1565151699,v1:172.21.15.190:6825/1565151699] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:18:18.984 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.idzoff{1:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.190:6826/2453728380,v1:172.21.15.190:6827/2453728380] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:18:18.984 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:18:18.984 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:18:18.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.981+0000 7f347b672700 1 -- 172.21.15.92:0/1006561967 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f346006e140 msgr2=0x7f34600705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:18.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.981+0000 7f347b672700 1 --2- 172.21.15.92:0/1006561967 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f346006e140 0x7f34600705f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3464009b80 tx=0x7f346400b040 comp rx=0 tx=0).stop 2024-05-06T12:18:18.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.981+0000 7f347b672700 1 -- 172.21.15.92:0/1006561967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34740fbcc0 msgr2=0x7f347410d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:18.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.981+0000 7f347b672700 1 --2- 172.21.15.92:0/1006561967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34740fbcc0 0x7f347410d890 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3470009150 tx=0x7f3470009e30 comp rx=0 tx=0).stop 2024-05-06T12:18:18.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.981+0000 7f347b672700 1 -- 172.21.15.92:0/1006561967 shutdown_connections 2024-05-06T12:18:18.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.981+0000 7f347b672700 1 --2- 172.21.15.92:0/1006561967 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f346006e140 0x7f34600705f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:18.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.981+0000 7f347b672700 1 --2- 172.21.15.92:0/1006561967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34740fbcc0 0x7f347410d890 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:18.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.981+0000 7f347b672700 1 --2- 172.21.15.92:0/1006561967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34740fcf70 0x7f347410ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:18.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.981+0000 7f347b672700 1 -- 172.21.15.92:0/1006561967 >> 172.21.15.92:0/1006561967 conn(0x7f34740f7440 msgr2=0x7f34740f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:18:18.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.981+0000 7f347b672700 1 -- 172.21.15.92:0/1006561967 shutdown_connections 2024-05-06T12:18:18.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:18.981+0000 7f347b672700 1 -- 172.21.15.92:0/1006561967 wait complete. 2024-05-06T12:18:18.986 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 20 2024-05-06T12:18:19.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.084+0000 7f78749e3700 1 -- 172.21.15.92:0/629021251 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78700fe080 msgr2=0x7f78700fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:19.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.084+0000 7f78749e3700 1 --2- 172.21.15.92:0/629021251 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78700fe080 0x7f78700fe4f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7864004940 tx=0x7f7864009e30 comp rx=0 tx=0).stop 2024-05-06T12:18:19.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.084+0000 7f78749e3700 1 -- 172.21.15.92:0/629021251 shutdown_connections 2024-05-06T12:18:19.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.084+0000 7f78749e3700 1 --2- 172.21.15.92:0/629021251 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78700fe080 0x7f78700fe4f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:19.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.084+0000 7f78749e3700 1 --2- 172.21.15.92:0/629021251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78700fce80 0x7f78700fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:19.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.084+0000 7f78749e3700 1 -- 172.21.15.92:0/629021251 >> 172.21.15.92:0/629021251 conn(0x7f78700f8590 msgr2=0x7f78700fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:18:19.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.084+0000 7f78749e3700 1 -- 172.21.15.92:0/629021251 shutdown_connections 2024-05-06T12:18:19.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.084+0000 7f78749e3700 1 -- 172.21.15.92:0/629021251 wait complete. 2024-05-06T12:18:19.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.084+0000 7f78749e3700 1 Processor -- start 2024-05-06T12:18:19.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.084+0000 7f78749e3700 1 -- start start 2024-05-06T12:18:19.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.084+0000 7f78749e3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78700fce80 0x7f787010da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:19.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.084+0000 7f78749e3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78700fe080 0x7f787010df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:19.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.084+0000 7f78749e3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787010e560 con 0x7f78700fce80 2024-05-06T12:18:19.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.084+0000 7f78749e3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787010edb0 con 0x7f78700fe080 2024-05-06T12:18:19.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.085+0000 7f786e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78700fce80 0x7f787010da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:19.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.085+0000 7f786e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78700fce80 0x7f787010da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35018/0 (socket says 172.21.15.92:35018) 2024-05-06T12:18:19.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.085+0000 7f786e59c700 1 -- 172.21.15.92:0/1016456940 learned_addr learned my addr 172.21.15.92:0/1016456940 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:18:19.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.085+0000 7f786dd9b700 1 --2- 172.21.15.92:0/1016456940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78700fe080 0x7f787010df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:19.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.085+0000 7f786e59c700 1 -- 172.21.15.92:0/1016456940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78700fe080 msgr2=0x7f787010df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:19.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.085+0000 7f786e59c700 1 --2- 172.21.15.92:0/1016456940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78700fe080 0x7f787010df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:19.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.085+0000 7f786e59c700 1 -- 172.21.15.92:0/1016456940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7864005040 con 0x7f78700fce80 2024-05-06T12:18:19.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.085+0000 7f786dd9b700 1 --2- 172.21.15.92:0/1016456940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78700fe080 0x7f787010df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-06T12:18:19.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.085+0000 7f786e59c700 1 --2- 172.21.15.92:0/1016456940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78700fce80 0x7f787010da20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f786000c8e0 tx=0x7f78600044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:18:19.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.085+0000 7f785effd700 1 -- 172.21.15.92:0/1016456940 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f786000a940 con 0x7f78700fce80 2024-05-06T12:18:19.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.085+0000 7f785effd700 1 -- 172.21.15.92:0/1016456940 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f786000ce80 con 0x7f78700fce80 2024-05-06T12:18:19.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.085+0000 7f78749e3700 1 -- 172.21.15.92:0/1016456940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f787010f090 con 0x7f78700fce80 2024-05-06T12:18:19.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.085+0000 7f785effd700 1 -- 172.21.15.92:0/1016456940 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78600113a0 con 0x7f78700fce80 2024-05-06T12:18:19.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.085+0000 7f78749e3700 1 -- 172.21.15.92:0/1016456940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f787010f5d0 con 0x7f78700fce80 2024-05-06T12:18:19.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.086+0000 7f78749e3700 1 -- 172.21.15.92:0/1016456940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f787005f5c0 con 0x7f78700fce80 2024-05-06T12:18:19.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.087+0000 7f785effd700 1 -- 172.21.15.92:0/1016456940 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7860012400 con 0x7f78700fce80 2024-05-06T12:18:19.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.087+0000 7f785effd700 1 --2- 172.21.15.92:0/1016456940 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f785806e140 0x7f78580705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:19.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.087+0000 7f785effd700 1 -- 172.21.15.92:0/1016456940 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f786008daf0 con 0x7f78700fce80 2024-05-06T12:18:19.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.087+0000 7f786dd9b700 1 --2- 172.21.15.92:0/1016456940 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f785806e140 0x7f78580705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:19.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.087+0000 7f786dd9b700 1 --2- 172.21.15.92:0/1016456940 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f785806e140 0x7f78580705f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7864000c00 tx=0x7f7864009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:18:19.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.091+0000 7f785effd700 1 -- 172.21.15.92:0/1016456940 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f786005aac0 con 0x7f78700fce80 2024-05-06T12:18:19.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.310+0000 7f78749e3700 1 -- 172.21.15.92:0/1016456940 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f787010f940 con 0x7f785806e140 2024-05-06T12:18:19.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.312+0000 7f785effd700 1 -- 172.21.15.92:0/1016456940 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f787010f940 con 0x7f785806e140 2024-05-06T12:18:19.315 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:18:19.315 INFO:teuthology.orchestra.run.smithi092.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb", 2024-05-06T12:18:19.315 INFO:teuthology.orchestra.run.smithi092.stdout: "in_progress": true, 2024-05-06T12:18:19.315 INFO:teuthology.orchestra.run.smithi092.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-06T12:18:19.315 INFO:teuthology.orchestra.run.smithi092.stdout: "services_complete": [ 2024-05-06T12:18:19.315 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr", 2024-05-06T12:18:19.315 INFO:teuthology.orchestra.run.smithi092.stdout: "mon", 2024-05-06T12:18:19.316 INFO:teuthology.orchestra.run.smithi092.stdout: "crash" 2024-05-06T12:18:19.316 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-05-06T12:18:19.316 INFO:teuthology.orchestra.run.smithi092.stdout: "progress": "7/21 daemons upgraded", 2024-05-06T12:18:19.316 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "Currently upgrading osd daemons", 2024-05-06T12:18:19.316 INFO:teuthology.orchestra.run.smithi092.stdout: "is_paused": false 2024-05-06T12:18:19.316 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:18:19.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.314+0000 7f78749e3700 1 -- 172.21.15.92:0/1016456940 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f785806e140 msgr2=0x7f78580705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:19.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.314+0000 7f78749e3700 1 --2- 172.21.15.92:0/1016456940 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f785806e140 0x7f78580705f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7864000c00 tx=0x7f7864009e30 comp rx=0 tx=0).stop 2024-05-06T12:18:19.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.314+0000 7f78749e3700 1 -- 172.21.15.92:0/1016456940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78700fce80 msgr2=0x7f787010da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:19.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.314+0000 7f78749e3700 1 --2- 172.21.15.92:0/1016456940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78700fce80 0x7f787010da20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f786000c8e0 tx=0x7f78600044c0 comp rx=0 tx=0).stop 2024-05-06T12:18:19.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.314+0000 7f78749e3700 1 -- 172.21.15.92:0/1016456940 shutdown_connections 2024-05-06T12:18:19.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.314+0000 7f78749e3700 1 --2- 172.21.15.92:0/1016456940 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f785806e140 0x7f78580705f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:19.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.314+0000 7f78749e3700 1 --2- 172.21.15.92:0/1016456940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78700fce80 0x7f787010da20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:19.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.314+0000 7f78749e3700 1 --2- 172.21.15.92:0/1016456940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78700fe080 0x7f787010df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:19.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.314+0000 7f78749e3700 1 -- 172.21.15.92:0/1016456940 >> 172.21.15.92:0/1016456940 conn(0x7f78700f8590 msgr2=0x7f78700f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:18:19.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.314+0000 7f78749e3700 1 -- 172.21.15.92:0/1016456940 shutdown_connections 2024-05-06T12:18:19.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.314+0000 7f78749e3700 1 -- 172.21.15.92:0/1016456940 wait complete. 2024-05-06T12:18:19.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:18 smithi190 ceph-mon[79744]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:18:19.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:18 smithi190 ceph-mon[79744]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:18:19.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:18 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/1419866789' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:18:19.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.414+0000 7fd8647e1700 1 -- 172.21.15.92:0/2698554736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd85c0fbce0 msgr2=0x7fd85c0fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:19.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.414+0000 7fd8647e1700 1 --2- 172.21.15.92:0/2698554736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd85c0fbce0 0x7fd85c0fc0f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd858004940 tx=0x7fd858009e30 comp rx=0 tx=0).stop 2024-05-06T12:18:19.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.414+0000 7fd8647e1700 1 -- 172.21.15.92:0/2698554736 shutdown_connections 2024-05-06T12:18:19.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.414+0000 7fd8647e1700 1 --2- 172.21.15.92:0/2698554736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd85c0fcf90 0x7fd85c0fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:19.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.414+0000 7fd8647e1700 1 --2- 172.21.15.92:0/2698554736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd85c0fbce0 0x7fd85c0fc0f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:19.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.414+0000 7fd8647e1700 1 -- 172.21.15.92:0/2698554736 >> 172.21.15.92:0/2698554736 conn(0x7fd85c0f7440 msgr2=0x7fd85c0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:18:19.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.414+0000 7fd8647e1700 1 -- 172.21.15.92:0/2698554736 shutdown_connections 2024-05-06T12:18:19.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.414+0000 7fd8647e1700 1 -- 172.21.15.92:0/2698554736 wait complete. 2024-05-06T12:18:19.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.415+0000 7fd8647e1700 1 Processor -- start 2024-05-06T12:18:19.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.415+0000 7fd8647e1700 1 -- start start 2024-05-06T12:18:19.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.415+0000 7fd8647e1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd85c0fbce0 0x7fd85c10fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:19.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.415+0000 7fd8647e1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd85c0fcf90 0x7fd85c1100c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:19.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.415+0000 7fd8647e1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c1106c0 con 0x7fd85c0fbce0 2024-05-06T12:18:19.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.415+0000 7fd8647e1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c110830 con 0x7fd85c0fcf90 2024-05-06T12:18:19.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.415+0000 7fd86257d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd85c0fbce0 0x7fd85c10fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:19.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.415+0000 7fd86257d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd85c0fbce0 0x7fd85c10fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35034/0 (socket says 172.21.15.92:35034) 2024-05-06T12:18:19.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.415+0000 7fd86257d700 1 -- 172.21.15.92:0/1002822422 learned_addr learned my addr 172.21.15.92:0/1002822422 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:18:19.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.415+0000 7fd861d7c700 1 --2- 172.21.15.92:0/1002822422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd85c0fcf90 0x7fd85c1100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:19.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.415+0000 7fd86257d700 1 -- 172.21.15.92:0/1002822422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd85c0fcf90 msgr2=0x7fd85c1100c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:19.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.415+0000 7fd86257d700 1 --2- 172.21.15.92:0/1002822422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd85c0fcf90 0x7fd85c1100c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:19.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.415+0000 7fd86257d700 1 -- 172.21.15.92:0/1002822422 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd858005040 con 0x7fd85c0fbce0 2024-05-06T12:18:19.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.415+0000 7fd861d7c700 1 --2- 172.21.15.92:0/1002822422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd85c0fcf90 0x7fd85c1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-06T12:18:19.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.416+0000 7fd86257d700 1 --2- 172.21.15.92:0/1002822422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd85c0fbce0 0x7fd85c10fb80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd858009150 tx=0x7fd858009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:18:19.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.416+0000 7fd852ffd700 1 -- 172.21.15.92:0/1002822422 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd85800e040 con 0x7fd85c0fbce0 2024-05-06T12:18:19.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.416+0000 7fd852ffd700 1 -- 172.21.15.92:0/1002822422 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd85800bcc0 con 0x7fd85c0fbce0 2024-05-06T12:18:19.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.416+0000 7fd8647e1700 1 -- 172.21.15.92:0/1002822422 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd85c1151e0 con 0x7fd85c0fbce0 2024-05-06T12:18:19.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.416+0000 7fd852ffd700 1 -- 172.21.15.92:0/1002822422 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd85801c630 con 0x7fd85c0fbce0 2024-05-06T12:18:19.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.416+0000 7fd8647e1700 1 -- 172.21.15.92:0/1002822422 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd85c1157a0 con 0x7fd85c0fbce0 2024-05-06T12:18:19.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.417+0000 7fd8647e1700 1 -- 172.21.15.92:0/1002822422 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd85c101920 con 0x7fd85c0fbce0 2024-05-06T12:18:19.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.417+0000 7fd852ffd700 1 -- 172.21.15.92:0/1002822422 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd858013420 con 0x7fd85c0fbce0 2024-05-06T12:18:19.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.418+0000 7fd852ffd700 1 --2- 172.21.15.92:0/1002822422 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd84806e140 0x7fd8480705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:19.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.418+0000 7fd852ffd700 1 -- 172.21.15.92:0/1002822422 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fd85808e6c0 con 0x7fd85c0fbce0 2024-05-06T12:18:19.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.418+0000 7fd861d7c700 1 --2- 172.21.15.92:0/1002822422 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd84806e140 0x7fd8480705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:19.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.418+0000 7fd861d7c700 1 --2- 172.21.15.92:0/1002822422 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd84806e140 0x7fd8480705f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0041d0 tx=0x7fd84c009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:18:19.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.423+0000 7fd852ffd700 1 -- 172.21.15.92:0/1002822422 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd85805b690 con 0x7fd85c0fbce0 2024-05-06T12:18:19.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.700+0000 7fd8647e1700 1 -- 172.21.15.92:0/1002822422 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd85c04c710 con 0x7fd85c0fbce0 2024-05-06T12:18:19.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.701+0000 7fd852ffd700 1 -- 172.21.15.92:0/1002822422 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fd85c04c710 con 0x7fd85c0fbce0 2024-05-06T12:18:19.704 INFO:teuthology.orchestra.run.smithi092.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-06T12:18:19.704 INFO:teuthology.orchestra.run.smithi092.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-06T12:18:19.704 INFO:teuthology.orchestra.run.smithi092.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-06T12:18:19.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.703+0000 7fd8647e1700 1 -- 172.21.15.92:0/1002822422 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd84806e140 msgr2=0x7fd8480705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:19.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.703+0000 7fd8647e1700 1 --2- 172.21.15.92:0/1002822422 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd84806e140 0x7fd8480705f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0041d0 tx=0x7fd84c009040 comp rx=0 tx=0).stop 2024-05-06T12:18:19.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.703+0000 7fd8647e1700 1 -- 172.21.15.92:0/1002822422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd85c0fbce0 msgr2=0x7fd85c10fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:19.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.703+0000 7fd8647e1700 1 --2- 172.21.15.92:0/1002822422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd85c0fbce0 0x7fd85c10fb80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd858009150 tx=0x7fd858009e30 comp rx=0 tx=0).stop 2024-05-06T12:18:19.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.703+0000 7fd8647e1700 1 -- 172.21.15.92:0/1002822422 shutdown_connections 2024-05-06T12:18:19.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.703+0000 7fd8647e1700 1 --2- 172.21.15.92:0/1002822422 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd84806e140 0x7fd8480705f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:19.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.703+0000 7fd8647e1700 1 --2- 172.21.15.92:0/1002822422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd85c0fbce0 0x7fd85c10fb80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:19.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.703+0000 7fd8647e1700 1 --2- 172.21.15.92:0/1002822422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd85c0fcf90 0x7fd85c1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:19.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.703+0000 7fd8647e1700 1 -- 172.21.15.92:0/1002822422 >> 172.21.15.92:0/1002822422 conn(0x7fd85c0f7440 msgr2=0x7fd85c0667f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:18:19.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.703+0000 7fd8647e1700 1 -- 172.21.15.92:0/1002822422 shutdown_connections 2024-05-06T12:18:19.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:19.703+0000 7fd8647e1700 1 -- 172.21.15.92:0/1002822422 wait complete. 2024-05-06T12:18:19.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:19 smithi092 ceph-mon[91933]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:18:19.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:19 smithi092 ceph-mon[91933]: pgmap v58: 65 pgs: 65 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 515 B/s wr, 2 op/s 2024-05-06T12:18:19.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:19 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:19.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:19 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/1006561967' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:18:19.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:19 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/1002822422' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:18:20.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:19 smithi190 ceph-mon[79744]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:18:20.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:19 smithi190 ceph-mon[79744]: pgmap v58: 65 pgs: 65 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 515 B/s wr, 2 op/s 2024-05-06T12:18:20.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:19 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:20.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:19 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/1006561967' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:18:20.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:19 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/1002822422' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:18:21.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:20 smithi092 ceph-mon[91933]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:18:21.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:20 smithi190 ceph-mon[79744]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:18:22.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:21 smithi092 ceph-mon[91933]: pgmap v59: 65 pgs: 65 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-05-06T12:18:22.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:21 smithi190 ceph-mon[79744]: pgmap v59: 65 pgs: 65 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-05-06T12:18:24.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:23 smithi092 ceph-mon[91933]: pgmap v60: 65 pgs: 65 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:18:24.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:23 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:24.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:23 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:18:24.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:23 smithi190 ceph-mon[79744]: pgmap v60: 65 pgs: 65 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:18:24.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:23 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:24.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:23 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:18:26.120 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:25 smithi092 ceph-mon[91933]: pgmap v61: 65 pgs: 65 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:18:26.120 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:25 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-06T12:18:26.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:25 smithi190 ceph-mon[79744]: pgmap v61: 65 pgs: 65 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:18:26.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:25 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-06T12:18:27.219 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:26 smithi092 ceph-mon[91933]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-06T12:18:27.219 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:26 smithi092 ceph-mon[91933]: Upgrade: osd.1 is safe to restart 2024-05-06T12:18:27.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:26 smithi190 ceph-mon[79744]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-06T12:18:27.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:26 smithi190 ceph-mon[79744]: Upgrade: osd.1 is safe to restart 2024-05-06T12:18:28.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:27 smithi092 ceph-mon[91933]: pgmap v62: 65 pgs: 65 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:18:28.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:27 smithi092 ceph-mon[91933]: Upgrade: Updating osd.1 2024-05-06T12:18:28.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:27 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:28.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:27 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-06T12:18:28.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:27 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:18:28.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:27 smithi092 ceph-mon[91933]: Deploying daemon osd.1 on smithi092 2024-05-06T12:18:28.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:27 smithi190 ceph-mon[79744]: pgmap v62: 65 pgs: 65 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:18:28.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:27 smithi190 ceph-mon[79744]: Upgrade: Updating osd.1 2024-05-06T12:18:28.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:27 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:28.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:27 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-06T12:18:28.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:27 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:18:28.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:27 smithi190 ceph-mon[79744]: Deploying daemon osd.1 on smithi092 2024-05-06T12:18:30.147 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:29 smithi092 ceph-mon[91933]: pgmap v63: 65 pgs: 65 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:18:30.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:29 smithi190 ceph-mon[79744]: pgmap v63: 65 pgs: 65 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:18:30.401 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:30 smithi092 systemd[1]: Stopping Ceph osd.1 for aee1d912-0ba0-11ef-bc97-c7b262605968... 2024-05-06T12:18:30.722 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:30 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1[53068]: 2024-05-06T12:18:30.397+0000 7f94b41ab700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-06T12:18:30.723 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:30 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1[53068]: 2024-05-06T12:18:30.397+0000 7f94b41ab700 -1 osd.1 55 *** Got signal Terminated *** 2024-05-06T12:18:30.723 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:30 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1[53068]: 2024-05-06T12:18:30.397+0000 7f94b41ab700 -1 osd.1 55 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-06T12:18:31.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:30 smithi092 ceph-mon[91933]: osd.1 marked itself down and dead 2024-05-06T12:18:31.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:30 smithi190 ceph-mon[79744]: osd.1 marked itself down and dead 2024-05-06T12:18:31.535 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:31 smithi092 podman[105751]: 2024-05-06 12:18:31.266115282 +0000 UTC m=+0.978938109 container died 8ab870f57eb2bcc117d23a07a3b72548b8bf8036d5d9ced60021a32cc353e560 (image=quay.ceph.io/ceph-ci/ceph@sha256:1a0b7fd3cefd2c5baddf682493f92b4e27596f3c0ff27b02fdccd2891f56859f, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-e81df42, org.label-schema.build-date=20240430) 2024-05-06T12:18:31.910 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:31 smithi092 podman[105751]: 2024-05-06 12:18:31.622930107 +0000 UTC m=+1.335752943 container cleanup 8ab870f57eb2bcc117d23a07a3b72548b8bf8036d5d9ced60021a32cc353e560 (image=quay.ceph.io/ceph-ci/ceph@sha256:1a0b7fd3cefd2c5baddf682493f92b4e27596f3c0ff27b02fdccd2891f56859f, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=quincy-e81df42, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-06T12:18:31.910 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:31 smithi092 bash[105751]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1 2024-05-06T12:18:32.222 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:32 smithi092 podman[105768]: 2024-05-06 12:18:32.058296778 +0000 UTC m=+0.790127192 container remove 8ab870f57eb2bcc117d23a07a3b72548b8bf8036d5d9ced60021a32cc353e560 (image=quay.ceph.io/ceph-ci/ceph@sha256:1a0b7fd3cefd2c5baddf682493f92b4e27596f3c0ff27b02fdccd2891f56859f, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=quincy-e81df42, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-06T12:18:32.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:31 smithi092 ceph-mon[91933]: pgmap v64: 65 pgs: 65 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:18:32.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:31 smithi092 ceph-mon[91933]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-06T12:18:32.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:31 smithi092 ceph-mon[91933]: osdmap e56: 6 total, 5 up, 6 in 2024-05-06T12:18:32.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:31 smithi190 ceph-mon[79744]: pgmap v64: 65 pgs: 65 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:18:32.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:31 smithi190 ceph-mon[79744]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-06T12:18:32.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:31 smithi190 ceph-mon[79744]: osdmap e56: 6 total, 5 up, 6 in 2024-05-06T12:18:32.672 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:32 smithi092 podman[105860]: 2024-05-06 12:18:32.284650706 +0000 UTC m=+0.018329658 image pull e4ce8ad5d0c4b9ea60bc5f6b8b0fdb5efbe59bf7771fb0920fc2200e6ae546b0 quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb 2024-05-06T12:18:32.672 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:32 smithi092 podman[105860]: 2024-05-06 12:18:32.459184009 +0000 UTC m=+0.192862952 container create c7a4aa136d1044ddb8649872ddd71ef35370ae772ee7f811578d03f91f1b7bb0 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-70ce953, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240430) 2024-05-06T12:18:32.672 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:32 smithi092 podman[105860]: 2024-05-06 12:18:32.670037499 +0000 UTC m=+0.403716444 container init c7a4aa136d1044ddb8649872ddd71ef35370ae772ee7f811578d03f91f1b7bb0 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-70ce953, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-05-06T12:18:32.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:32 smithi092 ceph-mon[91933]: osdmap e57: 6 total, 5 up, 6 in 2024-05-06T12:18:32.973 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:32 smithi092 podman[105860]: 2024-05-06 12:18:32.673281971 +0000 UTC m=+0.406960918 container start c7a4aa136d1044ddb8649872ddd71ef35370ae772ee7f811578d03f91f1b7bb0 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=squid-70ce953, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-06T12:18:32.973 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:32 smithi092 podman[105860]: 2024-05-06 12:18:32.753426511 +0000 UTC m=+0.487105467 container attach c7a4aa136d1044ddb8649872ddd71ef35370ae772ee7f811578d03f91f1b7bb0 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-70ce953, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-06T12:18:33.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:32 smithi190 ceph-mon[79744]: osdmap e57: 6 total, 5 up, 6 in 2024-05-06T12:18:33.464 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:32 smithi092 conmon[105875]: conmon c7a4aa136d1044ddb864 : Failed to open cgroups file: /sys/fs/cgroup/machine.slice/libpod-c7a4aa136d1044ddb8649872ddd71ef35370ae772ee7f811578d03f91f1b7bb0.scope/container/memory.events 2024-05-06T12:18:33.464 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:32 smithi092 podman[105860]: 2024-05-06 12:18:32.975205214 +0000 UTC m=+0.708884158 container died c7a4aa136d1044ddb8649872ddd71ef35370ae772ee7f811578d03f91f1b7bb0 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1-deactivate, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-70ce953, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-06T12:18:33.723 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:33 smithi092 podman[105886]: 2024-05-06 12:18:33.556924848 +0000 UTC m=+0.576315353 container remove c7a4aa136d1044ddb8649872ddd71ef35370ae772ee7f811578d03f91f1b7bb0 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1-deactivate, RELEASE=squid-70ce953, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-06T12:18:33.723 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:33 smithi092 systemd[1]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@osd.1.service: Deactivated successfully. 2024-05-06T12:18:33.723 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:33 smithi092 systemd[1]: Stopped Ceph osd.1 for aee1d912-0ba0-11ef-bc97-c7b262605968. 2024-05-06T12:18:33.723 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:33 smithi092 systemd[1]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@osd.1.service: Consumed 53.666s CPU time. 2024-05-06T12:18:34.035 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:33 smithi092 ceph-mon[91933]: pgmap v67: 65 pgs: 12 stale+active+clean, 53 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 3 op/s 2024-05-06T12:18:34.035 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:33 smithi092 systemd[1]: Starting Ceph osd.1 for aee1d912-0ba0-11ef-bc97-c7b262605968... 2024-05-06T12:18:34.035 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:33 smithi092 podman[105990]: 2024-05-06 12:18:33.899411159 +0000 UTC m=+0.018513114 image pull e4ce8ad5d0c4b9ea60bc5f6b8b0fdb5efbe59bf7771fb0920fc2200e6ae546b0 quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb 2024-05-06T12:18:34.035 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:34 smithi092 podman[105990]: 2024-05-06 12:18:34.033319188 +0000 UTC m=+0.152421114 container create 7dff2b3f15257017da8e937e6d16afa5b46af0a6a5b48f0c4f22471da5229cd9 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1-activate, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-70ce953, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD) 2024-05-06T12:18:34.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:33 smithi190 ceph-mon[79744]: pgmap v67: 65 pgs: 12 stale+active+clean, 53 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 3 op/s 2024-05-06T12:18:34.354 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:34 smithi092 podman[105990]: 2024-05-06 12:18:34.261093783 +0000 UTC m=+0.380195721 container init 7dff2b3f15257017da8e937e6d16afa5b46af0a6a5b48f0c4f22471da5229cd9 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, RELEASE=squid-70ce953, GIT_BRANCH=HEAD) 2024-05-06T12:18:34.354 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:34 smithi092 podman[105990]: 2024-05-06 12:18:34.264535528 +0000 UTC m=+0.383637464 container start 7dff2b3f15257017da8e937e6d16afa5b46af0a6a5b48f0c4f22471da5229cd9 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1-activate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-70ce953, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-05-06T12:18:34.973 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:34 smithi092 podman[105990]: 2024-05-06 12:18:34.352400616 +0000 UTC m=+0.471502559 container attach 7dff2b3f15257017da8e937e6d16afa5b46af0a6a5b48f0c4f22471da5229cd9 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1-activate, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-70ce953, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-06T12:18:34.973 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:34 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1-activate[106004]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-06T12:18:34.973 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:34 smithi092 bash[105990]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-06T12:18:34.974 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:34 smithi092 bash[105990]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-05-06T12:18:34.974 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:34 smithi092 bash[105990]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-05-06T12:18:34.974 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:34 smithi092 bash[105990]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-05-06T12:18:34.974 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:34 smithi092 bash[105990]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-05-06T12:18:34.974 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:34 smithi092 bash[105990]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-06T12:18:34.974 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:34 smithi092 bash[105990]: --> ceph-volume raw activate successful for osd ID: 1 2024-05-06T12:18:34.974 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:34 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1-activate[106004]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-05-06T12:18:34.974 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:34 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1-activate[106004]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-05-06T12:18:34.974 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:34 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1-activate[106004]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-05-06T12:18:34.975 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:34 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1-activate[106004]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-05-06T12:18:34.975 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:34 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1-activate[106004]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-06T12:18:34.975 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:34 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1-activate[106004]: --> ceph-volume raw activate successful for osd ID: 1 2024-05-06T12:18:34.975 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:34 smithi092 podman[105990]: 2024-05-06 12:18:34.734476153 +0000 UTC m=+0.853578086 container died 7dff2b3f15257017da8e937e6d16afa5b46af0a6a5b48f0c4f22471da5229cd9 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1-activate, org.label-schema.build-date=20240430, RELEASE=squid-70ce953, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-05-06T12:18:35.325 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:34 smithi092 ceph-mon[91933]: Health check failed: Degraded data redundancy: 49/372 objects degraded (13.172%), 14 pgs degraded (PG_DEGRADED) 2024-05-06T12:18:35.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:34 smithi190 ceph-mon[79744]: Health check failed: Degraded data redundancy: 49/372 objects degraded (13.172%), 14 pgs degraded (PG_DEGRADED) 2024-05-06T12:18:35.659 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:35 smithi092 podman[106143]: 2024-05-06 12:18:35.323038286 +0000 UTC m=+0.583334543 container remove 7dff2b3f15257017da8e937e6d16afa5b46af0a6a5b48f0c4f22471da5229cd9 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1-activate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-70ce953, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-06T12:18:35.659 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:35 smithi092 podman[106210]: 2024-05-06 12:18:35.437553422 +0000 UTC m=+0.018479585 image pull e4ce8ad5d0c4b9ea60bc5f6b8b0fdb5efbe59bf7771fb0920fc2200e6ae546b0 quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb 2024-05-06T12:18:35.659 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:35 smithi092 podman[106210]: 2024-05-06 12:18:35.55736274 +0000 UTC m=+0.138288930 container create e8007a510f08eae84f224f0551305393278f755fa1814b5c88fd14c114f32df0 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-70ce953, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-06T12:18:36.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:35 smithi092 ceph-mon[91933]: pgmap v68: 65 pgs: 14 active+undersized+degraded, 3 stale+active+clean, 15 active+undersized, 33 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 49/372 objects degraded (13.172%) 2024-05-06T12:18:36.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:35 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:36.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:35 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:36.223 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:35 smithi092 podman[106210]: 2024-05-06 12:18:35.784843922 +0000 UTC m=+0.365770070 container init e8007a510f08eae84f224f0551305393278f755fa1814b5c88fd14c114f32df0 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-70ce953, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-06T12:18:36.223 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:35 smithi092 podman[106210]: 2024-05-06 12:18:35.788026683 +0000 UTC m=+0.368952831 container start e8007a510f08eae84f224f0551305393278f755fa1814b5c88fd14c114f32df0 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, RELEASE=squid-70ce953, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430) 2024-05-06T12:18:36.223 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:35 smithi092 ceph-osd[106229]: -- 172.21.15.92:0/2169404323 <== mon.0 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d7d76e3680 con 0x55d7d76ad000 2024-05-06T12:18:36.223 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:35 smithi092 bash[106210]: e8007a510f08eae84f224f0551305393278f755fa1814b5c88fd14c114f32df0 2024-05-06T12:18:36.224 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:35 smithi092 systemd[1]: Started Ceph osd.1 for aee1d912-0ba0-11ef-bc97-c7b262605968. 2024-05-06T12:18:36.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:35 smithi190 ceph-mon[79744]: pgmap v68: 65 pgs: 14 active+undersized+degraded, 3 stale+active+clean, 15 active+undersized, 33 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 49/372 objects degraded (13.172%) 2024-05-06T12:18:36.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:35 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:36.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:35 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:36.722 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:36 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1[106225]: 2024-05-06T12:18:36.325+0000 7f87120257c0 -1 Falling back to public interface 2024-05-06T12:18:37.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:36 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:18:37.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:36 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:18:38.250 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:37 smithi190 ceph-mon[79744]: pgmap v69: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 57/372 objects degraded (15.323%) 2024-05-06T12:18:38.250 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:37 smithi190 ceph-mon[79744]: fsmap cephfs:2 {0=cephfs.smithi092.rosbll=up:active,1=cephfs.smithi190.byjvgf=up:active} 2 up:standby-replay 2024-05-06T12:18:38.263 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:37 smithi092 ceph-mon[91933]: pgmap v69: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 57/372 objects degraded (15.323%) 2024-05-06T12:18:38.263 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:37 smithi092 ceph-mon[91933]: fsmap cephfs:2 {0=cephfs.smithi092.rosbll=up:active,1=cephfs.smithi190.byjvgf=up:active} 2 up:standby-replay 2024-05-06T12:18:39.971 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:39 smithi092 ceph-mon[91933]: pgmap v70: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 57/372 objects degraded (15.323%) 2024-05-06T12:18:39.971 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:39 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:39.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:39 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:18:40.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:39 smithi190 ceph-mon[79744]: pgmap v70: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 57/372 objects degraded (15.323%) 2024-05-06T12:18:40.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:39 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:40.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:39 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:18:41.188 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:40 smithi092 ceph-mon[91933]: Health check update: Degraded data redundancy: 57/372 objects degraded (15.323%), 16 pgs degraded (PG_DEGRADED) 2024-05-06T12:18:41.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:40 smithi190 ceph-mon[79744]: Health check update: Degraded data redundancy: 57/372 objects degraded (15.323%), 16 pgs degraded (PG_DEGRADED) 2024-05-06T12:18:41.472 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:41 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1[106225]: 2024-05-06T12:18:41.184+0000 7f87120257c0 -1 osd.1 55 log_to_monitors true 2024-05-06T12:18:42.196 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:41 smithi092 ceph-mon[91933]: pgmap v71: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 57/372 objects degraded (15.323%) 2024-05-06T12:18:42.197 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:41 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:42.197 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:41 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:42.197 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:41 smithi092 ceph-mon[91933]: from='osd.1 [v2:172.21.15.92:6810/1875265240,v1:172.21.15.92:6811/1875265240]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-06T12:18:42.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:41 smithi190 ceph-mon[79744]: pgmap v71: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 323 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 57/372 objects degraded (15.323%) 2024-05-06T12:18:42.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:41 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:42.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:41 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:42.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:41 smithi190 ceph-mon[79744]: from='osd.1 [v2:172.21.15.92:6810/1875265240,v1:172.21.15.92:6811/1875265240]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-06T12:18:42.472 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:18:42 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1[106225]: 2024-05-06T12:18:42.193+0000 7f870347c700 -1 osd.1 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-06T12:18:43.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:43 smithi092 ceph-mon[91933]: from='osd.1 [v2:172.21.15.92:6810/1875265240,v1:172.21.15.92:6811/1875265240]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-06T12:18:43.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:43 smithi092 ceph-mon[91933]: osdmap e58: 6 total, 5 up, 6 in 2024-05-06T12:18:43.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:43 smithi092 ceph-mon[91933]: from='osd.1 [v2:172.21.15.92:6810/1875265240,v1:172.21.15.92:6811/1875265240]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-06T12:18:43.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:43 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:43.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:43 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:43.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:43 smithi092 ceph-mon[91933]: pgmap v73: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 57/372 objects degraded (15.323%) 2024-05-06T12:18:43.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:43 smithi190 ceph-mon[79744]: from='osd.1 [v2:172.21.15.92:6810/1875265240,v1:172.21.15.92:6811/1875265240]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-06T12:18:43.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:43 smithi190 ceph-mon[79744]: osdmap e58: 6 total, 5 up, 6 in 2024-05-06T12:18:43.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:43 smithi190 ceph-mon[79744]: from='osd.1 [v2:172.21.15.92:6810/1875265240,v1:172.21.15.92:6811/1875265240]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-06T12:18:43.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:43 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:43.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:43 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:43.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:43 smithi190 ceph-mon[79744]: pgmap v73: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 57/372 objects degraded (15.323%) 2024-05-06T12:18:44.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:44 smithi092 ceph-mon[91933]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-06T12:18:44.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:44 smithi092 ceph-mon[91933]: osd.1 [v2:172.21.15.92:6810/1875265240,v1:172.21.15.92:6811/1875265240] boot 2024-05-06T12:18:44.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:44 smithi092 ceph-mon[91933]: osdmap e59: 6 total, 6 up, 6 in 2024-05-06T12:18:44.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:44 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-06T12:18:44.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:44 smithi092 ceph-mon[91933]: osdmap e60: 6 total, 6 up, 6 in 2024-05-06T12:18:44.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:44 smithi190 ceph-mon[79744]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-06T12:18:44.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:44 smithi190 ceph-mon[79744]: osd.1 [v2:172.21.15.92:6810/1875265240,v1:172.21.15.92:6811/1875265240] boot 2024-05-06T12:18:44.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:44 smithi190 ceph-mon[79744]: osdmap e59: 6 total, 6 up, 6 in 2024-05-06T12:18:44.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:44 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-06T12:18:44.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:44 smithi190 ceph-mon[79744]: osdmap e60: 6 total, 6 up, 6 in 2024-05-06T12:18:45.330 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:45 smithi092 ceph-mon[91933]: pgmap v76: 65 pgs: 7 active+undersized+degraded, 7 active+undersized, 51 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 3.2 KiB/s rd, 5 op/s; 24/372 objects degraded (6.452%) 2024-05-06T12:18:45.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:45 smithi190 ceph-mon[79744]: pgmap v76: 65 pgs: 7 active+undersized+degraded, 7 active+undersized, 51 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 3.2 KiB/s rd, 5 op/s; 24/372 objects degraded (6.452%) 2024-05-06T12:18:46.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:46 smithi092 ceph-mon[91933]: Health check update: Degraded data redundancy: 24/372 objects degraded (6.452%), 7 pgs degraded (PG_DEGRADED) 2024-05-06T12:18:46.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:46 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:46.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:46 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:46.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:46 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:18:46.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:46 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:18:46.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:46 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:46.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:46 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:18:46.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:46 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:18:46.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:46 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:18:46.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:46 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:18:46.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:46 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-06T12:18:46.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:46 smithi092 ceph-mon[91933]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-06T12:18:46.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:46 smithi092 ceph-mon[91933]: Upgrade: unsafe to stop osd(s) at this time (3 PGs are or would become offline) 2024-05-06T12:18:46.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:46 smithi190 ceph-mon[79744]: Health check update: Degraded data redundancy: 24/372 objects degraded (6.452%), 7 pgs degraded (PG_DEGRADED) 2024-05-06T12:18:46.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:46 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:46.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:46 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:46.600 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:46 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:18:46.600 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:46 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:18:46.600 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:46 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:46.600 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:46 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:18:46.600 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:46 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:18:46.600 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:46 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:18:46.600 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:46 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:18:46.600 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:46 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-06T12:18:46.600 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:46 smithi190 ceph-mon[79744]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-06T12:18:46.600 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:46 smithi190 ceph-mon[79744]: Upgrade: unsafe to stop osd(s) at this time (3 PGs are or would become offline) 2024-05-06T12:18:47.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:47 smithi092 ceph-mon[91933]: pgmap v77: 65 pgs: 7 active+undersized+degraded, 5 active+undersized, 53 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s; 24/372 objects degraded (6.452%) 2024-05-06T12:18:47.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:47 smithi190 ceph-mon[79744]: pgmap v77: 65 pgs: 7 active+undersized+degraded, 5 active+undersized, 53 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s; 24/372 objects degraded (6.452%) 2024-05-06T12:18:49.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:48 smithi092 ceph-mon[91933]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 24/372 objects degraded (6.452%), 7 pgs degraded) 2024-05-06T12:18:49.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:48 smithi190 ceph-mon[79744]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 24/372 objects degraded (6.452%), 7 pgs degraded) 2024-05-06T12:18:49.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.816+0000 7fc30027d700 1 -- 172.21.15.92:0/1457733548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f80fbcb0 msgr2=0x7fc2f80fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:49.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.816+0000 7fc30027d700 1 --2- 172.21.15.92:0/1457733548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f80fbcb0 0x7fc2f80fc0c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8004940 tx=0x7fc2e8009e30 comp rx=0 tx=0).stop 2024-05-06T12:18:49.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.816+0000 7fc30027d700 1 -- 172.21.15.92:0/1457733548 shutdown_connections 2024-05-06T12:18:49.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.816+0000 7fc30027d700 1 --2- 172.21.15.92:0/1457733548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f80fcf60 0x7fc2f80fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:49.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.816+0000 7fc30027d700 1 --2- 172.21.15.92:0/1457733548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f80fbcb0 0x7fc2f80fc0c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:49.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.816+0000 7fc30027d700 1 -- 172.21.15.92:0/1457733548 >> 172.21.15.92:0/1457733548 conn(0x7fc2f80f7430 msgr2=0x7fc2f80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:18:49.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.816+0000 7fc30027d700 1 -- 172.21.15.92:0/1457733548 shutdown_connections 2024-05-06T12:18:49.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.816+0000 7fc30027d700 1 -- 172.21.15.92:0/1457733548 wait complete. 2024-05-06T12:18:49.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.817+0000 7fc30027d700 1 Processor -- start 2024-05-06T12:18:49.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.817+0000 7fc30027d700 1 -- start start 2024-05-06T12:18:49.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.817+0000 7fc30027d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f80fbcb0 0x7fc2f80ffd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:49.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.817+0000 7fc30027d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f80fcf60 0x7fc2f80fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:49.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.817+0000 7fc30027d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f80fe870 con 0x7fc2f80fbcb0 2024-05-06T12:18:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.817+0000 7fc30027d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f80fe9b0 con 0x7fc2f80fcf60 2024-05-06T12:18:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.817+0000 7fc2fe019700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f80fbcb0 0x7fc2f80ffd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.817+0000 7fc2fe019700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f80fbcb0 0x7fc2f80ffd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44312/0 (socket says 172.21.15.92:44312) 2024-05-06T12:18:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.817+0000 7fc2fe019700 1 -- 172.21.15.92:0/4030760467 learned_addr learned my addr 172.21.15.92:0/4030760467 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:18:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.817+0000 7fc2fd818700 1 --2- 172.21.15.92:0/4030760467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f80fcf60 0x7fc2f80fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.817+0000 7fc2fe019700 1 -- 172.21.15.92:0/4030760467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f80fcf60 msgr2=0x7fc2f80fe2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.817+0000 7fc2fe019700 1 --2- 172.21.15.92:0/4030760467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f80fcf60 0x7fc2f80fe2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.817+0000 7fc2fe019700 1 -- 172.21.15.92:0/4030760467 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e8005040 con 0x7fc2f80fbcb0 2024-05-06T12:18:49.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.817+0000 7fc2fd818700 1 --2- 172.21.15.92:0/4030760467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f80fcf60 0x7fc2f80fe2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-06T12:18:49.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.818+0000 7fc2fe019700 1 --2- 172.21.15.92:0/4030760467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f80fbcb0 0x7fc2f80ffd30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800b3b0 tx=0x7fc2e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:18:49.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.818+0000 7fc2eeffd700 1 -- 172.21.15.92:0/4030760467 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2e800e040 con 0x7fc2f80fbcb0 2024-05-06T12:18:49.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.818+0000 7fc30027d700 1 -- 172.21.15.92:0/4030760467 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f80fec30 con 0x7fc2f80fbcb0 2024-05-06T12:18:49.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.818+0000 7fc2eeffd700 1 -- 172.21.15.92:0/4030760467 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc2e800bcc0 con 0x7fc2f80fbcb0 2024-05-06T12:18:49.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.818+0000 7fc30027d700 1 -- 172.21.15.92:0/4030760467 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f8115b00 con 0x7fc2f80fbcb0 2024-05-06T12:18:49.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.818+0000 7fc2eeffd700 1 -- 172.21.15.92:0/4030760467 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2e801c630 con 0x7fc2f80fbcb0 2024-05-06T12:18:49.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.819+0000 7fc30027d700 1 -- 172.21.15.92:0/4030760467 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2f8101af0 con 0x7fc2f80fbcb0 2024-05-06T12:18:49.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.819+0000 7fc2eeffd700 1 -- 172.21.15.92:0/4030760467 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc2e8013980 con 0x7fc2f80fbcb0 2024-05-06T12:18:49.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.820+0000 7fc2eeffd700 1 --2- 172.21.15.92:0/4030760467 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fc2e406e140 0x7fc2e40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:49.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.820+0000 7fc2eeffd700 1 -- 172.21.15.92:0/4030760467 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc2e808e850 con 0x7fc2f80fbcb0 2024-05-06T12:18:49.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.820+0000 7fc2fd818700 1 --2- 172.21.15.92:0/4030760467 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fc2e406e140 0x7fc2e40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:49.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.820+0000 7fc2fd818700 1 --2- 172.21.15.92:0/4030760467 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fc2e406e140 0x7fc2e40705f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4006fd0 tx=0x7fc2f4008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:18:49.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:49.825+0000 7fc2eeffd700 1 -- 172.21.15.92:0/4030760467 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc2e8057bd0 con 0x7fc2f80fbcb0 2024-05-06T12:18:50.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.042+0000 7fc30027d700 1 -- 172.21.15.92:0/4030760467 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc2f8104990 con 0x7fc2e406e140 2024-05-06T12:18:50.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.043+0000 7fc2eeffd700 1 -- 172.21.15.92:0/4030760467 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fc2f8104990 con 0x7fc2e406e140 2024-05-06T12:18:50.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.045+0000 7fc30027d700 1 -- 172.21.15.92:0/4030760467 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fc2e406e140 msgr2=0x7fc2e40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:50.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.045+0000 7fc30027d700 1 --2- 172.21.15.92:0/4030760467 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fc2e406e140 0x7fc2e40705f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4006fd0 tx=0x7fc2f4008040 comp rx=0 tx=0).stop 2024-05-06T12:18:50.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.045+0000 7fc30027d700 1 -- 172.21.15.92:0/4030760467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f80fbcb0 msgr2=0x7fc2f80ffd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:50.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.045+0000 7fc30027d700 1 --2- 172.21.15.92:0/4030760467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f80fbcb0 0x7fc2f80ffd30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800b3b0 tx=0x7fc2e8009e30 comp rx=0 tx=0).stop 2024-05-06T12:18:50.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.045+0000 7fc30027d700 1 -- 172.21.15.92:0/4030760467 shutdown_connections 2024-05-06T12:18:50.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.045+0000 7fc30027d700 1 --2- 172.21.15.92:0/4030760467 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fc2e406e140 0x7fc2e40705f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:50.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.045+0000 7fc30027d700 1 --2- 172.21.15.92:0/4030760467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f80fbcb0 0x7fc2f80ffd30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:50.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.045+0000 7fc30027d700 1 --2- 172.21.15.92:0/4030760467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f80fcf60 0x7fc2f80fe2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:50.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.045+0000 7fc30027d700 1 -- 172.21.15.92:0/4030760467 >> 172.21.15.92:0/4030760467 conn(0x7fc2f80f7430 msgr2=0x7fc2f8065b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:18:50.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.046+0000 7fc30027d700 1 -- 172.21.15.92:0/4030760467 shutdown_connections 2024-05-06T12:18:50.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.046+0000 7fc30027d700 1 -- 172.21.15.92:0/4030760467 wait complete. 2024-05-06T12:18:50.058 INFO:teuthology.orchestra.run.smithi092.stdout:true 2024-05-06T12:18:50.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.150+0000 7f6abd246700 1 -- 172.21.15.92:0/2321820581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ab80fce90 msgr2=0x7f6ab80fd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:50.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.150+0000 7f6abd246700 1 --2- 172.21.15.92:0/2321820581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ab80fce90 0x7f6ab80fd2a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8004940 tx=0x7f6aa8009e30 comp rx=0 tx=0).stop 2024-05-06T12:18:50.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.150+0000 7f6abd246700 1 -- 172.21.15.92:0/2321820581 shutdown_connections 2024-05-06T12:18:50.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.150+0000 7f6abd246700 1 --2- 172.21.15.92:0/2321820581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ab80fe090 0x7f6ab80fe500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:50.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.150+0000 7f6abd246700 1 --2- 172.21.15.92:0/2321820581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ab80fce90 0x7f6ab80fd2a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:50.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.150+0000 7f6abd246700 1 -- 172.21.15.92:0/2321820581 >> 172.21.15.92:0/2321820581 conn(0x7f6ab80f85c0 msgr2=0x7f6ab80fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:18:50.153 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:49 smithi092 ceph-mon[91933]: pgmap v78: 65 pgs: 65 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.4 KiB/s rd, 2 op/s 2024-05-06T12:18:50.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.150+0000 7f6abd246700 1 -- 172.21.15.92:0/2321820581 shutdown_connections 2024-05-06T12:18:50.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.150+0000 7f6abd246700 1 -- 172.21.15.92:0/2321820581 wait complete. 2024-05-06T12:18:50.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.150+0000 7f6abd246700 1 Processor -- start 2024-05-06T12:18:50.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.150+0000 7f6abd246700 1 -- start start 2024-05-06T12:18:50.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.150+0000 7f6abd246700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ab80fce90 0x7f6ab810da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:50.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.150+0000 7f6abd246700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ab80fe090 0x7f6ab810df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:50.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.150+0000 7f6abd246700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab810e600 con 0x7f6ab80fe090 2024-05-06T12:18:50.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.151+0000 7f6abd246700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab810edf0 con 0x7f6ab80fce90 2024-05-06T12:18:50.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.151+0000 7f6aaffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ab80fe090 0x7f6ab810df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:50.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.151+0000 7f6aaffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ab80fe090 0x7f6ab810df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44330/0 (socket says 172.21.15.92:44330) 2024-05-06T12:18:50.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.151+0000 7f6aaffff700 1 -- 172.21.15.92:0/3547497636 learned_addr learned my addr 172.21.15.92:0/3547497636 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:18:50.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.151+0000 7f6ab6d9d700 1 --2- 172.21.15.92:0/3547497636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ab80fce90 0x7f6ab810da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:50.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.151+0000 7f6aaffff700 1 -- 172.21.15.92:0/3547497636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ab80fce90 msgr2=0x7f6ab810da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:50.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.151+0000 7f6aaffff700 1 --2- 172.21.15.92:0/3547497636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ab80fce90 0x7f6ab810da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:50.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.151+0000 7f6aaffff700 1 -- 172.21.15.92:0/3547497636 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa8005040 con 0x7f6ab80fe090 2024-05-06T12:18:50.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.151+0000 7f6aaffff700 1 --2- 172.21.15.92:0/3547497636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ab80fe090 0x7f6ab810df70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6aa000b790 tx=0x7f6aa0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:18:50.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.151+0000 7f6aaf7fe700 1 -- 172.21.15.92:0/3547497636 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6aa000f770 con 0x7f6ab80fe090 2024-05-06T12:18:50.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.151+0000 7f6abd246700 1 -- 172.21.15.92:0/3547497636 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ab810f0d0 con 0x7f6ab80fe090 2024-05-06T12:18:50.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.151+0000 7f6aaf7fe700 1 -- 172.21.15.92:0/3547497636 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6aa000fda0 con 0x7f6ab80fe090 2024-05-06T12:18:50.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.151+0000 7f6aaf7fe700 1 -- 172.21.15.92:0/3547497636 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6aa000d310 con 0x7f6ab80fe090 2024-05-06T12:18:50.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.151+0000 7f6abd246700 1 -- 172.21.15.92:0/3547497636 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ab810f690 con 0x7f6ab80fe090 2024-05-06T12:18:50.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.152+0000 7f6abd246700 1 -- 172.21.15.92:0/3547497636 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ab805f5c0 con 0x7f6ab80fe090 2024-05-06T12:18:50.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.153+0000 7f6aaf7fe700 1 -- 172.21.15.92:0/3547497636 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6aa0010420 con 0x7f6ab80fe090 2024-05-06T12:18:50.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.153+0000 7f6aaf7fe700 1 --2- 172.21.15.92:0/3547497636 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f6aa406e140 0x7f6aa40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:50.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.153+0000 7f6aaf7fe700 1 -- 172.21.15.92:0/3547497636 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f6aa008db40 con 0x7f6ab80fe090 2024-05-06T12:18:50.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.153+0000 7f6ab6d9d700 1 --2- 172.21.15.92:0/3547497636 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f6aa406e140 0x7f6aa40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:50.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.154+0000 7f6ab6d9d700 1 --2- 172.21.15.92:0/3547497636 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f6aa406e140 0x7f6aa40705f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6aa800b970 tx=0x7f6aa8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:18:50.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.158+0000 7f6aaf7fe700 1 -- 172.21.15.92:0/3547497636 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6aa005aa70 con 0x7f6ab80fe090 2024-05-06T12:18:50.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:49 smithi190 ceph-mon[79744]: pgmap v78: 65 pgs: 65 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.4 KiB/s rd, 2 op/s 2024-05-06T12:18:50.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.372+0000 7f6abd246700 1 -- 172.21.15.92:0/3547497636 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ab810f980 con 0x7f6aa406e140 2024-05-06T12:18:50.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.374+0000 7f6aaf7fe700 1 -- 172.21.15.92:0/3547497636 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f6ab810f980 con 0x7f6aa406e140 2024-05-06T12:18:50.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.376+0000 7f6abd246700 1 -- 172.21.15.92:0/3547497636 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f6aa406e140 msgr2=0x7f6aa40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:50.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.376+0000 7f6abd246700 1 --2- 172.21.15.92:0/3547497636 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f6aa406e140 0x7f6aa40705f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6aa800b970 tx=0x7f6aa8009e30 comp rx=0 tx=0).stop 2024-05-06T12:18:50.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.376+0000 7f6abd246700 1 -- 172.21.15.92:0/3547497636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ab80fe090 msgr2=0x7f6ab810df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:50.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.376+0000 7f6abd246700 1 --2- 172.21.15.92:0/3547497636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ab80fe090 0x7f6ab810df70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6aa000b790 tx=0x7f6aa0009250 comp rx=0 tx=0).stop 2024-05-06T12:18:50.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.376+0000 7f6abd246700 1 -- 172.21.15.92:0/3547497636 shutdown_connections 2024-05-06T12:18:50.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.376+0000 7f6abd246700 1 --2- 172.21.15.92:0/3547497636 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f6aa406e140 0x7f6aa40705f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:50.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.376+0000 7f6abd246700 1 --2- 172.21.15.92:0/3547497636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ab80fce90 0x7f6ab810da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:50.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.376+0000 7f6abd246700 1 --2- 172.21.15.92:0/3547497636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ab80fe090 0x7f6ab810df70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:50.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.376+0000 7f6abd246700 1 -- 172.21.15.92:0/3547497636 >> 172.21.15.92:0/3547497636 conn(0x7f6ab80f85c0 msgr2=0x7f6ab80f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:18:50.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.376+0000 7f6abd246700 1 -- 172.21.15.92:0/3547497636 shutdown_connections 2024-05-06T12:18:50.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.376+0000 7f6abd246700 1 -- 172.21.15.92:0/3547497636 wait complete. 2024-05-06T12:18:50.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.477+0000 7f48e568a700 1 -- 172.21.15.92:0/1534655350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48e00fce80 msgr2=0x7f48e00fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:50.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.477+0000 7f48e568a700 1 --2- 172.21.15.92:0/1534655350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48e00fce80 0x7f48e00fd290 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f48d8009ce0 tx=0x7f48d8004d10 comp rx=0 tx=0).stop 2024-05-06T12:18:50.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.477+0000 7f48e568a700 1 -- 172.21.15.92:0/1534655350 shutdown_connections 2024-05-06T12:18:50.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.477+0000 7f48e568a700 1 --2- 172.21.15.92:0/1534655350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48e00fe080 0x7f48e00fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:50.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.477+0000 7f48e568a700 1 --2- 172.21.15.92:0/1534655350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48e00fce80 0x7f48e00fd290 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:50.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.477+0000 7f48e568a700 1 -- 172.21.15.92:0/1534655350 >> 172.21.15.92:0/1534655350 conn(0x7f48e00f8590 msgr2=0x7f48e00fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:18:50.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.477+0000 7f48e568a700 1 -- 172.21.15.92:0/1534655350 shutdown_connections 2024-05-06T12:18:50.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.477+0000 7f48e568a700 1 -- 172.21.15.92:0/1534655350 wait complete. 2024-05-06T12:18:50.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.478+0000 7f48e568a700 1 Processor -- start 2024-05-06T12:18:50.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.478+0000 7f48e568a700 1 -- start start 2024-05-06T12:18:50.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.478+0000 7f48e568a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48e00fce80 0x7f48e010dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:50.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.478+0000 7f48e568a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48e00fe080 0x7f48e010e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:50.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.478+0000 7f48e568a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e010e6e0 con 0x7f48e00fce80 2024-05-06T12:18:50.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.478+0000 7f48e568a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e010ef30 con 0x7f48e00fe080 2024-05-06T12:18:50.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.478+0000 7f48deffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48e00fce80 0x7f48e010dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:50.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.478+0000 7f48deffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48e00fce80 0x7f48e010dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44346/0 (socket says 172.21.15.92:44346) 2024-05-06T12:18:50.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.478+0000 7f48deffd700 1 -- 172.21.15.92:0/1420452107 learned_addr learned my addr 172.21.15.92:0/1420452107 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:18:50.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.478+0000 7f48de7fc700 1 --2- 172.21.15.92:0/1420452107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48e00fe080 0x7f48e010e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:50.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.478+0000 7f48deffd700 1 -- 172.21.15.92:0/1420452107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48e00fe080 msgr2=0x7f48e010e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:50.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.478+0000 7f48deffd700 1 --2- 172.21.15.92:0/1420452107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48e00fe080 0x7f48e010e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:50.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.478+0000 7f48deffd700 1 -- 172.21.15.92:0/1420452107 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d8009940 con 0x7f48e00fce80 2024-05-06T12:18:50.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.478+0000 7f48deffd700 1 --2- 172.21.15.92:0/1420452107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48e00fce80 0x7f48e010dba0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f48d8009380 tx=0x7f48d8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:18:50.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.478+0000 7f48c77fe700 1 -- 172.21.15.92:0/1420452107 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48d800e040 con 0x7f48e00fce80 2024-05-06T12:18:50.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.479+0000 7f48c77fe700 1 -- 172.21.15.92:0/1420452107 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f48d80128e0 con 0x7f48e00fce80 2024-05-06T12:18:50.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.479+0000 7f48e568a700 1 -- 172.21.15.92:0/1420452107 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e010f1b0 con 0x7f48e00fce80 2024-05-06T12:18:50.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.479+0000 7f48c77fe700 1 -- 172.21.15.92:0/1420452107 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48d801c9f0 con 0x7f48e00fce80 2024-05-06T12:18:50.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.479+0000 7f48e568a700 1 -- 172.21.15.92:0/1420452107 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e010f6f0 con 0x7f48e00fce80 2024-05-06T12:18:50.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.480+0000 7f48e568a700 1 -- 172.21.15.92:0/1420452107 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48e005f5c0 con 0x7f48e00fce80 2024-05-06T12:18:50.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.480+0000 7f48c77fe700 1 -- 172.21.15.92:0/1420452107 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f48d8012a50 con 0x7f48e00fce80 2024-05-06T12:18:50.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.480+0000 7f48c77fe700 1 --2- 172.21.15.92:0/1420452107 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f48c806e140 0x7f48c80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:50.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.481+0000 7f48c77fe700 1 -- 172.21.15.92:0/1420452107 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f48d808f7b0 con 0x7f48e00fce80 2024-05-06T12:18:50.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.481+0000 7f48de7fc700 1 --2- 172.21.15.92:0/1420452107 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f48c806e140 0x7f48c80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:50.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.481+0000 7f48de7fc700 1 --2- 172.21.15.92:0/1420452107 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f48c806e140 0x7f48c80705f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f48d4007fd0 tx=0x7f48d4009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:18:50.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.485+0000 7f48c77fe700 1 -- 172.21.15.92:0/1420452107 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f48d805c660 con 0x7f48e00fce80 2024-05-06T12:18:50.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.692+0000 7f48e568a700 1 -- 172.21.15.92:0/1420452107 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f48e010fa20 con 0x7f48c806e140 2024-05-06T12:18:50.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.702+0000 7f48c77fe700 1 -- 172.21.15.92:0/1420452107 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f48e010fa20 con 0x7f48c806e140 2024-05-06T12:18:50.704 INFO:teuthology.orchestra.run.smithi092.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-06T12:18:50.704 INFO:teuthology.orchestra.run.smithi092.stdout:alertmanager.smithi092 smithi092 *:9093,9094 running (3m) 9s ago 12m 25.8M - 0.25.0 c8568f914cd2 be5637d7d25a 2024-05-06T12:18:50.704 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi092 smithi092 running (76s) 9s ago 12m 6660k - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 8da0494e1bdc 2024-05-06T12:18:50.704 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi190 smithi190 running (71s) 67s ago 11m 6665k - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 641a7cdaa56e 2024-05-06T12:18:50.704 INFO:teuthology.orchestra.run.smithi092.stdout:grafana.smithi092 smithi092 *:3000 running (3m) 9s ago 12m 73.9M - 9.4.7 954c08fa6188 9180cb3e5258 2024-05-06T12:18:50.704 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.ijsqgp smithi092 running (7m) 9s ago 7m 60.7M - 17.2.7-925-ge81df420 96d5ae7fd4b4 b4c2ed6d2c82 2024-05-06T12:18:50.705 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.rosbll smithi092 running (6m) 9s ago 6m 122M - 17.2.7-925-ge81df420 96d5ae7fd4b4 5f5426cd1bf5 2024-05-06T12:18:50.705 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.byjvgf smithi190 running (6m) 67s ago 6m 126M - 17.2.7-925-ge81df420 96d5ae7fd4b4 24581d2e1c5d 2024-05-06T12:18:50.705 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.idzoff smithi190 running (6m) 67s ago 6m 64.3M - 17.2.7-925-ge81df420 96d5ae7fd4b4 20f2641696ef 2024-05-06T12:18:50.705 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi092.gyzeqb smithi092 *:8443,9283,8765 running (3m) 9s ago 13m 502M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 de9d2198327a 2024-05-06T12:18:50.705 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi190.kczxkh smithi190 *:8443,9283,8765 running (2m) 67s ago 11m 449M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 e84ab3758ac7 2024-05-06T12:18:50.705 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi092 smithi092 running (2m) 9s ago 13m 43.9M 2048M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 ac07f7158087 2024-05-06T12:18:50.705 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi190 smithi190 running (117s) 67s ago 11m 34.0M 2048M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 7a3c08eb716f 2024-05-06T12:18:50.705 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi092 smithi092 *:9100 running (4m) 9s ago 12m 14.7M - 1.5.0 0da6a335fe13 c796446b206f 2024-05-06T12:18:50.705 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi190 smithi190 *:9100 running (3m) 67s ago 11m 16.9M - 1.5.0 0da6a335fe13 647e28111a76 2024-05-06T12:18:50.705 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 smithi092 running (51s) 9s ago 10m 81.5M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 8c96c75b25f4 2024-05-06T12:18:50.705 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 smithi092 running (15s) 9s ago 10m 11.9M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 e8007a510f08 2024-05-06T12:18:50.705 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 smithi092 running (9m) 9s ago 9m 164M 1557M 17.2.7-925-ge81df420 96d5ae7fd4b4 1e6ed04a0688 2024-05-06T12:18:50.705 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 smithi190 running (9m) 67s ago 9m 221M 2581M 17.2.7-925-ge81df420 96d5ae7fd4b4 52f9521286d5 2024-05-06T12:18:50.706 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 smithi190 running (8m) 67s ago 8m 204M 2581M 17.2.7-925-ge81df420 96d5ae7fd4b4 3a7da09e4c48 2024-05-06T12:18:50.706 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 smithi190 running (8m) 67s ago 8m 188M 2581M 17.2.7-925-ge81df420 96d5ae7fd4b4 e7452c6b5cec 2024-05-06T12:18:50.706 INFO:teuthology.orchestra.run.smithi092.stdout:prometheus.smithi092 smithi092 *:9095 running (2m) 9s ago 11m 51.7M - 2.43.0 a07b618ecd1d bc5445691527 2024-05-06T12:18:50.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.704+0000 7f48e568a700 1 -- 172.21.15.92:0/1420452107 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f48c806e140 msgr2=0x7f48c80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:50.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.704+0000 7f48e568a700 1 --2- 172.21.15.92:0/1420452107 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f48c806e140 0x7f48c80705f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f48d4007fd0 tx=0x7f48d4009040 comp rx=0 tx=0).stop 2024-05-06T12:18:50.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.704+0000 7f48e568a700 1 -- 172.21.15.92:0/1420452107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48e00fce80 msgr2=0x7f48e010dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:50.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.704+0000 7f48e568a700 1 --2- 172.21.15.92:0/1420452107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48e00fce80 0x7f48e010dba0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f48d8009380 tx=0x7f48d8004d10 comp rx=0 tx=0).stop 2024-05-06T12:18:50.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.704+0000 7f48e568a700 1 -- 172.21.15.92:0/1420452107 shutdown_connections 2024-05-06T12:18:50.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.704+0000 7f48e568a700 1 --2- 172.21.15.92:0/1420452107 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f48c806e140 0x7f48c80705f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:50.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.704+0000 7f48e568a700 1 --2- 172.21.15.92:0/1420452107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48e00fce80 0x7f48e010dba0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:50.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.704+0000 7f48e568a700 1 --2- 172.21.15.92:0/1420452107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48e00fe080 0x7f48e010e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:50.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.704+0000 7f48e568a700 1 -- 172.21.15.92:0/1420452107 >> 172.21.15.92:0/1420452107 conn(0x7f48e00f8590 msgr2=0x7f48e00f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:18:50.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.704+0000 7f48e568a700 1 -- 172.21.15.92:0/1420452107 shutdown_connections 2024-05-06T12:18:50.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.704+0000 7f48e568a700 1 -- 172.21.15.92:0/1420452107 wait complete. 2024-05-06T12:18:50.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.807+0000 7fdc823ef700 1 -- 172.21.15.92:0/3233869951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc7c071de0 msgr2=0x7fdc7c0702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:50.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.807+0000 7fdc823ef700 1 --2- 172.21.15.92:0/3233869951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc7c071de0 0x7fdc7c0702f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c004940 tx=0x7fdc6c009e30 comp rx=0 tx=0).stop 2024-05-06T12:18:50.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.807+0000 7fdc823ef700 1 -- 172.21.15.92:0/3233869951 shutdown_connections 2024-05-06T12:18:50.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.807+0000 7fdc823ef700 1 --2- 172.21.15.92:0/3233869951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc7c070830 0x7fdc7c070ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:50.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.807+0000 7fdc823ef700 1 --2- 172.21.15.92:0/3233869951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc7c071de0 0x7fdc7c0702f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:50.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.807+0000 7fdc823ef700 1 -- 172.21.15.92:0/3233869951 >> 172.21.15.92:0/3233869951 conn(0x7fdc7c0f8520 msgr2=0x7fdc7c0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:18:50.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.807+0000 7fdc823ef700 1 -- 172.21.15.92:0/3233869951 shutdown_connections 2024-05-06T12:18:50.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.807+0000 7fdc823ef700 1 -- 172.21.15.92:0/3233869951 wait complete. 2024-05-06T12:18:50.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.807+0000 7fdc823ef700 1 Processor -- start 2024-05-06T12:18:50.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.807+0000 7fdc823ef700 1 -- start start 2024-05-06T12:18:50.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.808+0000 7fdc823ef700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc7c070830 0x7fdc7c10fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:50.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.808+0000 7fdc823ef700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc7c071de0 0x7fdc7c110080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:50.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.808+0000 7fdc823ef700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c110680 con 0x7fdc7c071de0 2024-05-06T12:18:50.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.808+0000 7fdc823ef700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c1107f0 con 0x7fdc7c070830 2024-05-06T12:18:50.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.808+0000 7fdc7b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc7c071de0 0x7fdc7c110080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:50.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.808+0000 7fdc7b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc7c071de0 0x7fdc7c110080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44356/0 (socket says 172.21.15.92:44356) 2024-05-06T12:18:50.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.808+0000 7fdc7b7fe700 1 -- 172.21.15.92:0/2137552285 learned_addr learned my addr 172.21.15.92:0/2137552285 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:18:50.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.808+0000 7fdc7bfff700 1 --2- 172.21.15.92:0/2137552285 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc7c070830 0x7fdc7c10fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:50.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.808+0000 7fdc7b7fe700 1 -- 172.21.15.92:0/2137552285 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc7c070830 msgr2=0x7fdc7c10fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:50.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.808+0000 7fdc7b7fe700 1 --2- 172.21.15.92:0/2137552285 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc7c070830 0x7fdc7c10fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:50.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.808+0000 7fdc7b7fe700 1 -- 172.21.15.92:0/2137552285 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc6c005040 con 0x7fdc7c071de0 2024-05-06T12:18:50.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.808+0000 7fdc7b7fe700 1 --2- 172.21.15.92:0/2137552285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc7c071de0 0x7fdc7c110080 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdc70002a50 tx=0x7fdc700046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:18:50.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.808+0000 7fdc78ff9700 1 -- 172.21.15.92:0/2137552285 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc70007cc0 con 0x7fdc7c071de0 2024-05-06T12:18:50.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.808+0000 7fdc78ff9700 1 -- 172.21.15.92:0/2137552285 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc7000f040 con 0x7fdc7c071de0 2024-05-06T12:18:50.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.808+0000 7fdc823ef700 1 -- 172.21.15.92:0/2137552285 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc7c115250 con 0x7fdc7c071de0 2024-05-06T12:18:50.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.808+0000 7fdc78ff9700 1 -- 172.21.15.92:0/2137552285 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc7000da30 con 0x7fdc7c071de0 2024-05-06T12:18:50.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.809+0000 7fdc823ef700 1 -- 172.21.15.92:0/2137552285 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc7c115810 con 0x7fdc7c071de0 2024-05-06T12:18:50.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.810+0000 7fdc823ef700 1 -- 172.21.15.92:0/2137552285 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc7c04c710 con 0x7fdc7c071de0 2024-05-06T12:18:50.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.810+0000 7fdc78ff9700 1 -- 172.21.15.92:0/2137552285 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdc7000dbb0 con 0x7fdc7c071de0 2024-05-06T12:18:50.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.810+0000 7fdc78ff9700 1 --2- 172.21.15.92:0/2137552285 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fdc6406e140 0x7fdc640705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:50.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.810+0000 7fdc78ff9700 1 -- 172.21.15.92:0/2137552285 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fdc7008d8b0 con 0x7fdc7c071de0 2024-05-06T12:18:50.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.810+0000 7fdc7bfff700 1 --2- 172.21.15.92:0/2137552285 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fdc6406e140 0x7fdc640705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:50.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.811+0000 7fdc7bfff700 1 --2- 172.21.15.92:0/2137552285 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fdc6406e140 0x7fdc640705f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c004890 tx=0x7fdc6c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:18:50.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:50.815+0000 7fdc78ff9700 1 -- 172.21.15.92:0/2137552285 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fdc7005a760 con 0x7fdc7c071de0 2024-05-06T12:18:51.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.096+0000 7fdc823ef700 1 -- 172.21.15.92:0/2137552285 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdc7c107ac0 con 0x7fdc7c071de0 2024-05-06T12:18:51.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.097+0000 7fdc78ff9700 1 -- 172.21.15.92:0/2137552285 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fdc7005a2f0 con 0x7fdc7c071de0 2024-05-06T12:18:51.099 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:18:51.099 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": { 2024-05-06T12:18:51.099 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:18:51.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:18:51.099 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": { 2024-05-06T12:18:51.099 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:18:51.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:18:51.100 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": { 2024-05-06T12:18:51.100 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 4, 2024-05-06T12:18:51.100 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:18:51.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:18:51.100 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-05-06T12:18:51.100 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 4 2024-05-06T12:18:51.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:18:51.100 INFO:teuthology.orchestra.run.smithi092.stdout: "overall": { 2024-05-06T12:18:51.100 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 8, 2024-05-06T12:18:51.100 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 6 2024-05-06T12:18:51.100 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-06T12:18:51.100 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:18:51.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.099+0000 7fdc823ef700 1 -- 172.21.15.92:0/2137552285 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fdc6406e140 msgr2=0x7fdc640705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:51.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.099+0000 7fdc823ef700 1 --2- 172.21.15.92:0/2137552285 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fdc6406e140 0x7fdc640705f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c004890 tx=0x7fdc6c009e30 comp rx=0 tx=0).stop 2024-05-06T12:18:51.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.099+0000 7fdc823ef700 1 -- 172.21.15.92:0/2137552285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc7c071de0 msgr2=0x7fdc7c110080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:51.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.099+0000 7fdc823ef700 1 --2- 172.21.15.92:0/2137552285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc7c071de0 0x7fdc7c110080 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdc70002a50 tx=0x7fdc700046b0 comp rx=0 tx=0).stop 2024-05-06T12:18:51.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.099+0000 7fdc823ef700 1 -- 172.21.15.92:0/2137552285 shutdown_connections 2024-05-06T12:18:51.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.099+0000 7fdc823ef700 1 --2- 172.21.15.92:0/2137552285 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fdc6406e140 0x7fdc640705f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:51.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.099+0000 7fdc823ef700 1 --2- 172.21.15.92:0/2137552285 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc7c070830 0x7fdc7c10fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:51.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.099+0000 7fdc823ef700 1 --2- 172.21.15.92:0/2137552285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc7c071de0 0x7fdc7c110080 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:51.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.099+0000 7fdc823ef700 1 -- 172.21.15.92:0/2137552285 >> 172.21.15.92:0/2137552285 conn(0x7fdc7c0f8520 msgr2=0x7fdc7c0f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:18:51.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.099+0000 7fdc823ef700 1 -- 172.21.15.92:0/2137552285 shutdown_connections 2024-05-06T12:18:51.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.099+0000 7fdc823ef700 1 -- 172.21.15.92:0/2137552285 wait complete. 2024-05-06T12:18:51.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.199+0000 7f4ae9184700 1 -- 172.21.15.92:0/1609473371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ae40fcf10 msgr2=0x7f4ae40fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:51.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.199+0000 7f4ae9184700 1 --2- 172.21.15.92:0/1609473371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ae40fcf10 0x7f4ae40fd320 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8004940 tx=0x7f4ad8009e30 comp rx=0 tx=0).stop 2024-05-06T12:18:51.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.199+0000 7f4ae9184700 1 -- 172.21.15.92:0/1609473371 shutdown_connections 2024-05-06T12:18:51.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.199+0000 7f4ae9184700 1 --2- 172.21.15.92:0/1609473371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ae40fe080 0x7f4ae40fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:51.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.199+0000 7f4ae9184700 1 --2- 172.21.15.92:0/1609473371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ae40fcf10 0x7f4ae40fd320 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:51.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.199+0000 7f4ae9184700 1 -- 172.21.15.92:0/1609473371 >> 172.21.15.92:0/1609473371 conn(0x7f4ae40f8560 msgr2=0x7f4ae40fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:18:51.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.199+0000 7f4ae9184700 1 -- 172.21.15.92:0/1609473371 shutdown_connections 2024-05-06T12:18:51.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.199+0000 7f4ae9184700 1 -- 172.21.15.92:0/1609473371 wait complete. 2024-05-06T12:18:51.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.200+0000 7f4ae9184700 1 Processor -- start 2024-05-06T12:18:51.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.200+0000 7f4ae9184700 1 -- start start 2024-05-06T12:18:51.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.200+0000 7f4ae9184700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ae40fcf10 0x7f4ae4071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:51.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.200+0000 7f4ae9184700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ae40fe080 0x7f4ae40702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:51.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.200+0000 7f4ae9184700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae40708f0 con 0x7f4ae40fcf10 2024-05-06T12:18:51.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.200+0000 7f4ae9184700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae4070a60 con 0x7f4ae40fe080 2024-05-06T12:18:51.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.200+0000 7f4ae2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ae40fcf10 0x7f4ae4071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:51.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.200+0000 7f4ae2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ae40fcf10 0x7f4ae4071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44370/0 (socket says 172.21.15.92:44370) 2024-05-06T12:18:51.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.200+0000 7f4ae2d9d700 1 -- 172.21.15.92:0/1079600215 learned_addr learned my addr 172.21.15.92:0/1079600215 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:18:51.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.200+0000 7f4ae259c700 1 --2- 172.21.15.92:0/1079600215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ae40fe080 0x7f4ae40702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:51.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.200+0000 7f4ae2d9d700 1 -- 172.21.15.92:0/1079600215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ae40fe080 msgr2=0x7f4ae40702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:51.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.200+0000 7f4ae2d9d700 1 --2- 172.21.15.92:0/1079600215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ae40fe080 0x7f4ae40702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:51.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.200+0000 7f4ae2d9d700 1 -- 172.21.15.92:0/1079600215 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad8005040 con 0x7f4ae40fcf10 2024-05-06T12:18:51.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.201+0000 7f4ae2d9d700 1 --2- 172.21.15.92:0/1079600215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ae40fcf10 0x7f4ae4071de0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800b3b0 tx=0x7f4ad8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:18:51.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.201+0000 7f4acb7fe700 1 -- 172.21.15.92:0/1079600215 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ad800e040 con 0x7f4ae40fcf10 2024-05-06T12:18:51.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.201+0000 7f4ae9184700 1 -- 172.21.15.92:0/1079600215 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ae4070ce0 con 0x7f4ae40fcf10 2024-05-06T12:18:51.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.201+0000 7f4acb7fe700 1 -- 172.21.15.92:0/1079600215 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ad800bcc0 con 0x7f4ae40fcf10 2024-05-06T12:18:51.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.201+0000 7f4acb7fe700 1 -- 172.21.15.92:0/1079600215 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ad801c630 con 0x7f4ae40fcf10 2024-05-06T12:18:51.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.201+0000 7f4ae9184700 1 -- 172.21.15.92:0/1079600215 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ae4115a10 con 0x7f4ae40fcf10 2024-05-06T12:18:51.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.202+0000 7f4ae9184700 1 -- 172.21.15.92:0/1079600215 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ae405f5c0 con 0x7f4ae40fcf10 2024-05-06T12:18:51.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.202+0000 7f4acb7fe700 1 -- 172.21.15.92:0/1079600215 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4ad8013980 con 0x7f4ae40fcf10 2024-05-06T12:18:51.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.203+0000 7f4acb7fe700 1 --2- 172.21.15.92:0/1079600215 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4acc06e140 0x7f4acc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:51.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.203+0000 7f4acb7fe700 1 -- 172.21.15.92:0/1079600215 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f4ad808e830 con 0x7f4ae40fcf10 2024-05-06T12:18:51.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.203+0000 7f4ae259c700 1 --2- 172.21.15.92:0/1079600215 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4acc06e140 0x7f4acc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:51.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.203+0000 7f4ae259c700 1 --2- 172.21.15.92:0/1079600215 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4acc06e140 0x7f4acc0705f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009b10 tx=0x7f4ad4008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:18:51.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.207+0000 7f4acb7fe700 1 -- 172.21.15.92:0/1079600215 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4ad8057bb0 con 0x7f4ae40fcf10 2024-05-06T12:18:51.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.449+0000 7f4ae9184700 1 -- 172.21.15.92:0/1079600215 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4ae4115c60 con 0x7f4ae40fcf10 2024-05-06T12:18:51.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.449+0000 7f4acb7fe700 1 -- 172.21.15.92:0/1079600215 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1935 (secure 0 0 0) 0x7f4ad805b270 con 0x7f4ae40fcf10 2024-05-06T12:18:51.452 INFO:teuthology.orchestra.run.smithi092.stdout:e21 2024-05-06T12:18:51.452 INFO:teuthology.orchestra.run.smithi092.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-06T12:18:51.452 INFO:teuthology.orchestra.run.smithi092.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:18:51.452 INFO:teuthology.orchestra.run.smithi092.stdout:legacy client fscid: 1 2024-05-06T12:18:51.452 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:18:51.452 INFO:teuthology.orchestra.run.smithi092.stdout:Filesystem 'cephfs' (1) 2024-05-06T12:18:51.452 INFO:teuthology.orchestra.run.smithi092.stdout:fs_name cephfs 2024-05-06T12:18:51.452 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 21 2024-05-06T12:18:51.452 INFO:teuthology.orchestra.run.smithi092.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-06T12:18:51.452 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-05-06T12:11:44.952671+0000 2024-05-06T12:18:51.453 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2024-05-06T12:18:36.975472+0000 2024-05-06T12:18:51.453 INFO:teuthology.orchestra.run.smithi092.stdout:tableserver 0 2024-05-06T12:18:51.453 INFO:teuthology.orchestra.run.smithi092.stdout:root 0 2024-05-06T12:18:51.453 INFO:teuthology.orchestra.run.smithi092.stdout:session_timeout 60 2024-05-06T12:18:51.453 INFO:teuthology.orchestra.run.smithi092.stdout:session_autoclose 300 2024-05-06T12:18:51.453 INFO:teuthology.orchestra.run.smithi092.stdout:max_file_size 1099511627776 2024-05-06T12:18:51.453 INFO:teuthology.orchestra.run.smithi092.stdout:max_xattr_size 65536 2024-05-06T12:18:51.453 INFO:teuthology.orchestra.run.smithi092.stdout:required_client_features {} 2024-05-06T12:18:51.453 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure 0 2024-05-06T12:18:51.453 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure_osd_epoch 46 2024-05-06T12:18:51.453 INFO:teuthology.orchestra.run.smithi092.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:18:51.453 INFO:teuthology.orchestra.run.smithi092.stdout:max_mds 2 2024-05-06T12:18:51.453 INFO:teuthology.orchestra.run.smithi092.stdout:in 0,1 2024-05-06T12:18:51.453 INFO:teuthology.orchestra.run.smithi092.stdout:up {0=14472,1=24267} 2024-05-06T12:18:51.454 INFO:teuthology.orchestra.run.smithi092.stdout:failed 2024-05-06T12:18:51.454 INFO:teuthology.orchestra.run.smithi092.stdout:damaged 2024-05-06T12:18:51.454 INFO:teuthology.orchestra.run.smithi092.stdout:stopped 2024-05-06T12:18:51.454 INFO:teuthology.orchestra.run.smithi092.stdout:data_pools [3] 2024-05-06T12:18:51.454 INFO:teuthology.orchestra.run.smithi092.stdout:metadata_pool 2 2024-05-06T12:18:51.454 INFO:teuthology.orchestra.run.smithi092.stdout:inline_data enabled 2024-05-06T12:18:51.454 INFO:teuthology.orchestra.run.smithi092.stdout:balancer 2024-05-06T12:18:51.454 INFO:teuthology.orchestra.run.smithi092.stdout:bal_rank_mask -1 2024-05-06T12:18:51.454 INFO:teuthology.orchestra.run.smithi092.stdout:standby_count_wanted 1 2024-05-06T12:18:51.454 INFO:teuthology.orchestra.run.smithi092.stdout:qdb_cluster leader: 14472 members: 14472,24267 2024-05-06T12:18:51.454 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.rosbll{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.92:6828/1663253598,v1:172.21.15.92:6829/1663253598] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:18:51.454 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.ijsqgp{0:14484} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.92:6826/3613441054,v1:172.21.15.92:6827/3613441054] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:18:51.454 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.byjvgf{1:24267} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.190:6824/1565151699,v1:172.21.15.190:6825/1565151699] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:18:51.454 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.idzoff{1:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.190:6826/2453728380,v1:172.21.15.190:6827/2453728380] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:18:51.455 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:18:51.455 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:18:51.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.452+0000 7f4ae9184700 1 -- 172.21.15.92:0/1079600215 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4acc06e140 msgr2=0x7f4acc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:51.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.452+0000 7f4ae9184700 1 --2- 172.21.15.92:0/1079600215 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4acc06e140 0x7f4acc0705f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009b10 tx=0x7f4ad4008040 comp rx=0 tx=0).stop 2024-05-06T12:18:51.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.452+0000 7f4ae9184700 1 -- 172.21.15.92:0/1079600215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ae40fcf10 msgr2=0x7f4ae4071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:51.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.452+0000 7f4ae9184700 1 --2- 172.21.15.92:0/1079600215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ae40fcf10 0x7f4ae4071de0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800b3b0 tx=0x7f4ad8009e30 comp rx=0 tx=0).stop 2024-05-06T12:18:51.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.452+0000 7f4ae9184700 1 -- 172.21.15.92:0/1079600215 shutdown_connections 2024-05-06T12:18:51.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.452+0000 7f4ae9184700 1 --2- 172.21.15.92:0/1079600215 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4acc06e140 0x7f4acc0705f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:51.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.452+0000 7f4ae9184700 1 --2- 172.21.15.92:0/1079600215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ae40fcf10 0x7f4ae4071de0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:51.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.452+0000 7f4ae9184700 1 --2- 172.21.15.92:0/1079600215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ae40fe080 0x7f4ae40702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:51.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.452+0000 7f4ae9184700 1 -- 172.21.15.92:0/1079600215 >> 172.21.15.92:0/1079600215 conn(0x7f4ae40f8560 msgr2=0x7f4ae40f9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:18:51.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.453+0000 7f4ae9184700 1 -- 172.21.15.92:0/1079600215 shutdown_connections 2024-05-06T12:18:51.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.453+0000 7f4ae9184700 1 -- 172.21.15.92:0/1079600215 wait complete. 2024-05-06T12:18:51.456 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 21 2024-05-06T12:18:51.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.555+0000 7ff875c05700 1 -- 172.21.15.92:0/1244345678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8700fce60 msgr2=0x7ff8700fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:51.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.555+0000 7ff875c05700 1 --2- 172.21.15.92:0/1244345678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8700fce60 0x7ff8700fd270 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff860004940 tx=0x7ff860009e30 comp rx=0 tx=0).stop 2024-05-06T12:18:51.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.555+0000 7ff875c05700 1 -- 172.21.15.92:0/1244345678 shutdown_connections 2024-05-06T12:18:51.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.555+0000 7ff875c05700 1 --2- 172.21.15.92:0/1244345678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8700fdfd0 0x7ff8700fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:51.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.555+0000 7ff875c05700 1 --2- 172.21.15.92:0/1244345678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8700fce60 0x7ff8700fd270 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:51.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.555+0000 7ff875c05700 1 -- 172.21.15.92:0/1244345678 >> 172.21.15.92:0/1244345678 conn(0x7ff8700f8540 msgr2=0x7ff8700fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:18:51.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.555+0000 7ff875c05700 1 -- 172.21.15.92:0/1244345678 shutdown_connections 2024-05-06T12:18:51.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.555+0000 7ff875c05700 1 -- 172.21.15.92:0/1244345678 wait complete. 2024-05-06T12:18:51.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.555+0000 7ff875c05700 1 Processor -- start 2024-05-06T12:18:51.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.555+0000 7ff875c05700 1 -- start start 2024-05-06T12:18:51.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.556+0000 7ff875c05700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8700fce60 0x7ff870071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:51.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.556+0000 7ff875c05700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8700fdfd0 0x7ff8700702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:51.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.556+0000 7ff875c05700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8700707f0 con 0x7ff8700fce60 2024-05-06T12:18:51.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.556+0000 7ff875c05700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff870070960 con 0x7ff8700fdfd0 2024-05-06T12:18:51.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.556+0000 7ff86f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8700fce60 0x7ff870071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:51.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.556+0000 7ff86f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8700fce60 0x7ff870071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44394/0 (socket says 172.21.15.92:44394) 2024-05-06T12:18:51.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.556+0000 7ff86f7fe700 1 -- 172.21.15.92:0/3555225683 learned_addr learned my addr 172.21.15.92:0/3555225683 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:18:51.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.556+0000 7ff86f7fe700 1 -- 172.21.15.92:0/3555225683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8700fdfd0 msgr2=0x7ff8700702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:51.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.556+0000 7ff86f7fe700 1 --2- 172.21.15.92:0/3555225683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8700fdfd0 0x7ff8700702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:51.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.556+0000 7ff86f7fe700 1 -- 172.21.15.92:0/3555225683 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff860005040 con 0x7ff8700fce60 2024-05-06T12:18:51.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.556+0000 7ff86f7fe700 1 --2- 172.21.15.92:0/3555225683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8700fce60 0x7ff870071c00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff86000b3b0 tx=0x7ff860009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:18:51.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.556+0000 7ff86cff9700 1 -- 172.21.15.92:0/3555225683 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff86000e040 con 0x7ff8700fce60 2024-05-06T12:18:51.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.556+0000 7ff875c05700 1 -- 172.21.15.92:0/3555225683 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff870070be0 con 0x7ff8700fce60 2024-05-06T12:18:51.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.556+0000 7ff86cff9700 1 -- 172.21.15.92:0/3555225683 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff86000bcc0 con 0x7ff8700fce60 2024-05-06T12:18:51.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.556+0000 7ff86cff9700 1 -- 172.21.15.92:0/3555225683 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff86001c630 con 0x7ff8700fce60 2024-05-06T12:18:51.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.556+0000 7ff875c05700 1 -- 172.21.15.92:0/3555225683 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff870115a40 con 0x7ff8700fce60 2024-05-06T12:18:51.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.557+0000 7ff875c05700 1 -- 172.21.15.92:0/3555225683 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff87004c710 con 0x7ff8700fce60 2024-05-06T12:18:51.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.558+0000 7ff86cff9700 1 -- 172.21.15.92:0/3555225683 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff860013980 con 0x7ff8700fce60 2024-05-06T12:18:51.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.558+0000 7ff86cff9700 1 --2- 172.21.15.92:0/3555225683 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff85c06e140 0x7ff85c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:51.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.558+0000 7ff86cff9700 1 -- 172.21.15.92:0/3555225683 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff86008e830 con 0x7ff8700fce60 2024-05-06T12:18:51.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.558+0000 7ff866ffd700 1 --2- 172.21.15.92:0/3555225683 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff85c06e140 0x7ff85c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:51.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.559+0000 7ff866ffd700 1 --2- 172.21.15.92:0/3555225683 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff85c06e140 0x7ff85c0705f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff858006fd0 tx=0x7ff858006db0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:18:51.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.563+0000 7ff86cff9700 1 -- 172.21.15.92:0/3555225683 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff86005b6e0 con 0x7ff8700fce60 2024-05-06T12:18:51.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.785+0000 7ff875c05700 1 -- 172.21.15.92:0/3555225683 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff870102270 con 0x7ff85c06e140 2024-05-06T12:18:51.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.786+0000 7ff86cff9700 1 -- 172.21.15.92:0/3555225683 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7ff870102270 con 0x7ff85c06e140 2024-05-06T12:18:51.788 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:18:51.788 INFO:teuthology.orchestra.run.smithi092.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb", 2024-05-06T12:18:51.788 INFO:teuthology.orchestra.run.smithi092.stdout: "in_progress": true, 2024-05-06T12:18:51.789 INFO:teuthology.orchestra.run.smithi092.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-06T12:18:51.789 INFO:teuthology.orchestra.run.smithi092.stdout: "services_complete": [ 2024-05-06T12:18:51.789 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr", 2024-05-06T12:18:51.789 INFO:teuthology.orchestra.run.smithi092.stdout: "mon", 2024-05-06T12:18:51.789 INFO:teuthology.orchestra.run.smithi092.stdout: "crash" 2024-05-06T12:18:51.789 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-05-06T12:18:51.789 INFO:teuthology.orchestra.run.smithi092.stdout: "progress": "8/21 daemons upgraded", 2024-05-06T12:18:51.789 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "Currently upgrading osd daemons", 2024-05-06T12:18:51.789 INFO:teuthology.orchestra.run.smithi092.stdout: "is_paused": false 2024-05-06T12:18:51.789 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:18:51.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.788+0000 7ff875c05700 1 -- 172.21.15.92:0/3555225683 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff85c06e140 msgr2=0x7ff85c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:51.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.788+0000 7ff875c05700 1 --2- 172.21.15.92:0/3555225683 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff85c06e140 0x7ff85c0705f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff858006fd0 tx=0x7ff858006db0 comp rx=0 tx=0).stop 2024-05-06T12:18:51.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.788+0000 7ff875c05700 1 -- 172.21.15.92:0/3555225683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8700fce60 msgr2=0x7ff870071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:51.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.788+0000 7ff875c05700 1 --2- 172.21.15.92:0/3555225683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8700fce60 0x7ff870071c00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff86000b3b0 tx=0x7ff860009e30 comp rx=0 tx=0).stop 2024-05-06T12:18:51.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.788+0000 7ff875c05700 1 -- 172.21.15.92:0/3555225683 shutdown_connections 2024-05-06T12:18:51.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.788+0000 7ff875c05700 1 --2- 172.21.15.92:0/3555225683 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff85c06e140 0x7ff85c0705f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:51.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.788+0000 7ff875c05700 1 --2- 172.21.15.92:0/3555225683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8700fce60 0x7ff870071c00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:51.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.788+0000 7ff875c05700 1 --2- 172.21.15.92:0/3555225683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8700fdfd0 0x7ff8700702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:51.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.788+0000 7ff875c05700 1 -- 172.21.15.92:0/3555225683 >> 172.21.15.92:0/3555225683 conn(0x7ff8700f8540 msgr2=0x7ff8700f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:18:51.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.788+0000 7ff875c05700 1 -- 172.21.15.92:0/3555225683 shutdown_connections 2024-05-06T12:18:51.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.788+0000 7ff875c05700 1 -- 172.21.15.92:0/3555225683 wait complete. 2024-05-06T12:18:51.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.890+0000 7f8087bb9700 1 -- 172.21.15.92:0/3991647989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80800fcf10 msgr2=0x7f80800fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:51.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.890+0000 7f8087bb9700 1 --2- 172.21.15.92:0/3991647989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80800fcf10 0x7f80800fd320 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f807c009ce0 tx=0x7f807c004d10 comp rx=0 tx=0).stop 2024-05-06T12:18:51.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.890+0000 7f8087bb9700 1 -- 172.21.15.92:0/3991647989 shutdown_connections 2024-05-06T12:18:51.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.890+0000 7f8087bb9700 1 --2- 172.21.15.92:0/3991647989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80800fe080 0x7f80800fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:51.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.890+0000 7f8087bb9700 1 --2- 172.21.15.92:0/3991647989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80800fcf10 0x7f80800fd320 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:51.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.890+0000 7f8087bb9700 1 -- 172.21.15.92:0/3991647989 >> 172.21.15.92:0/3991647989 conn(0x7f80800f8560 msgr2=0x7f80800fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:18:51.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.890+0000 7f8087bb9700 1 -- 172.21.15.92:0/3991647989 shutdown_connections 2024-05-06T12:18:51.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.890+0000 7f8087bb9700 1 -- 172.21.15.92:0/3991647989 wait complete. 2024-05-06T12:18:51.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.890+0000 7f8087bb9700 1 Processor -- start 2024-05-06T12:18:51.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.890+0000 7f8087bb9700 1 -- start start 2024-05-06T12:18:51.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.891+0000 7f8087bb9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80800fcf10 0x7f808010db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:51.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.891+0000 7f8087bb9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80800fe080 0x7f808010e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:51.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.891+0000 7f8087bb9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808010e690 con 0x7f80800fe080 2024-05-06T12:18:51.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.891+0000 7f8087bb9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808010eee0 con 0x7f80800fcf10 2024-05-06T12:18:51.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.891+0000 7f8085154700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80800fe080 0x7f808010e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:51.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.891+0000 7f8085154700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80800fe080 0x7f808010e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44410/0 (socket says 172.21.15.92:44410) 2024-05-06T12:18:51.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.891+0000 7f8085154700 1 -- 172.21.15.92:0/638006736 learned_addr learned my addr 172.21.15.92:0/638006736 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:18:51.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.891+0000 7f8085955700 1 --2- 172.21.15.92:0/638006736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80800fcf10 0x7f808010db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:51.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.891+0000 7f8085154700 1 -- 172.21.15.92:0/638006736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80800fcf10 msgr2=0x7f808010db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:51.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.891+0000 7f8085154700 1 --2- 172.21.15.92:0/638006736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80800fcf10 0x7f808010db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:51.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.891+0000 7f8085154700 1 -- 172.21.15.92:0/638006736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f807c009940 con 0x7f80800fe080 2024-05-06T12:18:51.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.891+0000 7f8085154700 1 --2- 172.21.15.92:0/638006736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80800fe080 0x7f808010e090 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f807000c8c0 tx=0x7f80700044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:18:51.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.891+0000 7f80767fc700 1 -- 172.21.15.92:0/638006736 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f807000a980 con 0x7f80800fe080 2024-05-06T12:18:51.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.891+0000 7f80767fc700 1 -- 172.21.15.92:0/638006736 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f807000ce60 con 0x7f80800fe080 2024-05-06T12:18:51.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.891+0000 7f8087bb9700 1 -- 172.21.15.92:0/638006736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f808010f1c0 con 0x7f80800fe080 2024-05-06T12:18:51.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.891+0000 7f80767fc700 1 -- 172.21.15.92:0/638006736 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80700115e0 con 0x7f80800fe080 2024-05-06T12:18:51.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.891+0000 7f8087bb9700 1 -- 172.21.15.92:0/638006736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808010f700 con 0x7f80800fe080 2024-05-06T12:18:51.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.892+0000 7f8087bb9700 1 -- 172.21.15.92:0/638006736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f808005f5c0 con 0x7f80800fe080 2024-05-06T12:18:51.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.893+0000 7f80767fc700 1 -- 172.21.15.92:0/638006736 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8070012400 con 0x7f80800fe080 2024-05-06T12:18:51.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.893+0000 7f80767fc700 1 --2- 172.21.15.92:0/638006736 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f806c06e140 0x7f806c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:18:51.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.893+0000 7f80767fc700 1 -- 172.21.15.92:0/638006736 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f807008db20 con 0x7f80800fe080 2024-05-06T12:18:51.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.893+0000 7f8085955700 1 --2- 172.21.15.92:0/638006736 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f806c06e140 0x7f806c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:18:51.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.894+0000 7f8085955700 1 --2- 172.21.15.92:0/638006736 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f806c06e140 0x7f806c0705f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f807c0095a0 tx=0x7f807c004d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:18:51.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:51.898+0000 7f80767fc700 1 -- 172.21.15.92:0/638006736 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f807005aaf0 con 0x7f80800fe080 2024-05-06T12:18:52.184 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:52.181+0000 7f8087bb9700 1 -- 172.21.15.92:0/638006736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f808004bed0 con 0x7f80800fe080 2024-05-06T12:18:52.184 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:51 smithi092 ceph-mon[91933]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:18:52.184 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:51 smithi092 ceph-mon[91933]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:18:52.185 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:51 smithi092 ceph-mon[91933]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:18:52.185 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:51 smithi092 ceph-mon[91933]: pgmap v79: 65 pgs: 65 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-06T12:18:52.185 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:51 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/2137552285' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:18:52.185 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:51 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/1079600215' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:18:52.186 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:52.182+0000 7f80767fc700 1 -- 172.21.15.92:0/638006736 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f80700126b0 con 0x7f80800fe080 2024-05-06T12:18:52.186 INFO:teuthology.orchestra.run.smithi092.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-06T12:18:52.186 INFO:teuthology.orchestra.run.smithi092.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-06T12:18:52.187 INFO:teuthology.orchestra.run.smithi092.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-06T12:18:52.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:52.184+0000 7f8087bb9700 1 -- 172.21.15.92:0/638006736 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f806c06e140 msgr2=0x7f806c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:52.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:52.184+0000 7f8087bb9700 1 --2- 172.21.15.92:0/638006736 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f806c06e140 0x7f806c0705f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f807c0095a0 tx=0x7f807c004d10 comp rx=0 tx=0).stop 2024-05-06T12:18:52.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:52.184+0000 7f8087bb9700 1 -- 172.21.15.92:0/638006736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80800fe080 msgr2=0x7f808010e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:18:52.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:52.184+0000 7f8087bb9700 1 --2- 172.21.15.92:0/638006736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80800fe080 0x7f808010e090 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f807000c8c0 tx=0x7f80700044c0 comp rx=0 tx=0).stop 2024-05-06T12:18:52.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:52.184+0000 7f8087bb9700 1 -- 172.21.15.92:0/638006736 shutdown_connections 2024-05-06T12:18:52.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:52.184+0000 7f8087bb9700 1 --2- 172.21.15.92:0/638006736 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f806c06e140 0x7f806c0705f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:52.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:52.184+0000 7f8087bb9700 1 --2- 172.21.15.92:0/638006736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80800fcf10 0x7f808010db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:52.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:52.184+0000 7f8087bb9700 1 --2- 172.21.15.92:0/638006736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80800fe080 0x7f808010e090 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:18:52.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:52.184+0000 7f8087bb9700 1 -- 172.21.15.92:0/638006736 >> 172.21.15.92:0/638006736 conn(0x7f80800f8560 msgr2=0x7f80800f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:18:52.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:52.184+0000 7f8087bb9700 1 -- 172.21.15.92:0/638006736 shutdown_connections 2024-05-06T12:18:52.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:18:52.184+0000 7f8087bb9700 1 -- 172.21.15.92:0/638006736 wait complete. 2024-05-06T12:18:52.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:51 smithi190 ceph-mon[79744]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:18:52.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:51 smithi190 ceph-mon[79744]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:18:52.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:51 smithi190 ceph-mon[79744]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:18:52.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:51 smithi190 ceph-mon[79744]: pgmap v79: 65 pgs: 65 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-06T12:18:52.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:51 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/2137552285' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:18:52.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:51 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/1079600215' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:18:53.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:52 smithi092 ceph-mon[91933]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:18:53.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:52 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/638006736' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:18:53.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:52 smithi190 ceph-mon[79744]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:18:53.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:52 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/638006736' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:18:54.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:53 smithi092 ceph-mon[91933]: pgmap v80: 65 pgs: 65 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-05-06T12:18:54.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:53 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:54.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:53 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:18:54.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:53 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:54.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:53 smithi190 ceph-mon[79744]: pgmap v80: 65 pgs: 65 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-05-06T12:18:54.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:53 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:54.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:53 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:18:54.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:53 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:18:56.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:55 smithi092 ceph-mon[91933]: pgmap v81: 65 pgs: 65 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-06T12:18:56.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:55 smithi190 ceph-mon[79744]: pgmap v81: 65 pgs: 65 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-06T12:18:58.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:57 smithi092 ceph-mon[91933]: pgmap v82: 65 pgs: 65 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:18:58.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:57 smithi190 ceph-mon[79744]: pgmap v82: 65 pgs: 65 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:19:00.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:18:59 smithi092 ceph-mon[91933]: pgmap v83: 65 pgs: 65 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:19:00.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:18:59 smithi190 ceph-mon[79744]: pgmap v83: 65 pgs: 65 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:19:01.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:00 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-06T12:19:01.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:00 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-06T12:19:02.121 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:01 smithi092 ceph-mon[91933]: pgmap v84: 65 pgs: 65 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:19:02.121 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:01 smithi092 ceph-mon[91933]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-06T12:19:02.121 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:01 smithi092 ceph-mon[91933]: Upgrade: osd.2 is safe to restart 2024-05-06T12:19:02.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:01 smithi190 ceph-mon[79744]: pgmap v84: 65 pgs: 65 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:19:02.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:01 smithi190 ceph-mon[79744]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-06T12:19:02.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:01 smithi190 ceph-mon[79744]: Upgrade: osd.2 is safe to restart 2024-05-06T12:19:03.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:03 smithi092 ceph-mon[91933]: Upgrade: Updating osd.2 2024-05-06T12:19:03.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:03 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:03.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:03 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-06T12:19:03.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:03 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:19:03.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:03 smithi092 ceph-mon[91933]: Deploying daemon osd.2 on smithi092 2024-05-06T12:19:03.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:03 smithi092 ceph-mon[91933]: pgmap v85: 65 pgs: 65 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:19:03.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:03 smithi190 ceph-mon[79744]: Upgrade: Updating osd.2 2024-05-06T12:19:03.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:03 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:03.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:03 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-06T12:19:03.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:03 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:19:03.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:03 smithi190 ceph-mon[79744]: Deploying daemon osd.2 on smithi092 2024-05-06T12:19:03.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:03 smithi190 ceph-mon[79744]: pgmap v85: 65 pgs: 65 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:19:05.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:05 smithi092 ceph-mon[91933]: pgmap v86: 65 pgs: 65 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:19:05.973 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:05 smithi092 systemd[1]: Stopping Ceph osd.2 for aee1d912-0ba0-11ef-bc97-c7b262605968... 2024-05-06T12:19:06.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:05 smithi190 ceph-mon[79744]: pgmap v86: 65 pgs: 65 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:19:06.473 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:05 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2[60584]: 2024-05-06T12:19:05.988+0000 7fdc041df700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-06T12:19:06.473 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:05 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2[60584]: 2024-05-06T12:19:05.988+0000 7fdc041df700 -1 osd.2 60 *** Got signal Terminated *** 2024-05-06T12:19:06.473 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:05 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2[60584]: 2024-05-06T12:19:05.988+0000 7fdc041df700 -1 osd.2 60 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-06T12:19:07.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:06 smithi092 ceph-mon[91933]: osd.2 marked itself down and dead 2024-05-06T12:19:07.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:06 smithi190 ceph-mon[79744]: osd.2 marked itself down and dead 2024-05-06T12:19:07.596 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:07 smithi092 podman[111556]: 2024-05-06 12:19:07.267144058 +0000 UTC m=+1.403015392 container died 1e6ed04a0688ba5529ead14444e1f9961e2ad56233a453834ce0b3729bdf2e8c (image=quay.ceph.io/ceph-ci/ceph@sha256:1a0b7fd3cefd2c5baddf682493f92b4e27596f3c0ff27b02fdccd2891f56859f, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2, org.label-schema.build-date=20240430, RELEASE=quincy-e81df42, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-06T12:19:07.876 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:07 smithi092 podman[111556]: 2024-05-06 12:19:07.697871448 +0000 UTC m=+1.833742781 container cleanup 1e6ed04a0688ba5529ead14444e1f9961e2ad56233a453834ce0b3729bdf2e8c (image=quay.ceph.io/ceph-ci/ceph@sha256:1a0b7fd3cefd2c5baddf682493f92b4e27596f3c0ff27b02fdccd2891f56859f, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, RELEASE=quincy-e81df42) 2024-05-06T12:19:07.876 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:07 smithi092 bash[111556]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2 2024-05-06T12:19:08.223 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:07 smithi092 podman[111573]: 2024-05-06 12:19:07.873937015 +0000 UTC m=+0.609491042 container remove 1e6ed04a0688ba5529ead14444e1f9961e2ad56233a453834ce0b3729bdf2e8c (image=quay.ceph.io/ceph-ci/ceph@sha256:1a0b7fd3cefd2c5baddf682493f92b4e27596f3c0ff27b02fdccd2891f56859f, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-e81df42, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-06T12:19:08.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:07 smithi092 ceph-mon[91933]: pgmap v87: 65 pgs: 65 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:19:08.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:07 smithi092 ceph-mon[91933]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-06T12:19:08.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:07 smithi092 ceph-mon[91933]: osdmap e61: 6 total, 5 up, 6 in 2024-05-06T12:19:08.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:07 smithi092 ceph-mon[91933]: fsmap cephfs:2 {0=cephfs.smithi092.rosbll=up:active,1=cephfs.smithi190.byjvgf=up:active} 2 up:standby-replay 2024-05-06T12:19:08.250 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:07 smithi190 ceph-mon[79744]: pgmap v87: 65 pgs: 65 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:19:08.250 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:07 smithi190 ceph-mon[79744]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-06T12:19:08.250 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:07 smithi190 ceph-mon[79744]: osdmap e61: 6 total, 5 up, 6 in 2024-05-06T12:19:08.250 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:07 smithi190 ceph-mon[79744]: fsmap cephfs:2 {0=cephfs.smithi092.rosbll=up:active,1=cephfs.smithi190.byjvgf=up:active} 2 up:standby-replay 2024-05-06T12:19:08.549 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:08 smithi092 podman[111666]: 2024-05-06 12:19:08.198618563 +0000 UTC m=+0.018970169 image pull e4ce8ad5d0c4b9ea60bc5f6b8b0fdb5efbe59bf7771fb0920fc2200e6ae546b0 quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb 2024-05-06T12:19:08.550 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:08 smithi092 podman[111666]: 2024-05-06 12:19:08.324867476 +0000 UTC m=+0.145219063 container create 5fbae90089aca7a56ec7d57f2a5afbc534ff050d8087710b4bc1cc7d60f95cc8 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, RELEASE=squid-70ce953, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-05-06T12:19:08.550 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:08 smithi092 podman[111666]: 2024-05-06 12:19:08.543994796 +0000 UTC m=+0.364346393 container init 5fbae90089aca7a56ec7d57f2a5afbc534ff050d8087710b4bc1cc7d60f95cc8 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-70ce953, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-06T12:19:08.550 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:08 smithi092 podman[111666]: 2024-05-06 12:19:08.547375821 +0000 UTC m=+0.367727417 container start 5fbae90089aca7a56ec7d57f2a5afbc534ff050d8087710b4bc1cc7d60f95cc8 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-70ce953, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-05-06T12:19:08.837 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:08 smithi092 podman[111666]: 2024-05-06 12:19:08.652327079 +0000 UTC m=+0.472678679 container attach 5fbae90089aca7a56ec7d57f2a5afbc534ff050d8087710b4bc1cc7d60f95cc8 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-70ce953, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True) 2024-05-06T12:19:09.133 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:08 smithi092 ceph-mon[91933]: osdmap e62: 6 total, 5 up, 6 in 2024-05-06T12:19:09.133 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:08 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:09.133 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:08 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:19:09.133 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:08 smithi092 podman[111666]: 2024-05-06 12:19:08.835907626 +0000 UTC m=+0.656259205 container died 5fbae90089aca7a56ec7d57f2a5afbc534ff050d8087710b4bc1cc7d60f95cc8 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2-deactivate, RELEASE=squid-70ce953, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-06T12:19:09.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:08 smithi190 ceph-mon[79744]: osdmap e62: 6 total, 5 up, 6 in 2024-05-06T12:19:09.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:08 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:09.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:08 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:19:09.723 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:09 smithi092 podman[111691]: 2024-05-06 12:19:09.39817016 +0000 UTC m=+0.553179201 container remove 5fbae90089aca7a56ec7d57f2a5afbc534ff050d8087710b4bc1cc7d60f95cc8 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-70ce953, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-06T12:19:09.723 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:09 smithi092 systemd[1]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@osd.2.service: Deactivated successfully. 2024-05-06T12:19:09.723 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:09 smithi092 systemd[1]: Stopped Ceph osd.2 for aee1d912-0ba0-11ef-bc97-c7b262605968. 2024-05-06T12:19:09.723 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:09 smithi092 systemd[1]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@osd.2.service: Consumed 42.530s CPU time. 2024-05-06T12:19:09.723 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:09 smithi092 systemd[1]: Starting Ceph osd.2 for aee1d912-0ba0-11ef-bc97-c7b262605968... 2024-05-06T12:19:10.178 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:09 smithi092 ceph-mon[91933]: pgmap v90: 65 pgs: 7 peering, 5 stale+active+clean, 53 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-06T12:19:10.179 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:09 smithi092 ceph-mon[91933]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-05-06T12:19:10.179 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:09 smithi092 podman[111798]: 2024-05-06 12:19:09.74709713 +0000 UTC m=+0.018257054 image pull e4ce8ad5d0c4b9ea60bc5f6b8b0fdb5efbe59bf7771fb0920fc2200e6ae546b0 quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb 2024-05-06T12:19:10.179 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:09 smithi092 podman[111798]: 2024-05-06 12:19:09.857465656 +0000 UTC m=+0.128625623 container create 588db302c81bd6caa4b7bad54acb821190f877064d152434db7ccd79520aa8b9 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2-activate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , RELEASE=squid-70ce953, org.label-schema.build-date=20240430) 2024-05-06T12:19:10.179 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:10 smithi092 podman[111798]: 2024-05-06 12:19:10.076656858 +0000 UTC m=+0.347816782 container init 588db302c81bd6caa4b7bad54acb821190f877064d152434db7ccd79520aa8b9 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2-activate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-70ce953, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-06T12:19:10.179 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:10 smithi092 podman[111798]: 2024-05-06 12:19:10.079990111 +0000 UTC m=+0.351150044 container start 588db302c81bd6caa4b7bad54acb821190f877064d152434db7ccd79520aa8b9 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2-activate, org.label-schema.vendor=CentOS, RELEASE=squid-70ce953, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-06T12:19:10.179 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:10 smithi092 podman[111798]: 2024-05-06 12:19:10.176575442 +0000 UTC m=+0.447735381 container attach 588db302c81bd6caa4b7bad54acb821190f877064d152434db7ccd79520aa8b9 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2-activate, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-70ce953, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-06T12:19:10.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:09 smithi190 ceph-mon[79744]: pgmap v90: 65 pgs: 7 peering, 5 stale+active+clean, 53 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-06T12:19:10.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:09 smithi190 ceph-mon[79744]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-05-06T12:19:10.866 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:10 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2-activate[111813]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-06T12:19:10.866 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:10 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2-activate[111813]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-05-06T12:19:10.866 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:10 smithi092 bash[111798]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-06T12:19:10.866 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:10 smithi092 bash[111798]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-05-06T12:19:10.866 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:10 smithi092 bash[111798]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-05-06T12:19:10.866 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:10 smithi092 bash[111798]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-05-06T12:19:10.866 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:10 smithi092 bash[111798]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-05-06T12:19:10.866 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:10 smithi092 bash[111798]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-06T12:19:10.866 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:10 smithi092 bash[111798]: --> ceph-volume raw activate successful for osd ID: 2 2024-05-06T12:19:10.866 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:10 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2-activate[111813]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-05-06T12:19:10.867 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:10 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2-activate[111813]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-05-06T12:19:10.867 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:10 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2-activate[111813]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-05-06T12:19:10.867 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:10 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2-activate[111813]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-06T12:19:10.867 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:10 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2-activate[111813]: --> ceph-volume raw activate successful for osd ID: 2 2024-05-06T12:19:10.867 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:10 smithi092 podman[111798]: 2024-05-06 12:19:10.550633445 +0000 UTC m=+0.821793360 container died 588db302c81bd6caa4b7bad54acb821190f877064d152434db7ccd79520aa8b9 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2-activate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-70ce953, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=) 2024-05-06T12:19:11.124 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:10 smithi092 ceph-mon[91933]: Health check failed: Degraded data redundancy: 33/372 objects degraded (8.871%), 10 pgs degraded (PG_DEGRADED) 2024-05-06T12:19:11.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:10 smithi190 ceph-mon[79744]: Health check failed: Degraded data redundancy: 33/372 objects degraded (8.871%), 10 pgs degraded (PG_DEGRADED) 2024-05-06T12:19:11.457 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:11 smithi092 podman[111942]: 2024-05-06 12:19:11.122566193 +0000 UTC m=+0.567082885 container remove 588db302c81bd6caa4b7bad54acb821190f877064d152434db7ccd79520aa8b9 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2-activate, maintainer=Guillaume Abrioux , RELEASE=squid-70ce953, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-06T12:19:11.457 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:11 smithi092 podman[112007]: 2024-05-06 12:19:11.239809956 +0000 UTC m=+0.018868032 image pull e4ce8ad5d0c4b9ea60bc5f6b8b0fdb5efbe59bf7771fb0920fc2200e6ae546b0 quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb 2024-05-06T12:19:11.457 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:11 smithi092 podman[112007]: 2024-05-06 12:19:11.348486512 +0000 UTC m=+0.127544617 container create 2beca7448789c87dcd787d8ab5b224f68af7a62a5d44e9462ba2defce1ebeefa (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2, org.label-schema.schema-version=1.0, RELEASE=squid-70ce953, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-06T12:19:11.949 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:11 smithi092 podman[112007]: 2024-05-06 12:19:11.550286764 +0000 UTC m=+0.329344843 container init 2beca7448789c87dcd787d8ab5b224f68af7a62a5d44e9462ba2defce1ebeefa (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-70ce953, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240430) 2024-05-06T12:19:11.950 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:11 smithi092 podman[112007]: 2024-05-06 12:19:11.553248914 +0000 UTC m=+0.332306978 container start 2beca7448789c87dcd787d8ab5b224f68af7a62a5d44e9462ba2defce1ebeefa (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-70ce953, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240430) 2024-05-06T12:19:11.950 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:11 smithi092 ceph-osd[112027]: -- 172.21.15.92:0/941227667 <== mon.0 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c7531c5680 con 0x55c75318ec00 2024-05-06T12:19:11.950 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:11 smithi092 bash[112007]: 2beca7448789c87dcd787d8ab5b224f68af7a62a5d44e9462ba2defce1ebeefa 2024-05-06T12:19:11.950 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:11 smithi092 systemd[1]: Started Ceph osd.2 for aee1d912-0ba0-11ef-bc97-c7b262605968. 2024-05-06T12:19:12.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:11 smithi092 ceph-mon[91933]: pgmap v91: 65 pgs: 10 active+undersized+degraded, 7 peering, 10 active+undersized, 38 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 33/372 objects degraded (8.871%) 2024-05-06T12:19:12.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:11 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:12.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:11 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:12.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:11 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:19:12.223 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:12 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2[112023]: 2024-05-06T12:19:12.107+0000 7f67579d67c0 -1 Falling back to public interface 2024-05-06T12:19:12.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:11 smithi190 ceph-mon[79744]: pgmap v91: 65 pgs: 10 active+undersized+degraded, 7 peering, 10 active+undersized, 38 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 33/372 objects degraded (8.871%) 2024-05-06T12:19:12.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:11 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:12.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:11 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:12.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:11 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:19:13.318 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:12 smithi092 ceph-mon[91933]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-05-06T12:19:13.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:12 smithi190 ceph-mon[79744]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-05-06T12:19:14.270 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:14 smithi092 ceph-mon[91933]: pgmap v92: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 46/372 objects degraded (12.366%) 2024-05-06T12:19:14.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:14 smithi190 ceph-mon[79744]: pgmap v92: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 46/372 objects degraded (12.366%) 2024-05-06T12:19:16.270 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:16 smithi092 ceph-mon[91933]: pgmap v93: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 46/372 objects degraded (12.366%) 2024-05-06T12:19:16.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:16 smithi190 ceph-mon[79744]: pgmap v93: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 46/372 objects degraded (12.366%) 2024-05-06T12:19:16.541 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:16 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2[112023]: 2024-05-06T12:19:16.266+0000 7f67579d67c0 -1 osd.2 60 log_to_monitors true 2024-05-06T12:19:17.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:17 smithi190 ceph-mon[79744]: from='osd.2 [v2:172.21.15.92:6818/629593219,v1:172.21.15.92:6819/629593219]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-06T12:19:17.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:17 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:17.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:17 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:17.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:17 smithi092 ceph-mon[91933]: from='osd.2 [v2:172.21.15.92:6818/629593219,v1:172.21.15.92:6819/629593219]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-06T12:19:17.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:17 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:17.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:17 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:18.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:18 smithi190 ceph-mon[79744]: pgmap v94: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 46/372 objects degraded (12.366%) 2024-05-06T12:19:18.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:18 smithi190 ceph-mon[79744]: from='osd.2 [v2:172.21.15.92:6818/629593219,v1:172.21.15.92:6819/629593219]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-06T12:19:18.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:18 smithi190 ceph-mon[79744]: osdmap e63: 6 total, 5 up, 6 in 2024-05-06T12:19:18.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:18 smithi190 ceph-mon[79744]: from='osd.2 [v2:172.21.15.92:6818/629593219,v1:172.21.15.92:6819/629593219]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-06T12:19:18.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:18 smithi190 ceph-mon[79744]: from='osd.2 [v2:172.21.15.92:6818/629593219,v1:172.21.15.92:6819/629593219]' entity='osd.2' 2024-05-06T12:19:18.472 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:19:17 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2[112023]: 2024-05-06T12:19:17.985+0000 7f6744e25700 -1 osd.2 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-06T12:19:18.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:18 smithi092 ceph-mon[91933]: pgmap v94: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 323 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 46/372 objects degraded (12.366%) 2024-05-06T12:19:18.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:18 smithi092 ceph-mon[91933]: from='osd.2 [v2:172.21.15.92:6818/629593219,v1:172.21.15.92:6819/629593219]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-06T12:19:18.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:18 smithi092 ceph-mon[91933]: osdmap e63: 6 total, 5 up, 6 in 2024-05-06T12:19:18.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:18 smithi092 ceph-mon[91933]: from='osd.2 [v2:172.21.15.92:6818/629593219,v1:172.21.15.92:6819/629593219]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-06T12:19:18.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:18 smithi092 ceph-mon[91933]: from='osd.2 [v2:172.21.15.92:6818/629593219,v1:172.21.15.92:6819/629593219]' entity='osd.2' 2024-05-06T12:19:19.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:19 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:19.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:19 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:19.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:19 smithi092 ceph-mon[91933]: pgmap v96: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 46/372 objects degraded (12.366%) 2024-05-06T12:19:19.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:19 smithi092 ceph-mon[91933]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-06T12:19:19.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:19 smithi092 ceph-mon[91933]: osd.2 [v2:172.21.15.92:6818/629593219,v1:172.21.15.92:6819/629593219] boot 2024-05-06T12:19:19.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:19 smithi092 ceph-mon[91933]: osdmap e64: 6 total, 6 up, 6 in 2024-05-06T12:19:19.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:19 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-06T12:19:19.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:19 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:19.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:19 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:19.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:19 smithi190 ceph-mon[79744]: pgmap v96: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 46/372 objects degraded (12.366%) 2024-05-06T12:19:19.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:19 smithi190 ceph-mon[79744]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-06T12:19:19.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:19 smithi190 ceph-mon[79744]: osd.2 [v2:172.21.15.92:6818/629593219,v1:172.21.15.92:6819/629593219] boot 2024-05-06T12:19:19.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:19 smithi190 ceph-mon[79744]: osdmap e64: 6 total, 6 up, 6 in 2024-05-06T12:19:19.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:19 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-06T12:19:21.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:20 smithi092 ceph-mon[91933]: osdmap e65: 6 total, 6 up, 6 in 2024-05-06T12:19:21.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:20 smithi092 ceph-mon[91933]: Health check update: Degraded data redundancy: 46/372 objects degraded (12.366%), 13 pgs degraded (PG_DEGRADED) 2024-05-06T12:19:21.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:20 smithi190 ceph-mon[79744]: osdmap e65: 6 total, 6 up, 6 in 2024-05-06T12:19:21.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:20 smithi190 ceph-mon[79744]: Health check update: Degraded data redundancy: 46/372 objects degraded (12.366%), 13 pgs degraded (PG_DEGRADED) 2024-05-06T12:19:22.292 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:21 smithi092 ceph-mon[91933]: pgmap v99: 65 pgs: 15 peering, 6 active+undersized+degraded, 6 active+undersized, 38 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 19/372 objects degraded (5.108%) 2024-05-06T12:19:22.292 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:21 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:22.293 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:21 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:22.293 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:21 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:19:22.293 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:21 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:19:22.293 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:21 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:22.293 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:21 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:19:22.293 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:21 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:19:22.293 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:21 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:19:22.293 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:21 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:19:22.293 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:21 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-06T12:19:22.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.289+0000 7f2cd396b700 1 -- 172.21.15.92:0/1488062282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ccc0fcf70 msgr2=0x7f2ccc0fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:22.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.289+0000 7f2cd396b700 1 --2- 172.21.15.92:0/1488062282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ccc0fcf70 0x7f2ccc0fd3e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc004940 tx=0x7f2cbc009e30 comp rx=0 tx=0).stop 2024-05-06T12:19:22.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.289+0000 7f2cd396b700 1 -- 172.21.15.92:0/1488062282 shutdown_connections 2024-05-06T12:19:22.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.289+0000 7f2cd396b700 1 --2- 172.21.15.92:0/1488062282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ccc0fcf70 0x7f2ccc0fd3e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:22.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.289+0000 7f2cd396b700 1 --2- 172.21.15.92:0/1488062282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ccc0fbcc0 0x7f2ccc0fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:22.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.289+0000 7f2cd396b700 1 -- 172.21.15.92:0/1488062282 >> 172.21.15.92:0/1488062282 conn(0x7f2ccc0f7440 msgr2=0x7f2ccc0f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:19:22.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.289+0000 7f2cd396b700 1 -- 172.21.15.92:0/1488062282 shutdown_connections 2024-05-06T12:19:22.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.289+0000 7f2cd396b700 1 -- 172.21.15.92:0/1488062282 wait complete. 2024-05-06T12:19:22.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.290+0000 7f2cd396b700 1 Processor -- start 2024-05-06T12:19:22.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.290+0000 7f2cd396b700 1 -- start start 2024-05-06T12:19:22.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.290+0000 7f2cd396b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ccc0fbcc0 0x7f2ccc10d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:22.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.290+0000 7f2cd396b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ccc0fcf70 0x7f2ccc10def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:22.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.290+0000 7f2cd396b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ccc10e4f0 con 0x7f2ccc0fbcc0 2024-05-06T12:19:22.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.290+0000 7f2cd396b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ccc10e660 con 0x7f2ccc0fcf70 2024-05-06T12:19:22.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.290+0000 7f2cd1707700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ccc0fbcc0 0x7f2ccc10d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:22.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.290+0000 7f2cd1707700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ccc0fbcc0 0x7f2ccc10d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48302/0 (socket says 172.21.15.92:48302) 2024-05-06T12:19:22.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.290+0000 7f2cd1707700 1 -- 172.21.15.92:0/1612150195 learned_addr learned my addr 172.21.15.92:0/1612150195 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:19:22.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.290+0000 7f2cd0f06700 1 --2- 172.21.15.92:0/1612150195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ccc0fcf70 0x7f2ccc10def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:22.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.290+0000 7f2cd1707700 1 -- 172.21.15.92:0/1612150195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ccc0fcf70 msgr2=0x7f2ccc10def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:22.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.290+0000 7f2cd1707700 1 --2- 172.21.15.92:0/1612150195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ccc0fcf70 0x7f2ccc10def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:22.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.290+0000 7f2cd1707700 1 -- 172.21.15.92:0/1612150195 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc005040 con 0x7f2ccc0fbcc0 2024-05-06T12:19:22.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.291+0000 7f2cd1707700 1 --2- 172.21.15.92:0/1612150195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ccc0fbcc0 0x7f2ccc10d9b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800c830 tx=0x7f2cc8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:19:22.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.291+0000 7f2cc1ffb700 1 -- 172.21.15.92:0/1612150195 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cc800a760 con 0x7f2ccc0fbcc0 2024-05-06T12:19:22.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.291+0000 7f2cd396b700 1 -- 172.21.15.92:0/1612150195 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ccc113070 con 0x7f2ccc0fbcc0 2024-05-06T12:19:22.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.291+0000 7f2cc1ffb700 1 -- 172.21.15.92:0/1612150195 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2cc800ad90 con 0x7f2ccc0fbcc0 2024-05-06T12:19:22.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.291+0000 7f2cc1ffb700 1 -- 172.21.15.92:0/1612150195 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cc80113a0 con 0x7f2ccc0fbcc0 2024-05-06T12:19:22.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.291+0000 7f2cd396b700 1 -- 172.21.15.92:0/1612150195 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ccc113630 con 0x7f2ccc0fbcc0 2024-05-06T12:19:22.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.292+0000 7f2cd396b700 1 -- 172.21.15.92:0/1612150195 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ccc101b00 con 0x7f2ccc0fbcc0 2024-05-06T12:19:22.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.292+0000 7f2cc1ffb700 1 -- 172.21.15.92:0/1612150195 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2cc8012400 con 0x7f2ccc0fbcc0 2024-05-06T12:19:22.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.293+0000 7f2cc1ffb700 1 --2- 172.21.15.92:0/1612150195 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f2cb806e140 0x7f2cb80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:22.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.293+0000 7f2cc1ffb700 1 -- 172.21.15.92:0/1612150195 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f2cc808db90 con 0x7f2ccc0fbcc0 2024-05-06T12:19:22.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.293+0000 7f2cd0f06700 1 --2- 172.21.15.92:0/1612150195 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f2cb806e140 0x7f2cb80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:22.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.293+0000 7f2cd0f06700 1 --2- 172.21.15.92:0/1612150195 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f2cb806e140 0x7f2cb80705f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc004f90 tx=0x7f2cbc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:19:22.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.297+0000 7f2cc1ffb700 1 -- 172.21.15.92:0/1612150195 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2cc805aac0 con 0x7f2ccc0fbcc0 2024-05-06T12:19:22.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:21 smithi190 ceph-mon[79744]: pgmap v99: 65 pgs: 15 peering, 6 active+undersized+degraded, 6 active+undersized, 38 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 19/372 objects degraded (5.108%) 2024-05-06T12:19:22.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:21 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:22.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:21 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:22.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:21 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:19:22.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:21 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:19:22.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:21 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:22.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:21 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:19:22.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:21 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:19:22.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:21 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:19:22.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:21 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:19:22.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:21 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-06T12:19:22.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.512+0000 7f2cd396b700 1 -- 172.21.15.92:0/1612150195 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ccc02b340 con 0x7f2cb806e140 2024-05-06T12:19:22.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.513+0000 7f2cc1ffb700 1 -- 172.21.15.92:0/1612150195 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f2ccc02b340 con 0x7f2cb806e140 2024-05-06T12:19:22.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.515+0000 7f2cd396b700 1 -- 172.21.15.92:0/1612150195 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f2cb806e140 msgr2=0x7f2cb80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:22.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.515+0000 7f2cd396b700 1 --2- 172.21.15.92:0/1612150195 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f2cb806e140 0x7f2cb80705f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc004f90 tx=0x7f2cbc009e30 comp rx=0 tx=0).stop 2024-05-06T12:19:22.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.515+0000 7f2cd396b700 1 -- 172.21.15.92:0/1612150195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ccc0fbcc0 msgr2=0x7f2ccc10d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:22.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.515+0000 7f2cd396b700 1 --2- 172.21.15.92:0/1612150195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ccc0fbcc0 0x7f2ccc10d9b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800c830 tx=0x7f2cc8004470 comp rx=0 tx=0).stop 2024-05-06T12:19:22.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.515+0000 7f2cd396b700 1 -- 172.21.15.92:0/1612150195 shutdown_connections 2024-05-06T12:19:22.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.515+0000 7f2cd396b700 1 --2- 172.21.15.92:0/1612150195 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f2cb806e140 0x7f2cb80705f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:22.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.515+0000 7f2cd396b700 1 --2- 172.21.15.92:0/1612150195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ccc0fbcc0 0x7f2ccc10d9b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:22.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.515+0000 7f2cd396b700 1 --2- 172.21.15.92:0/1612150195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ccc0fcf70 0x7f2ccc10def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:22.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.515+0000 7f2cd396b700 1 -- 172.21.15.92:0/1612150195 >> 172.21.15.92:0/1612150195 conn(0x7f2ccc0f7440 msgr2=0x7f2ccc066870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:19:22.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.515+0000 7f2cd396b700 1 -- 172.21.15.92:0/1612150195 shutdown_connections 2024-05-06T12:19:22.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.515+0000 7f2cd396b700 1 -- 172.21.15.92:0/1612150195 wait complete. 2024-05-06T12:19:22.528 INFO:teuthology.orchestra.run.smithi092.stdout:true 2024-05-06T12:19:22.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.618+0000 7fea7a142700 1 -- 172.21.15.92:0/2262700127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea740fdfd0 msgr2=0x7fea740fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:22.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.618+0000 7fea7a142700 1 --2- 172.21.15.92:0/2262700127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea740fdfd0 0x7fea740fe440 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fea5c004940 tx=0x7fea5c009e30 comp rx=0 tx=0).stop 2024-05-06T12:19:22.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.618+0000 7fea7a142700 1 -- 172.21.15.92:0/2262700127 shutdown_connections 2024-05-06T12:19:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.618+0000 7fea7a142700 1 --2- 172.21.15.92:0/2262700127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea740fdfd0 0x7fea740fe440 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.618+0000 7fea7a142700 1 --2- 172.21.15.92:0/2262700127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea740fce60 0x7fea740fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.618+0000 7fea7a142700 1 -- 172.21.15.92:0/2262700127 >> 172.21.15.92:0/2262700127 conn(0x7fea740f8520 msgr2=0x7fea740fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:19:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.618+0000 7fea7a142700 1 -- 172.21.15.92:0/2262700127 shutdown_connections 2024-05-06T12:19:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.618+0000 7fea7a142700 1 -- 172.21.15.92:0/2262700127 wait complete. 2024-05-06T12:19:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.619+0000 7fea7a142700 1 Processor -- start 2024-05-06T12:19:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.619+0000 7fea7a142700 1 -- start start 2024-05-06T12:19:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.619+0000 7fea7a142700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea740fce60 0x7fea7410d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.619+0000 7fea7a142700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea740fdfd0 0x7fea7410de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.619+0000 7fea7a142700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7410e480 con 0x7fea740fce60 2024-05-06T12:19:22.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.619+0000 7fea7a142700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7410e5f0 con 0x7fea740fdfd0 2024-05-06T12:19:22.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.619+0000 7fea737fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea740fce60 0x7fea7410d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:22.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.619+0000 7fea737fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea740fce60 0x7fea7410d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47608/0 (socket says 172.21.15.92:47608) 2024-05-06T12:19:22.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.619+0000 7fea737fe700 1 -- 172.21.15.92:0/2107469507 learned_addr learned my addr 172.21.15.92:0/2107469507 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:19:22.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.619+0000 7fea6affd700 1 --2- 172.21.15.92:0/2107469507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea740fdfd0 0x7fea7410de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:22.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.620+0000 7fea737fe700 1 -- 172.21.15.92:0/2107469507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea740fdfd0 msgr2=0x7fea7410de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:22.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.620+0000 7fea737fe700 1 --2- 172.21.15.92:0/2107469507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea740fdfd0 0x7fea7410de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:22.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.620+0000 7fea737fe700 1 -- 172.21.15.92:0/2107469507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea5c005040 con 0x7fea740fce60 2024-05-06T12:19:22.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.620+0000 7fea737fe700 1 --2- 172.21.15.92:0/2107469507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea740fce60 0x7fea7410d940 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fea64002a50 tx=0x7fea640046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:19:22.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.620+0000 7fea70ff9700 1 -- 172.21.15.92:0/2107469507 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea64007cc0 con 0x7fea740fce60 2024-05-06T12:19:22.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.620+0000 7fea7a142700 1 -- 172.21.15.92:0/2107469507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea74112ff0 con 0x7fea740fce60 2024-05-06T12:19:22.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.620+0000 7fea70ff9700 1 -- 172.21.15.92:0/2107469507 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fea6400f040 con 0x7fea740fce60 2024-05-06T12:19:22.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.620+0000 7fea70ff9700 1 -- 172.21.15.92:0/2107469507 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea6400da70 con 0x7fea740fce60 2024-05-06T12:19:22.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.620+0000 7fea7a142700 1 -- 172.21.15.92:0/2107469507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea74113580 con 0x7fea740fce60 2024-05-06T12:19:22.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.621+0000 7fea7a142700 1 -- 172.21.15.92:0/2107469507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea7404c710 con 0x7fea740fce60 2024-05-06T12:19:22.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.621+0000 7fea70ff9700 1 -- 172.21.15.92:0/2107469507 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fea6400dc10 con 0x7fea740fce60 2024-05-06T12:19:22.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.622+0000 7fea70ff9700 1 --2- 172.21.15.92:0/2107469507 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fea6006e140 0x7fea600705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:22.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.622+0000 7fea70ff9700 1 -- 172.21.15.92:0/2107469507 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fea6408d140 con 0x7fea740fce60 2024-05-06T12:19:22.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.622+0000 7fea6affd700 1 --2- 172.21.15.92:0/2107469507 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fea6006e140 0x7fea600705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:22.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.622+0000 7fea6affd700 1 --2- 172.21.15.92:0/2107469507 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fea6006e140 0x7fea600705f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fea5c00b970 tx=0x7fea5c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:19:22.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.627+0000 7fea70ff9700 1 -- 172.21.15.92:0/2107469507 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fea64059ff0 con 0x7fea740fce60 2024-05-06T12:19:22.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.842+0000 7fea7a142700 1 -- 172.21.15.92:0/2107469507 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fea74113830 con 0x7fea6006e140 2024-05-06T12:19:22.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.843+0000 7fea70ff9700 1 -- 172.21.15.92:0/2107469507 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fea74113830 con 0x7fea6006e140 2024-05-06T12:19:22.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.845+0000 7fea7a142700 1 -- 172.21.15.92:0/2107469507 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fea6006e140 msgr2=0x7fea600705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:22.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.845+0000 7fea7a142700 1 --2- 172.21.15.92:0/2107469507 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fea6006e140 0x7fea600705f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fea5c00b970 tx=0x7fea5c009e30 comp rx=0 tx=0).stop 2024-05-06T12:19:22.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.845+0000 7fea7a142700 1 -- 172.21.15.92:0/2107469507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea740fce60 msgr2=0x7fea7410d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:22.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.845+0000 7fea7a142700 1 --2- 172.21.15.92:0/2107469507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea740fce60 0x7fea7410d940 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fea64002a50 tx=0x7fea640046b0 comp rx=0 tx=0).stop 2024-05-06T12:19:22.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.845+0000 7fea7a142700 1 -- 172.21.15.92:0/2107469507 shutdown_connections 2024-05-06T12:19:22.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.845+0000 7fea7a142700 1 --2- 172.21.15.92:0/2107469507 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fea6006e140 0x7fea600705f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:22.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.845+0000 7fea7a142700 1 --2- 172.21.15.92:0/2107469507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea740fce60 0x7fea7410d940 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:22.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.845+0000 7fea7a142700 1 --2- 172.21.15.92:0/2107469507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea740fdfd0 0x7fea7410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:22.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.845+0000 7fea7a142700 1 -- 172.21.15.92:0/2107469507 >> 172.21.15.92:0/2107469507 conn(0x7fea740f8520 msgr2=0x7fea740f9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:19:22.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.846+0000 7fea7a142700 1 -- 172.21.15.92:0/2107469507 shutdown_connections 2024-05-06T12:19:22.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.846+0000 7fea7a142700 1 -- 172.21.15.92:0/2107469507 wait complete. 2024-05-06T12:19:22.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.946+0000 7f5259f3c700 1 -- 172.21.15.92:0/1877386635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52540fcf60 msgr2=0x7f52540fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:22.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.946+0000 7f5259f3c700 1 --2- 172.21.15.92:0/1877386635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52540fcf60 0x7f52540fd3d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5248004940 tx=0x7f5248009e30 comp rx=0 tx=0).stop 2024-05-06T12:19:22.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.946+0000 7f5259f3c700 1 -- 172.21.15.92:0/1877386635 shutdown_connections 2024-05-06T12:19:22.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.946+0000 7f5259f3c700 1 --2- 172.21.15.92:0/1877386635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52540fcf60 0x7f52540fd3d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:22.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.946+0000 7f5259f3c700 1 --2- 172.21.15.92:0/1877386635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52540fbcb0 0x7f52540fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:22.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.946+0000 7f5259f3c700 1 -- 172.21.15.92:0/1877386635 >> 172.21.15.92:0/1877386635 conn(0x7f52540f7430 msgr2=0x7f52540f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:19:22.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.946+0000 7f5259f3c700 1 -- 172.21.15.92:0/1877386635 shutdown_connections 2024-05-06T12:19:22.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.946+0000 7f5259f3c700 1 -- 172.21.15.92:0/1877386635 wait complete. 2024-05-06T12:19:22.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.946+0000 7f5259f3c700 1 Processor -- start 2024-05-06T12:19:22.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.947+0000 7f5259f3c700 1 -- start start 2024-05-06T12:19:22.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.947+0000 7f5259f3c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52540fbcb0 0x7f5254112b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:22.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.947+0000 7f5259f3c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52540fcf60 0x7f5254115060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:22.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.947+0000 7f5259f3c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5254066220 con 0x7f52540fcf60 2024-05-06T12:19:22.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.947+0000 7f5259f3c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5254066360 con 0x7f52540fbcb0 2024-05-06T12:19:22.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.947+0000 7f5252ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52540fcf60 0x7f5254115060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:22.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.947+0000 7f5252ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52540fcf60 0x7f5254115060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47632/0 (socket says 172.21.15.92:47632) 2024-05-06T12:19:22.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.947+0000 7f5252ffd700 1 -- 172.21.15.92:0/3056677425 learned_addr learned my addr 172.21.15.92:0/3056677425 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:19:22.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.947+0000 7f52537fe700 1 --2- 172.21.15.92:0/3056677425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52540fbcb0 0x7f5254112b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:22.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.947+0000 7f5252ffd700 1 -- 172.21.15.92:0/3056677425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52540fbcb0 msgr2=0x7f5254112b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:22.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.947+0000 7f5252ffd700 1 --2- 172.21.15.92:0/3056677425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52540fbcb0 0x7f5254112b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:22.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.947+0000 7f5252ffd700 1 -- 172.21.15.92:0/3056677425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5248005040 con 0x7f52540fcf60 2024-05-06T12:19:22.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.947+0000 7f5252ffd700 1 --2- 172.21.15.92:0/3056677425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52540fcf60 0x7f5254115060 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5248009150 tx=0x7f5248009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:19:22.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.948+0000 7f523bfff700 1 -- 172.21.15.92:0/3056677425 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f524800e040 con 0x7f52540fcf60 2024-05-06T12:19:22.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.948+0000 7f523bfff700 1 -- 172.21.15.92:0/3056677425 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5248004290 con 0x7f52540fcf60 2024-05-06T12:19:22.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.948+0000 7f523bfff700 1 -- 172.21.15.92:0/3056677425 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f524801c630 con 0x7f52540fcf60 2024-05-06T12:19:22.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.948+0000 7f5259f3c700 1 -- 172.21.15.92:0/3056677425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52541155a0 con 0x7f52540fcf60 2024-05-06T12:19:22.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.948+0000 7f5259f3c700 1 -- 172.21.15.92:0/3056677425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52541159c0 con 0x7f52540fcf60 2024-05-06T12:19:22.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.949+0000 7f5259f3c700 1 -- 172.21.15.92:0/3056677425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f525405f5c0 con 0x7f52540fcf60 2024-05-06T12:19:22.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.949+0000 7f523bfff700 1 -- 172.21.15.92:0/3056677425 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f524800bbf0 con 0x7f52540fcf60 2024-05-06T12:19:22.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.950+0000 7f523bfff700 1 --2- 172.21.15.92:0/3056677425 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f523c06e140 0x7f523c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:22.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.950+0000 7f523bfff700 1 -- 172.21.15.92:0/3056677425 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f524808e760 con 0x7f52540fcf60 2024-05-06T12:19:22.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.950+0000 7f52537fe700 1 --2- 172.21.15.92:0/3056677425 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f523c06e140 0x7f523c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:22.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.950+0000 7f52537fe700 1 --2- 172.21.15.92:0/3056677425 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f523c06e140 0x7f523c0705f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5254065fb0 tx=0x7f524c009780 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:19:22.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:22.954+0000 7f523bfff700 1 -- 172.21.15.92:0/3056677425 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5248057ac0 con 0x7f52540fcf60 2024-05-06T12:19:23.165 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.163+0000 7f5259f3c700 1 -- 172.21.15.92:0/3056677425 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5254101200 con 0x7f523c06e140 2024-05-06T12:19:23.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.172+0000 7f523bfff700 1 -- 172.21.15.92:0/3056677425 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f5254101200 con 0x7f523c06e140 2024-05-06T12:19:23.177 INFO:teuthology.orchestra.run.smithi092.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-06T12:19:23.177 INFO:teuthology.orchestra.run.smithi092.stdout:alertmanager.smithi092 smithi092 *:9093,9094 running (4m) 6s ago 12m 25.2M - 0.25.0 c8568f914cd2 be5637d7d25a 2024-05-06T12:19:23.177 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi092 smithi092 running (109s) 6s ago 13m 6660k - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 8da0494e1bdc 2024-05-06T12:19:23.177 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi190 smithi190 running (103s) 100s ago 12m 6665k - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 641a7cdaa56e 2024-05-06T12:19:23.178 INFO:teuthology.orchestra.run.smithi092.stdout:grafana.smithi092 smithi092 *:3000 running (4m) 6s ago 12m 75.0M - 9.4.7 954c08fa6188 9180cb3e5258 2024-05-06T12:19:23.178 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.ijsqgp smithi092 running (7m) 6s ago 7m 60.7M - 17.2.7-925-ge81df420 96d5ae7fd4b4 b4c2ed6d2c82 2024-05-06T12:19:23.178 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.rosbll smithi092 running (7m) 6s ago 7m 122M - 17.2.7-925-ge81df420 96d5ae7fd4b4 5f5426cd1bf5 2024-05-06T12:19:23.178 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.byjvgf smithi190 running (7m) 100s ago 7m 126M - 17.2.7-925-ge81df420 96d5ae7fd4b4 24581d2e1c5d 2024-05-06T12:19:23.178 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.idzoff smithi190 running (7m) 100s ago 7m 64.3M - 17.2.7-925-ge81df420 96d5ae7fd4b4 20f2641696ef 2024-05-06T12:19:23.178 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi092.gyzeqb smithi092 *:8443,9283,8765 running (3m) 6s ago 14m 504M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 de9d2198327a 2024-05-06T12:19:23.178 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi190.kczxkh smithi190 *:8443,9283,8765 running (3m) 100s ago 12m 449M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 e84ab3758ac7 2024-05-06T12:19:23.178 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi092 smithi092 running (2m) 6s ago 14m 46.0M 2048M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 ac07f7158087 2024-05-06T12:19:23.178 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi190 smithi190 running (2m) 100s ago 11m 34.0M 2048M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 7a3c08eb716f 2024-05-06T12:19:23.178 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi092 smithi092 *:9100 running (4m) 6s ago 13m 14.9M - 1.5.0 0da6a335fe13 c796446b206f 2024-05-06T12:19:23.178 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi190 smithi190 *:9100 running (4m) 100s ago 12m 16.9M - 1.5.0 0da6a335fe13 647e28111a76 2024-05-06T12:19:23.178 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 smithi092 running (83s) 6s ago 11m 82.0M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 8c96c75b25f4 2024-05-06T12:19:23.178 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 smithi092 running (47s) 6s ago 10m 82.0M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 e8007a510f08 2024-05-06T12:19:23.179 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 smithi092 running (11s) 6s ago 10m 14.8M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 2beca7448789 2024-05-06T12:19:23.179 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 smithi190 running (9m) 100s ago 9m 221M 2581M 17.2.7-925-ge81df420 96d5ae7fd4b4 52f9521286d5 2024-05-06T12:19:23.179 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 smithi190 running (9m) 100s ago 9m 204M 2581M 17.2.7-925-ge81df420 96d5ae7fd4b4 3a7da09e4c48 2024-05-06T12:19:23.179 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 smithi190 running (9m) 100s ago 9m 188M 2581M 17.2.7-925-ge81df420 96d5ae7fd4b4 e7452c6b5cec 2024-05-06T12:19:23.179 INFO:teuthology.orchestra.run.smithi092.stdout:prometheus.smithi092 smithi092 *:9095 running (3m) 6s ago 12m 54.4M - 2.43.0 a07b618ecd1d bc5445691527 2024-05-06T12:19:23.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.175+0000 7f5259f3c700 1 -- 172.21.15.92:0/3056677425 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f523c06e140 msgr2=0x7f523c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:23.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.175+0000 7f5259f3c700 1 --2- 172.21.15.92:0/3056677425 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f523c06e140 0x7f523c0705f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5254065fb0 tx=0x7f524c009780 comp rx=0 tx=0).stop 2024-05-06T12:19:23.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.175+0000 7f5259f3c700 1 -- 172.21.15.92:0/3056677425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52540fcf60 msgr2=0x7f5254115060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:23.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.175+0000 7f5259f3c700 1 --2- 172.21.15.92:0/3056677425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52540fcf60 0x7f5254115060 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5248009150 tx=0x7f5248009e30 comp rx=0 tx=0).stop 2024-05-06T12:19:23.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.175+0000 7f5259f3c700 1 -- 172.21.15.92:0/3056677425 shutdown_connections 2024-05-06T12:19:23.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.175+0000 7f5259f3c700 1 --2- 172.21.15.92:0/3056677425 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f523c06e140 0x7f523c0705f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:23.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.175+0000 7f5259f3c700 1 --2- 172.21.15.92:0/3056677425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52540fbcb0 0x7f5254112b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:23.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.175+0000 7f5259f3c700 1 --2- 172.21.15.92:0/3056677425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52540fcf60 0x7f5254115060 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:23.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.175+0000 7f5259f3c700 1 -- 172.21.15.92:0/3056677425 >> 172.21.15.92:0/3056677425 conn(0x7f52540f7430 msgr2=0x7f52540f8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:19:23.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.175+0000 7f5259f3c700 1 -- 172.21.15.92:0/3056677425 shutdown_connections 2024-05-06T12:19:23.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.175+0000 7f5259f3c700 1 -- 172.21.15.92:0/3056677425 wait complete. 2024-05-06T12:19:23.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:22 smithi092 ceph-mon[91933]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-06T12:19:23.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:22 smithi092 ceph-mon[91933]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-05-06T12:19:23.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.275+0000 7f8ca5bdb700 1 -- 172.21.15.92:0/1135284398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ca00fce60 msgr2=0x7f8ca00fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:23.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.275+0000 7f8ca5bdb700 1 --2- 172.21.15.92:0/1135284398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ca00fce60 0x7f8ca00fd270 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8c94004940 tx=0x7f8c94009e30 comp rx=0 tx=0).stop 2024-05-06T12:19:23.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.275+0000 7f8ca5bdb700 1 -- 172.21.15.92:0/1135284398 shutdown_connections 2024-05-06T12:19:23.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.275+0000 7f8ca5bdb700 1 --2- 172.21.15.92:0/1135284398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ca00fdfd0 0x7f8ca00fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:23.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.275+0000 7f8ca5bdb700 1 --2- 172.21.15.92:0/1135284398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ca00fce60 0x7f8ca00fd270 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:23.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.275+0000 7f8ca5bdb700 1 -- 172.21.15.92:0/1135284398 >> 172.21.15.92:0/1135284398 conn(0x7f8ca00f8540 msgr2=0x7f8ca00fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:19:23.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.275+0000 7f8ca5bdb700 1 -- 172.21.15.92:0/1135284398 shutdown_connections 2024-05-06T12:19:23.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.275+0000 7f8ca5bdb700 1 -- 172.21.15.92:0/1135284398 wait complete. 2024-05-06T12:19:23.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.276+0000 7f8ca5bdb700 1 Processor -- start 2024-05-06T12:19:23.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.276+0000 7f8ca5bdb700 1 -- start start 2024-05-06T12:19:23.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.276+0000 7f8ca5bdb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ca00fce60 0x7f8ca010d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:23.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.276+0000 7f8ca5bdb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ca00fdfd0 0x7f8ca010de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:23.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.276+0000 7f8ca5bdb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca010e450 con 0x7f8ca00fdfd0 2024-05-06T12:19:23.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.276+0000 7f8ca5bdb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca010e5c0 con 0x7f8ca00fce60 2024-05-06T12:19:23.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.276+0000 7f8c9effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ca00fdfd0 0x7f8ca010de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:23.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.276+0000 7f8c9effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ca00fdfd0 0x7f8ca010de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47656/0 (socket says 172.21.15.92:47656) 2024-05-06T12:19:23.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.276+0000 7f8c9effd700 1 -- 172.21.15.92:0/3389113640 learned_addr learned my addr 172.21.15.92:0/3389113640 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:19:23.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.276+0000 7f8c9f7fe700 1 --2- 172.21.15.92:0/3389113640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ca00fce60 0x7f8ca010d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:23.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.276+0000 7f8c9effd700 1 -- 172.21.15.92:0/3389113640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ca00fce60 msgr2=0x7f8ca010d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:23.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.276+0000 7f8c9effd700 1 --2- 172.21.15.92:0/3389113640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ca00fce60 0x7f8ca010d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:23.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.276+0000 7f8c9effd700 1 -- 172.21.15.92:0/3389113640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c94005040 con 0x7f8ca00fdfd0 2024-05-06T12:19:23.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.277+0000 7f8c9effd700 1 --2- 172.21.15.92:0/3389113640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ca00fdfd0 0x7f8ca010de50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8c9000c830 tx=0x7f8c90004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:19:23.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.277+0000 7f8c87fff700 1 -- 172.21.15.92:0/3389113640 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c9000a760 con 0x7f8ca00fdfd0 2024-05-06T12:19:23.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.277+0000 7f8c87fff700 1 -- 172.21.15.92:0/3389113640 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8c9000ad90 con 0x7f8ca00fdfd0 2024-05-06T12:19:23.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.277+0000 7f8ca5bdb700 1 -- 172.21.15.92:0/3389113640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca0113020 con 0x7f8ca00fdfd0 2024-05-06T12:19:23.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.277+0000 7f8c87fff700 1 -- 172.21.15.92:0/3389113640 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c900113a0 con 0x7f8ca00fdfd0 2024-05-06T12:19:23.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.277+0000 7f8ca5bdb700 1 -- 172.21.15.92:0/3389113640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca01135b0 con 0x7f8ca00fdfd0 2024-05-06T12:19:23.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.278+0000 7f8ca5bdb700 1 -- 172.21.15.92:0/3389113640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ca004c710 con 0x7f8ca00fdfd0 2024-05-06T12:19:23.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.278+0000 7f8c87fff700 1 -- 172.21.15.92:0/3389113640 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8c90012400 con 0x7f8ca00fdfd0 2024-05-06T12:19:23.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.279+0000 7f8c87fff700 1 --2- 172.21.15.92:0/3389113640 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f8c8806e140 0x7f8c880705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:23.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.279+0000 7f8c87fff700 1 -- 172.21.15.92:0/3389113640 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f8c9008db40 con 0x7f8ca00fdfd0 2024-05-06T12:19:23.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.279+0000 7f8c9f7fe700 1 --2- 172.21.15.92:0/3389113640 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f8c8806e140 0x7f8c880705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:23.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.279+0000 7f8c9f7fe700 1 --2- 172.21.15.92:0/3389113640 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f8c8806e140 0x7f8c880705f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400b3b0 tx=0x7f8c94009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:19:23.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.283+0000 7f8c87fff700 1 -- 172.21.15.92:0/3389113640 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f8c9005aa70 con 0x7f8ca00fdfd0 2024-05-06T12:19:23.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:22 smithi190 ceph-mon[79744]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-06T12:19:23.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:22 smithi190 ceph-mon[79744]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-05-06T12:19:23.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.565+0000 7f8ca5bdb700 1 -- 172.21.15.92:0/3389113640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8ca01058c0 con 0x7f8ca00fdfd0 2024-05-06T12:19:23.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.566+0000 7f8c87fff700 1 -- 172.21.15.92:0/3389113640 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f8c9005a600 con 0x7f8ca00fdfd0 2024-05-06T12:19:23.568 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:19:23.569 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": { 2024-05-06T12:19:23.569 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:19:23.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:19:23.569 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": { 2024-05-06T12:19:23.569 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:19:23.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:19:23.570 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": { 2024-05-06T12:19:23.570 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 3, 2024-05-06T12:19:23.570 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 3 2024-05-06T12:19:23.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:19:23.570 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-05-06T12:19:23.571 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 4 2024-05-06T12:19:23.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:19:23.571 INFO:teuthology.orchestra.run.smithi092.stdout: "overall": { 2024-05-06T12:19:23.571 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 7, 2024-05-06T12:19:23.571 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 7 2024-05-06T12:19:23.571 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-06T12:19:23.571 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:19:23.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.568+0000 7f8ca5bdb700 1 -- 172.21.15.92:0/3389113640 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f8c8806e140 msgr2=0x7f8c880705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:23.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.569+0000 7f8ca5bdb700 1 --2- 172.21.15.92:0/3389113640 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f8c8806e140 0x7f8c880705f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400b3b0 tx=0x7f8c94009e30 comp rx=0 tx=0).stop 2024-05-06T12:19:23.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.569+0000 7f8ca5bdb700 1 -- 172.21.15.92:0/3389113640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ca00fdfd0 msgr2=0x7f8ca010de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:23.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.569+0000 7f8ca5bdb700 1 --2- 172.21.15.92:0/3389113640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ca00fdfd0 0x7f8ca010de50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8c9000c830 tx=0x7f8c90004470 comp rx=0 tx=0).stop 2024-05-06T12:19:23.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.569+0000 7f8ca5bdb700 1 -- 172.21.15.92:0/3389113640 shutdown_connections 2024-05-06T12:19:23.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.569+0000 7f8ca5bdb700 1 --2- 172.21.15.92:0/3389113640 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f8c8806e140 0x7f8c880705f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:23.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.569+0000 7f8ca5bdb700 1 --2- 172.21.15.92:0/3389113640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ca00fce60 0x7f8ca010d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:23.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.569+0000 7f8ca5bdb700 1 --2- 172.21.15.92:0/3389113640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ca00fdfd0 0x7f8ca010de50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:23.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.569+0000 7f8ca5bdb700 1 -- 172.21.15.92:0/3389113640 >> 172.21.15.92:0/3389113640 conn(0x7f8ca00f8540 msgr2=0x7f8ca00f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:19:23.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.569+0000 7f8ca5bdb700 1 -- 172.21.15.92:0/3389113640 shutdown_connections 2024-05-06T12:19:23.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.569+0000 7f8ca5bdb700 1 -- 172.21.15.92:0/3389113640 wait complete. 2024-05-06T12:19:23.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.669+0000 7ff33a283700 1 -- 172.21.15.92:0/647315566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3340fcc60 msgr2=0x7ff3340fd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:23.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.669+0000 7ff33a283700 1 --2- 172.21.15.92:0/647315566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3340fcc60 0x7ff3340fd0d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff328004940 tx=0x7ff328009e30 comp rx=0 tx=0).stop 2024-05-06T12:19:23.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.669+0000 7ff33a283700 1 -- 172.21.15.92:0/647315566 shutdown_connections 2024-05-06T12:19:23.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.669+0000 7ff33a283700 1 --2- 172.21.15.92:0/647315566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3340fcc60 0x7ff3340fd0d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:23.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.669+0000 7ff33a283700 1 --2- 172.21.15.92:0/647315566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3340fc310 0x7ff3340fc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:23.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.669+0000 7ff33a283700 1 -- 172.21.15.92:0/647315566 >> 172.21.15.92:0/647315566 conn(0x7ff3340f6b80 msgr2=0x7ff3340f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:19:23.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.669+0000 7ff33a283700 1 -- 172.21.15.92:0/647315566 shutdown_connections 2024-05-06T12:19:23.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.669+0000 7ff33a283700 1 -- 172.21.15.92:0/647315566 wait complete. 2024-05-06T12:19:23.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.670+0000 7ff33a283700 1 Processor -- start 2024-05-06T12:19:23.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.670+0000 7ff33a283700 1 -- start start 2024-05-06T12:19:23.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.670+0000 7ff33a283700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3340fc310 0x7ff334110740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:23.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.670+0000 7ff33a283700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3340fcc60 0x7ff334112c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:23.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.670+0000 7ff33a283700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3340fbd90 con 0x7ff3340fc310 2024-05-06T12:19:23.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.670+0000 7ff33a283700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3340fbed0 con 0x7ff3340fcc60 2024-05-06T12:19:23.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.670+0000 7ff333fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3340fc310 0x7ff334110740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:23.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.670+0000 7ff333fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3340fc310 0x7ff334110740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47684/0 (socket says 172.21.15.92:47684) 2024-05-06T12:19:23.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.670+0000 7ff333fff700 1 -- 172.21.15.92:0/1082422654 learned_addr learned my addr 172.21.15.92:0/1082422654 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:19:23.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.670+0000 7ff3337fe700 1 --2- 172.21.15.92:0/1082422654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3340fcc60 0x7ff334112c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:23.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.670+0000 7ff333fff700 1 -- 172.21.15.92:0/1082422654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3340fcc60 msgr2=0x7ff334112c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:23.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.670+0000 7ff333fff700 1 --2- 172.21.15.92:0/1082422654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3340fcc60 0x7ff334112c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:23.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.670+0000 7ff333fff700 1 -- 172.21.15.92:0/1082422654 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff328005040 con 0x7ff3340fc310 2024-05-06T12:19:23.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.671+0000 7ff333fff700 1 --2- 172.21.15.92:0/1082422654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3340fc310 0x7ff334110740 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff32400b790 tx=0x7ff324009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:19:23.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.671+0000 7ff330ff9700 1 -- 172.21.15.92:0/1082422654 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff32400f770 con 0x7ff3340fc310 2024-05-06T12:19:23.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.671+0000 7ff330ff9700 1 -- 172.21.15.92:0/1082422654 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff32400fda0 con 0x7ff3340fc310 2024-05-06T12:19:23.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.671+0000 7ff33a283700 1 -- 172.21.15.92:0/1082422654 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3341132f0 con 0x7ff3340fc310 2024-05-06T12:19:23.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.671+0000 7ff330ff9700 1 -- 172.21.15.92:0/1082422654 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff32400d310 con 0x7ff3340fc310 2024-05-06T12:19:23.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.671+0000 7ff33a283700 1 -- 172.21.15.92:0/1082422654 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff334113880 con 0x7ff3340fc310 2024-05-06T12:19:23.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.672+0000 7ff33a283700 1 -- 172.21.15.92:0/1082422654 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff33405f5c0 con 0x7ff3340fc310 2024-05-06T12:19:23.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.672+0000 7ff330ff9700 1 -- 172.21.15.92:0/1082422654 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff324010420 con 0x7ff3340fc310 2024-05-06T12:19:23.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.673+0000 7ff330ff9700 1 --2- 172.21.15.92:0/1082422654 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff31c06e140 0x7ff31c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:23.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.673+0000 7ff330ff9700 1 -- 172.21.15.92:0/1082422654 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff32408db40 con 0x7ff3340fc310 2024-05-06T12:19:23.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.673+0000 7ff3337fe700 1 --2- 172.21.15.92:0/1082422654 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff31c06e140 0x7ff31c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:23.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.673+0000 7ff3337fe700 1 --2- 172.21.15.92:0/1082422654 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff31c06e140 0x7ff31c0705f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff32800b970 tx=0x7ff328009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:19:23.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.677+0000 7ff330ff9700 1 -- 172.21.15.92:0/1082422654 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff324056f40 con 0x7ff3340fc310 2024-05-06T12:19:23.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.920+0000 7ff33a283700 1 -- 172.21.15.92:0/1082422654 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff334113b00 con 0x7ff3340fc310 2024-05-06T12:19:23.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.920+0000 7ff330ff9700 1 -- 172.21.15.92:0/1082422654 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1918 (secure 0 0 0) 0x7ff32405a600 con 0x7ff3340fc310 2024-05-06T12:19:23.922 INFO:teuthology.orchestra.run.smithi092.stdout:e22 2024-05-06T12:19:23.922 INFO:teuthology.orchestra.run.smithi092.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-06T12:19:23.922 INFO:teuthology.orchestra.run.smithi092.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:19:23.923 INFO:teuthology.orchestra.run.smithi092.stdout:legacy client fscid: 1 2024-05-06T12:19:23.923 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:19:23.923 INFO:teuthology.orchestra.run.smithi092.stdout:Filesystem 'cephfs' (1) 2024-05-06T12:19:23.923 INFO:teuthology.orchestra.run.smithi092.stdout:fs_name cephfs 2024-05-06T12:19:23.923 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 22 2024-05-06T12:19:23.923 INFO:teuthology.orchestra.run.smithi092.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-06T12:19:23.923 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-05-06T12:11:44.952671+0000 2024-05-06T12:19:23.923 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2024-05-06T12:19:06.643003+0000 2024-05-06T12:19:23.923 INFO:teuthology.orchestra.run.smithi092.stdout:tableserver 0 2024-05-06T12:19:23.923 INFO:teuthology.orchestra.run.smithi092.stdout:root 0 2024-05-06T12:19:23.923 INFO:teuthology.orchestra.run.smithi092.stdout:session_timeout 60 2024-05-06T12:19:23.923 INFO:teuthology.orchestra.run.smithi092.stdout:session_autoclose 300 2024-05-06T12:19:23.923 INFO:teuthology.orchestra.run.smithi092.stdout:max_file_size 1099511627776 2024-05-06T12:19:23.923 INFO:teuthology.orchestra.run.smithi092.stdout:max_xattr_size 65536 2024-05-06T12:19:23.923 INFO:teuthology.orchestra.run.smithi092.stdout:required_client_features {} 2024-05-06T12:19:23.923 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure 0 2024-05-06T12:19:23.924 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure_osd_epoch 46 2024-05-06T12:19:23.924 INFO:teuthology.orchestra.run.smithi092.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:19:23.924 INFO:teuthology.orchestra.run.smithi092.stdout:max_mds 2 2024-05-06T12:19:23.924 INFO:teuthology.orchestra.run.smithi092.stdout:in 0,1 2024-05-06T12:19:23.924 INFO:teuthology.orchestra.run.smithi092.stdout:up {0=14472,1=24267} 2024-05-06T12:19:23.924 INFO:teuthology.orchestra.run.smithi092.stdout:failed 2024-05-06T12:19:23.924 INFO:teuthology.orchestra.run.smithi092.stdout:damaged 2024-05-06T12:19:23.924 INFO:teuthology.orchestra.run.smithi092.stdout:stopped 2024-05-06T12:19:23.924 INFO:teuthology.orchestra.run.smithi092.stdout:data_pools [3] 2024-05-06T12:19:23.924 INFO:teuthology.orchestra.run.smithi092.stdout:metadata_pool 2 2024-05-06T12:19:23.924 INFO:teuthology.orchestra.run.smithi092.stdout:inline_data enabled 2024-05-06T12:19:23.924 INFO:teuthology.orchestra.run.smithi092.stdout:balancer 2024-05-06T12:19:23.924 INFO:teuthology.orchestra.run.smithi092.stdout:bal_rank_mask -1 2024-05-06T12:19:23.924 INFO:teuthology.orchestra.run.smithi092.stdout:standby_count_wanted 1 2024-05-06T12:19:23.925 INFO:teuthology.orchestra.run.smithi092.stdout:qdb_cluster leader: 14472 members: 24267,14472 2024-05-06T12:19:23.925 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.rosbll{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.92:6828/1663253598,v1:172.21.15.92:6829/1663253598] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:19:23.925 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.ijsqgp{0:14484} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.92:6826/3613441054,v1:172.21.15.92:6827/3613441054] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:19:23.925 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.byjvgf{1:24267} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.190:6824/1565151699,v1:172.21.15.190:6825/1565151699] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:19:23.925 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.idzoff{1:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.190:6826/2453728380,v1:172.21.15.190:6827/2453728380] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:19:23.925 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:19:23.925 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:19:23.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.922+0000 7ff33a283700 1 -- 172.21.15.92:0/1082422654 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff31c06e140 msgr2=0x7ff31c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:23.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.922+0000 7ff33a283700 1 --2- 172.21.15.92:0/1082422654 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff31c06e140 0x7ff31c0705f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff32800b970 tx=0x7ff328009e30 comp rx=0 tx=0).stop 2024-05-06T12:19:23.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.922+0000 7ff33a283700 1 -- 172.21.15.92:0/1082422654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3340fc310 msgr2=0x7ff334110740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:23.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.922+0000 7ff33a283700 1 --2- 172.21.15.92:0/1082422654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3340fc310 0x7ff334110740 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff32400b790 tx=0x7ff324009250 comp rx=0 tx=0).stop 2024-05-06T12:19:23.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.922+0000 7ff33a283700 1 -- 172.21.15.92:0/1082422654 shutdown_connections 2024-05-06T12:19:23.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.922+0000 7ff33a283700 1 --2- 172.21.15.92:0/1082422654 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff31c06e140 0x7ff31c0705f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:23.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.922+0000 7ff33a283700 1 --2- 172.21.15.92:0/1082422654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3340fc310 0x7ff334110740 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:23.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.922+0000 7ff33a283700 1 --2- 172.21.15.92:0/1082422654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3340fcc60 0x7ff334112c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:23.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.922+0000 7ff33a283700 1 -- 172.21.15.92:0/1082422654 >> 172.21.15.92:0/1082422654 conn(0x7ff3340f6b80 msgr2=0x7ff3340f8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:19:23.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.922+0000 7ff33a283700 1 -- 172.21.15.92:0/1082422654 shutdown_connections 2024-05-06T12:19:23.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:23.922+0000 7ff33a283700 1 -- 172.21.15.92:0/1082422654 wait complete. 2024-05-06T12:19:23.927 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 22 2024-05-06T12:19:24.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.026+0000 7fa843123700 1 -- 172.21.15.92:0/4136169768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa83c0fcf70 msgr2=0x7fa83c0fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:24.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.026+0000 7fa843123700 1 --2- 172.21.15.92:0/4136169768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa83c0fcf70 0x7fa83c0fd3e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa828004940 tx=0x7fa828009e30 comp rx=0 tx=0).stop 2024-05-06T12:19:24.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.026+0000 7fa843123700 1 -- 172.21.15.92:0/4136169768 shutdown_connections 2024-05-06T12:19:24.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.026+0000 7fa843123700 1 --2- 172.21.15.92:0/4136169768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa83c0fcf70 0x7fa83c0fd3e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:24.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.026+0000 7fa843123700 1 --2- 172.21.15.92:0/4136169768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa83c0fbcc0 0x7fa83c0fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:24.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.026+0000 7fa843123700 1 -- 172.21.15.92:0/4136169768 >> 172.21.15.92:0/4136169768 conn(0x7fa83c0f7440 msgr2=0x7fa83c0f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:19:24.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.026+0000 7fa843123700 1 -- 172.21.15.92:0/4136169768 shutdown_connections 2024-05-06T12:19:24.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.026+0000 7fa843123700 1 -- 172.21.15.92:0/4136169768 wait complete. 2024-05-06T12:19:24.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.026+0000 7fa843123700 1 Processor -- start 2024-05-06T12:19:24.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.026+0000 7fa843123700 1 -- start start 2024-05-06T12:19:24.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.027+0000 7fa843123700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa83c0fbcc0 0x7fa83c066cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:24.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.027+0000 7fa843123700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa83c0fcf70 0x7fa83c112740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:24.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.027+0000 7fa843123700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c112c80 con 0x7fa83c0fbcc0 2024-05-06T12:19:24.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.027+0000 7fa843123700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c112df0 con 0x7fa83c0fcf70 2024-05-06T12:19:24.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.027+0000 7fa840ebf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa83c0fbcc0 0x7fa83c066cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:24.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.027+0000 7fa840ebf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa83c0fbcc0 0x7fa83c066cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47700/0 (socket says 172.21.15.92:47700) 2024-05-06T12:19:24.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.027+0000 7fa840ebf700 1 -- 172.21.15.92:0/387780889 learned_addr learned my addr 172.21.15.92:0/387780889 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:19:24.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.027+0000 7fa840ebf700 1 -- 172.21.15.92:0/387780889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa83c0fcf70 msgr2=0x7fa83c112740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:24.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.027+0000 7fa840ebf700 1 --2- 172.21.15.92:0/387780889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa83c0fcf70 0x7fa83c112740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:24.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.027+0000 7fa840ebf700 1 -- 172.21.15.92:0/387780889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa828005040 con 0x7fa83c0fbcc0 2024-05-06T12:19:24.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.027+0000 7fa840ebf700 1 --2- 172.21.15.92:0/387780889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa83c0fbcc0 0x7fa83c066cf0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa83000b790 tx=0x7fa830009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:19:24.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.027+0000 7fa8397fa700 1 -- 172.21.15.92:0/387780889 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa83000f770 con 0x7fa83c0fbcc0 2024-05-06T12:19:24.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.028+0000 7fa843123700 1 -- 172.21.15.92:0/387780889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa83c1130d0 con 0x7fa83c0fbcc0 2024-05-06T12:19:24.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.028+0000 7fa8397fa700 1 -- 172.21.15.92:0/387780889 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa83000fda0 con 0x7fa83c0fbcc0 2024-05-06T12:19:24.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.028+0000 7fa8397fa700 1 -- 172.21.15.92:0/387780889 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa83000d310 con 0x7fa83c0fbcc0 2024-05-06T12:19:24.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.028+0000 7fa843123700 1 -- 172.21.15.92:0/387780889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa83c113690 con 0x7fa83c0fbcc0 2024-05-06T12:19:24.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.028+0000 7fa843123700 1 -- 172.21.15.92:0/387780889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa83c04c710 con 0x7fa83c0fbcc0 2024-05-06T12:19:24.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.029+0000 7fa8397fa700 1 -- 172.21.15.92:0/387780889 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa830010420 con 0x7fa83c0fbcc0 2024-05-06T12:19:24.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.029+0000 7fa8397fa700 1 --2- 172.21.15.92:0/387780889 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fa82c06e140 0x7fa82c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:24.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.029+0000 7fa8397fa700 1 -- 172.21.15.92:0/387780889 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa83008dbc0 con 0x7fa83c0fbcc0 2024-05-06T12:19:24.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.029+0000 7fa83bfff700 1 --2- 172.21.15.92:0/387780889 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fa82c06e140 0x7fa82c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:24.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.030+0000 7fa83bfff700 1 --2- 172.21.15.92:0/387780889 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fa82c06e140 0x7fa82c0705f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa828004f90 tx=0x7fa828009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:19:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.034+0000 7fa8397fa700 1 -- 172.21.15.92:0/387780889 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa83005aa90 con 0x7fa83c0fbcc0 2024-05-06T12:19:24.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:23 smithi092 ceph-mon[91933]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:19:24.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:23 smithi092 ceph-mon[91933]: pgmap v100: 65 pgs: 15 peering, 50 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-05-06T12:19:24.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:23 smithi092 ceph-mon[91933]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:19:24.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:23 smithi092 ceph-mon[91933]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 19/372 objects degraded (5.108%), 6 pgs degraded) 2024-05-06T12:19:24.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:23 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/3389113640' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:19:24.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:23 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:24.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:23 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:19:24.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:23 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/1082422654' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:19:24.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.253+0000 7fa843123700 1 -- 172.21.15.92:0/387780889 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa83c113910 con 0x7fa82c06e140 2024-05-06T12:19:24.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.254+0000 7fa8397fa700 1 -- 172.21.15.92:0/387780889 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fa83c113910 con 0x7fa82c06e140 2024-05-06T12:19:24.256 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:19:24.256 INFO:teuthology.orchestra.run.smithi092.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb", 2024-05-06T12:19:24.256 INFO:teuthology.orchestra.run.smithi092.stdout: "in_progress": true, 2024-05-06T12:19:24.256 INFO:teuthology.orchestra.run.smithi092.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-06T12:19:24.256 INFO:teuthology.orchestra.run.smithi092.stdout: "services_complete": [ 2024-05-06T12:19:24.256 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr", 2024-05-06T12:19:24.256 INFO:teuthology.orchestra.run.smithi092.stdout: "mon", 2024-05-06T12:19:24.256 INFO:teuthology.orchestra.run.smithi092.stdout: "crash" 2024-05-06T12:19:24.256 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-05-06T12:19:24.257 INFO:teuthology.orchestra.run.smithi092.stdout: "progress": "9/21 daemons upgraded", 2024-05-06T12:19:24.257 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "Currently upgrading osd daemons", 2024-05-06T12:19:24.257 INFO:teuthology.orchestra.run.smithi092.stdout: "is_paused": false 2024-05-06T12:19:24.257 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:19:24.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.256+0000 7fa843123700 1 -- 172.21.15.92:0/387780889 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fa82c06e140 msgr2=0x7fa82c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:24.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.256+0000 7fa843123700 1 --2- 172.21.15.92:0/387780889 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fa82c06e140 0x7fa82c0705f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa828004f90 tx=0x7fa828009e30 comp rx=0 tx=0).stop 2024-05-06T12:19:24.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.256+0000 7fa843123700 1 -- 172.21.15.92:0/387780889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa83c0fbcc0 msgr2=0x7fa83c066cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:24.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.256+0000 7fa843123700 1 --2- 172.21.15.92:0/387780889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa83c0fbcc0 0x7fa83c066cf0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa83000b790 tx=0x7fa830009250 comp rx=0 tx=0).stop 2024-05-06T12:19:24.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.256+0000 7fa843123700 1 -- 172.21.15.92:0/387780889 shutdown_connections 2024-05-06T12:19:24.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.256+0000 7fa843123700 1 --2- 172.21.15.92:0/387780889 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fa82c06e140 0x7fa82c0705f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:24.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.256+0000 7fa843123700 1 --2- 172.21.15.92:0/387780889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa83c0fbcc0 0x7fa83c066cf0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:24.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.256+0000 7fa843123700 1 --2- 172.21.15.92:0/387780889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa83c0fcf70 0x7fa83c112740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:24.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.256+0000 7fa843123700 1 -- 172.21.15.92:0/387780889 >> 172.21.15.92:0/387780889 conn(0x7fa83c0f7440 msgr2=0x7fa83c0f8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:19:24.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.256+0000 7fa843123700 1 -- 172.21.15.92:0/387780889 shutdown_connections 2024-05-06T12:19:24.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.256+0000 7fa843123700 1 -- 172.21.15.92:0/387780889 wait complete. 2024-05-06T12:19:24.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:23 smithi190 ceph-mon[79744]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:19:24.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:23 smithi190 ceph-mon[79744]: pgmap v100: 65 pgs: 15 peering, 50 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-05-06T12:19:24.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:23 smithi190 ceph-mon[79744]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:19:24.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:23 smithi190 ceph-mon[79744]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 19/372 objects degraded (5.108%), 6 pgs degraded) 2024-05-06T12:19:24.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:23 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/3389113640' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:19:24.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:23 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:24.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:23 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:19:24.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:23 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/1082422654' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:19:24.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.360+0000 7f7aa6add700 1 -- 172.21.15.92:0/3331256269 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7aa00fcf70 msgr2=0x7f7aa00fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:24.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.360+0000 7f7aa6add700 1 --2- 172.21.15.92:0/3331256269 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7aa00fcf70 0x7f7aa00fd3e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7a90004940 tx=0x7f7a90009e30 comp rx=0 tx=0).stop 2024-05-06T12:19:24.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.360+0000 7f7aa6add700 1 -- 172.21.15.92:0/3331256269 shutdown_connections 2024-05-06T12:19:24.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.360+0000 7f7aa6add700 1 --2- 172.21.15.92:0/3331256269 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7aa00fcf70 0x7f7aa00fd3e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:24.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.360+0000 7f7aa6add700 1 --2- 172.21.15.92:0/3331256269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7aa00fbcc0 0x7f7aa00fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:24.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.360+0000 7f7aa6add700 1 -- 172.21.15.92:0/3331256269 >> 172.21.15.92:0/3331256269 conn(0x7f7aa00f7440 msgr2=0x7f7aa00f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:19:24.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.360+0000 7f7aa6add700 1 -- 172.21.15.92:0/3331256269 shutdown_connections 2024-05-06T12:19:24.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.360+0000 7f7aa6add700 1 -- 172.21.15.92:0/3331256269 wait complete. 2024-05-06T12:19:24.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.361+0000 7f7aa6add700 1 Processor -- start 2024-05-06T12:19:24.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.361+0000 7f7aa6add700 1 -- start start 2024-05-06T12:19:24.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.361+0000 7f7aa6add700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7aa00fbcc0 0x7f7aa010c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:24.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.361+0000 7f7aa6add700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7aa00fcf70 0x7f7aa010cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:24.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.361+0000 7f7aa6add700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa010d3f0 con 0x7f7aa00fbcc0 2024-05-06T12:19:24.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.361+0000 7f7aa6add700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa010d560 con 0x7f7aa00fcf70 2024-05-06T12:19:24.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.361+0000 7f7aa4879700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7aa00fbcc0 0x7f7aa010c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:24.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.361+0000 7f7aa4879700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7aa00fbcc0 0x7f7aa010c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47720/0 (socket says 172.21.15.92:47720) 2024-05-06T12:19:24.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.361+0000 7f7aa4879700 1 -- 172.21.15.92:0/4071448731 learned_addr learned my addr 172.21.15.92:0/4071448731 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:19:24.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.361+0000 7f7a9ffff700 1 --2- 172.21.15.92:0/4071448731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7aa00fcf70 0x7f7aa010cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:24.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.362+0000 7f7aa4879700 1 -- 172.21.15.92:0/4071448731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7aa00fcf70 msgr2=0x7f7aa010cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:24.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.362+0000 7f7aa4879700 1 --2- 172.21.15.92:0/4071448731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7aa00fcf70 0x7f7aa010cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:24.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.362+0000 7f7aa4879700 1 -- 172.21.15.92:0/4071448731 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a90005040 con 0x7f7aa00fbcc0 2024-05-06T12:19:24.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.362+0000 7f7aa4879700 1 --2- 172.21.15.92:0/4071448731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7aa00fbcc0 0x7f7aa010c8b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7a98001d00 tx=0x7f7a98009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:19:24.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.362+0000 7f7a9d7fa700 1 -- 172.21.15.92:0/4071448731 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a98001e60 con 0x7f7aa00fbcc0 2024-05-06T12:19:24.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.362+0000 7f7a9d7fa700 1 -- 172.21.15.92:0/4071448731 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a9800f040 con 0x7f7aa00fbcc0 2024-05-06T12:19:24.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.362+0000 7f7aa6add700 1 -- 172.21.15.92:0/4071448731 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aa0111fc0 con 0x7f7aa00fbcc0 2024-05-06T12:19:24.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.362+0000 7f7a9d7fa700 1 -- 172.21.15.92:0/4071448731 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a98009310 con 0x7f7aa00fbcc0 2024-05-06T12:19:24.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.362+0000 7f7aa6add700 1 -- 172.21.15.92:0/4071448731 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa01124f0 con 0x7f7aa00fbcc0 2024-05-06T12:19:24.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.363+0000 7f7aa6add700 1 -- 172.21.15.92:0/4071448731 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aa004c710 con 0x7f7aa00fbcc0 2024-05-06T12:19:24.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.363+0000 7f7a9d7fa700 1 -- 172.21.15.92:0/4071448731 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7a98007920 con 0x7f7aa00fbcc0 2024-05-06T12:19:24.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.364+0000 7f7a9d7fa700 1 --2- 172.21.15.92:0/4071448731 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7a8c06e140 0x7f7a8c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:24.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.364+0000 7f7a9d7fa700 1 -- 172.21.15.92:0/4071448731 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f7a9808d490 con 0x7f7aa00fbcc0 2024-05-06T12:19:24.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.364+0000 7f7a9ffff700 1 --2- 172.21.15.92:0/4071448731 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7a8c06e140 0x7f7a8c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:24.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.364+0000 7f7a9ffff700 1 --2- 172.21.15.92:0/4071448731 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7a8c06e140 0x7f7a8c0705f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7a9000b3b0 tx=0x7f7a90009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:19:24.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.368+0000 7f7a9d7fa700 1 -- 172.21.15.92:0/4071448731 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7a9805a340 con 0x7f7aa00fbcc0 2024-05-06T12:19:24.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.649+0000 7f7aa6add700 1 -- 172.21.15.92:0/4071448731 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7aa01047d0 con 0x7f7aa00fbcc0 2024-05-06T12:19:24.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.649+0000 7f7a9d7fa700 1 -- 172.21.15.92:0/4071448731 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f7a98059ed0 con 0x7f7aa00fbcc0 2024-05-06T12:19:24.651 INFO:teuthology.orchestra.run.smithi092.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-06T12:19:24.651 INFO:teuthology.orchestra.run.smithi092.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-06T12:19:24.652 INFO:teuthology.orchestra.run.smithi092.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-06T12:19:24.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.651+0000 7f7aa6add700 1 -- 172.21.15.92:0/4071448731 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7a8c06e140 msgr2=0x7f7a8c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:24.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.651+0000 7f7aa6add700 1 --2- 172.21.15.92:0/4071448731 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7a8c06e140 0x7f7a8c0705f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7a9000b3b0 tx=0x7f7a90009e30 comp rx=0 tx=0).stop 2024-05-06T12:19:24.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.651+0000 7f7aa6add700 1 -- 172.21.15.92:0/4071448731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7aa00fbcc0 msgr2=0x7f7aa010c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:24.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.651+0000 7f7aa6add700 1 --2- 172.21.15.92:0/4071448731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7aa00fbcc0 0x7f7aa010c8b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7a98001d00 tx=0x7f7a98009f30 comp rx=0 tx=0).stop 2024-05-06T12:19:24.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.651+0000 7f7aa6add700 1 -- 172.21.15.92:0/4071448731 shutdown_connections 2024-05-06T12:19:24.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.651+0000 7f7aa6add700 1 --2- 172.21.15.92:0/4071448731 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7a8c06e140 0x7f7a8c0705f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:24.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.652+0000 7f7aa6add700 1 --2- 172.21.15.92:0/4071448731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7aa00fbcc0 0x7f7aa010c8b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:24.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.652+0000 7f7aa6add700 1 --2- 172.21.15.92:0/4071448731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7aa00fcf70 0x7f7aa010cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:24.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.652+0000 7f7aa6add700 1 -- 172.21.15.92:0/4071448731 >> 172.21.15.92:0/4071448731 conn(0x7f7aa00f7440 msgr2=0x7f7aa00f8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:19:24.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.652+0000 7f7aa6add700 1 -- 172.21.15.92:0/4071448731 shutdown_connections 2024-05-06T12:19:24.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:24.652+0000 7f7aa6add700 1 -- 172.21.15.92:0/4071448731 wait complete. 2024-05-06T12:19:25.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:25 smithi190 ceph-mon[79744]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:19:25.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:25 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/4071448731' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:19:25.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:25 smithi092 ceph-mon[91933]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:19:25.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:25 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/4071448731' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:19:26.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:26 smithi190 ceph-mon[79744]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:19:26.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:26 smithi190 ceph-mon[79744]: pgmap v101: 65 pgs: 5 peering, 60 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 2.3 KiB/s rd, 2 op/s 2024-05-06T12:19:26.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:26 smithi092 ceph-mon[91933]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:19:26.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:26 smithi092 ceph-mon[91933]: pgmap v101: 65 pgs: 5 peering, 60 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 2.3 KiB/s rd, 2 op/s 2024-05-06T12:19:28.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:28 smithi190 ceph-mon[79744]: pgmap v102: 65 pgs: 65 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 2.2 KiB/s rd, 2 op/s 2024-05-06T12:19:28.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:28 smithi092 ceph-mon[91933]: pgmap v102: 65 pgs: 65 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 2.2 KiB/s rd, 2 op/s 2024-05-06T12:19:30.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:30 smithi190 ceph-mon[79744]: pgmap v103: 65 pgs: 65 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 2 op/s 2024-05-06T12:19:30.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:30 smithi092 ceph-mon[91933]: pgmap v103: 65 pgs: 65 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 2 op/s 2024-05-06T12:19:32.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:32 smithi190 ceph-mon[79744]: pgmap v104: 65 pgs: 65 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 2 op/s 2024-05-06T12:19:32.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:32 smithi092 ceph-mon[91933]: pgmap v104: 65 pgs: 65 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 2 op/s 2024-05-06T12:19:34.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:34 smithi190 ceph-mon[79744]: pgmap v105: 65 pgs: 65 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-05-06T12:19:34.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:34 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:34.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:34 smithi092 ceph-mon[91933]: pgmap v105: 65 pgs: 65 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-05-06T12:19:34.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:34 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:36.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:36 smithi190 ceph-mon[79744]: pgmap v106: 65 pgs: 65 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:19:36.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:36 smithi092 ceph-mon[91933]: pgmap v106: 65 pgs: 65 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:19:37.292 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:37 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-06T12:19:37.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:37 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-06T12:19:38.295 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:38 smithi190 ceph-mon[79744]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-06T12:19:38.295 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:38 smithi190 ceph-mon[79744]: Upgrade: osd.3 is safe to restart 2024-05-06T12:19:38.295 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:38 smithi190 ceph-mon[79744]: pgmap v107: 65 pgs: 65 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:19:38.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:38 smithi092 ceph-mon[91933]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-06T12:19:38.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:38 smithi092 ceph-mon[91933]: Upgrade: osd.3 is safe to restart 2024-05-06T12:19:38.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:38 smithi092 ceph-mon[91933]: pgmap v107: 65 pgs: 65 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:19:39.565 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:39 smithi190 ceph-mon[79744]: Upgrade: Updating osd.3 2024-05-06T12:19:39.565 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:39 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:39.565 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:39 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-06T12:19:39.565 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:39 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:19:39.565 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:39 smithi190 ceph-mon[79744]: Deploying daemon osd.3 on smithi190 2024-05-06T12:19:39.565 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:39 smithi190 ceph-mon[79744]: pgmap v108: 65 pgs: 65 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:19:39.565 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:39 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:19:39.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:39 smithi092 ceph-mon[91933]: Upgrade: Updating osd.3 2024-05-06T12:19:39.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:39 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:39.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:39 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-06T12:19:39.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:39 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:19:39.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:39 smithi092 ceph-mon[91933]: Deploying daemon osd.3 on smithi190 2024-05-06T12:19:39.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:39 smithi092 ceph-mon[91933]: pgmap v108: 65 pgs: 65 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:19:39.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:39 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:19:41.781 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:41 smithi190 systemd[1]: Stopping Ceph osd.3 for aee1d912-0ba0-11ef-bc97-c7b262605968... 2024-05-06T12:19:42.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:41 smithi190 ceph-mon[79744]: pgmap v109: 65 pgs: 65 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:19:42.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:41 smithi190 ceph-mon[79744]: osd.3 marked itself down and dead 2024-05-06T12:19:42.099 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:41 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3[39370]: 2024-05-06T12:19:41.780+0000 7fd63bb28700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-06T12:19:42.099 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:41 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3[39370]: 2024-05-06T12:19:41.780+0000 7fd63bb28700 -1 osd.3 65 *** Got signal Terminated *** 2024-05-06T12:19:42.099 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:41 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3[39370]: 2024-05-06T12:19:41.780+0000 7fd63bb28700 -1 osd.3 65 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-06T12:19:42.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:41 smithi092 ceph-mon[91933]: pgmap v109: 65 pgs: 65 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:19:42.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:41 smithi092 ceph-mon[91933]: osd.3 marked itself down and dead 2024-05-06T12:19:42.493 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:42 smithi190 podman[88796]: 2024-05-06 12:19:42.23100025 +0000 UTC m=+0.555815257 container died 52f9521286d58b2fcaa5ce052f976e986514caeeeb69fae17e402ad0e6a9dc41 (image=quay.ceph.io/ceph-ci/ceph@sha256:1a0b7fd3cefd2c5baddf682493f92b4e27596f3c0ff27b02fdccd2891f56859f, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, RELEASE=quincy-e81df42, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-06T12:19:42.777 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:42 smithi190 podman[88796]: 2024-05-06 12:19:42.564863568 +0000 UTC m=+0.889678575 container cleanup 52f9521286d58b2fcaa5ce052f976e986514caeeeb69fae17e402ad0e6a9dc41 (image=quay.ceph.io/ceph-ci/ceph@sha256:1a0b7fd3cefd2c5baddf682493f92b4e27596f3c0ff27b02fdccd2891f56859f, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-e81df42, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-06T12:19:42.777 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:42 smithi190 bash[88796]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3 2024-05-06T12:19:43.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:42 smithi190 ceph-mon[79744]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-06T12:19:43.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:42 smithi190 ceph-mon[79744]: osdmap e66: 6 total, 5 up, 6 in 2024-05-06T12:19:43.098 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:42 smithi190 podman[88825]: 2024-05-06 12:19:42.775046635 +0000 UTC m=+0.541130182 container remove 52f9521286d58b2fcaa5ce052f976e986514caeeeb69fae17e402ad0e6a9dc41 (image=quay.ceph.io/ceph-ci/ceph@sha256:1a0b7fd3cefd2c5baddf682493f92b4e27596f3c0ff27b02fdccd2891f56859f, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=quincy-e81df42, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2) 2024-05-06T12:19:43.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:42 smithi092 ceph-mon[91933]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-06T12:19:43.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:42 smithi092 ceph-mon[91933]: osdmap e66: 6 total, 5 up, 6 in 2024-05-06T12:19:43.496 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:43 smithi190 podman[88918]: 2024-05-06 12:19:43.092863346 +0000 UTC m=+0.017671209 image pull e4ce8ad5d0c4b9ea60bc5f6b8b0fdb5efbe59bf7771fb0920fc2200e6ae546b0 quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb 2024-05-06T12:19:43.497 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:43 smithi190 podman[88918]: 2024-05-06 12:19:43.192318067 +0000 UTC m=+0.117125904 container create 7f35685dcf8a7cb15ad80c2384f62053f965c99fce2529604f12cf802dbfffb6 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, RELEASE=squid-70ce953, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux ) 2024-05-06T12:19:43.497 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:43 smithi190 podman[88918]: 2024-05-06 12:19:43.411967364 +0000 UTC m=+0.336775211 container init 7f35685dcf8a7cb15ad80c2384f62053f965c99fce2529604f12cf802dbfffb6 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3-deactivate, RELEASE=squid-70ce953, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-06T12:19:43.497 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:43 smithi190 podman[88918]: 2024-05-06 12:19:43.415419892 +0000 UTC m=+0.340227729 container start 7f35685dcf8a7cb15ad80c2384f62053f965c99fce2529604f12cf802dbfffb6 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-70ce953, maintainer=Guillaume Abrioux ) 2024-05-06T12:19:43.497 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:43 smithi190 podman[88918]: 2024-05-06 12:19:43.49469896 +0000 UTC m=+0.419506812 container attach 7f35685dcf8a7cb15ad80c2384f62053f965c99fce2529604f12cf802dbfffb6 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-70ce953, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-06T12:19:43.848 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:43 smithi190 podman[88918]: 2024-05-06 12:19:43.706131884 +0000 UTC m=+0.630939731 container died 7f35685dcf8a7cb15ad80c2384f62053f965c99fce2529604f12cf802dbfffb6 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-70ce953, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-05-06T12:19:44.197 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:43 smithi190 ceph-mon[79744]: pgmap v111: 65 pgs: 13 peering, 13 stale+active+clean, 39 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-06T12:19:44.197 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:43 smithi190 ceph-mon[79744]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-05-06T12:19:44.197 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:43 smithi190 ceph-mon[79744]: osdmap e67: 6 total, 5 up, 6 in 2024-05-06T12:19:44.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:43 smithi092 ceph-mon[91933]: pgmap v111: 65 pgs: 13 peering, 13 stale+active+clean, 39 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-06T12:19:44.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:43 smithi092 ceph-mon[91933]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-05-06T12:19:44.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:43 smithi092 ceph-mon[91933]: osdmap e67: 6 total, 5 up, 6 in 2024-05-06T12:19:44.598 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:44 smithi190 podman[88943]: 2024-05-06 12:19:44.365242537 +0000 UTC m=+0.651271388 container remove 7f35685dcf8a7cb15ad80c2384f62053f965c99fce2529604f12cf802dbfffb6 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-70ce953, org.label-schema.build-date=20240430, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-06T12:19:44.598 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:44 smithi190 systemd[1]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@osd.3.service: Deactivated successfully. 2024-05-06T12:19:44.598 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:44 smithi190 systemd[1]: Stopped Ceph osd.3 for aee1d912-0ba0-11ef-bc97-c7b262605968. 2024-05-06T12:19:44.598 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:44 smithi190 systemd[1]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@osd.3.service: Consumed 59.455s CPU time. 2024-05-06T12:19:44.848 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:44 smithi190 systemd[1]: Starting Ceph osd.3 for aee1d912-0ba0-11ef-bc97-c7b262605968... 2024-05-06T12:19:44.848 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:44 smithi190 podman[89043]: 2024-05-06 12:19:44.71591331 +0000 UTC m=+0.017922606 image pull e4ce8ad5d0c4b9ea60bc5f6b8b0fdb5efbe59bf7771fb0920fc2200e6ae546b0 quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb 2024-05-06T12:19:44.849 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:44 smithi190 podman[89043]: 2024-05-06 12:19:44.816705761 +0000 UTC m=+0.118715043 container create 47722ddd0acedda8156cb4141a1b796636a66115ede96837053545606417350c (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3-activate, ceph=True, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-70ce953, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-06T12:19:45.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:44 smithi092 ceph-mon[91933]: Health check failed: Degraded data redundancy: 43/372 objects degraded (11.559%), 9 pgs degraded (PG_DEGRADED) 2024-05-06T12:19:45.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:44 smithi190 ceph-mon[79744]: Health check failed: Degraded data redundancy: 43/372 objects degraded (11.559%), 9 pgs degraded (PG_DEGRADED) 2024-05-06T12:19:45.349 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:45 smithi190 podman[89043]: 2024-05-06 12:19:45.01932638 +0000 UTC m=+0.321335662 container init 47722ddd0acedda8156cb4141a1b796636a66115ede96837053545606417350c (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3-activate, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-70ce953, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430) 2024-05-06T12:19:45.349 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:45 smithi190 podman[89043]: 2024-05-06 12:19:45.022452692 +0000 UTC m=+0.324461976 container start 47722ddd0acedda8156cb4141a1b796636a66115ede96837053545606417350c (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3-activate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-70ce953, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-06T12:19:45.349 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:45 smithi190 podman[89043]: 2024-05-06 12:19:45.119471368 +0000 UTC m=+0.421480652 container attach 47722ddd0acedda8156cb4141a1b796636a66115ede96837053545606417350c (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3-activate, RELEASE=squid-70ce953, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_CLEAN=True) 2024-05-06T12:19:45.849 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:45 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3-activate[89058]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-06T12:19:45.849 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:45 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3-activate[89058]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-05-06T12:19:45.849 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:45 smithi190 bash[89043]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-06T12:19:45.849 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:45 smithi190 bash[89043]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-05-06T12:19:45.849 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:45 smithi190 bash[89043]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-05-06T12:19:45.849 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:45 smithi190 bash[89043]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-05-06T12:19:45.849 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:45 smithi190 bash[89043]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-05-06T12:19:45.849 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:45 smithi190 bash[89043]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-06T12:19:45.849 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:45 smithi190 bash[89043]: --> ceph-volume raw activate successful for osd ID: 3 2024-05-06T12:19:45.849 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:45 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3-activate[89058]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-05-06T12:19:45.849 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:45 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3-activate[89058]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-05-06T12:19:45.849 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:45 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3-activate[89058]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-05-06T12:19:45.850 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:45 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3-activate[89058]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-06T12:19:45.850 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:45 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3-activate[89058]: --> ceph-volume raw activate successful for osd ID: 3 2024-05-06T12:19:45.850 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:45 smithi190 podman[89043]: 2024-05-06 12:19:45.490106369 +0000 UTC m=+0.792115658 container died 47722ddd0acedda8156cb4141a1b796636a66115ede96837053545606417350c (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3-activate, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-70ce953, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-06T12:19:46.116 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:45 smithi190 ceph-mon[79744]: pgmap v113: 65 pgs: 9 active+undersized+degraded, 13 peering, 3 stale+active+clean, 9 active+undersized, 31 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 43/372 objects degraded (11.559%) 2024-05-06T12:19:46.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:45 smithi092 ceph-mon[91933]: pgmap v113: 65 pgs: 9 active+undersized+degraded, 13 peering, 3 stale+active+clean, 9 active+undersized, 31 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 43/372 objects degraded (11.559%) 2024-05-06T12:19:46.448 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:46 smithi190 podman[89193]: 2024-05-06 12:19:46.11479973 +0000 UTC m=+0.619418189 container remove 47722ddd0acedda8156cb4141a1b796636a66115ede96837053545606417350c (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3-activate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-70ce953, GIT_BRANCH=HEAD) 2024-05-06T12:19:46.448 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:46 smithi190 podman[89255]: 2024-05-06 12:19:46.232479877 +0000 UTC m=+0.018029707 image pull e4ce8ad5d0c4b9ea60bc5f6b8b0fdb5efbe59bf7771fb0920fc2200e6ae546b0 quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb 2024-05-06T12:19:46.449 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:46 smithi190 podman[89255]: 2024-05-06 12:19:46.340874947 +0000 UTC m=+0.126424824 container create 914ada199f9336cd9b65640fead36c31aaed835630ba514007feaa0bcffe83e3 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-70ce953, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-06T12:19:46.848 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:46 smithi190 podman[89255]: 2024-05-06 12:19:46.543101073 +0000 UTC m=+0.328650907 container init 914ada199f9336cd9b65640fead36c31aaed835630ba514007feaa0bcffe83e3 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-70ce953, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430) 2024-05-06T12:19:46.848 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:46 smithi190 podman[89255]: 2024-05-06 12:19:46.546327567 +0000 UTC m=+0.331877405 container start 914ada199f9336cd9b65640fead36c31aaed835630ba514007feaa0bcffe83e3 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-70ce953, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-06T12:19:46.848 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:46 smithi190 ceph-osd[89275]: -- 172.21.15.190:0/3268590818 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ae541b1680 con 0x55ae5417ac00 2024-05-06T12:19:46.848 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:46 smithi190 bash[89255]: 914ada199f9336cd9b65640fead36c31aaed835630ba514007feaa0bcffe83e3 2024-05-06T12:19:46.849 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:46 smithi190 systemd[1]: Started Ceph osd.3 for aee1d912-0ba0-11ef-bc97-c7b262605968. 2024-05-06T12:19:47.348 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:47 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3[89271]: 2024-05-06T12:19:47.087+0000 7f87566a87c0 -1 Falling back to public interface 2024-05-06T12:19:47.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:47 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:47.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:47 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:47.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:47 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:19:47.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:47 smithi190 ceph-mon[79744]: pgmap v114: 65 pgs: 12 active+undersized+degraded, 13 peering, 13 active+undersized, 27 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 52/372 objects degraded (13.978%) 2024-05-06T12:19:47.974 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:47 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:47.974 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:47 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:47.974 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:47 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:19:47.974 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:47 smithi092 ceph-mon[91933]: pgmap v114: 65 pgs: 12 active+undersized+degraded, 13 peering, 13 active+undersized, 27 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 52/372 objects degraded (13.978%) 2024-05-06T12:19:49.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:48 smithi092 ceph-mon[91933]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-05-06T12:19:49.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:48 smithi190 ceph-mon[79744]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-05-06T12:19:50.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:49 smithi092 ceph-mon[91933]: pgmap v115: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 76/372 objects degraded (20.430%) 2024-05-06T12:19:50.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:50.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:50.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:49 smithi190 ceph-mon[79744]: pgmap v115: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 76/372 objects degraded (20.430%) 2024-05-06T12:19:50.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:50.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:51.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:50 smithi092 ceph-mon[91933]: Health check update: Degraded data redundancy: 76/372 objects degraded (20.430%), 20 pgs degraded (PG_DEGRADED) 2024-05-06T12:19:51.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:50 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:51.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:50 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:51.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:50 smithi190 ceph-mon[79744]: Health check update: Degraded data redundancy: 76/372 objects degraded (20.430%), 20 pgs degraded (PG_DEGRADED) 2024-05-06T12:19:51.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:50 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:51.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:50 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:52.209 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:51 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3[89271]: 2024-05-06T12:19:51.918+0000 7f87566a87c0 -1 osd.3 65 log_to_monitors true 2024-05-06T12:19:52.210 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:51 smithi190 ceph-mon[79744]: pgmap v116: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 76/372 objects degraded (20.430%) 2024-05-06T12:19:52.210 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:51 smithi190 ceph-mon[79744]: from='osd.3 [v2:172.21.15.190:6800/3441479977,v1:172.21.15.190:6801/3441479977]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-06T12:19:52.210 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:51 smithi190 ceph-mon[79744]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-06T12:19:52.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:51 smithi092 ceph-mon[91933]: pgmap v116: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 323 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 76/372 objects degraded (20.430%) 2024-05-06T12:19:52.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:51 smithi092 ceph-mon[91933]: from='osd.3 [v2:172.21.15.190:6800/3441479977,v1:172.21.15.190:6801/3441479977]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-06T12:19:52.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:51 smithi092 ceph-mon[91933]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-06T12:19:52.848 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:19:52 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3[89271]: 2024-05-06T12:19:52.773+0000 7f8747aff700 -1 osd.3 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-06T12:19:53.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:53 smithi190 ceph-mon[79744]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-06T12:19:53.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:53 smithi190 ceph-mon[79744]: from='osd.3 [v2:172.21.15.190:6800/3441479977,v1:172.21.15.190:6801/3441479977]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-05-06T12:19:53.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:53 smithi190 ceph-mon[79744]: osdmap e68: 6 total, 5 up, 6 in 2024-05-06T12:19:53.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:53 smithi190 ceph-mon[79744]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-05-06T12:19:53.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:53 smithi190 ceph-mon[79744]: pgmap v118: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 76/372 objects degraded (20.430%) 2024-05-06T12:19:54.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:53 smithi092 ceph-mon[91933]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-06T12:19:54.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:53 smithi092 ceph-mon[91933]: from='osd.3 [v2:172.21.15.190:6800/3441479977,v1:172.21.15.190:6801/3441479977]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-05-06T12:19:54.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:53 smithi092 ceph-mon[91933]: osdmap e68: 6 total, 5 up, 6 in 2024-05-06T12:19:54.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:53 smithi092 ceph-mon[91933]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-05-06T12:19:54.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:53 smithi092 ceph-mon[91933]: pgmap v118: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 76/372 objects degraded (20.430%) 2024-05-06T12:19:54.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.756+0000 7f10d73d0700 1 -- 172.21.15.92:0/3447926975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10d00fdff0 msgr2=0x7f10d00fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:54.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.756+0000 7f10d73d0700 1 --2- 172.21.15.92:0/3447926975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10d00fdff0 0x7f10d00fe460 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f10c0004940 tx=0x7f10c0009e30 comp rx=0 tx=0).stop 2024-05-06T12:19:54.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.756+0000 7f10d73d0700 1 -- 172.21.15.92:0/3447926975 shutdown_connections 2024-05-06T12:19:54.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.756+0000 7f10d73d0700 1 --2- 172.21.15.92:0/3447926975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10d00fdff0 0x7f10d00fe460 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:54.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.756+0000 7f10d73d0700 1 --2- 172.21.15.92:0/3447926975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10d00fce80 0x7f10d00fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:54.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.756+0000 7f10d73d0700 1 -- 172.21.15.92:0/3447926975 >> 172.21.15.92:0/3447926975 conn(0x7f10d00f8580 msgr2=0x7f10d00fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:19:54.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.756+0000 7f10d73d0700 1 -- 172.21.15.92:0/3447926975 shutdown_connections 2024-05-06T12:19:54.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.756+0000 7f10d73d0700 1 -- 172.21.15.92:0/3447926975 wait complete. 2024-05-06T12:19:54.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.756+0000 7f10d73d0700 1 Processor -- start 2024-05-06T12:19:54.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.757+0000 7f10d73d0700 1 -- start start 2024-05-06T12:19:54.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.757+0000 7f10d73d0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10d00fce80 0x7f10d0071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:54.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.757+0000 7f10d73d0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10d00fdff0 0x7f10d00702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:54.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.757+0000 7f10d73d0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d0072200 con 0x7f10d00fce80 2024-05-06T12:19:54.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.757+0000 7f10d73d0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d00707f0 con 0x7f10d00fdff0 2024-05-06T12:19:54.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.757+0000 7f10d516c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10d00fce80 0x7f10d0071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:54.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.757+0000 7f10d516c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10d00fce80 0x7f10d0071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53370/0 (socket says 172.21.15.92:53370) 2024-05-06T12:19:54.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.757+0000 7f10d516c700 1 -- 172.21.15.92:0/679011450 learned_addr learned my addr 172.21.15.92:0/679011450 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:19:54.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.757+0000 7f10d516c700 1 -- 172.21.15.92:0/679011450 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10d00fdff0 msgr2=0x7f10d00702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:54.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.757+0000 7f10d516c700 1 --2- 172.21.15.92:0/679011450 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10d00fdff0 0x7f10d00702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:54.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.757+0000 7f10d516c700 1 -- 172.21.15.92:0/679011450 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c0005040 con 0x7f10d00fce80 2024-05-06T12:19:54.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.757+0000 7f10d516c700 1 --2- 172.21.15.92:0/679011450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10d00fce80 0x7f10d0071c00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f10cc002a50 tx=0x7f10cc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:19:54.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.757+0000 7f10c5ffb700 1 -- 172.21.15.92:0/679011450 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10cc007cc0 con 0x7f10d00fce80 2024-05-06T12:19:54.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.757+0000 7f10c5ffb700 1 -- 172.21.15.92:0/679011450 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f10cc00f040 con 0x7f10d00fce80 2024-05-06T12:19:54.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.757+0000 7f10d73d0700 1 -- 172.21.15.92:0/679011450 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10d0070aa0 con 0x7f10d00fce80 2024-05-06T12:19:54.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.758+0000 7f10c5ffb700 1 -- 172.21.15.92:0/679011450 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10cc009640 con 0x7f10d00fce80 2024-05-06T12:19:54.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.758+0000 7f10d73d0700 1 -- 172.21.15.92:0/679011450 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d0071060 con 0x7f10d00fce80 2024-05-06T12:19:54.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.759+0000 7f10d73d0700 1 -- 172.21.15.92:0/679011450 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10d004c710 con 0x7f10d00fce80 2024-05-06T12:19:54.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.759+0000 7f10c5ffb700 1 -- 172.21.15.92:0/679011450 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f10cc0097a0 con 0x7f10d00fce80 2024-05-06T12:19:54.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.759+0000 7f10c5ffb700 1 --2- 172.21.15.92:0/679011450 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f10bc06e140 0x7f10bc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:54.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.760+0000 7f10c5ffb700 1 -- 172.21.15.92:0/679011450 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f10cc08d940 con 0x7f10d00fce80 2024-05-06T12:19:54.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.760+0000 7f10d496b700 1 --2- 172.21.15.92:0/679011450 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f10bc06e140 0x7f10bc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:54.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.760+0000 7f10d496b700 1 --2- 172.21.15.92:0/679011450 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f10bc06e140 0x7f10bc0705f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f10c0004f90 tx=0x7f10c0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:19:54.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.764+0000 7f10c5ffb700 1 -- 172.21.15.92:0/679011450 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f10cc05a7f0 con 0x7f10d00fce80 2024-05-06T12:19:54.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.994+0000 7f10d73d0700 1 -- 172.21.15.92:0/679011450 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f10d0102230 con 0x7f10bc06e140 2024-05-06T12:19:54.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.995+0000 7f10c5ffb700 1 -- 172.21.15.92:0/679011450 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f10d0102230 con 0x7f10bc06e140 2024-05-06T12:19:55.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.997+0000 7f10d73d0700 1 -- 172.21.15.92:0/679011450 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f10bc06e140 msgr2=0x7f10bc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:55.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.997+0000 7f10d73d0700 1 --2- 172.21.15.92:0/679011450 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f10bc06e140 0x7f10bc0705f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f10c0004f90 tx=0x7f10c0009e30 comp rx=0 tx=0).stop 2024-05-06T12:19:55.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.997+0000 7f10d73d0700 1 -- 172.21.15.92:0/679011450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10d00fce80 msgr2=0x7f10d0071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:55.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.997+0000 7f10d73d0700 1 --2- 172.21.15.92:0/679011450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10d00fce80 0x7f10d0071c00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f10cc002a50 tx=0x7f10cc0046b0 comp rx=0 tx=0).stop 2024-05-06T12:19:55.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.997+0000 7f10d73d0700 1 -- 172.21.15.92:0/679011450 shutdown_connections 2024-05-06T12:19:55.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.997+0000 7f10d73d0700 1 --2- 172.21.15.92:0/679011450 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f10bc06e140 0x7f10bc0705f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:55.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.997+0000 7f10d73d0700 1 --2- 172.21.15.92:0/679011450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10d00fce80 0x7f10d0071c00 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:55.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.997+0000 7f10d73d0700 1 --2- 172.21.15.92:0/679011450 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10d00fdff0 0x7f10d00702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:55.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.997+0000 7f10d73d0700 1 -- 172.21.15.92:0/679011450 >> 172.21.15.92:0/679011450 conn(0x7f10d00f8580 msgr2=0x7f10d00f9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:19:55.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.997+0000 7f10d73d0700 1 -- 172.21.15.92:0/679011450 shutdown_connections 2024-05-06T12:19:55.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:54.997+0000 7f10d73d0700 1 -- 172.21.15.92:0/679011450 wait complete. 2024-05-06T12:19:55.011 INFO:teuthology.orchestra.run.smithi092.stdout:true 2024-05-06T12:19:55.011 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:54 smithi092 ceph-mon[91933]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-06T12:19:55.011 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:54 smithi092 ceph-mon[91933]: osd.3 [v2:172.21.15.190:6800/3441479977,v1:172.21.15.190:6801/3441479977] boot 2024-05-06T12:19:55.011 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:54 smithi092 ceph-mon[91933]: osdmap e69: 6 total, 6 up, 6 in 2024-05-06T12:19:55.011 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:54 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-06T12:19:55.011 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:54 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:55.012 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:54 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:19:55.012 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:54 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:55.012 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:54 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:55.012 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:54 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:19:55.012 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:54 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:19:55.012 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:54 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:55.012 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:54 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:19:55.012 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:54 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:19:55.012 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:54 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:19:55.012 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:54 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:19:55.012 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:54 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-06T12:19:55.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:54 smithi190 ceph-mon[79744]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-06T12:19:55.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:54 smithi190 ceph-mon[79744]: osd.3 [v2:172.21.15.190:6800/3441479977,v1:172.21.15.190:6801/3441479977] boot 2024-05-06T12:19:55.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:54 smithi190 ceph-mon[79744]: osdmap e69: 6 total, 6 up, 6 in 2024-05-06T12:19:55.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:54 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-06T12:19:55.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:54 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:55.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:54 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:19:55.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:54 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:55.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:54 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:55.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:54 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:19:55.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:54 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:19:55.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:54 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:19:55.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:54 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:19:55.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:54 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:19:55.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:54 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:19:55.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:54 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:19:55.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:54 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-06T12:19:55.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.100+0000 7f861497e700 1 -- 172.21.15.92:0/626152667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86100fe090 msgr2=0x7f86100fe500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:55.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.100+0000 7f861497e700 1 --2- 172.21.15.92:0/626152667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86100fe090 0x7f86100fe500 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f85f8004940 tx=0x7f85f8009e30 comp rx=0 tx=0).stop 2024-05-06T12:19:55.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.100+0000 7f861497e700 1 -- 172.21.15.92:0/626152667 shutdown_connections 2024-05-06T12:19:55.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.100+0000 7f861497e700 1 --2- 172.21.15.92:0/626152667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86100fe090 0x7f86100fe500 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:55.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.100+0000 7f861497e700 1 --2- 172.21.15.92:0/626152667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86100fce90 0x7f86100fd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:55.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.100+0000 7f861497e700 1 -- 172.21.15.92:0/626152667 >> 172.21.15.92:0/626152667 conn(0x7f86100f85c0 msgr2=0x7f86100fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:19:55.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.100+0000 7f861497e700 1 -- 172.21.15.92:0/626152667 shutdown_connections 2024-05-06T12:19:55.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.100+0000 7f861497e700 1 -- 172.21.15.92:0/626152667 wait complete. 2024-05-06T12:19:55.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.101+0000 7f861497e700 1 Processor -- start 2024-05-06T12:19:55.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.101+0000 7f861497e700 1 -- start start 2024-05-06T12:19:55.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.101+0000 7f861497e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86100fce90 0x7f861010da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:55.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.101+0000 7f861497e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86100fe090 0x7f861010df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:55.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.101+0000 7f861497e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861010e560 con 0x7f86100fe090 2024-05-06T12:19:55.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.101+0000 7f861497e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861010edb0 con 0x7f86100fce90 2024-05-06T12:19:55.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.101+0000 7f860dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86100fe090 0x7f861010df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:55.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.101+0000 7f860dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86100fe090 0x7f861010df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53386/0 (socket says 172.21.15.92:53386) 2024-05-06T12:19:55.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.101+0000 7f860dd9b700 1 -- 172.21.15.92:0/4109827887 learned_addr learned my addr 172.21.15.92:0/4109827887 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:19:55.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.101+0000 7f860dd9b700 1 -- 172.21.15.92:0/4109827887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86100fce90 msgr2=0x7f861010da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:55.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.101+0000 7f860e59c700 1 --2- 172.21.15.92:0/4109827887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86100fce90 0x7f861010da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:55.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.101+0000 7f860dd9b700 1 --2- 172.21.15.92:0/4109827887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86100fce90 0x7f861010da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:55.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.101+0000 7f860dd9b700 1 -- 172.21.15.92:0/4109827887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f8005040 con 0x7f86100fe090 2024-05-06T12:19:55.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.101+0000 7f860e59c700 1 --2- 172.21.15.92:0/4109827887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86100fce90 0x7f861010da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-06T12:19:55.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.101+0000 7f860dd9b700 1 --2- 172.21.15.92:0/4109827887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86100fe090 0x7f861010df60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f85f800b3b0 tx=0x7f85f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:19:55.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.101+0000 7f8606ffd700 1 -- 172.21.15.92:0/4109827887 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85f800e040 con 0x7f86100fe090 2024-05-06T12:19:55.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.102+0000 7f8606ffd700 1 -- 172.21.15.92:0/4109827887 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f85f800bcc0 con 0x7f86100fe090 2024-05-06T12:19:55.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.102+0000 7f861497e700 1 -- 172.21.15.92:0/4109827887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f861010f030 con 0x7f86100fe090 2024-05-06T12:19:55.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.102+0000 7f8606ffd700 1 -- 172.21.15.92:0/4109827887 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85f801c630 con 0x7f86100fe090 2024-05-06T12:19:55.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.102+0000 7f861497e700 1 -- 172.21.15.92:0/4109827887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f861010f570 con 0x7f86100fe090 2024-05-06T12:19:55.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.102+0000 7f861497e700 1 -- 172.21.15.92:0/4109827887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f861005f5c0 con 0x7f86100fe090 2024-05-06T12:19:55.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.103+0000 7f8606ffd700 1 -- 172.21.15.92:0/4109827887 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f85f8013980 con 0x7f86100fe090 2024-05-06T12:19:55.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.103+0000 7f8606ffd700 1 --2- 172.21.15.92:0/4109827887 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f85fc06e140 0x7f85fc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:55.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.104+0000 7f860e59c700 1 --2- 172.21.15.92:0/4109827887 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f85fc06e140 0x7f85fc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:55.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.104+0000 7f8606ffd700 1 -- 172.21.15.92:0/4109827887 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f85f808e830 con 0x7f86100fe090 2024-05-06T12:19:55.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.104+0000 7f860e59c700 1 --2- 172.21.15.92:0/4109827887 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f85fc06e140 0x7f85fc0705f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f86100fdc00 tx=0x7f8608009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:19:55.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.108+0000 7f8606ffd700 1 -- 172.21.15.92:0/4109827887 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f85f805b6e0 con 0x7f86100fe090 2024-05-06T12:19:55.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.322+0000 7f861497e700 1 -- 172.21.15.92:0/4109827887 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f861010f8a0 con 0x7f85fc06e140 2024-05-06T12:19:55.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.324+0000 7f8606ffd700 1 -- 172.21.15.92:0/4109827887 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f861010f8a0 con 0x7f85fc06e140 2024-05-06T12:19:55.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.326+0000 7f861497e700 1 -- 172.21.15.92:0/4109827887 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f85fc06e140 msgr2=0x7f85fc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:55.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.326+0000 7f861497e700 1 --2- 172.21.15.92:0/4109827887 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f85fc06e140 0x7f85fc0705f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f86100fdc00 tx=0x7f8608009040 comp rx=0 tx=0).stop 2024-05-06T12:19:55.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.326+0000 7f861497e700 1 -- 172.21.15.92:0/4109827887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86100fe090 msgr2=0x7f861010df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:55.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.326+0000 7f861497e700 1 --2- 172.21.15.92:0/4109827887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86100fe090 0x7f861010df60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f85f800b3b0 tx=0x7f85f8009e30 comp rx=0 tx=0).stop 2024-05-06T12:19:55.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.326+0000 7f861497e700 1 -- 172.21.15.92:0/4109827887 shutdown_connections 2024-05-06T12:19:55.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.326+0000 7f861497e700 1 --2- 172.21.15.92:0/4109827887 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f85fc06e140 0x7f85fc0705f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:55.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.326+0000 7f861497e700 1 --2- 172.21.15.92:0/4109827887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86100fce90 0x7f861010da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:55.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.326+0000 7f861497e700 1 --2- 172.21.15.92:0/4109827887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86100fe090 0x7f861010df60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:55.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.326+0000 7f861497e700 1 -- 172.21.15.92:0/4109827887 >> 172.21.15.92:0/4109827887 conn(0x7f86100f85c0 msgr2=0x7f86100f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:19:55.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.326+0000 7f861497e700 1 -- 172.21.15.92:0/4109827887 shutdown_connections 2024-05-06T12:19:55.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.326+0000 7f861497e700 1 -- 172.21.15.92:0/4109827887 wait complete. 2024-05-06T12:19:55.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.427+0000 7efff37be700 1 -- 172.21.15.92:0/3055712336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effec0662c0 msgr2=0x7effec0ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.427+0000 7efff37be700 1 --2- 172.21.15.92:0/3055712336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effec0662c0 0x7effec0ff9a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7effe4009ce0 tx=0x7effe4004d10 comp rx=0 tx=0).stop 2024-05-06T12:19:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.427+0000 7efff37be700 1 -- 172.21.15.92:0/3055712336 shutdown_connections 2024-05-06T12:19:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.427+0000 7efff37be700 1 --2- 172.21.15.92:0/3055712336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effec0ffee0 0x7effec102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.427+0000 7efff37be700 1 --2- 172.21.15.92:0/3055712336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effec0662c0 0x7effec0ff9a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.427+0000 7efff37be700 1 -- 172.21.15.92:0/3055712336 >> 172.21.15.92:0/3055712336 conn(0x7effec0f7430 msgr2=0x7effec0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:19:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.427+0000 7efff37be700 1 -- 172.21.15.92:0/3055712336 shutdown_connections 2024-05-06T12:19:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.427+0000 7efff37be700 1 -- 172.21.15.92:0/3055712336 wait complete. 2024-05-06T12:19:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.427+0000 7efff37be700 1 Processor -- start 2024-05-06T12:19:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.427+0000 7efff37be700 1 -- start start 2024-05-06T12:19:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.427+0000 7efff37be700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effec0662c0 0x7effec10da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.427+0000 7efff37be700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effec0ffee0 0x7effec10df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.428+0000 7efff37be700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effec10e560 con 0x7effec0ffee0 2024-05-06T12:19:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.428+0000 7efff37be700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effec10edb0 con 0x7effec0662c0 2024-05-06T12:19:55.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.428+0000 7efff0d59700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effec0ffee0 0x7effec10df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.428+0000 7efff0d59700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effec0ffee0 0x7effec10df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53412/0 (socket says 172.21.15.92:53412) 2024-05-06T12:19:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.428+0000 7efff0d59700 1 -- 172.21.15.92:0/1433245594 learned_addr learned my addr 172.21.15.92:0/1433245594 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:19:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.428+0000 7efff155a700 1 --2- 172.21.15.92:0/1433245594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effec0662c0 0x7effec10da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.428+0000 7efff0d59700 1 -- 172.21.15.92:0/1433245594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effec0662c0 msgr2=0x7effec10da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.428+0000 7efff0d59700 1 --2- 172.21.15.92:0/1433245594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effec0662c0 0x7effec10da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.428+0000 7efff0d59700 1 -- 172.21.15.92:0/1433245594 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe4009940 con 0x7effec0ffee0 2024-05-06T12:19:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.428+0000 7efff155a700 1 --2- 172.21.15.92:0/1433245594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effec0662c0 0x7effec10da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-06T12:19:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.428+0000 7efff0d59700 1 --2- 172.21.15.92:0/1433245594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effec0ffee0 0x7effec10df60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7effdc00eb40 tx=0x7effdc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:19:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.428+0000 7effe1ffb700 1 -- 172.21.15.92:0/1433245594 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7effdc007cc0 con 0x7effec0ffee0 2024-05-06T12:19:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.428+0000 7effe1ffb700 1 -- 172.21.15.92:0/1433245594 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7effdc00f040 con 0x7effec0ffee0 2024-05-06T12:19:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.428+0000 7efff37be700 1 -- 172.21.15.92:0/1433245594 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effec10f030 con 0x7effec0ffee0 2024-05-06T12:19:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.428+0000 7effe1ffb700 1 -- 172.21.15.92:0/1433245594 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7effdc009620 con 0x7effec0ffee0 2024-05-06T12:19:55.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.428+0000 7efff37be700 1 -- 172.21.15.92:0/1433245594 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effec10f570 con 0x7effec0ffee0 2024-05-06T12:19:55.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.429+0000 7efff37be700 1 -- 172.21.15.92:0/1433245594 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effec05f5c0 con 0x7effec0ffee0 2024-05-06T12:19:55.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.430+0000 7effe1ffb700 1 -- 172.21.15.92:0/1433245594 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7effdc009780 con 0x7effec0ffee0 2024-05-06T12:19:55.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.430+0000 7effe1ffb700 1 --2- 172.21.15.92:0/1433245594 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7effd806e140 0x7effd80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:55.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.430+0000 7effe1ffb700 1 -- 172.21.15.92:0/1433245594 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6410+0+0 (secure 0 0 0) 0x7effdc08d7e0 con 0x7effec0ffee0 2024-05-06T12:19:55.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.430+0000 7efff155a700 1 --2- 172.21.15.92:0/1433245594 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7effd806e140 0x7effd80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:55.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.431+0000 7efff155a700 1 --2- 172.21.15.92:0/1433245594 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7effd806e140 0x7effd80705f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7effe4015040 tx=0x7effe4004d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:19:55.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.435+0000 7effe1ffb700 1 -- 172.21.15.92:0/1433245594 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7effdc05a740 con 0x7effec0ffee0 2024-05-06T12:19:55.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.640+0000 7efff37be700 1 -- 172.21.15.92:0/1433245594 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7effec10f900 con 0x7effd806e140 2024-05-06T12:19:55.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.651+0000 7effe1ffb700 1 -- 172.21.15.92:0/1433245594 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7effec10f900 con 0x7effd806e140 2024-05-06T12:19:55.654 INFO:teuthology.orchestra.run.smithi092.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-06T12:19:55.654 INFO:teuthology.orchestra.run.smithi092.stdout:alertmanager.smithi092 smithi092 *:9093,9094 running (4m) 38s ago 13m 25.2M - 0.25.0 c8568f914cd2 be5637d7d25a 2024-05-06T12:19:55.654 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi092 smithi092 running (2m) 38s ago 13m 6660k - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 8da0494e1bdc 2024-05-06T12:19:55.654 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi190 smithi190 running (2m) 6s ago 12m 6665k - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 641a7cdaa56e 2024-05-06T12:19:55.654 INFO:teuthology.orchestra.run.smithi092.stdout:grafana.smithi092 smithi092 *:3000 running (4m) 38s ago 13m 75.0M - 9.4.7 954c08fa6188 9180cb3e5258 2024-05-06T12:19:55.654 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.ijsqgp smithi092 running (8m) 38s ago 8m 60.7M - 17.2.7-925-ge81df420 96d5ae7fd4b4 b4c2ed6d2c82 2024-05-06T12:19:55.654 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.rosbll smithi092 running (7m) 38s ago 7m 122M - 17.2.7-925-ge81df420 96d5ae7fd4b4 5f5426cd1bf5 2024-05-06T12:19:55.654 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.byjvgf smithi190 running (8m) 6s ago 8m 126M - 17.2.7-925-ge81df420 96d5ae7fd4b4 24581d2e1c5d 2024-05-06T12:19:55.654 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.idzoff smithi190 running (7m) 6s ago 7m 64.0M - 17.2.7-925-ge81df420 96d5ae7fd4b4 20f2641696ef 2024-05-06T12:19:55.655 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi092.gyzeqb smithi092 *:8443,9283,8765 running (4m) 38s ago 14m 504M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 de9d2198327a 2024-05-06T12:19:55.655 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi190.kczxkh smithi190 *:8443,9283,8765 running (3m) 6s ago 12m 450M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 e84ab3758ac7 2024-05-06T12:19:55.655 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi092 smithi092 running (3m) 38s ago 15m 46.0M 2048M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 ac07f7158087 2024-05-06T12:19:55.655 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi190 smithi190 running (3m) 6s ago 12m 39.3M 2048M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 7a3c08eb716f 2024-05-06T12:19:55.655 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi092 smithi092 *:9100 running (5m) 38s ago 13m 14.9M - 1.5.0 0da6a335fe13 c796446b206f 2024-05-06T12:19:55.655 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi190 smithi190 *:9100 running (4m) 6s ago 12m 16.5M - 1.5.0 0da6a335fe13 647e28111a76 2024-05-06T12:19:55.655 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 smithi092 running (116s) 38s ago 11m 82.0M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 8c96c75b25f4 2024-05-06T12:19:55.655 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 smithi092 running (80s) 38s ago 11m 82.0M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 e8007a510f08 2024-05-06T12:19:55.655 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 smithi092 running (44s) 38s ago 10m 14.8M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 2beca7448789 2024-05-06T12:19:55.655 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 smithi190 running (9s) 6s ago 10m 12.6M 2581M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 914ada199f93 2024-05-06T12:19:55.655 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 smithi190 running (10m) 6s ago 10m 206M 2581M 17.2.7-925-ge81df420 96d5ae7fd4b4 3a7da09e4c48 2024-05-06T12:19:55.655 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 smithi190 running (9m) 6s ago 9m 189M 2581M 17.2.7-925-ge81df420 96d5ae7fd4b4 e7452c6b5cec 2024-05-06T12:19:55.656 INFO:teuthology.orchestra.run.smithi092.stdout:prometheus.smithi092 smithi092 *:9095 running (3m) 38s ago 12m 54.4M - 2.43.0 a07b618ecd1d bc5445691527 2024-05-06T12:19:55.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.653+0000 7efff37be700 1 -- 172.21.15.92:0/1433245594 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7effd806e140 msgr2=0x7effd80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:55.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.653+0000 7efff37be700 1 --2- 172.21.15.92:0/1433245594 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7effd806e140 0x7effd80705f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7effe4015040 tx=0x7effe4004d10 comp rx=0 tx=0).stop 2024-05-06T12:19:55.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.653+0000 7efff37be700 1 -- 172.21.15.92:0/1433245594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effec0ffee0 msgr2=0x7effec10df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:55.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.653+0000 7efff37be700 1 --2- 172.21.15.92:0/1433245594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effec0ffee0 0x7effec10df60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7effdc00eb40 tx=0x7effdc00b040 comp rx=0 tx=0).stop 2024-05-06T12:19:55.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.653+0000 7efff37be700 1 -- 172.21.15.92:0/1433245594 shutdown_connections 2024-05-06T12:19:55.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.653+0000 7efff37be700 1 --2- 172.21.15.92:0/1433245594 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7effd806e140 0x7effd80705f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:55.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.653+0000 7efff37be700 1 --2- 172.21.15.92:0/1433245594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effec0662c0 0x7effec10da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:55.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.653+0000 7efff37be700 1 --2- 172.21.15.92:0/1433245594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effec0ffee0 0x7effec10df60 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:55.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.653+0000 7efff37be700 1 -- 172.21.15.92:0/1433245594 >> 172.21.15.92:0/1433245594 conn(0x7effec0f7430 msgr2=0x7effec0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:19:55.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.653+0000 7efff37be700 1 -- 172.21.15.92:0/1433245594 shutdown_connections 2024-05-06T12:19:55.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.653+0000 7efff37be700 1 -- 172.21.15.92:0/1433245594 wait complete. 2024-05-06T12:19:55.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.752+0000 7f6c58832700 1 -- 172.21.15.92:0/660285938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c540fce60 msgr2=0x7f6c540fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:55.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.752+0000 7f6c58832700 1 --2- 172.21.15.92:0/660285938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c540fce60 0x7f6c540fd270 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6c44004940 tx=0x7f6c44009e30 comp rx=0 tx=0).stop 2024-05-06T12:19:55.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.753+0000 7f6c58832700 1 -- 172.21.15.92:0/660285938 shutdown_connections 2024-05-06T12:19:55.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.753+0000 7f6c58832700 1 --2- 172.21.15.92:0/660285938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c540fdfd0 0x7f6c540fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:55.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.753+0000 7f6c58832700 1 --2- 172.21.15.92:0/660285938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c540fce60 0x7f6c540fd270 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:55.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.753+0000 7f6c58832700 1 -- 172.21.15.92:0/660285938 >> 172.21.15.92:0/660285938 conn(0x7f6c540f8540 msgr2=0x7f6c540fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:19:55.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.753+0000 7f6c58832700 1 -- 172.21.15.92:0/660285938 shutdown_connections 2024-05-06T12:19:55.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.753+0000 7f6c58832700 1 -- 172.21.15.92:0/660285938 wait complete. 2024-05-06T12:19:55.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.753+0000 7f6c58832700 1 Processor -- start 2024-05-06T12:19:55.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.753+0000 7f6c58832700 1 -- start start 2024-05-06T12:19:55.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.753+0000 7f6c58832700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c540fce60 0x7f6c5410d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:55.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.753+0000 7f6c58832700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c540fdfd0 0x7f6c5410de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:55.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.753+0000 7f6c58832700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5410e470 con 0x7f6c540fdfd0 2024-05-06T12:19:55.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.753+0000 7f6c58832700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5410e5e0 con 0x7f6c540fce60 2024-05-06T12:19:55.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.753+0000 7f6c51d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c540fdfd0 0x7f6c5410de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:55.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.753+0000 7f6c51d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c540fdfd0 0x7f6c5410de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53440/0 (socket says 172.21.15.92:53440) 2024-05-06T12:19:55.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.753+0000 7f6c51d9b700 1 -- 172.21.15.92:0/3461059001 learned_addr learned my addr 172.21.15.92:0/3461059001 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:19:55.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.754+0000 7f6c5259c700 1 --2- 172.21.15.92:0/3461059001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c540fce60 0x7f6c5410d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:55.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.754+0000 7f6c51d9b700 1 -- 172.21.15.92:0/3461059001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c540fce60 msgr2=0x7f6c5410d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:55.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.754+0000 7f6c51d9b700 1 --2- 172.21.15.92:0/3461059001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c540fce60 0x7f6c5410d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:55.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.754+0000 7f6c51d9b700 1 -- 172.21.15.92:0/3461059001 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c44005040 con 0x7f6c540fdfd0 2024-05-06T12:19:55.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.754+0000 7f6c51d9b700 1 --2- 172.21.15.92:0/3461059001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c540fdfd0 0x7f6c5410de70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6c48002a50 tx=0x7f6c480046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:19:55.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.754+0000 7f6c42ffd700 1 -- 172.21.15.92:0/3461059001 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c48007cc0 con 0x7f6c540fdfd0 2024-05-06T12:19:55.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.754+0000 7f6c42ffd700 1 -- 172.21.15.92:0/3461059001 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c4800f040 con 0x7f6c540fdfd0 2024-05-06T12:19:55.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.754+0000 7f6c58832700 1 -- 172.21.15.92:0/3461059001 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c54113040 con 0x7f6c540fdfd0 2024-05-06T12:19:55.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.754+0000 7f6c42ffd700 1 -- 172.21.15.92:0/3461059001 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c4800db70 con 0x7f6c540fdfd0 2024-05-06T12:19:55.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.754+0000 7f6c58832700 1 -- 172.21.15.92:0/3461059001 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c541135d0 con 0x7f6c540fdfd0 2024-05-06T12:19:55.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.755+0000 7f6c58832700 1 -- 172.21.15.92:0/3461059001 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c5404c710 con 0x7f6c540fdfd0 2024-05-06T12:19:55.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.755+0000 7f6c42ffd700 1 -- 172.21.15.92:0/3461059001 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6c4800dd10 con 0x7f6c540fdfd0 2024-05-06T12:19:55.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.756+0000 7f6c42ffd700 1 --2- 172.21.15.92:0/3461059001 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f6c3c06e140 0x7f6c3c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:55.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.756+0000 7f6c42ffd700 1 -- 172.21.15.92:0/3461059001 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f6c4808d150 con 0x7f6c540fdfd0 2024-05-06T12:19:55.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.756+0000 7f6c5259c700 1 --2- 172.21.15.92:0/3461059001 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f6c3c06e140 0x7f6c3c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:55.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.756+0000 7f6c5259c700 1 --2- 172.21.15.92:0/3461059001 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f6c3c06e140 0x7f6c3c0705f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400b3b0 tx=0x7f6c44009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:19:55.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:55.761+0000 7f6c42ffd700 1 -- 172.21.15.92:0/3461059001 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6c4805a080 con 0x7f6c540fdfd0 2024-05-06T12:19:55.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:55 smithi092 ceph-mon[91933]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-06T12:19:55.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:55 smithi092 ceph-mon[91933]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-05-06T12:19:55.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:55 smithi092 ceph-mon[91933]: osdmap e70: 6 total, 6 up, 6 in 2024-05-06T12:19:55.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:55 smithi092 ceph-mon[91933]: pgmap v121: 65 pgs: 17 active+undersized+degraded, 13 active+undersized, 35 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 65/372 objects degraded (17.473%) 2024-05-06T12:19:55.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:55 smithi092 ceph-mon[91933]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:19:56.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.044+0000 7f6c58832700 1 -- 172.21.15.92:0/3461059001 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6c541058e0 con 0x7f6c540fdfd0 2024-05-06T12:19:56.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.044+0000 7f6c42ffd700 1 -- 172.21.15.92:0/3461059001 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f6c48059c10 con 0x7f6c540fdfd0 2024-05-06T12:19:56.048 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:19:56.048 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": { 2024-05-06T12:19:56.048 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:19:56.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:19:56.048 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": { 2024-05-06T12:19:56.048 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:19:56.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:19:56.048 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": { 2024-05-06T12:19:56.048 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 2, 2024-05-06T12:19:56.048 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 4 2024-05-06T12:19:56.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:19:56.048 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-05-06T12:19:56.049 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 4 2024-05-06T12:19:56.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:19:56.049 INFO:teuthology.orchestra.run.smithi092.stdout: "overall": { 2024-05-06T12:19:56.049 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 6, 2024-05-06T12:19:56.049 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 8 2024-05-06T12:19:56.049 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-06T12:19:56.049 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:19:56.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.047+0000 7f6c58832700 1 -- 172.21.15.92:0/3461059001 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f6c3c06e140 msgr2=0x7f6c3c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:56.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.047+0000 7f6c58832700 1 --2- 172.21.15.92:0/3461059001 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f6c3c06e140 0x7f6c3c0705f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400b3b0 tx=0x7f6c44009e30 comp rx=0 tx=0).stop 2024-05-06T12:19:56.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.047+0000 7f6c58832700 1 -- 172.21.15.92:0/3461059001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c540fdfd0 msgr2=0x7f6c5410de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:56.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.047+0000 7f6c58832700 1 --2- 172.21.15.92:0/3461059001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c540fdfd0 0x7f6c5410de70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6c48002a50 tx=0x7f6c480046b0 comp rx=0 tx=0).stop 2024-05-06T12:19:56.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.047+0000 7f6c58832700 1 -- 172.21.15.92:0/3461059001 shutdown_connections 2024-05-06T12:19:56.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.047+0000 7f6c58832700 1 --2- 172.21.15.92:0/3461059001 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f6c3c06e140 0x7f6c3c0705f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:56.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.047+0000 7f6c58832700 1 --2- 172.21.15.92:0/3461059001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c540fce60 0x7f6c5410d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:56.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.047+0000 7f6c58832700 1 --2- 172.21.15.92:0/3461059001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c540fdfd0 0x7f6c5410de70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:56.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.047+0000 7f6c58832700 1 -- 172.21.15.92:0/3461059001 >> 172.21.15.92:0/3461059001 conn(0x7f6c540f8540 msgr2=0x7f6c540f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:19:56.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.047+0000 7f6c58832700 1 -- 172.21.15.92:0/3461059001 shutdown_connections 2024-05-06T12:19:56.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.047+0000 7f6c58832700 1 -- 172.21.15.92:0/3461059001 wait complete. 2024-05-06T12:19:56.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:55 smithi190 ceph-mon[79744]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-06T12:19:56.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:55 smithi190 ceph-mon[79744]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-05-06T12:19:56.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:55 smithi190 ceph-mon[79744]: osdmap e70: 6 total, 6 up, 6 in 2024-05-06T12:19:56.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:55 smithi190 ceph-mon[79744]: pgmap v121: 65 pgs: 17 active+undersized+degraded, 13 active+undersized, 35 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 65/372 objects degraded (17.473%) 2024-05-06T12:19:56.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:55 smithi190 ceph-mon[79744]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:19:56.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.147+0000 7ff2608c5700 1 -- 172.21.15.92:0/3309384890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff25c0fed60 msgr2=0x7ff25c0faea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:56.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.147+0000 7ff2608c5700 1 --2- 172.21.15.92:0/3309384890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff25c0fed60 0x7ff25c0faea0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff250004940 tx=0x7ff250009e30 comp rx=0 tx=0).stop 2024-05-06T12:19:56.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.147+0000 7ff2608c5700 1 -- 172.21.15.92:0/3309384890 shutdown_connections 2024-05-06T12:19:56.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.147+0000 7ff2608c5700 1 --2- 172.21.15.92:0/3309384890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff25c0fed60 0x7ff25c0faea0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:56.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.147+0000 7ff2608c5700 1 --2- 172.21.15.92:0/3309384890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff25c0fe410 0x7ff25c0fe820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:56.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.147+0000 7ff2608c5700 1 -- 172.21.15.92:0/3309384890 >> 172.21.15.92:0/3309384890 conn(0x7ff25c0f6bd0 msgr2=0x7ff25c0f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:19:56.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.147+0000 7ff2608c5700 1 -- 172.21.15.92:0/3309384890 shutdown_connections 2024-05-06T12:19:56.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.147+0000 7ff2608c5700 1 -- 172.21.15.92:0/3309384890 wait complete. 2024-05-06T12:19:56.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.148+0000 7ff2608c5700 1 Processor -- start 2024-05-06T12:19:56.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.148+0000 7ff2608c5700 1 -- start start 2024-05-06T12:19:56.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.148+0000 7ff2608c5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff25c0fe410 0x7ff25c10d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:56.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.148+0000 7ff2608c5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff25c0fed60 0x7ff25c10de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:56.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.148+0000 7ff2608c5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c10e420 con 0x7ff25c0fed60 2024-05-06T12:19:56.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.148+0000 7ff2608c5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c10e590 con 0x7ff25c0fe410 2024-05-06T12:19:56.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.148+0000 7ff259d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff25c0fed60 0x7ff25c10de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:56.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.148+0000 7ff259d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff25c0fed60 0x7ff25c10de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53450/0 (socket says 172.21.15.92:53450) 2024-05-06T12:19:56.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.148+0000 7ff259d9b700 1 -- 172.21.15.92:0/2165867686 learned_addr learned my addr 172.21.15.92:0/2165867686 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:19:56.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.148+0000 7ff25a59c700 1 --2- 172.21.15.92:0/2165867686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff25c0fe410 0x7ff25c10d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:56.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.149+0000 7ff259d9b700 1 -- 172.21.15.92:0/2165867686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff25c0fe410 msgr2=0x7ff25c10d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:56.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.149+0000 7ff259d9b700 1 --2- 172.21.15.92:0/2165867686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff25c0fe410 0x7ff25c10d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:56.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.149+0000 7ff259d9b700 1 -- 172.21.15.92:0/2165867686 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff250005040 con 0x7ff25c0fed60 2024-05-06T12:19:56.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.149+0000 7ff259d9b700 1 --2- 172.21.15.92:0/2165867686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff25c0fed60 0x7ff25c10de20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff250009150 tx=0x7ff250009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:19:56.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.149+0000 7ff24affd700 1 -- 172.21.15.92:0/2165867686 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff25000e040 con 0x7ff25c0fed60 2024-05-06T12:19:56.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.149+0000 7ff24affd700 1 -- 172.21.15.92:0/2165867686 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff25000bbf0 con 0x7ff25c0fed60 2024-05-06T12:19:56.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.149+0000 7ff2608c5700 1 -- 172.21.15.92:0/2165867686 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff25c112f90 con 0x7ff25c0fed60 2024-05-06T12:19:56.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.149+0000 7ff24affd700 1 -- 172.21.15.92:0/2165867686 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff25001c630 con 0x7ff25c0fed60 2024-05-06T12:19:56.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.149+0000 7ff2608c5700 1 -- 172.21.15.92:0/2165867686 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff25c113520 con 0x7ff25c0fed60 2024-05-06T12:19:56.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.150+0000 7ff2608c5700 1 -- 172.21.15.92:0/2165867686 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff25c04c710 con 0x7ff25c0fed60 2024-05-06T12:19:56.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.150+0000 7ff24affd700 1 -- 172.21.15.92:0/2165867686 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff25000bd60 con 0x7ff25c0fed60 2024-05-06T12:19:56.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.151+0000 7ff24affd700 1 --2- 172.21.15.92:0/2165867686 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff24406e140 0x7ff2440705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:56.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.151+0000 7ff25a59c700 1 --2- 172.21.15.92:0/2165867686 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff24406e140 0x7ff2440705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:56.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.151+0000 7ff24affd700 1 -- 172.21.15.92:0/2165867686 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff25008ed50 con 0x7ff25c0fed60 2024-05-06T12:19:56.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.151+0000 7ff25a59c700 1 --2- 172.21.15.92:0/2165867686 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff24406e140 0x7ff2440705f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff24c004f40 tx=0x7ff24c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:19:56.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.156+0000 7ff24affd700 1 -- 172.21.15.92:0/2165867686 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff25005bd20 con 0x7ff25c0fed60 2024-05-06T12:19:56.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.397+0000 7ff2608c5700 1 -- 172.21.15.92:0/2165867686 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff25c1058c0 con 0x7ff25c0fed60 2024-05-06T12:19:56.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.398+0000 7ff24affd700 1 -- 172.21.15.92:0/2165867686 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1918 (secure 0 0 0) 0x7ff250013e60 con 0x7ff25c0fed60 2024-05-06T12:19:56.401 INFO:teuthology.orchestra.run.smithi092.stdout:e22 2024-05-06T12:19:56.401 INFO:teuthology.orchestra.run.smithi092.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-06T12:19:56.401 INFO:teuthology.orchestra.run.smithi092.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:19:56.401 INFO:teuthology.orchestra.run.smithi092.stdout:legacy client fscid: 1 2024-05-06T12:19:56.401 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:19:56.401 INFO:teuthology.orchestra.run.smithi092.stdout:Filesystem 'cephfs' (1) 2024-05-06T12:19:56.401 INFO:teuthology.orchestra.run.smithi092.stdout:fs_name cephfs 2024-05-06T12:19:56.401 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 22 2024-05-06T12:19:56.402 INFO:teuthology.orchestra.run.smithi092.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-06T12:19:56.402 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-05-06T12:11:44.952671+0000 2024-05-06T12:19:56.402 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2024-05-06T12:19:06.643003+0000 2024-05-06T12:19:56.402 INFO:teuthology.orchestra.run.smithi092.stdout:tableserver 0 2024-05-06T12:19:56.402 INFO:teuthology.orchestra.run.smithi092.stdout:root 0 2024-05-06T12:19:56.402 INFO:teuthology.orchestra.run.smithi092.stdout:session_timeout 60 2024-05-06T12:19:56.402 INFO:teuthology.orchestra.run.smithi092.stdout:session_autoclose 300 2024-05-06T12:19:56.402 INFO:teuthology.orchestra.run.smithi092.stdout:max_file_size 1099511627776 2024-05-06T12:19:56.402 INFO:teuthology.orchestra.run.smithi092.stdout:max_xattr_size 65536 2024-05-06T12:19:56.402 INFO:teuthology.orchestra.run.smithi092.stdout:required_client_features {} 2024-05-06T12:19:56.402 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure 0 2024-05-06T12:19:56.402 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure_osd_epoch 46 2024-05-06T12:19:56.402 INFO:teuthology.orchestra.run.smithi092.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:19:56.402 INFO:teuthology.orchestra.run.smithi092.stdout:max_mds 2 2024-05-06T12:19:56.403 INFO:teuthology.orchestra.run.smithi092.stdout:in 0,1 2024-05-06T12:19:56.403 INFO:teuthology.orchestra.run.smithi092.stdout:up {0=14472,1=24267} 2024-05-06T12:19:56.403 INFO:teuthology.orchestra.run.smithi092.stdout:failed 2024-05-06T12:19:56.403 INFO:teuthology.orchestra.run.smithi092.stdout:damaged 2024-05-06T12:19:56.403 INFO:teuthology.orchestra.run.smithi092.stdout:stopped 2024-05-06T12:19:56.403 INFO:teuthology.orchestra.run.smithi092.stdout:data_pools [3] 2024-05-06T12:19:56.403 INFO:teuthology.orchestra.run.smithi092.stdout:metadata_pool 2 2024-05-06T12:19:56.403 INFO:teuthology.orchestra.run.smithi092.stdout:inline_data enabled 2024-05-06T12:19:56.403 INFO:teuthology.orchestra.run.smithi092.stdout:balancer 2024-05-06T12:19:56.403 INFO:teuthology.orchestra.run.smithi092.stdout:bal_rank_mask -1 2024-05-06T12:19:56.403 INFO:teuthology.orchestra.run.smithi092.stdout:standby_count_wanted 1 2024-05-06T12:19:56.403 INFO:teuthology.orchestra.run.smithi092.stdout:qdb_cluster leader: 14472 members: 24267,14472 2024-05-06T12:19:56.403 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.rosbll{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.92:6828/1663253598,v1:172.21.15.92:6829/1663253598] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:19:56.403 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.ijsqgp{0:14484} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.92:6826/3613441054,v1:172.21.15.92:6827/3613441054] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:19:56.403 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.byjvgf{1:24267} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.190:6824/1565151699,v1:172.21.15.190:6825/1565151699] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:19:56.404 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.idzoff{1:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.190:6826/2453728380,v1:172.21.15.190:6827/2453728380] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:19:56.404 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:19:56.404 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:19:56.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.400+0000 7ff2608c5700 1 -- 172.21.15.92:0/2165867686 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff24406e140 msgr2=0x7ff2440705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:56.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.400+0000 7ff2608c5700 1 --2- 172.21.15.92:0/2165867686 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff24406e140 0x7ff2440705f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff24c004f40 tx=0x7ff24c00b040 comp rx=0 tx=0).stop 2024-05-06T12:19:56.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.400+0000 7ff2608c5700 1 -- 172.21.15.92:0/2165867686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff25c0fed60 msgr2=0x7ff25c10de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:56.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.400+0000 7ff2608c5700 1 --2- 172.21.15.92:0/2165867686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff25c0fed60 0x7ff25c10de20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff250009150 tx=0x7ff250009e30 comp rx=0 tx=0).stop 2024-05-06T12:19:56.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.400+0000 7ff2608c5700 1 -- 172.21.15.92:0/2165867686 shutdown_connections 2024-05-06T12:19:56.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.400+0000 7ff2608c5700 1 --2- 172.21.15.92:0/2165867686 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff24406e140 0x7ff2440705f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:56.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.400+0000 7ff2608c5700 1 --2- 172.21.15.92:0/2165867686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff25c0fe410 0x7ff25c10d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:56.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.400+0000 7ff2608c5700 1 --2- 172.21.15.92:0/2165867686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff25c0fed60 0x7ff25c10de20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:56.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.400+0000 7ff2608c5700 1 -- 172.21.15.92:0/2165867686 >> 172.21.15.92:0/2165867686 conn(0x7ff25c0f6bd0 msgr2=0x7ff25c0f8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:19:56.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.400+0000 7ff2608c5700 1 -- 172.21.15.92:0/2165867686 shutdown_connections 2024-05-06T12:19:56.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.400+0000 7ff2608c5700 1 -- 172.21.15.92:0/2165867686 wait complete. 2024-05-06T12:19:56.405 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 22 2024-05-06T12:19:56.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.501+0000 7f95ffee0700 1 -- 172.21.15.92:0/2828747576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95f8066af0 msgr2=0x7f95f80ffa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:56.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.501+0000 7f95ffee0700 1 --2- 172.21.15.92:0/2828747576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95f8066af0 0x7f95f80ffa90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f95ec004940 tx=0x7f95ec009e30 comp rx=0 tx=0).stop 2024-05-06T12:19:56.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.501+0000 7f95ffee0700 1 -- 172.21.15.92:0/2828747576 shutdown_connections 2024-05-06T12:19:56.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.501+0000 7f95ffee0700 1 --2- 172.21.15.92:0/2828747576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95f80fffd0 0x7f95f81023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:56.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.501+0000 7f95ffee0700 1 --2- 172.21.15.92:0/2828747576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95f8066af0 0x7f95f80ffa90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:56.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.501+0000 7f95ffee0700 1 -- 172.21.15.92:0/2828747576 >> 172.21.15.92:0/2828747576 conn(0x7f95f80f74c0 msgr2=0x7f95f80f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:19:56.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.501+0000 7f95ffee0700 1 -- 172.21.15.92:0/2828747576 shutdown_connections 2024-05-06T12:19:56.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.501+0000 7f95ffee0700 1 -- 172.21.15.92:0/2828747576 wait complete. 2024-05-06T12:19:56.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.501+0000 7f95ffee0700 1 Processor -- start 2024-05-06T12:19:56.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.501+0000 7f95ffee0700 1 -- start start 2024-05-06T12:19:56.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.502+0000 7f95ffee0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95f8066af0 0x7f95f810da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:56.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.502+0000 7f95ffee0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95f80fffd0 0x7f95f810dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:56.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.502+0000 7f95ffee0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f810e5b0 con 0x7f95f80fffd0 2024-05-06T12:19:56.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.502+0000 7f95ffee0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f810ee00 con 0x7f95f8066af0 2024-05-06T12:19:56.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.502+0000 7f95fd47b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95f80fffd0 0x7f95f810dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:56.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.502+0000 7f95fd47b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95f80fffd0 0x7f95f810dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53466/0 (socket says 172.21.15.92:53466) 2024-05-06T12:19:56.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.502+0000 7f95fd47b700 1 -- 172.21.15.92:0/34029076 learned_addr learned my addr 172.21.15.92:0/34029076 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:19:56.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.502+0000 7f95fd47b700 1 -- 172.21.15.92:0/34029076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95f8066af0 msgr2=0x7f95f810da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:56.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.502+0000 7f95fdc7c700 1 --2- 172.21.15.92:0/34029076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95f8066af0 0x7f95f810da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:56.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.502+0000 7f95fd47b700 1 --2- 172.21.15.92:0/34029076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95f8066af0 0x7f95f810da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:56.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.502+0000 7f95fd47b700 1 -- 172.21.15.92:0/34029076 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ec005040 con 0x7f95f80fffd0 2024-05-06T12:19:56.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.502+0000 7f95fd47b700 1 --2- 172.21.15.92:0/34029076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95f80fffd0 0x7f95f810dfb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f95f4002a50 tx=0x7f95f40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:19:56.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.502+0000 7f95ea7fc700 1 -- 172.21.15.92:0/34029076 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95f4007cc0 con 0x7f95f80fffd0 2024-05-06T12:19:56.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.502+0000 7f95ffee0700 1 -- 172.21.15.92:0/34029076 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95f810f0e0 con 0x7f95f80fffd0 2024-05-06T12:19:56.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.502+0000 7f95ea7fc700 1 -- 172.21.15.92:0/34029076 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f95f400f040 con 0x7f95f80fffd0 2024-05-06T12:19:56.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.502+0000 7f95ea7fc700 1 -- 172.21.15.92:0/34029076 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95f4009640 con 0x7f95f80fffd0 2024-05-06T12:19:56.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.502+0000 7f95ffee0700 1 -- 172.21.15.92:0/34029076 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95f810f620 con 0x7f95f80fffd0 2024-05-06T12:19:56.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.504+0000 7f95ffee0700 1 -- 172.21.15.92:0/34029076 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95f805f5c0 con 0x7f95f80fffd0 2024-05-06T12:19:56.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.504+0000 7f95ea7fc700 1 -- 172.21.15.92:0/34029076 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f95f40097c0 con 0x7f95f80fffd0 2024-05-06T12:19:56.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.504+0000 7f95ea7fc700 1 --2- 172.21.15.92:0/34029076 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f95e406e140 0x7f95e40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:56.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.505+0000 7f95ea7fc700 1 -- 172.21.15.92:0/34029076 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f95f408d8c0 con 0x7f95f80fffd0 2024-05-06T12:19:56.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.505+0000 7f95fdc7c700 1 --2- 172.21.15.92:0/34029076 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f95e406e140 0x7f95e40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:56.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.505+0000 7f95fdc7c700 1 --2- 172.21.15.92:0/34029076 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f95e406e140 0x7f95e40705f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f95ec00b970 tx=0x7f95ec009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:19:56.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.509+0000 7f95ea7fc700 1 -- 172.21.15.92:0/34029076 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f95f405a770 con 0x7f95f80fffd0 2024-05-06T12:19:56.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.722+0000 7f95ffee0700 1 -- 172.21.15.92:0/34029076 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f95f810f970 con 0x7f95e406e140 2024-05-06T12:19:56.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.723+0000 7f95ea7fc700 1 -- 172.21.15.92:0/34029076 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f95f810f970 con 0x7f95e406e140 2024-05-06T12:19:56.726 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:19:56.726 INFO:teuthology.orchestra.run.smithi092.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb", 2024-05-06T12:19:56.726 INFO:teuthology.orchestra.run.smithi092.stdout: "in_progress": true, 2024-05-06T12:19:56.726 INFO:teuthology.orchestra.run.smithi092.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-06T12:19:56.726 INFO:teuthology.orchestra.run.smithi092.stdout: "services_complete": [ 2024-05-06T12:19:56.727 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr", 2024-05-06T12:19:56.727 INFO:teuthology.orchestra.run.smithi092.stdout: "mon", 2024-05-06T12:19:56.727 INFO:teuthology.orchestra.run.smithi092.stdout: "crash" 2024-05-06T12:19:56.727 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-05-06T12:19:56.727 INFO:teuthology.orchestra.run.smithi092.stdout: "progress": "10/21 daemons upgraded", 2024-05-06T12:19:56.727 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "Currently upgrading osd daemons", 2024-05-06T12:19:56.727 INFO:teuthology.orchestra.run.smithi092.stdout: "is_paused": false 2024-05-06T12:19:56.727 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:19:56.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.725+0000 7f95ffee0700 1 -- 172.21.15.92:0/34029076 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f95e406e140 msgr2=0x7f95e40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:56.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.725+0000 7f95ffee0700 1 --2- 172.21.15.92:0/34029076 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f95e406e140 0x7f95e40705f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f95ec00b970 tx=0x7f95ec009e30 comp rx=0 tx=0).stop 2024-05-06T12:19:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.725+0000 7f95ffee0700 1 -- 172.21.15.92:0/34029076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95f80fffd0 msgr2=0x7f95f810dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.725+0000 7f95ffee0700 1 --2- 172.21.15.92:0/34029076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95f80fffd0 0x7f95f810dfb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f95f4002a50 tx=0x7f95f40046b0 comp rx=0 tx=0).stop 2024-05-06T12:19:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.725+0000 7f95ffee0700 1 -- 172.21.15.92:0/34029076 shutdown_connections 2024-05-06T12:19:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.725+0000 7f95ffee0700 1 --2- 172.21.15.92:0/34029076 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f95e406e140 0x7f95e40705f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.725+0000 7f95ffee0700 1 --2- 172.21.15.92:0/34029076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95f8066af0 0x7f95f810da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.725+0000 7f95ffee0700 1 --2- 172.21.15.92:0/34029076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95f80fffd0 0x7f95f810dfb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.725+0000 7f95ffee0700 1 -- 172.21.15.92:0/34029076 >> 172.21.15.92:0/34029076 conn(0x7f95f80f74c0 msgr2=0x7f95f80f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:19:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.725+0000 7f95ffee0700 1 -- 172.21.15.92:0/34029076 shutdown_connections 2024-05-06T12:19:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.725+0000 7f95ffee0700 1 -- 172.21.15.92:0/34029076 wait complete. 2024-05-06T12:19:56.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.827+0000 7fa588504700 1 -- 172.21.15.92:0/2158100902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5800fdfd0 msgr2=0x7fa5800fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:56.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.827+0000 7fa588504700 1 --2- 172.21.15.92:0/2158100902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5800fdfd0 0x7fa5800fe440 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa57c004940 tx=0x7fa57c009e30 comp rx=0 tx=0).stop 2024-05-06T12:19:56.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.827+0000 7fa588504700 1 -- 172.21.15.92:0/2158100902 shutdown_connections 2024-05-06T12:19:56.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.827+0000 7fa588504700 1 --2- 172.21.15.92:0/2158100902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5800fdfd0 0x7fa5800fe440 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:56.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.827+0000 7fa588504700 1 --2- 172.21.15.92:0/2158100902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5800fce60 0x7fa5800fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:56.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.827+0000 7fa588504700 1 -- 172.21.15.92:0/2158100902 >> 172.21.15.92:0/2158100902 conn(0x7fa5800f8540 msgr2=0x7fa5800fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:19:56.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.827+0000 7fa588504700 1 -- 172.21.15.92:0/2158100902 shutdown_connections 2024-05-06T12:19:56.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.827+0000 7fa588504700 1 -- 172.21.15.92:0/2158100902 wait complete. 2024-05-06T12:19:56.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.828+0000 7fa588504700 1 Processor -- start 2024-05-06T12:19:56.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.828+0000 7fa588504700 1 -- start start 2024-05-06T12:19:56.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.828+0000 7fa588504700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5800fce60 0x7fa58010d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:56.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.828+0000 7fa588504700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5800fdfd0 0x7fa58010de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:56.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.828+0000 7fa588504700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58010e4f0 con 0x7fa5800fce60 2024-05-06T12:19:56.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.828+0000 7fa588504700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa580112e90 con 0x7fa5800fdfd0 2024-05-06T12:19:56.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.828+0000 7fa5862a0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5800fce60 0x7fa58010d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:56.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.828+0000 7fa5862a0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5800fce60 0x7fa58010d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53480/0 (socket says 172.21.15.92:53480) 2024-05-06T12:19:56.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.828+0000 7fa5862a0700 1 -- 172.21.15.92:0/205867615 learned_addr learned my addr 172.21.15.92:0/205867615 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:19:56.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.828+0000 7fa5862a0700 1 -- 172.21.15.92:0/205867615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5800fdfd0 msgr2=0x7fa58010de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:56.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.828+0000 7fa5862a0700 1 --2- 172.21.15.92:0/205867615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5800fdfd0 0x7fa58010de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:56.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.828+0000 7fa5862a0700 1 -- 172.21.15.92:0/205867615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa57c005040 con 0x7fa5800fce60 2024-05-06T12:19:56.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.828+0000 7fa5862a0700 1 --2- 172.21.15.92:0/205867615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5800fce60 0x7fa58010d920 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa57000b790 tx=0x7fa570009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:19:56.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.829+0000 7fa576ffd700 1 -- 172.21.15.92:0/205867615 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa57000f770 con 0x7fa5800fce60 2024-05-06T12:19:56.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.829+0000 7fa588504700 1 -- 172.21.15.92:0/205867615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5801130f0 con 0x7fa5800fce60 2024-05-06T12:19:56.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.829+0000 7fa576ffd700 1 -- 172.21.15.92:0/205867615 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa57000fda0 con 0x7fa5800fce60 2024-05-06T12:19:56.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.829+0000 7fa576ffd700 1 -- 172.21.15.92:0/205867615 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa57000d310 con 0x7fa5800fce60 2024-05-06T12:19:56.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.829+0000 7fa588504700 1 -- 172.21.15.92:0/205867615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa580113660 con 0x7fa5800fce60 2024-05-06T12:19:56.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.830+0000 7fa588504700 1 -- 172.21.15.92:0/205867615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa58004c710 con 0x7fa5800fce60 2024-05-06T12:19:56.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.830+0000 7fa576ffd700 1 -- 172.21.15.92:0/205867615 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa570010420 con 0x7fa5800fce60 2024-05-06T12:19:56.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.831+0000 7fa576ffd700 1 --2- 172.21.15.92:0/205867615 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fa56c06e140 0x7fa56c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:19:56.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.831+0000 7fa576ffd700 1 -- 172.21.15.92:0/205867615 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa57008db40 con 0x7fa5800fce60 2024-05-06T12:19:56.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.831+0000 7fa585a9f700 1 --2- 172.21.15.92:0/205867615 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fa56c06e140 0x7fa56c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:19:56.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.831+0000 7fa585a9f700 1 --2- 172.21.15.92:0/205867615 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fa56c06e140 0x7fa56c0705f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa57c004f90 tx=0x7fa57c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:19:56.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:56.835+0000 7fa576ffd700 1 -- 172.21.15.92:0/205867615 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa57005aa70 con 0x7fa5800fce60 2024-05-06T12:19:57.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:56 smithi190 ceph-mon[79744]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:19:57.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:56 smithi190 ceph-mon[79744]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:19:57.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:56 smithi190 ceph-mon[79744]: Health check update: Degraded data redundancy: 65/372 objects degraded (17.473%), 17 pgs degraded (PG_DEGRADED) 2024-05-06T12:19:57.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:56 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/3461059001' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:19:57.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:56 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/2165867686' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:19:57.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:57.114+0000 7fa588504700 1 -- 172.21.15.92:0/205867615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa580105940 con 0x7fa5800fce60 2024-05-06T12:19:57.118 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:56 smithi092 ceph-mon[91933]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:19:57.118 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:56 smithi092 ceph-mon[91933]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:19:57.118 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:56 smithi092 ceph-mon[91933]: Health check update: Degraded data redundancy: 65/372 objects degraded (17.473%), 17 pgs degraded (PG_DEGRADED) 2024-05-06T12:19:57.118 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:56 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/3461059001' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:19:57.118 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:56 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/2165867686' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:19:57.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:57.115+0000 7fa576ffd700 1 -- 172.21.15.92:0/205867615 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1316 (secure 0 0 0) 0x7fa57005a600 con 0x7fa5800fce60 2024-05-06T12:19:57.119 INFO:teuthology.orchestra.run.smithi092.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; Degraded data redundancy: 65/372 objects degraded (17.473%), 17 pgs degraded 2024-05-06T12:19:57.119 INFO:teuthology.orchestra.run.smithi092.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-06T12:19:57.119 INFO:teuthology.orchestra.run.smithi092.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-06T12:19:57.119 INFO:teuthology.orchestra.run.smithi092.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 65/372 objects degraded (17.473%), 17 pgs degraded 2024-05-06T12:19:57.119 INFO:teuthology.orchestra.run.smithi092.stdout: pg 1.0 is active+undersized+degraded, acting [0,1] 2024-05-06T12:19:57.119 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.0 is active+undersized+degraded, acting [1,0] 2024-05-06T12:19:57.119 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.5 is active+undersized+degraded, acting [0,4] 2024-05-06T12:19:57.119 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.6 is active+undersized+degraded, acting [1,4] 2024-05-06T12:19:57.119 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.7 is active+undersized+degraded, acting [4,2] 2024-05-06T12:19:57.119 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.8 is active+undersized+degraded, acting [5,0] 2024-05-06T12:19:57.119 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.a is active+undersized+degraded, acting [1,4] 2024-05-06T12:19:57.120 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.b is active+undersized+degraded, acting [4,5] 2024-05-06T12:19:57.120 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.c is active+undersized+degraded, acting [2,0] 2024-05-06T12:19:57.120 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.d is active+undersized+degraded, acting [1,2] 2024-05-06T12:19:57.120 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.e is active+undersized+degraded, acting [2,0] 2024-05-06T12:19:57.120 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.12 is active+undersized+degraded, acting [1,0] 2024-05-06T12:19:57.120 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.14 is active+undersized+degraded, acting [4,5] 2024-05-06T12:19:57.120 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.15 is active+undersized+degraded, acting [1,0] 2024-05-06T12:19:57.120 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.1a is active+undersized+degraded, acting [4,5] 2024-05-06T12:19:57.120 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.1d is active+undersized+degraded, acting [5,0] 2024-05-06T12:19:57.120 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.1f is active+undersized+degraded, acting [0,4] 2024-05-06T12:19:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:57.117+0000 7fa588504700 1 -- 172.21.15.92:0/205867615 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fa56c06e140 msgr2=0x7fa56c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:57.117+0000 7fa588504700 1 --2- 172.21.15.92:0/205867615 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fa56c06e140 0x7fa56c0705f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa57c004f90 tx=0x7fa57c009e30 comp rx=0 tx=0).stop 2024-05-06T12:19:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:57.117+0000 7fa588504700 1 -- 172.21.15.92:0/205867615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5800fce60 msgr2=0x7fa58010d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:19:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:57.117+0000 7fa588504700 1 --2- 172.21.15.92:0/205867615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5800fce60 0x7fa58010d920 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa57000b790 tx=0x7fa570009250 comp rx=0 tx=0).stop 2024-05-06T12:19:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:57.118+0000 7fa588504700 1 -- 172.21.15.92:0/205867615 shutdown_connections 2024-05-06T12:19:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:57.118+0000 7fa588504700 1 --2- 172.21.15.92:0/205867615 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fa56c06e140 0x7fa56c0705f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:57.118+0000 7fa588504700 1 --2- 172.21.15.92:0/205867615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5800fce60 0x7fa58010d920 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:57.118+0000 7fa588504700 1 --2- 172.21.15.92:0/205867615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5800fdfd0 0x7fa58010de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:19:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:57.118+0000 7fa588504700 1 -- 172.21.15.92:0/205867615 >> 172.21.15.92:0/205867615 conn(0x7fa5800f8540 msgr2=0x7fa5800f9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:19:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:57.118+0000 7fa588504700 1 -- 172.21.15.92:0/205867615 shutdown_connections 2024-05-06T12:19:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:19:57.118+0000 7fa588504700 1 -- 172.21.15.92:0/205867615 wait complete. 2024-05-06T12:19:58.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:57 smithi190 ceph-mon[79744]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:19:58.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:57 smithi190 ceph-mon[79744]: pgmap v122: 65 pgs: 16 active+undersized+degraded, 10 active+undersized, 39 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s; 61/372 objects degraded (16.398%) 2024-05-06T12:19:58.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:57 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/205867615' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:19:58.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:57 smithi092 ceph-mon[91933]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:19:58.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:57 smithi092 ceph-mon[91933]: pgmap v122: 65 pgs: 16 active+undersized+degraded, 10 active+undersized, 39 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s; 61/372 objects degraded (16.398%) 2024-05-06T12:19:58.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:57 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/205867615' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:19:59.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:58 smithi092 ceph-mon[91933]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 61/372 objects degraded (16.398%), 16 pgs degraded) 2024-05-06T12:19:59.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:58 smithi190 ceph-mon[79744]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 61/372 objects degraded (16.398%), 16 pgs degraded) 2024-05-06T12:20:00.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:19:59 smithi092 ceph-mon[91933]: pgmap v123: 65 pgs: 65 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-05-06T12:20:00.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:19:59 smithi190 ceph-mon[79744]: pgmap v123: 65 pgs: 65 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-05-06T12:20:01.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:00 smithi092 ceph-mon[91933]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-06T12:20:01.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:00 smithi190 ceph-mon[79744]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-06T12:20:02.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:01 smithi092 ceph-mon[91933]: pgmap v124: 65 pgs: 65 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-06T12:20:02.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:01 smithi190 ceph-mon[79744]: pgmap v124: 65 pgs: 65 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-06T12:20:04.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:03 smithi092 ceph-mon[91933]: pgmap v125: 65 pgs: 65 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 4 op/s 2024-05-06T12:20:04.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:03 smithi190 ceph-mon[79744]: pgmap v125: 65 pgs: 65 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 4 op/s 2024-05-06T12:20:05.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:05 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:05.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:05 smithi092 ceph-mon[91933]: pgmap v126: 65 pgs: 65 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 4 op/s 2024-05-06T12:20:05.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:05 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:05.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:05 smithi190 ceph-mon[79744]: pgmap v126: 65 pgs: 65 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 4 op/s 2024-05-06T12:20:08.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:07 smithi092 ceph-mon[91933]: pgmap v127: 65 pgs: 65 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:20:08.301 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:07 smithi190 ceph-mon[79744]: pgmap v127: 65 pgs: 65 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:20:10.200 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:09 smithi190 ceph-mon[79744]: pgmap v128: 65 pgs: 65 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-05-06T12:20:10.200 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:10.200 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:20:10.200 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-06T12:20:10.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:09 smithi092 ceph-mon[91933]: pgmap v128: 65 pgs: 65 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-05-06T12:20:10.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:10.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:20:10.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-06T12:20:11.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:10 smithi092 ceph-mon[91933]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-06T12:20:11.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:10 smithi092 ceph-mon[91933]: Upgrade: osd.4 is safe to restart 2024-05-06T12:20:11.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:10 smithi190 ceph-mon[79744]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-06T12:20:11.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:10 smithi190 ceph-mon[79744]: Upgrade: osd.4 is safe to restart 2024-05-06T12:20:11.979 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:11 smithi190 ceph-mon[79744]: pgmap v129: 65 pgs: 65 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:20:11.980 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:11 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:11.980 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:11 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-06T12:20:11.980 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:11 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:20:12.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:11 smithi092 ceph-mon[91933]: pgmap v129: 65 pgs: 65 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:20:12.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:11 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:12.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:11 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-06T12:20:12.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:11 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:20:13.078 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:12 smithi190 ceph-mon[79744]: Upgrade: Updating osd.4 2024-05-06T12:20:13.078 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:12 smithi190 ceph-mon[79744]: Deploying daemon osd.4 on smithi190 2024-05-06T12:20:13.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:12 smithi092 ceph-mon[91933]: Upgrade: Updating osd.4 2024-05-06T12:20:13.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:12 smithi092 ceph-mon[91933]: Deploying daemon osd.4 on smithi190 2024-05-06T12:20:13.899 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:13 smithi190 ceph-mon[79744]: pgmap v130: 65 pgs: 65 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:20:14.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:13 smithi092 ceph-mon[91933]: pgmap v130: 65 pgs: 65 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:20:14.598 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:14 smithi190 systemd[1]: Stopping Ceph osd.4 for aee1d912-0ba0-11ef-bc97-c7b262605968... 2024-05-06T12:20:14.599 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:14 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4[46190]: 2024-05-06T12:20:14.492+0000 7f3b370df700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-06T12:20:14.599 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:14 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4[46190]: 2024-05-06T12:20:14.492+0000 7f3b370df700 -1 osd.4 70 *** Got signal Terminated *** 2024-05-06T12:20:14.599 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:14 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4[46190]: 2024-05-06T12:20:14.492+0000 7f3b370df700 -1 osd.4 70 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-06T12:20:15.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:14 smithi092 ceph-mon[91933]: osd.4 marked itself down and dead 2024-05-06T12:20:15.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:14 smithi190 ceph-mon[79744]: osd.4 marked itself down and dead 2024-05-06T12:20:15.687 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:15 smithi190 podman[94131]: 2024-05-06 12:20:15.433187156 +0000 UTC m=+1.044527245 container died 3a7da09e4c48c141e9210a160aa35ea313ed7cdcaf60500cee085d37858e96a4 (image=quay.ceph.io/ceph-ci/ceph@sha256:1a0b7fd3cefd2c5baddf682493f92b4e27596f3c0ff27b02fdccd2891f56859f, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=quincy-e81df42, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-06T12:20:16.096 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:15 smithi190 ceph-mon[79744]: pgmap v131: 65 pgs: 65 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:20:16.096 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:15 smithi190 ceph-mon[79744]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-06T12:20:16.096 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:15 smithi190 ceph-mon[79744]: osdmap e71: 6 total, 5 up, 6 in 2024-05-06T12:20:16.096 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:15 smithi190 podman[94131]: 2024-05-06 12:20:15.784381659 +0000 UTC m=+1.395721741 container cleanup 3a7da09e4c48c141e9210a160aa35ea313ed7cdcaf60500cee085d37858e96a4 (image=quay.ceph.io/ceph-ci/ceph@sha256:1a0b7fd3cefd2c5baddf682493f92b4e27596f3c0ff27b02fdccd2891f56859f, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=quincy-e81df42, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-06T12:20:16.097 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:15 smithi190 bash[94131]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4 2024-05-06T12:20:16.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:15 smithi092 ceph-mon[91933]: pgmap v131: 65 pgs: 65 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:20:16.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:15 smithi092 ceph-mon[91933]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-06T12:20:16.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:15 smithi092 ceph-mon[91933]: osdmap e71: 6 total, 5 up, 6 in 2024-05-06T12:20:16.348 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:16 smithi190 podman[94150]: 2024-05-06 12:20:16.094211484 +0000 UTC m=+0.656571214 container remove 3a7da09e4c48c141e9210a160aa35ea313ed7cdcaf60500cee085d37858e96a4 (image=quay.ceph.io/ceph-ci/ceph@sha256:1a0b7fd3cefd2c5baddf682493f92b4e27596f3c0ff27b02fdccd2891f56859f, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=quincy-e81df42, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-06T12:20:16.730 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:16 smithi190 podman[94245]: 2024-05-06 12:20:16.292478452 +0000 UTC m=+0.017850559 image pull e4ce8ad5d0c4b9ea60bc5f6b8b0fdb5efbe59bf7771fb0920fc2200e6ae546b0 quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb 2024-05-06T12:20:16.730 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:16 smithi190 podman[94245]: 2024-05-06 12:20:16.403447119 +0000 UTC m=+0.128819255 container create 6592978f3296ced863f96a3f5dfad67370051676650c1df54a52d9d9bfb5fecc (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-70ce953, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-06T12:20:16.730 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:16 smithi190 podman[94245]: 2024-05-06 12:20:16.728598025 +0000 UTC m=+0.453970118 container init 6592978f3296ced863f96a3f5dfad67370051676650c1df54a52d9d9bfb5fecc (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-70ce953) 2024-05-06T12:20:17.027 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:16 smithi190 ceph-mon[79744]: osdmap e72: 6 total, 5 up, 6 in 2024-05-06T12:20:17.028 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:16 smithi190 podman[94245]: 2024-05-06 12:20:16.732156075 +0000 UTC m=+0.457528168 container start 6592978f3296ced863f96a3f5dfad67370051676650c1df54a52d9d9bfb5fecc (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-70ce953, ceph=True, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-06T12:20:17.028 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:16 smithi190 podman[94245]: 2024-05-06 12:20:16.822737891 +0000 UTC m=+0.548110009 container attach 6592978f3296ced863f96a3f5dfad67370051676650c1df54a52d9d9bfb5fecc (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-70ce953, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-06T12:20:17.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:16 smithi092 ceph-mon[91933]: osdmap e72: 6 total, 5 up, 6 in 2024-05-06T12:20:17.348 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:17 smithi190 podman[94245]: 2024-05-06 12:20:17.02617103 +0000 UTC m=+0.751543132 container died 6592978f3296ced863f96a3f5dfad67370051676650c1df54a52d9d9bfb5fecc (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-70ce953, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-06T12:20:17.624 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:17 smithi190 podman[94271]: 2024-05-06 12:20:17.601162484 +0000 UTC m=+0.569238968 container remove 6592978f3296ced863f96a3f5dfad67370051676650c1df54a52d9d9bfb5fecc (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-70ce953, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-06T12:20:17.889 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:17 smithi190 systemd[1]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@osd.4.service: Deactivated successfully. 2024-05-06T12:20:17.889 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:17 smithi190 systemd[1]: Stopped Ceph osd.4 for aee1d912-0ba0-11ef-bc97-c7b262605968. 2024-05-06T12:20:17.889 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:17 smithi190 systemd[1]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@osd.4.service: Consumed 51.322s CPU time. 2024-05-06T12:20:17.889 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:17 smithi190 systemd[1]: Starting Ceph osd.4 for aee1d912-0ba0-11ef-bc97-c7b262605968... 2024-05-06T12:20:18.167 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:17 smithi190 ceph-mon[79744]: pgmap v134: 65 pgs: 6 stale+active+clean, 59 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-05-06T12:20:18.167 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:18 smithi190 podman[94375]: 2024-05-06 12:20:17.94540204 +0000 UTC m=+0.018661216 image pull e4ce8ad5d0c4b9ea60bc5f6b8b0fdb5efbe59bf7771fb0920fc2200e6ae546b0 quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb 2024-05-06T12:20:18.167 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:18 smithi190 podman[94375]: 2024-05-06 12:20:18.052614844 +0000 UTC m=+0.125874059 container create 509a455ea9247fbcd4395862cbb12afff15feefd5ea28882a74a862de2ed794f (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4-activate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-70ce953, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-06T12:20:18.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:17 smithi092 ceph-mon[91933]: pgmap v134: 65 pgs: 6 stale+active+clean, 59 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-05-06T12:20:18.456 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:18 smithi190 podman[94375]: 2024-05-06 12:20:18.29708661 +0000 UTC m=+0.370345770 container init 509a455ea9247fbcd4395862cbb12afff15feefd5ea28882a74a862de2ed794f (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4-activate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-70ce953, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-06T12:20:18.456 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:18 smithi190 podman[94375]: 2024-05-06 12:20:18.300753037 +0000 UTC m=+0.374012189 container start 509a455ea9247fbcd4395862cbb12afff15feefd5ea28882a74a862de2ed794f (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4-activate, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-70ce953) 2024-05-06T12:20:18.456 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:18 smithi190 podman[94375]: 2024-05-06 12:20:18.454568363 +0000 UTC m=+0.527827522 container attach 509a455ea9247fbcd4395862cbb12afff15feefd5ea28882a74a862de2ed794f (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4-activate, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-70ce953) 2024-05-06T12:20:19.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:18 smithi190 ceph-mon[79744]: Health check failed: Degraded data redundancy: 42/372 objects degraded (11.290%), 10 pgs degraded (PG_DEGRADED) 2024-05-06T12:20:19.099 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:18 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4-activate[94390]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-06T12:20:19.099 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:18 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4-activate[94390]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-05-06T12:20:19.099 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:18 smithi190 bash[94375]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-06T12:20:19.099 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:18 smithi190 bash[94375]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-05-06T12:20:19.099 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:18 smithi190 bash[94375]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-05-06T12:20:19.099 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:18 smithi190 bash[94375]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-05-06T12:20:19.099 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:18 smithi190 bash[94375]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-05-06T12:20:19.099 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:18 smithi190 bash[94375]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-06T12:20:19.099 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:18 smithi190 bash[94375]: --> ceph-volume raw activate successful for osd ID: 4 2024-05-06T12:20:19.099 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:18 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4-activate[94390]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-05-06T12:20:19.099 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:18 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4-activate[94390]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-05-06T12:20:19.099 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:18 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4-activate[94390]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-05-06T12:20:19.100 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:18 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4-activate[94390]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-06T12:20:19.100 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:18 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4-activate[94390]: --> ceph-volume raw activate successful for osd ID: 4 2024-05-06T12:20:19.100 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:18 smithi190 podman[94375]: 2024-05-06 12:20:18.769785818 +0000 UTC m=+0.843044983 container died 509a455ea9247fbcd4395862cbb12afff15feefd5ea28882a74a862de2ed794f (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4-activate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-70ce953, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-06T12:20:19.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:18 smithi092 ceph-mon[91933]: Health check failed: Degraded data redundancy: 42/372 objects degraded (11.290%), 10 pgs degraded (PG_DEGRADED) 2024-05-06T12:20:19.653 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:19 smithi190 podman[94520]: 2024-05-06 12:20:19.400616266 +0000 UTC m=+0.624945049 container remove 509a455ea9247fbcd4395862cbb12afff15feefd5ea28882a74a862de2ed794f (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4-activate, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-70ce953, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-06T12:20:19.653 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:19 smithi190 podman[94587]: 2024-05-06 12:20:19.51647332 +0000 UTC m=+0.017918180 image pull e4ce8ad5d0c4b9ea60bc5f6b8b0fdb5efbe59bf7771fb0920fc2200e6ae546b0 quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb 2024-05-06T12:20:19.653 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:19 smithi190 podman[94587]: 2024-05-06 12:20:19.651569906 +0000 UTC m=+0.153014793 container create 371c0be776c7d2e21c34bc6bb7b3703d88bbecca2bae00c5b3b8a0c5faa8d5d1 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4, RELEASE=squid-70ce953, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430) 2024-05-06T12:20:19.942 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:19 smithi190 podman[94587]: 2024-05-06 12:20:19.870664949 +0000 UTC m=+0.372109791 container init 371c0be776c7d2e21c34bc6bb7b3703d88bbecca2bae00c5b3b8a0c5faa8d5d1 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-70ce953, org.label-schema.schema-version=1.0) 2024-05-06T12:20:19.943 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:19 smithi190 podman[94587]: 2024-05-06 12:20:19.873559572 +0000 UTC m=+0.375004414 container start 371c0be776c7d2e21c34bc6bb7b3703d88bbecca2bae00c5b3b8a0c5faa8d5d1 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-70ce953, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-06T12:20:19.943 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:19 smithi190 ceph-osd[94605]: -- 172.21.15.190:0/3152832983 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b00c467680 con 0x55b00c431000 2024-05-06T12:20:20.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:19 smithi092 ceph-mon[91933]: pgmap v135: 65 pgs: 10 active+undersized+degraded, 3 stale+active+clean, 11 active+undersized, 41 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 42/372 objects degraded (11.290%) 2024-05-06T12:20:20.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:19 smithi190 ceph-mon[79744]: pgmap v135: 65 pgs: 10 active+undersized+degraded, 3 stale+active+clean, 11 active+undersized, 41 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 42/372 objects degraded (11.290%) 2024-05-06T12:20:20.348 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:19 smithi190 bash[94587]: 371c0be776c7d2e21c34bc6bb7b3703d88bbecca2bae00c5b3b8a0c5faa8d5d1 2024-05-06T12:20:20.348 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:19 smithi190 systemd[1]: Started Ceph osd.4 for aee1d912-0ba0-11ef-bc97-c7b262605968. 2024-05-06T12:20:20.848 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:20 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4[94601]: 2024-05-06T12:20:20.410+0000 7fcbd26657c0 -1 Falling back to public interface 2024-05-06T12:20:21.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:21 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:21.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:21 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:21.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:21 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:20:21.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:21 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:21.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:21 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:21.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:21 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:20:22.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:22 smithi190 ceph-mon[79744]: pgmap v136: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 64/372 objects degraded (17.204%) 2024-05-06T12:20:22.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:22 smithi092 ceph-mon[91933]: pgmap v136: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 64/372 objects degraded (17.204%) 2024-05-06T12:20:23.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:23 smithi190 ceph-mon[79744]: pgmap v137: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 64/372 objects degraded (17.204%) 2024-05-06T12:20:23.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:23 smithi092 ceph-mon[91933]: pgmap v137: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 64/372 objects degraded (17.204%) 2024-05-06T12:20:24.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:24 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:24.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:24 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:24.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:24 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:24.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:24 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:20:24.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:24 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:24.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:24 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:24.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:24 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:24.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:24 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:20:25.598 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:25 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4[94601]: 2024-05-06T12:20:25.296+0000 7fcbd26657c0 -1 osd.4 70 log_to_monitors true 2024-05-06T12:20:25.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:25 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:25.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:25 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:25.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:25 smithi190 ceph-mon[79744]: pgmap v138: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 64/372 objects degraded (17.204%) 2024-05-06T12:20:25.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:25 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:25.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:25 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:25.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:25 smithi092 ceph-mon[91933]: pgmap v138: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 323 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 64/372 objects degraded (17.204%) 2024-05-06T12:20:26.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:26 smithi190 ceph-mon[79744]: from='osd.4 [v2:172.21.15.190:6808/839214066,v1:172.21.15.190:6809/839214066]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-06T12:20:26.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:26 smithi190 ceph-mon[79744]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-06T12:20:26.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:26 smithi190 ceph-mon[79744]: Health check update: Degraded data redundancy: 64/372 objects degraded (17.204%), 16 pgs degraded (PG_DEGRADED) 2024-05-06T12:20:26.599 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:20:26 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4[94601]: 2024-05-06T12:20:26.282+0000 7fcbc3abc700 -1 osd.4 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-06T12:20:26.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:26 smithi092 ceph-mon[91933]: from='osd.4 [v2:172.21.15.190:6808/839214066,v1:172.21.15.190:6809/839214066]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-06T12:20:26.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:26 smithi092 ceph-mon[91933]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-06T12:20:26.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:26 smithi092 ceph-mon[91933]: Health check update: Degraded data redundancy: 64/372 objects degraded (17.204%), 16 pgs degraded (PG_DEGRADED) 2024-05-06T12:20:27.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.234+0000 7f59c3de9700 1 -- 172.21.15.92:0/976991221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59bc0fce60 msgr2=0x7f59bc0fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:20:27.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.234+0000 7f59c3de9700 1 --2- 172.21.15.92:0/976991221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59bc0fce60 0x7f59bc0fd270 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009ce0 tx=0x7f59b4004d10 comp rx=0 tx=0).stop 2024-05-06T12:20:27.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.234+0000 7f59c3de9700 1 -- 172.21.15.92:0/976991221 shutdown_connections 2024-05-06T12:20:27.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.234+0000 7f59c3de9700 1 --2- 172.21.15.92:0/976991221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59bc0fdfd0 0x7f59bc0fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:27.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.234+0000 7f59c3de9700 1 --2- 172.21.15.92:0/976991221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59bc0fce60 0x7f59bc0fd270 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:27.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.234+0000 7f59c3de9700 1 -- 172.21.15.92:0/976991221 >> 172.21.15.92:0/976991221 conn(0x7f59bc0f8520 msgr2=0x7f59bc0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:20:27.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.234+0000 7f59c3de9700 1 -- 172.21.15.92:0/976991221 shutdown_connections 2024-05-06T12:20:27.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.234+0000 7f59c3de9700 1 -- 172.21.15.92:0/976991221 wait complete. 2024-05-06T12:20:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.235+0000 7f59c3de9700 1 Processor -- start 2024-05-06T12:20:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.235+0000 7f59c3de9700 1 -- start start 2024-05-06T12:20:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.235+0000 7f59c3de9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59bc0fce60 0x7f59bc10dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:20:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.235+0000 7f59c3de9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59bc0fdfd0 0x7f59bc10e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:20:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.235+0000 7f59c3de9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc10e620 con 0x7f59bc0fce60 2024-05-06T12:20:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.235+0000 7f59c3de9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc10e790 con 0x7f59bc0fdfd0 2024-05-06T12:20:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.235+0000 7f59c1b85700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59bc0fce60 0x7f59bc10dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:20:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.235+0000 7f59c1b85700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59bc0fce60 0x7f59bc10dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34258/0 (socket says 172.21.15.92:34258) 2024-05-06T12:20:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.235+0000 7f59c1b85700 1 -- 172.21.15.92:0/3751545279 learned_addr learned my addr 172.21.15.92:0/3751545279 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:20:27.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.235+0000 7f59c1384700 1 --2- 172.21.15.92:0/3751545279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59bc0fdfd0 0x7f59bc10e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:20:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.235+0000 7f59c1b85700 1 -- 172.21.15.92:0/3751545279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59bc0fdfd0 msgr2=0x7f59bc10e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:20:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.235+0000 7f59c1b85700 1 --2- 172.21.15.92:0/3751545279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59bc0fdfd0 0x7f59bc10e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.236+0000 7f59c1b85700 1 -- 172.21.15.92:0/3751545279 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b4009940 con 0x7f59bc0fce60 2024-05-06T12:20:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.236+0000 7f59c1b85700 1 --2- 172.21.15.92:0/3751545279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59bc0fce60 0x7f59bc10dae0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009380 tx=0x7f59b4004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:20:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.236+0000 7f59ba7fc700 1 -- 172.21.15.92:0/3751545279 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59b400e040 con 0x7f59bc0fce60 2024-05-06T12:20:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.236+0000 7f59ba7fc700 1 -- 172.21.15.92:0/3751545279 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f59b40128e0 con 0x7f59bc0fce60 2024-05-06T12:20:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.236+0000 7f59c3de9700 1 -- 172.21.15.92:0/3751545279 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59bc113190 con 0x7f59bc0fce60 2024-05-06T12:20:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.236+0000 7f59ba7fc700 1 -- 172.21.15.92:0/3751545279 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59b401c930 con 0x7f59bc0fce60 2024-05-06T12:20:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.236+0000 7f59c3de9700 1 -- 172.21.15.92:0/3751545279 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59bc113750 con 0x7f59bc0fce60 2024-05-06T12:20:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.237+0000 7f59c3de9700 1 -- 172.21.15.92:0/3751545279 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59bc04c710 con 0x7f59bc0fce60 2024-05-06T12:20:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.238+0000 7f59ba7fc700 1 -- 172.21.15.92:0/3751545279 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f59b4012a50 con 0x7f59bc0fce60 2024-05-06T12:20:27.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.238+0000 7f59ba7fc700 1 --2- 172.21.15.92:0/3751545279 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f59a406e140 0x7f59a40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:20:27.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.238+0000 7f59ba7fc700 1 -- 172.21.15.92:0/3751545279 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f59b408f6b0 con 0x7f59bc0fce60 2024-05-06T12:20:27.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.238+0000 7f59c1384700 1 --2- 172.21.15.92:0/3751545279 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f59a406e140 0x7f59a40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:20:27.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.238+0000 7f59c1384700 1 --2- 172.21.15.92:0/3751545279 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f59a406e140 0x7f59a40705f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f59b0004f40 tx=0x7f59b000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:20:27.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.243+0000 7f59ba7fc700 1 -- 172.21.15.92:0/3751545279 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f59b405c560 con 0x7f59bc0fce60 2024-05-06T12:20:27.477 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.474+0000 7f59c3de9700 1 -- 172.21.15.92:0/3751545279 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f59bc1023b0 con 0x7f59a406e140 2024-05-06T12:20:27.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.476+0000 7f59ba7fc700 1 -- 172.21.15.92:0/3751545279 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f59bc1023b0 con 0x7f59a406e140 2024-05-06T12:20:27.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.478+0000 7f59c3de9700 1 -- 172.21.15.92:0/3751545279 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f59a406e140 msgr2=0x7f59a40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:20:27.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.478+0000 7f59c3de9700 1 --2- 172.21.15.92:0/3751545279 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f59a406e140 0x7f59a40705f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f59b0004f40 tx=0x7f59b000b040 comp rx=0 tx=0).stop 2024-05-06T12:20:27.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.478+0000 7f59c3de9700 1 -- 172.21.15.92:0/3751545279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59bc0fce60 msgr2=0x7f59bc10dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:20:27.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.478+0000 7f59c3de9700 1 --2- 172.21.15.92:0/3751545279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59bc0fce60 0x7f59bc10dae0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009380 tx=0x7f59b4004d10 comp rx=0 tx=0).stop 2024-05-06T12:20:27.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.478+0000 7f59c3de9700 1 -- 172.21.15.92:0/3751545279 shutdown_connections 2024-05-06T12:20:27.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.478+0000 7f59c3de9700 1 --2- 172.21.15.92:0/3751545279 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f59a406e140 0x7f59a40705f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:27.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.478+0000 7f59c3de9700 1 --2- 172.21.15.92:0/3751545279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59bc0fce60 0x7f59bc10dae0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:27.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.478+0000 7f59c3de9700 1 --2- 172.21.15.92:0/3751545279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59bc0fdfd0 0x7f59bc10e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:27.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.478+0000 7f59c3de9700 1 -- 172.21.15.92:0/3751545279 >> 172.21.15.92:0/3751545279 conn(0x7f59bc0f8520 msgr2=0x7f59bc0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:20:27.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.478+0000 7f59c3de9700 1 -- 172.21.15.92:0/3751545279 shutdown_connections 2024-05-06T12:20:27.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.478+0000 7f59c3de9700 1 -- 172.21.15.92:0/3751545279 wait complete. 2024-05-06T12:20:27.491 INFO:teuthology.orchestra.run.smithi092.stdout:true 2024-05-06T12:20:27.492 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:27 smithi092 ceph-mon[91933]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-06T12:20:27.492 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:27 smithi092 ceph-mon[91933]: from='osd.4 [v2:172.21.15.190:6808/839214066,v1:172.21.15.190:6809/839214066]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-05-06T12:20:27.492 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:27 smithi092 ceph-mon[91933]: osdmap e73: 6 total, 5 up, 6 in 2024-05-06T12:20:27.492 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:27 smithi092 ceph-mon[91933]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-05-06T12:20:27.492 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:27 smithi092 ceph-mon[91933]: pgmap v140: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 323 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 64/372 objects degraded (17.204%) 2024-05-06T12:20:27.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.579+0000 7f774ce47700 1 -- 172.21.15.92:0/4057270873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77480fd270 msgr2=0x7f77480fd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:20:27.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.579+0000 7f774ce47700 1 --2- 172.21.15.92:0/4057270873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77480fd270 0x7f77480fd680 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7738004940 tx=0x7f7738009e30 comp rx=0 tx=0).stop 2024-05-06T12:20:27.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.580+0000 7f774ce47700 1 -- 172.21.15.92:0/4057270873 shutdown_connections 2024-05-06T12:20:27.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.580+0000 7f774ce47700 1 --2- 172.21.15.92:0/4057270873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77480fdbc0 0x7f77480fbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:27.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.580+0000 7f774ce47700 1 --2- 172.21.15.92:0/4057270873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77480fd270 0x7f77480fd680 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:27.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.580+0000 7f774ce47700 1 -- 172.21.15.92:0/4057270873 >> 172.21.15.92:0/4057270873 conn(0x7f77480f6b80 msgr2=0x7f77480f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:20:27.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.580+0000 7f774ce47700 1 -- 172.21.15.92:0/4057270873 shutdown_connections 2024-05-06T12:20:27.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.580+0000 7f774ce47700 1 -- 172.21.15.92:0/4057270873 wait complete. 2024-05-06T12:20:27.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.580+0000 7f774ce47700 1 Processor -- start 2024-05-06T12:20:27.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.580+0000 7f774ce47700 1 -- start start 2024-05-06T12:20:27.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.580+0000 7f774ce47700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77480fd270 0x7f77480ffdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:20:27.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.580+0000 7f774ce47700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77480fe2d0 0x7f77480fe740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:20:27.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.580+0000 7f774ce47700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77480fec80 con 0x7f77480fd270 2024-05-06T12:20:27.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.580+0000 7f774ce47700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77480fedf0 con 0x7f77480fe2d0 2024-05-06T12:20:27.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.581+0000 7f774659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77480fd270 0x7f77480ffdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:20:27.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.581+0000 7f774659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77480fd270 0x7f77480ffdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34278/0 (socket says 172.21.15.92:34278) 2024-05-06T12:20:27.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.581+0000 7f774659c700 1 -- 172.21.15.92:0/4158497904 learned_addr learned my addr 172.21.15.92:0/4158497904 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:20:27.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.581+0000 7f773dd9b700 1 --2- 172.21.15.92:0/4158497904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77480fe2d0 0x7f77480fe740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:20:27.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.581+0000 7f774659c700 1 -- 172.21.15.92:0/4158497904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77480fe2d0 msgr2=0x7f77480fe740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:20:27.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.581+0000 7f774659c700 1 --2- 172.21.15.92:0/4158497904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77480fe2d0 0x7f77480fe740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:27.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.581+0000 7f774659c700 1 -- 172.21.15.92:0/4158497904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7738005040 con 0x7f77480fd270 2024-05-06T12:20:27.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.581+0000 7f774659c700 1 --2- 172.21.15.92:0/4158497904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77480fd270 0x7f77480ffdb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7738000c00 tx=0x7f7738009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:20:27.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.581+0000 7f773f7fe700 1 -- 172.21.15.92:0/4158497904 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f773800e040 con 0x7f77480fd270 2024-05-06T12:20:27.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.581+0000 7f773f7fe700 1 -- 172.21.15.92:0/4158497904 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f773800bbf0 con 0x7f77480fd270 2024-05-06T12:20:27.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.581+0000 7f774ce47700 1 -- 172.21.15.92:0/4158497904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77480ff070 con 0x7f77480fd270 2024-05-06T12:20:27.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.581+0000 7f773f7fe700 1 -- 172.21.15.92:0/4158497904 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f773801c630 con 0x7f77480fd270 2024-05-06T12:20:27.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.581+0000 7f774ce47700 1 -- 172.21.15.92:0/4158497904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7748115d90 con 0x7f77480fd270 2024-05-06T12:20:27.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.582+0000 7f774ce47700 1 -- 172.21.15.92:0/4158497904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7748101b20 con 0x7f77480fd270 2024-05-06T12:20:27.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.583+0000 7f773f7fe700 1 -- 172.21.15.92:0/4158497904 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f773800bd60 con 0x7f77480fd270 2024-05-06T12:20:27.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.583+0000 7f773f7fe700 1 --2- 172.21.15.92:0/4158497904 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f773406e140 0x7f77340705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:20:27.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.583+0000 7f773f7fe700 1 -- 172.21.15.92:0/4158497904 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f773808ee00 con 0x7f77480fd270 2024-05-06T12:20:27.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.583+0000 7f773dd9b700 1 --2- 172.21.15.92:0/4158497904 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f773406e140 0x7f77340705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:20:27.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.584+0000 7f773dd9b700 1 --2- 172.21.15.92:0/4158497904 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f773406e140 0x7f77340705f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f77300102e0 tx=0x7f773000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:20:27.591 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.588+0000 7f773f7fe700 1 -- 172.21.15.92:0/4158497904 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7738058200 con 0x7f77480fd270 2024-05-06T12:20:27.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:27 smithi190 ceph-mon[79744]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-06T12:20:27.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:27 smithi190 ceph-mon[79744]: from='osd.4 [v2:172.21.15.190:6808/839214066,v1:172.21.15.190:6809/839214066]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-05-06T12:20:27.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:27 smithi190 ceph-mon[79744]: osdmap e73: 6 total, 5 up, 6 in 2024-05-06T12:20:27.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:27 smithi190 ceph-mon[79744]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-05-06T12:20:27.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:27 smithi190 ceph-mon[79744]: pgmap v140: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 323 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 64/372 objects degraded (17.204%) 2024-05-06T12:20:27.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.807+0000 7f774ce47700 1 -- 172.21.15.92:0/4158497904 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f77480fdbc0 con 0x7f773406e140 2024-05-06T12:20:27.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.808+0000 7f773f7fe700 1 -- 172.21.15.92:0/4158497904 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f77480fdbc0 con 0x7f773406e140 2024-05-06T12:20:27.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.810+0000 7f774ce47700 1 -- 172.21.15.92:0/4158497904 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f773406e140 msgr2=0x7f77340705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:20:27.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.810+0000 7f774ce47700 1 --2- 172.21.15.92:0/4158497904 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f773406e140 0x7f77340705f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f77300102e0 tx=0x7f773000b040 comp rx=0 tx=0).stop 2024-05-06T12:20:27.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.810+0000 7f774ce47700 1 -- 172.21.15.92:0/4158497904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77480fd270 msgr2=0x7f77480ffdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:20:27.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.810+0000 7f774ce47700 1 --2- 172.21.15.92:0/4158497904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77480fd270 0x7f77480ffdb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7738000c00 tx=0x7f7738009e30 comp rx=0 tx=0).stop 2024-05-06T12:20:27.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.810+0000 7f774ce47700 1 -- 172.21.15.92:0/4158497904 shutdown_connections 2024-05-06T12:20:27.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.810+0000 7f774ce47700 1 --2- 172.21.15.92:0/4158497904 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f773406e140 0x7f77340705f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:27.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.810+0000 7f774ce47700 1 --2- 172.21.15.92:0/4158497904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77480fd270 0x7f77480ffdb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:27.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.810+0000 7f774ce47700 1 --2- 172.21.15.92:0/4158497904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77480fe2d0 0x7f77480fe740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:27.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.810+0000 7f774ce47700 1 -- 172.21.15.92:0/4158497904 >> 172.21.15.92:0/4158497904 conn(0x7f77480f6b80 msgr2=0x7f77480fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:20:27.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.810+0000 7f774ce47700 1 -- 172.21.15.92:0/4158497904 shutdown_connections 2024-05-06T12:20:27.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.810+0000 7f774ce47700 1 -- 172.21.15.92:0/4158497904 wait complete. 2024-05-06T12:20:27.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.913+0000 7f84518c4700 1 -- 172.21.15.92:0/172547512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f844c0fed60 msgr2=0x7f844c0faea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:20:27.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.913+0000 7f84518c4700 1 --2- 172.21.15.92:0/172547512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f844c0fed60 0x7f844c0faea0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8440004940 tx=0x7f8440009e30 comp rx=0 tx=0).stop 2024-05-06T12:20:27.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.913+0000 7f84518c4700 1 -- 172.21.15.92:0/172547512 shutdown_connections 2024-05-06T12:20:27.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.913+0000 7f84518c4700 1 --2- 172.21.15.92:0/172547512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f844c0fed60 0x7f844c0faea0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:27.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.913+0000 7f84518c4700 1 --2- 172.21.15.92:0/172547512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f844c0fe410 0x7f844c0fe820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:27.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.913+0000 7f84518c4700 1 -- 172.21.15.92:0/172547512 >> 172.21.15.92:0/172547512 conn(0x7f844c0f6bd0 msgr2=0x7f844c0f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:20:27.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.913+0000 7f84518c4700 1 -- 172.21.15.92:0/172547512 shutdown_connections 2024-05-06T12:20:27.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.913+0000 7f84518c4700 1 -- 172.21.15.92:0/172547512 wait complete. 2024-05-06T12:20:27.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.913+0000 7f84518c4700 1 Processor -- start 2024-05-06T12:20:27.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.913+0000 7f84518c4700 1 -- start start 2024-05-06T12:20:27.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.914+0000 7f84518c4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f844c0fe410 0x7f844c10d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:20:27.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.914+0000 7f84518c4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f844c0fed60 0x7f844c10dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:20:27.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.914+0000 7f84518c4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844c10e3a0 con 0x7f844c0fe410 2024-05-06T12:20:27.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.914+0000 7f84518c4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844c10e510 con 0x7f844c0fed60 2024-05-06T12:20:27.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.914+0000 7f844affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f844c0fe410 0x7f844c10d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:20:27.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.914+0000 7f844affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f844c0fe410 0x7f844c10d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34300/0 (socket says 172.21.15.92:34300) 2024-05-06T12:20:27.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.914+0000 7f844affd700 1 -- 172.21.15.92:0/3957078234 learned_addr learned my addr 172.21.15.92:0/3957078234 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:20:27.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.914+0000 7f844a7fc700 1 --2- 172.21.15.92:0/3957078234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f844c0fed60 0x7f844c10dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:20:27.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.914+0000 7f844affd700 1 -- 172.21.15.92:0/3957078234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f844c0fed60 msgr2=0x7f844c10dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:20:27.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.914+0000 7f844affd700 1 --2- 172.21.15.92:0/3957078234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f844c0fed60 0x7f844c10dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:27.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.914+0000 7f844affd700 1 -- 172.21.15.92:0/3957078234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8440005040 con 0x7f844c0fe410 2024-05-06T12:20:27.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.914+0000 7f844affd700 1 --2- 172.21.15.92:0/3957078234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f844c0fe410 0x7f844c10d860 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8444001db0 tx=0x7f8444009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:20:27.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.914+0000 7f84337fe700 1 -- 172.21.15.92:0/3957078234 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f844400ed50 con 0x7f844c0fe410 2024-05-06T12:20:27.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.914+0000 7f84337fe700 1 -- 172.21.15.92:0/3957078234 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8444007df0 con 0x7f844c0fe410 2024-05-06T12:20:27.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.914+0000 7f84518c4700 1 -- 172.21.15.92:0/3957078234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f844c112f70 con 0x7f844c0fe410 2024-05-06T12:20:27.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.914+0000 7f84337fe700 1 -- 172.21.15.92:0/3957078234 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8444014640 con 0x7f844c0fe410 2024-05-06T12:20:27.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.914+0000 7f84518c4700 1 -- 172.21.15.92:0/3957078234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f844c1134a0 con 0x7f844c0fe410 2024-05-06T12:20:27.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.915+0000 7f84518c4700 1 -- 172.21.15.92:0/3957078234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f844c04c710 con 0x7f844c0fe410 2024-05-06T12:20:27.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.916+0000 7f84337fe700 1 -- 172.21.15.92:0/3957078234 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8444007920 con 0x7f844c0fe410 2024-05-06T12:20:27.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.916+0000 7f84337fe700 1 --2- 172.21.15.92:0/3957078234 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f843406e140 0x7f84340705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:20:27.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.916+0000 7f84337fe700 1 -- 172.21.15.92:0/3957078234 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f844408d8f0 con 0x7f844c0fe410 2024-05-06T12:20:27.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.916+0000 7f844a7fc700 1 --2- 172.21.15.92:0/3957078234 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f843406e140 0x7f84340705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:20:27.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.917+0000 7f844a7fc700 1 --2- 172.21.15.92:0/3957078234 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f843406e140 0x7f84340705f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f844000b970 tx=0x7f8440009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:20:27.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:27.921+0000 7f84337fe700 1 -- 172.21.15.92:0/3957078234 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f844405a850 con 0x7f844c0fe410 2024-05-06T12:20:28.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.125+0000 7f84518c4700 1 -- 172.21.15.92:0/3957078234 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f844c1136f0 con 0x7f843406e140 2024-05-06T12:20:28.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.135+0000 7f84337fe700 1 -- 172.21.15.92:0/3957078234 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f844c1136f0 con 0x7f843406e140 2024-05-06T12:20:28.139 INFO:teuthology.orchestra.run.smithi092.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-06T12:20:28.139 INFO:teuthology.orchestra.run.smithi092.stdout:alertmanager.smithi092 smithi092 *:9093,9094 running (5m) 71s ago 13m 25.2M - 0.25.0 c8568f914cd2 be5637d7d25a 2024-05-06T12:20:28.139 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi092 smithi092 running (2m) 71s ago 14m 6660k - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 8da0494e1bdc 2024-05-06T12:20:28.139 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi190 smithi190 running (2m) 5s ago 13m 6665k - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 641a7cdaa56e 2024-05-06T12:20:28.139 INFO:teuthology.orchestra.run.smithi092.stdout:grafana.smithi092 smithi092 *:3000 running (5m) 71s ago 13m 75.0M - 9.4.7 954c08fa6188 9180cb3e5258 2024-05-06T12:20:28.139 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.ijsqgp smithi092 running (8m) 71s ago 8m 60.7M - 17.2.7-925-ge81df420 96d5ae7fd4b4 b4c2ed6d2c82 2024-05-06T12:20:28.139 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.rosbll smithi092 running (8m) 71s ago 8m 122M - 17.2.7-925-ge81df420 96d5ae7fd4b4 5f5426cd1bf5 2024-05-06T12:20:28.139 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.byjvgf smithi190 running (8m) 5s ago 8m 126M - 17.2.7-925-ge81df420 96d5ae7fd4b4 24581d2e1c5d 2024-05-06T12:20:28.140 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.idzoff smithi190 running (8m) 5s ago 8m 64.0M - 17.2.7-925-ge81df420 96d5ae7fd4b4 20f2641696ef 2024-05-06T12:20:28.140 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi092.gyzeqb smithi092 *:8443,9283,8765 running (4m) 71s ago 15m 504M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 de9d2198327a 2024-05-06T12:20:28.140 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi190.kczxkh smithi190 *:8443,9283,8765 running (4m) 5s ago 13m 451M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 e84ab3758ac7 2024-05-06T12:20:28.140 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi092 smithi092 running (3m) 71s ago 15m 46.0M 2048M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 ac07f7158087 2024-05-06T12:20:28.140 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi190 smithi190 running (3m) 5s ago 13m 40.5M 2048M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 7a3c08eb716f 2024-05-06T12:20:28.140 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi092 smithi092 *:9100 running (5m) 71s ago 14m 14.9M - 1.5.0 0da6a335fe13 c796446b206f 2024-05-06T12:20:28.141 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi190 smithi190 *:9100 running (5m) 5s ago 13m 16.2M - 1.5.0 0da6a335fe13 647e28111a76 2024-05-06T12:20:28.141 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 smithi092 running (2m) 71s ago 12m 82.0M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 8c96c75b25f4 2024-05-06T12:20:28.141 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 smithi092 running (112s) 71s ago 11m 82.0M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 e8007a510f08 2024-05-06T12:20:28.141 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 smithi092 running (76s) 71s ago 11m 14.8M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 2beca7448789 2024-05-06T12:20:28.141 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 smithi190 running (41s) 5s ago 10m 86.7M 2581M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 914ada199f93 2024-05-06T12:20:28.141 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 smithi190 running (8s) 5s ago 10m 12.4M 2581M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 371c0be776c7 2024-05-06T12:20:28.141 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 smithi190 running (10m) 5s ago 10m 189M 2581M 17.2.7-925-ge81df420 96d5ae7fd4b4 e7452c6b5cec 2024-05-06T12:20:28.142 INFO:teuthology.orchestra.run.smithi092.stdout:prometheus.smithi092 smithi092 *:9095 running (4m) 71s ago 13m 54.4M - 2.43.0 a07b618ecd1d bc5445691527 2024-05-06T12:20:28.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.137+0000 7f84518c4700 1 -- 172.21.15.92:0/3957078234 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f843406e140 msgr2=0x7f84340705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:20:28.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.137+0000 7f84518c4700 1 --2- 172.21.15.92:0/3957078234 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f843406e140 0x7f84340705f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f844000b970 tx=0x7f8440009e30 comp rx=0 tx=0).stop 2024-05-06T12:20:28.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.137+0000 7f84518c4700 1 -- 172.21.15.92:0/3957078234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f844c0fe410 msgr2=0x7f844c10d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:20:28.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.137+0000 7f84518c4700 1 --2- 172.21.15.92:0/3957078234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f844c0fe410 0x7f844c10d860 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8444001db0 tx=0x7f8444009ee0 comp rx=0 tx=0).stop 2024-05-06T12:20:28.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.138+0000 7f84518c4700 1 -- 172.21.15.92:0/3957078234 shutdown_connections 2024-05-06T12:20:28.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.138+0000 7f84518c4700 1 --2- 172.21.15.92:0/3957078234 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f843406e140 0x7f84340705f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:28.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.138+0000 7f84518c4700 1 --2- 172.21.15.92:0/3957078234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f844c0fe410 0x7f844c10d860 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:28.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.138+0000 7f84518c4700 1 --2- 172.21.15.92:0/3957078234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f844c0fed60 0x7f844c10dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:28.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.138+0000 7f84518c4700 1 -- 172.21.15.92:0/3957078234 >> 172.21.15.92:0/3957078234 conn(0x7f844c0f6bd0 msgr2=0x7f844c0f81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:20:28.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.138+0000 7f84518c4700 1 -- 172.21.15.92:0/3957078234 shutdown_connections 2024-05-06T12:20:28.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.138+0000 7f84518c4700 1 -- 172.21.15.92:0/3957078234 wait complete. 2024-05-06T12:20:28.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.238+0000 7fc7ff931700 1 -- 172.21.15.92:0/1044364119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7f80fcf60 msgr2=0x7fc7f80fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:20:28.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.238+0000 7fc7ff931700 1 --2- 172.21.15.92:0/1044364119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7f80fcf60 0x7fc7f80fd3d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8004940 tx=0x7fc7e8009e30 comp rx=0 tx=0).stop 2024-05-06T12:20:28.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.238+0000 7fc7ff931700 1 -- 172.21.15.92:0/1044364119 shutdown_connections 2024-05-06T12:20:28.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.238+0000 7fc7ff931700 1 --2- 172.21.15.92:0/1044364119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7f80fcf60 0x7fc7f80fd3d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:28.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.238+0000 7fc7ff931700 1 --2- 172.21.15.92:0/1044364119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7f80fbcb0 0x7fc7f80fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:28.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.238+0000 7fc7ff931700 1 -- 172.21.15.92:0/1044364119 >> 172.21.15.92:0/1044364119 conn(0x7fc7f80f7430 msgr2=0x7fc7f80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:20:28.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.238+0000 7fc7ff931700 1 -- 172.21.15.92:0/1044364119 shutdown_connections 2024-05-06T12:20:28.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.238+0000 7fc7ff931700 1 -- 172.21.15.92:0/1044364119 wait complete. 2024-05-06T12:20:28.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.238+0000 7fc7ff931700 1 Processor -- start 2024-05-06T12:20:28.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.238+0000 7fc7ff931700 1 -- start start 2024-05-06T12:20:28.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.239+0000 7fc7ff931700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7f80fbcb0 0x7fc7f810db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:20:28.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.239+0000 7fc7ff931700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7f80fcf60 0x7fc7f810e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:20:28.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.239+0000 7fc7ff931700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f810e640 con 0x7fc7f80fbcb0 2024-05-06T12:20:28.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.239+0000 7fc7ff931700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f810edf0 con 0x7fc7f80fcf60 2024-05-06T12:20:28.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.239+0000 7fc7fd6cd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7f80fbcb0 0x7fc7f810db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:20:28.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.239+0000 7fc7fd6cd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7f80fbcb0 0x7fc7f810db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34314/0 (socket says 172.21.15.92:34314) 2024-05-06T12:20:28.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.239+0000 7fc7fd6cd700 1 -- 172.21.15.92:0/626737076 learned_addr learned my addr 172.21.15.92:0/626737076 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:20:28.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.239+0000 7fc7fcecc700 1 --2- 172.21.15.92:0/626737076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7f80fcf60 0x7fc7f810e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:20:28.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.239+0000 7fc7fd6cd700 1 -- 172.21.15.92:0/626737076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7f80fcf60 msgr2=0x7fc7f810e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:20:28.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.239+0000 7fc7fd6cd700 1 --2- 172.21.15.92:0/626737076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7f80fcf60 0x7fc7f810e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:28.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.239+0000 7fc7fd6cd700 1 -- 172.21.15.92:0/626737076 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e8005040 con 0x7fc7f80fbcb0 2024-05-06T12:20:28.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.239+0000 7fc7fd6cd700 1 --2- 172.21.15.92:0/626737076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7f80fbcb0 0x7fc7f810db00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4002a50 tx=0x7fc7f40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:20:28.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.239+0000 7fc7edffb700 1 -- 172.21.15.92:0/626737076 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7f4007cc0 con 0x7fc7f80fbcb0 2024-05-06T12:20:28.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.239+0000 7fc7ff931700 1 -- 172.21.15.92:0/626737076 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f810f0d0 con 0x7fc7f80fbcb0 2024-05-06T12:20:28.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.239+0000 7fc7edffb700 1 -- 172.21.15.92:0/626737076 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc7f400f040 con 0x7fc7f80fbcb0 2024-05-06T12:20:28.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.239+0000 7fc7edffb700 1 -- 172.21.15.92:0/626737076 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7f400da70 con 0x7fc7f80fbcb0 2024-05-06T12:20:28.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.240+0000 7fc7ff931700 1 -- 172.21.15.92:0/626737076 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f810f610 con 0x7fc7f80fbcb0 2024-05-06T12:20:28.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.241+0000 7fc7ff931700 1 -- 172.21.15.92:0/626737076 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7f8101af0 con 0x7fc7f80fbcb0 2024-05-06T12:20:28.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.241+0000 7fc7edffb700 1 -- 172.21.15.92:0/626737076 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc7f400dc10 con 0x7fc7f80fbcb0 2024-05-06T12:20:28.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.242+0000 7fc7edffb700 1 --2- 172.21.15.92:0/626737076 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fc7e406e140 0x7fc7e40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:20:28.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.242+0000 7fc7fcecc700 1 --2- 172.21.15.92:0/626737076 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fc7e406e140 0x7fc7e40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:20:28.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.242+0000 7fc7edffb700 1 -- 172.21.15.92:0/626737076 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc7f408d180 con 0x7fc7f80fbcb0 2024-05-06T12:20:28.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.242+0000 7fc7fcecc700 1 --2- 172.21.15.92:0/626737076 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fc7e406e140 0x7fc7e40705f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc7e800b970 tx=0x7fc7e8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:20:28.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.246+0000 7fc7edffb700 1 -- 172.21.15.92:0/626737076 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc7f405a030 con 0x7fc7f80fbcb0 2024-05-06T12:20:28.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:28 smithi092 ceph-mon[91933]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-06T12:20:28.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:28 smithi092 ceph-mon[91933]: osd.4 [v2:172.21.15.190:6808/839214066,v1:172.21.15.190:6809/839214066] boot 2024-05-06T12:20:28.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:28 smithi092 ceph-mon[91933]: osdmap e74: 6 total, 6 up, 6 in 2024-05-06T12:20:28.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:28 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-06T12:20:28.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:28 smithi092 ceph-mon[91933]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:20:28.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:28 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:28.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:28 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:28.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:28 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:20:28.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:28 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:20:28.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:28 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:28.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:28 smithi092 ceph-mon[91933]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:20:28.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:28 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:20:28.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:28 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:20:28.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:28 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:20:28.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:28 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:20:28.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:28 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-06T12:20:28.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:28 smithi092 ceph-mon[91933]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-06T12:20:28.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:28 smithi092 ceph-mon[91933]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-05-06T12:20:28.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:28 smithi092 ceph-mon[91933]: osdmap e75: 6 total, 6 up, 6 in 2024-05-06T12:20:28.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.545+0000 7fc7ff931700 1 -- 172.21.15.92:0/626737076 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc7f8104940 con 0x7fc7f80fbcb0 2024-05-06T12:20:28.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.546+0000 7fc7edffb700 1 -- 172.21.15.92:0/626737076 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fc7f4059bc0 con 0x7fc7f80fbcb0 2024-05-06T12:20:28.549 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:20:28.549 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": { 2024-05-06T12:20:28.549 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:20:28.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:20:28.549 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": { 2024-05-06T12:20:28.549 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:20:28.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:20:28.550 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": { 2024-05-06T12:20:28.550 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 1, 2024-05-06T12:20:28.550 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 5 2024-05-06T12:20:28.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:20:28.550 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-05-06T12:20:28.550 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 4 2024-05-06T12:20:28.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:20:28.550 INFO:teuthology.orchestra.run.smithi092.stdout: "overall": { 2024-05-06T12:20:28.550 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 5, 2024-05-06T12:20:28.550 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 9 2024-05-06T12:20:28.550 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-06T12:20:28.550 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:20:28.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.548+0000 7fc7ff931700 1 -- 172.21.15.92:0/626737076 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fc7e406e140 msgr2=0x7fc7e40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:20:28.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.548+0000 7fc7ff931700 1 --2- 172.21.15.92:0/626737076 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fc7e406e140 0x7fc7e40705f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc7e800b970 tx=0x7fc7e8009e30 comp rx=0 tx=0).stop 2024-05-06T12:20:28.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.548+0000 7fc7ff931700 1 -- 172.21.15.92:0/626737076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7f80fbcb0 msgr2=0x7fc7f810db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:20:28.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.548+0000 7fc7ff931700 1 --2- 172.21.15.92:0/626737076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7f80fbcb0 0x7fc7f810db00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4002a50 tx=0x7fc7f40046b0 comp rx=0 tx=0).stop 2024-05-06T12:20:28.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.548+0000 7fc7ff931700 1 -- 172.21.15.92:0/626737076 shutdown_connections 2024-05-06T12:20:28.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.548+0000 7fc7ff931700 1 --2- 172.21.15.92:0/626737076 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fc7e406e140 0x7fc7e40705f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:28.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.548+0000 7fc7ff931700 1 --2- 172.21.15.92:0/626737076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7f80fbcb0 0x7fc7f810db00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:28.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.548+0000 7fc7ff931700 1 --2- 172.21.15.92:0/626737076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7f80fcf60 0x7fc7f810e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:28.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.548+0000 7fc7ff931700 1 -- 172.21.15.92:0/626737076 >> 172.21.15.92:0/626737076 conn(0x7fc7f80f7430 msgr2=0x7fc7f8065bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:20:28.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.548+0000 7fc7ff931700 1 -- 172.21.15.92:0/626737076 shutdown_connections 2024-05-06T12:20:28.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.548+0000 7fc7ff931700 1 -- 172.21.15.92:0/626737076 wait complete. 2024-05-06T12:20:28.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:28 smithi190 ceph-mon[79744]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-06T12:20:28.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:28 smithi190 ceph-mon[79744]: osd.4 [v2:172.21.15.190:6808/839214066,v1:172.21.15.190:6809/839214066] boot 2024-05-06T12:20:28.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:28 smithi190 ceph-mon[79744]: osdmap e74: 6 total, 6 up, 6 in 2024-05-06T12:20:28.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:28 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-06T12:20:28.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:28 smithi190 ceph-mon[79744]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:20:28.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:28 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:28.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:28 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:28.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:28 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:20:28.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:28 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:20:28.600 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:28 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:28.600 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:28 smithi190 ceph-mon[79744]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:20:28.600 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:28 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:20:28.600 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:28 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:20:28.600 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:28 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:20:28.600 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:28 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:20:28.600 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:28 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-06T12:20:28.600 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:28 smithi190 ceph-mon[79744]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-06T12:20:28.600 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:28 smithi190 ceph-mon[79744]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-05-06T12:20:28.600 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:28 smithi190 ceph-mon[79744]: osdmap e75: 6 total, 6 up, 6 in 2024-05-06T12:20:28.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.649+0000 7efffbb93700 1 -- 172.21.15.92:0/1977679387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efff40fcf70 msgr2=0x7efff40fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:20:28.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.649+0000 7efffbb93700 1 --2- 172.21.15.92:0/1977679387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efff40fcf70 0x7efff40fd3e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efff0004940 tx=0x7efff0009e30 comp rx=0 tx=0).stop 2024-05-06T12:20:28.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.649+0000 7efffbb93700 1 -- 172.21.15.92:0/1977679387 shutdown_connections 2024-05-06T12:20:28.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.649+0000 7efffbb93700 1 --2- 172.21.15.92:0/1977679387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efff40fcf70 0x7efff40fd3e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:28.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.649+0000 7efffbb93700 1 --2- 172.21.15.92:0/1977679387 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efff40fbcc0 0x7efff40fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:28.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.649+0000 7efffbb93700 1 -- 172.21.15.92:0/1977679387 >> 172.21.15.92:0/1977679387 conn(0x7efff40f7440 msgr2=0x7efff40f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:20:28.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.649+0000 7efffbb93700 1 -- 172.21.15.92:0/1977679387 shutdown_connections 2024-05-06T12:20:28.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.649+0000 7efffbb93700 1 -- 172.21.15.92:0/1977679387 wait complete. 2024-05-06T12:20:28.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.650+0000 7efffbb93700 1 Processor -- start 2024-05-06T12:20:28.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.650+0000 7efffbb93700 1 -- start start 2024-05-06T12:20:28.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.650+0000 7efffbb93700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efff40fbcc0 0x7efff410d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:20:28.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.650+0000 7efffbb93700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efff40fcf70 0x7efff410de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:20:28.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.650+0000 7efffbb93700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff410e430 con 0x7efff40fcf70 2024-05-06T12:20:28.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.650+0000 7efffbb93700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff410e5a0 con 0x7efff40fbcc0 2024-05-06T12:20:28.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.650+0000 7efff912e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efff40fcf70 0x7efff410de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:20:28.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.650+0000 7efff912e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efff40fcf70 0x7efff410de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34330/0 (socket says 172.21.15.92:34330) 2024-05-06T12:20:28.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.650+0000 7efff912e700 1 -- 172.21.15.92:0/313092042 learned_addr learned my addr 172.21.15.92:0/313092042 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:20:28.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.650+0000 7efff992f700 1 --2- 172.21.15.92:0/313092042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efff40fbcc0 0x7efff410d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:20:28.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.650+0000 7efff912e700 1 -- 172.21.15.92:0/313092042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efff40fbcc0 msgr2=0x7efff410d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:20:28.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.650+0000 7efff912e700 1 --2- 172.21.15.92:0/313092042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efff40fbcc0 0x7efff410d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:28.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.650+0000 7efff912e700 1 -- 172.21.15.92:0/313092042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff0005040 con 0x7efff40fcf70 2024-05-06T12:20:28.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.651+0000 7efff912e700 1 --2- 172.21.15.92:0/313092042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efff40fcf70 0x7efff410de30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efff000b3b0 tx=0x7efff0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:20:28.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.651+0000 7effea7fc700 1 -- 172.21.15.92:0/313092042 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efff000e040 con 0x7efff40fcf70 2024-05-06T12:20:28.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.651+0000 7effea7fc700 1 -- 172.21.15.92:0/313092042 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efff0004290 con 0x7efff40fcf70 2024-05-06T12:20:28.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.651+0000 7effea7fc700 1 -- 172.21.15.92:0/313092042 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efff001c3f0 con 0x7efff40fcf70 2024-05-06T12:20:28.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.651+0000 7efffbb93700 1 -- 172.21.15.92:0/313092042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efff4112fa0 con 0x7efff40fcf70 2024-05-06T12:20:28.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.651+0000 7efffbb93700 1 -- 172.21.15.92:0/313092042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff4113530 con 0x7efff40fcf70 2024-05-06T12:20:28.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.652+0000 7efffbb93700 1 -- 172.21.15.92:0/313092042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efff4101b00 con 0x7efff40fcf70 2024-05-06T12:20:28.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.652+0000 7effea7fc700 1 -- 172.21.15.92:0/313092042 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7efff000bbf0 con 0x7efff40fcf70 2024-05-06T12:20:28.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.653+0000 7effea7fc700 1 --2- 172.21.15.92:0/313092042 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7effe006e140 0x7effe00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:20:28.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.653+0000 7effea7fc700 1 -- 172.21.15.92:0/313092042 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6410+0+0 (secure 0 0 0) 0x7efff008ede0 con 0x7efff40fcf70 2024-05-06T12:20:28.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.653+0000 7efff992f700 1 --2- 172.21.15.92:0/313092042 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7effe006e140 0x7effe00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:20:28.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.653+0000 7efff992f700 1 --2- 172.21.15.92:0/313092042 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7effe006e140 0x7effe00705f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7effe4009fd0 tx=0x7effe400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:20:28.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.658+0000 7effea7fc700 1 -- 172.21.15.92:0/313092042 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7efff005bd10 con 0x7efff40fcf70 2024-05-06T12:20:28.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.898+0000 7efffbb93700 1 -- 172.21.15.92:0/313092042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7efff404c710 con 0x7efff40fcf70 2024-05-06T12:20:28.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.898+0000 7effea7fc700 1 -- 172.21.15.92:0/313092042 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1918 (secure 0 0 0) 0x7efff005b8a0 con 0x7efff40fcf70 2024-05-06T12:20:28.901 INFO:teuthology.orchestra.run.smithi092.stdout:e22 2024-05-06T12:20:28.901 INFO:teuthology.orchestra.run.smithi092.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-06T12:20:28.901 INFO:teuthology.orchestra.run.smithi092.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:20:28.901 INFO:teuthology.orchestra.run.smithi092.stdout:legacy client fscid: 1 2024-05-06T12:20:28.902 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:20:28.902 INFO:teuthology.orchestra.run.smithi092.stdout:Filesystem 'cephfs' (1) 2024-05-06T12:20:28.902 INFO:teuthology.orchestra.run.smithi092.stdout:fs_name cephfs 2024-05-06T12:20:28.902 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 22 2024-05-06T12:20:28.902 INFO:teuthology.orchestra.run.smithi092.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-06T12:20:28.902 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-05-06T12:11:44.952671+0000 2024-05-06T12:20:28.902 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2024-05-06T12:19:06.643003+0000 2024-05-06T12:20:28.902 INFO:teuthology.orchestra.run.smithi092.stdout:tableserver 0 2024-05-06T12:20:28.902 INFO:teuthology.orchestra.run.smithi092.stdout:root 0 2024-05-06T12:20:28.902 INFO:teuthology.orchestra.run.smithi092.stdout:session_timeout 60 2024-05-06T12:20:28.902 INFO:teuthology.orchestra.run.smithi092.stdout:session_autoclose 300 2024-05-06T12:20:28.902 INFO:teuthology.orchestra.run.smithi092.stdout:max_file_size 1099511627776 2024-05-06T12:20:28.902 INFO:teuthology.orchestra.run.smithi092.stdout:max_xattr_size 65536 2024-05-06T12:20:28.903 INFO:teuthology.orchestra.run.smithi092.stdout:required_client_features {} 2024-05-06T12:20:28.903 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure 0 2024-05-06T12:20:28.903 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure_osd_epoch 46 2024-05-06T12:20:28.903 INFO:teuthology.orchestra.run.smithi092.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:20:28.903 INFO:teuthology.orchestra.run.smithi092.stdout:max_mds 2 2024-05-06T12:20:28.903 INFO:teuthology.orchestra.run.smithi092.stdout:in 0,1 2024-05-06T12:20:28.903 INFO:teuthology.orchestra.run.smithi092.stdout:up {0=14472,1=24267} 2024-05-06T12:20:28.903 INFO:teuthology.orchestra.run.smithi092.stdout:failed 2024-05-06T12:20:28.903 INFO:teuthology.orchestra.run.smithi092.stdout:damaged 2024-05-06T12:20:28.903 INFO:teuthology.orchestra.run.smithi092.stdout:stopped 2024-05-06T12:20:28.903 INFO:teuthology.orchestra.run.smithi092.stdout:data_pools [3] 2024-05-06T12:20:28.903 INFO:teuthology.orchestra.run.smithi092.stdout:metadata_pool 2 2024-05-06T12:20:28.903 INFO:teuthology.orchestra.run.smithi092.stdout:inline_data enabled 2024-05-06T12:20:28.903 INFO:teuthology.orchestra.run.smithi092.stdout:balancer 2024-05-06T12:20:28.903 INFO:teuthology.orchestra.run.smithi092.stdout:bal_rank_mask -1 2024-05-06T12:20:28.904 INFO:teuthology.orchestra.run.smithi092.stdout:standby_count_wanted 1 2024-05-06T12:20:28.904 INFO:teuthology.orchestra.run.smithi092.stdout:qdb_cluster leader: 14472 members: 24267,14472 2024-05-06T12:20:28.904 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.rosbll{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.92:6828/1663253598,v1:172.21.15.92:6829/1663253598] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:20:28.904 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.ijsqgp{0:14484} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.92:6826/3613441054,v1:172.21.15.92:6827/3613441054] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:20:28.904 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.byjvgf{1:24267} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.190:6824/1565151699,v1:172.21.15.190:6825/1565151699] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:20:28.904 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.idzoff{1:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.190:6826/2453728380,v1:172.21.15.190:6827/2453728380] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:20:28.904 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:20:28.904 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:20:28.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.900+0000 7efffbb93700 1 -- 172.21.15.92:0/313092042 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7effe006e140 msgr2=0x7effe00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:20:28.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.900+0000 7efffbb93700 1 --2- 172.21.15.92:0/313092042 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7effe006e140 0x7effe00705f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7effe4009fd0 tx=0x7effe400b040 comp rx=0 tx=0).stop 2024-05-06T12:20:28.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.900+0000 7efffbb93700 1 -- 172.21.15.92:0/313092042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efff40fcf70 msgr2=0x7efff410de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:20:28.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.900+0000 7efffbb93700 1 --2- 172.21.15.92:0/313092042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efff40fcf70 0x7efff410de30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efff000b3b0 tx=0x7efff0009e30 comp rx=0 tx=0).stop 2024-05-06T12:20:28.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.900+0000 7efffbb93700 1 -- 172.21.15.92:0/313092042 shutdown_connections 2024-05-06T12:20:28.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.900+0000 7efffbb93700 1 --2- 172.21.15.92:0/313092042 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7effe006e140 0x7effe00705f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:28.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.900+0000 7efffbb93700 1 --2- 172.21.15.92:0/313092042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efff40fbcc0 0x7efff410d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:28.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.900+0000 7efffbb93700 1 --2- 172.21.15.92:0/313092042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efff40fcf70 0x7efff410de30 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:28.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.900+0000 7efffbb93700 1 -- 172.21.15.92:0/313092042 >> 172.21.15.92:0/313092042 conn(0x7efff40f7440 msgr2=0x7efff4066800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:20:28.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.900+0000 7efffbb93700 1 -- 172.21.15.92:0/313092042 shutdown_connections 2024-05-06T12:20:28.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:28.900+0000 7efffbb93700 1 -- 172.21.15.92:0/313092042 wait complete. 2024-05-06T12:20:28.906 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 22 2024-05-06T12:20:29.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.002+0000 7f7b31268700 1 -- 172.21.15.92:0/416802110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b2c0fed30 msgr2=0x7f7b2c0fae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:20:29.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.002+0000 7f7b31268700 1 --2- 172.21.15.92:0/416802110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b2c0fed30 0x7f7b2c0fae70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009ce0 tx=0x7f7b20004d10 comp rx=0 tx=0).stop 2024-05-06T12:20:29.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.002+0000 7f7b31268700 1 -- 172.21.15.92:0/416802110 shutdown_connections 2024-05-06T12:20:29.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.002+0000 7f7b31268700 1 --2- 172.21.15.92:0/416802110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b2c0fed30 0x7f7b2c0fae70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:29.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.002+0000 7f7b31268700 1 --2- 172.21.15.92:0/416802110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b2c0fe3e0 0x7f7b2c0fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:29.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.002+0000 7f7b31268700 1 -- 172.21.15.92:0/416802110 >> 172.21.15.92:0/416802110 conn(0x7f7b2c0f6b80 msgr2=0x7f7b2c0f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:20:29.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.002+0000 7f7b31268700 1 -- 172.21.15.92:0/416802110 shutdown_connections 2024-05-06T12:20:29.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.002+0000 7f7b31268700 1 -- 172.21.15.92:0/416802110 wait complete. 2024-05-06T12:20:29.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.002+0000 7f7b31268700 1 Processor -- start 2024-05-06T12:20:29.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.002+0000 7f7b31268700 1 -- start start 2024-05-06T12:20:29.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.003+0000 7f7b31268700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b2c0fe3e0 0x7f7b2c10db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:20:29.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.003+0000 7f7b31268700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b2c0fed30 0x7f7b2c10e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:20:29.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.003+0000 7f7b31268700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c10e670 con 0x7f7b2c0fed30 2024-05-06T12:20:29.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.003+0000 7f7b31268700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c10eec0 con 0x7f7b2c0fe3e0 2024-05-06T12:20:29.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.003+0000 7f7b2a7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b2c0fed30 0x7f7b2c10e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:20:29.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.003+0000 7f7b2a7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b2c0fed30 0x7f7b2c10e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34356/0 (socket says 172.21.15.92:34356) 2024-05-06T12:20:29.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.003+0000 7f7b2a7fc700 1 -- 172.21.15.92:0/3903420437 learned_addr learned my addr 172.21.15.92:0/3903420437 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:20:29.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.003+0000 7f7b2affd700 1 --2- 172.21.15.92:0/3903420437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b2c0fe3e0 0x7f7b2c10db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:20:29.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.003+0000 7f7b2a7fc700 1 -- 172.21.15.92:0/3903420437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b2c0fe3e0 msgr2=0x7f7b2c10db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:20:29.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.003+0000 7f7b2a7fc700 1 --2- 172.21.15.92:0/3903420437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b2c0fe3e0 0x7f7b2c10db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:29.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.003+0000 7f7b2a7fc700 1 -- 172.21.15.92:0/3903420437 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b20009940 con 0x7f7b2c0fed30 2024-05-06T12:20:29.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.003+0000 7f7b2affd700 1 --2- 172.21.15.92:0/3903420437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b2c0fe3e0 0x7f7b2c10db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-06T12:20:29.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.003+0000 7f7b2a7fc700 1 --2- 172.21.15.92:0/3903420437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b2c0fed30 0x7f7b2c10e070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7b20004740 tx=0x7f7b20004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:20:29.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.003+0000 7f7b137fe700 1 -- 172.21.15.92:0/3903420437 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b2000e040 con 0x7f7b2c0fed30 2024-05-06T12:20:29.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.003+0000 7f7b137fe700 1 -- 172.21.15.92:0/3903420437 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b200128e0 con 0x7f7b2c0fed30 2024-05-06T12:20:29.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.003+0000 7f7b31268700 1 -- 172.21.15.92:0/3903420437 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b2c10f140 con 0x7f7b2c0fed30 2024-05-06T12:20:29.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.003+0000 7f7b137fe700 1 -- 172.21.15.92:0/3903420437 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b2001ca80 con 0x7f7b2c0fed30 2024-05-06T12:20:29.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.003+0000 7f7b31268700 1 -- 172.21.15.92:0/3903420437 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b2c10f680 con 0x7f7b2c0fed30 2024-05-06T12:20:29.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.004+0000 7f7b31268700 1 -- 172.21.15.92:0/3903420437 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b2c05f5c0 con 0x7f7b2c0fed30 2024-05-06T12:20:29.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.005+0000 7f7b137fe700 1 -- 172.21.15.92:0/3903420437 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7b20012a50 con 0x7f7b2c0fed30 2024-05-06T12:20:29.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.005+0000 7f7b137fe700 1 --2- 172.21.15.92:0/3903420437 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7b1406e140 0x7f7b140705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:20:29.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.005+0000 7f7b137fe700 1 -- 172.21.15.92:0/3903420437 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f7b2008f7b0 con 0x7f7b2c0fed30 2024-05-06T12:20:29.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.005+0000 7f7b2affd700 1 --2- 172.21.15.92:0/3903420437 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7b1406e140 0x7f7b140705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:20:29.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.006+0000 7f7b2affd700 1 --2- 172.21.15.92:0/3903420437 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7b1406e140 0x7f7b140705f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7b24007fd0 tx=0x7f7b24009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:20:29.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.010+0000 7f7b137fe700 1 -- 172.21.15.92:0/3903420437 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7b2005c660 con 0x7f7b2c0fed30 2024-05-06T12:20:29.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.226+0000 7f7b31268700 1 -- 172.21.15.92:0/3903420437 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b2c10f960 con 0x7f7b1406e140 2024-05-06T12:20:29.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.228+0000 7f7b137fe700 1 -- 172.21.15.92:0/3903420437 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f7b2c10f960 con 0x7f7b1406e140 2024-05-06T12:20:29.230 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:20:29.230 INFO:teuthology.orchestra.run.smithi092.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb", 2024-05-06T12:20:29.230 INFO:teuthology.orchestra.run.smithi092.stdout: "in_progress": true, 2024-05-06T12:20:29.231 INFO:teuthology.orchestra.run.smithi092.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-06T12:20:29.231 INFO:teuthology.orchestra.run.smithi092.stdout: "services_complete": [ 2024-05-06T12:20:29.231 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr", 2024-05-06T12:20:29.231 INFO:teuthology.orchestra.run.smithi092.stdout: "mon", 2024-05-06T12:20:29.231 INFO:teuthology.orchestra.run.smithi092.stdout: "crash" 2024-05-06T12:20:29.231 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-05-06T12:20:29.231 INFO:teuthology.orchestra.run.smithi092.stdout: "progress": "11/21 daemons upgraded", 2024-05-06T12:20:29.231 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "Currently upgrading osd daemons", 2024-05-06T12:20:29.231 INFO:teuthology.orchestra.run.smithi092.stdout: "is_paused": false 2024-05-06T12:20:29.231 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:20:29.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.230+0000 7f7b31268700 1 -- 172.21.15.92:0/3903420437 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7b1406e140 msgr2=0x7f7b140705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:20:29.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.230+0000 7f7b31268700 1 --2- 172.21.15.92:0/3903420437 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7b1406e140 0x7f7b140705f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7b24007fd0 tx=0x7f7b24009040 comp rx=0 tx=0).stop 2024-05-06T12:20:29.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.230+0000 7f7b31268700 1 -- 172.21.15.92:0/3903420437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b2c0fed30 msgr2=0x7f7b2c10e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:20:29.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.230+0000 7f7b31268700 1 --2- 172.21.15.92:0/3903420437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b2c0fed30 0x7f7b2c10e070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7b20004740 tx=0x7f7b20004d10 comp rx=0 tx=0).stop 2024-05-06T12:20:29.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.230+0000 7f7b31268700 1 -- 172.21.15.92:0/3903420437 shutdown_connections 2024-05-06T12:20:29.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.230+0000 7f7b31268700 1 --2- 172.21.15.92:0/3903420437 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7b1406e140 0x7f7b140705f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:29.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.230+0000 7f7b31268700 1 --2- 172.21.15.92:0/3903420437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b2c0fe3e0 0x7f7b2c10db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:29.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.230+0000 7f7b31268700 1 --2- 172.21.15.92:0/3903420437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b2c0fed30 0x7f7b2c10e070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:29.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.230+0000 7f7b31268700 1 -- 172.21.15.92:0/3903420437 >> 172.21.15.92:0/3903420437 conn(0x7f7b2c0f6b80 msgr2=0x7f7b2c0f8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:20:29.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.230+0000 7f7b31268700 1 -- 172.21.15.92:0/3903420437 shutdown_connections 2024-05-06T12:20:29.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.230+0000 7f7b31268700 1 -- 172.21.15.92:0/3903420437 wait complete. 2024-05-06T12:20:29.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.331+0000 7f3d218a6700 1 -- 172.21.15.92:0/2321427505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d1c066e10 msgr2=0x7f3d1c0ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:20:29.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.331+0000 7f3d218a6700 1 --2- 172.21.15.92:0/2321427505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d1c066e10 0x7f3d1c0ffa60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3d14009ce0 tx=0x7f3d14004d10 comp rx=0 tx=0).stop 2024-05-06T12:20:29.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.332+0000 7f3d218a6700 1 -- 172.21.15.92:0/2321427505 shutdown_connections 2024-05-06T12:20:29.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.332+0000 7f3d218a6700 1 --2- 172.21.15.92:0/2321427505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d1c0fffa0 0x7f3d1c102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:29.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.332+0000 7f3d218a6700 1 --2- 172.21.15.92:0/2321427505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d1c066e10 0x7f3d1c0ffa60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:29.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.332+0000 7f3d218a6700 1 -- 172.21.15.92:0/2321427505 >> 172.21.15.92:0/2321427505 conn(0x7f3d1c0f7430 msgr2=0x7f3d1c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:20:29.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.332+0000 7f3d218a6700 1 -- 172.21.15.92:0/2321427505 shutdown_connections 2024-05-06T12:20:29.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.332+0000 7f3d218a6700 1 -- 172.21.15.92:0/2321427505 wait complete. 2024-05-06T12:20:29.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.332+0000 7f3d218a6700 1 Processor -- start 2024-05-06T12:20:29.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.332+0000 7f3d218a6700 1 -- start start 2024-05-06T12:20:29.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.332+0000 7f3d218a6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d1c066e10 0x7f3d1c10dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:20:29.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.332+0000 7f3d218a6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d1c0fffa0 0x7f3d1c10e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:20:29.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.332+0000 7f3d218a6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c10e620 con 0x7f3d1c0fffa0 2024-05-06T12:20:29.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.332+0000 7f3d218a6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c10e790 con 0x7f3d1c066e10 2024-05-06T12:20:29.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.333+0000 7f3d1a7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d1c0fffa0 0x7f3d1c10e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:20:29.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.333+0000 7f3d1a7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d1c0fffa0 0x7f3d1c10e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34360/0 (socket says 172.21.15.92:34360) 2024-05-06T12:20:29.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.333+0000 7f3d1a7fc700 1 -- 172.21.15.92:0/621616209 learned_addr learned my addr 172.21.15.92:0/621616209 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:20:29.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.333+0000 7f3d1affd700 1 --2- 172.21.15.92:0/621616209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d1c066e10 0x7f3d1c10dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:20:29.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.333+0000 7f3d1a7fc700 1 -- 172.21.15.92:0/621616209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d1c066e10 msgr2=0x7f3d1c10dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:20:29.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.333+0000 7f3d1a7fc700 1 --2- 172.21.15.92:0/621616209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d1c066e10 0x7f3d1c10dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:29.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.333+0000 7f3d1a7fc700 1 -- 172.21.15.92:0/621616209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d14009940 con 0x7f3d1c0fffa0 2024-05-06T12:20:29.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.333+0000 7f3d1a7fc700 1 --2- 172.21.15.92:0/621616209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d1c0fffa0 0x7f3d1c10e020 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3d10002a50 tx=0x7f3d100046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:20:29.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.333+0000 7f3d037fe700 1 -- 172.21.15.92:0/621616209 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d10007cc0 con 0x7f3d1c0fffa0 2024-05-06T12:20:29.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.333+0000 7f3d218a6700 1 -- 172.21.15.92:0/621616209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d1c1131a0 con 0x7f3d1c0fffa0 2024-05-06T12:20:29.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.333+0000 7f3d037fe700 1 -- 172.21.15.92:0/621616209 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3d1000f040 con 0x7f3d1c0fffa0 2024-05-06T12:20:29.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.333+0000 7f3d037fe700 1 -- 172.21.15.92:0/621616209 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d1000db30 con 0x7f3d1c0fffa0 2024-05-06T12:20:29.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.333+0000 7f3d218a6700 1 -- 172.21.15.92:0/621616209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d1c113760 con 0x7f3d1c0fffa0 2024-05-06T12:20:29.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.334+0000 7f3d218a6700 1 -- 172.21.15.92:0/621616209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d1c04c710 con 0x7f3d1c0fffa0 2024-05-06T12:20:29.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.335+0000 7f3d037fe700 1 -- 172.21.15.92:0/621616209 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3d1000dc90 con 0x7f3d1c0fffa0 2024-05-06T12:20:29.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.335+0000 7f3d037fe700 1 --2- 172.21.15.92:0/621616209 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f3d0406e140 0x7f3d040705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:20:29.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.335+0000 7f3d1affd700 1 --2- 172.21.15.92:0/621616209 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f3d0406e140 0x7f3d040705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:20:29.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.335+0000 7f3d037fe700 1 -- 172.21.15.92:0/621616209 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f3d1008d8b0 con 0x7f3d1c0fffa0 2024-05-06T12:20:29.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.336+0000 7f3d1affd700 1 --2- 172.21.15.92:0/621616209 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f3d0406e140 0x7f3d040705f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3d14009470 tx=0x7f3d14004d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:20:29.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.340+0000 7f3d037fe700 1 -- 172.21.15.92:0/621616209 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3d1005a760 con 0x7f3d1c0fffa0 2024-05-06T12:20:29.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:29 smithi190 ceph-mon[79744]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:20:29.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:29 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/626737076' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:20:29.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:29 smithi190 ceph-mon[79744]: pgmap v143: 65 pgs: 10 peering, 11 active+undersized+degraded, 13 active+undersized, 31 active+clean; 323 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s; 38/372 objects degraded (10.215%) 2024-05-06T12:20:29.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:29 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/313092042' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:20:29.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.617+0000 7f3d218a6700 1 -- 172.21.15.92:0/621616209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3d1c1058c0 con 0x7f3d1c0fffa0 2024-05-06T12:20:29.620 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:29 smithi092 ceph-mon[91933]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:20:29.620 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:29 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/626737076' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:20:29.621 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:29 smithi092 ceph-mon[91933]: pgmap v143: 65 pgs: 10 peering, 11 active+undersized+degraded, 13 active+undersized, 31 active+clean; 323 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s; 38/372 objects degraded (10.215%) 2024-05-06T12:20:29.621 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:29 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/313092042' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:20:29.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.618+0000 7f3d037fe700 1 -- 172.21.15.92:0/621616209 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+986 (secure 0 0 0) 0x7f3d1005a2f0 con 0x7f3d1c0fffa0 2024-05-06T12:20:29.621 INFO:teuthology.orchestra.run.smithi092.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; Degraded data redundancy: 38/372 objects degraded (10.215%), 11 pgs degraded 2024-05-06T12:20:29.621 INFO:teuthology.orchestra.run.smithi092.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-06T12:20:29.621 INFO:teuthology.orchestra.run.smithi092.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-06T12:20:29.621 INFO:teuthology.orchestra.run.smithi092.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 38/372 objects degraded (10.215%), 11 pgs degraded 2024-05-06T12:20:29.621 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.4 is active+undersized+degraded, acting [1,0] 2024-05-06T12:20:29.621 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.6 is active+undersized+degraded, acting [1,3] 2024-05-06T12:20:29.622 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.9 is active+undersized+degraded, acting [1,0] 2024-05-06T12:20:29.622 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.a is active+undersized+degraded, acting [1,3] 2024-05-06T12:20:29.622 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.f is active+undersized+degraded, acting [0,5] 2024-05-06T12:20:29.622 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.11 is active+undersized+degraded, acting [3,1] 2024-05-06T12:20:29.622 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.13 is active+undersized+degraded, acting [0,2] 2024-05-06T12:20:29.622 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.18 is active+undersized+degraded, acting [5,3] 2024-05-06T12:20:29.622 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.19 is active+undersized+degraded, acting [0,2] 2024-05-06T12:20:29.622 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.1c is active+undersized+degraded, acting [5,2] 2024-05-06T12:20:29.622 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.1f is active+undersized+degraded, acting [0,3] 2024-05-06T12:20:29.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.620+0000 7f3d218a6700 1 -- 172.21.15.92:0/621616209 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f3d0406e140 msgr2=0x7f3d040705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:20:29.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.620+0000 7f3d218a6700 1 --2- 172.21.15.92:0/621616209 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f3d0406e140 0x7f3d040705f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3d14009470 tx=0x7f3d14004d10 comp rx=0 tx=0).stop 2024-05-06T12:20:29.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.620+0000 7f3d218a6700 1 -- 172.21.15.92:0/621616209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d1c0fffa0 msgr2=0x7f3d1c10e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:20:29.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.620+0000 7f3d218a6700 1 --2- 172.21.15.92:0/621616209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d1c0fffa0 0x7f3d1c10e020 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3d10002a50 tx=0x7f3d100046b0 comp rx=0 tx=0).stop 2024-05-06T12:20:29.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.620+0000 7f3d218a6700 1 -- 172.21.15.92:0/621616209 shutdown_connections 2024-05-06T12:20:29.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.620+0000 7f3d218a6700 1 --2- 172.21.15.92:0/621616209 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f3d0406e140 0x7f3d040705f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:29.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.620+0000 7f3d218a6700 1 --2- 172.21.15.92:0/621616209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d1c066e10 0x7f3d1c10dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:29.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.620+0000 7f3d218a6700 1 --2- 172.21.15.92:0/621616209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d1c0fffa0 0x7f3d1c10e020 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:29.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.620+0000 7f3d218a6700 1 -- 172.21.15.92:0/621616209 >> 172.21.15.92:0/621616209 conn(0x7f3d1c0f7430 msgr2=0x7f3d1c0f9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:20:29.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.620+0000 7f3d218a6700 1 -- 172.21.15.92:0/621616209 shutdown_connections 2024-05-06T12:20:29.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:29.620+0000 7f3d218a6700 1 -- 172.21.15.92:0/621616209 wait complete. 2024-05-06T12:20:30.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:30 smithi190 ceph-mon[79744]: from='client.34362 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:20:30.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:30 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/621616209' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:20:30.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:30 smithi092 ceph-mon[91933]: from='client.34362 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:20:30.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:30 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/621616209' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:20:31.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:31 smithi190 ceph-mon[79744]: Health check update: Degraded data redundancy: 38/372 objects degraded (10.215%), 11 pgs degraded (PG_DEGRADED) 2024-05-06T12:20:31.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:31 smithi190 ceph-mon[79744]: pgmap v144: 65 pgs: 10 peering, 7 active+undersized+degraded, 7 active+undersized, 41 active+clean; 323 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s; 20/372 objects degraded (5.376%) 2024-05-06T12:20:31.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:31 smithi092 ceph-mon[91933]: Health check update: Degraded data redundancy: 38/372 objects degraded (10.215%), 11 pgs degraded (PG_DEGRADED) 2024-05-06T12:20:31.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:31 smithi092 ceph-mon[91933]: pgmap v144: 65 pgs: 10 peering, 7 active+undersized+degraded, 7 active+undersized, 41 active+clean; 323 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s; 20/372 objects degraded (5.376%) 2024-05-06T12:20:33.226 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:32 smithi092 ceph-mon[91933]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 20/372 objects degraded (5.376%), 7 pgs degraded) 2024-05-06T12:20:33.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:32 smithi190 ceph-mon[79744]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 20/372 objects degraded (5.376%), 7 pgs degraded) 2024-05-06T12:20:34.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:33 smithi092 ceph-mon[91933]: pgmap v145: 65 pgs: 65 active+clean; 323 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:20:34.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:33 smithi190 ceph-mon[79744]: pgmap v145: 65 pgs: 65 active+clean; 323 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:20:36.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:35 smithi092 ceph-mon[91933]: pgmap v146: 65 pgs: 65 active+clean; 323 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 3 op/s 2024-05-06T12:20:36.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:35 smithi190 ceph-mon[79744]: pgmap v146: 65 pgs: 65 active+clean; 323 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 3 op/s 2024-05-06T12:20:38.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:37 smithi092 ceph-mon[91933]: pgmap v147: 65 pgs: 65 active+clean; 323 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-06T12:20:38.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:37 smithi190 ceph-mon[79744]: pgmap v147: 65 pgs: 65 active+clean; 323 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-06T12:20:40.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:39 smithi092 ceph-mon[91933]: pgmap v148: 65 pgs: 65 active+clean; 323 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-05-06T12:20:40.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:39 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:40.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:39 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:20:40.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:39 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:40.327 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:39 smithi190 ceph-mon[79744]: pgmap v148: 65 pgs: 65 active+clean; 323 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-05-06T12:20:40.328 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:39 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:40.328 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:39 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:20:40.328 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:39 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:42.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:41 smithi092 ceph-mon[91933]: pgmap v149: 65 pgs: 65 active+clean; 323 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:20:42.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:41 smithi190 ceph-mon[79744]: pgmap v149: 65 pgs: 65 active+clean; 323 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:20:44.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:43 smithi190 ceph-mon[79744]: pgmap v150: 65 pgs: 65 active+clean; 323 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:20:44.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:43 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-06T12:20:44.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:43 smithi190 ceph-mon[79744]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-06T12:20:44.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:43 smithi190 ceph-mon[79744]: Upgrade: osd.5 is safe to restart 2024-05-06T12:20:44.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:43 smithi092 ceph-mon[91933]: pgmap v150: 65 pgs: 65 active+clean; 323 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:20:44.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:43 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-06T12:20:44.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:43 smithi092 ceph-mon[91933]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-06T12:20:44.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:43 smithi092 ceph-mon[91933]: Upgrade: osd.5 is safe to restart 2024-05-06T12:20:45.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:45 smithi190 ceph-mon[79744]: Upgrade: Updating osd.5 2024-05-06T12:20:45.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:45 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:45.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:45 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-06T12:20:45.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:45 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:20:45.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:45 smithi190 ceph-mon[79744]: Deploying daemon osd.5 on smithi190 2024-05-06T12:20:45.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:45 smithi190 ceph-mon[79744]: pgmap v151: 65 pgs: 65 active+clean; 323 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:20:45.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:45 smithi092 ceph-mon[91933]: Upgrade: Updating osd.5 2024-05-06T12:20:45.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:45 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:45.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:45 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-06T12:20:45.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:45 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:20:45.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:45 smithi092 ceph-mon[91933]: Deploying daemon osd.5 on smithi190 2024-05-06T12:20:45.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:45 smithi092 ceph-mon[91933]: pgmap v151: 65 pgs: 65 active+clean; 323 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:20:47.957 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:47 smithi190 ceph-mon[79744]: pgmap v152: 65 pgs: 65 active+clean; 323 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:20:47.957 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:47 smithi190 systemd[1]: Stopping Ceph osd.5 for aee1d912-0ba0-11ef-bc97-c7b262605968... 2024-05-06T12:20:48.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:47 smithi092 ceph-mon[91933]: pgmap v152: 65 pgs: 65 active+clean; 323 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:20:48.348 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:47 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[52689]: 2024-05-06T12:20:47.953+0000 7fbd7dcf4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-06T12:20:48.348 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:47 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[52689]: 2024-05-06T12:20:47.953+0000 7fbd7dcf4700 -1 osd.5 75 *** Got signal Terminated *** 2024-05-06T12:20:48.348 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:47 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[52689]: 2024-05-06T12:20:47.953+0000 7fbd7dcf4700 -1 osd.5 75 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-06T12:20:49.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:48 smithi092 ceph-mon[91933]: osd.5 marked itself down and dead 2024-05-06T12:20:49.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:48 smithi190 ceph-mon[79744]: osd.5 marked itself down and dead 2024-05-06T12:20:49.640 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:49 smithi190 podman[99461]: 2024-05-06 12:20:49.355412971 +0000 UTC m=+1.505422189 container died e7452c6b5cecd247b5ebec5c13e0d0bbd9ef14bf24907d4af2df82af856aa2f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:1a0b7fd3cefd2c5baddf682493f92b4e27596f3c0ff27b02fdccd2891f56859f, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, RELEASE=quincy-e81df42, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-06T12:20:49.905 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:49 smithi190 podman[99461]: 2024-05-06 12:20:49.720681165 +0000 UTC m=+1.870690375 container cleanup e7452c6b5cecd247b5ebec5c13e0d0bbd9ef14bf24907d4af2df82af856aa2f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:1a0b7fd3cefd2c5baddf682493f92b4e27596f3c0ff27b02fdccd2891f56859f, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=quincy-e81df42, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-06T12:20:49.905 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:49 smithi190 bash[99461]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5 2024-05-06T12:20:50.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:49 smithi092 ceph-mon[91933]: pgmap v153: 65 pgs: 65 active+clean; 323 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:20:50.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:49 smithi092 ceph-mon[91933]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-06T12:20:50.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:49 smithi092 ceph-mon[91933]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-05-06T12:20:50.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:49 smithi092 ceph-mon[91933]: osdmap e76: 6 total, 5 up, 6 in 2024-05-06T12:20:50.338 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:49 smithi190 ceph-mon[79744]: pgmap v153: 65 pgs: 65 active+clean; 323 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:20:50.338 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:49 smithi190 ceph-mon[79744]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-06T12:20:50.338 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:49 smithi190 ceph-mon[79744]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-05-06T12:20:50.338 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:49 smithi190 ceph-mon[79744]: osdmap e76: 6 total, 5 up, 6 in 2024-05-06T12:20:50.338 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:50 smithi190 podman[99480]: 2024-05-06 12:20:50.005506771 +0000 UTC m=+0.653311291 container remove e7452c6b5cecd247b5ebec5c13e0d0bbd9ef14bf24907d4af2df82af856aa2f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:1a0b7fd3cefd2c5baddf682493f92b4e27596f3c0ff27b02fdccd2891f56859f, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=quincy-e81df42, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-06T12:20:50.598 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:50 smithi190 podman[99573]: 2024-05-06 12:20:50.239303657 +0000 UTC m=+0.017759660 image pull e4ce8ad5d0c4b9ea60bc5f6b8b0fdb5efbe59bf7771fb0920fc2200e6ae546b0 quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb 2024-05-06T12:20:50.598 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:50 smithi190 podman[99573]: 2024-05-06 12:20:50.356367734 +0000 UTC m=+0.134823709 container create 92dd2d24031600c91fa3840409bb7cd06b2ba370b9bdffc90b0b927ab510f11f (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5-deactivate, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=squid-70ce953, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-06T12:20:50.598 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:50 smithi190 podman[99573]: 2024-05-06 12:20:50.567234079 +0000 UTC m=+0.345690059 container init 92dd2d24031600c91fa3840409bb7cd06b2ba370b9bdffc90b0b927ab510f11f (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-70ce953, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-05-06T12:20:50.598 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:50 smithi190 podman[99573]: 2024-05-06 12:20:50.570434968 +0000 UTC m=+0.348890943 container start 92dd2d24031600c91fa3840409bb7cd06b2ba370b9bdffc90b0b927ab510f11f (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-70ce953, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5) 2024-05-06T12:20:51.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:50 smithi190 ceph-mon[79744]: osdmap e77: 6 total, 5 up, 6 in 2024-05-06T12:20:51.099 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:50 smithi190 podman[99573]: 2024-05-06 12:20:50.6672574 +0000 UTC m=+0.445713379 container attach 92dd2d24031600c91fa3840409bb7cd06b2ba370b9bdffc90b0b927ab510f11f (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=squid-70ce953, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5) 2024-05-06T12:20:51.099 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:50 smithi190 podman[99573]: 2024-05-06 12:20:50.859549036 +0000 UTC m=+0.638005017 container died 92dd2d24031600c91fa3840409bb7cd06b2ba370b9bdffc90b0b927ab510f11f (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-70ce953, org.label-schema.schema-version=1.0) 2024-05-06T12:20:51.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:50 smithi092 ceph-mon[91933]: osdmap e77: 6 total, 5 up, 6 in 2024-05-06T12:20:51.457 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:51 smithi190 podman[99599]: 2024-05-06 12:20:51.429191984 +0000 UTC m=+0.564311739 container remove 92dd2d24031600c91fa3840409bb7cd06b2ba370b9bdffc90b0b927ab510f11f (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-70ce953, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-06T12:20:51.457 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:51 smithi190 systemd[1]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@osd.5.service: Deactivated successfully. 2024-05-06T12:20:51.457 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:51 smithi190 systemd[1]: Stopped Ceph osd.5 for aee1d912-0ba0-11ef-bc97-c7b262605968. 2024-05-06T12:20:51.457 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:51 smithi190 systemd[1]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@osd.5.service: Consumed 51.839s CPU time. 2024-05-06T12:20:51.850 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:51 smithi190 systemd[1]: Starting Ceph osd.5 for aee1d912-0ba0-11ef-bc97-c7b262605968... 2024-05-06T12:20:52.209 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:51 smithi190 ceph-mon[79744]: pgmap v156: 65 pgs: 3 peering, 11 stale+active+clean, 51 active+clean; 323 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-06T12:20:52.209 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:51 smithi190 ceph-mon[79744]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-05-06T12:20:52.210 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:51 smithi190 podman[99702]: 2024-05-06 12:20:51.76731031 +0000 UTC m=+0.018359615 image pull e4ce8ad5d0c4b9ea60bc5f6b8b0fdb5efbe59bf7771fb0920fc2200e6ae546b0 quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb 2024-05-06T12:20:52.210 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:51 smithi190 podman[99702]: 2024-05-06 12:20:51.880260941 +0000 UTC m=+0.131310231 container create 2eb6c467629698fa6d9ae670dd578d4f63f2e4f6adbfb6541175b527c5e01949 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5-activate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-70ce953) 2024-05-06T12:20:52.210 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:52 smithi190 podman[99702]: 2024-05-06 12:20:52.099557499 +0000 UTC m=+0.350606790 container init 2eb6c467629698fa6d9ae670dd578d4f63f2e4f6adbfb6541175b527c5e01949 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5-activate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-70ce953, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-06T12:20:52.210 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:52 smithi190 podman[99702]: 2024-05-06 12:20:52.102780544 +0000 UTC m=+0.353829838 container start 2eb6c467629698fa6d9ae670dd578d4f63f2e4f6adbfb6541175b527c5e01949 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5-activate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-70ce953, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-06T12:20:52.210 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:52 smithi190 podman[99702]: 2024-05-06 12:20:52.207955878 +0000 UTC m=+0.459005180 container attach 2eb6c467629698fa6d9ae670dd578d4f63f2e4f6adbfb6541175b527c5e01949 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5-activate, GIT_CLEAN=True, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-70ce953, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-06T12:20:52.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:51 smithi092 ceph-mon[91933]: pgmap v156: 65 pgs: 3 peering, 11 stale+active+clean, 51 active+clean; 323 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-06T12:20:52.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:51 smithi092 ceph-mon[91933]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-05-06T12:20:52.848 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:52 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5-activate[99717]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-06T12:20:52.848 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:52 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5-activate[99717]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-05-06T12:20:52.848 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:52 smithi190 bash[99702]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-06T12:20:52.848 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:52 smithi190 bash[99702]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-05-06T12:20:52.849 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:52 smithi190 bash[99702]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-05-06T12:20:52.849 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:52 smithi190 bash[99702]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-05-06T12:20:52.849 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:52 smithi190 bash[99702]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-05-06T12:20:52.849 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:52 smithi190 bash[99702]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-06T12:20:52.849 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:52 smithi190 bash[99702]: --> ceph-volume raw activate successful for osd ID: 5 2024-05-06T12:20:52.849 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:52 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5-activate[99717]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-05-06T12:20:52.849 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:52 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5-activate[99717]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-05-06T12:20:52.849 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:52 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5-activate[99717]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-05-06T12:20:52.849 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:52 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5-activate[99717]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-06T12:20:52.849 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:52 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5-activate[99717]: --> ceph-volume raw activate successful for osd ID: 5 2024-05-06T12:20:52.849 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:52 smithi190 podman[99702]: 2024-05-06 12:20:52.570770754 +0000 UTC m=+0.821820037 container died 2eb6c467629698fa6d9ae670dd578d4f63f2e4f6adbfb6541175b527c5e01949 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5-activate, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-70ce953, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-06T12:20:53.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:52 smithi092 ceph-mon[91933]: Health check failed: Degraded data redundancy: 56/372 objects degraded (15.054%), 14 pgs degraded (PG_DEGRADED) 2024-05-06T12:20:53.238 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:52 smithi190 ceph-mon[79744]: Health check failed: Degraded data redundancy: 56/372 objects degraded (15.054%), 14 pgs degraded (PG_DEGRADED) 2024-05-06T12:20:53.560 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:53 smithi190 podman[99847]: 2024-05-06 12:20:53.237122526 +0000 UTC m=+0.661147304 container remove 2eb6c467629698fa6d9ae670dd578d4f63f2e4f6adbfb6541175b527c5e01949 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5-activate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-70ce953, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-06T12:20:53.560 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:53 smithi190 podman[99908]: 2024-05-06 12:20:53.356395818 +0000 UTC m=+0.018223238 image pull e4ce8ad5d0c4b9ea60bc5f6b8b0fdb5efbe59bf7771fb0920fc2200e6ae546b0 quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb 2024-05-06T12:20:53.560 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:53 smithi190 podman[99908]: 2024-05-06 12:20:53.454631269 +0000 UTC m=+0.116458676 container create 067bc34cdfb18f8ad7b176d1c3cf140c1b22c4f15aff4f6de93ac05bf9fa621b (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5, RELEASE=squid-70ce953, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-06T12:20:53.848 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:53 smithi190 podman[99908]: 2024-05-06 12:20:53.640323138 +0000 UTC m=+0.302150545 container init 067bc34cdfb18f8ad7b176d1c3cf140c1b22c4f15aff4f6de93ac05bf9fa621b (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, RELEASE=squid-70ce953, org.label-schema.license=GPLv2) 2024-05-06T12:20:53.848 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:53 smithi190 podman[99908]: 2024-05-06 12:20:53.643354364 +0000 UTC m=+0.305181774 container start 067bc34cdfb18f8ad7b176d1c3cf140c1b22c4f15aff4f6de93ac05bf9fa621b (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , RELEASE=squid-70ce953, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430) 2024-05-06T12:20:53.848 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:53 smithi190 ceph-osd[99927]: -- 172.21.15.190:0/3496016311 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5632478cd680 con 0x563247897000 2024-05-06T12:20:53.848 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:53 smithi190 bash[99908]: 067bc34cdfb18f8ad7b176d1c3cf140c1b22c4f15aff4f6de93ac05bf9fa621b 2024-05-06T12:20:53.848 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:53 smithi190 systemd[1]: Started Ceph osd.5 for aee1d912-0ba0-11ef-bc97-c7b262605968. 2024-05-06T12:20:54.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:53 smithi092 ceph-mon[91933]: pgmap v157: 65 pgs: 14 active+undersized+degraded, 3 peering, 11 active+undersized, 37 active+clean; 323 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 56/372 objects degraded (15.054%) 2024-05-06T12:20:54.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:53 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:54.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:53 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:54.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:53 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:20:54.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:53 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:54.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:53 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:20:54.348 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:54 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:20:54.186+0000 7f33b95cd7c0 -1 Falling back to public interface 2024-05-06T12:20:54.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:53 smithi190 ceph-mon[79744]: pgmap v157: 65 pgs: 14 active+undersized+degraded, 3 peering, 11 active+undersized, 37 active+clean; 323 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 56/372 objects degraded (15.054%) 2024-05-06T12:20:54.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:53 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:54.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:53 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:54.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:53 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:20:54.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:53 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:54.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:53 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:20:56.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:55 smithi092 ceph-mon[91933]: pgmap v158: 65 pgs: 14 active+undersized+degraded, 3 peering, 11 active+undersized, 37 active+clean; 323 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 56/372 objects degraded (15.054%) 2024-05-06T12:20:56.309 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:55 smithi190 ceph-mon[79744]: pgmap v158: 65 pgs: 14 active+undersized+degraded, 3 peering, 11 active+undersized, 37 active+clean; 323 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 56/372 objects degraded (15.054%) 2024-05-06T12:20:57.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:57 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:57.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:57 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:57.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:57 smithi092 ceph-mon[91933]: pgmap v159: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 323 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 56/372 objects degraded (15.054%) 2024-05-06T12:20:58.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:57 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:58.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:57 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:58.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:57 smithi190 ceph-mon[79744]: pgmap v159: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 323 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 56/372 objects degraded (15.054%) 2024-05-06T12:20:58.654 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:58 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:20:58.399+0000 7f33b95cd7c0 -1 osd.5 75 log_to_monitors true 2024-05-06T12:20:58.917 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:58 smithi092 ceph-mon[91933]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-05-06T12:20:58.917 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:58 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:58.917 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:58 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:58.917 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:58 smithi092 ceph-mon[91933]: from='osd.5 [v2:172.21.15.190:6816/10700099,v1:172.21.15.190:6817/10700099]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-06T12:20:58.917 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:58 smithi092 ceph-mon[91933]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-06T12:20:59.098 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:20:58 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:20:58.818+0000 7f33aaa24700 -1 osd.5 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-06T12:20:59.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:58 smithi190 ceph-mon[79744]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-05-06T12:20:59.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:58 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:59.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:58 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:20:59.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:58 smithi190 ceph-mon[79744]: from='osd.5 [v2:172.21.15.190:6816/10700099,v1:172.21.15.190:6817/10700099]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-06T12:20:59.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:58 smithi190 ceph-mon[79744]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-06T12:20:59.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.724+0000 7fe16fe2e700 1 -- 172.21.15.92:0/2611695861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1680fcf70 msgr2=0x7fe1680fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:20:59.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.724+0000 7fe16fe2e700 1 --2- 172.21.15.92:0/2611695861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1680fcf70 0x7fe1680fd3e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe158004940 tx=0x7fe158009e30 comp rx=0 tx=0).stop 2024-05-06T12:20:59.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.724+0000 7fe16fe2e700 1 -- 172.21.15.92:0/2611695861 shutdown_connections 2024-05-06T12:20:59.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.724+0000 7fe16fe2e700 1 --2- 172.21.15.92:0/2611695861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1680fcf70 0x7fe1680fd3e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:59.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.724+0000 7fe16fe2e700 1 --2- 172.21.15.92:0/2611695861 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1680fbcc0 0x7fe1680fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:59.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.725+0000 7fe16fe2e700 1 -- 172.21.15.92:0/2611695861 >> 172.21.15.92:0/2611695861 conn(0x7fe1680f7440 msgr2=0x7fe1680f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:20:59.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.725+0000 7fe16fe2e700 1 -- 172.21.15.92:0/2611695861 shutdown_connections 2024-05-06T12:20:59.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.725+0000 7fe16fe2e700 1 -- 172.21.15.92:0/2611695861 wait complete. 2024-05-06T12:20:59.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.725+0000 7fe16fe2e700 1 Processor -- start 2024-05-06T12:20:59.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.725+0000 7fe16fe2e700 1 -- start start 2024-05-06T12:20:59.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.725+0000 7fe16fe2e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1680fbcc0 0x7fe16810c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:20:59.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.725+0000 7fe16fe2e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1680fcf70 0x7fe16810cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:20:59.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.725+0000 7fe16fe2e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16810d3b0 con 0x7fe1680fcf70 2024-05-06T12:20:59.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.725+0000 7fe16fe2e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16810d520 con 0x7fe1680fbcc0 2024-05-06T12:20:59.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.726+0000 7fe16d3c9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1680fcf70 0x7fe16810cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:20:59.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.726+0000 7fe16d3c9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1680fcf70 0x7fe16810cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43546/0 (socket says 172.21.15.92:43546) 2024-05-06T12:20:59.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.726+0000 7fe16d3c9700 1 -- 172.21.15.92:0/2560358605 learned_addr learned my addr 172.21.15.92:0/2560358605 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:20:59.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.726+0000 7fe16dbca700 1 --2- 172.21.15.92:0/2560358605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1680fbcc0 0x7fe16810c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:20:59.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.726+0000 7fe16d3c9700 1 -- 172.21.15.92:0/2560358605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1680fbcc0 msgr2=0x7fe16810c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:20:59.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.726+0000 7fe16d3c9700 1 --2- 172.21.15.92:0/2560358605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1680fbcc0 0x7fe16810c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:59.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.726+0000 7fe16d3c9700 1 -- 172.21.15.92:0/2560358605 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe158005040 con 0x7fe1680fcf70 2024-05-06T12:20:59.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.726+0000 7fe16d3c9700 1 --2- 172.21.15.92:0/2560358605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1680fcf70 0x7fe16810cdb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe158009150 tx=0x7fe158009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:20:59.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.726+0000 7fe15e7fc700 1 -- 172.21.15.92:0/2560358605 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe15800e040 con 0x7fe1680fcf70 2024-05-06T12:20:59.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.726+0000 7fe15e7fc700 1 -- 172.21.15.92:0/2560358605 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe158004290 con 0x7fe1680fcf70 2024-05-06T12:20:59.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.726+0000 7fe15e7fc700 1 -- 172.21.15.92:0/2560358605 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe15801c630 con 0x7fe1680fcf70 2024-05-06T12:20:59.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.726+0000 7fe16fe2e700 1 -- 172.21.15.92:0/2560358605 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe168066250 con 0x7fe1680fcf70 2024-05-06T12:20:59.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.726+0000 7fe16fe2e700 1 -- 172.21.15.92:0/2560358605 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe168066790 con 0x7fe1680fcf70 2024-05-06T12:20:59.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.727+0000 7fe16fe2e700 1 -- 172.21.15.92:0/2560358605 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe16804c710 con 0x7fe1680fcf70 2024-05-06T12:20:59.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.728+0000 7fe15e7fc700 1 -- 172.21.15.92:0/2560358605 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe15800bbf0 con 0x7fe1680fcf70 2024-05-06T12:20:59.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.728+0000 7fe15e7fc700 1 --2- 172.21.15.92:0/2560358605 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fe15406e140 0x7fe1540705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:20:59.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.728+0000 7fe15e7fc700 1 -- 172.21.15.92:0/2560358605 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe15808e6e0 con 0x7fe1680fcf70 2024-05-06T12:20:59.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.728+0000 7fe16dbca700 1 --2- 172.21.15.92:0/2560358605 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fe15406e140 0x7fe1540705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:20:59.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.729+0000 7fe16dbca700 1 --2- 172.21.15.92:0/2560358605 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fe15406e140 0x7fe1540705f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe164004f40 tx=0x7fe16400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:20:59.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.733+0000 7fe15e7fc700 1 -- 172.21.15.92:0/2560358605 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe15805b610 con 0x7fe1680fcf70 2024-05-06T12:20:59.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.964+0000 7fe16fe2e700 1 -- 172.21.15.92:0/2560358605 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe168066cb0 con 0x7fe15406e140 2024-05-06T12:20:59.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.965+0000 7fe15e7fc700 1 -- 172.21.15.92:0/2560358605 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe168066cb0 con 0x7fe15406e140 2024-05-06T12:20:59.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.967+0000 7fe16fe2e700 1 -- 172.21.15.92:0/2560358605 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fe15406e140 msgr2=0x7fe1540705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:20:59.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.967+0000 7fe16fe2e700 1 --2- 172.21.15.92:0/2560358605 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fe15406e140 0x7fe1540705f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe164004f40 tx=0x7fe16400b040 comp rx=0 tx=0).stop 2024-05-06T12:20:59.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.967+0000 7fe16fe2e700 1 -- 172.21.15.92:0/2560358605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1680fcf70 msgr2=0x7fe16810cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:20:59.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.967+0000 7fe16fe2e700 1 --2- 172.21.15.92:0/2560358605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1680fcf70 0x7fe16810cdb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe158009150 tx=0x7fe158009e30 comp rx=0 tx=0).stop 2024-05-06T12:20:59.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.967+0000 7fe16fe2e700 1 -- 172.21.15.92:0/2560358605 shutdown_connections 2024-05-06T12:20:59.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.967+0000 7fe16fe2e700 1 --2- 172.21.15.92:0/2560358605 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fe15406e140 0x7fe1540705f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:59.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.967+0000 7fe16fe2e700 1 --2- 172.21.15.92:0/2560358605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1680fbcc0 0x7fe16810c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:59.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.967+0000 7fe16fe2e700 1 --2- 172.21.15.92:0/2560358605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1680fcf70 0x7fe16810cdb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:20:59.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.967+0000 7fe16fe2e700 1 -- 172.21.15.92:0/2560358605 >> 172.21.15.92:0/2560358605 conn(0x7fe1680f7440 msgr2=0x7fe1680f8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:20:59.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.967+0000 7fe16fe2e700 1 -- 172.21.15.92:0/2560358605 shutdown_connections 2024-05-06T12:20:59.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:20:59.967+0000 7fe16fe2e700 1 -- 172.21.15.92:0/2560358605 wait complete. 2024-05-06T12:20:59.980 INFO:teuthology.orchestra.run.smithi092.stdout:true 2024-05-06T12:21:00.072 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.069+0000 7f0e62037700 1 -- 172.21.15.92:0/1486182588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e5c0fbcc0 msgr2=0x7f0e5c0fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:00.072 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.069+0000 7f0e62037700 1 --2- 172.21.15.92:0/1486182588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e5c0fbcc0 0x7f0e5c0fc0d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0e54009ce0 tx=0x7f0e54004d10 comp rx=0 tx=0).stop 2024-05-06T12:21:00.072 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.069+0000 7f0e62037700 1 -- 172.21.15.92:0/1486182588 shutdown_connections 2024-05-06T12:21:00.072 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.069+0000 7f0e62037700 1 --2- 172.21.15.92:0/1486182588 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e5c0fcf70 0x7f0e5c0fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:00.072 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.069+0000 7f0e62037700 1 --2- 172.21.15.92:0/1486182588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e5c0fbcc0 0x7f0e5c0fc0d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:00.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.069+0000 7f0e62037700 1 -- 172.21.15.92:0/1486182588 >> 172.21.15.92:0/1486182588 conn(0x7f0e5c0f7440 msgr2=0x7f0e5c0f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:21:00.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.069+0000 7f0e62037700 1 -- 172.21.15.92:0/1486182588 shutdown_connections 2024-05-06T12:21:00.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.069+0000 7f0e62037700 1 -- 172.21.15.92:0/1486182588 wait complete. 2024-05-06T12:21:00.073 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:59 smithi092 ceph-mon[91933]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-06T12:21:00.073 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:59 smithi092 ceph-mon[91933]: osdmap e78: 6 total, 5 up, 6 in 2024-05-06T12:21:00.074 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:59 smithi092 ceph-mon[91933]: from='osd.5 [v2:172.21.15.190:6816/10700099,v1:172.21.15.190:6817/10700099]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-05-06T12:21:00.074 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:59 smithi092 ceph-mon[91933]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-05-06T12:21:00.074 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:20:59 smithi092 ceph-mon[91933]: pgmap v161: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 323 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 56/372 objects degraded (15.054%) 2024-05-06T12:21:00.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.070+0000 7f0e62037700 1 Processor -- start 2024-05-06T12:21:00.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.070+0000 7f0e62037700 1 -- start start 2024-05-06T12:21:00.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.070+0000 7f0e62037700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e5c0fbcc0 0x7f0e5c066cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:00.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.070+0000 7f0e62037700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e5c0fcf70 0x7f0e5c1128d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:00.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.070+0000 7f0e62037700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c112e10 con 0x7f0e5c0fcf70 2024-05-06T12:21:00.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.070+0000 7f0e62037700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c112f50 con 0x7f0e5c0fbcc0 2024-05-06T12:21:00.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.070+0000 7f0e5affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e5c0fcf70 0x7f0e5c1128d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:00.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.070+0000 7f0e5b7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e5c0fbcc0 0x7f0e5c066cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:00.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.070+0000 7f0e5affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e5c0fcf70 0x7f0e5c1128d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43556/0 (socket says 172.21.15.92:43556) 2024-05-06T12:21:00.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.070+0000 7f0e5b7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e5c0fbcc0 0x7f0e5c066cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.92:58016/0 (socket says 172.21.15.92:58016) 2024-05-06T12:21:00.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.070+0000 7f0e5affd700 1 -- 172.21.15.92:0/3726265700 learned_addr learned my addr 172.21.15.92:0/3726265700 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:21:00.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.071+0000 7f0e5affd700 1 -- 172.21.15.92:0/3726265700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e5c0fbcc0 msgr2=0x7f0e5c066cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:00.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.071+0000 7f0e5affd700 1 --2- 172.21.15.92:0/3726265700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e5c0fbcc0 0x7f0e5c066cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:00.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.071+0000 7f0e5affd700 1 -- 172.21.15.92:0/3726265700 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e54009940 con 0x7f0e5c0fcf70 2024-05-06T12:21:00.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.071+0000 7f0e5affd700 1 --2- 172.21.15.92:0/3726265700 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e5c0fcf70 0x7f0e5c1128d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0e44002a50 tx=0x7f0e440046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:21:00.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.071+0000 7f0e53fff700 1 -- 172.21.15.92:0/3726265700 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e44007cc0 con 0x7f0e5c0fcf70 2024-05-06T12:21:00.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.071+0000 7f0e62037700 1 -- 172.21.15.92:0/3726265700 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e5c1131e0 con 0x7f0e5c0fcf70 2024-05-06T12:21:00.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.071+0000 7f0e53fff700 1 -- 172.21.15.92:0/3726265700 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e4400f040 con 0x7f0e5c0fcf70 2024-05-06T12:21:00.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.071+0000 7f0e53fff700 1 -- 172.21.15.92:0/3726265700 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e4400d9d0 con 0x7f0e5c0fcf70 2024-05-06T12:21:00.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.071+0000 7f0e62037700 1 -- 172.21.15.92:0/3726265700 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e5c1137a0 con 0x7f0e5c0fcf70 2024-05-06T12:21:00.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.072+0000 7f0e62037700 1 -- 172.21.15.92:0/3726265700 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e5c04c710 con 0x7f0e5c0fcf70 2024-05-06T12:21:00.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.073+0000 7f0e53fff700 1 -- 172.21.15.92:0/3726265700 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0e4400db50 con 0x7f0e5c0fcf70 2024-05-06T12:21:00.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.073+0000 7f0e53fff700 1 --2- 172.21.15.92:0/3726265700 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f0e4806e140 0x7f0e480705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:00.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.073+0000 7f0e53fff700 1 -- 172.21.15.92:0/3726265700 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f0e4408d7f0 con 0x7f0e5c0fcf70 2024-05-06T12:21:00.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.073+0000 7f0e5b7fe700 1 --2- 172.21.15.92:0/3726265700 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f0e4806e140 0x7f0e480705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:00.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.074+0000 7f0e5b7fe700 1 --2- 172.21.15.92:0/3726265700 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f0e4806e140 0x7f0e480705f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400b740 tx=0x7f0e54004d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:21:00.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.078+0000 7f0e53fff700 1 -- 172.21.15.92:0/3726265700 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0e4405a720 con 0x7f0e5c0fcf70 2024-05-06T12:21:00.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:59 smithi190 ceph-mon[79744]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-06T12:21:00.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:59 smithi190 ceph-mon[79744]: osdmap e78: 6 total, 5 up, 6 in 2024-05-06T12:21:00.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:59 smithi190 ceph-mon[79744]: from='osd.5 [v2:172.21.15.190:6816/10700099,v1:172.21.15.190:6817/10700099]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-05-06T12:21:00.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:59 smithi190 ceph-mon[79744]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-05-06T12:21:00.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:20:59 smithi190 ceph-mon[79744]: pgmap v161: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 323 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 56/372 objects degraded (15.054%) 2024-05-06T12:21:00.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.298+0000 7f0e62037700 1 -- 172.21.15.92:0/3726265700 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e5c1012d0 con 0x7f0e4806e140 2024-05-06T12:21:00.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.300+0000 7f0e53fff700 1 -- 172.21.15.92:0/3726265700 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0e5c1012d0 con 0x7f0e4806e140 2024-05-06T12:21:00.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.302+0000 7f0e62037700 1 -- 172.21.15.92:0/3726265700 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f0e4806e140 msgr2=0x7f0e480705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:00.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.302+0000 7f0e62037700 1 --2- 172.21.15.92:0/3726265700 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f0e4806e140 0x7f0e480705f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400b740 tx=0x7f0e54004d10 comp rx=0 tx=0).stop 2024-05-06T12:21:00.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.302+0000 7f0e62037700 1 -- 172.21.15.92:0/3726265700 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e5c0fcf70 msgr2=0x7f0e5c1128d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:00.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.302+0000 7f0e62037700 1 --2- 172.21.15.92:0/3726265700 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e5c0fcf70 0x7f0e5c1128d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0e44002a50 tx=0x7f0e440046b0 comp rx=0 tx=0).stop 2024-05-06T12:21:00.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.302+0000 7f0e62037700 1 -- 172.21.15.92:0/3726265700 shutdown_connections 2024-05-06T12:21:00.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.302+0000 7f0e62037700 1 --2- 172.21.15.92:0/3726265700 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f0e4806e140 0x7f0e480705f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:00.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.302+0000 7f0e62037700 1 --2- 172.21.15.92:0/3726265700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e5c0fbcc0 0x7f0e5c066cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:00.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.302+0000 7f0e62037700 1 --2- 172.21.15.92:0/3726265700 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e5c0fcf70 0x7f0e5c1128d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:00.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.302+0000 7f0e62037700 1 -- 172.21.15.92:0/3726265700 >> 172.21.15.92:0/3726265700 conn(0x7f0e5c0f7440 msgr2=0x7f0e5c0f8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:21:00.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.302+0000 7f0e62037700 1 -- 172.21.15.92:0/3726265700 shutdown_connections 2024-05-06T12:21:00.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.302+0000 7f0e62037700 1 -- 172.21.15.92:0/3726265700 wait complete. 2024-05-06T12:21:00.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.405+0000 7f99848df700 1 -- 172.21.15.92:0/3958149479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99800fdfd0 msgr2=0x7f99800fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:00.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.405+0000 7f99848df700 1 --2- 172.21.15.92:0/3958149479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99800fdfd0 0x7f99800fe440 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9974004940 tx=0x7f9974009e30 comp rx=0 tx=0).stop 2024-05-06T12:21:00.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.405+0000 7f99848df700 1 -- 172.21.15.92:0/3958149479 shutdown_connections 2024-05-06T12:21:00.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.405+0000 7f99848df700 1 --2- 172.21.15.92:0/3958149479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99800fdfd0 0x7f99800fe440 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:00.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.405+0000 7f99848df700 1 --2- 172.21.15.92:0/3958149479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99800fce60 0x7f99800fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:00.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.405+0000 7f99848df700 1 -- 172.21.15.92:0/3958149479 >> 172.21.15.92:0/3958149479 conn(0x7f99800f8540 msgr2=0x7f99800fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:21:00.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.406+0000 7f99848df700 1 -- 172.21.15.92:0/3958149479 shutdown_connections 2024-05-06T12:21:00.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.406+0000 7f99848df700 1 -- 172.21.15.92:0/3958149479 wait complete. 2024-05-06T12:21:00.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.406+0000 7f99848df700 1 Processor -- start 2024-05-06T12:21:00.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.406+0000 7f99848df700 1 -- start start 2024-05-06T12:21:00.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.406+0000 7f99848df700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99800fce60 0x7f998010d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:00.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.406+0000 7f99848df700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99800fdfd0 0x7f998010de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:00.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.406+0000 7f99848df700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998010e400 con 0x7f99800fdfd0 2024-05-06T12:21:00.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.406+0000 7f99848df700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998010e570 con 0x7f99800fce60 2024-05-06T12:21:00.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.406+0000 7f997dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99800fdfd0 0x7f998010de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:00.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.406+0000 7f997dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99800fdfd0 0x7f998010de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43578/0 (socket says 172.21.15.92:43578) 2024-05-06T12:21:00.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.406+0000 7f997dd9b700 1 -- 172.21.15.92:0/2762120854 learned_addr learned my addr 172.21.15.92:0/2762120854 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:21:00.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.406+0000 7f997e59c700 1 --2- 172.21.15.92:0/2762120854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99800fce60 0x7f998010d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:00.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.407+0000 7f997dd9b700 1 -- 172.21.15.92:0/2762120854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99800fce60 msgr2=0x7f998010d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:00.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.407+0000 7f997dd9b700 1 --2- 172.21.15.92:0/2762120854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99800fce60 0x7f998010d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:00.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.407+0000 7f997dd9b700 1 -- 172.21.15.92:0/2762120854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9974005040 con 0x7f99800fdfd0 2024-05-06T12:21:00.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.407+0000 7f997e59c700 1 --2- 172.21.15.92:0/2762120854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99800fce60 0x7f998010d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-06T12:21:00.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.407+0000 7f997dd9b700 1 --2- 172.21.15.92:0/2762120854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99800fdfd0 0x7f998010de00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9974013870 tx=0x7f9974009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:21:00.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.407+0000 7f996effd700 1 -- 172.21.15.92:0/2762120854 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9974021070 con 0x7f99800fdfd0 2024-05-06T12:21:00.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.407+0000 7f996effd700 1 -- 172.21.15.92:0/2762120854 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9974009430 con 0x7f99800fdfd0 2024-05-06T12:21:00.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.407+0000 7f99848df700 1 -- 172.21.15.92:0/2762120854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9980112f70 con 0x7f99800fdfd0 2024-05-06T12:21:00.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.407+0000 7f996effd700 1 -- 172.21.15.92:0/2762120854 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f997400e070 con 0x7f99800fdfd0 2024-05-06T12:21:00.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.407+0000 7f99848df700 1 -- 172.21.15.92:0/2762120854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9980113480 con 0x7f99800fdfd0 2024-05-06T12:21:00.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.408+0000 7f99848df700 1 -- 172.21.15.92:0/2762120854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f998004c710 con 0x7f99800fdfd0 2024-05-06T12:21:00.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.409+0000 7f996effd700 1 -- 172.21.15.92:0/2762120854 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f997401c750 con 0x7f99800fdfd0 2024-05-06T12:21:00.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.409+0000 7f996effd700 1 --2- 172.21.15.92:0/2762120854 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f996806e0e0 0x7f9968070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:00.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.409+0000 7f996effd700 1 -- 172.21.15.92:0/2762120854 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f997408ec70 con 0x7f99800fdfd0 2024-05-06T12:21:00.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.409+0000 7f997e59c700 1 --2- 172.21.15.92:0/2762120854 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f996806e0e0 0x7f9968070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:00.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.410+0000 7f997e59c700 1 --2- 172.21.15.92:0/2762120854 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f996806e0e0 0x7f9968070590 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9970009b80 tx=0x7f997000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:21:00.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.414+0000 7f996effd700 1 -- 172.21.15.92:0/2762120854 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f997405bba0 con 0x7f99800fdfd0 2024-05-06T12:21:00.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.620+0000 7f99848df700 1 -- 172.21.15.92:0/2762120854 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f99801022a0 con 0x7f996806e0e0 2024-05-06T12:21:00.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.630+0000 7f996effd700 1 -- 172.21.15.92:0/2762120854 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f99801022a0 con 0x7f996806e0e0 2024-05-06T12:21:00.633 INFO:teuthology.orchestra.run.smithi092.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-06T12:21:00.633 INFO:teuthology.orchestra.run.smithi092.stdout:alertmanager.smithi092 smithi092 *:9093,9094 running (6m) 103s ago 14m 25.2M - 0.25.0 c8568f914cd2 be5637d7d25a 2024-05-06T12:21:00.633 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi092 smithi092 running (3m) 103s ago 14m 6660k - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 8da0494e1bdc 2024-05-06T12:21:00.633 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi190 smithi190 running (3m) 3s ago 13m 6665k - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 641a7cdaa56e 2024-05-06T12:21:00.634 INFO:teuthology.orchestra.run.smithi092.stdout:grafana.smithi092 smithi092 *:3000 running (5m) 103s ago 14m 75.0M - 9.4.7 954c08fa6188 9180cb3e5258 2024-05-06T12:21:00.634 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.ijsqgp smithi092 running (9m) 103s ago 9m 60.7M - 17.2.7-925-ge81df420 96d5ae7fd4b4 b4c2ed6d2c82 2024-05-06T12:21:00.634 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.rosbll smithi092 running (9m) 103s ago 9m 122M - 17.2.7-925-ge81df420 96d5ae7fd4b4 5f5426cd1bf5 2024-05-06T12:21:00.634 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.byjvgf smithi190 running (9m) 3s ago 9m 126M - 17.2.7-925-ge81df420 96d5ae7fd4b4 24581d2e1c5d 2024-05-06T12:21:00.634 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.idzoff smithi190 running (9m) 3s ago 9m 64.0M - 17.2.7-925-ge81df420 96d5ae7fd4b4 20f2641696ef 2024-05-06T12:21:00.634 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi092.gyzeqb smithi092 *:8443,9283,8765 running (5m) 103s ago 16m 504M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 de9d2198327a 2024-05-06T12:21:00.634 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi190.kczxkh smithi190 *:8443,9283,8765 running (4m) 3s ago 13m 451M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 e84ab3758ac7 2024-05-06T12:21:00.634 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi092 smithi092 running (4m) 103s ago 16m 46.0M 2048M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 ac07f7158087 2024-05-06T12:21:00.634 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi190 smithi190 running (4m) 3s ago 13m 41.7M 2048M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 7a3c08eb716f 2024-05-06T12:21:00.634 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi092 smithi092 *:9100 running (6m) 103s ago 14m 14.9M - 1.5.0 0da6a335fe13 c796446b206f 2024-05-06T12:21:00.634 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi190 smithi190 *:9100 running (5m) 3s ago 13m 16.5M - 1.5.0 0da6a335fe13 647e28111a76 2024-05-06T12:21:00.634 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 smithi092 running (3m) 103s ago 12m 82.0M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 8c96c75b25f4 2024-05-06T12:21:00.634 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 smithi092 running (2m) 103s ago 12m 82.0M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 e8007a510f08 2024-05-06T12:21:00.634 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 smithi092 running (109s) 103s ago 11m 14.8M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 2beca7448789 2024-05-06T12:21:00.635 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 smithi190 running (74s) 3s ago 11m 87.7M 2581M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 914ada199f93 2024-05-06T12:21:00.635 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 smithi190 running (41s) 3s ago 11m 82.6M 2581M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 371c0be776c7 2024-05-06T12:21:00.635 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 smithi190 running (7s) 3s ago 10m 12.0M 2581M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 067bc34cdfb1 2024-05-06T12:21:00.635 INFO:teuthology.orchestra.run.smithi092.stdout:prometheus.smithi092 smithi092 *:9095 running (4m) 103s ago 14m 54.4M - 2.43.0 a07b618ecd1d bc5445691527 2024-05-06T12:21:00.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.632+0000 7f99848df700 1 -- 172.21.15.92:0/2762120854 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f996806e0e0 msgr2=0x7f9968070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:00.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.632+0000 7f99848df700 1 --2- 172.21.15.92:0/2762120854 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f996806e0e0 0x7f9968070590 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9970009b80 tx=0x7f997000b040 comp rx=0 tx=0).stop 2024-05-06T12:21:00.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.632+0000 7f99848df700 1 -- 172.21.15.92:0/2762120854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99800fdfd0 msgr2=0x7f998010de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:00.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.632+0000 7f99848df700 1 --2- 172.21.15.92:0/2762120854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99800fdfd0 0x7f998010de00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9974013870 tx=0x7f9974009e30 comp rx=0 tx=0).stop 2024-05-06T12:21:00.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.632+0000 7f99848df700 1 -- 172.21.15.92:0/2762120854 shutdown_connections 2024-05-06T12:21:00.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.632+0000 7f99848df700 1 --2- 172.21.15.92:0/2762120854 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f996806e0e0 0x7f9968070590 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:00.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.632+0000 7f99848df700 1 --2- 172.21.15.92:0/2762120854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99800fce60 0x7f998010d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:00.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.632+0000 7f99848df700 1 --2- 172.21.15.92:0/2762120854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99800fdfd0 0x7f998010de00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:00.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.633+0000 7f99848df700 1 -- 172.21.15.92:0/2762120854 >> 172.21.15.92:0/2762120854 conn(0x7f99800f8540 msgr2=0x7f99800f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:21:00.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.633+0000 7f99848df700 1 -- 172.21.15.92:0/2762120854 shutdown_connections 2024-05-06T12:21:00.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.633+0000 7f99848df700 1 -- 172.21.15.92:0/2762120854 wait complete. 2024-05-06T12:21:00.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.732+0000 7f7360d29700 1 -- 172.21.15.92:0/2464276982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f735c0fed70 msgr2=0x7f735c0faeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:00.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.732+0000 7f7360d29700 1 --2- 172.21.15.92:0/2464276982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f735c0fed70 0x7f735c0faeb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7350004940 tx=0x7f7350009e30 comp rx=0 tx=0).stop 2024-05-06T12:21:00.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.732+0000 7f7360d29700 1 -- 172.21.15.92:0/2464276982 shutdown_connections 2024-05-06T12:21:00.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.732+0000 7f7360d29700 1 --2- 172.21.15.92:0/2464276982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f735c0fed70 0x7f735c0faeb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:00.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.732+0000 7f7360d29700 1 --2- 172.21.15.92:0/2464276982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f735c0fe420 0x7f735c0fe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:00.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.732+0000 7f7360d29700 1 -- 172.21.15.92:0/2464276982 >> 172.21.15.92:0/2464276982 conn(0x7f735c0f6be0 msgr2=0x7f735c0f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:21:00.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.732+0000 7f7360d29700 1 -- 172.21.15.92:0/2464276982 shutdown_connections 2024-05-06T12:21:00.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.732+0000 7f7360d29700 1 -- 172.21.15.92:0/2464276982 wait complete. 2024-05-06T12:21:00.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.733+0000 7f7360d29700 1 Processor -- start 2024-05-06T12:21:00.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.733+0000 7f7360d29700 1 -- start start 2024-05-06T12:21:00.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.733+0000 7f7360d29700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f735c0fe420 0x7f735c10d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:00.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.733+0000 7f7360d29700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f735c0fed70 0x7f735c10de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:00.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.733+0000 7f7360d29700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735c10e440 con 0x7f735c0fed70 2024-05-06T12:21:00.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.733+0000 7f7360d29700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735c10e5b0 con 0x7f735c0fe420 2024-05-06T12:21:00.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.733+0000 7f7359d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f735c0fed70 0x7f735c10de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:00.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.733+0000 7f7359d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f735c0fed70 0x7f735c10de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43608/0 (socket says 172.21.15.92:43608) 2024-05-06T12:21:00.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.733+0000 7f7359d9b700 1 -- 172.21.15.92:0/124765045 learned_addr learned my addr 172.21.15.92:0/124765045 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:21:00.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.733+0000 7f735a59c700 1 --2- 172.21.15.92:0/124765045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f735c0fe420 0x7f735c10d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:00.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.733+0000 7f7359d9b700 1 -- 172.21.15.92:0/124765045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f735c0fe420 msgr2=0x7f735c10d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:00.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.733+0000 7f7359d9b700 1 --2- 172.21.15.92:0/124765045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f735c0fe420 0x7f735c10d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:00.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.733+0000 7f7359d9b700 1 -- 172.21.15.92:0/124765045 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7350005040 con 0x7f735c0fed70 2024-05-06T12:21:00.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.734+0000 7f7359d9b700 1 --2- 172.21.15.92:0/124765045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f735c0fed70 0x7f735c10de40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f735000b3b0 tx=0x7f7350009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:21:00.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.734+0000 7f734affd700 1 -- 172.21.15.92:0/124765045 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f735000e040 con 0x7f735c0fed70 2024-05-06T12:21:00.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.734+0000 7f734affd700 1 -- 172.21.15.92:0/124765045 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f735000bbf0 con 0x7f735c0fed70 2024-05-06T12:21:00.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.734+0000 7f7360d29700 1 -- 172.21.15.92:0/124765045 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f735c112fb0 con 0x7f735c0fed70 2024-05-06T12:21:00.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.734+0000 7f734affd700 1 -- 172.21.15.92:0/124765045 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f735001c630 con 0x7f735c0fed70 2024-05-06T12:21:00.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.734+0000 7f7360d29700 1 -- 172.21.15.92:0/124765045 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f735c113540 con 0x7f735c0fed70 2024-05-06T12:21:00.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.735+0000 7f7360d29700 1 -- 172.21.15.92:0/124765045 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f735c04c710 con 0x7f735c0fed70 2024-05-06T12:21:00.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.735+0000 7f734affd700 1 -- 172.21.15.92:0/124765045 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f735000bd60 con 0x7f735c0fed70 2024-05-06T12:21:00.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.736+0000 7f734affd700 1 --2- 172.21.15.92:0/124765045 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f734406e140 0x7f73440705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:00.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.736+0000 7f735a59c700 1 --2- 172.21.15.92:0/124765045 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f734406e140 0x7f73440705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:00.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.736+0000 7f734affd700 1 -- 172.21.15.92:0/124765045 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f735008e6c0 con 0x7f735c0fed70 2024-05-06T12:21:00.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.736+0000 7f735a59c700 1 --2- 172.21.15.92:0/124765045 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f734406e140 0x7f73440705f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f734c004f40 tx=0x7f734c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:21:00.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:00.741+0000 7f734affd700 1 -- 172.21.15.92:0/124765045 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f735005b5f0 con 0x7f735c0fed70 2024-05-06T12:21:00.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:00 smithi190 ceph-mon[79744]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-06T12:21:00.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:00 smithi190 ceph-mon[79744]: osd.5 [v2:172.21.15.190:6816/10700099,v1:172.21.15.190:6817/10700099] boot 2024-05-06T12:21:00.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:00 smithi190 ceph-mon[79744]: osdmap e79: 6 total, 6 up, 6 in 2024-05-06T12:21:00.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:00 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-06T12:21:00.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:00 smithi190 ceph-mon[79744]: from='client.34370 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:21:00.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:00 smithi190 ceph-mon[79744]: osdmap e80: 6 total, 6 up, 6 in 2024-05-06T12:21:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.040+0000 7f7360d29700 1 -- 172.21.15.92:0/124765045 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f735c1058d0 con 0x7f735c0fed70 2024-05-06T12:21:01.044 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:00 smithi092 ceph-mon[91933]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-06T12:21:01.044 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:00 smithi092 ceph-mon[91933]: osd.5 [v2:172.21.15.190:6816/10700099,v1:172.21.15.190:6817/10700099] boot 2024-05-06T12:21:01.044 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:00 smithi092 ceph-mon[91933]: osdmap e79: 6 total, 6 up, 6 in 2024-05-06T12:21:01.044 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:00 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-06T12:21:01.044 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:00 smithi092 ceph-mon[91933]: from='client.34370 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:21:01.044 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:00 smithi092 ceph-mon[91933]: osdmap e80: 6 total, 6 up, 6 in 2024-05-06T12:21:01.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.041+0000 7f734affd700 1 -- 172.21.15.92:0/124765045 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+733 (secure 0 0 0) 0x7f735005b180 con 0x7f735c0fed70 2024-05-06T12:21:01.045 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:21:01.045 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": { 2024-05-06T12:21:01.045 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:21:01.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:21:01.045 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": { 2024-05-06T12:21:01.045 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:21:01.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:21:01.045 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": { 2024-05-06T12:21:01.045 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 6 2024-05-06T12:21:01.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:21:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-05-06T12:21:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 4 2024-05-06T12:21:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:21:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: "overall": { 2024-05-06T12:21:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 4, 2024-05-06T12:21:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 10 2024-05-06T12:21:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-06T12:21:01.046 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:21:01.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.043+0000 7f7360d29700 1 -- 172.21.15.92:0/124765045 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f734406e140 msgr2=0x7f73440705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:01.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.043+0000 7f7360d29700 1 --2- 172.21.15.92:0/124765045 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f734406e140 0x7f73440705f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f734c004f40 tx=0x7f734c00b040 comp rx=0 tx=0).stop 2024-05-06T12:21:01.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.043+0000 7f7360d29700 1 -- 172.21.15.92:0/124765045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f735c0fed70 msgr2=0x7f735c10de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:01.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.043+0000 7f7360d29700 1 --2- 172.21.15.92:0/124765045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f735c0fed70 0x7f735c10de40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f735000b3b0 tx=0x7f7350009e30 comp rx=0 tx=0).stop 2024-05-06T12:21:01.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.043+0000 7f7360d29700 1 -- 172.21.15.92:0/124765045 shutdown_connections 2024-05-06T12:21:01.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.043+0000 7f7360d29700 1 --2- 172.21.15.92:0/124765045 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f734406e140 0x7f73440705f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:01.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.043+0000 7f7360d29700 1 --2- 172.21.15.92:0/124765045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f735c0fe420 0x7f735c10d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:01.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.043+0000 7f7360d29700 1 --2- 172.21.15.92:0/124765045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f735c0fed70 0x7f735c10de40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:01.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.043+0000 7f7360d29700 1 -- 172.21.15.92:0/124765045 >> 172.21.15.92:0/124765045 conn(0x7f735c0f6be0 msgr2=0x7f735c0f8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:21:01.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.043+0000 7f7360d29700 1 -- 172.21.15.92:0/124765045 shutdown_connections 2024-05-06T12:21:01.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.043+0000 7f7360d29700 1 -- 172.21.15.92:0/124765045 wait complete. 2024-05-06T12:21:01.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.144+0000 7f941777d700 1 -- 172.21.15.92:0/1823208794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94100fbcc0 msgr2=0x7f94100fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:01.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.144+0000 7f941777d700 1 --2- 172.21.15.92:0/1823208794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94100fbcc0 0x7f94100fc0d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f940c004940 tx=0x7f940c009e30 comp rx=0 tx=0).stop 2024-05-06T12:21:01.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.144+0000 7f941777d700 1 -- 172.21.15.92:0/1823208794 shutdown_connections 2024-05-06T12:21:01.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.144+0000 7f941777d700 1 --2- 172.21.15.92:0/1823208794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94100fcf70 0x7f94100fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:01.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.144+0000 7f941777d700 1 --2- 172.21.15.92:0/1823208794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94100fbcc0 0x7f94100fc0d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:01.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.144+0000 7f941777d700 1 -- 172.21.15.92:0/1823208794 >> 172.21.15.92:0/1823208794 conn(0x7f94100f7440 msgr2=0x7f94100f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:21:01.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.145+0000 7f941777d700 1 -- 172.21.15.92:0/1823208794 shutdown_connections 2024-05-06T12:21:01.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.145+0000 7f941777d700 1 -- 172.21.15.92:0/1823208794 wait complete. 2024-05-06T12:21:01.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.145+0000 7f941777d700 1 Processor -- start 2024-05-06T12:21:01.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.145+0000 7f941777d700 1 -- start start 2024-05-06T12:21:01.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.145+0000 7f941777d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94100fbcc0 0x7f94100ffc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:01.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.145+0000 7f941777d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94100fcf70 0x7f94100fe2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:01.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.145+0000 7f941777d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9410100200 con 0x7f94100fbcc0 2024-05-06T12:21:01.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.145+0000 7f941777d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94100fe7f0 con 0x7f94100fcf70 2024-05-06T12:21:01.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.145+0000 7f9415519700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94100fbcc0 0x7f94100ffc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:01.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.145+0000 7f9415519700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94100fbcc0 0x7f94100ffc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43636/0 (socket says 172.21.15.92:43636) 2024-05-06T12:21:01.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.145+0000 7f9415519700 1 -- 172.21.15.92:0/1948904589 learned_addr learned my addr 172.21.15.92:0/1948904589 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:21:01.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.145+0000 7f9414d18700 1 --2- 172.21.15.92:0/1948904589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94100fcf70 0x7f94100fe2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:01.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.146+0000 7f9415519700 1 -- 172.21.15.92:0/1948904589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94100fcf70 msgr2=0x7f94100fe2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:01.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.146+0000 7f9415519700 1 --2- 172.21.15.92:0/1948904589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94100fcf70 0x7f94100fe2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:01.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.146+0000 7f9415519700 1 -- 172.21.15.92:0/1948904589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f940c005040 con 0x7f94100fbcc0 2024-05-06T12:21:01.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.146+0000 7f9414d18700 1 --2- 172.21.15.92:0/1948904589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94100fcf70 0x7f94100fe2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-06T12:21:01.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.146+0000 7f9415519700 1 --2- 172.21.15.92:0/1948904589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94100fbcc0 0x7f94100ffc00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f940c009150 tx=0x7f940c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:21:01.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.146+0000 7f9405ffb700 1 -- 172.21.15.92:0/1948904589 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f940c00e040 con 0x7f94100fbcc0 2024-05-06T12:21:01.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.146+0000 7f9405ffb700 1 -- 172.21.15.92:0/1948904589 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f940c00bcc0 con 0x7f94100fbcc0 2024-05-06T12:21:01.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.146+0000 7f941777d700 1 -- 172.21.15.92:0/1948904589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94100fea40 con 0x7f94100fbcc0 2024-05-06T12:21:01.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.146+0000 7f9405ffb700 1 -- 172.21.15.92:0/1948904589 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f940c01c630 con 0x7f94100fbcc0 2024-05-06T12:21:01.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.146+0000 7f941777d700 1 -- 172.21.15.92:0/1948904589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94100ff000 con 0x7f94100fbcc0 2024-05-06T12:21:01.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.147+0000 7f941777d700 1 -- 172.21.15.92:0/1948904589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9410101b00 con 0x7f94100fbcc0 2024-05-06T12:21:01.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.148+0000 7f9405ffb700 1 -- 172.21.15.92:0/1948904589 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f940c013420 con 0x7f94100fbcc0 2024-05-06T12:21:01.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.148+0000 7f9405ffb700 1 --2- 172.21.15.92:0/1948904589 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f93fc06e140 0x7f93fc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:01.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.148+0000 7f9405ffb700 1 -- 172.21.15.92:0/1948904589 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f940c08e6c0 con 0x7f94100fbcc0 2024-05-06T12:21:01.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.148+0000 7f9414d18700 1 --2- 172.21.15.92:0/1948904589 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f93fc06e140 0x7f93fc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:01.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.149+0000 7f9414d18700 1 --2- 172.21.15.92:0/1948904589 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f93fc06e140 0x7f93fc0705f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9400007fd0 tx=0x7f9400009f60 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:21:01.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.153+0000 7f9405ffb700 1 -- 172.21.15.92:0/1948904589 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f940c05b690 con 0x7f94100fbcc0 2024-05-06T12:21:01.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.401+0000 7f941777d700 1 -- 172.21.15.92:0/1948904589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f941002b340 con 0x7f94100fbcc0 2024-05-06T12:21:01.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.402+0000 7f9405ffb700 1 -- 172.21.15.92:0/1948904589 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1918 (secure 0 0 0) 0x7f940c05b220 con 0x7f94100fbcc0 2024-05-06T12:21:01.405 INFO:teuthology.orchestra.run.smithi092.stdout:e22 2024-05-06T12:21:01.405 INFO:teuthology.orchestra.run.smithi092.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-06T12:21:01.405 INFO:teuthology.orchestra.run.smithi092.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:21:01.405 INFO:teuthology.orchestra.run.smithi092.stdout:legacy client fscid: 1 2024-05-06T12:21:01.405 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:21:01.405 INFO:teuthology.orchestra.run.smithi092.stdout:Filesystem 'cephfs' (1) 2024-05-06T12:21:01.405 INFO:teuthology.orchestra.run.smithi092.stdout:fs_name cephfs 2024-05-06T12:21:01.405 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 22 2024-05-06T12:21:01.405 INFO:teuthology.orchestra.run.smithi092.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-06T12:21:01.405 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-05-06T12:11:44.952671+0000 2024-05-06T12:21:01.405 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2024-05-06T12:19:06.643003+0000 2024-05-06T12:21:01.405 INFO:teuthology.orchestra.run.smithi092.stdout:tableserver 0 2024-05-06T12:21:01.405 INFO:teuthology.orchestra.run.smithi092.stdout:root 0 2024-05-06T12:21:01.405 INFO:teuthology.orchestra.run.smithi092.stdout:session_timeout 60 2024-05-06T12:21:01.406 INFO:teuthology.orchestra.run.smithi092.stdout:session_autoclose 300 2024-05-06T12:21:01.406 INFO:teuthology.orchestra.run.smithi092.stdout:max_file_size 1099511627776 2024-05-06T12:21:01.406 INFO:teuthology.orchestra.run.smithi092.stdout:max_xattr_size 65536 2024-05-06T12:21:01.406 INFO:teuthology.orchestra.run.smithi092.stdout:required_client_features {} 2024-05-06T12:21:01.406 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure 0 2024-05-06T12:21:01.406 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure_osd_epoch 46 2024-05-06T12:21:01.406 INFO:teuthology.orchestra.run.smithi092.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:21:01.406 INFO:teuthology.orchestra.run.smithi092.stdout:max_mds 2 2024-05-06T12:21:01.406 INFO:teuthology.orchestra.run.smithi092.stdout:in 0,1 2024-05-06T12:21:01.406 INFO:teuthology.orchestra.run.smithi092.stdout:up {0=14472,1=24267} 2024-05-06T12:21:01.406 INFO:teuthology.orchestra.run.smithi092.stdout:failed 2024-05-06T12:21:01.406 INFO:teuthology.orchestra.run.smithi092.stdout:damaged 2024-05-06T12:21:01.406 INFO:teuthology.orchestra.run.smithi092.stdout:stopped 2024-05-06T12:21:01.407 INFO:teuthology.orchestra.run.smithi092.stdout:data_pools [3] 2024-05-06T12:21:01.407 INFO:teuthology.orchestra.run.smithi092.stdout:metadata_pool 2 2024-05-06T12:21:01.407 INFO:teuthology.orchestra.run.smithi092.stdout:inline_data enabled 2024-05-06T12:21:01.407 INFO:teuthology.orchestra.run.smithi092.stdout:balancer 2024-05-06T12:21:01.407 INFO:teuthology.orchestra.run.smithi092.stdout:bal_rank_mask -1 2024-05-06T12:21:01.407 INFO:teuthology.orchestra.run.smithi092.stdout:standby_count_wanted 1 2024-05-06T12:21:01.407 INFO:teuthology.orchestra.run.smithi092.stdout:qdb_cluster leader: 14472 members: 24267,14472 2024-05-06T12:21:01.407 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.rosbll{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.92:6828/1663253598,v1:172.21.15.92:6829/1663253598] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:21:01.407 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.ijsqgp{0:14484} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.92:6826/3613441054,v1:172.21.15.92:6827/3613441054] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:21:01.407 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.byjvgf{1:24267} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.190:6824/1565151699,v1:172.21.15.190:6825/1565151699] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:21:01.407 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.idzoff{1:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.190:6826/2453728380,v1:172.21.15.190:6827/2453728380] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:21:01.407 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:21:01.407 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:21:01.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.404+0000 7f941777d700 1 -- 172.21.15.92:0/1948904589 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f93fc06e140 msgr2=0x7f93fc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:01.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.404+0000 7f941777d700 1 --2- 172.21.15.92:0/1948904589 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f93fc06e140 0x7f93fc0705f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9400007fd0 tx=0x7f9400009f60 comp rx=0 tx=0).stop 2024-05-06T12:21:01.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.404+0000 7f941777d700 1 -- 172.21.15.92:0/1948904589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94100fbcc0 msgr2=0x7f94100ffc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:01.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.404+0000 7f941777d700 1 --2- 172.21.15.92:0/1948904589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94100fbcc0 0x7f94100ffc00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f940c009150 tx=0x7f940c009e30 comp rx=0 tx=0).stop 2024-05-06T12:21:01.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.404+0000 7f941777d700 1 -- 172.21.15.92:0/1948904589 shutdown_connections 2024-05-06T12:21:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.404+0000 7f941777d700 1 --2- 172.21.15.92:0/1948904589 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f93fc06e140 0x7f93fc0705f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.404+0000 7f941777d700 1 --2- 172.21.15.92:0/1948904589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94100fbcc0 0x7f94100ffc00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.404+0000 7f941777d700 1 --2- 172.21.15.92:0/1948904589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94100fcf70 0x7f94100fe2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.404+0000 7f941777d700 1 -- 172.21.15.92:0/1948904589 >> 172.21.15.92:0/1948904589 conn(0x7f94100f7440 msgr2=0x7f9410066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:21:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.404+0000 7f941777d700 1 -- 172.21.15.92:0/1948904589 shutdown_connections 2024-05-06T12:21:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.404+0000 7f941777d700 1 -- 172.21.15.92:0/1948904589 wait complete. 2024-05-06T12:21:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 22 2024-05-06T12:21:01.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.512+0000 7f18f4c35700 1 -- 172.21.15.92:0/956763142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18f00fce60 msgr2=0x7f18f00fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:01.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.512+0000 7f18f4c35700 1 --2- 172.21.15.92:0/956763142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18f00fce60 0x7f18f00fd270 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f18e0004940 tx=0x7f18e0009e30 comp rx=0 tx=0).stop 2024-05-06T12:21:01.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.512+0000 7f18f4c35700 1 -- 172.21.15.92:0/956763142 shutdown_connections 2024-05-06T12:21:01.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.512+0000 7f18f4c35700 1 --2- 172.21.15.92:0/956763142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18f00fdfd0 0x7f18f00fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:01.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.512+0000 7f18f4c35700 1 --2- 172.21.15.92:0/956763142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18f00fce60 0x7f18f00fd270 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:01.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.512+0000 7f18f4c35700 1 -- 172.21.15.92:0/956763142 >> 172.21.15.92:0/956763142 conn(0x7f18f00f8540 msgr2=0x7f18f00fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:21:01.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.512+0000 7f18f4c35700 1 -- 172.21.15.92:0/956763142 shutdown_connections 2024-05-06T12:21:01.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.512+0000 7f18f4c35700 1 -- 172.21.15.92:0/956763142 wait complete. 2024-05-06T12:21:01.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.513+0000 7f18f4c35700 1 Processor -- start 2024-05-06T12:21:01.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.513+0000 7f18f4c35700 1 -- start start 2024-05-06T12:21:01.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.513+0000 7f18f4c35700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18f00fce60 0x7f18f010d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:01.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.513+0000 7f18f4c35700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18f00fdfd0 0x7f18f010de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:01.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.513+0000 7f18f4c35700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f010e480 con 0x7f18f00fce60 2024-05-06T12:21:01.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.513+0000 7f18f4c35700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f010e5f0 con 0x7f18f00fdfd0 2024-05-06T12:21:01.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.513+0000 7f18ee59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18f00fce60 0x7f18f010d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:01.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.513+0000 7f18ee59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18f00fce60 0x7f18f010d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43662/0 (socket says 172.21.15.92:43662) 2024-05-06T12:21:01.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.513+0000 7f18ee59c700 1 -- 172.21.15.92:0/1854532059 learned_addr learned my addr 172.21.15.92:0/1854532059 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:21:01.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.513+0000 7f18edd9b700 1 --2- 172.21.15.92:0/1854532059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18f00fdfd0 0x7f18f010de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:01.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.513+0000 7f18ee59c700 1 -- 172.21.15.92:0/1854532059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18f00fdfd0 msgr2=0x7f18f010de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:01.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.513+0000 7f18ee59c700 1 --2- 172.21.15.92:0/1854532059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18f00fdfd0 0x7f18f010de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:01.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.513+0000 7f18ee59c700 1 -- 172.21.15.92:0/1854532059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e0005040 con 0x7f18f00fce60 2024-05-06T12:21:01.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.514+0000 7f18ee59c700 1 --2- 172.21.15.92:0/1854532059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18f00fce60 0x7f18f010d940 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f18e0009150 tx=0x7f18e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:21:01.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.514+0000 7f18deffd700 1 -- 172.21.15.92:0/1854532059 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18e000e040 con 0x7f18f00fce60 2024-05-06T12:21:01.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.514+0000 7f18deffd700 1 -- 172.21.15.92:0/1854532059 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f18e0004290 con 0x7f18f00fce60 2024-05-06T12:21:01.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.514+0000 7f18f4c35700 1 -- 172.21.15.92:0/1854532059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18f0112ff0 con 0x7f18f00fce60 2024-05-06T12:21:01.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.514+0000 7f18deffd700 1 -- 172.21.15.92:0/1854532059 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18e001c630 con 0x7f18f00fce60 2024-05-06T12:21:01.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.514+0000 7f18f4c35700 1 -- 172.21.15.92:0/1854532059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f0113520 con 0x7f18f00fce60 2024-05-06T12:21:01.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.515+0000 7f18f4c35700 1 -- 172.21.15.92:0/1854532059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f004c710 con 0x7f18f00fce60 2024-05-06T12:21:01.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.515+0000 7f18deffd700 1 -- 172.21.15.92:0/1854532059 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f18e000bbf0 con 0x7f18f00fce60 2024-05-06T12:21:01.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.516+0000 7f18deffd700 1 --2- 172.21.15.92:0/1854532059 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f18d806e140 0x7f18d80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:01.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.516+0000 7f18deffd700 1 -- 172.21.15.92:0/1854532059 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f18e008e5d0 con 0x7f18f00fce60 2024-05-06T12:21:01.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.516+0000 7f18edd9b700 1 --2- 172.21.15.92:0/1854532059 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f18d806e140 0x7f18d80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:01.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.516+0000 7f18edd9b700 1 --2- 172.21.15.92:0/1854532059 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f18d806e140 0x7f18d80705f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f18e4007fd0 tx=0x7f18e4009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:21:01.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.520+0000 7f18deffd700 1 -- 172.21.15.92:0/1854532059 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f18e005b5a0 con 0x7f18f00fce60 2024-05-06T12:21:01.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.736+0000 7f18f4c35700 1 -- 172.21.15.92:0/1854532059 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f18f0102330 con 0x7f18d806e140 2024-05-06T12:21:01.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.738+0000 7f18deffd700 1 -- 172.21.15.92:0/1854532059 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f18f0102330 con 0x7f18d806e140 2024-05-06T12:21:01.740 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:21:01.740 INFO:teuthology.orchestra.run.smithi092.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb", 2024-05-06T12:21:01.740 INFO:teuthology.orchestra.run.smithi092.stdout: "in_progress": true, 2024-05-06T12:21:01.740 INFO:teuthology.orchestra.run.smithi092.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-06T12:21:01.741 INFO:teuthology.orchestra.run.smithi092.stdout: "services_complete": [ 2024-05-06T12:21:01.741 INFO:teuthology.orchestra.run.smithi092.stdout: "osd", 2024-05-06T12:21:01.741 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr", 2024-05-06T12:21:01.741 INFO:teuthology.orchestra.run.smithi092.stdout: "mon", 2024-05-06T12:21:01.741 INFO:teuthology.orchestra.run.smithi092.stdout: "crash" 2024-05-06T12:21:01.741 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-05-06T12:21:01.741 INFO:teuthology.orchestra.run.smithi092.stdout: "progress": "12/21 daemons upgraded", 2024-05-06T12:21:01.741 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "Currently upgrading osd daemons", 2024-05-06T12:21:01.741 INFO:teuthology.orchestra.run.smithi092.stdout: "is_paused": false 2024-05-06T12:21:01.741 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:21:01.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.740+0000 7f18f4c35700 1 -- 172.21.15.92:0/1854532059 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f18d806e140 msgr2=0x7f18d80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:01.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.740+0000 7f18f4c35700 1 --2- 172.21.15.92:0/1854532059 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f18d806e140 0x7f18d80705f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f18e4007fd0 tx=0x7f18e4009040 comp rx=0 tx=0).stop 2024-05-06T12:21:01.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.740+0000 7f18f4c35700 1 -- 172.21.15.92:0/1854532059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18f00fce60 msgr2=0x7f18f010d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:01.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.740+0000 7f18f4c35700 1 --2- 172.21.15.92:0/1854532059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18f00fce60 0x7f18f010d940 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f18e0009150 tx=0x7f18e0009e30 comp rx=0 tx=0).stop 2024-05-06T12:21:01.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.740+0000 7f18f4c35700 1 -- 172.21.15.92:0/1854532059 shutdown_connections 2024-05-06T12:21:01.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.740+0000 7f18f4c35700 1 --2- 172.21.15.92:0/1854532059 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f18d806e140 0x7f18d80705f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:01.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.740+0000 7f18f4c35700 1 --2- 172.21.15.92:0/1854532059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18f00fce60 0x7f18f010d940 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:01.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.740+0000 7f18f4c35700 1 --2- 172.21.15.92:0/1854532059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18f00fdfd0 0x7f18f010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:01.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.740+0000 7f18f4c35700 1 -- 172.21.15.92:0/1854532059 >> 172.21.15.92:0/1854532059 conn(0x7f18f00f8540 msgr2=0x7f18f00f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:21:01.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.740+0000 7f18f4c35700 1 -- 172.21.15.92:0/1854532059 shutdown_connections 2024-05-06T12:21:01.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.740+0000 7f18f4c35700 1 -- 172.21.15.92:0/1854532059 wait complete. 2024-05-06T12:21:01.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.843+0000 7f137dae7700 1 -- 172.21.15.92:0/1079079252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13780fbce0 msgr2=0x7f13780fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:01.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.843+0000 7f137dae7700 1 --2- 172.21.15.92:0/1079079252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13780fbce0 0x7f13780fc0f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1368004940 tx=0x7f1368009e30 comp rx=0 tx=0).stop 2024-05-06T12:21:01.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.843+0000 7f137dae7700 1 -- 172.21.15.92:0/1079079252 shutdown_connections 2024-05-06T12:21:01.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.843+0000 7f137dae7700 1 --2- 172.21.15.92:0/1079079252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13780fcf90 0x7f13780fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:01.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.843+0000 7f137dae7700 1 --2- 172.21.15.92:0/1079079252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13780fbce0 0x7f13780fc0f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:01.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.843+0000 7f137dae7700 1 -- 172.21.15.92:0/1079079252 >> 172.21.15.92:0/1079079252 conn(0x7f13780f7480 msgr2=0x7f13780f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:21:01.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.843+0000 7f137dae7700 1 -- 172.21.15.92:0/1079079252 shutdown_connections 2024-05-06T12:21:01.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.843+0000 7f137dae7700 1 -- 172.21.15.92:0/1079079252 wait complete. 2024-05-06T12:21:01.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.844+0000 7f137dae7700 1 Processor -- start 2024-05-06T12:21:01.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.844+0000 7f137dae7700 1 -- start start 2024-05-06T12:21:01.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.844+0000 7f137dae7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13780fbce0 0x7f1378066cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:01.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.844+0000 7f137dae7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13780fcf90 0x7f1378112750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:01.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.844+0000 7f137dae7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1378112c90 con 0x7f13780fcf90 2024-05-06T12:21:01.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.844+0000 7f137dae7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1378112e00 con 0x7f13780fbce0 2024-05-06T12:21:01.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.844+0000 7f1376ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13780fcf90 0x7f1378112750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:01.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.844+0000 7f1376ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13780fcf90 0x7f1378112750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43676/0 (socket says 172.21.15.92:43676) 2024-05-06T12:21:01.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.845+0000 7f1376ffd700 1 -- 172.21.15.92:0/2321403096 learned_addr learned my addr 172.21.15.92:0/2321403096 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:21:01.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.845+0000 7f13777fe700 1 --2- 172.21.15.92:0/2321403096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13780fbce0 0x7f1378066cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:01.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.845+0000 7f1376ffd700 1 -- 172.21.15.92:0/2321403096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13780fbce0 msgr2=0x7f1378066cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:01.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.845+0000 7f1376ffd700 1 --2- 172.21.15.92:0/2321403096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13780fbce0 0x7f1378066cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:01.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.845+0000 7f1376ffd700 1 -- 172.21.15.92:0/2321403096 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1368005040 con 0x7f13780fcf90 2024-05-06T12:21:01.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.845+0000 7f1376ffd700 1 --2- 172.21.15.92:0/2321403096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13780fcf90 0x7f1378112750 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f136c00b790 tx=0x7f136c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:21:01.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.845+0000 7f135ffff700 1 -- 172.21.15.92:0/2321403096 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f136c00f6f0 con 0x7f13780fcf90 2024-05-06T12:21:01.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.845+0000 7f135ffff700 1 -- 172.21.15.92:0/2321403096 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f136c00fd20 con 0x7f13780fcf90 2024-05-06T12:21:01.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.845+0000 7f137dae7700 1 -- 172.21.15.92:0/2321403096 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13781130e0 con 0x7f13780fcf90 2024-05-06T12:21:01.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.845+0000 7f135ffff700 1 -- 172.21.15.92:0/2321403096 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f136c0103a0 con 0x7f13780fcf90 2024-05-06T12:21:01.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.845+0000 7f137dae7700 1 -- 172.21.15.92:0/2321403096 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13781136a0 con 0x7f13780fcf90 2024-05-06T12:21:01.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.846+0000 7f137dae7700 1 -- 172.21.15.92:0/2321403096 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f137804c710 con 0x7f13780fcf90 2024-05-06T12:21:01.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.847+0000 7f135ffff700 1 -- 172.21.15.92:0/2321403096 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f136c011400 con 0x7f13780fcf90 2024-05-06T12:21:01.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.847+0000 7f135ffff700 1 --2- 172.21.15.92:0/2321403096 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f136006e140 0x7f13600705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:01.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.847+0000 7f135ffff700 1 -- 172.21.15.92:0/2321403096 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f136c08d960 con 0x7f13780fcf90 2024-05-06T12:21:01.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.847+0000 7f13777fe700 1 --2- 172.21.15.92:0/2321403096 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f136006e140 0x7f13600705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:01.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.847+0000 7f13777fe700 1 --2- 172.21.15.92:0/2321403096 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f136006e140 0x7f13600705f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f136800b970 tx=0x7f1368009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:21:01.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:01.852+0000 7f135ffff700 1 -- 172.21.15.92:0/2321403096 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f136c05a890 con 0x7f13780fcf90 2024-05-06T12:21:02.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:01 smithi190 ceph-mon[79744]: from='client.34374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:21:02.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:01 smithi190 ceph-mon[79744]: from='client.34378 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:21:02.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:01 smithi190 ceph-mon[79744]: pgmap v164: 65 pgs: 1 peering, 14 active+undersized+degraded, 13 active+undersized, 37 active+clean; 323 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s; 56/372 objects degraded (15.054%) 2024-05-06T12:21:02.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:01 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/124765045' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:02.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:01 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:02.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:01 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:02.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:01 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:21:02.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:01 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:21:02.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:01 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:02.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:01 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/1948904589' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:21:02.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:01 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:21:02.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:01 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:02.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:01 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:02.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:01 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:02.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:01 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:02.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:01 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:02.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:01 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-05-06T12:21:02.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:01 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-05-06T12:21:02.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:01 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-05-06T12:21:02.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:01 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-05-06T12:21:02.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:01 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-05-06T12:21:02.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:01 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-05-06T12:21:02.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:01 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-05-06T12:21:02.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:01 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-05-06T12:21:02.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:01 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-05-06T12:21:02.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:01 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-05-06T12:21:02.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:01 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-05-06T12:21:02.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:01 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-05-06T12:21:02.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:01 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-05-06T12:21:02.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:02.135+0000 7f137dae7700 1 -- 172.21.15.92:0/2321403096 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1378104850 con 0x7f13780fcf90 2024-05-06T12:21:02.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:02.135+0000 7f135ffff700 1 -- 172.21.15.92:0/2321403096 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1391 (secure 0 0 0) 0x7f136c05a420 con 0x7f13780fcf90 2024-05-06T12:21:02.138 INFO:teuthology.orchestra.run.smithi092.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; all OSDs are running squid or later but require_osd_release < squid; Degraded data redundancy: 56/372 objects degraded (15.054%), 14 pgs degraded 2024-05-06T12:21:02.139 INFO:teuthology.orchestra.run.smithi092.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-06T12:21:02.139 INFO:teuthology.orchestra.run.smithi092.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-06T12:21:02.139 INFO:teuthology.orchestra.run.smithi092.stdout:[WRN] OSD_UPGRADE_FINISHED: all OSDs are running squid or later but require_osd_release < squid 2024-05-06T12:21:02.139 INFO:teuthology.orchestra.run.smithi092.stdout: all OSDs are running squid or later but require_osd_release < squid 2024-05-06T12:21:02.139 INFO:teuthology.orchestra.run.smithi092.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 56/372 objects degraded (15.054%), 14 pgs degraded 2024-05-06T12:21:02.139 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.2 is active+undersized+degraded, acting [1,0] 2024-05-06T12:21:02.139 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.3 is active+undersized+degraded, acting [2,1] 2024-05-06T12:21:02.139 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.8 is active+undersized+degraded, acting [3,0] 2024-05-06T12:21:02.139 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.b is active+undersized+degraded, acting [3,4] 2024-05-06T12:21:02.139 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.f is active+undersized+degraded, acting [4,0] 2024-05-06T12:21:02.139 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.14 is active+undersized+degraded, acting [3,4] 2024-05-06T12:21:02.139 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.16 is active+undersized+degraded, acting [3,2] 2024-05-06T12:21:02.139 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.17 is active+undersized+degraded, acting [2,1] 2024-05-06T12:21:02.139 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.18 is active+undersized+degraded, acting [4,3] 2024-05-06T12:21:02.140 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.1a is active+undersized+degraded, acting [3,4] 2024-05-06T12:21:02.140 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.1b is active+undersized+degraded, acting [1,0] 2024-05-06T12:21:02.140 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.1c is active+undersized+degraded, acting [4,2] 2024-05-06T12:21:02.140 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.1d is active+undersized+degraded, acting [3,0] 2024-05-06T12:21:02.140 INFO:teuthology.orchestra.run.smithi092.stdout: pg 2.1e is active+undersized+degraded, acting [2,0] 2024-05-06T12:21:02.140 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:01 smithi092 ceph-mon[91933]: from='client.34374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:21:02.140 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:01 smithi092 ceph-mon[91933]: from='client.34378 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:21:02.140 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:01 smithi092 ceph-mon[91933]: pgmap v164: 65 pgs: 1 peering, 14 active+undersized+degraded, 13 active+undersized, 37 active+clean; 323 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s; 56/372 objects degraded (15.054%) 2024-05-06T12:21:02.141 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:01 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/124765045' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:02.141 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:01 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:02.141 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:01 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:02.141 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:01 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:21:02.141 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:01 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:21:02.141 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:01 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:02.141 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:01 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/1948904589' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:21:02.141 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:01 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:21:02.141 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:01 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:02.141 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:01 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:02.141 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:01 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:02.141 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:01 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:02.141 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:01 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:02.142 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:01 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-05-06T12:21:02.142 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:01 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-05-06T12:21:02.142 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:01 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-05-06T12:21:02.142 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:01 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-05-06T12:21:02.142 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:01 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-05-06T12:21:02.142 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:01 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-05-06T12:21:02.142 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:01 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-05-06T12:21:02.143 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:01 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-05-06T12:21:02.145 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:01 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-05-06T12:21:02.145 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:01 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-05-06T12:21:02.145 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:01 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-05-06T12:21:02.145 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:01 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-05-06T12:21:02.145 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:01 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-05-06T12:21:02.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:02.137+0000 7f137dae7700 1 -- 172.21.15.92:0/2321403096 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f136006e140 msgr2=0x7f13600705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:02.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:02.137+0000 7f137dae7700 1 --2- 172.21.15.92:0/2321403096 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f136006e140 0x7f13600705f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f136800b970 tx=0x7f1368009e30 comp rx=0 tx=0).stop 2024-05-06T12:21:02.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:02.137+0000 7f137dae7700 1 -- 172.21.15.92:0/2321403096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13780fcf90 msgr2=0x7f1378112750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:02.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:02.137+0000 7f137dae7700 1 --2- 172.21.15.92:0/2321403096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13780fcf90 0x7f1378112750 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f136c00b790 tx=0x7f136c009250 comp rx=0 tx=0).stop 2024-05-06T12:21:02.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:02.138+0000 7f137dae7700 1 -- 172.21.15.92:0/2321403096 shutdown_connections 2024-05-06T12:21:02.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:02.138+0000 7f137dae7700 1 --2- 172.21.15.92:0/2321403096 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f136006e140 0x7f13600705f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:02.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:02.138+0000 7f137dae7700 1 --2- 172.21.15.92:0/2321403096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13780fbce0 0x7f1378066cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:02.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:02.138+0000 7f137dae7700 1 --2- 172.21.15.92:0/2321403096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13780fcf90 0x7f1378112750 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:02.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:02.138+0000 7f137dae7700 1 -- 172.21.15.92:0/2321403096 >> 172.21.15.92:0/2321403096 conn(0x7f13780f7480 msgr2=0x7f13780f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:21:02.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:02.138+0000 7f137dae7700 1 -- 172.21.15.92:0/2321403096 shutdown_connections 2024-05-06T12:21:02.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:02.138+0000 7f137dae7700 1 -- 172.21.15.92:0/2321403096 wait complete. 2024-05-06T12:21:03.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:02 smithi190 ceph-mon[79744]: Upgrade: Setting container_image for all osd 2024-05-06T12:21:03.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:02 smithi190 ceph-mon[79744]: Upgrade: Setting require_osd_release to 19 squid 2024-05-06T12:21:03.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:02 smithi190 ceph-mon[79744]: from='client.34390 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:21:03.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:02 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/2321403096' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:21:03.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:02 smithi190 ceph-mon[79744]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-05-06T12:21:03.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:02 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-05-06T12:21:03.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:02 smithi190 ceph-mon[79744]: osdmap e81: 6 total, 6 up, 6 in 2024-05-06T12:21:03.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:02 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:03.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:02 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-05-06T12:21:03.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:02 smithi092 ceph-mon[91933]: Upgrade: Setting container_image for all osd 2024-05-06T12:21:03.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:02 smithi092 ceph-mon[91933]: Upgrade: Setting require_osd_release to 19 squid 2024-05-06T12:21:03.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:02 smithi092 ceph-mon[91933]: from='client.34390 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:21:03.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:02 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/2321403096' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:21:03.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:02 smithi092 ceph-mon[91933]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-05-06T12:21:03.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:02 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-05-06T12:21:03.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:02 smithi092 ceph-mon[91933]: osdmap e81: 6 total, 6 up, 6 in 2024-05-06T12:21:03.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:02 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:03.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:02 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-05-06T12:21:04.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:03 smithi190 ceph-mon[79744]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-05-06T12:21:04.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:03 smithi190 ceph-mon[79744]: osdmap e82: 6 total, 6 up, 6 in 2024-05-06T12:21:04.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:03 smithi190 ceph-mon[79744]: pgmap v167: 65 pgs: 1 peering, 5 active+undersized+degraded, 8 active+undersized, 51 active+clean; 323 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 2 op/s; 18/372 objects degraded (4.839%) 2024-05-06T12:21:04.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:03 smithi190 ceph-mon[79744]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-05-06T12:21:04.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:03 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-05-06T12:21:04.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:03 smithi190 ceph-mon[79744]: fsmap cephfs:2 {0=cephfs.smithi092.rosbll=up:active,1=cephfs.smithi190.byjvgf=up:active} 2024-05-06T12:21:04.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:03 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:21:04.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:03 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:21:04.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:03 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:21:04.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:03 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:04.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:03 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:21:04.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:03 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:04.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:03 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:04.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:03 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:04.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:03 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:04.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:03 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:04.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:03 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-05-06T12:21:04.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:03 smithi092 ceph-mon[91933]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-05-06T12:21:04.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:03 smithi092 ceph-mon[91933]: osdmap e82: 6 total, 6 up, 6 in 2024-05-06T12:21:04.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:03 smithi092 ceph-mon[91933]: pgmap v167: 65 pgs: 1 peering, 5 active+undersized+degraded, 8 active+undersized, 51 active+clean; 323 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 2 op/s; 18/372 objects degraded (4.839%) 2024-05-06T12:21:04.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:03 smithi092 ceph-mon[91933]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-05-06T12:21:04.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:03 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-05-06T12:21:04.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:03 smithi092 ceph-mon[91933]: fsmap cephfs:2 {0=cephfs.smithi092.rosbll=up:active,1=cephfs.smithi190.byjvgf=up:active} 2024-05-06T12:21:04.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:03 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:21:04.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:03 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:21:04.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:03 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:21:04.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:03 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:04.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:03 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:21:04.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:03 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:04.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:03 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:04.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:03 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:04.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:03 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:04.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:03 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:04.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:03 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-05-06T12:21:05.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:04 smithi190 ceph-mon[79744]: Upgrade: Scaling down filesystem cephfs 2024-05-06T12:21:05.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:04 smithi190 ceph-mon[79744]: Health check update: Degraded data redundancy: 18/372 objects degraded (4.839%), 5 pgs degraded (PG_DEGRADED) 2024-05-06T12:21:05.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:04 smithi190 ceph-mon[79744]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-05-06T12:21:05.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:04 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-05-06T12:21:05.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:04 smithi190 ceph-mon[79744]: mds.? [v2:172.21.15.190:6826/1642657019,v1:172.21.15.190:6827/1642657019] up:boot 2024-05-06T12:21:05.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:04 smithi190 ceph-mon[79744]: mds.? [v2:172.21.15.92:6826/3131141280,v1:172.21.15.92:6827/3131141280] up:boot 2024-05-06T12:21:05.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:04 smithi190 ceph-mon[79744]: stopping daemon mds.cephfs.smithi190.byjvgf 2024-05-06T12:21:05.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:04 smithi190 ceph-mon[79744]: fsmap cephfs:2 {0=cephfs.smithi092.rosbll=up:active,1=cephfs.smithi190.byjvgf=up:active} 2 up:standby 2024-05-06T12:21:05.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:04 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi092.ijsqgp"}]: dispatch 2024-05-06T12:21:05.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:04 smithi190 ceph-mon[79744]: fsmap cephfs:2 {0=cephfs.smithi092.rosbll=up:active,1=cephfs.smithi190.byjvgf=up:stopping} 2 up:standby 2024-05-06T12:21:05.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:04 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi190.idzoff"}]: dispatch 2024-05-06T12:21:05.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:04 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:21:05.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:04 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:21:05.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:04 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:21:05.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:04 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:05.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:04 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:21:05.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:04 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:05.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:04 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:05.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:04 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:05.100 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:04 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:05.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:04 smithi092 ceph-mon[91933]: Upgrade: Scaling down filesystem cephfs 2024-05-06T12:21:05.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:04 smithi092 ceph-mon[91933]: Health check update: Degraded data redundancy: 18/372 objects degraded (4.839%), 5 pgs degraded (PG_DEGRADED) 2024-05-06T12:21:05.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:04 smithi092 ceph-mon[91933]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-05-06T12:21:05.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:04 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-05-06T12:21:05.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:04 smithi092 ceph-mon[91933]: mds.? [v2:172.21.15.190:6826/1642657019,v1:172.21.15.190:6827/1642657019] up:boot 2024-05-06T12:21:05.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:04 smithi092 ceph-mon[91933]: mds.? [v2:172.21.15.92:6826/3131141280,v1:172.21.15.92:6827/3131141280] up:boot 2024-05-06T12:21:05.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:04 smithi092 ceph-mon[91933]: stopping daemon mds.cephfs.smithi190.byjvgf 2024-05-06T12:21:05.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:04 smithi092 ceph-mon[91933]: fsmap cephfs:2 {0=cephfs.smithi092.rosbll=up:active,1=cephfs.smithi190.byjvgf=up:active} 2 up:standby 2024-05-06T12:21:05.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:04 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi092.ijsqgp"}]: dispatch 2024-05-06T12:21:05.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:04 smithi092 ceph-mon[91933]: fsmap cephfs:2 {0=cephfs.smithi092.rosbll=up:active,1=cephfs.smithi190.byjvgf=up:stopping} 2 up:standby 2024-05-06T12:21:05.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:04 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi190.idzoff"}]: dispatch 2024-05-06T12:21:05.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:04 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:21:05.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:04 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:21:05.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:04 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:21:05.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:04 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:05.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:04 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:21:05.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:04 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:05.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:04 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:05.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:04 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:05.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:04 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:06.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:05 smithi190 ceph-mon[79744]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-05-06T12:21:06.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:05 smithi190 ceph-mon[79744]: pgmap v168: 65 pgs: 1 peering, 64 active+clean; 323 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-05-06T12:21:06.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:05 smithi092 ceph-mon[91933]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-05-06T12:21:06.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:05 smithi092 ceph-mon[91933]: pgmap v168: 65 pgs: 1 peering, 64 active+clean; 323 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-05-06T12:21:07.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:06 smithi190 ceph-mon[79744]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/372 objects degraded (4.839%), 5 pgs degraded) 2024-05-06T12:21:07.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:06 smithi092 ceph-mon[91933]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/372 objects degraded (4.839%), 5 pgs degraded) 2024-05-06T12:21:08.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:07 smithi190 ceph-mon[79744]: pgmap v169: 65 pgs: 65 active+clean; 323 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-06T12:21:08.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:07 smithi092 ceph-mon[91933]: pgmap v169: 65 pgs: 65 active+clean; 323 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-06T12:21:10.165 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:09 smithi190 ceph-mon[79744]: pgmap v170: 65 pgs: 65 active+clean; 323 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 127 B/s wr, 0 op/s 2024-05-06T12:21:10.165 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:10.165 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:21:10.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:09 smithi092 ceph-mon[91933]: pgmap v170: 65 pgs: 65 active+clean; 323 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 127 B/s wr, 0 op/s 2024-05-06T12:21:10.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:10.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:21:12.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:11 smithi092 ceph-mon[91933]: pgmap v171: 65 pgs: 65 active+clean; 323 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 489 B/s wr, 0 op/s 2024-05-06T12:21:12.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:11 smithi190 ceph-mon[79744]: pgmap v171: 65 pgs: 65 active+clean; 323 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 489 B/s wr, 0 op/s 2024-05-06T12:21:14.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:13 smithi092 ceph-mon[91933]: pgmap v172: 65 pgs: 65 active+clean; 287 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 510 B/s wr, 4 op/s 2024-05-06T12:21:14.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:13 smithi190 ceph-mon[79744]: pgmap v172: 65 pgs: 65 active+clean; 287 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 510 B/s wr, 4 op/s 2024-05-06T12:21:15.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:15 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:15.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:15 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:21:15.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:15 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:21:15.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:15 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:21:15.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:15 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:15.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:15 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:21:15.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:15 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:15.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:15 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:15.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:15 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:15.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:15 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:15.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:15 smithi092 ceph-mon[91933]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-05-06T12:21:15.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:15 smithi092 ceph-mon[91933]: pgmap v173: 65 pgs: 65 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 938 B/s wr, 5 op/s 2024-05-06T12:21:15.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:15 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:15.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:15 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:21:15.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:15 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:21:15.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:15 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:21:15.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:15 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:15.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:15 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:21:15.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:15 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:15.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:15 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:15.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:15 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:15.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:15 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:15.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:15 smithi190 ceph-mon[79744]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-05-06T12:21:15.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:15 smithi190 ceph-mon[79744]: pgmap v173: 65 pgs: 65 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 938 B/s wr, 5 op/s 2024-05-06T12:21:18.176 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:17 smithi092 ceph-mon[91933]: pgmap v174: 65 pgs: 65 active+clean; 267 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 1023 B/s wr, 6 op/s 2024-05-06T12:21:18.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:17 smithi190 ceph-mon[79744]: pgmap v174: 65 pgs: 65 active+clean; 267 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 1023 B/s wr, 6 op/s 2024-05-06T12:21:20.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:19 smithi092 ceph-mon[91933]: pgmap v175: 65 pgs: 65 active+clean; 187 MiB data, 1008 MiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 1.2 KiB/s wr, 8 op/s 2024-05-06T12:21:20.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:19 smithi190 ceph-mon[79744]: pgmap v175: 65 pgs: 65 active+clean; 187 MiB data, 1008 MiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 1.2 KiB/s wr, 8 op/s 2024-05-06T12:21:22.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:21 smithi092 ceph-mon[91933]: pgmap v176: 65 pgs: 65 active+clean; 179 MiB data, 948 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1.1 KiB/s wr, 8 op/s 2024-05-06T12:21:22.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:21 smithi190 ceph-mon[79744]: pgmap v176: 65 pgs: 65 active+clean; 179 MiB data, 948 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1.1 KiB/s wr, 8 op/s 2024-05-06T12:21:24.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:23 smithi092 ceph-mon[91933]: pgmap v177: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 853 B/s wr, 8 op/s 2024-05-06T12:21:24.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:23 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:21:24.354 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:23 smithi190 ceph-mon[79744]: pgmap v177: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 853 B/s wr, 8 op/s 2024-05-06T12:21:24.355 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:23 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:21:25.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:24 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:21:25.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:24 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:21:26.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:25 smithi092 ceph-mon[91933]: pgmap v178: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1023 B/s wr, 4 op/s 2024-05-06T12:21:26.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:25 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:21:26.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:25 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:21:26.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:25 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:26.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:25 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:21:26.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:25 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:26.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:25 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:26.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:25 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:26.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:25 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:26.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:25 smithi190 ceph-mon[79744]: pgmap v178: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1023 B/s wr, 4 op/s 2024-05-06T12:21:26.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:25 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:21:26.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:25 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:21:26.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:25 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:26.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:25 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:21:26.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:25 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:26.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:25 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:26.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:25 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:26.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:25 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:27.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:26 smithi092 ceph-mon[91933]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-05-06T12:21:27.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:26 smithi092 ceph-mon[91933]: daemon mds.cephfs.smithi190.byjvgf finished stopping rank 1 in filesystem cephfs (now has 1 ranks) 2024-05-06T12:21:27.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:26 smithi190 ceph-mon[79744]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-05-06T12:21:27.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:26 smithi190 ceph-mon[79744]: daemon mds.cephfs.smithi190.byjvgf finished stopping rank 1 in filesystem cephfs (now has 1 ranks) 2024-05-06T12:21:28.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:27 smithi092 ceph-mon[91933]: pgmap v179: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 511 B/s wr, 2 op/s 2024-05-06T12:21:28.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:27 smithi092 ceph-mon[91933]: mds.1 [v2:172.21.15.190:6824/1565151699,v1:172.21.15.190:6825/1565151699] down:stopped 2024-05-06T12:21:28.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:27 smithi092 ceph-mon[91933]: fsmap cephfs:1 {0=cephfs.smithi092.rosbll=up:active} 2 up:standby 2024-05-06T12:21:28.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:27 smithi190 ceph-mon[79744]: pgmap v179: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 511 B/s wr, 2 op/s 2024-05-06T12:21:28.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:27 smithi190 ceph-mon[79744]: mds.1 [v2:172.21.15.190:6824/1565151699,v1:172.21.15.190:6825/1565151699] down:stopped 2024-05-06T12:21:28.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:27 smithi190 ceph-mon[79744]: fsmap cephfs:1 {0=cephfs.smithi092.rosbll=up:active} 2 up:standby 2024-05-06T12:21:29.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:28 smithi092 ceph-mon[91933]: mds.? [v2:172.21.15.190:6824/3031977277,v1:172.21.15.190:6825/3031977277] up:boot 2024-05-06T12:21:29.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:28 smithi092 ceph-mon[91933]: fsmap cephfs:1 {0=cephfs.smithi092.rosbll=up:active} 3 up:standby 2024-05-06T12:21:29.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:28 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi190.byjvgf"}]: dispatch 2024-05-06T12:21:29.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:28 smithi190 ceph-mon[79744]: mds.? [v2:172.21.15.190:6824/3031977277,v1:172.21.15.190:6825/3031977277] up:boot 2024-05-06T12:21:29.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:28 smithi190 ceph-mon[79744]: fsmap cephfs:1 {0=cephfs.smithi092.rosbll=up:active} 3 up:standby 2024-05-06T12:21:29.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:28 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi190.byjvgf"}]: dispatch 2024-05-06T12:21:30.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:29 smithi092 ceph-mon[91933]: pgmap v180: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 2 op/s 2024-05-06T12:21:30.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:29 smithi190 ceph-mon[79744]: pgmap v180: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 2 op/s 2024-05-06T12:21:32.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:31 smithi092 ceph-mon[91933]: pgmap v181: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-05-06T12:21:32.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.250+0000 7f66abbbc700 1 -- 172.21.15.92:0/1945217978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66a4066f70 msgr2=0x7f66a40ff9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:32.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.250+0000 7f66abbbc700 1 --2- 172.21.15.92:0/1945217978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66a4066f70 0x7f66a40ff9d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6694004940 tx=0x7f6694009e30 comp rx=0 tx=0).stop 2024-05-06T12:21:32.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.250+0000 7f66abbbc700 1 -- 172.21.15.92:0/1945217978 shutdown_connections 2024-05-06T12:21:32.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.250+0000 7f66abbbc700 1 --2- 172.21.15.92:0/1945217978 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66a40fff10 0x7f66a4102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:32.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.250+0000 7f66abbbc700 1 --2- 172.21.15.92:0/1945217978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66a4066f70 0x7f66a40ff9d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:32.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.251+0000 7f66abbbc700 1 -- 172.21.15.92:0/1945217978 >> 172.21.15.92:0/1945217978 conn(0x7f66a40f7440 msgr2=0x7f66a40f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:21:32.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.251+0000 7f66abbbc700 1 -- 172.21.15.92:0/1945217978 shutdown_connections 2024-05-06T12:21:32.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.251+0000 7f66abbbc700 1 -- 172.21.15.92:0/1945217978 wait complete. 2024-05-06T12:21:32.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.251+0000 7f66abbbc700 1 Processor -- start 2024-05-06T12:21:32.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.251+0000 7f66abbbc700 1 -- start start 2024-05-06T12:21:32.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.251+0000 7f66abbbc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66a4066f70 0x7f66a410d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:32.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.251+0000 7f66abbbc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66a40fff10 0x7f66a410ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:32.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.251+0000 7f66abbbc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a410e560 con 0x7f66a4066f70 2024-05-06T12:21:32.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.251+0000 7f66abbbc700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a4112eb0 con 0x7f66a40fff10 2024-05-06T12:21:32.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.252+0000 7f66a9958700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66a4066f70 0x7f66a410d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:32.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.252+0000 7f66a9958700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66a4066f70 0x7f66a410d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40284/0 (socket says 172.21.15.92:40284) 2024-05-06T12:21:32.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.252+0000 7f66a9958700 1 -- 172.21.15.92:0/1539796888 learned_addr learned my addr 172.21.15.92:0/1539796888 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:21:32.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.252+0000 7f66a9157700 1 --2- 172.21.15.92:0/1539796888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66a40fff10 0x7f66a410ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:32.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.252+0000 7f66a9958700 1 -- 172.21.15.92:0/1539796888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66a40fff10 msgr2=0x7f66a410ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:32.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.252+0000 7f66a9958700 1 --2- 172.21.15.92:0/1539796888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66a40fff10 0x7f66a410ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:32.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.252+0000 7f66a9958700 1 -- 172.21.15.92:0/1539796888 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6694005040 con 0x7f66a4066f70 2024-05-06T12:21:32.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.252+0000 7f66a9157700 1 --2- 172.21.15.92:0/1539796888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66a40fff10 0x7f66a410ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-06T12:21:32.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.252+0000 7f66a9958700 1 --2- 172.21.15.92:0/1539796888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66a4066f70 0x7f66a410d990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f669400b3b0 tx=0x7f6694009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:21:32.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.252+0000 7f669a7fc700 1 -- 172.21.15.92:0/1539796888 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f669400e040 con 0x7f66a4066f70 2024-05-06T12:21:32.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.252+0000 7f66abbbc700 1 -- 172.21.15.92:0/1539796888 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66a41130b0 con 0x7f66a4066f70 2024-05-06T12:21:32.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.252+0000 7f669a7fc700 1 -- 172.21.15.92:0/1539796888 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f669400bcc0 con 0x7f66a4066f70 2024-05-06T12:21:32.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.252+0000 7f669a7fc700 1 -- 172.21.15.92:0/1539796888 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f669401c630 con 0x7f66a4066f70 2024-05-06T12:21:32.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.252+0000 7f66abbbc700 1 -- 172.21.15.92:0/1539796888 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a4113620 con 0x7f66a4066f70 2024-05-06T12:21:32.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.253+0000 7f66abbbc700 1 -- 172.21.15.92:0/1539796888 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66a404c710 con 0x7f66a4066f70 2024-05-06T12:21:32.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.254+0000 7f669a7fc700 1 -- 172.21.15.92:0/1539796888 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f66940139f0 con 0x7f66a4066f70 2024-05-06T12:21:32.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.254+0000 7f669a7fc700 1 --2- 172.21.15.92:0/1539796888 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f669006e140 0x7f66900705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:32.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.254+0000 7f669a7fc700 1 -- 172.21.15.92:0/1539796888 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f669408e7e0 con 0x7f66a4066f70 2024-05-06T12:21:32.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.254+0000 7f66a9157700 1 --2- 172.21.15.92:0/1539796888 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f669006e140 0x7f66900705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:32.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.255+0000 7f66a9157700 1 --2- 172.21.15.92:0/1539796888 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f669006e140 0x7f66900705f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f66a0006fd0 tx=0x7f66a0008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:21:32.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.259+0000 7f669a7fc700 1 -- 172.21.15.92:0/1539796888 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f669405b710 con 0x7f66a4066f70 2024-05-06T12:21:32.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:31 smithi190 ceph-mon[79744]: pgmap v181: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-05-06T12:21:32.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.478+0000 7f66abbbc700 1 -- 172.21.15.92:0/1539796888 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f66a40f8f80 con 0x7f669006e140 2024-05-06T12:21:32.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.479+0000 7f669a7fc700 1 -- 172.21.15.92:0/1539796888 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f66a40f8f80 con 0x7f669006e140 2024-05-06T12:21:32.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.481+0000 7f66abbbc700 1 -- 172.21.15.92:0/1539796888 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f669006e140 msgr2=0x7f66900705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:32.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.481+0000 7f66abbbc700 1 --2- 172.21.15.92:0/1539796888 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f669006e140 0x7f66900705f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f66a0006fd0 tx=0x7f66a0008040 comp rx=0 tx=0).stop 2024-05-06T12:21:32.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.481+0000 7f66abbbc700 1 -- 172.21.15.92:0/1539796888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66a4066f70 msgr2=0x7f66a410d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:32.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.481+0000 7f66abbbc700 1 --2- 172.21.15.92:0/1539796888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66a4066f70 0x7f66a410d990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f669400b3b0 tx=0x7f6694009e30 comp rx=0 tx=0).stop 2024-05-06T12:21:32.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.481+0000 7f66abbbc700 1 -- 172.21.15.92:0/1539796888 shutdown_connections 2024-05-06T12:21:32.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.481+0000 7f66abbbc700 1 --2- 172.21.15.92:0/1539796888 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f669006e140 0x7f66900705f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:32.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.481+0000 7f66abbbc700 1 --2- 172.21.15.92:0/1539796888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66a4066f70 0x7f66a410d990 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:32.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.481+0000 7f66abbbc700 1 --2- 172.21.15.92:0/1539796888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66a40fff10 0x7f66a410ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:32.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.481+0000 7f66abbbc700 1 -- 172.21.15.92:0/1539796888 >> 172.21.15.92:0/1539796888 conn(0x7f66a40f7440 msgr2=0x7f66a40f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:21:32.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.481+0000 7f66abbbc700 1 -- 172.21.15.92:0/1539796888 shutdown_connections 2024-05-06T12:21:32.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.481+0000 7f66abbbc700 1 -- 172.21.15.92:0/1539796888 wait complete. 2024-05-06T12:21:32.494 INFO:teuthology.orchestra.run.smithi092.stdout:true 2024-05-06T12:21:32.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.583+0000 7f4232717700 1 -- 172.21.15.92:0/3826446025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f422c0fce60 msgr2=0x7f422c0fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:32.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.583+0000 7f4232717700 1 --2- 172.21.15.92:0/3826446025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f422c0fce60 0x7f422c0fd270 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f421c004940 tx=0x7f421c009e30 comp rx=0 tx=0).stop 2024-05-06T12:21:32.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.583+0000 7f4232717700 1 -- 172.21.15.92:0/3826446025 shutdown_connections 2024-05-06T12:21:32.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.583+0000 7f4232717700 1 --2- 172.21.15.92:0/3826446025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f422c0fdfd0 0x7f422c0fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:32.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.583+0000 7f4232717700 1 --2- 172.21.15.92:0/3826446025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f422c0fce60 0x7f422c0fd270 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:32.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.583+0000 7f4232717700 1 -- 172.21.15.92:0/3826446025 >> 172.21.15.92:0/3826446025 conn(0x7f422c0f8540 msgr2=0x7f422c0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:21:32.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.583+0000 7f4232717700 1 -- 172.21.15.92:0/3826446025 shutdown_connections 2024-05-06T12:21:32.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.583+0000 7f4232717700 1 -- 172.21.15.92:0/3826446025 wait complete. 2024-05-06T12:21:32.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.583+0000 7f4232717700 1 Processor -- start 2024-05-06T12:21:32.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.583+0000 7f4232717700 1 -- start start 2024-05-06T12:21:32.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.584+0000 7f4232717700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f422c0fce60 0x7f422c10d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:32.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.584+0000 7f4232717700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f422c0fdfd0 0x7f422c10de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:32.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.584+0000 7f4232717700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c10e480 con 0x7f422c0fce60 2024-05-06T12:21:32.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.584+0000 7f4232717700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c10e5f0 con 0x7f422c0fdfd0 2024-05-06T12:21:32.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.584+0000 7f422bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f422c0fce60 0x7f422c10d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:32.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.584+0000 7f422bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f422c0fce60 0x7f422c10d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39826/0 (socket says 172.21.15.92:39826) 2024-05-06T12:21:32.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.584+0000 7f422bfff700 1 -- 172.21.15.92:0/628765855 learned_addr learned my addr 172.21.15.92:0/628765855 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:21:32.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.584+0000 7f422b7fe700 1 --2- 172.21.15.92:0/628765855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f422c0fdfd0 0x7f422c10de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:32.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.584+0000 7f422bfff700 1 -- 172.21.15.92:0/628765855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f422c0fdfd0 msgr2=0x7f422c10de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:32.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.584+0000 7f422bfff700 1 --2- 172.21.15.92:0/628765855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f422c0fdfd0 0x7f422c10de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:32.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.584+0000 7f422bfff700 1 -- 172.21.15.92:0/628765855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f421c005040 con 0x7f422c0fce60 2024-05-06T12:21:32.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.584+0000 7f422bfff700 1 --2- 172.21.15.92:0/628765855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f422c0fce60 0x7f422c10d940 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f421c00b3b0 tx=0x7f421c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:21:32.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.584+0000 7f4228ff9700 1 -- 172.21.15.92:0/628765855 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f421c00e040 con 0x7f422c0fce60 2024-05-06T12:21:32.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.585+0000 7f4228ff9700 1 -- 172.21.15.92:0/628765855 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f421c00bbf0 con 0x7f422c0fce60 2024-05-06T12:21:32.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.585+0000 7f4232717700 1 -- 172.21.15.92:0/628765855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f422c112ff0 con 0x7f422c0fce60 2024-05-06T12:21:32.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.585+0000 7f4228ff9700 1 -- 172.21.15.92:0/628765855 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f421c01c630 con 0x7f422c0fce60 2024-05-06T12:21:32.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.585+0000 7f4232717700 1 -- 172.21.15.92:0/628765855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f422c113520 con 0x7f422c0fce60 2024-05-06T12:21:32.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.586+0000 7f4232717700 1 -- 172.21.15.92:0/628765855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f422c04c710 con 0x7f422c0fce60 2024-05-06T12:21:32.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.587+0000 7f4228ff9700 1 -- 172.21.15.92:0/628765855 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f421c00bd60 con 0x7f422c0fce60 2024-05-06T12:21:32.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.587+0000 7f4228ff9700 1 --2- 172.21.15.92:0/628765855 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f421406e140 0x7f42140705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:32.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.587+0000 7f4228ff9700 1 -- 172.21.15.92:0/628765855 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f421c08e5f0 con 0x7f422c0fce60 2024-05-06T12:21:32.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.587+0000 7f422b7fe700 1 --2- 172.21.15.92:0/628765855 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f421406e140 0x7f42140705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:32.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.588+0000 7f422b7fe700 1 --2- 172.21.15.92:0/628765855 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f421406e140 0x7f42140705f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4220004f40 tx=0x7f422000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:21:32.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.592+0000 7f4228ff9700 1 -- 172.21.15.92:0/628765855 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f421c05b5c0 con 0x7f422c0fce60 2024-05-06T12:21:32.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.806+0000 7f4232717700 1 -- 172.21.15.92:0/628765855 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f422c113790 con 0x7f421406e140 2024-05-06T12:21:32.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.808+0000 7f4228ff9700 1 -- 172.21.15.92:0/628765855 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f422c113790 con 0x7f421406e140 2024-05-06T12:21:32.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.810+0000 7f4232717700 1 -- 172.21.15.92:0/628765855 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f421406e140 msgr2=0x7f42140705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:32.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.810+0000 7f4232717700 1 --2- 172.21.15.92:0/628765855 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f421406e140 0x7f42140705f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4220004f40 tx=0x7f422000b040 comp rx=0 tx=0).stop 2024-05-06T12:21:32.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.810+0000 7f4232717700 1 -- 172.21.15.92:0/628765855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f422c0fce60 msgr2=0x7f422c10d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:32.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.810+0000 7f4232717700 1 --2- 172.21.15.92:0/628765855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f422c0fce60 0x7f422c10d940 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f421c00b3b0 tx=0x7f421c009e30 comp rx=0 tx=0).stop 2024-05-06T12:21:32.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.810+0000 7f4232717700 1 -- 172.21.15.92:0/628765855 shutdown_connections 2024-05-06T12:21:32.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.810+0000 7f4232717700 1 --2- 172.21.15.92:0/628765855 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f421406e140 0x7f42140705f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:32.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.810+0000 7f4232717700 1 --2- 172.21.15.92:0/628765855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f422c0fce60 0x7f422c10d940 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:32.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.810+0000 7f4232717700 1 --2- 172.21.15.92:0/628765855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f422c0fdfd0 0x7f422c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:32.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.810+0000 7f4232717700 1 -- 172.21.15.92:0/628765855 >> 172.21.15.92:0/628765855 conn(0x7f422c0f8540 msgr2=0x7f422c0f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:21:32.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.810+0000 7f4232717700 1 -- 172.21.15.92:0/628765855 shutdown_connections 2024-05-06T12:21:32.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.810+0000 7f4232717700 1 -- 172.21.15.92:0/628765855 wait complete. 2024-05-06T12:21:32.915 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.913+0000 7f59fa820700 1 -- 172.21.15.92:0/1921281429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59f4102360 msgr2=0x7f59f4104740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:32.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.913+0000 7f59fa820700 1 --2- 172.21.15.92:0/1921281429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59f4102360 0x7f59f4104740 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f59e8004940 tx=0x7f59e8009e30 comp rx=0 tx=0).stop 2024-05-06T12:21:32.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.913+0000 7f59fa820700 1 -- 172.21.15.92:0/1921281429 shutdown_connections 2024-05-06T12:21:32.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.913+0000 7f59fa820700 1 --2- 172.21.15.92:0/1921281429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59f4102360 0x7f59f4104740 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:32.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.913+0000 7f59fa820700 1 --2- 172.21.15.92:0/1921281429 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59f4066240 0x7f59f4101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:32.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.913+0000 7f59fa820700 1 -- 172.21.15.92:0/1921281429 >> 172.21.15.92:0/1921281429 conn(0x7f59f40f7430 msgr2=0x7f59f40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:21:32.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.913+0000 7f59fa820700 1 -- 172.21.15.92:0/1921281429 shutdown_connections 2024-05-06T12:21:32.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.913+0000 7f59fa820700 1 -- 172.21.15.92:0/1921281429 wait complete. 2024-05-06T12:21:32.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.913+0000 7f59fa820700 1 Processor -- start 2024-05-06T12:21:32.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.914+0000 7f59fa820700 1 -- start start 2024-05-06T12:21:32.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.914+0000 7f59fa820700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59f4066240 0x7f59f410fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:32.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.914+0000 7f59fa820700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59f4102360 0x7f59f4110270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:32.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.914+0000 7f59fa820700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f4110870 con 0x7f59f4066240 2024-05-06T12:21:32.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.914+0000 7f59fa820700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f4110fb0 con 0x7f59f4102360 2024-05-06T12:21:32.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.914+0000 7f59f3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59f4066240 0x7f59f410fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:32.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.914+0000 7f59f3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59f4066240 0x7f59f410fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39834/0 (socket says 172.21.15.92:39834) 2024-05-06T12:21:32.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.914+0000 7f59f3fff700 1 -- 172.21.15.92:0/2478411453 learned_addr learned my addr 172.21.15.92:0/2478411453 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:21:32.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.914+0000 7f59f37fe700 1 --2- 172.21.15.92:0/2478411453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59f4102360 0x7f59f4110270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:32.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.914+0000 7f59f3fff700 1 -- 172.21.15.92:0/2478411453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59f4102360 msgr2=0x7f59f4110270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:32.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.914+0000 7f59f3fff700 1 --2- 172.21.15.92:0/2478411453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59f4102360 0x7f59f4110270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:32.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.914+0000 7f59f3fff700 1 -- 172.21.15.92:0/2478411453 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e8005040 con 0x7f59f4066240 2024-05-06T12:21:32.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.915+0000 7f59f3fff700 1 --2- 172.21.15.92:0/2478411453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59f4066240 0x7f59f410fd30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f59e400c8b0 tx=0x7f59e4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:21:32.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.915+0000 7f59f0ff9700 1 -- 172.21.15.92:0/2478411453 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59e40106f0 con 0x7f59f4066240 2024-05-06T12:21:32.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.915+0000 7f59fa820700 1 -- 172.21.15.92:0/2478411453 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59f4111290 con 0x7f59f4066240 2024-05-06T12:21:32.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.915+0000 7f59f0ff9700 1 -- 172.21.15.92:0/2478411453 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f59e4010d20 con 0x7f59f4066240 2024-05-06T12:21:32.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.915+0000 7f59f0ff9700 1 -- 172.21.15.92:0/2478411453 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59e4012e90 con 0x7f59f4066240 2024-05-06T12:21:32.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.915+0000 7f59fa820700 1 -- 172.21.15.92:0/2478411453 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f4111850 con 0x7f59f4066240 2024-05-06T12:21:32.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.916+0000 7f59fa820700 1 -- 172.21.15.92:0/2478411453 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59f405f5c0 con 0x7f59f4066240 2024-05-06T12:21:32.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.916+0000 7f59f0ff9700 1 -- 172.21.15.92:0/2478411453 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f59e4010850 con 0x7f59f4066240 2024-05-06T12:21:32.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.917+0000 7f59f0ff9700 1 --2- 172.21.15.92:0/2478411453 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f59dc06e140 0x7f59dc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:32.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.917+0000 7f59f0ff9700 1 -- 172.21.15.92:0/2478411453 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f59e408da80 con 0x7f59f4066240 2024-05-06T12:21:32.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.917+0000 7f59f37fe700 1 --2- 172.21.15.92:0/2478411453 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f59dc06e140 0x7f59dc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:32.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.917+0000 7f59f37fe700 1 --2- 172.21.15.92:0/2478411453 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f59dc06e140 0x7f59dc0705f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f59e800b3b0 tx=0x7f59e8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:21:32.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:32.922+0000 7f59f0ff9700 1 -- 172.21.15.92:0/2478411453 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f59e405a930 con 0x7f59f4066240 2024-05-06T12:21:33.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.127+0000 7f59fa820700 1 -- 172.21.15.92:0/2478411453 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f59f4111c40 con 0x7f59dc06e140 2024-05-06T12:21:33.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.137+0000 7f59f0ff9700 1 -- 172.21.15.92:0/2478411453 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f59f4111c40 con 0x7f59dc06e140 2024-05-06T12:21:33.140 INFO:teuthology.orchestra.run.smithi092.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-06T12:21:33.140 INFO:teuthology.orchestra.run.smithi092.stdout:alertmanager.smithi092 smithi092 *:9093,9094 running (6m) 2m ago 15m 25.2M - 0.25.0 c8568f914cd2 be5637d7d25a 2024-05-06T12:21:33.140 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi092 smithi092 running (3m) 2m ago 15m 6660k - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 8da0494e1bdc 2024-05-06T12:21:33.140 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi190 smithi190 running (3m) 36s ago 14m 6665k - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 641a7cdaa56e 2024-05-06T12:21:33.140 INFO:teuthology.orchestra.run.smithi092.stdout:grafana.smithi092 smithi092 *:3000 running (6m) 2m ago 14m 75.0M - 9.4.7 954c08fa6188 9180cb3e5258 2024-05-06T12:21:33.140 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.ijsqgp smithi092 running (9m) 2m ago 9m 60.7M - 17.2.7-925-ge81df420 96d5ae7fd4b4 b4c2ed6d2c82 2024-05-06T12:21:33.141 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.rosbll smithi092 running (9m) 2m ago 9m 122M - 17.2.7-925-ge81df420 96d5ae7fd4b4 5f5426cd1bf5 2024-05-06T12:21:33.141 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.byjvgf smithi190 running (9m) 36s ago 9m 126M - 17.2.7-925-ge81df420 96d5ae7fd4b4 24581d2e1c5d 2024-05-06T12:21:33.141 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.idzoff smithi190 running (9m) 36s ago 9m 64.0M - 17.2.7-925-ge81df420 96d5ae7fd4b4 20f2641696ef 2024-05-06T12:21:33.141 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi092.gyzeqb smithi092 *:8443,9283,8765 running (6m) 2m ago 16m 504M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 de9d2198327a 2024-05-06T12:21:33.141 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi190.kczxkh smithi190 *:8443,9283,8765 running (5m) 36s ago 14m 451M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 e84ab3758ac7 2024-05-06T12:21:33.141 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi092 smithi092 running (5m) 2m ago 16m 46.0M 2048M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 ac07f7158087 2024-05-06T12:21:33.141 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi190 smithi190 running (4m) 36s ago 14m 41.7M 2048M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 7a3c08eb716f 2024-05-06T12:21:33.141 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi092 smithi092 *:9100 running (6m) 2m ago 15m 14.9M - 1.5.0 0da6a335fe13 c796446b206f 2024-05-06T12:21:33.141 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi190 smithi190 *:9100 running (6m) 36s ago 14m 16.5M - 1.5.0 0da6a335fe13 647e28111a76 2024-05-06T12:21:33.141 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 smithi092 running (3m) 2m ago 13m 82.0M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 8c96c75b25f4 2024-05-06T12:21:33.141 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 smithi092 running (2m) 2m ago 12m 82.0M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 e8007a510f08 2024-05-06T12:21:33.141 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 smithi092 running (2m) 2m ago 12m 14.8M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 2beca7448789 2024-05-06T12:21:33.141 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 smithi190 running (106s) 36s ago 12m 87.7M 2581M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 914ada199f93 2024-05-06T12:21:33.141 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 smithi190 running (73s) 36s ago 11m 82.6M 2581M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 371c0be776c7 2024-05-06T12:21:33.141 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 smithi190 running (39s) 36s ago 11m 12.0M 2581M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 067bc34cdfb1 2024-05-06T12:21:33.141 INFO:teuthology.orchestra.run.smithi092.stdout:prometheus.smithi092 smithi092 *:9095 running (5m) 2m ago 14m 54.4M - 2.43.0 a07b618ecd1d bc5445691527 2024-05-06T12:21:33.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.139+0000 7f59fa820700 1 -- 172.21.15.92:0/2478411453 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f59dc06e140 msgr2=0x7f59dc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:33.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.139+0000 7f59fa820700 1 --2- 172.21.15.92:0/2478411453 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f59dc06e140 0x7f59dc0705f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f59e800b3b0 tx=0x7f59e8009e30 comp rx=0 tx=0).stop 2024-05-06T12:21:33.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.139+0000 7f59fa820700 1 -- 172.21.15.92:0/2478411453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59f4066240 msgr2=0x7f59f410fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:33.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.139+0000 7f59fa820700 1 --2- 172.21.15.92:0/2478411453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59f4066240 0x7f59f410fd30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f59e400c8b0 tx=0x7f59e4004470 comp rx=0 tx=0).stop 2024-05-06T12:21:33.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.139+0000 7f59fa820700 1 -- 172.21.15.92:0/2478411453 shutdown_connections 2024-05-06T12:21:33.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.139+0000 7f59fa820700 1 --2- 172.21.15.92:0/2478411453 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f59dc06e140 0x7f59dc0705f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:33.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.139+0000 7f59fa820700 1 --2- 172.21.15.92:0/2478411453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59f4066240 0x7f59f410fd30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:33.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.139+0000 7f59fa820700 1 --2- 172.21.15.92:0/2478411453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59f4102360 0x7f59f4110270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:33.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.139+0000 7f59fa820700 1 -- 172.21.15.92:0/2478411453 >> 172.21.15.92:0/2478411453 conn(0x7f59f40f7430 msgr2=0x7f59f40f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:21:33.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.140+0000 7f59fa820700 1 -- 172.21.15.92:0/2478411453 shutdown_connections 2024-05-06T12:21:33.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.140+0000 7f59fa820700 1 -- 172.21.15.92:0/2478411453 wait complete. 2024-05-06T12:21:33.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.239+0000 7f68b3f3d700 1 -- 172.21.15.92:0/149186719 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68ac0fe430 msgr2=0x7f68ac0fe840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:33.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.239+0000 7f68b3f3d700 1 --2- 172.21.15.92:0/149186719 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68ac0fe430 0x7f68ac0fe840 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f68a8004940 tx=0x7f68a8009e30 comp rx=0 tx=0).stop 2024-05-06T12:21:33.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.239+0000 7f68b3f3d700 1 -- 172.21.15.92:0/149186719 shutdown_connections 2024-05-06T12:21:33.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.239+0000 7f68b3f3d700 1 --2- 172.21.15.92:0/149186719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68ac0fed80 0x7f68ac0faec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:33.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.239+0000 7f68b3f3d700 1 --2- 172.21.15.92:0/149186719 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68ac0fe430 0x7f68ac0fe840 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:33.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.239+0000 7f68b3f3d700 1 -- 172.21.15.92:0/149186719 >> 172.21.15.92:0/149186719 conn(0x7f68ac0f6bd0 msgr2=0x7f68ac0f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:21:33.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.239+0000 7f68b3f3d700 1 -- 172.21.15.92:0/149186719 shutdown_connections 2024-05-06T12:21:33.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.239+0000 7f68b3f3d700 1 -- 172.21.15.92:0/149186719 wait complete. 2024-05-06T12:21:33.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.239+0000 7f68b3f3d700 1 Processor -- start 2024-05-06T12:21:33.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.240+0000 7f68b3f3d700 1 -- start start 2024-05-06T12:21:33.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.240+0000 7f68b3f3d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68ac0fe430 0x7f68ac10d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:33.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.240+0000 7f68b3f3d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68ac0fed80 0x7f68ac10de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:33.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.240+0000 7f68b3f3d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac10e460 con 0x7f68ac0fed80 2024-05-06T12:21:33.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.240+0000 7f68b3f3d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac10e5d0 con 0x7f68ac0fe430 2024-05-06T12:21:33.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.240+0000 7f68b14d8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68ac0fed80 0x7f68ac10de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:33.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.240+0000 7f68b14d8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68ac0fed80 0x7f68ac10de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39850/0 (socket says 172.21.15.92:39850) 2024-05-06T12:21:33.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.240+0000 7f68b14d8700 1 -- 172.21.15.92:0/2837173071 learned_addr learned my addr 172.21.15.92:0/2837173071 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:21:33.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.240+0000 7f68b1cd9700 1 --2- 172.21.15.92:0/2837173071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68ac0fe430 0x7f68ac10d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:33.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.240+0000 7f68b14d8700 1 -- 172.21.15.92:0/2837173071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68ac0fe430 msgr2=0x7f68ac10d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:33.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.240+0000 7f68b14d8700 1 --2- 172.21.15.92:0/2837173071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68ac0fe430 0x7f68ac10d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:33.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.240+0000 7f68b14d8700 1 -- 172.21.15.92:0/2837173071 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a8005040 con 0x7f68ac0fed80 2024-05-06T12:21:33.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.240+0000 7f68b14d8700 1 --2- 172.21.15.92:0/2837173071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68ac0fed80 0x7f68ac10de60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f689c002a50 tx=0x7f689c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:21:33.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.240+0000 7f68a27fc700 1 -- 172.21.15.92:0/2837173071 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f689c007cc0 con 0x7f68ac0fed80 2024-05-06T12:21:33.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.240+0000 7f68a27fc700 1 -- 172.21.15.92:0/2837173071 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f689c00f040 con 0x7f68ac0fed80 2024-05-06T12:21:33.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.240+0000 7f68b3f3d700 1 -- 172.21.15.92:0/2837173071 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68ac113030 con 0x7f68ac0fed80 2024-05-06T12:21:33.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.240+0000 7f68a27fc700 1 -- 172.21.15.92:0/2837173071 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f689c009640 con 0x7f68ac0fed80 2024-05-06T12:21:33.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.240+0000 7f68b3f3d700 1 -- 172.21.15.92:0/2837173071 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68ac1135c0 con 0x7f68ac0fed80 2024-05-06T12:21:33.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.241+0000 7f68b3f3d700 1 -- 172.21.15.92:0/2837173071 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68ac04c710 con 0x7f68ac0fed80 2024-05-06T12:21:33.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.242+0000 7f68a27fc700 1 -- 172.21.15.92:0/2837173071 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f689c00d690 con 0x7f68ac0fed80 2024-05-06T12:21:33.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.242+0000 7f68a27fc700 1 --2- 172.21.15.92:0/2837173071 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f689806e140 0x7f68980705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:33.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.242+0000 7f68a27fc700 1 -- 172.21.15.92:0/2837173071 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f689c08d800 con 0x7f68ac0fed80 2024-05-06T12:21:33.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.242+0000 7f68b1cd9700 1 --2- 172.21.15.92:0/2837173071 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f689806e140 0x7f68980705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:33.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.243+0000 7f68b1cd9700 1 --2- 172.21.15.92:0/2837173071 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f689806e140 0x7f68980705f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f68a800b3b0 tx=0x7f68a8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:21:33.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.247+0000 7f68a27fc700 1 -- 172.21.15.92:0/2837173071 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f689c05a730 con 0x7f68ac0fed80 2024-05-06T12:21:33.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.530+0000 7f68b3f3d700 1 -- 172.21.15.92:0/2837173071 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f68ac1058e0 con 0x7f68ac0fed80 2024-05-06T12:21:33.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.531+0000 7f68a27fc700 1 -- 172.21.15.92:0/2837173071 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+733 (secure 0 0 0) 0x7f689c05a2c0 con 0x7f68ac0fed80 2024-05-06T12:21:33.534 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:21:33.534 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": { 2024-05-06T12:21:33.534 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:21:33.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:21:33.534 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": { 2024-05-06T12:21:33.534 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:21:33.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:21:33.534 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": { 2024-05-06T12:21:33.534 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 6 2024-05-06T12:21:33.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:21:33.535 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-05-06T12:21:33.535 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 4 2024-05-06T12:21:33.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:21:33.535 INFO:teuthology.orchestra.run.smithi092.stdout: "overall": { 2024-05-06T12:21:33.535 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 4, 2024-05-06T12:21:33.535 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 10 2024-05-06T12:21:33.535 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-06T12:21:33.535 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:21:33.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.534+0000 7f68b3f3d700 1 -- 172.21.15.92:0/2837173071 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f689806e140 msgr2=0x7f68980705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:33.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.534+0000 7f68b3f3d700 1 --2- 172.21.15.92:0/2837173071 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f689806e140 0x7f68980705f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f68a800b3b0 tx=0x7f68a8009e30 comp rx=0 tx=0).stop 2024-05-06T12:21:33.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.534+0000 7f68b3f3d700 1 -- 172.21.15.92:0/2837173071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68ac0fed80 msgr2=0x7f68ac10de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:33.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.534+0000 7f68b3f3d700 1 --2- 172.21.15.92:0/2837173071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68ac0fed80 0x7f68ac10de60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f689c002a50 tx=0x7f689c0046b0 comp rx=0 tx=0).stop 2024-05-06T12:21:33.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.534+0000 7f68b3f3d700 1 -- 172.21.15.92:0/2837173071 shutdown_connections 2024-05-06T12:21:33.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.534+0000 7f68b3f3d700 1 --2- 172.21.15.92:0/2837173071 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f689806e140 0x7f68980705f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:33.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.534+0000 7f68b3f3d700 1 --2- 172.21.15.92:0/2837173071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68ac0fe430 0x7f68ac10d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:33.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.534+0000 7f68b3f3d700 1 --2- 172.21.15.92:0/2837173071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68ac0fed80 0x7f68ac10de60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:33.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.534+0000 7f68b3f3d700 1 -- 172.21.15.92:0/2837173071 >> 172.21.15.92:0/2837173071 conn(0x7f68ac0f6bd0 msgr2=0x7f68ac0f82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:21:33.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.534+0000 7f68b3f3d700 1 -- 172.21.15.92:0/2837173071 shutdown_connections 2024-05-06T12:21:33.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.534+0000 7f68b3f3d700 1 -- 172.21.15.92:0/2837173071 wait complete. 2024-05-06T12:21:33.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.634+0000 7fbaab992700 1 -- 172.21.15.92:0/3625104356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbaa40fff70 msgr2=0x7fbaa4102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:33.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.634+0000 7fbaab992700 1 --2- 172.21.15.92:0/3625104356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbaa40fff70 0x7fbaa4102350 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0004940 tx=0x7fbaa0009e30 comp rx=0 tx=0).stop 2024-05-06T12:21:33.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.634+0000 7fbaab992700 1 -- 172.21.15.92:0/3625104356 shutdown_connections 2024-05-06T12:21:33.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.634+0000 7fbaab992700 1 --2- 172.21.15.92:0/3625104356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbaa40fff70 0x7fbaa4102350 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:33.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.634+0000 7fbaab992700 1 --2- 172.21.15.92:0/3625104356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbaa4066f30 0x7fbaa40ffa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:33.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.634+0000 7fbaab992700 1 -- 172.21.15.92:0/3625104356 >> 172.21.15.92:0/3625104356 conn(0x7fbaa40f7420 msgr2=0x7fbaa40f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:21:33.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.634+0000 7fbaab992700 1 -- 172.21.15.92:0/3625104356 shutdown_connections 2024-05-06T12:21:33.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.634+0000 7fbaab992700 1 -- 172.21.15.92:0/3625104356 wait complete. 2024-05-06T12:21:33.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.634+0000 7fbaab992700 1 Processor -- start 2024-05-06T12:21:33.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.634+0000 7fbaab992700 1 -- start start 2024-05-06T12:21:33.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.635+0000 7fbaab992700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbaa4066f30 0x7fbaa410d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:33.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.635+0000 7fbaab992700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbaa40fff70 0x7fbaa410dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:33.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.635+0000 7fbaab992700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa410e4e0 con 0x7fbaa4066f30 2024-05-06T12:21:33.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.635+0000 7fbaab992700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa410e650 con 0x7fbaa40fff70 2024-05-06T12:21:33.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.635+0000 7fbaa972e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbaa4066f30 0x7fbaa410d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:33.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.635+0000 7fbaa972e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbaa4066f30 0x7fbaa410d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39860/0 (socket says 172.21.15.92:39860) 2024-05-06T12:21:33.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.635+0000 7fbaa972e700 1 -- 172.21.15.92:0/3423818855 learned_addr learned my addr 172.21.15.92:0/3423818855 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:21:33.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.635+0000 7fbaa8f2d700 1 --2- 172.21.15.92:0/3423818855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbaa40fff70 0x7fbaa410dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:33.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.635+0000 7fbaa972e700 1 -- 172.21.15.92:0/3423818855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbaa40fff70 msgr2=0x7fbaa410dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:33.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.635+0000 7fbaa972e700 1 --2- 172.21.15.92:0/3423818855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbaa40fff70 0x7fbaa410dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:33.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.635+0000 7fbaa972e700 1 -- 172.21.15.92:0/3423818855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa0005040 con 0x7fbaa4066f30 2024-05-06T12:21:33.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.635+0000 7fbaa972e700 1 --2- 172.21.15.92:0/3423818855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbaa4066f30 0x7fbaa410d9a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fba9400c800 tx=0x7fba94004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:21:33.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.635+0000 7fba99ffb700 1 -- 172.21.15.92:0/3423818855 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba940106f0 con 0x7fbaa4066f30 2024-05-06T12:21:33.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.635+0000 7fba99ffb700 1 -- 172.21.15.92:0/3423818855 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fba94010d20 con 0x7fbaa4066f30 2024-05-06T12:21:33.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.635+0000 7fbaab992700 1 -- 172.21.15.92:0/3423818855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaa4113060 con 0x7fbaa4066f30 2024-05-06T12:21:33.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.636+0000 7fba99ffb700 1 -- 172.21.15.92:0/3423818855 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba940113a0 con 0x7fbaa4066f30 2024-05-06T12:21:33.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.636+0000 7fbaab992700 1 -- 172.21.15.92:0/3423818855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaa4113620 con 0x7fbaa4066f30 2024-05-06T12:21:33.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.637+0000 7fbaab992700 1 -- 172.21.15.92:0/3423818855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaa404c710 con 0x7fbaa4066f30 2024-05-06T12:21:33.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.637+0000 7fba99ffb700 1 -- 172.21.15.92:0/3423818855 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fba94012400 con 0x7fbaa4066f30 2024-05-06T12:21:33.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.637+0000 7fba99ffb700 1 --2- 172.21.15.92:0/3423818855 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fba9006e140 0x7fba900705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:33.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.637+0000 7fba99ffb700 1 -- 172.21.15.92:0/3423818855 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fba9408d970 con 0x7fbaa4066f30 2024-05-06T12:21:33.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.637+0000 7fbaa8f2d700 1 --2- 172.21.15.92:0/3423818855 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fba9006e140 0x7fba900705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:33.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.638+0000 7fbaa8f2d700 1 --2- 172.21.15.92:0/3423818855 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fba9006e140 0x7fba900705f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbaa000b3b0 tx=0x7fbaa0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:21:33.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.642+0000 7fba99ffb700 1 -- 172.21.15.92:0/3423818855 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fba9405a940 con 0x7fbaa4066f30 2024-05-06T12:21:33.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.884+0000 7fbaab992700 1 -- 172.21.15.92:0/3423818855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbaa4113b10 con 0x7fbaa4066f30 2024-05-06T12:21:33.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.884+0000 7fba99ffb700 1 -- 172.21.15.92:0/3423818855 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) v1 ==== 76+0+1890 (secure 0 0 0) 0x7fba9405a4d0 con 0x7fbaa4066f30 2024-05-06T12:21:33.888 INFO:teuthology.orchestra.run.smithi092.stdout:e27 2024-05-06T12:21:33.888 INFO:teuthology.orchestra.run.smithi092.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-06T12:21:33.888 INFO:teuthology.orchestra.run.smithi092.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:21:33.888 INFO:teuthology.orchestra.run.smithi092.stdout:legacy client fscid: 1 2024-05-06T12:21:33.888 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:21:33.888 INFO:teuthology.orchestra.run.smithi092.stdout:Filesystem 'cephfs' (1) 2024-05-06T12:21:33.888 INFO:teuthology.orchestra.run.smithi092.stdout:fs_name cephfs 2024-05-06T12:21:33.888 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 26 2024-05-06T12:21:33.888 INFO:teuthology.orchestra.run.smithi092.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-06T12:21:33.889 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-05-06T12:11:44.952671+0000 2024-05-06T12:21:33.889 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2024-05-06T12:21:26.647044+0000 2024-05-06T12:21:33.889 INFO:teuthology.orchestra.run.smithi092.stdout:tableserver 0 2024-05-06T12:21:33.889 INFO:teuthology.orchestra.run.smithi092.stdout:root 0 2024-05-06T12:21:33.889 INFO:teuthology.orchestra.run.smithi092.stdout:session_timeout 60 2024-05-06T12:21:33.889 INFO:teuthology.orchestra.run.smithi092.stdout:session_autoclose 300 2024-05-06T12:21:33.889 INFO:teuthology.orchestra.run.smithi092.stdout:max_file_size 1099511627776 2024-05-06T12:21:33.889 INFO:teuthology.orchestra.run.smithi092.stdout:max_xattr_size 65536 2024-05-06T12:21:33.889 INFO:teuthology.orchestra.run.smithi092.stdout:required_client_features {} 2024-05-06T12:21:33.889 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure 0 2024-05-06T12:21:33.889 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure_osd_epoch 0 2024-05-06T12:21:33.889 INFO:teuthology.orchestra.run.smithi092.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:21:33.889 INFO:teuthology.orchestra.run.smithi092.stdout:max_mds 1 2024-05-06T12:21:33.889 INFO:teuthology.orchestra.run.smithi092.stdout:in 0 2024-05-06T12:21:33.890 INFO:teuthology.orchestra.run.smithi092.stdout:up {0=14472} 2024-05-06T12:21:33.890 INFO:teuthology.orchestra.run.smithi092.stdout:failed 2024-05-06T12:21:33.890 INFO:teuthology.orchestra.run.smithi092.stdout:damaged 2024-05-06T12:21:33.890 INFO:teuthology.orchestra.run.smithi092.stdout:stopped 1 2024-05-06T12:21:33.890 INFO:teuthology.orchestra.run.smithi092.stdout:data_pools [3] 2024-05-06T12:21:33.890 INFO:teuthology.orchestra.run.smithi092.stdout:metadata_pool 2 2024-05-06T12:21:33.890 INFO:teuthology.orchestra.run.smithi092.stdout:inline_data enabled 2024-05-06T12:21:33.890 INFO:teuthology.orchestra.run.smithi092.stdout:balancer 2024-05-06T12:21:33.890 INFO:teuthology.orchestra.run.smithi092.stdout:bal_rank_mask -1 2024-05-06T12:21:33.890 INFO:teuthology.orchestra.run.smithi092.stdout:standby_count_wanted 1 2024-05-06T12:21:33.890 INFO:teuthology.orchestra.run.smithi092.stdout:qdb_cluster leader: 14472 members: 14472 2024-05-06T12:21:33.890 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.rosbll{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.92:6828/1663253598,v1:172.21.15.92:6829/1663253598] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:21:33.890 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:21:33.890 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:21:33.891 INFO:teuthology.orchestra.run.smithi092.stdout:Standby daemons: 2024-05-06T12:21:33.891 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:21:33.891 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.ijsqgp{-1:34402} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.92:6826/3131141280,v1:172.21.15.92:6827/3131141280] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:21:33.891 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.idzoff{-1:44261} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.190:6826/1642657019,v1:172.21.15.190:6827/1642657019] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:21:33.891 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.byjvgf{-1:44267} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.190:6824/3031977277,v1:172.21.15.190:6825/3031977277] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:21:33.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.887+0000 7fbaab992700 1 -- 172.21.15.92:0/3423818855 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fba9006e140 msgr2=0x7fba900705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:33.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.887+0000 7fbaab992700 1 --2- 172.21.15.92:0/3423818855 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fba9006e140 0x7fba900705f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbaa000b3b0 tx=0x7fbaa0009e30 comp rx=0 tx=0).stop 2024-05-06T12:21:33.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.887+0000 7fbaab992700 1 -- 172.21.15.92:0/3423818855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbaa4066f30 msgr2=0x7fbaa410d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:33.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.887+0000 7fbaab992700 1 --2- 172.21.15.92:0/3423818855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbaa4066f30 0x7fbaa410d9a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fba9400c800 tx=0x7fba94004470 comp rx=0 tx=0).stop 2024-05-06T12:21:33.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.888+0000 7fbaab992700 1 -- 172.21.15.92:0/3423818855 shutdown_connections 2024-05-06T12:21:33.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.888+0000 7fbaab992700 1 --2- 172.21.15.92:0/3423818855 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fba9006e140 0x7fba900705f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:33.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.888+0000 7fbaab992700 1 --2- 172.21.15.92:0/3423818855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbaa4066f30 0x7fbaa410d9a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:33.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.888+0000 7fbaab992700 1 --2- 172.21.15.92:0/3423818855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbaa40fff70 0x7fbaa410dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:33.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.888+0000 7fbaab992700 1 -- 172.21.15.92:0/3423818855 >> 172.21.15.92:0/3423818855 conn(0x7fbaa40f7420 msgr2=0x7fbaa40f7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:21:33.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.888+0000 7fbaab992700 1 -- 172.21.15.92:0/3423818855 shutdown_connections 2024-05-06T12:21:33.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.888+0000 7fbaab992700 1 -- 172.21.15.92:0/3423818855 wait complete. 2024-05-06T12:21:33.893 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 27 2024-05-06T12:21:33.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.990+0000 7f8db0c3a700 1 -- 172.21.15.92:0/4225376384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dac0fcf10 msgr2=0x7f8dac0fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:33.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.990+0000 7f8db0c3a700 1 --2- 172.21.15.92:0/4225376384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dac0fcf10 0x7f8dac0fd320 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c004940 tx=0x7f8d9c009e30 comp rx=0 tx=0).stop 2024-05-06T12:21:33.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.990+0000 7f8db0c3a700 1 -- 172.21.15.92:0/4225376384 shutdown_connections 2024-05-06T12:21:33.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.990+0000 7f8db0c3a700 1 --2- 172.21.15.92:0/4225376384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dac0fe080 0x7f8dac0fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:33.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.990+0000 7f8db0c3a700 1 --2- 172.21.15.92:0/4225376384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dac0fcf10 0x7f8dac0fd320 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:33.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.990+0000 7f8db0c3a700 1 -- 172.21.15.92:0/4225376384 >> 172.21.15.92:0/4225376384 conn(0x7f8dac0f8560 msgr2=0x7f8dac0fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:21:33.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.990+0000 7f8db0c3a700 1 -- 172.21.15.92:0/4225376384 shutdown_connections 2024-05-06T12:21:33.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.990+0000 7f8db0c3a700 1 -- 172.21.15.92:0/4225376384 wait complete. 2024-05-06T12:21:33.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.990+0000 7f8db0c3a700 1 Processor -- start 2024-05-06T12:21:33.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.990+0000 7f8db0c3a700 1 -- start start 2024-05-06T12:21:33.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.990+0000 7f8db0c3a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dac0fcf10 0x7f8dac10d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:33.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.991+0000 7f8db0c3a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dac0fe080 0x7f8dac10dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:33.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.991+0000 7f8db0c3a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac10e4a0 con 0x7f8dac0fcf10 2024-05-06T12:21:33.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.991+0000 7f8db0c3a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac10ecf0 con 0x7f8dac0fe080 2024-05-06T12:21:33.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.991+0000 7f8daa59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dac0fcf10 0x7f8dac10d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:33.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.991+0000 7f8daa59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dac0fcf10 0x7f8dac10d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39886/0 (socket says 172.21.15.92:39886) 2024-05-06T12:21:33.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.991+0000 7f8daa59c700 1 -- 172.21.15.92:0/2158614394 learned_addr learned my addr 172.21.15.92:0/2158614394 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:21:33.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.991+0000 7f8da9d9b700 1 --2- 172.21.15.92:0/2158614394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dac0fe080 0x7f8dac10dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:33.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.991+0000 7f8daa59c700 1 -- 172.21.15.92:0/2158614394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dac0fe080 msgr2=0x7f8dac10dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:33.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.991+0000 7f8daa59c700 1 --2- 172.21.15.92:0/2158614394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dac0fe080 0x7f8dac10dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:33.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.991+0000 7f8daa59c700 1 -- 172.21.15.92:0/2158614394 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9c005040 con 0x7f8dac0fcf10 2024-05-06T12:21:33.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.991+0000 7f8daa59c700 1 --2- 172.21.15.92:0/2158614394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dac0fcf10 0x7f8dac10d960 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c013870 tx=0x7f8d9c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:21:33.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.991+0000 7f8d9affd700 1 -- 172.21.15.92:0/2158614394 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d9c021070 con 0x7f8dac0fcf10 2024-05-06T12:21:33.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.991+0000 7f8d9affd700 1 -- 172.21.15.92:0/2158614394 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d9c01c400 con 0x7f8dac0fcf10 2024-05-06T12:21:33.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.991+0000 7f8db0c3a700 1 -- 172.21.15.92:0/2158614394 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dac10ef70 con 0x7f8dac0fcf10 2024-05-06T12:21:33.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.991+0000 7f8d9affd700 1 -- 172.21.15.92:0/2158614394 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d9c00e070 con 0x7f8dac0fcf10 2024-05-06T12:21:33.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.991+0000 7f8db0c3a700 1 -- 172.21.15.92:0/2158614394 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dac10f4b0 con 0x7f8dac0fcf10 2024-05-06T12:21:33.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.992+0000 7f8db0c3a700 1 -- 172.21.15.92:0/2158614394 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dac05f5c0 con 0x7f8dac0fcf10 2024-05-06T12:21:33.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.993+0000 7f8d9affd700 1 -- 172.21.15.92:0/2158614394 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8d9c01ca20 con 0x7f8dac0fcf10 2024-05-06T12:21:33.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.993+0000 7f8d9affd700 1 --2- 172.21.15.92:0/2158614394 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f8d9406e140 0x7f8d940705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:33.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.993+0000 7f8d9affd700 1 -- 172.21.15.92:0/2158614394 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f8d9c08ed90 con 0x7f8dac0fcf10 2024-05-06T12:21:33.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.993+0000 7f8da9d9b700 1 --2- 172.21.15.92:0/2158614394 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f8d9406e140 0x7f8d940705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:33.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.994+0000 7f8da9d9b700 1 --2- 172.21.15.92:0/2158614394 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f8d9406e140 0x7f8d940705f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8da0004600 tx=0x7f8da000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:21:34.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:33.998+0000 7f8d9affd700 1 -- 172.21.15.92:0/2158614394 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f8d9c05bcc0 con 0x7f8dac0fcf10 2024-05-06T12:21:34.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.214+0000 7f8db0c3a700 1 -- 172.21.15.92:0/2158614394 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8dac10f7e0 con 0x7f8d9406e140 2024-05-06T12:21:34.217 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:33 smithi092 ceph-mon[91933]: from='client.34410 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:21:34.217 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:33 smithi092 ceph-mon[91933]: from='client.34414 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:21:34.217 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:33 smithi092 ceph-mon[91933]: pgmap v182: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-05-06T12:21:34.218 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:33 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/2837173071' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:34.218 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:33 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/3423818855' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:21:34.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.215+0000 7f8d9affd700 1 -- 172.21.15.92:0/2158614394 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8dac10f7e0 con 0x7f8d9406e140 2024-05-06T12:21:34.218 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:21:34.218 INFO:teuthology.orchestra.run.smithi092.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb", 2024-05-06T12:21:34.219 INFO:teuthology.orchestra.run.smithi092.stdout: "in_progress": true, 2024-05-06T12:21:34.219 INFO:teuthology.orchestra.run.smithi092.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-06T12:21:34.219 INFO:teuthology.orchestra.run.smithi092.stdout: "services_complete": [ 2024-05-06T12:21:34.219 INFO:teuthology.orchestra.run.smithi092.stdout: "osd", 2024-05-06T12:21:34.219 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr", 2024-05-06T12:21:34.219 INFO:teuthology.orchestra.run.smithi092.stdout: "mon", 2024-05-06T12:21:34.219 INFO:teuthology.orchestra.run.smithi092.stdout: "crash" 2024-05-06T12:21:34.219 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-05-06T12:21:34.219 INFO:teuthology.orchestra.run.smithi092.stdout: "progress": "12/21 daemons upgraded", 2024-05-06T12:21:34.219 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "Currently upgrading osd daemons", 2024-05-06T12:21:34.219 INFO:teuthology.orchestra.run.smithi092.stdout: "is_paused": false 2024-05-06T12:21:34.219 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:21:34.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.218+0000 7f8db0c3a700 1 -- 172.21.15.92:0/2158614394 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f8d9406e140 msgr2=0x7f8d940705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:34.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.218+0000 7f8db0c3a700 1 --2- 172.21.15.92:0/2158614394 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f8d9406e140 0x7f8d940705f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8da0004600 tx=0x7f8da000b040 comp rx=0 tx=0).stop 2024-05-06T12:21:34.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.218+0000 7f8db0c3a700 1 -- 172.21.15.92:0/2158614394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dac0fcf10 msgr2=0x7f8dac10d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:34.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.218+0000 7f8db0c3a700 1 --2- 172.21.15.92:0/2158614394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dac0fcf10 0x7f8dac10d960 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c013870 tx=0x7f8d9c009e30 comp rx=0 tx=0).stop 2024-05-06T12:21:34.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.218+0000 7f8db0c3a700 1 -- 172.21.15.92:0/2158614394 shutdown_connections 2024-05-06T12:21:34.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.218+0000 7f8db0c3a700 1 --2- 172.21.15.92:0/2158614394 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f8d9406e140 0x7f8d940705f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:34.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.218+0000 7f8db0c3a700 1 --2- 172.21.15.92:0/2158614394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dac0fcf10 0x7f8dac10d960 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:34.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.218+0000 7f8db0c3a700 1 --2- 172.21.15.92:0/2158614394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dac0fe080 0x7f8dac10dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:34.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.218+0000 7f8db0c3a700 1 -- 172.21.15.92:0/2158614394 >> 172.21.15.92:0/2158614394 conn(0x7f8dac0f8560 msgr2=0x7f8dac0f9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:21:34.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.218+0000 7f8db0c3a700 1 -- 172.21.15.92:0/2158614394 shutdown_connections 2024-05-06T12:21:34.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.218+0000 7f8db0c3a700 1 -- 172.21.15.92:0/2158614394 wait complete. 2024-05-06T12:21:34.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.318+0000 7f8af3abb700 1 -- 172.21.15.92:0/2672875897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8aec0fce60 msgr2=0x7f8aec0fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:34.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.318+0000 7f8af3abb700 1 --2- 172.21.15.92:0/2672875897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8aec0fce60 0x7f8aec0fd270 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8004940 tx=0x7f8ae8009e30 comp rx=0 tx=0).stop 2024-05-06T12:21:34.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.318+0000 7f8af3abb700 1 -- 172.21.15.92:0/2672875897 shutdown_connections 2024-05-06T12:21:34.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.318+0000 7f8af3abb700 1 --2- 172.21.15.92:0/2672875897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8aec0fdfd0 0x7f8aec0fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:34.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.318+0000 7f8af3abb700 1 --2- 172.21.15.92:0/2672875897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8aec0fce60 0x7f8aec0fd270 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:34.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.318+0000 7f8af3abb700 1 -- 172.21.15.92:0/2672875897 >> 172.21.15.92:0/2672875897 conn(0x7f8aec0f8540 msgr2=0x7f8aec0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:21:34.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.319+0000 7f8af3abb700 1 -- 172.21.15.92:0/2672875897 shutdown_connections 2024-05-06T12:21:34.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.319+0000 7f8af3abb700 1 -- 172.21.15.92:0/2672875897 wait complete. 2024-05-06T12:21:34.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.319+0000 7f8af3abb700 1 Processor -- start 2024-05-06T12:21:34.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.319+0000 7f8af3abb700 1 -- start start 2024-05-06T12:21:34.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.319+0000 7f8af3abb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8aec0fce60 0x7f8aec10d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:34.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.319+0000 7f8af3abb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8aec0fdfd0 0x7f8aec10de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:34.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.319+0000 7f8af3abb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec10e450 con 0x7f8aec0fce60 2024-05-06T12:21:34.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.319+0000 7f8af3abb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec10e5c0 con 0x7f8aec0fdfd0 2024-05-06T12:21:34.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.319+0000 7f8af1857700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8aec0fce60 0x7f8aec10d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:34.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.319+0000 7f8af1857700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8aec0fce60 0x7f8aec10d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39910/0 (socket says 172.21.15.92:39910) 2024-05-06T12:21:34.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.319+0000 7f8af1857700 1 -- 172.21.15.92:0/2256546176 learned_addr learned my addr 172.21.15.92:0/2256546176 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:21:34.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.319+0000 7f8af1056700 1 --2- 172.21.15.92:0/2256546176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8aec0fdfd0 0x7f8aec10de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:34.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.320+0000 7f8af1857700 1 -- 172.21.15.92:0/2256546176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8aec0fdfd0 msgr2=0x7f8aec10de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:34.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.320+0000 7f8af1857700 1 --2- 172.21.15.92:0/2256546176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8aec0fdfd0 0x7f8aec10de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:34.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.320+0000 7f8af1857700 1 -- 172.21.15.92:0/2256546176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae8005040 con 0x7f8aec0fce60 2024-05-06T12:21:34.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.320+0000 7f8af1857700 1 --2- 172.21.15.92:0/2256546176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8aec0fce60 0x7f8aec10d910 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8ae800b3b0 tx=0x7f8ae8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:21:34.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.320+0000 7f8ae27fc700 1 -- 172.21.15.92:0/2256546176 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ae800e040 con 0x7f8aec0fce60 2024-05-06T12:21:34.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.320+0000 7f8ae27fc700 1 -- 172.21.15.92:0/2256546176 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ae800bbf0 con 0x7f8aec0fce60 2024-05-06T12:21:34.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.320+0000 7f8af3abb700 1 -- 172.21.15.92:0/2256546176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aec112fc0 con 0x7f8aec0fce60 2024-05-06T12:21:34.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.320+0000 7f8ae27fc700 1 -- 172.21.15.92:0/2256546176 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ae801c630 con 0x7f8aec0fce60 2024-05-06T12:21:34.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.320+0000 7f8af3abb700 1 -- 172.21.15.92:0/2256546176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aec1134f0 con 0x7f8aec0fce60 2024-05-06T12:21:34.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.321+0000 7f8af3abb700 1 -- 172.21.15.92:0/2256546176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aec04c710 con 0x7f8aec0fce60 2024-05-06T12:21:34.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.321+0000 7f8ae27fc700 1 -- 172.21.15.92:0/2256546176 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8ae800bd60 con 0x7f8aec0fce60 2024-05-06T12:21:34.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.322+0000 7f8ae27fc700 1 --2- 172.21.15.92:0/2256546176 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f8ad806e140 0x7f8ad80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:21:34.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.322+0000 7f8ae27fc700 1 -- 172.21.15.92:0/2256546176 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f8ae808e6e0 con 0x7f8aec0fce60 2024-05-06T12:21:34.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.322+0000 7f8af1056700 1 --2- 172.21.15.92:0/2256546176 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f8ad806e140 0x7f8ad80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:21:34.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.322+0000 7f8af1056700 1 --2- 172.21.15.92:0/2256546176 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f8ad806e140 0x7f8ad80705f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8adc004f40 tx=0x7f8adc00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:21:34.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.326+0000 7f8ae27fc700 1 -- 172.21.15.92:0/2256546176 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f8ae805b610 con 0x7f8aec0fce60 2024-05-06T12:21:34.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:33 smithi190 ceph-mon[79744]: from='client.34410 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:21:34.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:33 smithi190 ceph-mon[79744]: from='client.34414 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:21:34.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:33 smithi190 ceph-mon[79744]: pgmap v182: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-05-06T12:21:34.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:33 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/2837173071' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:34.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:33 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/3423818855' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:21:34.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.607+0000 7f8af3abb700 1 -- 172.21.15.92:0/2256546176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8aec113a00 con 0x7f8aec0fce60 2024-05-06T12:21:34.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.607+0000 7f8ae27fc700 1 -- 172.21.15.92:0/2256546176 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f8ae805b1a0 con 0x7f8aec0fce60 2024-05-06T12:21:34.610 INFO:teuthology.orchestra.run.smithi092.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-06T12:21:34.610 INFO:teuthology.orchestra.run.smithi092.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-06T12:21:34.610 INFO:teuthology.orchestra.run.smithi092.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-06T12:21:34.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.609+0000 7f8af3abb700 1 -- 172.21.15.92:0/2256546176 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f8ad806e140 msgr2=0x7f8ad80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:34.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.609+0000 7f8af3abb700 1 --2- 172.21.15.92:0/2256546176 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f8ad806e140 0x7f8ad80705f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8adc004f40 tx=0x7f8adc00b040 comp rx=0 tx=0).stop 2024-05-06T12:21:34.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.609+0000 7f8af3abb700 1 -- 172.21.15.92:0/2256546176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8aec0fce60 msgr2=0x7f8aec10d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:21:34.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.609+0000 7f8af3abb700 1 --2- 172.21.15.92:0/2256546176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8aec0fce60 0x7f8aec10d910 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8ae800b3b0 tx=0x7f8ae8009e30 comp rx=0 tx=0).stop 2024-05-06T12:21:34.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.609+0000 7f8af3abb700 1 -- 172.21.15.92:0/2256546176 shutdown_connections 2024-05-06T12:21:34.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.609+0000 7f8af3abb700 1 --2- 172.21.15.92:0/2256546176 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f8ad806e140 0x7f8ad80705f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:34.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.609+0000 7f8af3abb700 1 --2- 172.21.15.92:0/2256546176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8aec0fce60 0x7f8aec10d910 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:34.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.609+0000 7f8af3abb700 1 --2- 172.21.15.92:0/2256546176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8aec0fdfd0 0x7f8aec10de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:21:34.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.609+0000 7f8af3abb700 1 -- 172.21.15.92:0/2256546176 >> 172.21.15.92:0/2256546176 conn(0x7f8aec0f8540 msgr2=0x7f8aec0f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:21:34.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.609+0000 7f8af3abb700 1 -- 172.21.15.92:0/2256546176 shutdown_connections 2024-05-06T12:21:34.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:21:34.609+0000 7f8af3abb700 1 -- 172.21.15.92:0/2256546176 wait complete. 2024-05-06T12:21:35.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:34 smithi092 ceph-mon[91933]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:21:35.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:34 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/2256546176' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:21:35.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:34 smithi190 ceph-mon[79744]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:21:35.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:34 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/2256546176' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:21:36.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:35 smithi092 ceph-mon[91933]: from='client.34430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:21:36.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:35 smithi092 ceph-mon[91933]: pgmap v183: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-05-06T12:21:36.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:35 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:21:36.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:35 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:21:36.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:35 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:21:36.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:35 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:36.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:35 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:21:36.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:35 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:36.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:35 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:36.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:35 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:36.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:35 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:36.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:35 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi092.ijsqgp"]}]: dispatch 2024-05-06T12:21:36.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:35 smithi190 ceph-mon[79744]: from='client.34430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:21:36.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:35 smithi190 ceph-mon[79744]: pgmap v183: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-05-06T12:21:36.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:35 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:21:36.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:35 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:21:36.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:35 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:21:36.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:35 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:36.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:35 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:21:36.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:35 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:36.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:35 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:36.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:35 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:36.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:35 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:36.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:35 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi092.ijsqgp"]}]: dispatch 2024-05-06T12:21:37.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:36 smithi092 ceph-mon[91933]: Upgrade: It appears safe to stop mds.cephfs.smithi092.ijsqgp 2024-05-06T12:21:37.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:36 smithi190 ceph-mon[79744]: Upgrade: It appears safe to stop mds.cephfs.smithi092.ijsqgp 2024-05-06T12:21:38.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:37 smithi092 ceph-mon[91933]: pgmap v184: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:21:38.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:37 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:38.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:37 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi092.ijsqgp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-06T12:21:38.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:37 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:21:38.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:37 smithi190 ceph-mon[79744]: pgmap v184: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:21:38.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:37 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:38.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:37 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi092.ijsqgp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-06T12:21:38.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:37 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:21:39.105 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:38 smithi092 ceph-mon[91933]: Upgrade: Updating mds.cephfs.smithi092.ijsqgp 2024-05-06T12:21:39.105 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:38 smithi092 ceph-mon[91933]: Deploying daemon mds.cephfs.smithi092.ijsqgp on smithi092 2024-05-06T12:21:39.105 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:38 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:21:39.249 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:38 smithi190 ceph-mon[79744]: Upgrade: Updating mds.cephfs.smithi092.ijsqgp 2024-05-06T12:21:39.249 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:38 smithi190 ceph-mon[79744]: Deploying daemon mds.cephfs.smithi092.ijsqgp on smithi092 2024-05-06T12:21:39.249 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:38 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:21:40.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:39 smithi092 ceph-mon[91933]: pgmap v185: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:21:40.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:39 smithi190 ceph-mon[79744]: pgmap v185: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:21:41.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:40 smithi092 ceph-mon[91933]: osdmap e83: 6 total, 6 up, 6 in 2024-05-06T12:21:41.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:40 smithi092 ceph-mon[91933]: fsmap cephfs:1 {0=cephfs.smithi092.rosbll=up:active} 2 up:standby 2024-05-06T12:21:41.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:40 smithi190 ceph-mon[79744]: osdmap e83: 6 total, 6 up, 6 in 2024-05-06T12:21:41.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:40 smithi190 ceph-mon[79744]: fsmap cephfs:1 {0=cephfs.smithi092.rosbll=up:active} 2 up:standby 2024-05-06T12:21:42.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:41 smithi190 ceph-mon[79744]: pgmap v187: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:21:42.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:41 smithi092 ceph-mon[91933]: pgmap v187: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:21:44.060 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:43 smithi092 ceph-mon[91933]: pgmap v188: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:21:44.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:43 smithi190 ceph-mon[79744]: pgmap v188: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:21:46.111 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:45 smithi092 ceph-mon[91933]: pgmap v189: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:21:46.111 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:45 smithi092 ceph-mon[91933]: mds.? [v2:172.21.15.92:6826/4190573007,v1:172.21.15.92:6827/4190573007] up:boot 2024-05-06T12:21:46.111 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:45 smithi092 ceph-mon[91933]: fsmap cephfs:1 {0=cephfs.smithi092.rosbll=up:active} 3 up:standby 2024-05-06T12:21:46.111 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:45 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi092.ijsqgp"}]: dispatch 2024-05-06T12:21:46.111 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:45 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:46.111 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:45 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:46.111 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:45 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:21:46.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:45 smithi190 ceph-mon[79744]: pgmap v189: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:21:46.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:45 smithi190 ceph-mon[79744]: mds.? [v2:172.21.15.92:6826/4190573007,v1:172.21.15.92:6827/4190573007] up:boot 2024-05-06T12:21:46.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:45 smithi190 ceph-mon[79744]: fsmap cephfs:1 {0=cephfs.smithi092.rosbll=up:active} 3 up:standby 2024-05-06T12:21:46.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:45 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi092.ijsqgp"}]: dispatch 2024-05-06T12:21:46.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:45 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:46.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:45 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:46.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:45 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:21:48.313 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:47 smithi092 ceph-mon[91933]: pgmap v190: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:21:48.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:47 smithi190 ceph-mon[79744]: pgmap v190: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:21:50.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:49 smithi092 ceph-mon[91933]: pgmap v191: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:21:50.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:49 smithi190 ceph-mon[79744]: pgmap v191: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:21:51.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:51 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:51.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:51 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:51.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:51 smithi190 ceph-mon[79744]: pgmap v192: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:21:51.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:51 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:51.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:51 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:51.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:51 smithi092 ceph-mon[91933]: pgmap v192: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:21:52.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:52 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:52.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:52 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:52.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:52 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:52.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:52 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:53.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:53 smithi092 ceph-mon[91933]: pgmap v193: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:21:53.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:53 smithi190 ceph-mon[79744]: pgmap v193: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:21:54.675 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:54 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:21:54.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:54 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:21:56.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:55 smithi190 ceph-mon[79744]: Detected new or changed devices on smithi092 2024-05-06T12:21:56.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:55 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:56.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:55 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:56.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:55 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:21:56.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:55 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:21:56.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:55 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:56.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:55 smithi190 ceph-mon[79744]: pgmap v194: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:21:56.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:55 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:21:56.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:55 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:56.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:55 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:56.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:55 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:56.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:55 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:56.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:55 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi092.rosbll"]}]: dispatch 2024-05-06T12:21:56.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:55 smithi190 ceph-mon[79744]: Upgrade: It appears safe to stop mds.cephfs.smithi092.rosbll 2024-05-06T12:21:56.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:55 smithi092 ceph-mon[91933]: Detected new or changed devices on smithi092 2024-05-06T12:21:56.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:55 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:56.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:55 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:56.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:55 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:21:56.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:55 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:21:56.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:55 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:56.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:55 smithi092 ceph-mon[91933]: pgmap v194: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:21:56.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:55 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:21:56.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:55 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:56.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:55 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:56.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:55 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:56.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:55 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:21:56.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:55 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi092.rosbll"]}]: dispatch 2024-05-06T12:21:56.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:55 smithi092 ceph-mon[91933]: Upgrade: It appears safe to stop mds.cephfs.smithi092.rosbll 2024-05-06T12:21:57.714 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:57 smithi092 ceph-mon[91933]: Upgrade: Updating mds.cephfs.smithi092.rosbll 2024-05-06T12:21:57.714 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:57 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:57.714 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:57 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi092.rosbll", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-06T12:21:57.714 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:57 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:21:57.714 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:57 smithi092 ceph-mon[91933]: Deploying daemon mds.cephfs.smithi092.rosbll on smithi092 2024-05-06T12:21:57.714 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:57 smithi092 ceph-mon[91933]: pgmap v195: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:21:57.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:57 smithi190 ceph-mon[79744]: Upgrade: Updating mds.cephfs.smithi092.rosbll 2024-05-06T12:21:57.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:57 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:21:57.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:57 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi092.rosbll", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-06T12:21:57.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:57 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:21:57.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:57 smithi190 ceph-mon[79744]: Deploying daemon mds.cephfs.smithi092.rosbll on smithi092 2024-05-06T12:21:57.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:57 smithi190 ceph-mon[79744]: pgmap v195: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:21:58.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:58 smithi190 ceph-mon[79744]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-06T12:21:58.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:58 smithi190 ceph-mon[79744]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-06T12:21:58.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:58 smithi190 ceph-mon[79744]: osdmap e84: 6 total, 6 up, 6 in 2024-05-06T12:21:58.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:58 smithi190 ceph-mon[79744]: Standby daemon mds.cephfs.smithi092.ijsqgp assigned to filesystem cephfs as rank 0 2024-05-06T12:21:58.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:58 smithi190 ceph-mon[79744]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-06T12:21:58.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:58 smithi190 ceph-mon[79744]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-05-06T12:21:58.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:58 smithi190 ceph-mon[79744]: fsmap cephfs:1/1 {0=cephfs.smithi092.ijsqgp=up:replay} 2 up:standby 2024-05-06T12:21:58.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:58 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi092[91929]: 2024-05-06T12:21:58.516+0000 7f73d600c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-06T12:21:58.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:58 smithi092 ceph-mon[91933]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-06T12:21:58.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:58 smithi092 ceph-mon[91933]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-06T12:21:58.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:58 smithi092 ceph-mon[91933]: osdmap e84: 6 total, 6 up, 6 in 2024-05-06T12:21:58.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:58 smithi092 ceph-mon[91933]: Standby daemon mds.cephfs.smithi092.ijsqgp assigned to filesystem cephfs as rank 0 2024-05-06T12:21:58.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:58 smithi092 ceph-mon[91933]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-06T12:21:58.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:58 smithi092 ceph-mon[91933]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-05-06T12:21:58.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:58 smithi092 ceph-mon[91933]: fsmap cephfs:1/1 {0=cephfs.smithi092.ijsqgp=up:replay} 2 up:standby 2024-05-06T12:21:59.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:21:59 smithi092 ceph-mon[91933]: pgmap v197: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 410 KiB/s rd, 0 op/s 2024-05-06T12:21:59.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:21:59 smithi190 ceph-mon[79744]: pgmap v197: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 410 KiB/s rd, 0 op/s 2024-05-06T12:22:00.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:00 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi092.rosbll"}]: dispatch 2024-05-06T12:22:00.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:00 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi092.rosbll"}]: dispatch 2024-05-06T12:22:01.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:01 smithi190 ceph-mon[79744]: pgmap v198: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 0 op/s 2024-05-06T12:22:01.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:01 smithi092 ceph-mon[91933]: pgmap v198: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 0 op/s 2024-05-06T12:22:04.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:03 smithi092 ceph-mon[91933]: pgmap v199: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-05-06T12:22:04.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:03 smithi190 ceph-mon[79744]: pgmap v199: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-05-06T12:22:04.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.715+0000 7f1282eca700 1 -- 172.21.15.92:0/232559263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f127c0fbb30 msgr2=0x7f127c0fe390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:04.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.715+0000 7f1282eca700 1 --2- 172.21.15.92:0/232559263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f127c0fbb30 0x7f127c0fe390 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1268004940 tx=0x7f1268009e30 comp rx=0 tx=0).stop 2024-05-06T12:22:04.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.715+0000 7f1282eca700 1 -- 172.21.15.92:0/232559263 shutdown_connections 2024-05-06T12:22:04.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.715+0000 7f1282eca700 1 --2- 172.21.15.92:0/232559263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f127c0fbb30 0x7f127c0fe390 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:04.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.715+0000 7f1282eca700 1 --2- 172.21.15.92:0/232559263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f127c0fb1e0 0x7f127c0fb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:04.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.715+0000 7f1282eca700 1 -- 172.21.15.92:0/232559263 >> 172.21.15.92:0/232559263 conn(0x7f127c0f6be0 msgr2=0x7f127c0f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:22:04.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.715+0000 7f1282eca700 1 -- 172.21.15.92:0/232559263 shutdown_connections 2024-05-06T12:22:04.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.715+0000 7f1282eca700 1 -- 172.21.15.92:0/232559263 wait complete. 2024-05-06T12:22:04.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.715+0000 7f1282eca700 1 Processor -- start 2024-05-06T12:22:04.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.715+0000 7f1282eca700 1 -- start start 2024-05-06T12:22:04.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.715+0000 7f1282eca700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f127c0fb1e0 0x7f127c10db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:04.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.715+0000 7f1282eca700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f127c0fbb30 0x7f127c10e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:04.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.715+0000 7f1282eca700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f127c10e700 con 0x7f127c0fbb30 2024-05-06T12:22:04.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.715+0000 7f1282eca700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f127c1130a0 con 0x7f127c0fb1e0 2024-05-06T12:22:04.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.716+0000 7f127bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f127c0fbb30 0x7f127c10e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:04.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.716+0000 7f127bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f127c0fbb30 0x7f127c10e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39364/0 (socket says 172.21.15.92:39364) 2024-05-06T12:22:04.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.716+0000 7f127bfff700 1 -- 172.21.15.92:0/2298941085 learned_addr learned my addr 172.21.15.92:0/2298941085 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:22:04.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.716+0000 7f1280c66700 1 --2- 172.21.15.92:0/2298941085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f127c0fb1e0 0x7f127c10db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:04.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.716+0000 7f127bfff700 1 -- 172.21.15.92:0/2298941085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f127c0fb1e0 msgr2=0x7f127c10db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:04.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.716+0000 7f127bfff700 1 --2- 172.21.15.92:0/2298941085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f127c0fb1e0 0x7f127c10db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:04.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.716+0000 7f127bfff700 1 -- 172.21.15.92:0/2298941085 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1268005040 con 0x7f127c0fbb30 2024-05-06T12:22:04.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.716+0000 7f1280c66700 1 --2- 172.21.15.92:0/2298941085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f127c0fb1e0 0x7f127c10db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-06T12:22:04.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.716+0000 7f127bfff700 1 --2- 172.21.15.92:0/2298941085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f127c0fbb30 0x7f127c10e070 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1268009150 tx=0x7f1268009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:22:04.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.716+0000 7f12797fa700 1 -- 172.21.15.92:0/2298941085 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f126800e040 con 0x7f127c0fbb30 2024-05-06T12:22:04.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.716+0000 7f1282eca700 1 -- 172.21.15.92:0/2298941085 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f127c1132a0 con 0x7f127c0fbb30 2024-05-06T12:22:04.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.716+0000 7f12797fa700 1 -- 172.21.15.92:0/2298941085 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f126800bcc0 con 0x7f127c0fbb30 2024-05-06T12:22:04.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.716+0000 7f12797fa700 1 -- 172.21.15.92:0/2298941085 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f126801c630 con 0x7f127c0fbb30 2024-05-06T12:22:04.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.716+0000 7f1282eca700 1 -- 172.21.15.92:0/2298941085 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f127c113810 con 0x7f127c0fbb30 2024-05-06T12:22:04.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.717+0000 7f1282eca700 1 -- 172.21.15.92:0/2298941085 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f127c04c710 con 0x7f127c0fbb30 2024-05-06T12:22:04.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.718+0000 7f12797fa700 1 -- 172.21.15.92:0/2298941085 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1268013420 con 0x7f127c0fbb30 2024-05-06T12:22:04.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.718+0000 7f12797fa700 1 --2- 172.21.15.92:0/2298941085 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f126c06e140 0x7f126c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:04.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.718+0000 7f1280c66700 1 --2- 172.21.15.92:0/2298941085 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f126c06e140 0x7f126c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:04.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.718+0000 7f12797fa700 1 -- 172.21.15.92:0/2298941085 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f126808e800 con 0x7f127c0fbb30 2024-05-06T12:22:04.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.719+0000 7f1280c66700 1 --2- 172.21.15.92:0/2298941085 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f126c06e140 0x7f126c0705f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1270009b10 tx=0x7f1270008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:22:04.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.723+0000 7f12797fa700 1 -- 172.21.15.92:0/2298941085 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f126805b6e0 con 0x7f127c0fbb30 2024-05-06T12:22:04.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.950+0000 7f1282eca700 1 -- 172.21.15.92:0/2298941085 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f127c102630 con 0x7f126c06e140 2024-05-06T12:22:04.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.951+0000 7f12797fa700 1 -- 172.21.15.92:0/2298941085 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f127c102630 con 0x7f126c06e140 2024-05-06T12:22:04.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.953+0000 7f1282eca700 1 -- 172.21.15.92:0/2298941085 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f126c06e140 msgr2=0x7f126c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:04.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.953+0000 7f1282eca700 1 --2- 172.21.15.92:0/2298941085 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f126c06e140 0x7f126c0705f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1270009b10 tx=0x7f1270008040 comp rx=0 tx=0).stop 2024-05-06T12:22:04.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.953+0000 7f1282eca700 1 -- 172.21.15.92:0/2298941085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f127c0fbb30 msgr2=0x7f127c10e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:04.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.953+0000 7f1282eca700 1 --2- 172.21.15.92:0/2298941085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f127c0fbb30 0x7f127c10e070 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1268009150 tx=0x7f1268009e30 comp rx=0 tx=0).stop 2024-05-06T12:22:04.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.953+0000 7f1282eca700 1 -- 172.21.15.92:0/2298941085 shutdown_connections 2024-05-06T12:22:04.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.953+0000 7f1282eca700 1 --2- 172.21.15.92:0/2298941085 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f126c06e140 0x7f126c0705f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:04.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.953+0000 7f1282eca700 1 --2- 172.21.15.92:0/2298941085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f127c0fb1e0 0x7f127c10db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:04.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.953+0000 7f1282eca700 1 --2- 172.21.15.92:0/2298941085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f127c0fbb30 0x7f127c10e070 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:04.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.953+0000 7f1282eca700 1 -- 172.21.15.92:0/2298941085 >> 172.21.15.92:0/2298941085 conn(0x7f127c0f6be0 msgr2=0x7f127c0f8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:22:04.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.953+0000 7f1282eca700 1 -- 172.21.15.92:0/2298941085 shutdown_connections 2024-05-06T12:22:04.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:04.953+0000 7f1282eca700 1 -- 172.21.15.92:0/2298941085 wait complete. 2024-05-06T12:22:04.966 INFO:teuthology.orchestra.run.smithi092.stdout:true 2024-05-06T12:22:05.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.055+0000 7fad17774700 1 -- 172.21.15.92:0/1997428969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad100fdfd0 msgr2=0x7fad100fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:05.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.055+0000 7fad17774700 1 --2- 172.21.15.92:0/1997428969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad100fdfd0 0x7fad100fe440 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fad00004940 tx=0x7fad00009e30 comp rx=0 tx=0).stop 2024-05-06T12:22:05.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.055+0000 7fad17774700 1 -- 172.21.15.92:0/1997428969 shutdown_connections 2024-05-06T12:22:05.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.055+0000 7fad17774700 1 --2- 172.21.15.92:0/1997428969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad100fdfd0 0x7fad100fe440 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:05.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.055+0000 7fad17774700 1 --2- 172.21.15.92:0/1997428969 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad100fce60 0x7fad100fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:05.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.055+0000 7fad17774700 1 -- 172.21.15.92:0/1997428969 >> 172.21.15.92:0/1997428969 conn(0x7fad100f8540 msgr2=0x7fad100fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:22:05.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.055+0000 7fad17774700 1 -- 172.21.15.92:0/1997428969 shutdown_connections 2024-05-06T12:22:05.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.055+0000 7fad17774700 1 -- 172.21.15.92:0/1997428969 wait complete. 2024-05-06T12:22:05.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.055+0000 7fad17774700 1 Processor -- start 2024-05-06T12:22:05.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.055+0000 7fad17774700 1 -- start start 2024-05-06T12:22:05.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.055+0000 7fad17774700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad100fce60 0x7fad1010d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:05.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.055+0000 7fad17774700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad100fdfd0 0x7fad1010df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:05.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.055+0000 7fad17774700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1010e5b0 con 0x7fad100fce60 2024-05-06T12:22:05.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.055+0000 7fad17774700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1010eda0 con 0x7fad100fdfd0 2024-05-06T12:22:05.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.056+0000 7fad15510700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad100fce60 0x7fad1010d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:05.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.056+0000 7fad15510700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad100fce60 0x7fad1010d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39388/0 (socket says 172.21.15.92:39388) 2024-05-06T12:22:05.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.056+0000 7fad15510700 1 -- 172.21.15.92:0/320632415 learned_addr learned my addr 172.21.15.92:0/320632415 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:22:05.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.056+0000 7fad15510700 1 -- 172.21.15.92:0/320632415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad100fdfd0 msgr2=0x7fad1010df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:05.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.056+0000 7fad15510700 1 --2- 172.21.15.92:0/320632415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad100fdfd0 0x7fad1010df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:05.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.056+0000 7fad15510700 1 -- 172.21.15.92:0/320632415 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad00005040 con 0x7fad100fce60 2024-05-06T12:22:05.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.056+0000 7fad15510700 1 --2- 172.21.15.92:0/320632415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad100fce60 0x7fad1010d9e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00bbb0 tx=0x7fad0c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:22:05.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.056+0000 7fad05ffb700 1 -- 172.21.15.92:0/320632415 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad0c0092e0 con 0x7fad100fce60 2024-05-06T12:22:05.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.056+0000 7fad05ffb700 1 -- 172.21.15.92:0/320632415 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad0c00cbd0 con 0x7fad100fce60 2024-05-06T12:22:05.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.056+0000 7fad17774700 1 -- 172.21.15.92:0/320632415 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad1010f080 con 0x7fad100fce60 2024-05-06T12:22:05.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.056+0000 7fad05ffb700 1 -- 172.21.15.92:0/320632415 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad0c007400 con 0x7fad100fce60 2024-05-06T12:22:05.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.056+0000 7fad17774700 1 -- 172.21.15.92:0/320632415 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad1010f640 con 0x7fad100fce60 2024-05-06T12:22:05.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.057+0000 7fad17774700 1 -- 172.21.15.92:0/320632415 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad1005f5c0 con 0x7fad100fce60 2024-05-06T12:22:05.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.058+0000 7fad05ffb700 1 -- 172.21.15.92:0/320632415 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fad0c012070 con 0x7fad100fce60 2024-05-06T12:22:05.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.058+0000 7fad05ffb700 1 --2- 172.21.15.92:0/320632415 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7facfc06e150 0x7facfc070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:05.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.058+0000 7fad05ffb700 1 -- 172.21.15.92:0/320632415 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fad0c08d760 con 0x7fad100fce60 2024-05-06T12:22:05.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.058+0000 7fad14d0f700 1 --2- 172.21.15.92:0/320632415 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7facfc06e150 0x7facfc070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:05.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.059+0000 7fad14d0f700 1 --2- 172.21.15.92:0/320632415 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7facfc06e150 0x7facfc070600 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fad00004f90 tx=0x7fad00009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:22:05.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.063+0000 7fad05ffb700 1 -- 172.21.15.92:0/320632415 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fad0c05a640 con 0x7fad100fce60 2024-05-06T12:22:05.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.287+0000 7fad17774700 1 -- 172.21.15.92:0/320632415 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fad1010fa10 con 0x7facfc06e150 2024-05-06T12:22:05.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.288+0000 7fad05ffb700 1 -- 172.21.15.92:0/320632415 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fad1010fa10 con 0x7facfc06e150 2024-05-06T12:22:05.293 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.290+0000 7fad17774700 1 -- 172.21.15.92:0/320632415 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7facfc06e150 msgr2=0x7facfc070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:05.293 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.290+0000 7fad17774700 1 --2- 172.21.15.92:0/320632415 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7facfc06e150 0x7facfc070600 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fad00004f90 tx=0x7fad00009e30 comp rx=0 tx=0).stop 2024-05-06T12:22:05.293 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.290+0000 7fad17774700 1 -- 172.21.15.92:0/320632415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad100fce60 msgr2=0x7fad1010d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:05.293 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.290+0000 7fad17774700 1 --2- 172.21.15.92:0/320632415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad100fce60 0x7fad1010d9e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00bbb0 tx=0x7fad0c009840 comp rx=0 tx=0).stop 2024-05-06T12:22:05.293 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.290+0000 7fad17774700 1 -- 172.21.15.92:0/320632415 shutdown_connections 2024-05-06T12:22:05.293 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.290+0000 7fad17774700 1 --2- 172.21.15.92:0/320632415 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7facfc06e150 0x7facfc070600 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:05.293 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.290+0000 7fad17774700 1 --2- 172.21.15.92:0/320632415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad100fce60 0x7fad1010d9e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:05.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.290+0000 7fad17774700 1 --2- 172.21.15.92:0/320632415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad100fdfd0 0x7fad1010df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:05.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.290+0000 7fad17774700 1 -- 172.21.15.92:0/320632415 >> 172.21.15.92:0/320632415 conn(0x7fad100f8540 msgr2=0x7fad100f9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:22:05.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.291+0000 7fad17774700 1 -- 172.21.15.92:0/320632415 shutdown_connections 2024-05-06T12:22:05.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.291+0000 7fad17774700 1 -- 172.21.15.92:0/320632415 wait complete. 2024-05-06T12:22:05.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.393+0000 7fd5824df700 1 -- 172.21.15.92:0/4019632190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd57c0fdfd0 msgr2=0x7fd57c0fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:05.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.393+0000 7fd5824df700 1 --2- 172.21.15.92:0/4019632190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd57c0fdfd0 0x7fd57c0fe440 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd570004940 tx=0x7fd570009e30 comp rx=0 tx=0).stop 2024-05-06T12:22:05.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.393+0000 7fd5824df700 1 -- 172.21.15.92:0/4019632190 shutdown_connections 2024-05-06T12:22:05.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.393+0000 7fd5824df700 1 --2- 172.21.15.92:0/4019632190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd57c0fdfd0 0x7fd57c0fe440 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:05.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.393+0000 7fd5824df700 1 --2- 172.21.15.92:0/4019632190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd57c0fce60 0x7fd57c0fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:05.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.393+0000 7fd5824df700 1 -- 172.21.15.92:0/4019632190 >> 172.21.15.92:0/4019632190 conn(0x7fd57c0f8540 msgr2=0x7fd57c0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:22:05.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.393+0000 7fd5824df700 1 -- 172.21.15.92:0/4019632190 shutdown_connections 2024-05-06T12:22:05.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.393+0000 7fd5824df700 1 -- 172.21.15.92:0/4019632190 wait complete. 2024-05-06T12:22:05.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.393+0000 7fd5824df700 1 Processor -- start 2024-05-06T12:22:05.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.393+0000 7fd5824df700 1 -- start start 2024-05-06T12:22:05.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.394+0000 7fd5824df700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd57c0fce60 0x7fd57c10d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:05.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.394+0000 7fd5824df700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd57c0fdfd0 0x7fd57c10de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:05.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.394+0000 7fd5824df700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c10e450 con 0x7fd57c0fce60 2024-05-06T12:22:05.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.394+0000 7fd5824df700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c10e5c0 con 0x7fd57c0fdfd0 2024-05-06T12:22:05.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.394+0000 7fd57bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd57c0fce60 0x7fd57c10d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:05.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.394+0000 7fd57bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd57c0fce60 0x7fd57c10d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39410/0 (socket says 172.21.15.92:39410) 2024-05-06T12:22:05.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.394+0000 7fd57bfff700 1 -- 172.21.15.92:0/3787433275 learned_addr learned my addr 172.21.15.92:0/3787433275 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:22:05.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.394+0000 7fd57b7fe700 1 --2- 172.21.15.92:0/3787433275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd57c0fdfd0 0x7fd57c10de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:05.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.394+0000 7fd57bfff700 1 -- 172.21.15.92:0/3787433275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd57c0fdfd0 msgr2=0x7fd57c10de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:05.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.394+0000 7fd57bfff700 1 --2- 172.21.15.92:0/3787433275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd57c0fdfd0 0x7fd57c10de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:05.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.394+0000 7fd57bfff700 1 -- 172.21.15.92:0/3787433275 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd570005040 con 0x7fd57c0fce60 2024-05-06T12:22:05.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.394+0000 7fd57b7fe700 1 --2- 172.21.15.92:0/3787433275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd57c0fdfd0 0x7fd57c10de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-06T12:22:05.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.394+0000 7fd57bfff700 1 --2- 172.21.15.92:0/3787433275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd57c0fce60 0x7fd57c10d910 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd56c002a50 tx=0x7fd56c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:22:05.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.394+0000 7fd578ff9700 1 -- 172.21.15.92:0/3787433275 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd56c007cc0 con 0x7fd57c0fce60 2024-05-06T12:22:05.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.394+0000 7fd5824df700 1 -- 172.21.15.92:0/3787433275 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd57c112fc0 con 0x7fd57c0fce60 2024-05-06T12:22:05.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.394+0000 7fd578ff9700 1 -- 172.21.15.92:0/3787433275 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd56c00f040 con 0x7fd57c0fce60 2024-05-06T12:22:05.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.394+0000 7fd578ff9700 1 -- 172.21.15.92:0/3787433275 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd56c009640 con 0x7fd57c0fce60 2024-05-06T12:22:05.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.394+0000 7fd5824df700 1 -- 172.21.15.92:0/3787433275 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd57c113550 con 0x7fd57c0fce60 2024-05-06T12:22:05.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.395+0000 7fd5824df700 1 -- 172.21.15.92:0/3787433275 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd57c04c710 con 0x7fd57c0fce60 2024-05-06T12:22:05.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.396+0000 7fd578ff9700 1 -- 172.21.15.92:0/3787433275 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd56c0097a0 con 0x7fd57c0fce60 2024-05-06T12:22:05.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.396+0000 7fd578ff9700 1 --2- 172.21.15.92:0/3787433275 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd56406e140 0x7fd5640705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:05.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.396+0000 7fd578ff9700 1 -- 172.21.15.92:0/3787433275 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd56c08d970 con 0x7fd57c0fce60 2024-05-06T12:22:05.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.396+0000 7fd57b7fe700 1 --2- 172.21.15.92:0/3787433275 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd56406e140 0x7fd5640705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:05.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.397+0000 7fd57b7fe700 1 --2- 172.21.15.92:0/3787433275 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd56406e140 0x7fd5640705f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd57000b3b0 tx=0x7fd570009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:22:05.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.401+0000 7fd578ff9700 1 -- 172.21.15.92:0/3787433275 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd56c05a7d0 con 0x7fd57c0fce60 2024-05-06T12:22:05.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.620+0000 7fd5824df700 1 -- 172.21.15.92:0/3787433275 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd57c113770 con 0x7fd56406e140 2024-05-06T12:22:05.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.631+0000 7fd578ff9700 1 -- 172.21.15.92:0/3787433275 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fd57c113770 con 0x7fd56406e140 2024-05-06T12:22:05.634 INFO:teuthology.orchestra.run.smithi092.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-06T12:22:05.634 INFO:teuthology.orchestra.run.smithi092.stdout:alertmanager.smithi092 smithi092 *:9093,9094 running (7m) 15s ago 15m 25.2M - 0.25.0 c8568f914cd2 be5637d7d25a 2024-05-06T12:22:05.634 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi092 smithi092 running (4m) 15s ago 15m 6660k - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 8da0494e1bdc 2024-05-06T12:22:05.634 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi190 smithi190 running (4m) 68s ago 14m 6665k - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 641a7cdaa56e 2024-05-06T12:22:05.634 INFO:teuthology.orchestra.run.smithi092.stdout:grafana.smithi092 smithi092 *:3000 running (7m) 15s ago 15m 82.0M - 9.4.7 954c08fa6188 9180cb3e5258 2024-05-06T12:22:05.634 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.ijsqgp smithi092 running (21s) 15s ago 10m 15.0M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 40b4cdd3be40 2024-05-06T12:22:05.635 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.rosbll smithi092 running (10m) 15s ago 10m 122M - 17.2.7-925-ge81df420 96d5ae7fd4b4 5f5426cd1bf5 2024-05-06T12:22:05.635 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.byjvgf smithi190 running (10m) 68s ago 10m 126M - 17.2.7-925-ge81df420 96d5ae7fd4b4 24581d2e1c5d 2024-05-06T12:22:05.635 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.idzoff smithi190 running (10m) 68s ago 10m 64.0M - 17.2.7-925-ge81df420 96d5ae7fd4b4 20f2641696ef 2024-05-06T12:22:05.635 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi092.gyzeqb smithi092 *:8443,9283,8765 running (6m) 15s ago 17m 508M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 de9d2198327a 2024-05-06T12:22:05.635 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi190.kczxkh smithi190 *:8443,9283,8765 running (5m) 68s ago 14m 451M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 e84ab3758ac7 2024-05-06T12:22:05.635 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi092 smithi092 running (5m) 15s ago 17m 50.5M 2048M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 ac07f7158087 2024-05-06T12:22:05.635 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi190 smithi190 running (5m) 68s ago 14m 41.7M 2048M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 7a3c08eb716f 2024-05-06T12:22:05.635 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi092 smithi092 *:9100 running (7m) 15s ago 15m 14.9M - 1.5.0 0da6a335fe13 c796446b206f 2024-05-06T12:22:05.635 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi190 smithi190 *:9100 running (6m) 68s ago 14m 16.5M - 1.5.0 0da6a335fe13 647e28111a76 2024-05-06T12:22:05.635 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 smithi092 running (4m) 15s ago 13m 88.4M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 8c96c75b25f4 2024-05-06T12:22:05.635 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 smithi092 running (3m) 15s ago 13m 89.6M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 e8007a510f08 2024-05-06T12:22:05.635 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 smithi092 running (2m) 15s ago 13m 84.9M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 2beca7448789 2024-05-06T12:22:05.635 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 smithi190 running (2m) 68s ago 12m 87.7M 2581M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 914ada199f93 2024-05-06T12:22:05.635 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 smithi190 running (106s) 68s ago 12m 82.6M 2581M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 371c0be776c7 2024-05-06T12:22:05.636 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 smithi190 running (72s) 68s ago 11m 12.0M 2581M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 067bc34cdfb1 2024-05-06T12:22:05.636 INFO:teuthology.orchestra.run.smithi092.stdout:prometheus.smithi092 smithi092 *:9095 running (5m) 15s ago 15m 63.0M - 2.43.0 a07b618ecd1d bc5445691527 2024-05-06T12:22:05.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.634+0000 7fd5824df700 1 -- 172.21.15.92:0/3787433275 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd56406e140 msgr2=0x7fd5640705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:05.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.634+0000 7fd5824df700 1 --2- 172.21.15.92:0/3787433275 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd56406e140 0x7fd5640705f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd57000b3b0 tx=0x7fd570009e30 comp rx=0 tx=0).stop 2024-05-06T12:22:05.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.634+0000 7fd5824df700 1 -- 172.21.15.92:0/3787433275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd57c0fce60 msgr2=0x7fd57c10d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:05.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.634+0000 7fd5824df700 1 --2- 172.21.15.92:0/3787433275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd57c0fce60 0x7fd57c10d910 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd56c002a50 tx=0x7fd56c0046b0 comp rx=0 tx=0).stop 2024-05-06T12:22:05.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.634+0000 7fd5824df700 1 -- 172.21.15.92:0/3787433275 shutdown_connections 2024-05-06T12:22:05.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.634+0000 7fd5824df700 1 --2- 172.21.15.92:0/3787433275 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd56406e140 0x7fd5640705f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:05.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.634+0000 7fd5824df700 1 --2- 172.21.15.92:0/3787433275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd57c0fce60 0x7fd57c10d910 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:05.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.634+0000 7fd5824df700 1 --2- 172.21.15.92:0/3787433275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd57c0fdfd0 0x7fd57c10de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:05.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.634+0000 7fd5824df700 1 -- 172.21.15.92:0/3787433275 >> 172.21.15.92:0/3787433275 conn(0x7fd57c0f8540 msgr2=0x7fd57c0f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:22:05.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.634+0000 7fd5824df700 1 -- 172.21.15.92:0/3787433275 shutdown_connections 2024-05-06T12:22:05.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.634+0000 7fd5824df700 1 -- 172.21.15.92:0/3787433275 wait complete. 2024-05-06T12:22:05.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.744+0000 7fa548d1d700 1 -- 172.21.15.92:0/3708174646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5440fbcf0 msgr2=0x7fa5440fc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:05.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.744+0000 7fa548d1d700 1 --2- 172.21.15.92:0/3708174646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5440fbcf0 0x7fa5440fc100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa534004940 tx=0x7fa534009e30 comp rx=0 tx=0).stop 2024-05-06T12:22:05.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.744+0000 7fa548d1d700 1 -- 172.21.15.92:0/3708174646 shutdown_connections 2024-05-06T12:22:05.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.744+0000 7fa548d1d700 1 --2- 172.21.15.92:0/3708174646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5440fcfa0 0x7fa5440fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:05.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.744+0000 7fa548d1d700 1 --2- 172.21.15.92:0/3708174646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5440fbcf0 0x7fa5440fc100 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:05.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.745+0000 7fa548d1d700 1 -- 172.21.15.92:0/3708174646 >> 172.21.15.92:0/3708174646 conn(0x7fa5440f7450 msgr2=0x7fa5440f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:22:05.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.745+0000 7fa548d1d700 1 -- 172.21.15.92:0/3708174646 shutdown_connections 2024-05-06T12:22:05.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.745+0000 7fa548d1d700 1 -- 172.21.15.92:0/3708174646 wait complete. 2024-05-06T12:22:05.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.745+0000 7fa548d1d700 1 Processor -- start 2024-05-06T12:22:05.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.745+0000 7fa548d1d700 1 -- start start 2024-05-06T12:22:05.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.745+0000 7fa548d1d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5440fbcf0 0x7fa54410d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:05.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.745+0000 7fa548d1d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5440fcfa0 0x7fa54410de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:05.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.745+0000 7fa548d1d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54410e460 con 0x7fa5440fcfa0 2024-05-06T12:22:05.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.745+0000 7fa548d1d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54410e5d0 con 0x7fa5440fbcf0 2024-05-06T12:22:05.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.745+0000 7fa541d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5440fcfa0 0x7fa54410de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:05.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.745+0000 7fa541d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5440fcfa0 0x7fa54410de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39440/0 (socket says 172.21.15.92:39440) 2024-05-06T12:22:05.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.745+0000 7fa541d9b700 1 -- 172.21.15.92:0/1744193623 learned_addr learned my addr 172.21.15.92:0/1744193623 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:22:05.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.746+0000 7fa54259c700 1 --2- 172.21.15.92:0/1744193623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5440fbcf0 0x7fa54410d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:05.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.746+0000 7fa541d9b700 1 -- 172.21.15.92:0/1744193623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5440fbcf0 msgr2=0x7fa54410d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:05.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.746+0000 7fa541d9b700 1 --2- 172.21.15.92:0/1744193623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5440fbcf0 0x7fa54410d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:05.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.746+0000 7fa541d9b700 1 -- 172.21.15.92:0/1744193623 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa534005040 con 0x7fa5440fcfa0 2024-05-06T12:22:05.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.746+0000 7fa541d9b700 1 --2- 172.21.15.92:0/1744193623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5440fcfa0 0x7fa54410de60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa538002a50 tx=0x7fa5380046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:22:05.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.746+0000 7fa532ffd700 1 -- 172.21.15.92:0/1744193623 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa538007cc0 con 0x7fa5440fcfa0 2024-05-06T12:22:05.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.746+0000 7fa532ffd700 1 -- 172.21.15.92:0/1744193623 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa53800f040 con 0x7fa5440fcfa0 2024-05-06T12:22:05.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.746+0000 7fa548d1d700 1 -- 172.21.15.92:0/1744193623 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa544113030 con 0x7fa5440fcfa0 2024-05-06T12:22:05.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.746+0000 7fa532ffd700 1 -- 172.21.15.92:0/1744193623 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa53800da30 con 0x7fa5440fcfa0 2024-05-06T12:22:05.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.746+0000 7fa548d1d700 1 -- 172.21.15.92:0/1744193623 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5441135c0 con 0x7fa5440fcfa0 2024-05-06T12:22:05.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.747+0000 7fa548d1d700 1 -- 172.21.15.92:0/1744193623 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa544101b30 con 0x7fa5440fcfa0 2024-05-06T12:22:05.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.748+0000 7fa532ffd700 1 -- 172.21.15.92:0/1744193623 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa53800db90 con 0x7fa5440fcfa0 2024-05-06T12:22:05.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.748+0000 7fa532ffd700 1 --2- 172.21.15.92:0/1744193623 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fa52c06e140 0x7fa52c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:05.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.748+0000 7fa532ffd700 1 -- 172.21.15.92:0/1744193623 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa53808d840 con 0x7fa5440fcfa0 2024-05-06T12:22:05.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.748+0000 7fa54259c700 1 --2- 172.21.15.92:0/1744193623 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fa52c06e140 0x7fa52c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:05.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.748+0000 7fa54259c700 1 --2- 172.21.15.92:0/1744193623 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fa52c06e140 0x7fa52c0705f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa53400b3b0 tx=0x7fa534009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:22:05.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:05.753+0000 7fa532ffd700 1 -- 172.21.15.92:0/1744193623 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa53805a720 con 0x7fa5440fcfa0 2024-05-06T12:22:06.054 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:05 smithi092 ceph-mon[91933]: pgmap v200: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-05-06T12:22:06.054 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:05 smithi092 ceph-mon[91933]: reconnect by client.14524 192.168.0.1:0/1455526237 after 0.000999972 2024-05-06T12:22:06.054 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:05 smithi092 ceph-mon[91933]: reconnect by client.24315 192.168.0.1:0/3983235695 after 0.000999972 2024-05-06T12:22:06.054 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:05 smithi092 ceph-mon[91933]: mds.? [v2:172.21.15.92:6826/4190573007,v1:172.21.15.92:6827/4190573007] up:reconnect 2024-05-06T12:22:06.054 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:05 smithi092 ceph-mon[91933]: fsmap cephfs:1/1 {0=cephfs.smithi092.ijsqgp=up:reconnect} 2 up:standby 2024-05-06T12:22:06.054 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:05 smithi092 ceph-mon[91933]: from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:22:06.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.104+0000 7fa548d1d700 1 -- 172.21.15.92:0/1744193623 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa54404c710 con 0x7fa5440fcfa0 2024-05-06T12:22:06.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.105+0000 7fa532ffd700 1 -- 172.21.15.92:0/1744193623 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7fa53805a2b0 con 0x7fa5440fcfa0 2024-05-06T12:22:06.108 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:22:06.108 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": { 2024-05-06T12:22:06.108 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:22:06.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:22:06.108 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": { 2024-05-06T12:22:06.108 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:22:06.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:22:06.108 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": { 2024-05-06T12:22:06.109 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 6 2024-05-06T12:22:06.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:22:06.109 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-05-06T12:22:06.109 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 2, 2024-05-06T12:22:06.109 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 1 2024-05-06T12:22:06.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:22:06.109 INFO:teuthology.orchestra.run.smithi092.stdout: "overall": { 2024-05-06T12:22:06.109 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 17.2.7-925-ge81df420 (e81df4201214a93810b2eaca609e475a6319e220) quincy (stable)": 2, 2024-05-06T12:22:06.109 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 11 2024-05-06T12:22:06.109 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-06T12:22:06.109 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:22:06.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.107+0000 7fa548d1d700 1 -- 172.21.15.92:0/1744193623 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fa52c06e140 msgr2=0x7fa52c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:06.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.107+0000 7fa548d1d700 1 --2- 172.21.15.92:0/1744193623 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fa52c06e140 0x7fa52c0705f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa53400b3b0 tx=0x7fa534009e30 comp rx=0 tx=0).stop 2024-05-06T12:22:06.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.107+0000 7fa548d1d700 1 -- 172.21.15.92:0/1744193623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5440fcfa0 msgr2=0x7fa54410de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:06.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.107+0000 7fa548d1d700 1 --2- 172.21.15.92:0/1744193623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5440fcfa0 0x7fa54410de60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa538002a50 tx=0x7fa5380046b0 comp rx=0 tx=0).stop 2024-05-06T12:22:06.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.107+0000 7fa548d1d700 1 -- 172.21.15.92:0/1744193623 shutdown_connections 2024-05-06T12:22:06.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.107+0000 7fa548d1d700 1 --2- 172.21.15.92:0/1744193623 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fa52c06e140 0x7fa52c0705f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:06.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.107+0000 7fa548d1d700 1 --2- 172.21.15.92:0/1744193623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5440fbcf0 0x7fa54410d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:06.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.107+0000 7fa548d1d700 1 --2- 172.21.15.92:0/1744193623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5440fcfa0 0x7fa54410de60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:06.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.107+0000 7fa548d1d700 1 -- 172.21.15.92:0/1744193623 >> 172.21.15.92:0/1744193623 conn(0x7fa5440f7450 msgr2=0x7fa5440f8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:22:06.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.108+0000 7fa548d1d700 1 -- 172.21.15.92:0/1744193623 shutdown_connections 2024-05-06T12:22:06.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.108+0000 7fa548d1d700 1 -- 172.21.15.92:0/1744193623 wait complete. 2024-05-06T12:22:06.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.217+0000 7fdfe1d65700 1 -- 172.21.15.92:0/1545589117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfdc0fbcb0 msgr2=0x7fdfdc0fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:06.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.217+0000 7fdfe1d65700 1 --2- 172.21.15.92:0/1545589117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfdc0fbcb0 0x7fdfdc0fc0c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc004940 tx=0x7fdfcc009e30 comp rx=0 tx=0).stop 2024-05-06T12:22:06.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.217+0000 7fdfe1d65700 1 -- 172.21.15.92:0/1545589117 shutdown_connections 2024-05-06T12:22:06.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.217+0000 7fdfe1d65700 1 --2- 172.21.15.92:0/1545589117 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfdc0fcf60 0x7fdfdc0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:06.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.217+0000 7fdfe1d65700 1 --2- 172.21.15.92:0/1545589117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfdc0fbcb0 0x7fdfdc0fc0c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:06.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.217+0000 7fdfe1d65700 1 -- 172.21.15.92:0/1545589117 >> 172.21.15.92:0/1545589117 conn(0x7fdfdc0f7430 msgr2=0x7fdfdc0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:22:06.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.217+0000 7fdfe1d65700 1 -- 172.21.15.92:0/1545589117 shutdown_connections 2024-05-06T12:22:06.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.217+0000 7fdfe1d65700 1 -- 172.21.15.92:0/1545589117 wait complete. 2024-05-06T12:22:06.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.218+0000 7fdfe1d65700 1 Processor -- start 2024-05-06T12:22:06.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.218+0000 7fdfe1d65700 1 -- start start 2024-05-06T12:22:06.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.218+0000 7fdfe1d65700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfdc0fbcb0 0x7fdfdc112b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:06.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.218+0000 7fdfe1d65700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfdc0fcf60 0x7fdfdc115070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:06.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.218+0000 7fdfe1d65700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfdc066220 con 0x7fdfdc0fcf60 2024-05-06T12:22:06.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.218+0000 7fdfe1d65700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfdc066360 con 0x7fdfdc0fbcb0 2024-05-06T12:22:06.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.218+0000 7fdfd2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfdc0fcf60 0x7fdfdc115070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:06.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.218+0000 7fdfd2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfdc0fcf60 0x7fdfdc115070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39458/0 (socket says 172.21.15.92:39458) 2024-05-06T12:22:06.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.218+0000 7fdfd2ffd700 1 -- 172.21.15.92:0/2621457490 learned_addr learned my addr 172.21.15.92:0/2621457490 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:22:06.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.218+0000 7fdfdb7fe700 1 --2- 172.21.15.92:0/2621457490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfdc0fbcb0 0x7fdfdc112b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:06.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.219+0000 7fdfd2ffd700 1 -- 172.21.15.92:0/2621457490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfdc0fbcb0 msgr2=0x7fdfdc112b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:06.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.219+0000 7fdfd2ffd700 1 --2- 172.21.15.92:0/2621457490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfdc0fbcb0 0x7fdfdc112b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:06.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.219+0000 7fdfd2ffd700 1 -- 172.21.15.92:0/2621457490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfcc005040 con 0x7fdfdc0fcf60 2024-05-06T12:22:06.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.219+0000 7fdfd2ffd700 1 --2- 172.21.15.92:0/2621457490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfdc0fcf60 0x7fdfdc115070 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4002a50 tx=0x7fdfc40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:22:06.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.219+0000 7fdfd8ff9700 1 -- 172.21.15.92:0/2621457490 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfc4007cc0 con 0x7fdfdc0fcf60 2024-05-06T12:22:06.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.219+0000 7fdfd8ff9700 1 -- 172.21.15.92:0/2621457490 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdfc400f040 con 0x7fdfdc0fcf60 2024-05-06T12:22:06.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.219+0000 7fdfe1d65700 1 -- 172.21.15.92:0/2621457490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfdc1155b0 con 0x7fdfdc0fcf60 2024-05-06T12:22:06.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.219+0000 7fdfd8ff9700 1 -- 172.21.15.92:0/2621457490 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfc400da30 con 0x7fdfdc0fcf60 2024-05-06T12:22:06.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.219+0000 7fdfe1d65700 1 -- 172.21.15.92:0/2621457490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfdc1159d0 con 0x7fdfdc0fcf60 2024-05-06T12:22:06.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.220+0000 7fdfe1d65700 1 -- 172.21.15.92:0/2621457490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfdc05f5c0 con 0x7fdfdc0fcf60 2024-05-06T12:22:06.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.220+0000 7fdfd8ff9700 1 -- 172.21.15.92:0/2621457490 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdfc400dbb0 con 0x7fdfdc0fcf60 2024-05-06T12:22:06.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.221+0000 7fdfd8ff9700 1 --2- 172.21.15.92:0/2621457490 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fdfc806e140 0x7fdfc80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:06.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.221+0000 7fdfd8ff9700 1 -- 172.21.15.92:0/2621457490 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fdfc408d910 con 0x7fdfdc0fcf60 2024-05-06T12:22:06.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.221+0000 7fdfdb7fe700 1 --2- 172.21.15.92:0/2621457490 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fdfc806e140 0x7fdfc80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:06.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.221+0000 7fdfdb7fe700 1 --2- 172.21.15.92:0/2621457490 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fdfc806e140 0x7fdfc80705f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc004890 tx=0x7fdfcc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:22:06.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.226+0000 7fdfd8ff9700 1 -- 172.21.15.92:0/2621457490 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fdfc4056c40 con 0x7fdfdc0fcf60 2024-05-06T12:22:06.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:05 smithi190 ceph-mon[79744]: pgmap v200: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-05-06T12:22:06.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:05 smithi190 ceph-mon[79744]: reconnect by client.14524 192.168.0.1:0/1455526237 after 0.000999972 2024-05-06T12:22:06.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:05 smithi190 ceph-mon[79744]: reconnect by client.24315 192.168.0.1:0/3983235695 after 0.000999972 2024-05-06T12:22:06.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:05 smithi190 ceph-mon[79744]: mds.? [v2:172.21.15.92:6826/4190573007,v1:172.21.15.92:6827/4190573007] up:reconnect 2024-05-06T12:22:06.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:05 smithi190 ceph-mon[79744]: fsmap cephfs:1/1 {0=cephfs.smithi092.ijsqgp=up:reconnect} 2 up:standby 2024-05-06T12:22:06.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:05 smithi190 ceph-mon[79744]: from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:22:06.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.507+0000 7fdfe1d65700 1 -- 172.21.15.92:0/2621457490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdfdc04bed0 con 0x7fdfdc0fcf60 2024-05-06T12:22:06.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.507+0000 7fdfd8ff9700 1 -- 172.21.15.92:0/2621457490 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1751 (secure 0 0 0) 0x7fdfc405a300 con 0x7fdfdc0fcf60 2024-05-06T12:22:06.511 INFO:teuthology.orchestra.run.smithi092.stdout:e33 2024-05-06T12:22:06.511 INFO:teuthology.orchestra.run.smithi092.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-06T12:22:06.511 INFO:teuthology.orchestra.run.smithi092.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:22:06.511 INFO:teuthology.orchestra.run.smithi092.stdout:legacy client fscid: 1 2024-05-06T12:22:06.511 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:22:06.511 INFO:teuthology.orchestra.run.smithi092.stdout:Filesystem 'cephfs' (1) 2024-05-06T12:22:06.511 INFO:teuthology.orchestra.run.smithi092.stdout:fs_name cephfs 2024-05-06T12:22:06.511 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 33 2024-05-06T12:22:06.511 INFO:teuthology.orchestra.run.smithi092.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-06T12:22:06.511 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-05-06T12:11:44.952671+0000 2024-05-06T12:22:06.511 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2024-05-06T12:22:04.922678+0000 2024-05-06T12:22:06.512 INFO:teuthology.orchestra.run.smithi092.stdout:tableserver 0 2024-05-06T12:22:06.512 INFO:teuthology.orchestra.run.smithi092.stdout:root 0 2024-05-06T12:22:06.512 INFO:teuthology.orchestra.run.smithi092.stdout:session_timeout 60 2024-05-06T12:22:06.512 INFO:teuthology.orchestra.run.smithi092.stdout:session_autoclose 300 2024-05-06T12:22:06.512 INFO:teuthology.orchestra.run.smithi092.stdout:max_file_size 1099511627776 2024-05-06T12:22:06.512 INFO:teuthology.orchestra.run.smithi092.stdout:max_xattr_size 65536 2024-05-06T12:22:06.512 INFO:teuthology.orchestra.run.smithi092.stdout:required_client_features {} 2024-05-06T12:22:06.512 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure 0 2024-05-06T12:22:06.512 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure_osd_epoch 84 2024-05-06T12:22:06.512 INFO:teuthology.orchestra.run.smithi092.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-06T12:22:06.512 INFO:teuthology.orchestra.run.smithi092.stdout:max_mds 1 2024-05-06T12:22:06.513 INFO:teuthology.orchestra.run.smithi092.stdout:in 0 2024-05-06T12:22:06.513 INFO:teuthology.orchestra.run.smithi092.stdout:up {0=34438} 2024-05-06T12:22:06.513 INFO:teuthology.orchestra.run.smithi092.stdout:failed 2024-05-06T12:22:06.513 INFO:teuthology.orchestra.run.smithi092.stdout:damaged 2024-05-06T12:22:06.513 INFO:teuthology.orchestra.run.smithi092.stdout:stopped 1 2024-05-06T12:22:06.513 INFO:teuthology.orchestra.run.smithi092.stdout:data_pools [3] 2024-05-06T12:22:06.513 INFO:teuthology.orchestra.run.smithi092.stdout:metadata_pool 2 2024-05-06T12:22:06.513 INFO:teuthology.orchestra.run.smithi092.stdout:inline_data enabled 2024-05-06T12:22:06.513 INFO:teuthology.orchestra.run.smithi092.stdout:balancer 2024-05-06T12:22:06.513 INFO:teuthology.orchestra.run.smithi092.stdout:bal_rank_mask -1 2024-05-06T12:22:06.513 INFO:teuthology.orchestra.run.smithi092.stdout:standby_count_wanted 1 2024-05-06T12:22:06.513 INFO:teuthology.orchestra.run.smithi092.stdout:qdb_cluster leader: 0 members: 2024-05-06T12:22:06.514 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.ijsqgp{0:34438} state up:rejoin seq 7 join_fscid=1 addr [v2:172.21.15.92:6826/4190573007,v1:172.21.15.92:6827/4190573007] compat {c=[1],r=[1],i=[1fff]}] 2024-05-06T12:22:06.514 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:22:06.514 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:22:06.514 INFO:teuthology.orchestra.run.smithi092.stdout:Standby daemons: 2024-05-06T12:22:06.514 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:22:06.514 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.idzoff{-1:44261} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.190:6826/1642657019,v1:172.21.15.190:6827/1642657019] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:22:06.514 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.byjvgf{-1:44267} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.190:6824/3031977277,v1:172.21.15.190:6825/3031977277] compat {c=[1],r=[1],i=[7ff]}] 2024-05-06T12:22:06.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.509+0000 7fdfe1d65700 1 -- 172.21.15.92:0/2621457490 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fdfc806e140 msgr2=0x7fdfc80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:06.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.509+0000 7fdfe1d65700 1 --2- 172.21.15.92:0/2621457490 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fdfc806e140 0x7fdfc80705f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc004890 tx=0x7fdfcc009e30 comp rx=0 tx=0).stop 2024-05-06T12:22:06.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.509+0000 7fdfe1d65700 1 -- 172.21.15.92:0/2621457490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfdc0fcf60 msgr2=0x7fdfdc115070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:06.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.509+0000 7fdfe1d65700 1 --2- 172.21.15.92:0/2621457490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfdc0fcf60 0x7fdfdc115070 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4002a50 tx=0x7fdfc40046b0 comp rx=0 tx=0).stop 2024-05-06T12:22:06.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.509+0000 7fdfe1d65700 1 -- 172.21.15.92:0/2621457490 shutdown_connections 2024-05-06T12:22:06.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.509+0000 7fdfe1d65700 1 --2- 172.21.15.92:0/2621457490 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fdfc806e140 0x7fdfc80705f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:06.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.509+0000 7fdfe1d65700 1 --2- 172.21.15.92:0/2621457490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfdc0fbcb0 0x7fdfdc112b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:06.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.509+0000 7fdfe1d65700 1 --2- 172.21.15.92:0/2621457490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfdc0fcf60 0x7fdfdc115070 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:06.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.509+0000 7fdfe1d65700 1 -- 172.21.15.92:0/2621457490 >> 172.21.15.92:0/2621457490 conn(0x7fdfdc0f7430 msgr2=0x7fdfdc0f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:22:06.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.509+0000 7fdfe1d65700 1 -- 172.21.15.92:0/2621457490 shutdown_connections 2024-05-06T12:22:06.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.509+0000 7fdfe1d65700 1 -- 172.21.15.92:0/2621457490 wait complete. 2024-05-06T12:22:06.515 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 33 2024-05-06T12:22:06.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.614+0000 7f46402be700 1 -- 172.21.15.92:0/3646843237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46380fbcb0 msgr2=0x7f46380fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:06.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.614+0000 7f46402be700 1 --2- 172.21.15.92:0/3646843237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46380fbcb0 0x7f46380fc0c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4634004940 tx=0x7f4634009e30 comp rx=0 tx=0).stop 2024-05-06T12:22:06.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.614+0000 7f46402be700 1 -- 172.21.15.92:0/3646843237 shutdown_connections 2024-05-06T12:22:06.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.614+0000 7f46402be700 1 --2- 172.21.15.92:0/3646843237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46380fcf60 0x7f46380fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:06.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.614+0000 7f46402be700 1 --2- 172.21.15.92:0/3646843237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46380fbcb0 0x7f46380fc0c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:06.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.614+0000 7f46402be700 1 -- 172.21.15.92:0/3646843237 >> 172.21.15.92:0/3646843237 conn(0x7f46380f7430 msgr2=0x7f46380f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:22:06.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.614+0000 7f46402be700 1 -- 172.21.15.92:0/3646843237 shutdown_connections 2024-05-06T12:22:06.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.614+0000 7f46402be700 1 -- 172.21.15.92:0/3646843237 wait complete. 2024-05-06T12:22:06.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.615+0000 7f46402be700 1 Processor -- start 2024-05-06T12:22:06.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.615+0000 7f46402be700 1 -- start start 2024-05-06T12:22:06.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.615+0000 7f46402be700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46380fbcb0 0x7f4638110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:06.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.615+0000 7f46402be700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46380fcf60 0x7f4638112c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:06.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.615+0000 7f46402be700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4638066330 con 0x7f46380fbcb0 2024-05-06T12:22:06.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.615+0000 7f46402be700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4638066470 con 0x7f46380fcf60 2024-05-06T12:22:06.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.615+0000 7f463e05a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46380fbcb0 0x7f4638110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:06.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.615+0000 7f463e05a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46380fbcb0 0x7f4638110720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39474/0 (socket says 172.21.15.92:39474) 2024-05-06T12:22:06.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.615+0000 7f463e05a700 1 -- 172.21.15.92:0/3911312804 learned_addr learned my addr 172.21.15.92:0/3911312804 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:22:06.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.615+0000 7f463e05a700 1 -- 172.21.15.92:0/3911312804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46380fcf60 msgr2=0x7f4638112c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:06.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.615+0000 7f463d859700 1 --2- 172.21.15.92:0/3911312804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46380fcf60 0x7f4638112c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:06.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.615+0000 7f463e05a700 1 --2- 172.21.15.92:0/3911312804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46380fcf60 0x7f4638112c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:06.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.615+0000 7f463e05a700 1 -- 172.21.15.92:0/3911312804 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4634005040 con 0x7f46380fbcb0 2024-05-06T12:22:06.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.615+0000 7f463d859700 1 --2- 172.21.15.92:0/3911312804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46380fcf60 0x7f4638112c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-06T12:22:06.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.615+0000 7f463e05a700 1 --2- 172.21.15.92:0/3911312804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46380fbcb0 0x7f4638110720 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4634009150 tx=0x7f4634009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:22:06.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.616+0000 7f462effd700 1 -- 172.21.15.92:0/3911312804 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f463400e040 con 0x7f46380fbcb0 2024-05-06T12:22:06.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.616+0000 7f46402be700 1 -- 172.21.15.92:0/3911312804 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46381131b0 con 0x7f46380fbcb0 2024-05-06T12:22:06.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.616+0000 7f462effd700 1 -- 172.21.15.92:0/3911312804 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f463400bcc0 con 0x7f46380fbcb0 2024-05-06T12:22:06.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.616+0000 7f46402be700 1 -- 172.21.15.92:0/3911312804 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4638113770 con 0x7f46380fbcb0 2024-05-06T12:22:06.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.616+0000 7f462effd700 1 -- 172.21.15.92:0/3911312804 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f463401c630 con 0x7f46380fbcb0 2024-05-06T12:22:06.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.617+0000 7f46402be700 1 -- 172.21.15.92:0/3911312804 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f463805f5c0 con 0x7f46380fbcb0 2024-05-06T12:22:06.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.617+0000 7f462effd700 1 -- 172.21.15.92:0/3911312804 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4634013420 con 0x7f46380fbcb0 2024-05-06T12:22:06.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.618+0000 7f462effd700 1 --2- 172.21.15.92:0/3911312804 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f462406e140 0x7f46240705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:06.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.618+0000 7f463d859700 1 --2- 172.21.15.92:0/3911312804 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f462406e140 0x7f46240705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:06.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.618+0000 7f462effd700 1 -- 172.21.15.92:0/3911312804 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f463408e840 con 0x7f46380fbcb0 2024-05-06T12:22:06.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.618+0000 7f463d859700 1 --2- 172.21.15.92:0/3911312804 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f462406e140 0x7f46240705f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4628009fd0 tx=0x7f462800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:22:06.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.622+0000 7f462effd700 1 -- 172.21.15.92:0/3911312804 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4634057bf0 con 0x7f46380fbcb0 2024-05-06T12:22:06.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.846+0000 7f46402be700 1 -- 172.21.15.92:0/3911312804 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f46381012c0 con 0x7f462406e140 2024-05-06T12:22:06.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.848+0000 7f462effd700 1 -- 172.21.15.92:0/3911312804 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f46381012c0 con 0x7f462406e140 2024-05-06T12:22:06.850 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:22:06.850 INFO:teuthology.orchestra.run.smithi092.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb", 2024-05-06T12:22:06.851 INFO:teuthology.orchestra.run.smithi092.stdout: "in_progress": true, 2024-05-06T12:22:06.851 INFO:teuthology.orchestra.run.smithi092.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-06T12:22:06.851 INFO:teuthology.orchestra.run.smithi092.stdout: "services_complete": [ 2024-05-06T12:22:06.851 INFO:teuthology.orchestra.run.smithi092.stdout: "osd", 2024-05-06T12:22:06.851 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr", 2024-05-06T12:22:06.851 INFO:teuthology.orchestra.run.smithi092.stdout: "mon", 2024-05-06T12:22:06.851 INFO:teuthology.orchestra.run.smithi092.stdout: "crash" 2024-05-06T12:22:06.851 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-05-06T12:22:06.851 INFO:teuthology.orchestra.run.smithi092.stdout: "progress": "13/21 daemons upgraded", 2024-05-06T12:22:06.851 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "Currently upgrading mds daemons", 2024-05-06T12:22:06.851 INFO:teuthology.orchestra.run.smithi092.stdout: "is_paused": false 2024-05-06T12:22:06.851 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:22:06.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.850+0000 7f46402be700 1 -- 172.21.15.92:0/3911312804 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f462406e140 msgr2=0x7f46240705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:06.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.850+0000 7f46402be700 1 --2- 172.21.15.92:0/3911312804 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f462406e140 0x7f46240705f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4628009fd0 tx=0x7f462800b040 comp rx=0 tx=0).stop 2024-05-06T12:22:06.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.850+0000 7f46402be700 1 -- 172.21.15.92:0/3911312804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46380fbcb0 msgr2=0x7f4638110720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:06.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.850+0000 7f46402be700 1 --2- 172.21.15.92:0/3911312804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46380fbcb0 0x7f4638110720 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4634009150 tx=0x7f4634009e30 comp rx=0 tx=0).stop 2024-05-06T12:22:06.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.850+0000 7f46402be700 1 -- 172.21.15.92:0/3911312804 shutdown_connections 2024-05-06T12:22:06.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.850+0000 7f46402be700 1 --2- 172.21.15.92:0/3911312804 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f462406e140 0x7f46240705f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:06.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.850+0000 7f46402be700 1 --2- 172.21.15.92:0/3911312804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46380fbcb0 0x7f4638110720 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:06.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.850+0000 7f46402be700 1 --2- 172.21.15.92:0/3911312804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46380fcf60 0x7f4638112c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:06.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.850+0000 7f46402be700 1 -- 172.21.15.92:0/3911312804 >> 172.21.15.92:0/3911312804 conn(0x7f46380f7430 msgr2=0x7f46380f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:22:06.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.850+0000 7f46402be700 1 -- 172.21.15.92:0/3911312804 shutdown_connections 2024-05-06T12:22:06.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.850+0000 7f46402be700 1 -- 172.21.15.92:0/3911312804 wait complete. 2024-05-06T12:22:06.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.963+0000 7feae5ea2700 1 -- 172.21.15.92:0/2120840084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feae00fcf60 msgr2=0x7feae00fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:06.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.963+0000 7feae5ea2700 1 --2- 172.21.15.92:0/2120840084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feae00fcf60 0x7feae00fd3d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fead4004940 tx=0x7fead4009e30 comp rx=0 tx=0).stop 2024-05-06T12:22:06.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.963+0000 7feae5ea2700 1 -- 172.21.15.92:0/2120840084 shutdown_connections 2024-05-06T12:22:06.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.963+0000 7feae5ea2700 1 --2- 172.21.15.92:0/2120840084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feae00fcf60 0x7feae00fd3d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:06.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.963+0000 7feae5ea2700 1 --2- 172.21.15.92:0/2120840084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feae00fbcb0 0x7feae00fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:06.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.963+0000 7feae5ea2700 1 -- 172.21.15.92:0/2120840084 >> 172.21.15.92:0/2120840084 conn(0x7feae00f7430 msgr2=0x7feae00f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:22:06.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.963+0000 7feae5ea2700 1 -- 172.21.15.92:0/2120840084 shutdown_connections 2024-05-06T12:22:06.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.964+0000 7feae5ea2700 1 -- 172.21.15.92:0/2120840084 wait complete. 2024-05-06T12:22:06.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.964+0000 7feae5ea2700 1 Processor -- start 2024-05-06T12:22:06.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.964+0000 7feae5ea2700 1 -- start start 2024-05-06T12:22:06.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.964+0000 7feae5ea2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feae00fbcb0 0x7feae010da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:06.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.964+0000 7feae5ea2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feae00fcf60 0x7feae010dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:06.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.964+0000 7feae5ea2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae010e5b0 con 0x7feae00fcf60 2024-05-06T12:22:06.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.964+0000 7feae5ea2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae010edb0 con 0x7feae00fbcb0 2024-05-06T12:22:06.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.965+0000 7feadeffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feae00fcf60 0x7feae010dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:06.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.965+0000 7feadeffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feae00fcf60 0x7feae010dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39512/0 (socket says 172.21.15.92:39512) 2024-05-06T12:22:06.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.965+0000 7feadeffd700 1 -- 172.21.15.92:0/1419232221 learned_addr learned my addr 172.21.15.92:0/1419232221 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:22:06.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.965+0000 7feadf7fe700 1 --2- 172.21.15.92:0/1419232221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feae00fbcb0 0x7feae010da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:06.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.965+0000 7feadeffd700 1 -- 172.21.15.92:0/1419232221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feae00fbcb0 msgr2=0x7feae010da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:06.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.965+0000 7feadeffd700 1 --2- 172.21.15.92:0/1419232221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feae00fbcb0 0x7feae010da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:06.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.965+0000 7feadeffd700 1 -- 172.21.15.92:0/1419232221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead4005040 con 0x7feae00fcf60 2024-05-06T12:22:06.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.965+0000 7feadeffd700 1 --2- 172.21.15.92:0/1419232221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feae00fcf60 0x7feae010dfb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fead4009150 tx=0x7fead4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:22:06.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.965+0000 7feac7fff700 1 -- 172.21.15.92:0/1419232221 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fead400e040 con 0x7feae00fcf60 2024-05-06T12:22:06.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.965+0000 7feae5ea2700 1 -- 172.21.15.92:0/1419232221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae010f030 con 0x7feae00fcf60 2024-05-06T12:22:06.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.965+0000 7feac7fff700 1 -- 172.21.15.92:0/1419232221 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fead4004290 con 0x7feae00fcf60 2024-05-06T12:22:06.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.965+0000 7feac7fff700 1 -- 172.21.15.92:0/1419232221 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fead401c630 con 0x7feae00fcf60 2024-05-06T12:22:06.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.965+0000 7feae5ea2700 1 -- 172.21.15.92:0/1419232221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae010f570 con 0x7feae00fcf60 2024-05-06T12:22:06.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.966+0000 7feae5ea2700 1 -- 172.21.15.92:0/1419232221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae0101af0 con 0x7feae00fcf60 2024-05-06T12:22:06.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.967+0000 7feac7fff700 1 -- 172.21.15.92:0/1419232221 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fead400bbf0 con 0x7feae00fcf60 2024-05-06T12:22:06.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.967+0000 7feac7fff700 1 --2- 172.21.15.92:0/1419232221 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7feac806e140 0x7feac80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:06.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.967+0000 7feac7fff700 1 -- 172.21.15.92:0/1419232221 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fead408e710 con 0x7feae00fcf60 2024-05-06T12:22:06.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.967+0000 7feadf7fe700 1 --2- 172.21.15.92:0/1419232221 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7feac806e140 0x7feac80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:06.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.968+0000 7feadf7fe700 1 --2- 172.21.15.92:0/1419232221 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7feac806e140 0x7feac80705f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fead0004f60 tx=0x7fead000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:22:06.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:06.972+0000 7feac7fff700 1 -- 172.21.15.92:0/1419232221 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fead405b5f0 con 0x7feae00fcf60 2024-05-06T12:22:07.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:06 smithi092 ceph-mon[91933]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:22:07.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:06 smithi092 ceph-mon[91933]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:22:07.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:06 smithi092 ceph-mon[91933]: mds.? [v2:172.21.15.92:6826/4190573007,v1:172.21.15.92:6827/4190573007] up:rejoin 2024-05-06T12:22:07.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:06 smithi092 ceph-mon[91933]: fsmap cephfs:1/1 {0=cephfs.smithi092.ijsqgp=up:rejoin} 2 up:standby 2024-05-06T12:22:07.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:06 smithi092 ceph-mon[91933]: daemon mds.cephfs.smithi092.ijsqgp is now active in filesystem cephfs as rank 0 2024-05-06T12:22:07.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:06 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/1744193623' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:07.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:06 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/2621457490' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:22:07.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:07.269+0000 7feae5ea2700 1 -- 172.21.15.92:0/1419232221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7feae005f5c0 con 0x7feae00fcf60 2024-05-06T12:22:07.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:07.269+0000 7feac7fff700 1 -- 172.21.15.92:0/1419232221 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fead405b180 con 0x7feae00fcf60 2024-05-06T12:22:07.272 INFO:teuthology.orchestra.run.smithi092.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-06T12:22:07.272 INFO:teuthology.orchestra.run.smithi092.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-06T12:22:07.272 INFO:teuthology.orchestra.run.smithi092.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-06T12:22:07.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:07.271+0000 7feae5ea2700 1 -- 172.21.15.92:0/1419232221 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7feac806e140 msgr2=0x7feac80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:07.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:07.271+0000 7feae5ea2700 1 --2- 172.21.15.92:0/1419232221 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7feac806e140 0x7feac80705f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fead0004f60 tx=0x7fead000b040 comp rx=0 tx=0).stop 2024-05-06T12:22:07.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:07.271+0000 7feae5ea2700 1 -- 172.21.15.92:0/1419232221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feae00fcf60 msgr2=0x7feae010dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:07.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:07.271+0000 7feae5ea2700 1 --2- 172.21.15.92:0/1419232221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feae00fcf60 0x7feae010dfb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fead4009150 tx=0x7fead4009e30 comp rx=0 tx=0).stop 2024-05-06T12:22:07.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:07.271+0000 7feae5ea2700 1 -- 172.21.15.92:0/1419232221 shutdown_connections 2024-05-06T12:22:07.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:07.271+0000 7feae5ea2700 1 --2- 172.21.15.92:0/1419232221 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7feac806e140 0x7feac80705f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:07.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:07.271+0000 7feae5ea2700 1 --2- 172.21.15.92:0/1419232221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feae00fbcb0 0x7feae010da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:07.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:07.271+0000 7feae5ea2700 1 --2- 172.21.15.92:0/1419232221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feae00fcf60 0x7feae010dfb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:07.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:07.271+0000 7feae5ea2700 1 -- 172.21.15.92:0/1419232221 >> 172.21.15.92:0/1419232221 conn(0x7feae00f7430 msgr2=0x7feae0065bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:22:07.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:07.272+0000 7feae5ea2700 1 -- 172.21.15.92:0/1419232221 shutdown_connections 2024-05-06T12:22:07.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:07.272+0000 7feae5ea2700 1 -- 172.21.15.92:0/1419232221 wait complete. 2024-05-06T12:22:07.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:06 smithi190 ceph-mon[79744]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:22:07.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:06 smithi190 ceph-mon[79744]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:22:07.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:06 smithi190 ceph-mon[79744]: mds.? [v2:172.21.15.92:6826/4190573007,v1:172.21.15.92:6827/4190573007] up:rejoin 2024-05-06T12:22:07.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:06 smithi190 ceph-mon[79744]: fsmap cephfs:1/1 {0=cephfs.smithi092.ijsqgp=up:rejoin} 2 up:standby 2024-05-06T12:22:07.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:06 smithi190 ceph-mon[79744]: daemon mds.cephfs.smithi092.ijsqgp is now active in filesystem cephfs as rank 0 2024-05-06T12:22:07.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:06 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/1744193623' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:07.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:06 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/2621457490' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:22:08.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:07 smithi092 ceph-mon[91933]: from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:22:08.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:07 smithi092 ceph-mon[91933]: pgmap v201: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-05-06T12:22:08.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:07 smithi092 ceph-mon[91933]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-06T12:22:08.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:07 smithi092 ceph-mon[91933]: mds.? [v2:172.21.15.92:6826/4190573007,v1:172.21.15.92:6827/4190573007] up:active 2024-05-06T12:22:08.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:07 smithi092 ceph-mon[91933]: mds.? [v2:172.21.15.92:6828/2398082173,v1:172.21.15.92:6829/2398082173] up:boot 2024-05-06T12:22:08.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:07 smithi092 ceph-mon[91933]: fsmap cephfs:1 {0=cephfs.smithi092.ijsqgp=up:active} 3 up:standby 2024-05-06T12:22:08.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi092.rosbll"}]: dispatch 2024-05-06T12:22:08.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:08.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:08.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:22:08.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:07 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/1419232221' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:22:08.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:07 smithi190 ceph-mon[79744]: from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:22:08.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:07 smithi190 ceph-mon[79744]: pgmap v201: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-05-06T12:22:08.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:07 smithi190 ceph-mon[79744]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-06T12:22:08.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:07 smithi190 ceph-mon[79744]: mds.? [v2:172.21.15.92:6826/4190573007,v1:172.21.15.92:6827/4190573007] up:active 2024-05-06T12:22:08.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:07 smithi190 ceph-mon[79744]: mds.? [v2:172.21.15.92:6828/2398082173,v1:172.21.15.92:6829/2398082173] up:boot 2024-05-06T12:22:08.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:07 smithi190 ceph-mon[79744]: fsmap cephfs:1 {0=cephfs.smithi092.ijsqgp=up:active} 3 up:standby 2024-05-06T12:22:08.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi092.rosbll"}]: dispatch 2024-05-06T12:22:08.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:08.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:08.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:22:08.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:07 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/1419232221' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:22:09.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:08 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:22:09.248 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:08 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:22:10.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:09 smithi092 ceph-mon[91933]: pgmap v202: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 495 B/s wr, 8 op/s 2024-05-06T12:22:10.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:09 smithi190 ceph-mon[79744]: pgmap v202: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 495 B/s wr, 8 op/s 2024-05-06T12:22:12.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:11 smithi092 ceph-mon[91933]: pgmap v203: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 426 B/s wr, 7 op/s 2024-05-06T12:22:12.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:11 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:12.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:11 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:12.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:11 smithi190 ceph-mon[79744]: pgmap v203: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 426 B/s wr, 7 op/s 2024-05-06T12:22:12.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:11 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:12.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:11 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:14.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:13 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:14.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:13 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:14.099 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:13 smithi190 ceph-mon[79744]: pgmap v204: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-05-06T12:22:14.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:13 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:14.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:13 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:14.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:13 smithi092 ceph-mon[91933]: pgmap v204: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-05-06T12:22:16.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:15 smithi092 ceph-mon[91933]: pgmap v205: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 3.1 MiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-05-06T12:22:16.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:15 smithi190 ceph-mon[79744]: pgmap v205: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 3.1 MiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-05-06T12:22:17.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:17 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:17.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:17 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:17.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:17 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:22:17.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:17 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:22:17.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:17 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:17.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:17 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:22:17.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:17 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:17.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:17 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:17.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:17 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:17.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:17 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:17.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:17 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi190.byjvgf"]}]: dispatch 2024-05-06T12:22:17.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:17 smithi190 ceph-mon[79744]: Upgrade: It appears safe to stop mds.cephfs.smithi190.byjvgf 2024-05-06T12:22:17.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:17 smithi190 ceph-mon[79744]: pgmap v206: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-05-06T12:22:17.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:17 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:17.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:17 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:17.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:17 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:22:17.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:17 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:22:17.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:17 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:17.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:17 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:22:17.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:17 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:17.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:17 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:17.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:17 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:17.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:17 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:17.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:17 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi190.byjvgf"]}]: dispatch 2024-05-06T12:22:17.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:17 smithi092 ceph-mon[91933]: Upgrade: It appears safe to stop mds.cephfs.smithi190.byjvgf 2024-05-06T12:22:17.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:17 smithi092 ceph-mon[91933]: pgmap v206: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-05-06T12:22:19.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:19 smithi190 ceph-mon[79744]: Upgrade: Updating mds.cephfs.smithi190.byjvgf 2024-05-06T12:22:19.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:19 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:19.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:19 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi190.byjvgf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-06T12:22:19.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:19 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:22:19.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:19 smithi190 ceph-mon[79744]: Deploying daemon mds.cephfs.smithi190.byjvgf on smithi190 2024-05-06T12:22:19.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:19 smithi190 ceph-mon[79744]: pgmap v207: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 430 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-05-06T12:22:19.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:19 smithi092 ceph-mon[91933]: Upgrade: Updating mds.cephfs.smithi190.byjvgf 2024-05-06T12:22:19.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:19 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:19.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:19 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi190.byjvgf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-06T12:22:19.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:19 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:22:19.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:19 smithi092 ceph-mon[91933]: Deploying daemon mds.cephfs.smithi190.byjvgf on smithi190 2024-05-06T12:22:19.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:19 smithi092 ceph-mon[91933]: pgmap v207: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 430 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-05-06T12:22:21.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:21 smithi190 ceph-mon[79744]: osdmap e85: 6 total, 6 up, 6 in 2024-05-06T12:22:21.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:21 smithi190 ceph-mon[79744]: fsmap cephfs:1 {0=cephfs.smithi092.ijsqgp=up:active} 2 up:standby 2024-05-06T12:22:21.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:21 smithi190 ceph-mon[79744]: pgmap v209: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 307 B/s rd, 5.1 KiB/s wr, 2 op/s 2024-05-06T12:22:21.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:21 smithi092 ceph-mon[91933]: osdmap e85: 6 total, 6 up, 6 in 2024-05-06T12:22:21.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:21 smithi092 ceph-mon[91933]: fsmap cephfs:1 {0=cephfs.smithi092.ijsqgp=up:active} 2 up:standby 2024-05-06T12:22:21.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:21 smithi092 ceph-mon[91933]: pgmap v209: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 307 B/s rd, 5.1 KiB/s wr, 2 op/s 2024-05-06T12:22:24.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:23 smithi092 ceph-mon[91933]: pgmap v210: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:22:24.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:23 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi190.byjvgf"}]: dispatch 2024-05-06T12:22:24.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:23 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:22:24.262 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:23 smithi190 ceph-mon[79744]: pgmap v210: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:22:24.262 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:23 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi190.byjvgf"}]: dispatch 2024-05-06T12:22:24.262 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:23 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:22:25.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:25 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:25.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:25 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:25.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:25 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:22:25.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:25 smithi190 ceph-mon[79744]: pgmap v211: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:22:25.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:25 smithi190 ceph-mon[79744]: mds.? [v2:172.21.15.190:6824/1219274291,v1:172.21.15.190:6825/1219274291] up:boot 2024-05-06T12:22:25.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:25 smithi190 ceph-mon[79744]: fsmap cephfs:1 {0=cephfs.smithi092.ijsqgp=up:active} 3 up:standby 2024-05-06T12:22:25.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:25 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi190.byjvgf"}]: dispatch 2024-05-06T12:22:25.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:25 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:25.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:25 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:25.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:25 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:22:25.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:25 smithi092 ceph-mon[91933]: pgmap v211: 65 pgs: 65 active+clean; 175 MiB data, 772 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:22:25.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:25 smithi092 ceph-mon[91933]: mds.? [v2:172.21.15.190:6824/1219274291,v1:172.21.15.190:6825/1219274291] up:boot 2024-05-06T12:22:25.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:25 smithi092 ceph-mon[91933]: fsmap cephfs:1 {0=cephfs.smithi092.ijsqgp=up:active} 3 up:standby 2024-05-06T12:22:25.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:25 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi190.byjvgf"}]: dispatch 2024-05-06T12:22:28.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:27 smithi092 ceph-mon[91933]: pgmap v212: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:22:28.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:27 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:28.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:27 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:28.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:27 smithi190 ceph-mon[79744]: pgmap v212: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:22:28.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:27 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:28.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:27 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:29.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:29 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:29.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:29 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:29.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:29 smithi092 ceph-mon[91933]: pgmap v213: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:22:30.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:29 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:30.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:29 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:30.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:29 smithi190 ceph-mon[79744]: pgmap v213: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:22:32.058 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:31 smithi190 ceph-mon[79744]: pgmap v214: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:22:32.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:31 smithi092 ceph-mon[91933]: pgmap v214: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:22:33.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:33 smithi190 ceph-mon[79744]: Detected new or changed devices on smithi190 2024-05-06T12:22:33.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:33 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:33.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:33 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:33.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:33 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:22:33.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:33 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:22:33.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:33 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:33.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:33 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:22:33.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:33 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:33.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:33 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:33.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:33 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:33.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:33 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:33.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:33 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi190.idzoff"]}]: dispatch 2024-05-06T12:22:33.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:33 smithi190 ceph-mon[79744]: Upgrade: It appears safe to stop mds.cephfs.smithi190.idzoff 2024-05-06T12:22:33.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:33 smithi190 ceph-mon[79744]: pgmap v215: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:22:33.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:33 smithi092 ceph-mon[91933]: Detected new or changed devices on smithi190 2024-05-06T12:22:33.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:33 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:33.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:33 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:33.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:33 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:22:33.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:33 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:22:33.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:33 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:33.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:33 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:22:33.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:33 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:33.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:33 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:33.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:33 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:33.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:33 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:33.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:33 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi190.idzoff"]}]: dispatch 2024-05-06T12:22:33.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:33 smithi092 ceph-mon[91933]: Upgrade: It appears safe to stop mds.cephfs.smithi190.idzoff 2024-05-06T12:22:33.475 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:33 smithi092 ceph-mon[91933]: pgmap v215: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:22:35.095 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:34 smithi190 ceph-mon[79744]: Upgrade: Updating mds.cephfs.smithi190.idzoff 2024-05-06T12:22:35.095 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:34 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:35.095 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:34 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi190.idzoff", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-06T12:22:35.095 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:34 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:22:35.096 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:34 smithi190 ceph-mon[79744]: Deploying daemon mds.cephfs.smithi190.idzoff on smithi190 2024-05-06T12:22:35.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:34 smithi092 ceph-mon[91933]: Upgrade: Updating mds.cephfs.smithi190.idzoff 2024-05-06T12:22:35.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:34 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:35.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:34 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi190.idzoff", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-06T12:22:35.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:34 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:22:35.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:34 smithi092 ceph-mon[91933]: Deploying daemon mds.cephfs.smithi190.idzoff on smithi190 2024-05-06T12:22:36.093 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:35 smithi190 ceph-mon[79744]: pgmap v216: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:22:36.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:35 smithi092 ceph-mon[91933]: pgmap v216: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:22:37.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.388+0000 7f9a70e41700 1 -- 172.21.15.92:0/1232321438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a6c0fdfd0 msgr2=0x7f9a6c0fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:37.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.388+0000 7f9a70e41700 1 --2- 172.21.15.92:0/1232321438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a6c0fdfd0 0x7f9a6c0fe440 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9a60004940 tx=0x7f9a60009e30 comp rx=0 tx=0).stop 2024-05-06T12:22:37.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.388+0000 7f9a70e41700 1 -- 172.21.15.92:0/1232321438 shutdown_connections 2024-05-06T12:22:37.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.388+0000 7f9a70e41700 1 --2- 172.21.15.92:0/1232321438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a6c0fdfd0 0x7f9a6c0fe440 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:37.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.388+0000 7f9a70e41700 1 --2- 172.21.15.92:0/1232321438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a6c0fce60 0x7f9a6c0fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:37.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.388+0000 7f9a70e41700 1 -- 172.21.15.92:0/1232321438 >> 172.21.15.92:0/1232321438 conn(0x7f9a6c0f8540 msgr2=0x7f9a6c0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:22:37.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.388+0000 7f9a70e41700 1 -- 172.21.15.92:0/1232321438 shutdown_connections 2024-05-06T12:22:37.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.388+0000 7f9a70e41700 1 -- 172.21.15.92:0/1232321438 wait complete. 2024-05-06T12:22:37.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.389+0000 7f9a70e41700 1 Processor -- start 2024-05-06T12:22:37.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.389+0000 7f9a70e41700 1 -- start start 2024-05-06T12:22:37.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.389+0000 7f9a70e41700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a6c0fce60 0x7f9a6c10d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:37.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.389+0000 7f9a70e41700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a6c0fdfd0 0x7f9a6c10ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:37.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.389+0000 7f9a70e41700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6c10e3c0 con 0x7f9a6c0fdfd0 2024-05-06T12:22:37.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.389+0000 7f9a70e41700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6c10e530 con 0x7f9a6c0fce60 2024-05-06T12:22:37.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.389+0000 7f9a69d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a6c0fdfd0 0x7f9a6c10ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:37.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.389+0000 7f9a69d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a6c0fdfd0 0x7f9a6c10ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55482/0 (socket says 172.21.15.92:55482) 2024-05-06T12:22:37.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.389+0000 7f9a69d9b700 1 -- 172.21.15.92:0/2020826571 learned_addr learned my addr 172.21.15.92:0/2020826571 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:22:37.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.389+0000 7f9a6a59c700 1 --2- 172.21.15.92:0/2020826571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a6c0fce60 0x7f9a6c10d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:37.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.389+0000 7f9a69d9b700 1 -- 172.21.15.92:0/2020826571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a6c0fce60 msgr2=0x7f9a6c10d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:37.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.389+0000 7f9a69d9b700 1 --2- 172.21.15.92:0/2020826571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a6c0fce60 0x7f9a6c10d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:37.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.389+0000 7f9a69d9b700 1 -- 172.21.15.92:0/2020826571 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a60005040 con 0x7f9a6c0fdfd0 2024-05-06T12:22:37.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.389+0000 7f9a69d9b700 1 --2- 172.21.15.92:0/2020826571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a6c0fdfd0 0x7f9a6c10ddc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000b3b0 tx=0x7f9a60009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:22:37.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.390+0000 7f9a5affd700 1 -- 172.21.15.92:0/2020826571 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a6000e040 con 0x7f9a6c0fdfd0 2024-05-06T12:22:37.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.390+0000 7f9a5affd700 1 -- 172.21.15.92:0/2020826571 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9a60004290 con 0x7f9a6c0fdfd0 2024-05-06T12:22:37.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.390+0000 7f9a5affd700 1 -- 172.21.15.92:0/2020826571 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a6001c630 con 0x7f9a6c0fdfd0 2024-05-06T12:22:37.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.390+0000 7f9a70e41700 1 -- 172.21.15.92:0/2020826571 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a6c112f30 con 0x7f9a6c0fdfd0 2024-05-06T12:22:37.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.390+0000 7f9a70e41700 1 -- 172.21.15.92:0/2020826571 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a6c1134c0 con 0x7f9a6c0fdfd0 2024-05-06T12:22:37.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.391+0000 7f9a70e41700 1 -- 172.21.15.92:0/2020826571 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a6c04c710 con 0x7f9a6c0fdfd0 2024-05-06T12:22:37.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.391+0000 7f9a5affd700 1 -- 172.21.15.92:0/2020826571 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9a6000bbf0 con 0x7f9a6c0fdfd0 2024-05-06T12:22:37.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.392+0000 7f9a5affd700 1 --2- 172.21.15.92:0/2020826571 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f9a5406e140 0x7f9a540705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:37.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.392+0000 7f9a5affd700 1 -- 172.21.15.92:0/2020826571 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f9a6008ef40 con 0x7f9a6c0fdfd0 2024-05-06T12:22:37.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.392+0000 7f9a6a59c700 1 --2- 172.21.15.92:0/2020826571 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f9a5406e140 0x7f9a540705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:37.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.392+0000 7f9a6a59c700 1 --2- 172.21.15.92:0/2020826571 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f9a5406e140 0x7f9a540705f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c009fd0 tx=0x7f9a5c009f60 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:22:37.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.396+0000 7f9a5affd700 1 -- 172.21.15.92:0/2020826571 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9a6005be20 con 0x7f9a6c0fdfd0 2024-05-06T12:22:37.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.612+0000 7f9a70e41700 1 -- 172.21.15.92:0/2020826571 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a6c1136b0 con 0x7f9a5406e140 2024-05-06T12:22:37.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.614+0000 7f9a5affd700 1 -- 172.21.15.92:0/2020826571 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9a6c1136b0 con 0x7f9a5406e140 2024-05-06T12:22:37.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.616+0000 7f9a70e41700 1 -- 172.21.15.92:0/2020826571 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f9a5406e140 msgr2=0x7f9a540705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:37.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.616+0000 7f9a70e41700 1 --2- 172.21.15.92:0/2020826571 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f9a5406e140 0x7f9a540705f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c009fd0 tx=0x7f9a5c009f60 comp rx=0 tx=0).stop 2024-05-06T12:22:37.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.616+0000 7f9a70e41700 1 -- 172.21.15.92:0/2020826571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a6c0fdfd0 msgr2=0x7f9a6c10ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:37.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.616+0000 7f9a70e41700 1 --2- 172.21.15.92:0/2020826571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a6c0fdfd0 0x7f9a6c10ddc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000b3b0 tx=0x7f9a60009e30 comp rx=0 tx=0).stop 2024-05-06T12:22:37.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.616+0000 7f9a70e41700 1 -- 172.21.15.92:0/2020826571 shutdown_connections 2024-05-06T12:22:37.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.616+0000 7f9a70e41700 1 --2- 172.21.15.92:0/2020826571 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f9a5406e140 0x7f9a540705f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:37.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.616+0000 7f9a70e41700 1 --2- 172.21.15.92:0/2020826571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a6c0fce60 0x7f9a6c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:37.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.616+0000 7f9a70e41700 1 --2- 172.21.15.92:0/2020826571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a6c0fdfd0 0x7f9a6c10ddc0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:37.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.616+0000 7f9a70e41700 1 -- 172.21.15.92:0/2020826571 >> 172.21.15.92:0/2020826571 conn(0x7f9a6c0f8540 msgr2=0x7f9a6c0f9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:22:37.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.616+0000 7f9a70e41700 1 -- 172.21.15.92:0/2020826571 shutdown_connections 2024-05-06T12:22:37.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.616+0000 7f9a70e41700 1 -- 172.21.15.92:0/2020826571 wait complete. 2024-05-06T12:22:37.629 INFO:teuthology.orchestra.run.smithi092.stdout:true 2024-05-06T12:22:37.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.718+0000 7fd0abfb2700 1 -- 172.21.15.92:0/840577893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0a40fe080 msgr2=0x7fd0a40fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:37.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.718+0000 7fd0abfb2700 1 --2- 172.21.15.92:0/840577893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0a40fe080 0x7fd0a40fe4f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0004940 tx=0x7fd0a0009e30 comp rx=0 tx=0).stop 2024-05-06T12:22:37.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.718+0000 7fd0abfb2700 1 -- 172.21.15.92:0/840577893 shutdown_connections 2024-05-06T12:22:37.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.718+0000 7fd0abfb2700 1 --2- 172.21.15.92:0/840577893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0a40fe080 0x7fd0a40fe4f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:37.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.718+0000 7fd0abfb2700 1 --2- 172.21.15.92:0/840577893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0a40fcf10 0x7fd0a40fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:37.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.718+0000 7fd0abfb2700 1 -- 172.21.15.92:0/840577893 >> 172.21.15.92:0/840577893 conn(0x7fd0a40f8560 msgr2=0x7fd0a40fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:22:37.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.718+0000 7fd0abfb2700 1 -- 172.21.15.92:0/840577893 shutdown_connections 2024-05-06T12:22:37.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.718+0000 7fd0abfb2700 1 -- 172.21.15.92:0/840577893 wait complete. 2024-05-06T12:22:37.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.719+0000 7fd0abfb2700 1 Processor -- start 2024-05-06T12:22:37.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.719+0000 7fd0abfb2700 1 -- start start 2024-05-06T12:22:37.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.719+0000 7fd0abfb2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0a40fcf10 0x7fd0a410da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:37.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.719+0000 7fd0abfb2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0a40fe080 0x7fd0a410df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:37.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.719+0000 7fd0abfb2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a410e5d0 con 0x7fd0a40fe080 2024-05-06T12:22:37.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.719+0000 7fd0abfb2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a410edc0 con 0x7fd0a40fcf10 2024-05-06T12:22:37.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.719+0000 7fd0a954d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0a40fe080 0x7fd0a410df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:37.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.719+0000 7fd0a954d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0a40fe080 0x7fd0a410df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55506/0 (socket says 172.21.15.92:55506) 2024-05-06T12:22:37.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.719+0000 7fd0a954d700 1 -- 172.21.15.92:0/2440948537 learned_addr learned my addr 172.21.15.92:0/2440948537 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:22:37.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.719+0000 7fd0a9d4e700 1 --2- 172.21.15.92:0/2440948537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0a40fcf10 0x7fd0a410da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:37.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.719+0000 7fd0a954d700 1 -- 172.21.15.92:0/2440948537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0a40fcf10 msgr2=0x7fd0a410da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:37.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.719+0000 7fd0a954d700 1 --2- 172.21.15.92:0/2440948537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0a40fcf10 0x7fd0a410da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:37.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.719+0000 7fd0a954d700 1 -- 172.21.15.92:0/2440948537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a0005040 con 0x7fd0a40fe080 2024-05-06T12:22:37.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.719+0000 7fd0a9d4e700 1 --2- 172.21.15.92:0/2440948537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0a40fcf10 0x7fd0a410da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-06T12:22:37.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.719+0000 7fd0a954d700 1 --2- 172.21.15.92:0/2440948537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0a40fe080 0x7fd0a410df40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009150 tx=0x7fd0a0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:22:37.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.720+0000 7fd09a7fc700 1 -- 172.21.15.92:0/2440948537 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0a000e040 con 0x7fd0a40fe080 2024-05-06T12:22:37.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.720+0000 7fd09a7fc700 1 -- 172.21.15.92:0/2440948537 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd0a000bcc0 con 0x7fd0a40fe080 2024-05-06T12:22:37.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.720+0000 7fd0abfb2700 1 -- 172.21.15.92:0/2440948537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0a410f040 con 0x7fd0a40fe080 2024-05-06T12:22:37.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.720+0000 7fd09a7fc700 1 -- 172.21.15.92:0/2440948537 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0a001c630 con 0x7fd0a40fe080 2024-05-06T12:22:37.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.720+0000 7fd0abfb2700 1 -- 172.21.15.92:0/2440948537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0a410f600 con 0x7fd0a40fe080 2024-05-06T12:22:37.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.721+0000 7fd0abfb2700 1 -- 172.21.15.92:0/2440948537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0a405f5c0 con 0x7fd0a40fe080 2024-05-06T12:22:37.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.721+0000 7fd09a7fc700 1 -- 172.21.15.92:0/2440948537 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd0a0013420 con 0x7fd0a40fe080 2024-05-06T12:22:37.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.722+0000 7fd09a7fc700 1 --2- 172.21.15.92:0/2440948537 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd09006e140 0x7fd0900705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:37.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.722+0000 7fd09a7fc700 1 -- 172.21.15.92:0/2440948537 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd0a008e830 con 0x7fd0a40fe080 2024-05-06T12:22:37.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.722+0000 7fd0a9d4e700 1 --2- 172.21.15.92:0/2440948537 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd09006e140 0x7fd0900705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:37.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.722+0000 7fd0a9d4e700 1 --2- 172.21.15.92:0/2440948537 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd09006e140 0x7fd0900705f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd094009b10 tx=0x7fd094008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:22:37.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.727+0000 7fd09a7fc700 1 -- 172.21.15.92:0/2440948537 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd0a005b710 con 0x7fd0a40fe080 2024-05-06T12:22:37.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.946+0000 7fd0abfb2700 1 -- 172.21.15.92:0/2440948537 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd0a410f8e0 con 0x7fd09006e140 2024-05-06T12:22:37.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.947+0000 7fd09a7fc700 1 -- 172.21.15.92:0/2440948537 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd0a410f8e0 con 0x7fd09006e140 2024-05-06T12:22:37.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.949+0000 7fd0abfb2700 1 -- 172.21.15.92:0/2440948537 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd09006e140 msgr2=0x7fd0900705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:37.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.949+0000 7fd0abfb2700 1 --2- 172.21.15.92:0/2440948537 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd09006e140 0x7fd0900705f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd094009b10 tx=0x7fd094008040 comp rx=0 tx=0).stop 2024-05-06T12:22:37.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.949+0000 7fd0abfb2700 1 -- 172.21.15.92:0/2440948537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0a40fe080 msgr2=0x7fd0a410df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:37.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.949+0000 7fd0abfb2700 1 --2- 172.21.15.92:0/2440948537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0a40fe080 0x7fd0a410df40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009150 tx=0x7fd0a0009e30 comp rx=0 tx=0).stop 2024-05-06T12:22:37.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.950+0000 7fd0abfb2700 1 -- 172.21.15.92:0/2440948537 shutdown_connections 2024-05-06T12:22:37.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.950+0000 7fd0abfb2700 1 --2- 172.21.15.92:0/2440948537 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd09006e140 0x7fd0900705f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:37.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.950+0000 7fd0abfb2700 1 --2- 172.21.15.92:0/2440948537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0a40fcf10 0x7fd0a410da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:37.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.950+0000 7fd0abfb2700 1 --2- 172.21.15.92:0/2440948537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0a40fe080 0x7fd0a410df40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:37.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.950+0000 7fd0abfb2700 1 -- 172.21.15.92:0/2440948537 >> 172.21.15.92:0/2440948537 conn(0x7fd0a40f8560 msgr2=0x7fd0a40f9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:22:37.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.950+0000 7fd0abfb2700 1 -- 172.21.15.92:0/2440948537 shutdown_connections 2024-05-06T12:22:37.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:37.950+0000 7fd0abfb2700 1 -- 172.21.15.92:0/2440948537 wait complete. 2024-05-06T12:22:37.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:37 smithi092 ceph-mon[91933]: osdmap e86: 6 total, 6 up, 6 in 2024-05-06T12:22:37.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:37 smithi092 ceph-mon[91933]: fsmap cephfs:1 {0=cephfs.smithi092.ijsqgp=up:active} 2 up:standby 2024-05-06T12:22:37.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:37 smithi092 ceph-mon[91933]: pgmap v218: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:22:38.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.051+0000 7f4d9cc95700 1 -- 172.21.15.92:0/4267507688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d980fcff0 msgr2=0x7f4d980fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:38.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.051+0000 7f4d9cc95700 1 --2- 172.21.15.92:0/4267507688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d980fcff0 0x7f4d980fd460 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c004940 tx=0x7f4d8c009e30 comp rx=0 tx=0).stop 2024-05-06T12:22:38.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.051+0000 7f4d9cc95700 1 -- 172.21.15.92:0/4267507688 shutdown_connections 2024-05-06T12:22:38.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.051+0000 7f4d9cc95700 1 --2- 172.21.15.92:0/4267507688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d980fcff0 0x7f4d980fd460 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:38.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.051+0000 7f4d9cc95700 1 --2- 172.21.15.92:0/4267507688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d980fbd40 0x7f4d980fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:38.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.051+0000 7f4d9cc95700 1 -- 172.21.15.92:0/4267507688 >> 172.21.15.92:0/4267507688 conn(0x7f4d980f74c0 msgr2=0x7f4d980f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:22:38.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.051+0000 7f4d9cc95700 1 -- 172.21.15.92:0/4267507688 shutdown_connections 2024-05-06T12:22:38.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.051+0000 7f4d9cc95700 1 -- 172.21.15.92:0/4267507688 wait complete. 2024-05-06T12:22:38.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.051+0000 7f4d9cc95700 1 Processor -- start 2024-05-06T12:22:38.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.051+0000 7f4d9cc95700 1 -- start start 2024-05-06T12:22:38.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.052+0000 7f4d9cc95700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d980fbd40 0x7f4d9810da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:38.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.052+0000 7f4d9cc95700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d980fcff0 0x7f4d9810df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:38.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.052+0000 7f4d9cc95700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9810e550 con 0x7f4d980fbd40 2024-05-06T12:22:38.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.052+0000 7f4d9cc95700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9810eda0 con 0x7f4d980fcff0 2024-05-06T12:22:38.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.052+0000 7f4d9659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d980fbd40 0x7f4d9810da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:38.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.052+0000 7f4d9659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d980fbd40 0x7f4d9810da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55530/0 (socket says 172.21.15.92:55530) 2024-05-06T12:22:38.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.052+0000 7f4d9659c700 1 -- 172.21.15.92:0/3391831909 learned_addr learned my addr 172.21.15.92:0/3391831909 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:22:38.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.052+0000 7f4d95d9b700 1 --2- 172.21.15.92:0/3391831909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d980fcff0 0x7f4d9810df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:38.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.052+0000 7f4d9659c700 1 -- 172.21.15.92:0/3391831909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d980fcff0 msgr2=0x7f4d9810df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:38.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.052+0000 7f4d9659c700 1 --2- 172.21.15.92:0/3391831909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d980fcff0 0x7f4d9810df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:38.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.052+0000 7f4d9659c700 1 -- 172.21.15.92:0/3391831909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d8c005040 con 0x7f4d980fbd40 2024-05-06T12:22:38.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.052+0000 7f4d9659c700 1 --2- 172.21.15.92:0/3391831909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d980fbd40 0x7f4d9810da10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4d8800c830 tx=0x7f4d88004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:22:38.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.052+0000 7f4d86ffd700 1 -- 172.21.15.92:0/3391831909 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d8800a760 con 0x7f4d980fbd40 2024-05-06T12:22:38.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.052+0000 7f4d9cc95700 1 -- 172.21.15.92:0/3391831909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d9810f080 con 0x7f4d980fbd40 2024-05-06T12:22:38.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.052+0000 7f4d86ffd700 1 -- 172.21.15.92:0/3391831909 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d8800ad90 con 0x7f4d980fbd40 2024-05-06T12:22:38.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.052+0000 7f4d86ffd700 1 -- 172.21.15.92:0/3391831909 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d880113a0 con 0x7f4d980fbd40 2024-05-06T12:22:38.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.052+0000 7f4d9cc95700 1 -- 172.21.15.92:0/3391831909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d9810f5c0 con 0x7f4d980fbd40 2024-05-06T12:22:38.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.053+0000 7f4d9cc95700 1 -- 172.21.15.92:0/3391831909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d98101b80 con 0x7f4d980fbd40 2024-05-06T12:22:38.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.054+0000 7f4d86ffd700 1 -- 172.21.15.92:0/3391831909 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4d88012400 con 0x7f4d980fbd40 2024-05-06T12:22:38.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.054+0000 7f4d86ffd700 1 --2- 172.21.15.92:0/3391831909 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4d8006e150 0x7f4d80070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:38.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.054+0000 7f4d86ffd700 1 -- 172.21.15.92:0/3391831909 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4d8808dbb0 con 0x7f4d980fbd40 2024-05-06T12:22:38.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.054+0000 7f4d95d9b700 1 --2- 172.21.15.92:0/3391831909 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4d8006e150 0x7f4d80070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:38.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.055+0000 7f4d95d9b700 1 --2- 172.21.15.92:0/3391831909 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4d8006e150 0x7f4d80070600 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c00b3b0 tx=0x7f4d8c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:22:38.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.059+0000 7f4d86ffd700 1 -- 172.21.15.92:0/3391831909 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4d8805aa90 con 0x7f4d980fbd40 2024-05-06T12:22:38.094 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:37 smithi190 ceph-mon[79744]: osdmap e86: 6 total, 6 up, 6 in 2024-05-06T12:22:38.094 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:37 smithi190 ceph-mon[79744]: fsmap cephfs:1 {0=cephfs.smithi092.ijsqgp=up:active} 2 up:standby 2024-05-06T12:22:38.094 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:37 smithi190 ceph-mon[79744]: pgmap v218: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:22:38.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.262+0000 7f4d9cc95700 1 -- 172.21.15.92:0/3391831909 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d9810f8a0 con 0x7f4d8006e150 2024-05-06T12:22:38.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.272+0000 7f4d86ffd700 1 -- 172.21.15.92:0/3391831909 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f4d9810f8a0 con 0x7f4d8006e150 2024-05-06T12:22:38.275 INFO:teuthology.orchestra.run.smithi092.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-06T12:22:38.275 INFO:teuthology.orchestra.run.smithi092.stdout:alertmanager.smithi092 smithi092 *:9093,9094 running (7m) 26s ago 16m 25.2M - 0.25.0 c8568f914cd2 be5637d7d25a 2024-05-06T12:22:38.275 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi092 smithi092 running (5m) 26s ago 16m 6660k - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 8da0494e1bdc 2024-05-06T12:22:38.275 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi190 smithi190 running (4m) 10s ago 15m 6665k - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 641a7cdaa56e 2024-05-06T12:22:38.275 INFO:teuthology.orchestra.run.smithi092.stdout:grafana.smithi092 smithi092 *:3000 running (7m) 26s ago 15m 81.1M - 9.4.7 954c08fa6188 9180cb3e5258 2024-05-06T12:22:38.275 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.ijsqgp smithi092 running (53s) 26s ago 10m 203M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 40b4cdd3be40 2024-05-06T12:22:38.276 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.rosbll smithi092 running (31s) 26s ago 10m 12.5M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 550f8238d95f 2024-05-06T12:22:38.276 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.byjvgf smithi190 running (14s) 10s ago 10m 11.5M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 0d26051c908e 2024-05-06T12:22:38.276 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.idzoff smithi190 running (10m) 10s ago 10m 10.9M - 17.2.7-925-ge81df420 96d5ae7fd4b4 20f2641696ef 2024-05-06T12:22:38.276 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi092.gyzeqb smithi092 *:8443,9283,8765 running (7m) 26s ago 17m 509M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 de9d2198327a 2024-05-06T12:22:38.276 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi190.kczxkh smithi190 *:8443,9283,8765 running (6m) 10s ago 15m 451M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 e84ab3758ac7 2024-05-06T12:22:38.276 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi092 smithi092 running (6m) 26s ago 17m 51.4M 2048M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 ac07f7158087 2024-05-06T12:22:38.276 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi190 smithi190 running (5m) 10s ago 15m 39.7M 2048M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 7a3c08eb716f 2024-05-06T12:22:38.276 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi092 smithi092 *:9100 running (7m) 26s ago 16m 15.1M - 1.5.0 0da6a335fe13 c796446b206f 2024-05-06T12:22:38.276 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi190 smithi190 *:9100 running (7m) 10s ago 15m 16.4M - 1.5.0 0da6a335fe13 647e28111a76 2024-05-06T12:22:38.276 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 smithi092 running (4m) 26s ago 14m 92.3M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 8c96c75b25f4 2024-05-06T12:22:38.276 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 smithi092 running (4m) 26s ago 14m 95.6M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 e8007a510f08 2024-05-06T12:22:38.276 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 smithi092 running (3m) 26s ago 13m 88.8M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 2beca7448789 2024-05-06T12:22:38.276 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 smithi190 running (2m) 10s ago 13m 98.5M 2581M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 914ada199f93 2024-05-06T12:22:38.276 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 smithi190 running (2m) 10s ago 12m 88.2M 2581M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 371c0be776c7 2024-05-06T12:22:38.277 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 smithi190 running (104s) 10s ago 12m 84.5M 2581M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 067bc34cdfb1 2024-05-06T12:22:38.277 INFO:teuthology.orchestra.run.smithi092.stdout:prometheus.smithi092 smithi092 *:9095 running (6m) 26s ago 15m 63.0M - 2.43.0 a07b618ecd1d bc5445691527 2024-05-06T12:22:38.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.275+0000 7f4d9cc95700 1 -- 172.21.15.92:0/3391831909 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4d8006e150 msgr2=0x7f4d80070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:38.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.275+0000 7f4d9cc95700 1 --2- 172.21.15.92:0/3391831909 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4d8006e150 0x7f4d80070600 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c00b3b0 tx=0x7f4d8c009e30 comp rx=0 tx=0).stop 2024-05-06T12:22:38.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.275+0000 7f4d9cc95700 1 -- 172.21.15.92:0/3391831909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d980fbd40 msgr2=0x7f4d9810da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:38.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.275+0000 7f4d9cc95700 1 --2- 172.21.15.92:0/3391831909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d980fbd40 0x7f4d9810da10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4d8800c830 tx=0x7f4d88004470 comp rx=0 tx=0).stop 2024-05-06T12:22:38.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.275+0000 7f4d9cc95700 1 -- 172.21.15.92:0/3391831909 shutdown_connections 2024-05-06T12:22:38.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.275+0000 7f4d9cc95700 1 --2- 172.21.15.92:0/3391831909 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4d8006e150 0x7f4d80070600 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:38.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.275+0000 7f4d9cc95700 1 --2- 172.21.15.92:0/3391831909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d980fbd40 0x7f4d9810da10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:38.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.275+0000 7f4d9cc95700 1 --2- 172.21.15.92:0/3391831909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d980fcff0 0x7f4d9810df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:38.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.275+0000 7f4d9cc95700 1 -- 172.21.15.92:0/3391831909 >> 172.21.15.92:0/3391831909 conn(0x7f4d980f74c0 msgr2=0x7f4d980f8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:22:38.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.275+0000 7f4d9cc95700 1 -- 172.21.15.92:0/3391831909 shutdown_connections 2024-05-06T12:22:38.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.275+0000 7f4d9cc95700 1 -- 172.21.15.92:0/3391831909 wait complete. 2024-05-06T12:22:38.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.375+0000 7f7216f99700 1 -- 172.21.15.92:0/917503039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72100fcf70 msgr2=0x7f72100fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:38.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.375+0000 7f7216f99700 1 --2- 172.21.15.92:0/917503039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72100fcf70 0x7f72100fd3e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f71fc004940 tx=0x7f71fc009e30 comp rx=0 tx=0).stop 2024-05-06T12:22:38.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.375+0000 7f7216f99700 1 -- 172.21.15.92:0/917503039 shutdown_connections 2024-05-06T12:22:38.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.375+0000 7f7216f99700 1 --2- 172.21.15.92:0/917503039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72100fcf70 0x7f72100fd3e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:38.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.375+0000 7f7216f99700 1 --2- 172.21.15.92:0/917503039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72100fbcc0 0x7f72100fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:38.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.375+0000 7f7216f99700 1 -- 172.21.15.92:0/917503039 >> 172.21.15.92:0/917503039 conn(0x7f72100f7440 msgr2=0x7f72100f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:22:38.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.376+0000 7f7216f99700 1 -- 172.21.15.92:0/917503039 shutdown_connections 2024-05-06T12:22:38.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.376+0000 7f7216f99700 1 -- 172.21.15.92:0/917503039 wait complete. 2024-05-06T12:22:38.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.376+0000 7f7216f99700 1 Processor -- start 2024-05-06T12:22:38.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.376+0000 7f7216f99700 1 -- start start 2024-05-06T12:22:38.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.377+0000 7f7216f99700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72100fbcc0 0x7f7210066cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:38.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.377+0000 7f7216f99700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72100fcf70 0x7f7210112790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:38.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.377+0000 7f7216f99700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7210112cd0 con 0x7f72100fbcc0 2024-05-06T12:22:38.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.377+0000 7f7216f99700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7210112e40 con 0x7f72100fcf70 2024-05-06T12:22:38.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.377+0000 7f7214d35700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72100fbcc0 0x7f7210066cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:38.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.377+0000 7f7214d35700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72100fbcc0 0x7f7210066cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55546/0 (socket says 172.21.15.92:55546) 2024-05-06T12:22:38.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.377+0000 7f7214d35700 1 -- 172.21.15.92:0/4052959845 learned_addr learned my addr 172.21.15.92:0/4052959845 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:22:38.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.377+0000 7f7214d35700 1 -- 172.21.15.92:0/4052959845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72100fcf70 msgr2=0x7f7210112790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:38.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.377+0000 7f7214d35700 1 --2- 172.21.15.92:0/4052959845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72100fcf70 0x7f7210112790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:38.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.377+0000 7f7214d35700 1 -- 172.21.15.92:0/4052959845 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71fc005040 con 0x7f72100fbcc0 2024-05-06T12:22:38.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.377+0000 7f7214d35700 1 --2- 172.21.15.92:0/4052959845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72100fbcc0 0x7f7210066cd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f720400b790 tx=0x7f7204009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:22:38.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.377+0000 7f720d7fa700 1 -- 172.21.15.92:0/4052959845 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f720400f770 con 0x7f72100fbcc0 2024-05-06T12:22:38.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.378+0000 7f720d7fa700 1 -- 172.21.15.92:0/4052959845 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f720400fda0 con 0x7f72100fbcc0 2024-05-06T12:22:38.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.378+0000 7f7216f99700 1 -- 172.21.15.92:0/4052959845 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7210113120 con 0x7f72100fbcc0 2024-05-06T12:22:38.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.378+0000 7f720d7fa700 1 -- 172.21.15.92:0/4052959845 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f720400d310 con 0x7f72100fbcc0 2024-05-06T12:22:38.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.378+0000 7f7216f99700 1 -- 172.21.15.92:0/4052959845 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72101136e0 con 0x7f72100fbcc0 2024-05-06T12:22:38.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.379+0000 7f7216f99700 1 -- 172.21.15.92:0/4052959845 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f721004c710 con 0x7f72100fbcc0 2024-05-06T12:22:38.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.379+0000 7f720d7fa700 1 -- 172.21.15.92:0/4052959845 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f720400f8d0 con 0x7f72100fbcc0 2024-05-06T12:22:38.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.380+0000 7f720d7fa700 1 --2- 172.21.15.92:0/4052959845 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f720006e140 0x7f72000705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:38.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.380+0000 7f720ffff700 1 --2- 172.21.15.92:0/4052959845 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f720006e140 0x7f72000705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:38.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.380+0000 7f720d7fa700 1 -- 172.21.15.92:0/4052959845 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f720408da90 con 0x7f72100fbcc0 2024-05-06T12:22:38.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.380+0000 7f720ffff700 1 --2- 172.21.15.92:0/4052959845 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f720006e140 0x7f72000705f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f71fc004f90 tx=0x7f71fc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:22:38.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.384+0000 7f720d7fa700 1 -- 172.21.15.92:0/4052959845 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f720405aa30 con 0x7f72100fbcc0 2024-05-06T12:22:38.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.665+0000 7f7216f99700 1 -- 172.21.15.92:0/4052959845 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f72101139b0 con 0x7f72100fbcc0 2024-05-06T12:22:38.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.666+0000 7f720d7fa700 1 -- 172.21.15.92:0/4052959845 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f720405a5c0 con 0x7f72100fbcc0 2024-05-06T12:22:38.668 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:22:38.668 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": { 2024-05-06T12:22:38.668 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:22:38.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:22:38.669 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": { 2024-05-06T12:22:38.669 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:22:38.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:22:38.669 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": { 2024-05-06T12:22:38.669 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 6 2024-05-06T12:22:38.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:22:38.669 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-05-06T12:22:38.669 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 3 2024-05-06T12:22:38.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:22:38.669 INFO:teuthology.orchestra.run.smithi092.stdout: "overall": { 2024-05-06T12:22:38.669 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 13 2024-05-06T12:22:38.669 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-06T12:22:38.669 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:22:38.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.668+0000 7f7216f99700 1 -- 172.21.15.92:0/4052959845 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f720006e140 msgr2=0x7f72000705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:38.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.668+0000 7f7216f99700 1 --2- 172.21.15.92:0/4052959845 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f720006e140 0x7f72000705f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f71fc004f90 tx=0x7f71fc009e30 comp rx=0 tx=0).stop 2024-05-06T12:22:38.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.668+0000 7f7216f99700 1 -- 172.21.15.92:0/4052959845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72100fbcc0 msgr2=0x7f7210066cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:38.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.668+0000 7f7216f99700 1 --2- 172.21.15.92:0/4052959845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72100fbcc0 0x7f7210066cd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f720400b790 tx=0x7f7204009250 comp rx=0 tx=0).stop 2024-05-06T12:22:38.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.668+0000 7f7216f99700 1 -- 172.21.15.92:0/4052959845 shutdown_connections 2024-05-06T12:22:38.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.668+0000 7f7216f99700 1 --2- 172.21.15.92:0/4052959845 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f720006e140 0x7f72000705f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:38.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.668+0000 7f7216f99700 1 --2- 172.21.15.92:0/4052959845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72100fbcc0 0x7f7210066cd0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:38.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.668+0000 7f7216f99700 1 --2- 172.21.15.92:0/4052959845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72100fcf70 0x7f7210112790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:38.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.668+0000 7f7216f99700 1 -- 172.21.15.92:0/4052959845 >> 172.21.15.92:0/4052959845 conn(0x7f72100f7440 msgr2=0x7f72100f8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:22:38.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.668+0000 7f7216f99700 1 -- 172.21.15.92:0/4052959845 shutdown_connections 2024-05-06T12:22:38.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.668+0000 7f7216f99700 1 -- 172.21.15.92:0/4052959845 wait complete. 2024-05-06T12:22:38.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.768+0000 7f7a76f09700 1 -- 172.21.15.92:0/3334499686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a700fbd40 msgr2=0x7f7a700fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:38.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.768+0000 7f7a76f09700 1 --2- 172.21.15.92:0/3334499686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a700fbd40 0x7f7a700fc150 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c009ce0 tx=0x7f7a6c004d10 comp rx=0 tx=0).stop 2024-05-06T12:22:38.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.768+0000 7f7a76f09700 1 -- 172.21.15.92:0/3334499686 shutdown_connections 2024-05-06T12:22:38.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.768+0000 7f7a76f09700 1 --2- 172.21.15.92:0/3334499686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a700fcff0 0x7f7a700fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:38.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.768+0000 7f7a76f09700 1 --2- 172.21.15.92:0/3334499686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a700fbd40 0x7f7a700fc150 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:38.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.768+0000 7f7a76f09700 1 -- 172.21.15.92:0/3334499686 >> 172.21.15.92:0/3334499686 conn(0x7f7a700f74c0 msgr2=0x7f7a700f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:22:38.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.768+0000 7f7a76f09700 1 -- 172.21.15.92:0/3334499686 shutdown_connections 2024-05-06T12:22:38.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.768+0000 7f7a76f09700 1 -- 172.21.15.92:0/3334499686 wait complete. 2024-05-06T12:22:38.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.768+0000 7f7a76f09700 1 Processor -- start 2024-05-06T12:22:38.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.768+0000 7f7a76f09700 1 -- start start 2024-05-06T12:22:38.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.769+0000 7f7a76f09700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a700fbd40 0x7f7a70110820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:38.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.769+0000 7f7a76f09700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a700fcff0 0x7f7a70112d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:38.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.769+0000 7f7a76f09700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a70066bc0 con 0x7f7a700fbd40 2024-05-06T12:22:38.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.769+0000 7f7a76f09700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a70066d00 con 0x7f7a700fcff0 2024-05-06T12:22:38.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.769+0000 7f7a74ca5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a700fbd40 0x7f7a70110820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:38.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.769+0000 7f7a74ca5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a700fbd40 0x7f7a70110820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55562/0 (socket says 172.21.15.92:55562) 2024-05-06T12:22:38.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.769+0000 7f7a74ca5700 1 -- 172.21.15.92:0/2192135457 learned_addr learned my addr 172.21.15.92:0/2192135457 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:22:38.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.769+0000 7f7a67fff700 1 --2- 172.21.15.92:0/2192135457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a700fcff0 0x7f7a70112d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:38.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.769+0000 7f7a74ca5700 1 -- 172.21.15.92:0/2192135457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a700fcff0 msgr2=0x7f7a70112d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:38.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.769+0000 7f7a74ca5700 1 --2- 172.21.15.92:0/2192135457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a700fcff0 0x7f7a70112d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:38.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.769+0000 7f7a74ca5700 1 -- 172.21.15.92:0/2192135457 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6c009940 con 0x7f7a700fbd40 2024-05-06T12:22:38.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.769+0000 7f7a67fff700 1 --2- 172.21.15.92:0/2192135457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a700fcff0 0x7f7a70112d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-06T12:22:38.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.769+0000 7f7a74ca5700 1 --2- 172.21.15.92:0/2192135457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a700fbd40 0x7f7a70110820 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c009380 tx=0x7f7a6c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:22:38.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.769+0000 7f7a657fa700 1 -- 172.21.15.92:0/2192135457 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a6c00e040 con 0x7f7a700fbd40 2024-05-06T12:22:38.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.769+0000 7f7a76f09700 1 -- 172.21.15.92:0/2192135457 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a701132b0 con 0x7f7a700fbd40 2024-05-06T12:22:38.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.769+0000 7f7a657fa700 1 -- 172.21.15.92:0/2192135457 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a6c0128e0 con 0x7f7a700fbd40 2024-05-06T12:22:38.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.769+0000 7f7a657fa700 1 -- 172.21.15.92:0/2192135457 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a6c01ca10 con 0x7f7a700fbd40 2024-05-06T12:22:38.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.769+0000 7f7a76f09700 1 -- 172.21.15.92:0/2192135457 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a70113870 con 0x7f7a700fbd40 2024-05-06T12:22:38.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.770+0000 7f7a76f09700 1 -- 172.21.15.92:0/2192135457 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a7005f5c0 con 0x7f7a700fbd40 2024-05-06T12:22:38.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.771+0000 7f7a657fa700 1 -- 172.21.15.92:0/2192135457 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7a6c012a50 con 0x7f7a700fbd40 2024-05-06T12:22:38.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.771+0000 7f7a657fa700 1 --2- 172.21.15.92:0/2192135457 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7a5806e140 0x7f7a580705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:38.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.771+0000 7f7a657fa700 1 -- 172.21.15.92:0/2192135457 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7a6c08f750 con 0x7f7a700fbd40 2024-05-06T12:22:38.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.771+0000 7f7a67fff700 1 --2- 172.21.15.92:0/2192135457 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7a5806e140 0x7f7a580705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:38.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.772+0000 7f7a67fff700 1 --2- 172.21.15.92:0/2192135457 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7a5806e140 0x7f7a580705f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7a60009b80 tx=0x7f7a6000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:22:38.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:38.776+0000 7f7a657fa700 1 -- 172.21.15.92:0/2192135457 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7a6c058a80 con 0x7f7a700fbd40 2024-05-06T12:22:38.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:38 smithi092 ceph-mon[91933]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:22:38.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:38 smithi092 ceph-mon[91933]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:22:38.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:38 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/4052959845' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:39.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.023+0000 7f7a76f09700 1 -- 172.21.15.92:0/2192135457 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7a701047c0 con 0x7f7a700fbd40 2024-05-06T12:22:39.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.023+0000 7f7a657fa700 1 -- 172.21.15.92:0/2192135457 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1760 (secure 0 0 0) 0x7f7a6c012d00 con 0x7f7a700fbd40 2024-05-06T12:22:39.026 INFO:teuthology.orchestra.run.smithi092.stdout:e37 2024-05-06T12:22:39.026 INFO:teuthology.orchestra.run.smithi092.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-06T12:22:39.026 INFO:teuthology.orchestra.run.smithi092.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:22:39.026 INFO:teuthology.orchestra.run.smithi092.stdout:legacy client fscid: 1 2024-05-06T12:22:39.026 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:22:39.026 INFO:teuthology.orchestra.run.smithi092.stdout:Filesystem 'cephfs' (1) 2024-05-06T12:22:39.026 INFO:teuthology.orchestra.run.smithi092.stdout:fs_name cephfs 2024-05-06T12:22:39.026 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 34 2024-05-06T12:22:39.026 INFO:teuthology.orchestra.run.smithi092.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-06T12:22:39.027 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-05-06T12:11:44.952671+0000 2024-05-06T12:22:39.027 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2024-05-06T12:22:06.925528+0000 2024-05-06T12:22:39.027 INFO:teuthology.orchestra.run.smithi092.stdout:tableserver 0 2024-05-06T12:22:39.027 INFO:teuthology.orchestra.run.smithi092.stdout:root 0 2024-05-06T12:22:39.027 INFO:teuthology.orchestra.run.smithi092.stdout:session_timeout 60 2024-05-06T12:22:39.027 INFO:teuthology.orchestra.run.smithi092.stdout:session_autoclose 300 2024-05-06T12:22:39.027 INFO:teuthology.orchestra.run.smithi092.stdout:max_file_size 1099511627776 2024-05-06T12:22:39.027 INFO:teuthology.orchestra.run.smithi092.stdout:max_xattr_size 65536 2024-05-06T12:22:39.027 INFO:teuthology.orchestra.run.smithi092.stdout:required_client_features {} 2024-05-06T12:22:39.027 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure 0 2024-05-06T12:22:39.027 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure_osd_epoch 84 2024-05-06T12:22:39.027 INFO:teuthology.orchestra.run.smithi092.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-06T12:22:39.028 INFO:teuthology.orchestra.run.smithi092.stdout:max_mds 1 2024-05-06T12:22:39.028 INFO:teuthology.orchestra.run.smithi092.stdout:in 0 2024-05-06T12:22:39.028 INFO:teuthology.orchestra.run.smithi092.stdout:up {0=34438} 2024-05-06T12:22:39.028 INFO:teuthology.orchestra.run.smithi092.stdout:failed 2024-05-06T12:22:39.028 INFO:teuthology.orchestra.run.smithi092.stdout:damaged 2024-05-06T12:22:39.028 INFO:teuthology.orchestra.run.smithi092.stdout:stopped 1 2024-05-06T12:22:39.028 INFO:teuthology.orchestra.run.smithi092.stdout:data_pools [3] 2024-05-06T12:22:39.028 INFO:teuthology.orchestra.run.smithi092.stdout:metadata_pool 2 2024-05-06T12:22:39.028 INFO:teuthology.orchestra.run.smithi092.stdout:inline_data enabled 2024-05-06T12:22:39.028 INFO:teuthology.orchestra.run.smithi092.stdout:balancer 2024-05-06T12:22:39.028 INFO:teuthology.orchestra.run.smithi092.stdout:bal_rank_mask -1 2024-05-06T12:22:39.028 INFO:teuthology.orchestra.run.smithi092.stdout:standby_count_wanted 1 2024-05-06T12:22:39.028 INFO:teuthology.orchestra.run.smithi092.stdout:qdb_cluster leader: 34438 members: 34438 2024-05-06T12:22:39.028 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.ijsqgp{0:34438} state up:active seq 8 join_fscid=1 addr [v2:172.21.15.92:6826/4190573007,v1:172.21.15.92:6827/4190573007] compat {c=[1],r=[1],i=[1fff]}] 2024-05-06T12:22:39.029 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:22:39.029 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:22:39.029 INFO:teuthology.orchestra.run.smithi092.stdout:Standby daemons: 2024-05-06T12:22:39.029 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:22:39.029 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.rosbll{-1:34466} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.92:6828/2398082173,v1:172.21.15.92:6829/2398082173] compat {c=[1],r=[1],i=[1fff]}] 2024-05-06T12:22:39.029 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.byjvgf{-1:44309} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.190:6824/1219274291,v1:172.21.15.190:6825/1219274291] compat {c=[1],r=[1],i=[1fff]}] 2024-05-06T12:22:39.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.025+0000 7f7a76f09700 1 -- 172.21.15.92:0/2192135457 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7a5806e140 msgr2=0x7f7a580705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:39.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.025+0000 7f7a76f09700 1 --2- 172.21.15.92:0/2192135457 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7a5806e140 0x7f7a580705f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7a60009b80 tx=0x7f7a6000b040 comp rx=0 tx=0).stop 2024-05-06T12:22:39.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.025+0000 7f7a76f09700 1 -- 172.21.15.92:0/2192135457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a700fbd40 msgr2=0x7f7a70110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:39.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.025+0000 7f7a76f09700 1 --2- 172.21.15.92:0/2192135457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a700fbd40 0x7f7a70110820 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c009380 tx=0x7f7a6c004d10 comp rx=0 tx=0).stop 2024-05-06T12:22:39.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.025+0000 7f7a76f09700 1 -- 172.21.15.92:0/2192135457 shutdown_connections 2024-05-06T12:22:39.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.025+0000 7f7a76f09700 1 --2- 172.21.15.92:0/2192135457 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7a5806e140 0x7f7a580705f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:39.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.025+0000 7f7a76f09700 1 --2- 172.21.15.92:0/2192135457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a700fbd40 0x7f7a70110820 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:39.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.025+0000 7f7a76f09700 1 --2- 172.21.15.92:0/2192135457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a700fcff0 0x7f7a70112d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:39.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.025+0000 7f7a76f09700 1 -- 172.21.15.92:0/2192135457 >> 172.21.15.92:0/2192135457 conn(0x7f7a700f74c0 msgr2=0x7f7a700f8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:22:39.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.025+0000 7f7a76f09700 1 -- 172.21.15.92:0/2192135457 shutdown_connections 2024-05-06T12:22:39.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.025+0000 7f7a76f09700 1 -- 172.21.15.92:0/2192135457 wait complete. 2024-05-06T12:22:39.032 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 37 2024-05-06T12:22:39.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:38 smithi190 ceph-mon[79744]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:22:39.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:38 smithi190 ceph-mon[79744]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:22:39.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:38 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/4052959845' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:39.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.128+0000 7fdb3e1dc700 1 -- 172.21.15.92:0/739280540 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb380fcf60 msgr2=0x7fdb380fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:39.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.128+0000 7fdb3e1dc700 1 --2- 172.21.15.92:0/739280540 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb380fcf60 0x7fdb380fd3d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdb20004940 tx=0x7fdb20009e30 comp rx=0 tx=0).stop 2024-05-06T12:22:39.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.128+0000 7fdb3e1dc700 1 -- 172.21.15.92:0/739280540 shutdown_connections 2024-05-06T12:22:39.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.128+0000 7fdb3e1dc700 1 --2- 172.21.15.92:0/739280540 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb380fcf60 0x7fdb380fd3d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:39.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.128+0000 7fdb3e1dc700 1 --2- 172.21.15.92:0/739280540 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb380fbcb0 0x7fdb380fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:39.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.128+0000 7fdb3e1dc700 1 -- 172.21.15.92:0/739280540 >> 172.21.15.92:0/739280540 conn(0x7fdb380f7430 msgr2=0x7fdb380f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:22:39.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.128+0000 7fdb3e1dc700 1 -- 172.21.15.92:0/739280540 shutdown_connections 2024-05-06T12:22:39.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.128+0000 7fdb3e1dc700 1 -- 172.21.15.92:0/739280540 wait complete. 2024-05-06T12:22:39.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.128+0000 7fdb3e1dc700 1 Processor -- start 2024-05-06T12:22:39.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.128+0000 7fdb3e1dc700 1 -- start start 2024-05-06T12:22:39.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.129+0000 7fdb3e1dc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb380fbcb0 0x7fdb3810c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:39.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.129+0000 7fdb3e1dc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb380fcf60 0x7fdb3810ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:39.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.129+0000 7fdb3e1dc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3810d440 con 0x7fdb380fcf60 2024-05-06T12:22:39.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.129+0000 7fdb3e1dc700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3810dc90 con 0x7fdb380fbcb0 2024-05-06T12:22:39.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.129+0000 7fdb2effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb380fcf60 0x7fdb3810ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:39.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.129+0000 7fdb2effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb380fcf60 0x7fdb3810ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55586/0 (socket says 172.21.15.92:55586) 2024-05-06T12:22:39.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.129+0000 7fdb2effd700 1 -- 172.21.15.92:0/1806445076 learned_addr learned my addr 172.21.15.92:0/1806445076 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:22:39.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.129+0000 7fdb377fe700 1 --2- 172.21.15.92:0/1806445076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb380fbcb0 0x7fdb3810c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:39.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.129+0000 7fdb2effd700 1 -- 172.21.15.92:0/1806445076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb380fbcb0 msgr2=0x7fdb3810c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:39.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.129+0000 7fdb2effd700 1 --2- 172.21.15.92:0/1806445076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb380fbcb0 0x7fdb3810c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:39.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.129+0000 7fdb2effd700 1 -- 172.21.15.92:0/1806445076 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb20005040 con 0x7fdb380fcf60 2024-05-06T12:22:39.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.129+0000 7fdb2effd700 1 --2- 172.21.15.92:0/1806445076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb380fcf60 0x7fdb3810ce40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdb20009150 tx=0x7fdb20009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:22:39.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.129+0000 7fdb34ff9700 1 -- 172.21.15.92:0/1806445076 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb2000e040 con 0x7fdb380fcf60 2024-05-06T12:22:39.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.129+0000 7fdb34ff9700 1 -- 172.21.15.92:0/1806445076 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb2000bbf0 con 0x7fdb380fcf60 2024-05-06T12:22:39.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.129+0000 7fdb3e1dc700 1 -- 172.21.15.92:0/1806445076 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb3810df10 con 0x7fdb380fcf60 2024-05-06T12:22:39.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.129+0000 7fdb34ff9700 1 -- 172.21.15.92:0/1806445076 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb2001c630 con 0x7fdb380fcf60 2024-05-06T12:22:39.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.130+0000 7fdb3e1dc700 1 -- 172.21.15.92:0/1806445076 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb3810e450 con 0x7fdb380fcf60 2024-05-06T12:22:39.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.130+0000 7fdb3e1dc700 1 -- 172.21.15.92:0/1806445076 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb3805f5c0 con 0x7fdb380fcf60 2024-05-06T12:22:39.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.131+0000 7fdb34ff9700 1 -- 172.21.15.92:0/1806445076 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdb2000bd60 con 0x7fdb380fcf60 2024-05-06T12:22:39.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.131+0000 7fdb34ff9700 1 --2- 172.21.15.92:0/1806445076 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fdb2406e150 0x7fdb24070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:39.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.132+0000 7fdb34ff9700 1 -- 172.21.15.92:0/1806445076 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fdb2008ee70 con 0x7fdb380fcf60 2024-05-06T12:22:39.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.132+0000 7fdb377fe700 1 --2- 172.21.15.92:0/1806445076 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fdb2406e150 0x7fdb24070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:39.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.132+0000 7fdb377fe700 1 --2- 172.21.15.92:0/1806445076 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fdb2406e150 0x7fdb24070600 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdb28004f40 tx=0x7fdb2800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:22:39.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.136+0000 7fdb34ff9700 1 -- 172.21.15.92:0/1806445076 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fdb2005bd50 con 0x7fdb380fcf60 2024-05-06T12:22:39.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.359+0000 7fdb3e1dc700 1 -- 172.21.15.92:0/1806445076 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb3810e7d0 con 0x7fdb2406e150 2024-05-06T12:22:39.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.360+0000 7fdb34ff9700 1 -- 172.21.15.92:0/1806445076 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fdb3810e7d0 con 0x7fdb2406e150 2024-05-06T12:22:39.362 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:22:39.363 INFO:teuthology.orchestra.run.smithi092.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb", 2024-05-06T12:22:39.363 INFO:teuthology.orchestra.run.smithi092.stdout: "in_progress": true, 2024-05-06T12:22:39.363 INFO:teuthology.orchestra.run.smithi092.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-06T12:22:39.363 INFO:teuthology.orchestra.run.smithi092.stdout: "services_complete": [ 2024-05-06T12:22:39.363 INFO:teuthology.orchestra.run.smithi092.stdout: "osd", 2024-05-06T12:22:39.363 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr", 2024-05-06T12:22:39.363 INFO:teuthology.orchestra.run.smithi092.stdout: "mon", 2024-05-06T12:22:39.363 INFO:teuthology.orchestra.run.smithi092.stdout: "crash" 2024-05-06T12:22:39.363 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-05-06T12:22:39.363 INFO:teuthology.orchestra.run.smithi092.stdout: "progress": "15/21 daemons upgraded", 2024-05-06T12:22:39.363 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "Currently upgrading mds daemons", 2024-05-06T12:22:39.363 INFO:teuthology.orchestra.run.smithi092.stdout: "is_paused": false 2024-05-06T12:22:39.363 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:22:39.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.362+0000 7fdb3e1dc700 1 -- 172.21.15.92:0/1806445076 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fdb2406e150 msgr2=0x7fdb24070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:39.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.362+0000 7fdb3e1dc700 1 --2- 172.21.15.92:0/1806445076 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fdb2406e150 0x7fdb24070600 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdb28004f40 tx=0x7fdb2800b040 comp rx=0 tx=0).stop 2024-05-06T12:22:39.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.362+0000 7fdb3e1dc700 1 -- 172.21.15.92:0/1806445076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb380fcf60 msgr2=0x7fdb3810ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:39.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.362+0000 7fdb3e1dc700 1 --2- 172.21.15.92:0/1806445076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb380fcf60 0x7fdb3810ce40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdb20009150 tx=0x7fdb20009e30 comp rx=0 tx=0).stop 2024-05-06T12:22:39.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.362+0000 7fdb3e1dc700 1 -- 172.21.15.92:0/1806445076 shutdown_connections 2024-05-06T12:22:39.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.362+0000 7fdb3e1dc700 1 --2- 172.21.15.92:0/1806445076 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fdb2406e150 0x7fdb24070600 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:39.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.362+0000 7fdb3e1dc700 1 --2- 172.21.15.92:0/1806445076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb380fbcb0 0x7fdb3810c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:39.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.362+0000 7fdb3e1dc700 1 --2- 172.21.15.92:0/1806445076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb380fcf60 0x7fdb3810ce40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:39.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.362+0000 7fdb3e1dc700 1 -- 172.21.15.92:0/1806445076 >> 172.21.15.92:0/1806445076 conn(0x7fdb380f7430 msgr2=0x7fdb380f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:22:39.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.362+0000 7fdb3e1dc700 1 -- 172.21.15.92:0/1806445076 shutdown_connections 2024-05-06T12:22:39.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.362+0000 7fdb3e1dc700 1 -- 172.21.15.92:0/1806445076 wait complete. 2024-05-06T12:22:39.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.464+0000 7f3c5e400700 1 -- 172.21.15.92:0/92742069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c580fdfd0 msgr2=0x7f3c580fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:39.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.464+0000 7f3c5e400700 1 --2- 172.21.15.92:0/92742069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c580fdfd0 0x7f3c580fe440 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3c48004940 tx=0x7f3c48009e30 comp rx=0 tx=0).stop 2024-05-06T12:22:39.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.464+0000 7f3c5e400700 1 -- 172.21.15.92:0/92742069 shutdown_connections 2024-05-06T12:22:39.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.464+0000 7f3c5e400700 1 --2- 172.21.15.92:0/92742069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c580fdfd0 0x7f3c580fe440 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:39.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.464+0000 7f3c5e400700 1 --2- 172.21.15.92:0/92742069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c580fce60 0x7f3c580fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:39.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.464+0000 7f3c5e400700 1 -- 172.21.15.92:0/92742069 >> 172.21.15.92:0/92742069 conn(0x7f3c580f8560 msgr2=0x7f3c580fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:22:39.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.464+0000 7f3c5e400700 1 -- 172.21.15.92:0/92742069 shutdown_connections 2024-05-06T12:22:39.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.464+0000 7f3c5e400700 1 -- 172.21.15.92:0/92742069 wait complete. 2024-05-06T12:22:39.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.465+0000 7f3c5e400700 1 Processor -- start 2024-05-06T12:22:39.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.465+0000 7f3c5e400700 1 -- start start 2024-05-06T12:22:39.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.465+0000 7f3c5e400700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c580fce60 0x7f3c58071d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:39.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.465+0000 7f3c5e400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c580fdfd0 0x7f3c580702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:39.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.465+0000 7f3c5e400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c58070840 con 0x7f3c580fdfd0 2024-05-06T12:22:39.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.465+0000 7f3c5e400700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c58070980 con 0x7f3c580fce60 2024-05-06T12:22:39.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.465+0000 7f3c577fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c580fdfd0 0x7f3c580702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:39.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.465+0000 7f3c577fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c580fdfd0 0x7f3c580702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55614/0 (socket says 172.21.15.92:55614) 2024-05-06T12:22:39.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.465+0000 7f3c577fe700 1 -- 172.21.15.92:0/3924492662 learned_addr learned my addr 172.21.15.92:0/3924492662 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:22:39.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.465+0000 7f3c57fff700 1 --2- 172.21.15.92:0/3924492662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c580fce60 0x7f3c58071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:39.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.465+0000 7f3c577fe700 1 -- 172.21.15.92:0/3924492662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c580fce60 msgr2=0x7f3c58071d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:39.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.465+0000 7f3c577fe700 1 --2- 172.21.15.92:0/3924492662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c580fce60 0x7f3c58071d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:39.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.465+0000 7f3c577fe700 1 -- 172.21.15.92:0/3924492662 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c48005040 con 0x7f3c580fdfd0 2024-05-06T12:22:39.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.465+0000 7f3c577fe700 1 --2- 172.21.15.92:0/3924492662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c580fdfd0 0x7f3c580702b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3c4800b970 tx=0x7f3c48009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:22:39.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.466+0000 7f3c54ff9700 1 -- 172.21.15.92:0/3924492662 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c4800e040 con 0x7f3c580fdfd0 2024-05-06T12:22:39.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.466+0000 7f3c54ff9700 1 -- 172.21.15.92:0/3924492662 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c4800bcc0 con 0x7f3c580fdfd0 2024-05-06T12:22:39.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.466+0000 7f3c5e400700 1 -- 172.21.15.92:0/3924492662 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c58070c00 con 0x7f3c580fdfd0 2024-05-06T12:22:39.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.466+0000 7f3c54ff9700 1 -- 172.21.15.92:0/3924492662 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c48013660 con 0x7f3c580fdfd0 2024-05-06T12:22:39.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.466+0000 7f3c5e400700 1 -- 172.21.15.92:0/3924492662 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c58115a90 con 0x7f3c580fdfd0 2024-05-06T12:22:39.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.467+0000 7f3c5e400700 1 -- 172.21.15.92:0/3924492662 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c5805f5c0 con 0x7f3c580fdfd0 2024-05-06T12:22:39.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.467+0000 7f3c54ff9700 1 -- 172.21.15.92:0/3924492662 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3c4801ca00 con 0x7f3c580fdfd0 2024-05-06T12:22:39.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.468+0000 7f3c54ff9700 1 --2- 172.21.15.92:0/3924492662 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f3c4006e140 0x7f3c400705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:22:39.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.468+0000 7f3c54ff9700 1 -- 172.21.15.92:0/3924492662 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f3c4808e8a0 con 0x7f3c580fdfd0 2024-05-06T12:22:39.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.468+0000 7f3c57fff700 1 --2- 172.21.15.92:0/3924492662 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f3c4006e140 0x7f3c400705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:22:39.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.468+0000 7f3c57fff700 1 --2- 172.21.15.92:0/3924492662 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f3c4006e140 0x7f3c400705f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3c58071b20 tx=0x7f3c4c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:22:39.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.472+0000 7f3c54ff9700 1 -- 172.21.15.92:0/3924492662 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3c48057d00 con 0x7f3c580fdfd0 2024-05-06T12:22:39.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.750+0000 7f3c5e400700 1 -- 172.21.15.92:0/3924492662 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3c58115c20 con 0x7f3c580fdfd0 2024-05-06T12:22:39.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.751+0000 7f3c54ff9700 1 -- 172.21.15.92:0/3924492662 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f3c4805b3c0 con 0x7f3c580fdfd0 2024-05-06T12:22:39.753 INFO:teuthology.orchestra.run.smithi092.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-06T12:22:39.753 INFO:teuthology.orchestra.run.smithi092.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-06T12:22:39.754 INFO:teuthology.orchestra.run.smithi092.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-06T12:22:39.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.753+0000 7f3c5e400700 1 -- 172.21.15.92:0/3924492662 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f3c4006e140 msgr2=0x7f3c400705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:39.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.753+0000 7f3c5e400700 1 --2- 172.21.15.92:0/3924492662 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f3c4006e140 0x7f3c400705f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3c58071b20 tx=0x7f3c4c00b040 comp rx=0 tx=0).stop 2024-05-06T12:22:39.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.753+0000 7f3c5e400700 1 -- 172.21.15.92:0/3924492662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c580fdfd0 msgr2=0x7f3c580702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:22:39.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.753+0000 7f3c5e400700 1 --2- 172.21.15.92:0/3924492662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c580fdfd0 0x7f3c580702b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3c4800b970 tx=0x7f3c48009e30 comp rx=0 tx=0).stop 2024-05-06T12:22:39.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.753+0000 7f3c5e400700 1 -- 172.21.15.92:0/3924492662 shutdown_connections 2024-05-06T12:22:39.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.753+0000 7f3c5e400700 1 --2- 172.21.15.92:0/3924492662 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f3c4006e140 0x7f3c400705f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:39.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.753+0000 7f3c5e400700 1 --2- 172.21.15.92:0/3924492662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c580fce60 0x7f3c58071d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:39.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.753+0000 7f3c5e400700 1 --2- 172.21.15.92:0/3924492662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c580fdfd0 0x7f3c580702b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:22:39.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.753+0000 7f3c5e400700 1 -- 172.21.15.92:0/3924492662 >> 172.21.15.92:0/3924492662 conn(0x7f3c580f8560 msgr2=0x7f3c580f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:22:39.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.753+0000 7f3c5e400700 1 -- 172.21.15.92:0/3924492662 shutdown_connections 2024-05-06T12:22:39.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:22:39.753+0000 7f3c5e400700 1 -- 172.21.15.92:0/3924492662 wait complete. 2024-05-06T12:22:40.097 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:39 smithi190 ceph-mon[79744]: from='client.34484 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:22:40.097 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:39 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:22:40.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:39 smithi190 ceph-mon[79744]: pgmap v219: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:22:40.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:39 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/2192135457' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:22:40.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:39 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/3924492662' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:22:40.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:39 smithi092 ceph-mon[91933]: from='client.34484 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:22:40.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:39 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:22:40.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:39 smithi092 ceph-mon[91933]: pgmap v219: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:22:40.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:39 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/2192135457' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:22:40.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:39 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/3924492662' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:22:41.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:40 smithi190 ceph-mon[79744]: from='client.34496 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:22:41.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:40 smithi092 ceph-mon[91933]: from='client.34496 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:22:42.095 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:41 smithi190 ceph-mon[79744]: pgmap v220: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:22:42.095 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:41 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:42.095 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:41 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:42.095 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:41 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:22:42.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:41 smithi092 ceph-mon[91933]: pgmap v220: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:22:42.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:41 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:42.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:41 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:42.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:41 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:22:43.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:42 smithi190 ceph-mon[79744]: mds.? [v2:172.21.15.190:6826/2486629325,v1:172.21.15.190:6827/2486629325] up:boot 2024-05-06T12:22:43.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:42 smithi190 ceph-mon[79744]: fsmap cephfs:1 {0=cephfs.smithi092.ijsqgp=up:active} 3 up:standby 2024-05-06T12:22:43.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:42 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi190.idzoff"}]: dispatch 2024-05-06T12:22:43.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:42 smithi092 ceph-mon[91933]: mds.? [v2:172.21.15.190:6826/2486629325,v1:172.21.15.190:6827/2486629325] up:boot 2024-05-06T12:22:43.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:42 smithi092 ceph-mon[91933]: fsmap cephfs:1 {0=cephfs.smithi092.ijsqgp=up:active} 3 up:standby 2024-05-06T12:22:43.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:42 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi190.idzoff"}]: dispatch 2024-05-06T12:22:44.097 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:43 smithi190 ceph-mon[79744]: pgmap v221: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:22:44.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:43 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:44.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:43 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:44.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:43 smithi092 ceph-mon[91933]: pgmap v221: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:22:44.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:43 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:44.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:43 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:45.847 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:45 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:45.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:45 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:45.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:45 smithi190 ceph-mon[79744]: pgmap v222: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:22:45.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:45 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:45.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:45 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:45.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:45 smithi092 ceph-mon[91933]: pgmap v222: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:22:48.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:47 smithi190 ceph-mon[79744]: pgmap v223: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:22:48.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:47 smithi092 ceph-mon[91933]: pgmap v223: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:22:49.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:49.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:49.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:22:49.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:22:49.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:49.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:22:49.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:49.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:49.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:49.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:49.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:49.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:49 smithi190 ceph-mon[79744]: Upgrade: Setting container_image for all mds 2024-05-06T12:22:49.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:49.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi092.ijsqgp"}]: dispatch 2024-05-06T12:22:49.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi092.ijsqgp"}]': finished 2024-05-06T12:22:49.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi092.rosbll"}]: dispatch 2024-05-06T12:22:49.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi092.rosbll"}]': finished 2024-05-06T12:22:49.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi190.byjvgf"}]: dispatch 2024-05-06T12:22:49.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi190.byjvgf"}]': finished 2024-05-06T12:22:49.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi190.idzoff"}]: dispatch 2024-05-06T12:22:49.599 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi190.idzoff"}]': finished 2024-05-06T12:22:49.600 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:49 smithi190 ceph-mon[79744]: Upgrade: Scaling up filesystem cephfs max_mds to 2 2024-05-06T12:22:49.600 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:49 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-05-06T12:22:49.600 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:49 smithi190 ceph-mon[79744]: pgmap v224: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:22:49.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:49.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:49.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:22:49.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:22:49.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:49.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:22:49.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:49.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:49.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:49.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:49.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:49.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:49 smithi092 ceph-mon[91933]: Upgrade: Setting container_image for all mds 2024-05-06T12:22:49.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:49.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi092.ijsqgp"}]: dispatch 2024-05-06T12:22:49.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi092.ijsqgp"}]': finished 2024-05-06T12:22:49.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi092.rosbll"}]: dispatch 2024-05-06T12:22:49.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi092.rosbll"}]': finished 2024-05-06T12:22:49.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi190.byjvgf"}]: dispatch 2024-05-06T12:22:49.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi190.byjvgf"}]': finished 2024-05-06T12:22:49.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi190.idzoff"}]: dispatch 2024-05-06T12:22:49.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi190.idzoff"}]': finished 2024-05-06T12:22:49.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:49 smithi092 ceph-mon[91933]: Upgrade: Scaling up filesystem cephfs max_mds to 2 2024-05-06T12:22:49.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:49 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-05-06T12:22:49.724 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:49 smithi092 ceph-mon[91933]: pgmap v224: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail 2024-05-06T12:22:50.601 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:50 smithi190 ceph-mon[79744]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-06T12:22:50.601 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:50 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-05-06T12:22:50.601 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:50 smithi190 ceph-mon[79744]: daemon mds.cephfs.smithi092.rosbll assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-06T12:22:50.602 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:50 smithi190 ceph-mon[79744]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-06T12:22:50.602 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:50 smithi190 ceph-mon[79744]: fsmap cephfs:1 {0=cephfs.smithi092.ijsqgp=up:active} 3 up:standby 2024-05-06T12:22:50.602 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:50 smithi190 ceph-mon[79744]: fsmap cephfs:2 {0=cephfs.smithi092.ijsqgp=up:active,1=cephfs.smithi092.rosbll=up:starting} 2 up:standby 2024-05-06T12:22:50.602 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:50 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:50.602 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:50 smithi190 ceph-mon[79744]: daemon mds.cephfs.smithi092.rosbll is now active in filesystem cephfs as rank 1 2024-05-06T12:22:50.602 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:50 smithi190 ceph-mon[79744]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-05-06T12:22:50.602 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:50 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-05-06T12:22:50.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:50 smithi092 ceph-mon[91933]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-06T12:22:50.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:50 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-05-06T12:22:50.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:50 smithi092 ceph-mon[91933]: daemon mds.cephfs.smithi092.rosbll assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-06T12:22:50.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:50 smithi092 ceph-mon[91933]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-06T12:22:50.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:50 smithi092 ceph-mon[91933]: fsmap cephfs:1 {0=cephfs.smithi092.ijsqgp=up:active} 3 up:standby 2024-05-06T12:22:50.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:50 smithi092 ceph-mon[91933]: fsmap cephfs:2 {0=cephfs.smithi092.ijsqgp=up:active,1=cephfs.smithi092.rosbll=up:starting} 2 up:standby 2024-05-06T12:22:50.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:50 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:50.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:50 smithi092 ceph-mon[91933]: daemon mds.cephfs.smithi092.rosbll is now active in filesystem cephfs as rank 1 2024-05-06T12:22:50.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:50 smithi092 ceph-mon[91933]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-05-06T12:22:50.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:50 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-05-06T12:22:51.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:51 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-05-06T12:22:51.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:51 smithi190 ceph-mon[79744]: mds.? [v2:172.21.15.92:6828/2398082173,v1:172.21.15.92:6829/2398082173] up:active 2024-05-06T12:22:51.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:51 smithi190 ceph-mon[79744]: fsmap cephfs:2 {0=cephfs.smithi092.ijsqgp=up:active,1=cephfs.smithi092.rosbll=up:active} 2 up:standby 2024-05-06T12:22:51.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:51 smithi190 ceph-mon[79744]: fsmap cephfs:2 {0=cephfs.smithi092.ijsqgp=up:active,1=cephfs.smithi092.rosbll=up:active} 2 up:standby-replay 2024-05-06T12:22:51.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:51 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:51.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:51 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:51.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:51 smithi190 ceph-mon[79744]: Upgrade: Setting container_image for all rgw 2024-05-06T12:22:51.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:51 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:51.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:51 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:51.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:51 smithi190 ceph-mon[79744]: Upgrade: Setting container_image for all rbd-mirror 2024-05-06T12:22:51.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:51 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:51.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:51 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:51.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:51 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:51.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:51 smithi190 ceph-mon[79744]: Upgrade: Setting container_image for all ceph-exporter 2024-05-06T12:22:51.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:51 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:51.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:51 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:51.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:51 smithi190 ceph-mon[79744]: Upgrade: Setting container_image for all iscsi 2024-05-06T12:22:51.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:51 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:51.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:51 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:51.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:51 smithi190 ceph-mon[79744]: Upgrade: Setting container_image for all nfs 2024-05-06T12:22:51.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:51 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:51.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:51 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:51.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:51 smithi190 ceph-mon[79744]: Upgrade: Setting container_image for all nvmeof 2024-05-06T12:22:51.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:51 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:51.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:51 smithi190 ceph-mon[79744]: pgmap v225: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-06T12:22:51.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:51 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-05-06T12:22:51.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:51 smithi092 ceph-mon[91933]: mds.? [v2:172.21.15.92:6828/2398082173,v1:172.21.15.92:6829/2398082173] up:active 2024-05-06T12:22:51.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:51 smithi092 ceph-mon[91933]: fsmap cephfs:2 {0=cephfs.smithi092.ijsqgp=up:active,1=cephfs.smithi092.rosbll=up:active} 2 up:standby 2024-05-06T12:22:51.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:51 smithi092 ceph-mon[91933]: fsmap cephfs:2 {0=cephfs.smithi092.ijsqgp=up:active,1=cephfs.smithi092.rosbll=up:active} 2 up:standby-replay 2024-05-06T12:22:51.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:51 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:51.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:51 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:51.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:51 smithi092 ceph-mon[91933]: Upgrade: Setting container_image for all rgw 2024-05-06T12:22:51.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:51 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:51.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:51 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:51.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:51 smithi092 ceph-mon[91933]: Upgrade: Setting container_image for all rbd-mirror 2024-05-06T12:22:51.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:51 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:51.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:51 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:51.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:51 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:51.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:51 smithi092 ceph-mon[91933]: Upgrade: Setting container_image for all ceph-exporter 2024-05-06T12:22:51.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:51 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:51.973 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:51 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:51.974 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:51 smithi092 ceph-mon[91933]: Upgrade: Setting container_image for all iscsi 2024-05-06T12:22:51.974 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:51 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:51.974 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:51 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:51.974 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:51 smithi092 ceph-mon[91933]: Upgrade: Setting container_image for all nfs 2024-05-06T12:22:51.974 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:51 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:51.974 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:51 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:22:51.974 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:51 smithi092 ceph-mon[91933]: Upgrade: Setting container_image for all nvmeof 2024-05-06T12:22:51.974 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:51 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:51.974 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:51 smithi092 ceph-mon[91933]: pgmap v225: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-06T12:22:54.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:53 smithi092 ceph-mon[91933]: Upgrade: Updating node-exporter.smithi092 (1/2) 2024-05-06T12:22:54.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:53 smithi092 ceph-mon[91933]: Deploying daemon node-exporter.smithi092 on smithi092 2024-05-06T12:22:54.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:53 smithi092 ceph-mon[91933]: pgmap v226: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 6.0 MiB/s rd, 0 B/s wr, 6 op/s 2024-05-06T12:22:54.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:53 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:22:54.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:53 smithi190 ceph-mon[79744]: Upgrade: Updating node-exporter.smithi092 (1/2) 2024-05-06T12:22:54.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:53 smithi190 ceph-mon[79744]: Deploying daemon node-exporter.smithi092 on smithi092 2024-05-06T12:22:54.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:53 smithi190 ceph-mon[79744]: pgmap v226: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 6.0 MiB/s rd, 0 B/s wr, 6 op/s 2024-05-06T12:22:54.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:53 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:22:55.998 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:55 smithi190 ceph-mon[79744]: pgmap v227: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 7.7 MiB/s rd, 85 B/s wr, 8 op/s 2024-05-06T12:22:55.998 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:55 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:55.998 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:55 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:56.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:55 smithi092 ceph-mon[91933]: pgmap v227: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 7.7 MiB/s rd, 85 B/s wr, 8 op/s 2024-05-06T12:22:56.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:55 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:56.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:55 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:22:58.090 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:57 smithi190 ceph-mon[79744]: Upgrade: Updating node-exporter.smithi190 (2/2) 2024-05-06T12:22:58.090 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:57 smithi190 ceph-mon[79744]: Deploying daemon node-exporter.smithi190 on smithi190 2024-05-06T12:22:58.090 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:57 smithi190 ceph-mon[79744]: pgmap v228: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 10 MiB/s rd, 170 B/s wr, 10 op/s 2024-05-06T12:22:58.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:57 smithi092 ceph-mon[91933]: Upgrade: Updating node-exporter.smithi190 (2/2) 2024-05-06T12:22:58.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:57 smithi092 ceph-mon[91933]: Deploying daemon node-exporter.smithi190 on smithi190 2024-05-06T12:22:58.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:57 smithi092 ceph-mon[91933]: pgmap v228: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 10 MiB/s rd, 170 B/s wr, 10 op/s 2024-05-06T12:23:00.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:59 smithi092 ceph-mon[91933]: pgmap v229: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 170 B/s wr, 12 op/s 2024-05-06T12:23:00.226 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:59 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:00.226 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:59 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:00.226 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:22:59 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:23:00.266 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:59 smithi190 ceph-mon[79744]: pgmap v229: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 170 B/s wr, 12 op/s 2024-05-06T12:23:00.266 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:59 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:00.266 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:59 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:00.266 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:22:59 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:23:01.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:01 smithi092 ceph-mon[91933]: pgmap v230: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 170 B/s wr, 12 op/s 2024-05-06T12:23:02.097 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:01 smithi190 ceph-mon[79744]: pgmap v230: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 170 B/s wr, 12 op/s 2024-05-06T12:23:03.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:03 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:03.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:03 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:03.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:03 smithi092 ceph-mon[91933]: pgmap v231: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 170 B/s wr, 13 op/s 2024-05-06T12:23:03.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:03 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:03.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:03 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:03.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:03 smithi190 ceph-mon[79744]: pgmap v231: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 170 B/s wr, 13 op/s 2024-05-06T12:23:04.541 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:04 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:04.541 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:04 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:04.541 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:04 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:04.541 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:04 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:04.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:04 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:04.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:04 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:04.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:04 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:04.723 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:04 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:06.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:05 smithi092 ceph-mon[91933]: pgmap v232: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 8.1 MiB/s rd, 170 B/s wr, 7 op/s 2024-05-06T12:23:06.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:05 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:06.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:05 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:06.265 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:05 smithi190 ceph-mon[79744]: pgmap v232: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 8.1 MiB/s rd, 170 B/s wr, 7 op/s 2024-05-06T12:23:06.265 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:05 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:06.265 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:05 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:07.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:07.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:07.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:07 smithi092 ceph-mon[91933]: pgmap v233: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 6.4 MiB/s rd, 85 B/s wr, 5 op/s 2024-05-06T12:23:08.097 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:08.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:08.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:07 smithi190 ceph-mon[79744]: pgmap v233: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 6.4 MiB/s rd, 85 B/s wr, 5 op/s 2024-05-06T12:23:09.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:09.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:09.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:23:09.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:23:09.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:09.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:23:09.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:09.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:09.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:09.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:09.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:09.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:09.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:09.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:09.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:09.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:09.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:09.849 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:09.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:09.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:23:09.850 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:09 smithi190 ceph-mon[79744]: pgmap v234: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 4.1 MiB/s rd, 4 op/s 2024-05-06T12:23:09.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.858+0000 7faa23b1f700 1 -- 172.21.15.92:0/428881230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa1c071de0 msgr2=0x7faa1c0702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:09.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.858+0000 7faa23b1f700 1 --2- 172.21.15.92:0/428881230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa1c071de0 0x7faa1c0702f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7faa18004940 tx=0x7faa18009e30 comp rx=0 tx=0).stop 2024-05-06T12:23:09.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.858+0000 7faa23b1f700 1 -- 172.21.15.92:0/428881230 shutdown_connections 2024-05-06T12:23:09.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.858+0000 7faa23b1f700 1 --2- 172.21.15.92:0/428881230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa1c070830 0x7faa1c070ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:09.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.858+0000 7faa23b1f700 1 --2- 172.21.15.92:0/428881230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa1c071de0 0x7faa1c0702f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:09.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.858+0000 7faa23b1f700 1 -- 172.21.15.92:0/428881230 >> 172.21.15.92:0/428881230 conn(0x7faa1c0f8520 msgr2=0x7faa1c0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:23:09.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.858+0000 7faa23b1f700 1 -- 172.21.15.92:0/428881230 shutdown_connections 2024-05-06T12:23:09.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.858+0000 7faa23b1f700 1 -- 172.21.15.92:0/428881230 wait complete. 2024-05-06T12:23:09.861 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:09.862 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:09.862 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:23:09.862 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:23:09.862 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:09.862 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:23:09.862 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:09.862 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:09.862 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:09.862 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:09.862 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:09.862 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:09.862 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:09.862 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:09.862 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:09.863 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:09.863 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:09.863 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:09.863 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:09.863 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:23:09.863 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:09 smithi092 ceph-mon[91933]: pgmap v234: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 4.1 MiB/s rd, 4 op/s 2024-05-06T12:23:09.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.859+0000 7faa23b1f700 1 Processor -- start 2024-05-06T12:23:09.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.859+0000 7faa23b1f700 1 -- start start 2024-05-06T12:23:09.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.859+0000 7faa23b1f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa1c070830 0x7faa1c10fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:09.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.859+0000 7faa23b1f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa1c071de0 0x7faa1c110080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:09.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.859+0000 7faa23b1f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c110680 con 0x7faa1c070830 2024-05-06T12:23:09.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.859+0000 7faa23b1f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c1107f0 con 0x7faa1c071de0 2024-05-06T12:23:09.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.859+0000 7faa218bb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa1c070830 0x7faa1c10fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:09.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.859+0000 7faa218bb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa1c070830 0x7faa1c10fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49444/0 (socket says 172.21.15.92:49444) 2024-05-06T12:23:09.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.859+0000 7faa218bb700 1 -- 172.21.15.92:0/3148944420 learned_addr learned my addr 172.21.15.92:0/3148944420 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:23:09.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.859+0000 7faa210ba700 1 --2- 172.21.15.92:0/3148944420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa1c071de0 0x7faa1c110080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:09.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.859+0000 7faa218bb700 1 -- 172.21.15.92:0/3148944420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa1c071de0 msgr2=0x7faa1c110080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:09.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.859+0000 7faa218bb700 1 --2- 172.21.15.92:0/3148944420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa1c071de0 0x7faa1c110080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:09.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.859+0000 7faa218bb700 1 -- 172.21.15.92:0/3148944420 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa18005040 con 0x7faa1c070830 2024-05-06T12:23:09.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.859+0000 7faa210ba700 1 --2- 172.21.15.92:0/3148944420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa1c071de0 0x7faa1c110080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-06T12:23:09.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.859+0000 7faa218bb700 1 --2- 172.21.15.92:0/3148944420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa1c070830 0x7faa1c10fb40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7faa1800b970 tx=0x7faa18009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:23:09.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.860+0000 7faa127fc700 1 -- 172.21.15.92:0/3148944420 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa1800e040 con 0x7faa1c070830 2024-05-06T12:23:09.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.860+0000 7faa23b1f700 1 -- 172.21.15.92:0/3148944420 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa1c1151f0 con 0x7faa1c070830 2024-05-06T12:23:09.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.860+0000 7faa127fc700 1 -- 172.21.15.92:0/3148944420 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa1800bbf0 con 0x7faa1c070830 2024-05-06T12:23:09.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.860+0000 7faa127fc700 1 -- 172.21.15.92:0/3148944420 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa1801c630 con 0x7faa1c070830 2024-05-06T12:23:09.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.860+0000 7faa23b1f700 1 -- 172.21.15.92:0/3148944420 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa1c1157b0 con 0x7faa1c070830 2024-05-06T12:23:09.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.861+0000 7faa23b1f700 1 -- 172.21.15.92:0/3148944420 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa1c04c710 con 0x7faa1c070830 2024-05-06T12:23:09.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.861+0000 7faa127fc700 1 -- 172.21.15.92:0/3148944420 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7faa1800bd60 con 0x7faa1c070830 2024-05-06T12:23:09.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.861+0000 7faa127fc700 1 --2- 172.21.15.92:0/3148944420 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7faa0806e140 0x7faa080705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:09.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.862+0000 7faa127fc700 1 -- 172.21.15.92:0/3148944420 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7faa1808eef0 con 0x7faa1c070830 2024-05-06T12:23:09.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.862+0000 7faa210ba700 1 --2- 172.21.15.92:0/3148944420 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7faa0806e140 0x7faa080705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:09.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.862+0000 7faa210ba700 1 --2- 172.21.15.92:0/3148944420 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7faa0806e140 0x7faa080705f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faa0c0041d0 tx=0x7faa0c009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:23:09.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:09.866+0000 7faa127fc700 1 -- 172.21.15.92:0/3148944420 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7faa1805bdd0 con 0x7faa1c070830 2024-05-06T12:23:10.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.087+0000 7faa23b1f700 1 -- 172.21.15.92:0/3148944420 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faa1c103ee0 con 0x7faa0806e140 2024-05-06T12:23:10.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.088+0000 7faa127fc700 1 -- 172.21.15.92:0/3148944420 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+438 (secure 0 0 0) 0x7faa1c103ee0 con 0x7faa0806e140 2024-05-06T12:23:10.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.090+0000 7faa23b1f700 1 -- 172.21.15.92:0/3148944420 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7faa0806e140 msgr2=0x7faa080705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:10.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.090+0000 7faa23b1f700 1 --2- 172.21.15.92:0/3148944420 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7faa0806e140 0x7faa080705f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faa0c0041d0 tx=0x7faa0c009040 comp rx=0 tx=0).stop 2024-05-06T12:23:10.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.090+0000 7faa23b1f700 1 -- 172.21.15.92:0/3148944420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa1c070830 msgr2=0x7faa1c10fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:10.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.090+0000 7faa23b1f700 1 --2- 172.21.15.92:0/3148944420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa1c070830 0x7faa1c10fb40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7faa1800b970 tx=0x7faa18009e30 comp rx=0 tx=0).stop 2024-05-06T12:23:10.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.090+0000 7faa23b1f700 1 -- 172.21.15.92:0/3148944420 shutdown_connections 2024-05-06T12:23:10.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.090+0000 7faa23b1f700 1 --2- 172.21.15.92:0/3148944420 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7faa0806e140 0x7faa080705f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:10.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.090+0000 7faa23b1f700 1 --2- 172.21.15.92:0/3148944420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa1c070830 0x7faa1c10fb40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:10.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.090+0000 7faa23b1f700 1 --2- 172.21.15.92:0/3148944420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa1c071de0 0x7faa1c110080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:10.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.090+0000 7faa23b1f700 1 -- 172.21.15.92:0/3148944420 >> 172.21.15.92:0/3148944420 conn(0x7faa1c0f8520 msgr2=0x7faa1c0f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:23:10.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.090+0000 7faa23b1f700 1 -- 172.21.15.92:0/3148944420 shutdown_connections 2024-05-06T12:23:10.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.090+0000 7faa23b1f700 1 -- 172.21.15.92:0/3148944420 wait complete. 2024-05-06T12:23:10.103 INFO:teuthology.orchestra.run.smithi092.stdout:true 2024-05-06T12:23:10.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.193+0000 7f4d2312d700 1 -- 172.21.15.92:0/785285432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d1c0fff60 msgr2=0x7f4d1c102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:10.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.193+0000 7f4d2312d700 1 --2- 172.21.15.92:0/785285432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d1c0fff60 0x7f4d1c102340 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4d08004940 tx=0x7f4d08009e30 comp rx=0 tx=0).stop 2024-05-06T12:23:10.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.193+0000 7f4d2312d700 1 -- 172.21.15.92:0/785285432 shutdown_connections 2024-05-06T12:23:10.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.193+0000 7f4d2312d700 1 --2- 172.21.15.92:0/785285432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d1c0fff60 0x7f4d1c102340 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:10.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.193+0000 7f4d2312d700 1 --2- 172.21.15.92:0/785285432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1c066240 0x7f4d1c0ffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:10.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.193+0000 7f4d2312d700 1 -- 172.21.15.92:0/785285432 >> 172.21.15.92:0/785285432 conn(0x7f4d1c0f7430 msgr2=0x7f4d1c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:23:10.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.193+0000 7f4d2312d700 1 -- 172.21.15.92:0/785285432 shutdown_connections 2024-05-06T12:23:10.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.193+0000 7f4d2312d700 1 -- 172.21.15.92:0/785285432 wait complete. 2024-05-06T12:23:10.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.194+0000 7f4d2312d700 1 Processor -- start 2024-05-06T12:23:10.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.194+0000 7f4d2312d700 1 -- start start 2024-05-06T12:23:10.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.194+0000 7f4d2312d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1c066240 0x7f4d1c10d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:10.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.194+0000 7f4d2312d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d1c0fff60 0x7f4d1c10df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:10.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.194+0000 7f4d2312d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c10e530 con 0x7f4d1c0fff60 2024-05-06T12:23:10.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.194+0000 7f4d2312d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c10ed30 con 0x7f4d1c066240 2024-05-06T12:23:10.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.194+0000 7f4d1bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d1c0fff60 0x7f4d1c10df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:10.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.194+0000 7f4d1bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d1c0fff60 0x7f4d1c10df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49472/0 (socket says 172.21.15.92:49472) 2024-05-06T12:23:10.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.194+0000 7f4d1bfff700 1 -- 172.21.15.92:0/130949166 learned_addr learned my addr 172.21.15.92:0/130949166 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:23:10.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.195+0000 7f4d20ec9700 1 --2- 172.21.15.92:0/130949166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1c066240 0x7f4d1c10d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:10.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.195+0000 7f4d1bfff700 1 -- 172.21.15.92:0/130949166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1c066240 msgr2=0x7f4d1c10d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:10.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.195+0000 7f4d1bfff700 1 --2- 172.21.15.92:0/130949166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1c066240 0x7f4d1c10d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:10.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.195+0000 7f4d1bfff700 1 -- 172.21.15.92:0/130949166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d08005040 con 0x7f4d1c0fff60 2024-05-06T12:23:10.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.195+0000 7f4d1bfff700 1 --2- 172.21.15.92:0/130949166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d1c0fff60 0x7f4d1c10df30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4d08013870 tx=0x7f4d08009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:23:10.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.195+0000 7f4d197fa700 1 -- 172.21.15.92:0/130949166 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d08021070 con 0x7f4d1c0fff60 2024-05-06T12:23:10.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.195+0000 7f4d197fa700 1 -- 172.21.15.92:0/130949166 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d0801c400 con 0x7f4d1c0fff60 2024-05-06T12:23:10.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.195+0000 7f4d2312d700 1 -- 172.21.15.92:0/130949166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d1c10efb0 con 0x7f4d1c0fff60 2024-05-06T12:23:10.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.195+0000 7f4d197fa700 1 -- 172.21.15.92:0/130949166 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d0800e070 con 0x7f4d1c0fff60 2024-05-06T12:23:10.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.195+0000 7f4d2312d700 1 -- 172.21.15.92:0/130949166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d1c10f4f0 con 0x7f4d1c0fff60 2024-05-06T12:23:10.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.196+0000 7f4d2312d700 1 -- 172.21.15.92:0/130949166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d1c05f5c0 con 0x7f4d1c0fff60 2024-05-06T12:23:10.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.197+0000 7f4d197fa700 1 -- 172.21.15.92:0/130949166 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4d0801ca20 con 0x7f4d1c0fff60 2024-05-06T12:23:10.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.197+0000 7f4d197fa700 1 --2- 172.21.15.92:0/130949166 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4d0c06e140 0x7f4d0c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:10.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.197+0000 7f4d197fa700 1 -- 172.21.15.92:0/130949166 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4d0808ede0 con 0x7f4d1c0fff60 2024-05-06T12:23:10.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.197+0000 7f4d20ec9700 1 --2- 172.21.15.92:0/130949166 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4d0c06e140 0x7f4d0c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:10.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.197+0000 7f4d20ec9700 1 --2- 172.21.15.92:0/130949166 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4d0c06e140 0x7f4d0c0705f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4d10004600 tx=0x7f4d1000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:23:10.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.202+0000 7f4d197fa700 1 -- 172.21.15.92:0/130949166 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4d0805bcc0 con 0x7f4d1c0fff60 2024-05-06T12:23:10.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.429+0000 7f4d2312d700 1 -- 172.21.15.92:0/130949166 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d1c10f710 con 0x7f4d0c06e140 2024-05-06T12:23:10.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.430+0000 7f4d197fa700 1 -- 172.21.15.92:0/130949166 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+438 (secure 0 0 0) 0x7f4d1c10f710 con 0x7f4d0c06e140 2024-05-06T12:23:10.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.433+0000 7f4d2312d700 1 -- 172.21.15.92:0/130949166 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4d0c06e140 msgr2=0x7f4d0c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:10.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.433+0000 7f4d2312d700 1 --2- 172.21.15.92:0/130949166 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4d0c06e140 0x7f4d0c0705f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4d10004600 tx=0x7f4d1000b040 comp rx=0 tx=0).stop 2024-05-06T12:23:10.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.433+0000 7f4d2312d700 1 -- 172.21.15.92:0/130949166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d1c0fff60 msgr2=0x7f4d1c10df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:10.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.433+0000 7f4d2312d700 1 --2- 172.21.15.92:0/130949166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d1c0fff60 0x7f4d1c10df30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4d08013870 tx=0x7f4d08009e30 comp rx=0 tx=0).stop 2024-05-06T12:23:10.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.433+0000 7f4d2312d700 1 -- 172.21.15.92:0/130949166 shutdown_connections 2024-05-06T12:23:10.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.433+0000 7f4d2312d700 1 --2- 172.21.15.92:0/130949166 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4d0c06e140 0x7f4d0c0705f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:10.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.433+0000 7f4d2312d700 1 --2- 172.21.15.92:0/130949166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1c066240 0x7f4d1c10d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:10.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.433+0000 7f4d2312d700 1 --2- 172.21.15.92:0/130949166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d1c0fff60 0x7f4d1c10df30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:10.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.433+0000 7f4d2312d700 1 -- 172.21.15.92:0/130949166 >> 172.21.15.92:0/130949166 conn(0x7f4d1c0f7430 msgr2=0x7f4d1c0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:23:10.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.433+0000 7f4d2312d700 1 -- 172.21.15.92:0/130949166 shutdown_connections 2024-05-06T12:23:10.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.433+0000 7f4d2312d700 1 -- 172.21.15.92:0/130949166 wait complete. 2024-05-06T12:23:10.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.543+0000 7f4f7b224700 1 -- 172.21.15.92:0/240071838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f740fcf90 msgr2=0x7f4f740fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:10.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.543+0000 7f4f7b224700 1 --2- 172.21.15.92:0/240071838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f740fcf90 0x7f4f740fd400 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4f60004940 tx=0x7f4f60009e30 comp rx=0 tx=0).stop 2024-05-06T12:23:10.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.544+0000 7f4f7b224700 1 -- 172.21.15.92:0/240071838 shutdown_connections 2024-05-06T12:23:10.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.544+0000 7f4f7b224700 1 --2- 172.21.15.92:0/240071838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f740fcf90 0x7f4f740fd400 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:10.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.544+0000 7f4f7b224700 1 --2- 172.21.15.92:0/240071838 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f740fbce0 0x7f4f740fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:10.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.544+0000 7f4f7b224700 1 -- 172.21.15.92:0/240071838 >> 172.21.15.92:0/240071838 conn(0x7f4f740f7420 msgr2=0x7f4f740f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:23:10.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.544+0000 7f4f7b224700 1 -- 172.21.15.92:0/240071838 shutdown_connections 2024-05-06T12:23:10.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.544+0000 7f4f7b224700 1 -- 172.21.15.92:0/240071838 wait complete. 2024-05-06T12:23:10.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.544+0000 7f4f7b224700 1 Processor -- start 2024-05-06T12:23:10.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.544+0000 7f4f7b224700 1 -- start start 2024-05-06T12:23:10.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.545+0000 7f4f7b224700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f740fbce0 0x7f4f74066c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:10.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.545+0000 7f4f7b224700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f740fcf90 0x7f4f74112730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:10.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.545+0000 7f4f7b224700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f74112c70 con 0x7f4f740fbce0 2024-05-06T12:23:10.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.545+0000 7f4f7b224700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f74112db0 con 0x7f4f740fcf90 2024-05-06T12:23:10.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.545+0000 7f4f78fc0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f740fbce0 0x7f4f74066c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:10.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.545+0000 7f4f78fc0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f740fbce0 0x7f4f74066c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49502/0 (socket says 172.21.15.92:49502) 2024-05-06T12:23:10.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.545+0000 7f4f78fc0700 1 -- 172.21.15.92:0/4226917607 learned_addr learned my addr 172.21.15.92:0/4226917607 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:23:10.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.545+0000 7f4f73fff700 1 --2- 172.21.15.92:0/4226917607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f740fcf90 0x7f4f74112730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:10.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.545+0000 7f4f78fc0700 1 -- 172.21.15.92:0/4226917607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f740fcf90 msgr2=0x7f4f74112730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:10.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.545+0000 7f4f78fc0700 1 --2- 172.21.15.92:0/4226917607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f740fcf90 0x7f4f74112730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:10.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.545+0000 7f4f78fc0700 1 -- 172.21.15.92:0/4226917607 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f60005040 con 0x7f4f740fbce0 2024-05-06T12:23:10.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.546+0000 7f4f78fc0700 1 --2- 172.21.15.92:0/4226917607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f740fbce0 0x7f4f74066c50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4f68002a50 tx=0x7f4f680046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:23:10.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.546+0000 7f4f717fa700 1 -- 172.21.15.92:0/4226917607 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f68007cc0 con 0x7f4f740fbce0 2024-05-06T12:23:10.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.546+0000 7f4f717fa700 1 -- 172.21.15.92:0/4226917607 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f6800f040 con 0x7f4f740fbce0 2024-05-06T12:23:10.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.546+0000 7f4f7b224700 1 -- 172.21.15.92:0/4226917607 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f74112fb0 con 0x7f4f740fbce0 2024-05-06T12:23:10.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.546+0000 7f4f717fa700 1 -- 172.21.15.92:0/4226917607 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f6800da30 con 0x7f4f740fbce0 2024-05-06T12:23:10.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.546+0000 7f4f7b224700 1 -- 172.21.15.92:0/4226917607 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f74113540 con 0x7f4f740fbce0 2024-05-06T12:23:10.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.548+0000 7f4f7b224700 1 -- 172.21.15.92:0/4226917607 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f7404c710 con 0x7f4f740fbce0 2024-05-06T12:23:10.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.548+0000 7f4f717fa700 1 -- 172.21.15.92:0/4226917607 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4f6800db90 con 0x7f4f740fbce0 2024-05-06T12:23:10.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.548+0000 7f4f717fa700 1 --2- 172.21.15.92:0/4226917607 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4f6406e140 0x7f4f640705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:10.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.548+0000 7f4f717fa700 1 -- 172.21.15.92:0/4226917607 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4f6808d8b0 con 0x7f4f740fbce0 2024-05-06T12:23:10.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.549+0000 7f4f73fff700 1 --2- 172.21.15.92:0/4226917607 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4f6406e140 0x7f4f640705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:10.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.549+0000 7f4f73fff700 1 --2- 172.21.15.92:0/4226917607 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4f6406e140 0x7f4f640705f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4f60009fd0 tx=0x7f4f60009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:23:10.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.553+0000 7f4f717fa700 1 -- 172.21.15.92:0/4226917607 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4f6805a710 con 0x7f4f740fbce0 2024-05-06T12:23:10.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.788+0000 7f4f7b224700 1 -- 172.21.15.92:0/4226917607 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f741012a0 con 0x7f4f6406e140 2024-05-06T12:23:10.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.798+0000 7f4f717fa700 1 -- 172.21.15.92:0/4226917607 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f4f741012a0 con 0x7f4f6406e140 2024-05-06T12:23:10.801 INFO:teuthology.orchestra.run.smithi092.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-06T12:23:10.801 INFO:teuthology.orchestra.run.smithi092.stdout:alertmanager.smithi092 smithi092 *:9093,9094 running (8m) 6s ago 16m 25.2M - 0.25.0 c8568f914cd2 be5637d7d25a 2024-05-06T12:23:10.801 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi092 smithi092 running (5m) 6s ago 16m 6660k - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 8da0494e1bdc 2024-05-06T12:23:10.801 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi190 smithi190 running (5m) 8s ago 15m 6665k - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 641a7cdaa56e 2024-05-06T12:23:10.801 INFO:teuthology.orchestra.run.smithi092.stdout:grafana.smithi092 smithi092 *:3000 running (8m) 6s ago 16m 81.1M - 9.4.7 954c08fa6188 9180cb3e5258 2024-05-06T12:23:10.802 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.ijsqgp smithi092 running (86s) 6s ago 11m 66.7M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 40b4cdd3be40 2024-05-06T12:23:10.802 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.rosbll smithi092 running (64s) 6s ago 11m 15.3M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 550f8238d95f 2024-05-06T12:23:10.802 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.byjvgf smithi190 running (46s) 8s ago 11m 63.4M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 0d26051c908e 2024-05-06T12:23:10.802 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.idzoff smithi190 running (30s) 8s ago 11m 12.8M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 7cfd0097389b 2024-05-06T12:23:10.802 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi092.gyzeqb smithi092 *:8443,9283,8765 running (7m) 6s ago 18m 511M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 de9d2198327a 2024-05-06T12:23:10.802 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi190.kczxkh smithi190 *:8443,9283,8765 running (6m) 8s ago 15m 452M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 e84ab3758ac7 2024-05-06T12:23:10.802 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi092 smithi092 running (6m) 6s ago 18m 52.6M 2048M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 ac07f7158087 2024-05-06T12:23:10.802 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi190 smithi190 running (6m) 8s ago 15m 39.7M 2048M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 7a3c08eb716f 2024-05-06T12:23:10.802 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi092 smithi092 *:9100 running (16s) 6s ago 16m 3472k - 1.5.0 0da6a335fe13 590fa41f0538 2024-05-06T12:23:10.802 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi190 smithi190 *:9100 running (11s) 8s ago 15m 3590k - 1.5.0 0da6a335fe13 a57191dac5b5 2024-05-06T12:23:10.802 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 smithi092 running (5m) 6s ago 15m 94.1M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 8c96c75b25f4 2024-05-06T12:23:10.802 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 smithi092 running (4m) 6s ago 14m 97.0M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 e8007a510f08 2024-05-06T12:23:10.802 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 smithi092 running (3m) 6s ago 14m 91.3M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 2beca7448789 2024-05-06T12:23:10.802 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 smithi190 running (3m) 8s ago 13m 101M 2581M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 914ada199f93 2024-05-06T12:23:10.803 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 smithi190 running (2m) 8s ago 13m 89.4M 2581M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 371c0be776c7 2024-05-06T12:23:10.803 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 smithi190 running (2m) 8s ago 12m 86.8M 2581M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 067bc34cdfb1 2024-05-06T12:23:10.803 INFO:teuthology.orchestra.run.smithi092.stdout:prometheus.smithi092 smithi092 *:9095 running (7m) 6s ago 16m 63.0M - 2.43.0 a07b618ecd1d bc5445691527 2024-05-06T12:23:10.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.801+0000 7f4f7b224700 1 -- 172.21.15.92:0/4226917607 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4f6406e140 msgr2=0x7f4f640705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:10.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.801+0000 7f4f7b224700 1 --2- 172.21.15.92:0/4226917607 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4f6406e140 0x7f4f640705f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4f60009fd0 tx=0x7f4f60009e30 comp rx=0 tx=0).stop 2024-05-06T12:23:10.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.801+0000 7f4f7b224700 1 -- 172.21.15.92:0/4226917607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f740fbce0 msgr2=0x7f4f74066c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:10.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.801+0000 7f4f7b224700 1 --2- 172.21.15.92:0/4226917607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f740fbce0 0x7f4f74066c50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4f68002a50 tx=0x7f4f680046b0 comp rx=0 tx=0).stop 2024-05-06T12:23:10.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.801+0000 7f4f7b224700 1 -- 172.21.15.92:0/4226917607 shutdown_connections 2024-05-06T12:23:10.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.801+0000 7f4f7b224700 1 --2- 172.21.15.92:0/4226917607 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4f6406e140 0x7f4f640705f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:10.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.801+0000 7f4f7b224700 1 --2- 172.21.15.92:0/4226917607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f740fbce0 0x7f4f74066c50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:10.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.801+0000 7f4f7b224700 1 --2- 172.21.15.92:0/4226917607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f740fcf90 0x7f4f74112730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:10.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.801+0000 7f4f7b224700 1 -- 172.21.15.92:0/4226917607 >> 172.21.15.92:0/4226917607 conn(0x7f4f740f7420 msgr2=0x7f4f740f8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:23:10.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.801+0000 7f4f7b224700 1 -- 172.21.15.92:0/4226917607 shutdown_connections 2024-05-06T12:23:10.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.801+0000 7f4f7b224700 1 -- 172.21.15.92:0/4226917607 wait complete. 2024-05-06T12:23:10.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.905+0000 7f9ec6748700 1 -- 172.21.15.92:0/3251527538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ec0100030 msgr2=0x7f9ec0102410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:10.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.905+0000 7f9ec6748700 1 --2- 172.21.15.92:0/3251527538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ec0100030 0x7f9ec0102410 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4004940 tx=0x7f9eb4009e30 comp rx=0 tx=0).stop 2024-05-06T12:23:10.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.905+0000 7f9ec6748700 1 -- 172.21.15.92:0/3251527538 shutdown_connections 2024-05-06T12:23:10.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.905+0000 7f9ec6748700 1 --2- 172.21.15.92:0/3251527538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ec0100030 0x7f9ec0102410 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:10.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.905+0000 7f9ec6748700 1 --2- 172.21.15.92:0/3251527538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ec0066a90 0x7f9ec00ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:10.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.905+0000 7f9ec6748700 1 -- 172.21.15.92:0/3251527538 >> 172.21.15.92:0/3251527538 conn(0x7f9ec00f74c0 msgr2=0x7f9ec00f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:23:10.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.905+0000 7f9ec6748700 1 -- 172.21.15.92:0/3251527538 shutdown_connections 2024-05-06T12:23:10.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.905+0000 7f9ec6748700 1 -- 172.21.15.92:0/3251527538 wait complete. 2024-05-06T12:23:10.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.906+0000 7f9ec6748700 1 Processor -- start 2024-05-06T12:23:10.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.906+0000 7f9ec6748700 1 -- start start 2024-05-06T12:23:10.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.906+0000 7f9ec6748700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ec0066a90 0x7f9ec010da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:10.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.906+0000 7f9ec6748700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ec0100030 0x7f9ec010df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:10.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.906+0000 7f9ec6748700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec010e590 con 0x7f9ec0066a90 2024-05-06T12:23:10.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.906+0000 7f9ec6748700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec010ede0 con 0x7f9ec0100030 2024-05-06T12:23:10.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.906+0000 7f9ebffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ec0066a90 0x7f9ec010da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:10.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.906+0000 7f9ebffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ec0066a90 0x7f9ec010da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49522/0 (socket says 172.21.15.92:49522) 2024-05-06T12:23:10.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.906+0000 7f9ebffff700 1 -- 172.21.15.92:0/4051932268 learned_addr learned my addr 172.21.15.92:0/4051932268 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:23:10.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.906+0000 7f9ebf7fe700 1 --2- 172.21.15.92:0/4051932268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ec0100030 0x7f9ec010df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:10.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.906+0000 7f9ebffff700 1 -- 172.21.15.92:0/4051932268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ec0100030 msgr2=0x7f9ec010df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:10.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.906+0000 7f9ebffff700 1 --2- 172.21.15.92:0/4051932268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ec0100030 0x7f9ec010df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:10.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.906+0000 7f9ebffff700 1 -- 172.21.15.92:0/4051932268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb4005040 con 0x7f9ec0066a90 2024-05-06T12:23:10.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.907+0000 7f9ebffff700 1 --2- 172.21.15.92:0/4051932268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ec0066a90 0x7f9ec010da50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0002a80 tx=0x7f9eb0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:23:10.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.907+0000 7f9ebcff9700 1 -- 172.21.15.92:0/4051932268 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9eb0007cc0 con 0x7f9ec0066a90 2024-05-06T12:23:10.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.907+0000 7f9ec6748700 1 -- 172.21.15.92:0/4051932268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ec010f0c0 con 0x7f9ec0066a90 2024-05-06T12:23:10.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.907+0000 7f9ebcff9700 1 -- 172.21.15.92:0/4051932268 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9eb000f040 con 0x7f9ec0066a90 2024-05-06T12:23:10.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.907+0000 7f9ebcff9700 1 -- 172.21.15.92:0/4051932268 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9eb000da30 con 0x7f9ec0066a90 2024-05-06T12:23:10.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.907+0000 7f9ec6748700 1 -- 172.21.15.92:0/4051932268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ec010f600 con 0x7f9ec0066a90 2024-05-06T12:23:10.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.908+0000 7f9ec6748700 1 -- 172.21.15.92:0/4051932268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ec005f5c0 con 0x7f9ec0066a90 2024-05-06T12:23:10.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.908+0000 7f9ebcff9700 1 -- 172.21.15.92:0/4051932268 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9eb000db90 con 0x7f9ec0066a90 2024-05-06T12:23:10.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.909+0000 7f9ebcff9700 1 --2- 172.21.15.92:0/4051932268 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f9ea806e140 0x7f9ea80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:10.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.909+0000 7f9ebcff9700 1 -- 172.21.15.92:0/4051932268 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f9eb008d910 con 0x7f9ec0066a90 2024-05-06T12:23:10.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.909+0000 7f9ebf7fe700 1 --2- 172.21.15.92:0/4051932268 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f9ea806e140 0x7f9ea80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:10.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.909+0000 7f9ebf7fe700 1 --2- 172.21.15.92:0/4051932268 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f9ea806e140 0x7f9ea80705f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4004890 tx=0x7f9eb4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:23:10.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:10.914+0000 7f9ebcff9700 1 -- 172.21.15.92:0/4051932268 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9eb005a770 con 0x7f9ec0066a90 2024-05-06T12:23:11.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.207+0000 7f9ec6748700 1 -- 172.21.15.92:0/4051932268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9ec010f910 con 0x7f9ec0066a90 2024-05-06T12:23:11.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.207+0000 7f9ebcff9700 1 -- 172.21.15.92:0/4051932268 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f9eb005a300 con 0x7f9ec0066a90 2024-05-06T12:23:11.210 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:23:11.210 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": { 2024-05-06T12:23:11.210 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:23:11.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:23:11.210 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": { 2024-05-06T12:23:11.210 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:23:11.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:23:11.211 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": { 2024-05-06T12:23:11.211 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 6 2024-05-06T12:23:11.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:23:11.211 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-05-06T12:23:11.211 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 4 2024-05-06T12:23:11.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:23:11.211 INFO:teuthology.orchestra.run.smithi092.stdout: "overall": { 2024-05-06T12:23:11.211 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 14 2024-05-06T12:23:11.211 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-06T12:23:11.211 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:23:11.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.209+0000 7f9ec6748700 1 -- 172.21.15.92:0/4051932268 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f9ea806e140 msgr2=0x7f9ea80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:11.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.209+0000 7f9ec6748700 1 --2- 172.21.15.92:0/4051932268 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f9ea806e140 0x7f9ea80705f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4004890 tx=0x7f9eb4009e30 comp rx=0 tx=0).stop 2024-05-06T12:23:11.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.209+0000 7f9ec6748700 1 -- 172.21.15.92:0/4051932268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ec0066a90 msgr2=0x7f9ec010da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:11.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.209+0000 7f9ec6748700 1 --2- 172.21.15.92:0/4051932268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ec0066a90 0x7f9ec010da50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0002a80 tx=0x7f9eb0004600 comp rx=0 tx=0).stop 2024-05-06T12:23:11.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.209+0000 7f9ec6748700 1 -- 172.21.15.92:0/4051932268 shutdown_connections 2024-05-06T12:23:11.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.209+0000 7f9ec6748700 1 --2- 172.21.15.92:0/4051932268 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f9ea806e140 0x7f9ea80705f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:11.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.209+0000 7f9ec6748700 1 --2- 172.21.15.92:0/4051932268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ec0066a90 0x7f9ec010da50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:11.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.209+0000 7f9ec6748700 1 --2- 172.21.15.92:0/4051932268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ec0100030 0x7f9ec010df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:11.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.209+0000 7f9ec6748700 1 -- 172.21.15.92:0/4051932268 >> 172.21.15.92:0/4051932268 conn(0x7f9ec00f74c0 msgr2=0x7f9ec00f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:23:11.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.210+0000 7f9ec6748700 1 -- 172.21.15.92:0/4051932268 shutdown_connections 2024-05-06T12:23:11.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.210+0000 7f9ec6748700 1 -- 172.21.15.92:0/4051932268 wait complete. 2024-05-06T12:23:11.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.312+0000 7eff00dab700 1 -- 172.21.15.92:0/648105801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efefc0fce60 msgr2=0x7efefc0fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:11.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.312+0000 7eff00dab700 1 --2- 172.21.15.92:0/648105801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efefc0fce60 0x7efefc0fd270 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efef4009ce0 tx=0x7efef4004d10 comp rx=0 tx=0).stop 2024-05-06T12:23:11.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.312+0000 7eff00dab700 1 -- 172.21.15.92:0/648105801 shutdown_connections 2024-05-06T12:23:11.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.312+0000 7eff00dab700 1 --2- 172.21.15.92:0/648105801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efefc0fdfd0 0x7efefc0fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:11.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.312+0000 7eff00dab700 1 --2- 172.21.15.92:0/648105801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efefc0fce60 0x7efefc0fd270 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:11.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.312+0000 7eff00dab700 1 -- 172.21.15.92:0/648105801 >> 172.21.15.92:0/648105801 conn(0x7efefc0f8560 msgr2=0x7efefc0fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:23:11.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.312+0000 7eff00dab700 1 -- 172.21.15.92:0/648105801 shutdown_connections 2024-05-06T12:23:11.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.312+0000 7eff00dab700 1 -- 172.21.15.92:0/648105801 wait complete. 2024-05-06T12:23:11.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.312+0000 7eff00dab700 1 Processor -- start 2024-05-06T12:23:11.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.312+0000 7eff00dab700 1 -- start start 2024-05-06T12:23:11.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.313+0000 7eff00dab700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efefc0fce60 0x7efefc10db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:11.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.313+0000 7eff00dab700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efefc0fdfd0 0x7efefc10e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:11.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.313+0000 7eff00dab700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efefc10e760 con 0x7efefc0fdfd0 2024-05-06T12:23:11.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.313+0000 7eff00dab700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efefc10ef50 con 0x7efefc0fce60 2024-05-06T12:23:11.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.313+0000 7efef9d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efefc0fdfd0 0x7efefc10e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:11.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.313+0000 7efef9d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efefc0fdfd0 0x7efefc10e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49534/0 (socket says 172.21.15.92:49534) 2024-05-06T12:23:11.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.313+0000 7efef9d9b700 1 -- 172.21.15.92:0/626716978 learned_addr learned my addr 172.21.15.92:0/626716978 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:23:11.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.313+0000 7efefa59c700 1 --2- 172.21.15.92:0/626716978 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efefc0fce60 0x7efefc10db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:11.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.313+0000 7efef9d9b700 1 -- 172.21.15.92:0/626716978 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efefc0fce60 msgr2=0x7efefc10db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:11.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.313+0000 7efef9d9b700 1 --2- 172.21.15.92:0/626716978 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efefc0fce60 0x7efefc10db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:11.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.313+0000 7efef9d9b700 1 -- 172.21.15.92:0/626716978 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef4009940 con 0x7efefc0fdfd0 2024-05-06T12:23:11.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.313+0000 7efef9d9b700 1 --2- 172.21.15.92:0/626716978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efefc0fdfd0 0x7efefc10e0d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efef000b790 tx=0x7efef0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:23:11.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.313+0000 7efeeaffd700 1 -- 172.21.15.92:0/626716978 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efef000f6f0 con 0x7efefc0fdfd0 2024-05-06T12:23:11.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.313+0000 7eff00dab700 1 -- 172.21.15.92:0/626716978 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efefc10f230 con 0x7efefc0fdfd0 2024-05-06T12:23:11.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.313+0000 7efeeaffd700 1 -- 172.21.15.92:0/626716978 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efef000fd20 con 0x7efefc0fdfd0 2024-05-06T12:23:11.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.313+0000 7eff00dab700 1 -- 172.21.15.92:0/626716978 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efefc10f7f0 con 0x7efefc0fdfd0 2024-05-06T12:23:11.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.313+0000 7efeeaffd700 1 -- 172.21.15.92:0/626716978 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efef00103a0 con 0x7efefc0fdfd0 2024-05-06T12:23:11.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.315+0000 7eff00dab700 1 -- 172.21.15.92:0/626716978 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efefc05f5c0 con 0x7efefc0fdfd0 2024-05-06T12:23:11.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.315+0000 7efeeaffd700 1 -- 172.21.15.92:0/626716978 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7efef0011400 con 0x7efefc0fdfd0 2024-05-06T12:23:11.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.315+0000 7efeeaffd700 1 --2- 172.21.15.92:0/626716978 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7efee406e150 0x7efee4070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:11.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.316+0000 7efeeaffd700 1 -- 172.21.15.92:0/626716978 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7efef008d840 con 0x7efefc0fdfd0 2024-05-06T12:23:11.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.316+0000 7efefa59c700 1 --2- 172.21.15.92:0/626716978 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7efee406e150 0x7efee4070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:11.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.316+0000 7efefa59c700 1 --2- 172.21.15.92:0/626716978 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7efee406e150 0x7efee4070600 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efef4015040 tx=0x7efef4004d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:23:11.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.320+0000 7efeeaffd700 1 -- 172.21.15.92:0/626716978 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7efef005a7e0 con 0x7efefc0fdfd0 2024-05-06T12:23:11.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.572+0000 7eff00dab700 1 -- 172.21.15.92:0/626716978 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7efefc04bed0 con 0x7efefc0fdfd0 2024-05-06T12:23:11.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.573+0000 7efeeaffd700 1 -- 172.21.15.92:0/626716978 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 42 v42) v1 ==== 76+0+1967 (secure 0 0 0) 0x7efef005a370 con 0x7efefc0fdfd0 2024-05-06T12:23:11.575 INFO:teuthology.orchestra.run.smithi092.stdout:e42 2024-05-06T12:23:11.575 INFO:teuthology.orchestra.run.smithi092.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-06T12:23:11.575 INFO:teuthology.orchestra.run.smithi092.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:23:11.576 INFO:teuthology.orchestra.run.smithi092.stdout:legacy client fscid: 1 2024-05-06T12:23:11.576 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:23:11.576 INFO:teuthology.orchestra.run.smithi092.stdout:Filesystem 'cephfs' (1) 2024-05-06T12:23:11.576 INFO:teuthology.orchestra.run.smithi092.stdout:fs_name cephfs 2024-05-06T12:23:11.576 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 42 2024-05-06T12:23:11.576 INFO:teuthology.orchestra.run.smithi092.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-06T12:23:11.576 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-05-06T12:11:44.952671+0000 2024-05-06T12:23:11.576 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2024-05-06T12:22:50.526300+0000 2024-05-06T12:23:11.576 INFO:teuthology.orchestra.run.smithi092.stdout:tableserver 0 2024-05-06T12:23:11.576 INFO:teuthology.orchestra.run.smithi092.stdout:root 0 2024-05-06T12:23:11.576 INFO:teuthology.orchestra.run.smithi092.stdout:session_timeout 60 2024-05-06T12:23:11.576 INFO:teuthology.orchestra.run.smithi092.stdout:session_autoclose 300 2024-05-06T12:23:11.576 INFO:teuthology.orchestra.run.smithi092.stdout:max_file_size 1099511627776 2024-05-06T12:23:11.576 INFO:teuthology.orchestra.run.smithi092.stdout:max_xattr_size 65536 2024-05-06T12:23:11.577 INFO:teuthology.orchestra.run.smithi092.stdout:required_client_features {} 2024-05-06T12:23:11.577 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure 0 2024-05-06T12:23:11.577 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure_osd_epoch 84 2024-05-06T12:23:11.577 INFO:teuthology.orchestra.run.smithi092.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-06T12:23:11.577 INFO:teuthology.orchestra.run.smithi092.stdout:max_mds 2 2024-05-06T12:23:11.577 INFO:teuthology.orchestra.run.smithi092.stdout:in 0,1 2024-05-06T12:23:11.577 INFO:teuthology.orchestra.run.smithi092.stdout:up {0=34438,1=34466} 2024-05-06T12:23:11.577 INFO:teuthology.orchestra.run.smithi092.stdout:failed 2024-05-06T12:23:11.577 INFO:teuthology.orchestra.run.smithi092.stdout:damaged 2024-05-06T12:23:11.577 INFO:teuthology.orchestra.run.smithi092.stdout:stopped 2024-05-06T12:23:11.577 INFO:teuthology.orchestra.run.smithi092.stdout:data_pools [3] 2024-05-06T12:23:11.577 INFO:teuthology.orchestra.run.smithi092.stdout:metadata_pool 2 2024-05-06T12:23:11.577 INFO:teuthology.orchestra.run.smithi092.stdout:inline_data enabled 2024-05-06T12:23:11.577 INFO:teuthology.orchestra.run.smithi092.stdout:balancer 2024-05-06T12:23:11.577 INFO:teuthology.orchestra.run.smithi092.stdout:bal_rank_mask -1 2024-05-06T12:23:11.578 INFO:teuthology.orchestra.run.smithi092.stdout:standby_count_wanted 1 2024-05-06T12:23:11.578 INFO:teuthology.orchestra.run.smithi092.stdout:qdb_cluster leader: 34438 members: 34466,34438 2024-05-06T12:23:11.578 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.ijsqgp{0:34438} state up:active seq 8 join_fscid=1 addr [v2:172.21.15.92:6826/4190573007,v1:172.21.15.92:6827/4190573007] compat {c=[1],r=[1],i=[1fff]}] 2024-05-06T12:23:11.578 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.byjvgf{0:44309} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.190:6824/1219274291,v1:172.21.15.190:6825/1219274291] compat {c=[1],r=[1],i=[1fff]}] 2024-05-06T12:23:11.578 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.rosbll{1:34466} state up:active seq 12 join_fscid=1 addr [v2:172.21.15.92:6828/2398082173,v1:172.21.15.92:6829/2398082173] compat {c=[1],r=[1],i=[1fff]}] 2024-05-06T12:23:11.578 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.idzoff{1:44329} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.190:6826/2486629325,v1:172.21.15.190:6827/2486629325] compat {c=[1],r=[1],i=[1fff]}] 2024-05-06T12:23:11.578 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:23:11.578 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:23:11.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.575+0000 7eff00dab700 1 -- 172.21.15.92:0/626716978 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7efee406e150 msgr2=0x7efee4070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:11.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.575+0000 7eff00dab700 1 --2- 172.21.15.92:0/626716978 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7efee406e150 0x7efee4070600 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efef4015040 tx=0x7efef4004d10 comp rx=0 tx=0).stop 2024-05-06T12:23:11.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.575+0000 7eff00dab700 1 -- 172.21.15.92:0/626716978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efefc0fdfd0 msgr2=0x7efefc10e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:11.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.575+0000 7eff00dab700 1 --2- 172.21.15.92:0/626716978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efefc0fdfd0 0x7efefc10e0d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efef000b790 tx=0x7efef0009250 comp rx=0 tx=0).stop 2024-05-06T12:23:11.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.575+0000 7eff00dab700 1 -- 172.21.15.92:0/626716978 shutdown_connections 2024-05-06T12:23:11.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.575+0000 7eff00dab700 1 --2- 172.21.15.92:0/626716978 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7efee406e150 0x7efee4070600 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:11.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.575+0000 7eff00dab700 1 --2- 172.21.15.92:0/626716978 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efefc0fce60 0x7efefc10db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:11.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.575+0000 7eff00dab700 1 --2- 172.21.15.92:0/626716978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efefc0fdfd0 0x7efefc10e0d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:11.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.575+0000 7eff00dab700 1 -- 172.21.15.92:0/626716978 >> 172.21.15.92:0/626716978 conn(0x7efefc0f8560 msgr2=0x7efefc0f9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:23:11.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.575+0000 7eff00dab700 1 -- 172.21.15.92:0/626716978 shutdown_connections 2024-05-06T12:23:11.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.575+0000 7eff00dab700 1 -- 172.21.15.92:0/626716978 wait complete. 2024-05-06T12:23:11.580 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 42 2024-05-06T12:23:11.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.680+0000 7f3176ff6700 1 -- 172.21.15.92:0/1496952338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31700fdfd0 msgr2=0x7f31700fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:11.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.680+0000 7f3176ff6700 1 --2- 172.21.15.92:0/1496952338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31700fdfd0 0x7f31700fe440 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3160004940 tx=0x7f3160009e30 comp rx=0 tx=0).stop 2024-05-06T12:23:11.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.680+0000 7f3176ff6700 1 -- 172.21.15.92:0/1496952338 shutdown_connections 2024-05-06T12:23:11.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.680+0000 7f3176ff6700 1 --2- 172.21.15.92:0/1496952338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31700fdfd0 0x7f31700fe440 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:11.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.680+0000 7f3176ff6700 1 --2- 172.21.15.92:0/1496952338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31700fce60 0x7f31700fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:11.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.680+0000 7f3176ff6700 1 -- 172.21.15.92:0/1496952338 >> 172.21.15.92:0/1496952338 conn(0x7f31700f8540 msgr2=0x7f31700fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:23:11.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.680+0000 7f3176ff6700 1 -- 172.21.15.92:0/1496952338 shutdown_connections 2024-05-06T12:23:11.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.680+0000 7f3176ff6700 1 -- 172.21.15.92:0/1496952338 wait complete. 2024-05-06T12:23:11.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.680+0000 7f3176ff6700 1 Processor -- start 2024-05-06T12:23:11.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.680+0000 7f3176ff6700 1 -- start start 2024-05-06T12:23:11.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.680+0000 7f3176ff6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31700fce60 0x7f317010d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:11.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.680+0000 7f3176ff6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31700fdfd0 0x7f317010de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:11.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.680+0000 7f3176ff6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317010e450 con 0x7f31700fce60 2024-05-06T12:23:11.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.681+0000 7f3176ff6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317010e5c0 con 0x7f31700fdfd0 2024-05-06T12:23:11.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.681+0000 7f3174d92700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31700fce60 0x7f317010d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:11.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.681+0000 7f3174d92700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31700fce60 0x7f317010d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49540/0 (socket says 172.21.15.92:49540) 2024-05-06T12:23:11.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.681+0000 7f3174d92700 1 -- 172.21.15.92:0/855304767 learned_addr learned my addr 172.21.15.92:0/855304767 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:23:11.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.681+0000 7f316ffff700 1 --2- 172.21.15.92:0/855304767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31700fdfd0 0x7f317010de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:11.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.681+0000 7f3174d92700 1 -- 172.21.15.92:0/855304767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31700fdfd0 msgr2=0x7f317010de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:11.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.681+0000 7f3174d92700 1 --2- 172.21.15.92:0/855304767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31700fdfd0 0x7f317010de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:11.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.681+0000 7f3174d92700 1 -- 172.21.15.92:0/855304767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3160005040 con 0x7f31700fce60 2024-05-06T12:23:11.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.681+0000 7f3174d92700 1 --2- 172.21.15.92:0/855304767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31700fce60 0x7f317010d910 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f316800cac0 tx=0x7f3168007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:23:11.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.681+0000 7f316d7fa700 1 -- 172.21.15.92:0/855304767 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f316800aa50 con 0x7f31700fce60 2024-05-06T12:23:11.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.681+0000 7f316d7fa700 1 -- 172.21.15.92:0/855304767 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f316800abb0 con 0x7f31700fce60 2024-05-06T12:23:11.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.681+0000 7f3176ff6700 1 -- 172.21.15.92:0/855304767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3170112fc0 con 0x7f31700fce60 2024-05-06T12:23:11.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.681+0000 7f316d7fa700 1 -- 172.21.15.92:0/855304767 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31680115e0 con 0x7f31700fce60 2024-05-06T12:23:11.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.681+0000 7f3176ff6700 1 -- 172.21.15.92:0/855304767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3170113550 con 0x7f31700fce60 2024-05-06T12:23:11.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.682+0000 7f3176ff6700 1 -- 172.21.15.92:0/855304767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f317004c710 con 0x7f31700fce60 2024-05-06T12:23:11.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.683+0000 7f316d7fa700 1 -- 172.21.15.92:0/855304767 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f316800ad20 con 0x7f31700fce60 2024-05-06T12:23:11.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.683+0000 7f316d7fa700 1 --2- 172.21.15.92:0/855304767 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f315c06e140 0x7f315c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:11.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.683+0000 7f316d7fa700 1 -- 172.21.15.92:0/855304767 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f316808e1e0 con 0x7f31700fce60 2024-05-06T12:23:11.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.683+0000 7f316ffff700 1 --2- 172.21.15.92:0/855304767 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f315c06e140 0x7f315c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:11.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.684+0000 7f316ffff700 1 --2- 172.21.15.92:0/855304767 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f315c06e140 0x7f315c0705f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3160000c00 tx=0x7f3160009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:23:11.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.688+0000 7f316d7fa700 1 -- 172.21.15.92:0/855304767 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f316805b0c0 con 0x7f31700fce60 2024-05-06T12:23:11.847 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:11 smithi190 ceph-mon[79744]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:23:11.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:11 smithi190 ceph-mon[79744]: Upgrade: Updating prometheus.smithi092 2024-05-06T12:23:11.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:11 smithi190 ceph-mon[79744]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:23:11.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:11 smithi190 ceph-mon[79744]: Deploying daemon prometheus.smithi092 on smithi092 2024-05-06T12:23:11.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:11 smithi190 ceph-mon[79744]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:23:11.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:11 smithi190 ceph-mon[79744]: pgmap v235: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:23:11.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:11 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/4051932268' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:11.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:11 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/626716978' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:23:11.896 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:11 smithi092 ceph-mon[91933]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:23:11.896 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:11 smithi092 ceph-mon[91933]: Upgrade: Updating prometheus.smithi092 2024-05-06T12:23:11.897 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:11 smithi092 ceph-mon[91933]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:23:11.897 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:11 smithi092 ceph-mon[91933]: Deploying daemon prometheus.smithi092 on smithi092 2024-05-06T12:23:11.897 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:11 smithi092 ceph-mon[91933]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:23:11.897 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:11 smithi092 ceph-mon[91933]: pgmap v235: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:23:11.897 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:11 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/4051932268' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:11.897 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:11 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/626716978' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:23:11.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.906+0000 7f3176ff6700 1 -- 172.21.15.92:0/855304767 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3170102390 con 0x7f315c06e140 2024-05-06T12:23:11.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.908+0000 7f316d7fa700 1 -- 172.21.15.92:0/855304767 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+438 (secure 0 0 0) 0x7f3170102390 con 0x7f315c06e140 2024-05-06T12:23:11.910 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:23:11.910 INFO:teuthology.orchestra.run.smithi092.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb", 2024-05-06T12:23:11.910 INFO:teuthology.orchestra.run.smithi092.stdout: "in_progress": true, 2024-05-06T12:23:11.910 INFO:teuthology.orchestra.run.smithi092.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-06T12:23:11.910 INFO:teuthology.orchestra.run.smithi092.stdout: "services_complete": [ 2024-05-06T12:23:11.910 INFO:teuthology.orchestra.run.smithi092.stdout: "mds", 2024-05-06T12:23:11.910 INFO:teuthology.orchestra.run.smithi092.stdout: "osd", 2024-05-06T12:23:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: "mon", 2024-05-06T12:23:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr", 2024-05-06T12:23:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: "crash" 2024-05-06T12:23:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-05-06T12:23:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: "progress": "16/21 daemons upgraded", 2024-05-06T12:23:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "Currently upgrading prometheus daemons", 2024-05-06T12:23:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: "is_paused": false 2024-05-06T12:23:11.911 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:23:11.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.910+0000 7f3176ff6700 1 -- 172.21.15.92:0/855304767 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f315c06e140 msgr2=0x7f315c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:11.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.910+0000 7f3176ff6700 1 --2- 172.21.15.92:0/855304767 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f315c06e140 0x7f315c0705f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3160000c00 tx=0x7f3160009e30 comp rx=0 tx=0).stop 2024-05-06T12:23:11.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.910+0000 7f3176ff6700 1 -- 172.21.15.92:0/855304767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31700fce60 msgr2=0x7f317010d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:11.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.910+0000 7f3176ff6700 1 --2- 172.21.15.92:0/855304767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31700fce60 0x7f317010d910 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f316800cac0 tx=0x7f3168007ee0 comp rx=0 tx=0).stop 2024-05-06T12:23:11.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.910+0000 7f3176ff6700 1 -- 172.21.15.92:0/855304767 shutdown_connections 2024-05-06T12:23:11.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.910+0000 7f3176ff6700 1 --2- 172.21.15.92:0/855304767 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f315c06e140 0x7f315c0705f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:11.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.910+0000 7f3176ff6700 1 --2- 172.21.15.92:0/855304767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31700fce60 0x7f317010d910 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:11.914 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.910+0000 7f3176ff6700 1 --2- 172.21.15.92:0/855304767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31700fdfd0 0x7f317010de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:11.914 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.910+0000 7f3176ff6700 1 -- 172.21.15.92:0/855304767 >> 172.21.15.92:0/855304767 conn(0x7f31700f8540 msgr2=0x7f31700f9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:23:11.914 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.910+0000 7f3176ff6700 1 -- 172.21.15.92:0/855304767 shutdown_connections 2024-05-06T12:23:11.914 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:11.910+0000 7f3176ff6700 1 -- 172.21.15.92:0/855304767 wait complete. 2024-05-06T12:23:12.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.012+0000 7f248bf93700 1 -- 172.21.15.92:0/2158740675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24840fbce0 msgr2=0x7f24840fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:12.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.012+0000 7f248bf93700 1 --2- 172.21.15.92:0/2158740675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24840fbce0 0x7f24840fc0f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2480004940 tx=0x7f2480009e30 comp rx=0 tx=0).stop 2024-05-06T12:23:12.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.012+0000 7f248bf93700 1 -- 172.21.15.92:0/2158740675 shutdown_connections 2024-05-06T12:23:12.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.012+0000 7f248bf93700 1 --2- 172.21.15.92:0/2158740675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24840fcf90 0x7f24840fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:12.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.012+0000 7f248bf93700 1 --2- 172.21.15.92:0/2158740675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24840fbce0 0x7f24840fc0f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:12.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.012+0000 7f248bf93700 1 -- 172.21.15.92:0/2158740675 >> 172.21.15.92:0/2158740675 conn(0x7f24840f7440 msgr2=0x7f24840f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:23:12.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.012+0000 7f248bf93700 1 -- 172.21.15.92:0/2158740675 shutdown_connections 2024-05-06T12:23:12.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.012+0000 7f248bf93700 1 -- 172.21.15.92:0/2158740675 wait complete. 2024-05-06T12:23:12.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.012+0000 7f248bf93700 1 Processor -- start 2024-05-06T12:23:12.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.012+0000 7f248bf93700 1 -- start start 2024-05-06T12:23:12.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.012+0000 7f248bf93700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24840fbce0 0x7f24840ffc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:12.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.012+0000 7f248bf93700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24840fcf90 0x7f24840fe2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:12.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.012+0000 7f248bf93700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2484100220 con 0x7f24840fbce0 2024-05-06T12:23:12.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.012+0000 7f248bf93700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24840fe810 con 0x7f24840fcf90 2024-05-06T12:23:12.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.013+0000 7f2489d2f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24840fbce0 0x7f24840ffc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:12.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.013+0000 7f2489d2f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24840fbce0 0x7f24840ffc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49558/0 (socket says 172.21.15.92:49558) 2024-05-06T12:23:12.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.013+0000 7f2489d2f700 1 -- 172.21.15.92:0/1578972961 learned_addr learned my addr 172.21.15.92:0/1578972961 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:23:12.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.013+0000 7f248952e700 1 --2- 172.21.15.92:0/1578972961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24840fcf90 0x7f24840fe2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:12.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.013+0000 7f2489d2f700 1 -- 172.21.15.92:0/1578972961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24840fcf90 msgr2=0x7f24840fe2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:12.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.013+0000 7f2489d2f700 1 --2- 172.21.15.92:0/1578972961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24840fcf90 0x7f24840fe2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:12.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.013+0000 7f2489d2f700 1 -- 172.21.15.92:0/1578972961 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2480005040 con 0x7f24840fbce0 2024-05-06T12:23:12.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.013+0000 7f2489d2f700 1 --2- 172.21.15.92:0/1578972961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24840fbce0 0x7f24840ffc20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2480009150 tx=0x7f2480009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:23:12.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.013+0000 7f247a7fc700 1 -- 172.21.15.92:0/1578972961 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f248000e040 con 0x7f24840fbce0 2024-05-06T12:23:12.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.013+0000 7f247a7fc700 1 -- 172.21.15.92:0/1578972961 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f248000bbf0 con 0x7f24840fbce0 2024-05-06T12:23:12.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.013+0000 7f248bf93700 1 -- 172.21.15.92:0/1578972961 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24840fea60 con 0x7f24840fbce0 2024-05-06T12:23:12.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.013+0000 7f247a7fc700 1 -- 172.21.15.92:0/1578972961 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f248001c630 con 0x7f24840fbce0 2024-05-06T12:23:12.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.013+0000 7f248bf93700 1 -- 172.21.15.92:0/1578972961 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24840ff020 con 0x7f24840fbce0 2024-05-06T12:23:12.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.014+0000 7f248bf93700 1 -- 172.21.15.92:0/1578972961 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2484101b20 con 0x7f24840fbce0 2024-05-06T12:23:12.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.015+0000 7f247a7fc700 1 -- 172.21.15.92:0/1578972961 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f248000bd60 con 0x7f24840fbce0 2024-05-06T12:23:12.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.015+0000 7f247a7fc700 1 --2- 172.21.15.92:0/1578972961 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f247006e140 0x7f24700705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:12.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.015+0000 7f247a7fc700 1 -- 172.21.15.92:0/1578972961 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f248008ee50 con 0x7f24840fbce0 2024-05-06T12:23:12.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.015+0000 7f248952e700 1 --2- 172.21.15.92:0/1578972961 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f247006e140 0x7f24700705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:12.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.016+0000 7f248952e700 1 --2- 172.21.15.92:0/1578972961 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f247006e140 0x7f24700705f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2474007fd0 tx=0x7f2474009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:23:12.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.020+0000 7f247a7fc700 1 -- 172.21.15.92:0/1578972961 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f248005bdf0 con 0x7f24840fbce0 2024-05-06T12:23:12.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.305+0000 7f248bf93700 1 -- 172.21.15.92:0/1578972961 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f248404c710 con 0x7f24840fbce0 2024-05-06T12:23:12.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.305+0000 7f247a7fc700 1 -- 172.21.15.92:0/1578972961 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f248005b980 con 0x7f24840fbce0 2024-05-06T12:23:12.308 INFO:teuthology.orchestra.run.smithi092.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-06T12:23:12.308 INFO:teuthology.orchestra.run.smithi092.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-06T12:23:12.308 INFO:teuthology.orchestra.run.smithi092.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-06T12:23:12.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.310+0000 7f248bf93700 1 -- 172.21.15.92:0/1578972961 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f247006e140 msgr2=0x7f24700705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:12.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.310+0000 7f248bf93700 1 --2- 172.21.15.92:0/1578972961 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f247006e140 0x7f24700705f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2474007fd0 tx=0x7f2474009040 comp rx=0 tx=0).stop 2024-05-06T12:23:12.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.310+0000 7f248bf93700 1 -- 172.21.15.92:0/1578972961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24840fbce0 msgr2=0x7f24840ffc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:12.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.310+0000 7f248bf93700 1 --2- 172.21.15.92:0/1578972961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24840fbce0 0x7f24840ffc20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2480009150 tx=0x7f2480009e30 comp rx=0 tx=0).stop 2024-05-06T12:23:12.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.310+0000 7f248bf93700 1 -- 172.21.15.92:0/1578972961 shutdown_connections 2024-05-06T12:23:12.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.310+0000 7f248bf93700 1 --2- 172.21.15.92:0/1578972961 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f247006e140 0x7f24700705f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:12.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.310+0000 7f248bf93700 1 --2- 172.21.15.92:0/1578972961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24840fbce0 0x7f24840ffc20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:12.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.310+0000 7f248bf93700 1 --2- 172.21.15.92:0/1578972961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24840fcf90 0x7f24840fe2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:12.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.310+0000 7f248bf93700 1 -- 172.21.15.92:0/1578972961 >> 172.21.15.92:0/1578972961 conn(0x7f24840f7440 msgr2=0x7f2484066830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:23:12.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.310+0000 7f248bf93700 1 -- 172.21.15.92:0/1578972961 shutdown_connections 2024-05-06T12:23:12.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:12.310+0000 7f248bf93700 1 -- 172.21.15.92:0/1578972961 wait complete. 2024-05-06T12:23:12.847 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:12 smithi190 ceph-mon[79744]: from='client.34526 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:23:12.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:12 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/1578972961' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:23:12.877 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:12 smithi092 ceph-mon[91933]: from='client.34526 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:23:12.877 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:12 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/1578972961' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:23:13.847 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:13 smithi190 ceph-mon[79744]: pgmap v236: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:23:13.972 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:13 smithi092 ceph-mon[91933]: pgmap v236: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:23:16.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:15 smithi092 ceph-mon[91933]: pgmap v237: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:23:16.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:15 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:16.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:15 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:16.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:15 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:23:16.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:15 smithi190 ceph-mon[79744]: pgmap v237: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:23:16.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:15 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:16.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:15 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:16.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:15 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:23:18.217 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:17 smithi092 ceph-mon[91933]: pgmap v238: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:23:18.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:17 smithi190 ceph-mon[79744]: pgmap v238: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:23:20.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:19 smithi092 ceph-mon[91933]: pgmap v239: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:23:20.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:19 smithi190 ceph-mon[79744]: pgmap v239: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:23:21.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:21 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:21.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:21 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:21.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:21 smithi190 ceph-mon[79744]: pgmap v240: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:23:21.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:21 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:21.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:21 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:21.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:21 smithi092 ceph-mon[91933]: pgmap v240: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:23:22.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:22 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:22.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:22 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:22.847 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:22 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:22.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:22 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:23.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:23 smithi092 ceph-mon[91933]: pgmap v241: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:23:23.847 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:23 smithi190 ceph-mon[79744]: pgmap v241: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:23:24.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:24 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:23:24.848 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:24 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:23:26.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:26 smithi190 ceph-mon[79744]: pgmap v242: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:23:26.466 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:26 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:26.466 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:26 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:26.467 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:26 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:23:26.467 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:26 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:23:26.467 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:26 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:26.467 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:26 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-06T12:23:26.467 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:26 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:23:26.467 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:26 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:26.467 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:26 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:26.467 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:26 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:26.468 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:26 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:26.468 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:26 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:26.468 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:26 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:26.468 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:26 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:26.468 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:26 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:26.468 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:26 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:26.468 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:26 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:26.468 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:26 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:26.468 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:26 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:26.468 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:26 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:26.468 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:26 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:26.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:26 smithi092 ceph-mon[91933]: pgmap v242: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:23:26.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:26 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:26.473 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:26 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:26.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:26 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:23:26.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:26 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:23:26.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:26 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:26.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:26 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-06T12:23:26.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:26 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:23:26.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:26 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:26.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:26 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:26.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:26 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:26.475 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:26 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:26.475 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:26 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:26.475 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:26 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:26.475 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:26 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:26.475 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:26 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:26.475 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:26 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:26.475 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:26 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:26.475 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:26 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:26.475 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:26 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:26.475 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:26 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:26.476 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:26 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:27.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:27 smithi092 ceph-mon[91933]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-06T12:23:27.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:27 smithi190 ceph-mon[79744]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-06T12:23:28.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:28 smithi092 ceph-mon[91933]: pgmap v243: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:23:28.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:28 smithi092 ceph-mon[91933]: Upgrade: Updating alertmanager.smithi092 2024-05-06T12:23:28.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:28 smithi092 ceph-mon[91933]: Deploying daemon alertmanager.smithi092 on smithi092 2024-05-06T12:23:28.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:28 smithi190 ceph-mon[79744]: pgmap v243: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:23:28.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:28 smithi190 ceph-mon[79744]: Upgrade: Updating alertmanager.smithi092 2024-05-06T12:23:28.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:28 smithi190 ceph-mon[79744]: Deploying daemon alertmanager.smithi092 on smithi092 2024-05-06T12:23:30.159 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:30 smithi092 ceph-mon[91933]: pgmap v244: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:23:30.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:30 smithi190 ceph-mon[79744]: pgmap v244: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:23:32.319 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:32 smithi092 ceph-mon[91933]: pgmap v245: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:23:32.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:32 smithi190 ceph-mon[79744]: pgmap v245: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:23:33.898 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:33 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:33.898 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:33 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:33.898 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:33 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:23:33.898 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:33 smithi092 ceph-mon[91933]: pgmap v246: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:23:34.097 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:33 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:34.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:33 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:34.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:33 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:23:34.098 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:33 smithi190 ceph-mon[79744]: pgmap v246: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:23:36.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:35 smithi092 ceph-mon[91933]: pgmap v247: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:23:36.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:35 smithi190 ceph-mon[79744]: pgmap v247: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:23:38.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:37 smithi092 ceph-mon[91933]: pgmap v248: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:23:38.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:37 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:38.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:37 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:38.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:37 smithi190 ceph-mon[79744]: pgmap v248: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:23:38.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:37 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:38.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:37 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:39.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:39 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:39.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:39 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:39.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:39 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:23:39.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:39 smithi092 ceph-mon[91933]: pgmap v249: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:23:39.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:39 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:39.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:39 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:39.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:39 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:23:39.598 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:39 smithi190 ceph-mon[79744]: pgmap v249: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:23:42.075 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:41 smithi092 ceph-mon[91933]: pgmap v250: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:23:42.075 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:41 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:42.075 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:41 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:42.075 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:41 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:23:42.075 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:41 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:23:42.075 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:41 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:42.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:41 smithi190 ceph-mon[79744]: pgmap v250: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:23:42.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:41 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:42.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:41 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:42.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:41 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:23:42.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:41 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:23:42.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:41 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:42.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.420+0000 7f1ea64a5700 1 -- 172.21.15.92:0/4139024469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ea00fe430 msgr2=0x7f1ea00fe840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:42.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.420+0000 7f1ea64a5700 1 --2- 172.21.15.92:0/4139024469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ea00fe430 0x7f1ea00fe840 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1e90004940 tx=0x7f1e90009e30 comp rx=0 tx=0).stop 2024-05-06T12:23:42.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.420+0000 7f1ea64a5700 1 -- 172.21.15.92:0/4139024469 shutdown_connections 2024-05-06T12:23:42.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.420+0000 7f1ea64a5700 1 --2- 172.21.15.92:0/4139024469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ea00fed80 0x7f1ea00faec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:42.424 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.420+0000 7f1ea64a5700 1 --2- 172.21.15.92:0/4139024469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ea00fe430 0x7f1ea00fe840 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:42.424 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.420+0000 7f1ea64a5700 1 -- 172.21.15.92:0/4139024469 >> 172.21.15.92:0/4139024469 conn(0x7f1ea00f6bb0 msgr2=0x7f1ea00f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:23:42.424 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.420+0000 7f1ea64a5700 1 -- 172.21.15.92:0/4139024469 shutdown_connections 2024-05-06T12:23:42.424 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.420+0000 7f1ea64a5700 1 -- 172.21.15.92:0/4139024469 wait complete. 2024-05-06T12:23:42.424 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.421+0000 7f1ea64a5700 1 Processor -- start 2024-05-06T12:23:42.424 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.421+0000 7f1ea64a5700 1 -- start start 2024-05-06T12:23:42.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.421+0000 7f1ea64a5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ea00fe430 0x7f1ea00fda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:42.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.421+0000 7f1ea64a5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ea00fed80 0x7f1ea00fc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:42.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.421+0000 7f1ea64a5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea00fc630 con 0x7f1ea00fe430 2024-05-06T12:23:42.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.421+0000 7f1ea64a5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea00fc7a0 con 0x7f1ea00fed80 2024-05-06T12:23:42.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.421+0000 7f1e9ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ea00fe430 0x7f1ea00fda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:42.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.421+0000 7f1e9ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ea00fe430 0x7f1ea00fda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53822/0 (socket says 172.21.15.92:53822) 2024-05-06T12:23:42.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.421+0000 7f1e9ffff700 1 -- 172.21.15.92:0/2579406997 learned_addr learned my addr 172.21.15.92:0/2579406997 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:23:42.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.421+0000 7f1e9f7fe700 1 --2- 172.21.15.92:0/2579406997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ea00fed80 0x7f1ea00fc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:42.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.422+0000 7f1e9ffff700 1 -- 172.21.15.92:0/2579406997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ea00fed80 msgr2=0x7f1ea00fc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:42.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.422+0000 7f1e9ffff700 1 --2- 172.21.15.92:0/2579406997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ea00fed80 0x7f1ea00fc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:42.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.422+0000 7f1e9ffff700 1 -- 172.21.15.92:0/2579406997 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e90005040 con 0x7f1ea00fe430 2024-05-06T12:23:42.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.422+0000 7f1e9ffff700 1 --2- 172.21.15.92:0/2579406997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ea00fe430 0x7f1ea00fda40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1e9000b3b0 tx=0x7f1e90009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:23:42.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.422+0000 7f1e9cff9700 1 -- 172.21.15.92:0/2579406997 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e9000e040 con 0x7f1ea00fe430 2024-05-06T12:23:42.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.422+0000 7f1e9cff9700 1 -- 172.21.15.92:0/2579406997 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e9000bbf0 con 0x7f1ea00fe430 2024-05-06T12:23:42.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.422+0000 7f1e9cff9700 1 -- 172.21.15.92:0/2579406997 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e9001c630 con 0x7f1ea00fe430 2024-05-06T12:23:42.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.422+0000 7f1ea64a5700 1 -- 172.21.15.92:0/2579406997 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea00fca20 con 0x7f1ea00fe430 2024-05-06T12:23:42.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.422+0000 7f1ea64a5700 1 -- 172.21.15.92:0/2579406997 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea0115a70 con 0x7f1ea00fe430 2024-05-06T12:23:42.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.423+0000 7f1ea64a5700 1 -- 172.21.15.92:0/2579406997 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ea004c710 con 0x7f1ea00fe430 2024-05-06T12:23:42.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.424+0000 7f1e9cff9700 1 -- 172.21.15.92:0/2579406997 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1e9000bd60 con 0x7f1ea00fe430 2024-05-06T12:23:42.428 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.424+0000 7f1e9cff9700 1 --2- 172.21.15.92:0/2579406997 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f1e8806e140 0x7f1e880705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:42.428 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.424+0000 7f1e9cff9700 1 -- 172.21.15.92:0/2579406997 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f1e9008ed80 con 0x7f1ea00fe430 2024-05-06T12:23:42.428 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.424+0000 7f1e9f7fe700 1 --2- 172.21.15.92:0/2579406997 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f1e8806e140 0x7f1e880705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:42.428 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.425+0000 7f1e9f7fe700 1 --2- 172.21.15.92:0/2579406997 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f1e8806e140 0x7f1e880705f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1e94009bf0 tx=0x7f1e940091a0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:23:42.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.433+0000 7f1e9cff9700 1 -- 172.21.15.92:0/2579406997 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1e9005bd20 con 0x7f1ea00fe430 2024-05-06T12:23:42.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.670+0000 7f1ea64a5700 1 -- 172.21.15.92:0/2579406997 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ea0102500 con 0x7f1e8806e140 2024-05-06T12:23:42.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.672+0000 7f1e9cff9700 1 -- 172.21.15.92:0/2579406997 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+435 (secure 0 0 0) 0x7f1ea0102500 con 0x7f1e8806e140 2024-05-06T12:23:42.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.674+0000 7f1ea64a5700 1 -- 172.21.15.92:0/2579406997 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f1e8806e140 msgr2=0x7f1e880705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:42.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.674+0000 7f1ea64a5700 1 --2- 172.21.15.92:0/2579406997 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f1e8806e140 0x7f1e880705f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1e94009bf0 tx=0x7f1e940091a0 comp rx=0 tx=0).stop 2024-05-06T12:23:42.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.674+0000 7f1ea64a5700 1 -- 172.21.15.92:0/2579406997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ea00fe430 msgr2=0x7f1ea00fda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:42.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.674+0000 7f1ea64a5700 1 --2- 172.21.15.92:0/2579406997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ea00fe430 0x7f1ea00fda40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1e9000b3b0 tx=0x7f1e90009e30 comp rx=0 tx=0).stop 2024-05-06T12:23:42.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.674+0000 7f1ea64a5700 1 -- 172.21.15.92:0/2579406997 shutdown_connections 2024-05-06T12:23:42.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.674+0000 7f1ea64a5700 1 --2- 172.21.15.92:0/2579406997 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f1e8806e140 0x7f1e880705f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:42.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.674+0000 7f1ea64a5700 1 --2- 172.21.15.92:0/2579406997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ea00fe430 0x7f1ea00fda40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:42.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.674+0000 7f1ea64a5700 1 --2- 172.21.15.92:0/2579406997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ea00fed80 0x7f1ea00fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:42.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.674+0000 7f1ea64a5700 1 -- 172.21.15.92:0/2579406997 >> 172.21.15.92:0/2579406997 conn(0x7f1ea00f6bb0 msgr2=0x7f1ea00f8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:23:42.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.674+0000 7f1ea64a5700 1 -- 172.21.15.92:0/2579406997 shutdown_connections 2024-05-06T12:23:42.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.674+0000 7f1ea64a5700 1 -- 172.21.15.92:0/2579406997 wait complete. 2024-05-06T12:23:42.686 INFO:teuthology.orchestra.run.smithi092.stdout:true 2024-05-06T12:23:42.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.780+0000 7f5d1373f700 1 -- 172.21.15.92:0/1160441785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d0c0fbcb0 msgr2=0x7f5d0c0fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:42.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.780+0000 7f5d1373f700 1 --2- 172.21.15.92:0/1160441785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d0c0fbcb0 0x7f5d0c0fc0c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5d08004940 tx=0x7f5d08009e30 comp rx=0 tx=0).stop 2024-05-06T12:23:42.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.780+0000 7f5d1373f700 1 -- 172.21.15.92:0/1160441785 shutdown_connections 2024-05-06T12:23:42.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.780+0000 7f5d1373f700 1 --2- 172.21.15.92:0/1160441785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d0c0fcf60 0x7f5d0c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:42.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.780+0000 7f5d1373f700 1 --2- 172.21.15.92:0/1160441785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d0c0fbcb0 0x7f5d0c0fc0c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:42.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.780+0000 7f5d1373f700 1 -- 172.21.15.92:0/1160441785 >> 172.21.15.92:0/1160441785 conn(0x7f5d0c0f7430 msgr2=0x7f5d0c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:23:42.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.780+0000 7f5d1373f700 1 -- 172.21.15.92:0/1160441785 shutdown_connections 2024-05-06T12:23:42.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.780+0000 7f5d1373f700 1 -- 172.21.15.92:0/1160441785 wait complete. 2024-05-06T12:23:42.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.781+0000 7f5d1373f700 1 Processor -- start 2024-05-06T12:23:42.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.781+0000 7f5d1373f700 1 -- start start 2024-05-06T12:23:42.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.781+0000 7f5d1373f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d0c0fbcb0 0x7f5d0c10c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:42.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.781+0000 7f5d1373f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d0c0fcf60 0x7f5d0c10ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:42.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.781+0000 7f5d1373f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c10d440 con 0x7f5d0c0fbcb0 2024-05-06T12:23:42.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.781+0000 7f5d1373f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c10dc90 con 0x7f5d0c0fcf60 2024-05-06T12:23:42.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.781+0000 7f5d114db700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d0c0fbcb0 0x7f5d0c10c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:42.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.781+0000 7f5d114db700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d0c0fbcb0 0x7f5d0c10c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51246/0 (socket says 172.21.15.92:51246) 2024-05-06T12:23:42.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.781+0000 7f5d114db700 1 -- 172.21.15.92:0/2764780518 learned_addr learned my addr 172.21.15.92:0/2764780518 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:23:42.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.781+0000 7f5d10cda700 1 --2- 172.21.15.92:0/2764780518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d0c0fcf60 0x7f5d0c10ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:42.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.781+0000 7f5d114db700 1 -- 172.21.15.92:0/2764780518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d0c0fcf60 msgr2=0x7f5d0c10ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:42.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.781+0000 7f5d114db700 1 --2- 172.21.15.92:0/2764780518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d0c0fcf60 0x7f5d0c10ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:42.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.781+0000 7f5d114db700 1 -- 172.21.15.92:0/2764780518 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d08005040 con 0x7f5d0c0fbcb0 2024-05-06T12:23:42.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.782+0000 7f5d114db700 1 --2- 172.21.15.92:0/2764780518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d0c0fbcb0 0x7f5d0c10c900 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5d08009150 tx=0x7f5d08009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:23:42.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.782+0000 7f5d01ffb700 1 -- 172.21.15.92:0/2764780518 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d0800e040 con 0x7f5d0c0fbcb0 2024-05-06T12:23:42.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.782+0000 7f5d01ffb700 1 -- 172.21.15.92:0/2764780518 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d0800bcc0 con 0x7f5d0c0fbcb0 2024-05-06T12:23:42.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.782+0000 7f5d1373f700 1 -- 172.21.15.92:0/2764780518 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d0c10df10 con 0x7f5d0c0fbcb0 2024-05-06T12:23:42.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.782+0000 7f5d01ffb700 1 -- 172.21.15.92:0/2764780518 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d0801c630 con 0x7f5d0c0fbcb0 2024-05-06T12:23:42.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.782+0000 7f5d1373f700 1 -- 172.21.15.92:0/2764780518 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d0c10e450 con 0x7f5d0c0fbcb0 2024-05-06T12:23:42.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.783+0000 7f5d1373f700 1 -- 172.21.15.92:0/2764780518 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d0c05f5c0 con 0x7f5d0c0fbcb0 2024-05-06T12:23:42.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.783+0000 7f5d01ffb700 1 -- 172.21.15.92:0/2764780518 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5d08013420 con 0x7f5d0c0fbcb0 2024-05-06T12:23:42.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.784+0000 7f5d01ffb700 1 --2- 172.21.15.92:0/2764780518 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f5cf806e140 0x7f5cf80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:42.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.784+0000 7f5d01ffb700 1 -- 172.21.15.92:0/2764780518 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f5d0808e6f0 con 0x7f5d0c0fbcb0 2024-05-06T12:23:42.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.784+0000 7f5d10cda700 1 --2- 172.21.15.92:0/2764780518 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f5cf806e140 0x7f5cf80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:42.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.784+0000 7f5d10cda700 1 --2- 172.21.15.92:0/2764780518 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f5cf806e140 0x7f5cf80705f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc004990 tx=0x7f5cfc009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:23:42.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:42.789+0000 7f5d01ffb700 1 -- 172.21.15.92:0/2764780518 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5d0805b690 con 0x7f5d0c0fbcb0 2024-05-06T12:23:43.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.024+0000 7f5d1373f700 1 -- 172.21.15.92:0/2764780518 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d0c10e7d0 con 0x7f5cf806e140 2024-05-06T12:23:43.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.025+0000 7f5d01ffb700 1 -- 172.21.15.92:0/2764780518 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+435 (secure 0 0 0) 0x7f5d0c10e7d0 con 0x7f5cf806e140 2024-05-06T12:23:43.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.027+0000 7f5d1373f700 1 -- 172.21.15.92:0/2764780518 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f5cf806e140 msgr2=0x7f5cf80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:43.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.027+0000 7f5d1373f700 1 --2- 172.21.15.92:0/2764780518 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f5cf806e140 0x7f5cf80705f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc004990 tx=0x7f5cfc009ee0 comp rx=0 tx=0).stop 2024-05-06T12:23:43.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.027+0000 7f5d1373f700 1 -- 172.21.15.92:0/2764780518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d0c0fbcb0 msgr2=0x7f5d0c10c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:43.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.027+0000 7f5d1373f700 1 --2- 172.21.15.92:0/2764780518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d0c0fbcb0 0x7f5d0c10c900 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5d08009150 tx=0x7f5d08009e30 comp rx=0 tx=0).stop 2024-05-06T12:23:43.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.027+0000 7f5d1373f700 1 -- 172.21.15.92:0/2764780518 shutdown_connections 2024-05-06T12:23:43.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.027+0000 7f5d1373f700 1 --2- 172.21.15.92:0/2764780518 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f5cf806e140 0x7f5cf80705f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:43.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.027+0000 7f5d1373f700 1 --2- 172.21.15.92:0/2764780518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d0c0fbcb0 0x7f5d0c10c900 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:43.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.027+0000 7f5d1373f700 1 --2- 172.21.15.92:0/2764780518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d0c0fcf60 0x7f5d0c10ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:43.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.027+0000 7f5d1373f700 1 -- 172.21.15.92:0/2764780518 >> 172.21.15.92:0/2764780518 conn(0x7f5d0c0f7430 msgr2=0x7f5d0c0f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:23:43.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.027+0000 7f5d1373f700 1 -- 172.21.15.92:0/2764780518 shutdown_connections 2024-05-06T12:23:43.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.027+0000 7f5d1373f700 1 -- 172.21.15.92:0/2764780518 wait complete. 2024-05-06T12:23:43.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.129+0000 7f83f8d35700 1 -- 172.21.15.92:0/1594548456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83f40fdfd0 msgr2=0x7f83f40fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:43.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.129+0000 7f83f8d35700 1 --2- 172.21.15.92:0/1594548456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83f40fdfd0 0x7f83f40fe440 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f83e8004940 tx=0x7f83e8009e30 comp rx=0 tx=0).stop 2024-05-06T12:23:43.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.129+0000 7f83f8d35700 1 -- 172.21.15.92:0/1594548456 shutdown_connections 2024-05-06T12:23:43.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.129+0000 7f83f8d35700 1 --2- 172.21.15.92:0/1594548456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83f40fdfd0 0x7f83f40fe440 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:43.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.129+0000 7f83f8d35700 1 --2- 172.21.15.92:0/1594548456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83f40fce60 0x7f83f40fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:43.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.129+0000 7f83f8d35700 1 -- 172.21.15.92:0/1594548456 >> 172.21.15.92:0/1594548456 conn(0x7f83f40f8540 msgr2=0x7f83f40fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:23:43.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.129+0000 7f83f8d35700 1 -- 172.21.15.92:0/1594548456 shutdown_connections 2024-05-06T12:23:43.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.129+0000 7f83f8d35700 1 -- 172.21.15.92:0/1594548456 wait complete. 2024-05-06T12:23:43.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.129+0000 7f83f8d35700 1 Processor -- start 2024-05-06T12:23:43.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.129+0000 7f83f8d35700 1 -- start start 2024-05-06T12:23:43.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.130+0000 7f83f8d35700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83f40fce60 0x7f83f410d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:43.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.130+0000 7f83f8d35700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83f40fdfd0 0x7f83f410de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:43.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.130+0000 7f83f8d35700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f410e4f0 con 0x7f83f40fdfd0 2024-05-06T12:23:43.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.130+0000 7f83f8d35700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f4112e90 con 0x7f83f40fce60 2024-05-06T12:23:43.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.130+0000 7f83f1d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83f40fdfd0 0x7f83f410de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:43.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.130+0000 7f83f1d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83f40fdfd0 0x7f83f410de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51266/0 (socket says 172.21.15.92:51266) 2024-05-06T12:23:43.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.130+0000 7f83f1d9b700 1 -- 172.21.15.92:0/2781892287 learned_addr learned my addr 172.21.15.92:0/2781892287 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:23:43.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.130+0000 7f83f259c700 1 --2- 172.21.15.92:0/2781892287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83f40fce60 0x7f83f410d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:43.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.130+0000 7f83f1d9b700 1 -- 172.21.15.92:0/2781892287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83f40fce60 msgr2=0x7f83f410d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:43.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.130+0000 7f83f1d9b700 1 --2- 172.21.15.92:0/2781892287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83f40fce60 0x7f83f410d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:43.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.130+0000 7f83f1d9b700 1 -- 172.21.15.92:0/2781892287 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e8005040 con 0x7f83f40fdfd0 2024-05-06T12:23:43.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.130+0000 7f83f259c700 1 --2- 172.21.15.92:0/2781892287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83f40fce60 0x7f83f410d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-06T12:23:43.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.130+0000 7f83f1d9b700 1 --2- 172.21.15.92:0/2781892287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83f40fdfd0 0x7f83f410de60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f83e800b3b0 tx=0x7f83e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:23:43.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.130+0000 7f83e2ffd700 1 -- 172.21.15.92:0/2781892287 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83e800e040 con 0x7f83f40fdfd0 2024-05-06T12:23:43.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.130+0000 7f83e2ffd700 1 -- 172.21.15.92:0/2781892287 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f83e800bcc0 con 0x7f83f40fdfd0 2024-05-06T12:23:43.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.130+0000 7f83f8d35700 1 -- 172.21.15.92:0/2781892287 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f4113090 con 0x7f83f40fdfd0 2024-05-06T12:23:43.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.130+0000 7f83e2ffd700 1 -- 172.21.15.92:0/2781892287 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83e801c630 con 0x7f83f40fdfd0 2024-05-06T12:23:43.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.130+0000 7f83f8d35700 1 -- 172.21.15.92:0/2781892287 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f4113600 con 0x7f83f40fdfd0 2024-05-06T12:23:43.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.131+0000 7f83f8d35700 1 -- 172.21.15.92:0/2781892287 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83f404c710 con 0x7f83f40fdfd0 2024-05-06T12:23:43.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.132+0000 7f83e2ffd700 1 -- 172.21.15.92:0/2781892287 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f83e8013980 con 0x7f83f40fdfd0 2024-05-06T12:23:43.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.132+0000 7f83e2ffd700 1 --2- 172.21.15.92:0/2781892287 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f83dc06e140 0x7f83dc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:43.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.132+0000 7f83e2ffd700 1 -- 172.21.15.92:0/2781892287 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f83e808e880 con 0x7f83f40fdfd0 2024-05-06T12:23:43.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.132+0000 7f83f259c700 1 --2- 172.21.15.92:0/2781892287 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f83dc06e140 0x7f83dc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:43.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.133+0000 7f83f259c700 1 --2- 172.21.15.92:0/2781892287 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f83dc06e140 0x7f83dc0705f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f83e4009b10 tx=0x7f83e4008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:23:43.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.137+0000 7f83e2ffd700 1 -- 172.21.15.92:0/2781892287 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f83e805b6e0 con 0x7f83f40fdfd0 2024-05-06T12:23:43.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:42 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-06T12:23:43.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:42 smithi092 ceph-mon[91933]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-06T12:23:43.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:42 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi092:9093"}]: dispatch 2024-05-06T12:23:43.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:42 smithi092 ceph-mon[91933]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi092:9093"}]: dispatch 2024-05-06T12:23:43.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:42 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:43.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:42 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:23:43.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:42 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:43.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:42 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:43.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:42 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:43.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:42 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:43.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:42 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:43.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:42 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:43.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:42 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:43.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:42 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:43.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:42 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:43.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:42 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:43.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:42 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:43.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:42 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:43.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:42 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:43.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:42 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:43.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:42 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:43.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:42 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-06T12:23:43.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:42 smithi190 ceph-mon[79744]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-06T12:23:43.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:42 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi092:9093"}]: dispatch 2024-05-06T12:23:43.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:42 smithi190 ceph-mon[79744]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi092:9093"}]: dispatch 2024-05-06T12:23:43.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:42 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:43.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:42 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:23:43.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:42 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:43.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:42 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:43.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:42 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:43.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:42 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:43.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:42 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:43.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:42 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:43.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:42 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:43.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:42 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:43.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:42 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:43.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:42 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:43.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:42 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:43.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:42 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:43.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:42 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:43.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:42 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:43.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:42 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:43.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.351+0000 7f83f8d35700 1 -- 172.21.15.92:0/2781892287 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f83f41022f0 con 0x7f83dc06e140 2024-05-06T12:23:43.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.361+0000 7f83e2ffd700 1 -- 172.21.15.92:0/2781892287 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f83f41022f0 con 0x7f83dc06e140 2024-05-06T12:23:43.363 INFO:teuthology.orchestra.run.smithi092.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-06T12:23:43.363 INFO:teuthology.orchestra.run.smithi092.stdout:alertmanager.smithi092 smithi092 *:9093,9094 running (11s) 6s ago 17m 15.1M - 0.25.0 c8568f914cd2 c9a1046648dc 2024-05-06T12:23:43.363 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi092 smithi092 running (6m) 6s ago 17m 6660k - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 8da0494e1bdc 2024-05-06T12:23:43.364 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi190 smithi190 running (6m) 41s ago 16m 6665k - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 641a7cdaa56e 2024-05-06T12:23:43.364 INFO:teuthology.orchestra.run.smithi092.stdout:grafana.smithi092 smithi092 *:3000 running (8m) 6s ago 16m 81.1M - 9.4.7 954c08fa6188 9180cb3e5258 2024-05-06T12:23:43.365 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.ijsqgp smithi092 running (118s) 6s ago 11m 66.7M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 40b4cdd3be40 2024-05-06T12:23:43.365 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.rosbll smithi092 running (96s) 6s ago 11m 16.4M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 550f8238d95f 2024-05-06T12:23:43.365 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.byjvgf smithi190 running (79s) 41s ago 11m 63.4M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 0d26051c908e 2024-05-06T12:23:43.365 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.idzoff smithi190 running (62s) 41s ago 11m 12.8M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 7cfd0097389b 2024-05-06T12:23:43.366 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi092.gyzeqb smithi092 *:8443,9283,8765 running (8m) 6s ago 18m 513M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 de9d2198327a 2024-05-06T12:23:43.366 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi190.kczxkh smithi190 *:8443,9283,8765 running (7m) 41s ago 16m 452M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 e84ab3758ac7 2024-05-06T12:23:43.366 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi092 smithi092 running (7m) 6s ago 18m 54.9M 2048M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 ac07f7158087 2024-05-06T12:23:43.366 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi190 smithi190 running (6m) 41s ago 16m 39.7M 2048M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 7a3c08eb716f 2024-05-06T12:23:43.366 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi092 smithi092 *:9100 running (48s) 6s ago 17m 7796k - 1.5.0 0da6a335fe13 590fa41f0538 2024-05-06T12:23:43.366 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi190 smithi190 *:9100 running (44s) 41s ago 16m 3590k - 1.5.0 0da6a335fe13 a57191dac5b5 2024-05-06T12:23:43.366 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 smithi092 running (5m) 6s ago 15m 94.2M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 8c96c75b25f4 2024-05-06T12:23:43.366 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 smithi092 running (5m) 6s ago 15m 97.3M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 e8007a510f08 2024-05-06T12:23:43.366 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 smithi092 running (4m) 6s ago 14m 91.5M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 2beca7448789 2024-05-06T12:23:43.366 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 smithi190 running (3m) 41s ago 14m 101M 2581M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 914ada199f93 2024-05-06T12:23:43.366 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 smithi190 running (3m) 41s ago 13m 89.4M 2581M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 371c0be776c7 2024-05-06T12:23:43.366 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 smithi190 running (2m) 41s ago 13m 86.8M 2581M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 067bc34cdfb1 2024-05-06T12:23:43.366 INFO:teuthology.orchestra.run.smithi092.stdout:prometheus.smithi092 smithi092 *:9095 running (27s) 6s ago 16m 46.7M - 2.43.0 a07b618ecd1d eadf61684ae7 2024-05-06T12:23:43.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.363+0000 7f83f8d35700 1 -- 172.21.15.92:0/2781892287 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f83dc06e140 msgr2=0x7f83dc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:43.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.363+0000 7f83f8d35700 1 --2- 172.21.15.92:0/2781892287 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f83dc06e140 0x7f83dc0705f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f83e4009b10 tx=0x7f83e4008040 comp rx=0 tx=0).stop 2024-05-06T12:23:43.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.363+0000 7f83f8d35700 1 -- 172.21.15.92:0/2781892287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83f40fdfd0 msgr2=0x7f83f410de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:43.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.363+0000 7f83f8d35700 1 --2- 172.21.15.92:0/2781892287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83f40fdfd0 0x7f83f410de60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f83e800b3b0 tx=0x7f83e8009e30 comp rx=0 tx=0).stop 2024-05-06T12:23:43.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.363+0000 7f83f8d35700 1 -- 172.21.15.92:0/2781892287 shutdown_connections 2024-05-06T12:23:43.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.363+0000 7f83f8d35700 1 --2- 172.21.15.92:0/2781892287 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f83dc06e140 0x7f83dc0705f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:43.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.363+0000 7f83f8d35700 1 --2- 172.21.15.92:0/2781892287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83f40fce60 0x7f83f410d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:43.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.363+0000 7f83f8d35700 1 --2- 172.21.15.92:0/2781892287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83f40fdfd0 0x7f83f410de60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:43.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.363+0000 7f83f8d35700 1 -- 172.21.15.92:0/2781892287 >> 172.21.15.92:0/2781892287 conn(0x7f83f40f8540 msgr2=0x7f83f40f9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:23:43.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.363+0000 7f83f8d35700 1 -- 172.21.15.92:0/2781892287 shutdown_connections 2024-05-06T12:23:43.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.363+0000 7f83f8d35700 1 -- 172.21.15.92:0/2781892287 wait complete. 2024-05-06T12:23:43.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.463+0000 7f254a53a700 1 -- 172.21.15.92:0/4052980157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2544066260 msgr2=0x7f25440ffa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:43.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.463+0000 7f254a53a700 1 --2- 172.21.15.92:0/4052980157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2544066260 0x7f25440ffa00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2534004940 tx=0x7f2534009e30 comp rx=0 tx=0).stop 2024-05-06T12:23:43.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.463+0000 7f254a53a700 1 -- 172.21.15.92:0/4052980157 shutdown_connections 2024-05-06T12:23:43.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.463+0000 7f254a53a700 1 --2- 172.21.15.92:0/4052980157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25440fff40 0x7f2544102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:43.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.463+0000 7f254a53a700 1 --2- 172.21.15.92:0/4052980157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2544066260 0x7f25440ffa00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:43.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.463+0000 7f254a53a700 1 -- 172.21.15.92:0/4052980157 >> 172.21.15.92:0/4052980157 conn(0x7f25440f7430 msgr2=0x7f25440f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:23:43.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.463+0000 7f254a53a700 1 -- 172.21.15.92:0/4052980157 shutdown_connections 2024-05-06T12:23:43.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.463+0000 7f254a53a700 1 -- 172.21.15.92:0/4052980157 wait complete. 2024-05-06T12:23:43.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.464+0000 7f254a53a700 1 Processor -- start 2024-05-06T12:23:43.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.464+0000 7f254a53a700 1 -- start start 2024-05-06T12:23:43.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.464+0000 7f254a53a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2544066260 0x7f254410fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:43.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.464+0000 7f254a53a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25440fff40 0x7f2544110250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:43.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.464+0000 7f254a53a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2544110850 con 0x7f25440fff40 2024-05-06T12:23:43.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.464+0000 7f254a53a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2544110f90 con 0x7f2544066260 2024-05-06T12:23:43.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.464+0000 7f25437fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25440fff40 0x7f2544110250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:43.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.464+0000 7f25437fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25440fff40 0x7f2544110250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51282/0 (socket says 172.21.15.92:51282) 2024-05-06T12:23:43.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.464+0000 7f25437fe700 1 -- 172.21.15.92:0/2246228214 learned_addr learned my addr 172.21.15.92:0/2246228214 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:23:43.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.465+0000 7f2543fff700 1 --2- 172.21.15.92:0/2246228214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2544066260 0x7f254410fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:43.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.465+0000 7f25437fe700 1 -- 172.21.15.92:0/2246228214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2544066260 msgr2=0x7f254410fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:43.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.465+0000 7f25437fe700 1 --2- 172.21.15.92:0/2246228214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2544066260 0x7f254410fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:43.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.465+0000 7f25437fe700 1 -- 172.21.15.92:0/2246228214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2534005040 con 0x7f25440fff40 2024-05-06T12:23:43.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.465+0000 7f25437fe700 1 --2- 172.21.15.92:0/2246228214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25440fff40 0x7f2544110250 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2538002a50 tx=0x7f25380046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:23:43.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.465+0000 7f2540ff9700 1 -- 172.21.15.92:0/2246228214 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2538007cc0 con 0x7f25440fff40 2024-05-06T12:23:43.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.465+0000 7f2540ff9700 1 -- 172.21.15.92:0/2246228214 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f253800f040 con 0x7f25440fff40 2024-05-06T12:23:43.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.465+0000 7f254a53a700 1 -- 172.21.15.92:0/2246228214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2544111270 con 0x7f25440fff40 2024-05-06T12:23:43.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.465+0000 7f2540ff9700 1 -- 172.21.15.92:0/2246228214 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2538009640 con 0x7f25440fff40 2024-05-06T12:23:43.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.465+0000 7f254a53a700 1 -- 172.21.15.92:0/2246228214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2544111830 con 0x7f25440fff40 2024-05-06T12:23:43.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.466+0000 7f254a53a700 1 -- 172.21.15.92:0/2246228214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25440f8f60 con 0x7f25440fff40 2024-05-06T12:23:43.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.467+0000 7f2540ff9700 1 -- 172.21.15.92:0/2246228214 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f25380097a0 con 0x7f25440fff40 2024-05-06T12:23:43.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.467+0000 7f2540ff9700 1 --2- 172.21.15.92:0/2246228214 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f252c06e140 0x7f252c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:43.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.467+0000 7f2540ff9700 1 -- 172.21.15.92:0/2246228214 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f253808d990 con 0x7f25440fff40 2024-05-06T12:23:43.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.467+0000 7f2543fff700 1 --2- 172.21.15.92:0/2246228214 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f252c06e140 0x7f252c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:43.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.468+0000 7f2543fff700 1 --2- 172.21.15.92:0/2246228214 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f252c06e140 0x7f252c0705f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f253400b970 tx=0x7f2534009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:23:43.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.472+0000 7f2540ff9700 1 -- 172.21.15.92:0/2246228214 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f253805a7f0 con 0x7f25440fff40 2024-05-06T12:23:43.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.769+0000 7f254a53a700 1 -- 172.21.15.92:0/2246228214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f254405f5c0 con 0x7f25440fff40 2024-05-06T12:23:43.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.770+0000 7f2540ff9700 1 -- 172.21.15.92:0/2246228214 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f253805a380 con 0x7f25440fff40 2024-05-06T12:23:43.772 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:23:43.772 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": { 2024-05-06T12:23:43.772 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:23:43.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:23:43.772 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": { 2024-05-06T12:23:43.773 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:23:43.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:23:43.773 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": { 2024-05-06T12:23:43.773 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 6 2024-05-06T12:23:43.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:23:43.773 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-05-06T12:23:43.773 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 4 2024-05-06T12:23:43.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:23:43.773 INFO:teuthology.orchestra.run.smithi092.stdout: "overall": { 2024-05-06T12:23:43.773 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 14 2024-05-06T12:23:43.773 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-06T12:23:43.773 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:23:43.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.772+0000 7f254a53a700 1 -- 172.21.15.92:0/2246228214 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f252c06e140 msgr2=0x7f252c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:43.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.772+0000 7f254a53a700 1 --2- 172.21.15.92:0/2246228214 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f252c06e140 0x7f252c0705f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f253400b970 tx=0x7f2534009e30 comp rx=0 tx=0).stop 2024-05-06T12:23:43.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.772+0000 7f254a53a700 1 -- 172.21.15.92:0/2246228214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25440fff40 msgr2=0x7f2544110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:43.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.772+0000 7f254a53a700 1 --2- 172.21.15.92:0/2246228214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25440fff40 0x7f2544110250 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2538002a50 tx=0x7f25380046b0 comp rx=0 tx=0).stop 2024-05-06T12:23:43.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.772+0000 7f254a53a700 1 -- 172.21.15.92:0/2246228214 shutdown_connections 2024-05-06T12:23:43.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.772+0000 7f254a53a700 1 --2- 172.21.15.92:0/2246228214 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f252c06e140 0x7f252c0705f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:43.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.772+0000 7f254a53a700 1 --2- 172.21.15.92:0/2246228214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2544066260 0x7f254410fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:43.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.772+0000 7f254a53a700 1 --2- 172.21.15.92:0/2246228214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25440fff40 0x7f2544110250 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:43.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.772+0000 7f254a53a700 1 -- 172.21.15.92:0/2246228214 >> 172.21.15.92:0/2246228214 conn(0x7f25440f7430 msgr2=0x7f25440fbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:23:43.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.772+0000 7f254a53a700 1 -- 172.21.15.92:0/2246228214 shutdown_connections 2024-05-06T12:23:43.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.772+0000 7f254a53a700 1 -- 172.21.15.92:0/2246228214 wait complete. 2024-05-06T12:23:43.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.878+0000 7f488435f700 1 -- 172.21.15.92:0/1786033638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f487c0662c0 msgr2=0x7f487c0ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:43.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.878+0000 7f488435f700 1 --2- 172.21.15.92:0/1786033638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f487c0662c0 0x7f487c0ff9a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4878004940 tx=0x7f4878009e30 comp rx=0 tx=0).stop 2024-05-06T12:23:43.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.878+0000 7f488435f700 1 -- 172.21.15.92:0/1786033638 shutdown_connections 2024-05-06T12:23:43.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.878+0000 7f488435f700 1 --2- 172.21.15.92:0/1786033638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f487c0ffee0 0x7f487c102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:43.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.878+0000 7f488435f700 1 --2- 172.21.15.92:0/1786033638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f487c0662c0 0x7f487c0ff9a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:43.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.878+0000 7f488435f700 1 -- 172.21.15.92:0/1786033638 >> 172.21.15.92:0/1786033638 conn(0x7f487c0f7430 msgr2=0x7f487c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:23:43.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.878+0000 7f488435f700 1 -- 172.21.15.92:0/1786033638 shutdown_connections 2024-05-06T12:23:43.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.878+0000 7f488435f700 1 -- 172.21.15.92:0/1786033638 wait complete. 2024-05-06T12:23:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.879+0000 7f488435f700 1 Processor -- start 2024-05-06T12:23:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.879+0000 7f488435f700 1 -- start start 2024-05-06T12:23:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.879+0000 7f488435f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f487c0662c0 0x7f487c10da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.879+0000 7f488435f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f487c0ffee0 0x7f487c10df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.879+0000 7f488435f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c10e560 con 0x7f487c0ffee0 2024-05-06T12:23:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.879+0000 7f488435f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c10edb0 con 0x7f487c0662c0 2024-05-06T12:23:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.879+0000 7f48818fa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f487c0ffee0 0x7f487c10df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.879+0000 7f48818fa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f487c0ffee0 0x7f487c10df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51298/0 (socket says 172.21.15.92:51298) 2024-05-06T12:23:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.879+0000 7f48818fa700 1 -- 172.21.15.92:0/435767573 learned_addr learned my addr 172.21.15.92:0/435767573 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:23:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.879+0000 7f48820fb700 1 --2- 172.21.15.92:0/435767573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f487c0662c0 0x7f487c10da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.880+0000 7f48818fa700 1 -- 172.21.15.92:0/435767573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f487c0662c0 msgr2=0x7f487c10da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:43.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.880+0000 7f48818fa700 1 --2- 172.21.15.92:0/435767573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f487c0662c0 0x7f487c10da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:43.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.880+0000 7f48818fa700 1 -- 172.21.15.92:0/435767573 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4878005040 con 0x7f487c0ffee0 2024-05-06T12:23:43.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.880+0000 7f48818fa700 1 --2- 172.21.15.92:0/435767573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f487c0ffee0 0x7f487c10df60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f486c002a50 tx=0x7f486c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:23:43.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.880+0000 7f4872ffd700 1 -- 172.21.15.92:0/435767573 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f486c007cc0 con 0x7f487c0ffee0 2024-05-06T12:23:43.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.880+0000 7f488435f700 1 -- 172.21.15.92:0/435767573 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f487c10f090 con 0x7f487c0ffee0 2024-05-06T12:23:43.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.880+0000 7f4872ffd700 1 -- 172.21.15.92:0/435767573 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f486c00f040 con 0x7f487c0ffee0 2024-05-06T12:23:43.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.880+0000 7f4872ffd700 1 -- 172.21.15.92:0/435767573 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f486c00da30 con 0x7f487c0ffee0 2024-05-06T12:23:43.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.880+0000 7f488435f700 1 -- 172.21.15.92:0/435767573 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f487c10f5d0 con 0x7f487c0ffee0 2024-05-06T12:23:43.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.881+0000 7f488435f700 1 -- 172.21.15.92:0/435767573 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f487c05f5c0 con 0x7f487c0ffee0 2024-05-06T12:23:43.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.882+0000 7f4872ffd700 1 -- 172.21.15.92:0/435767573 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f486c00db90 con 0x7f487c0ffee0 2024-05-06T12:23:43.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.883+0000 7f4872ffd700 1 --2- 172.21.15.92:0/435767573 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f486806e140 0x7f48680705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:43.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.883+0000 7f4872ffd700 1 -- 172.21.15.92:0/435767573 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f486c08d910 con 0x7f487c0ffee0 2024-05-06T12:23:43.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.883+0000 7f48820fb700 1 --2- 172.21.15.92:0/435767573 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f486806e140 0x7f48680705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:43.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.883+0000 7f48820fb700 1 --2- 172.21.15.92:0/435767573 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f486806e140 0x7f48680705f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4878004890 tx=0x7f4878009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:23:43.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:43.888+0000 7f4872ffd700 1 -- 172.21.15.92:0/435767573 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f486c05a770 con 0x7f487c0ffee0 2024-05-06T12:23:44.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.139+0000 7f488435f700 1 -- 172.21.15.92:0/435767573 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f487c10f950 con 0x7f487c0ffee0 2024-05-06T12:23:44.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.140+0000 7f4872ffd700 1 -- 172.21.15.92:0/435767573 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 42 v42) v1 ==== 76+0+1967 (secure 0 0 0) 0x7f486c05a300 con 0x7f487c0ffee0 2024-05-06T12:23:44.143 INFO:teuthology.orchestra.run.smithi092.stdout:e42 2024-05-06T12:23:44.143 INFO:teuthology.orchestra.run.smithi092.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-06T12:23:44.143 INFO:teuthology.orchestra.run.smithi092.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:23:44.143 INFO:teuthology.orchestra.run.smithi092.stdout:legacy client fscid: 1 2024-05-06T12:23:44.143 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:23:44.143 INFO:teuthology.orchestra.run.smithi092.stdout:Filesystem 'cephfs' (1) 2024-05-06T12:23:44.143 INFO:teuthology.orchestra.run.smithi092.stdout:fs_name cephfs 2024-05-06T12:23:44.143 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 42 2024-05-06T12:23:44.143 INFO:teuthology.orchestra.run.smithi092.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-06T12:23:44.143 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-05-06T12:11:44.952671+0000 2024-05-06T12:23:44.143 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2024-05-06T12:22:50.526300+0000 2024-05-06T12:23:44.143 INFO:teuthology.orchestra.run.smithi092.stdout:tableserver 0 2024-05-06T12:23:44.143 INFO:teuthology.orchestra.run.smithi092.stdout:root 0 2024-05-06T12:23:44.144 INFO:teuthology.orchestra.run.smithi092.stdout:session_timeout 60 2024-05-06T12:23:44.144 INFO:teuthology.orchestra.run.smithi092.stdout:session_autoclose 300 2024-05-06T12:23:44.144 INFO:teuthology.orchestra.run.smithi092.stdout:max_file_size 1099511627776 2024-05-06T12:23:44.144 INFO:teuthology.orchestra.run.smithi092.stdout:max_xattr_size 65536 2024-05-06T12:23:44.144 INFO:teuthology.orchestra.run.smithi092.stdout:required_client_features {} 2024-05-06T12:23:44.144 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure 0 2024-05-06T12:23:44.144 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure_osd_epoch 84 2024-05-06T12:23:44.144 INFO:teuthology.orchestra.run.smithi092.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-06T12:23:44.144 INFO:teuthology.orchestra.run.smithi092.stdout:max_mds 2 2024-05-06T12:23:44.144 INFO:teuthology.orchestra.run.smithi092.stdout:in 0,1 2024-05-06T12:23:44.144 INFO:teuthology.orchestra.run.smithi092.stdout:up {0=34438,1=34466} 2024-05-06T12:23:44.144 INFO:teuthology.orchestra.run.smithi092.stdout:failed 2024-05-06T12:23:44.144 INFO:teuthology.orchestra.run.smithi092.stdout:damaged 2024-05-06T12:23:44.144 INFO:teuthology.orchestra.run.smithi092.stdout:stopped 2024-05-06T12:23:44.144 INFO:teuthology.orchestra.run.smithi092.stdout:data_pools [3] 2024-05-06T12:23:44.145 INFO:teuthology.orchestra.run.smithi092.stdout:metadata_pool 2 2024-05-06T12:23:44.145 INFO:teuthology.orchestra.run.smithi092.stdout:inline_data enabled 2024-05-06T12:23:44.145 INFO:teuthology.orchestra.run.smithi092.stdout:balancer 2024-05-06T12:23:44.145 INFO:teuthology.orchestra.run.smithi092.stdout:bal_rank_mask -1 2024-05-06T12:23:44.145 INFO:teuthology.orchestra.run.smithi092.stdout:standby_count_wanted 1 2024-05-06T12:23:44.145 INFO:teuthology.orchestra.run.smithi092.stdout:qdb_cluster leader: 34438 members: 34466,34438 2024-05-06T12:23:44.145 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.ijsqgp{0:34438} state up:active seq 8 join_fscid=1 addr [v2:172.21.15.92:6826/4190573007,v1:172.21.15.92:6827/4190573007] compat {c=[1],r=[1],i=[1fff]}] 2024-05-06T12:23:44.145 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.byjvgf{0:44309} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.190:6824/1219274291,v1:172.21.15.190:6825/1219274291] compat {c=[1],r=[1],i=[1fff]}] 2024-05-06T12:23:44.145 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.rosbll{1:34466} state up:active seq 12 join_fscid=1 addr [v2:172.21.15.92:6828/2398082173,v1:172.21.15.92:6829/2398082173] compat {c=[1],r=[1],i=[1fff]}] 2024-05-06T12:23:44.145 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.idzoff{1:44329} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.190:6826/2486629325,v1:172.21.15.190:6827/2486629325] compat {c=[1],r=[1],i=[1fff]}] 2024-05-06T12:23:44.145 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:23:44.145 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:23:44.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.142+0000 7f488435f700 1 -- 172.21.15.92:0/435767573 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f486806e140 msgr2=0x7f48680705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:44.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.142+0000 7f488435f700 1 --2- 172.21.15.92:0/435767573 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f486806e140 0x7f48680705f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4878004890 tx=0x7f4878009e30 comp rx=0 tx=0).stop 2024-05-06T12:23:44.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.142+0000 7f488435f700 1 -- 172.21.15.92:0/435767573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f487c0ffee0 msgr2=0x7f487c10df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:44.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.142+0000 7f488435f700 1 --2- 172.21.15.92:0/435767573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f487c0ffee0 0x7f487c10df60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f486c002a50 tx=0x7f486c0046b0 comp rx=0 tx=0).stop 2024-05-06T12:23:44.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.142+0000 7f488435f700 1 -- 172.21.15.92:0/435767573 shutdown_connections 2024-05-06T12:23:44.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.142+0000 7f488435f700 1 --2- 172.21.15.92:0/435767573 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f486806e140 0x7f48680705f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:44.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.142+0000 7f488435f700 1 --2- 172.21.15.92:0/435767573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f487c0662c0 0x7f487c10da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:44.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.142+0000 7f488435f700 1 --2- 172.21.15.92:0/435767573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f487c0ffee0 0x7f487c10df60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:44.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.142+0000 7f488435f700 1 -- 172.21.15.92:0/435767573 >> 172.21.15.92:0/435767573 conn(0x7f487c0f7430 msgr2=0x7f487c0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:23:44.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.142+0000 7f488435f700 1 -- 172.21.15.92:0/435767573 shutdown_connections 2024-05-06T12:23:44.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.143+0000 7f488435f700 1 -- 172.21.15.92:0/435767573 wait complete. 2024-05-06T12:23:44.147 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 42 2024-05-06T12:23:44.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:43 smithi092 ceph-mon[91933]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:23:44.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:43 smithi092 ceph-mon[91933]: pgmap v251: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:23:44.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:43 smithi092 ceph-mon[91933]: from='client.34538 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:23:44.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:43 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/2246228214' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:44.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.258+0000 7f8667ee8700 1 -- 172.21.15.92:0/3756274113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8660066af0 msgr2=0x7f86600ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:44.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.258+0000 7f8667ee8700 1 --2- 172.21.15.92:0/3756274113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8660066af0 0x7f86600ffa30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8650004940 tx=0x7f8650009e30 comp rx=0 tx=0).stop 2024-05-06T12:23:44.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.258+0000 7f8667ee8700 1 -- 172.21.15.92:0/3756274113 shutdown_connections 2024-05-06T12:23:44.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.258+0000 7f8667ee8700 1 --2- 172.21.15.92:0/3756274113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86600fff70 0x7f8660102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:44.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.258+0000 7f8667ee8700 1 --2- 172.21.15.92:0/3756274113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8660066af0 0x7f86600ffa30 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:44.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.258+0000 7f8667ee8700 1 -- 172.21.15.92:0/3756274113 >> 172.21.15.92:0/3756274113 conn(0x7f86600f7440 msgr2=0x7f86600f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:23:44.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.258+0000 7f8667ee8700 1 -- 172.21.15.92:0/3756274113 shutdown_connections 2024-05-06T12:23:44.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.258+0000 7f8667ee8700 1 -- 172.21.15.92:0/3756274113 wait complete. 2024-05-06T12:23:44.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.259+0000 7f8667ee8700 1 Processor -- start 2024-05-06T12:23:44.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.259+0000 7f8667ee8700 1 -- start start 2024-05-06T12:23:44.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.259+0000 7f8667ee8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8660066af0 0x7f86600fd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:44.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.259+0000 7f8667ee8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86600fff70 0x7f86600fb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:44.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.259+0000 7f8667ee8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86600fbf70 con 0x7f8660066af0 2024-05-06T12:23:44.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.259+0000 7f8667ee8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86600fc0e0 con 0x7f86600fff70 2024-05-06T12:23:44.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.259+0000 7f8665c84700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8660066af0 0x7f86600fd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:44.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.259+0000 7f8665c84700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8660066af0 0x7f86600fd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51306/0 (socket says 172.21.15.92:51306) 2024-05-06T12:23:44.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.259+0000 7f8665c84700 1 -- 172.21.15.92:0/1175864646 learned_addr learned my addr 172.21.15.92:0/1175864646 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:23:44.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.259+0000 7f8665483700 1 --2- 172.21.15.92:0/1175864646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86600fff70 0x7f86600fb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:44.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.259+0000 7f8665c84700 1 -- 172.21.15.92:0/1175864646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86600fff70 msgr2=0x7f86600fb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:44.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.260+0000 7f8665c84700 1 --2- 172.21.15.92:0/1175864646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86600fff70 0x7f86600fb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:44.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.260+0000 7f8665c84700 1 -- 172.21.15.92:0/1175864646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8650005040 con 0x7f8660066af0 2024-05-06T12:23:44.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.260+0000 7f8665c84700 1 --2- 172.21.15.92:0/1175864646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8660066af0 0x7f86600fd460 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f865000b3b0 tx=0x7f8650009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:23:44.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.260+0000 7f86567fc700 1 -- 172.21.15.92:0/1175864646 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f865000e040 con 0x7f8660066af0 2024-05-06T12:23:44.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.260+0000 7f86567fc700 1 -- 172.21.15.92:0/1175864646 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f865000bcc0 con 0x7f8660066af0 2024-05-06T12:23:44.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.260+0000 7f86567fc700 1 -- 172.21.15.92:0/1175864646 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f865001c630 con 0x7f8660066af0 2024-05-06T12:23:44.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.260+0000 7f8667ee8700 1 -- 172.21.15.92:0/1175864646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86600fc360 con 0x7f8660066af0 2024-05-06T12:23:44.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.260+0000 7f8667ee8700 1 -- 172.21.15.92:0/1175864646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8660115a90 con 0x7f8660066af0 2024-05-06T12:23:44.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.261+0000 7f8667ee8700 1 -- 172.21.15.92:0/1175864646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f866005f5c0 con 0x7f8660066af0 2024-05-06T12:23:44.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.262+0000 7f86567fc700 1 -- 172.21.15.92:0/1175864646 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f86500139f0 con 0x7f8660066af0 2024-05-06T12:23:44.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.262+0000 7f86567fc700 1 --2- 172.21.15.92:0/1175864646 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f864c06e210 0x7f864c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:44.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.262+0000 7f86567fc700 1 -- 172.21.15.92:0/1175864646 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f865008e740 con 0x7f8660066af0 2024-05-06T12:23:44.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.262+0000 7f8665483700 1 --2- 172.21.15.92:0/1175864646 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f864c06e210 0x7f864c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:44.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.263+0000 7f8665483700 1 --2- 172.21.15.92:0/1175864646 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f864c06e210 0x7f864c0706c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f865c009b10 tx=0x7f865c008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:23:44.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.267+0000 7f86567fc700 1 -- 172.21.15.92:0/1175864646 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f8650057bb0 con 0x7f8660066af0 2024-05-06T12:23:44.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:43 smithi190 ceph-mon[79744]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:23:44.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:43 smithi190 ceph-mon[79744]: pgmap v251: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:23:44.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:43 smithi190 ceph-mon[79744]: from='client.34538 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:23:44.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:43 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/2246228214' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:23:44.492 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.489+0000 7f8667ee8700 1 -- 172.21.15.92:0/1175864646 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f86600f8f80 con 0x7f864c06e210 2024-05-06T12:23:44.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.491+0000 7f86567fc700 1 -- 172.21.15.92:0/1175864646 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+435 (secure 0 0 0) 0x7f86600f8f80 con 0x7f864c06e210 2024-05-06T12:23:44.493 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:23:44.493 INFO:teuthology.orchestra.run.smithi092.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb", 2024-05-06T12:23:44.493 INFO:teuthology.orchestra.run.smithi092.stdout: "in_progress": true, 2024-05-06T12:23:44.493 INFO:teuthology.orchestra.run.smithi092.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-06T12:23:44.493 INFO:teuthology.orchestra.run.smithi092.stdout: "services_complete": [ 2024-05-06T12:23:44.494 INFO:teuthology.orchestra.run.smithi092.stdout: "mds", 2024-05-06T12:23:44.494 INFO:teuthology.orchestra.run.smithi092.stdout: "osd", 2024-05-06T12:23:44.494 INFO:teuthology.orchestra.run.smithi092.stdout: "mon", 2024-05-06T12:23:44.494 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr", 2024-05-06T12:23:44.494 INFO:teuthology.orchestra.run.smithi092.stdout: "crash" 2024-05-06T12:23:44.494 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-05-06T12:23:44.494 INFO:teuthology.orchestra.run.smithi092.stdout: "progress": "16/21 daemons upgraded", 2024-05-06T12:23:44.494 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "Currently upgrading grafana daemons", 2024-05-06T12:23:44.494 INFO:teuthology.orchestra.run.smithi092.stdout: "is_paused": false 2024-05-06T12:23:44.494 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:23:44.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.493+0000 7f8667ee8700 1 -- 172.21.15.92:0/1175864646 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f864c06e210 msgr2=0x7f864c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:44.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.493+0000 7f8667ee8700 1 --2- 172.21.15.92:0/1175864646 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f864c06e210 0x7f864c0706c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f865c009b10 tx=0x7f865c008040 comp rx=0 tx=0).stop 2024-05-06T12:23:44.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.493+0000 7f8667ee8700 1 -- 172.21.15.92:0/1175864646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8660066af0 msgr2=0x7f86600fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:44.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.493+0000 7f8667ee8700 1 --2- 172.21.15.92:0/1175864646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8660066af0 0x7f86600fd460 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f865000b3b0 tx=0x7f8650009e30 comp rx=0 tx=0).stop 2024-05-06T12:23:44.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.493+0000 7f8667ee8700 1 -- 172.21.15.92:0/1175864646 shutdown_connections 2024-05-06T12:23:44.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.493+0000 7f8667ee8700 1 --2- 172.21.15.92:0/1175864646 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f864c06e210 0x7f864c0706c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:44.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.493+0000 7f8667ee8700 1 --2- 172.21.15.92:0/1175864646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8660066af0 0x7f86600fd460 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:44.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.493+0000 7f8667ee8700 1 --2- 172.21.15.92:0/1175864646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86600fff70 0x7f86600fb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:44.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.493+0000 7f8667ee8700 1 -- 172.21.15.92:0/1175864646 >> 172.21.15.92:0/1175864646 conn(0x7f86600f7440 msgr2=0x7f86600f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:23:44.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.493+0000 7f8667ee8700 1 -- 172.21.15.92:0/1175864646 shutdown_connections 2024-05-06T12:23:44.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.493+0000 7f8667ee8700 1 -- 172.21.15.92:0/1175864646 wait complete. 2024-05-06T12:23:44.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.598+0000 7f473440f700 1 -- 172.21.15.92:0/2633821048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f472c0fce80 msgr2=0x7f472c0fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:44.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.598+0000 7f473440f700 1 --2- 172.21.15.92:0/2633821048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f472c0fce80 0x7f472c0fd290 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4724009ce0 tx=0x7f4724004d10 comp rx=0 tx=0).stop 2024-05-06T12:23:44.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.598+0000 7f473440f700 1 -- 172.21.15.92:0/2633821048 shutdown_connections 2024-05-06T12:23:44.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.598+0000 7f473440f700 1 --2- 172.21.15.92:0/2633821048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f472c0fe080 0x7f472c0fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:44.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.598+0000 7f473440f700 1 --2- 172.21.15.92:0/2633821048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f472c0fce80 0x7f472c0fd290 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:44.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.598+0000 7f473440f700 1 -- 172.21.15.92:0/2633821048 >> 172.21.15.92:0/2633821048 conn(0x7f472c0f8590 msgr2=0x7f472c0fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:23:44.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.598+0000 7f473440f700 1 -- 172.21.15.92:0/2633821048 shutdown_connections 2024-05-06T12:23:44.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.598+0000 7f473440f700 1 -- 172.21.15.92:0/2633821048 wait complete. 2024-05-06T12:23:44.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.598+0000 7f473440f700 1 Processor -- start 2024-05-06T12:23:44.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.598+0000 7f473440f700 1 -- start start 2024-05-06T12:23:44.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.599+0000 7f473440f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f472c0fce80 0x7f472c071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:44.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.599+0000 7f473440f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f472c0fe080 0x7f472c0702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:44.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.599+0000 7f473440f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472c070860 con 0x7f472c0fe080 2024-05-06T12:23:44.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.599+0000 7f473440f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472c0709d0 con 0x7f472c0fce80 2024-05-06T12:23:44.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.599+0000 7f47319aa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f472c0fe080 0x7f472c0702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:44.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.599+0000 7f47319aa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f472c0fe080 0x7f472c0702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51320/0 (socket says 172.21.15.92:51320) 2024-05-06T12:23:44.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.599+0000 7f47319aa700 1 -- 172.21.15.92:0/143643959 learned_addr learned my addr 172.21.15.92:0/143643959 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:23:44.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.599+0000 7f47319aa700 1 -- 172.21.15.92:0/143643959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f472c0fce80 msgr2=0x7f472c071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:44.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.599+0000 7f47319aa700 1 --2- 172.21.15.92:0/143643959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f472c0fce80 0x7f472c071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:44.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.599+0000 7f47319aa700 1 -- 172.21.15.92:0/143643959 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4724009940 con 0x7f472c0fe080 2024-05-06T12:23:44.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.599+0000 7f47319aa700 1 --2- 172.21.15.92:0/143643959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f472c0fe080 0x7f472c0702b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f472000c830 tx=0x7f4720004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:23:44.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.599+0000 7f472affd700 1 -- 172.21.15.92:0/143643959 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f472000a760 con 0x7f472c0fe080 2024-05-06T12:23:44.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.599+0000 7f472affd700 1 -- 172.21.15.92:0/143643959 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f472000ad90 con 0x7f472c0fe080 2024-05-06T12:23:44.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.599+0000 7f473440f700 1 -- 172.21.15.92:0/143643959 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f472c070cb0 con 0x7f472c0fe080 2024-05-06T12:23:44.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.599+0000 7f472affd700 1 -- 172.21.15.92:0/143643959 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47200113a0 con 0x7f472c0fe080 2024-05-06T12:23:44.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.599+0000 7f473440f700 1 -- 172.21.15.92:0/143643959 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f472c115c50 con 0x7f472c0fe080 2024-05-06T12:23:44.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.600+0000 7f473440f700 1 -- 172.21.15.92:0/143643959 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f472c05f5c0 con 0x7f472c0fe080 2024-05-06T12:23:44.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.601+0000 7f472affd700 1 -- 172.21.15.92:0/143643959 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4720012400 con 0x7f472c0fe080 2024-05-06T12:23:44.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.601+0000 7f472affd700 1 --2- 172.21.15.92:0/143643959 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f471406e150 0x7f4714070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:23:44.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.601+0000 7f472affd700 1 -- 172.21.15.92:0/143643959 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f472008dbe0 con 0x7f472c0fe080 2024-05-06T12:23:44.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.601+0000 7f47321ab700 1 --2- 172.21.15.92:0/143643959 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f471406e150 0x7f4714070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:23:44.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.602+0000 7f47321ab700 1 --2- 172.21.15.92:0/143643959 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f471406e150 0x7f4714070600 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4724000f80 tx=0x7f4724004d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:23:44.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.606+0000 7f472affd700 1 -- 172.21.15.92:0/143643959 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4720056f90 con 0x7f472c0fe080 2024-05-06T12:23:44.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.901+0000 7f473440f700 1 -- 172.21.15.92:0/143643959 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f472c116000 con 0x7f472c0fe080 2024-05-06T12:23:44.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.902+0000 7f472affd700 1 -- 172.21.15.92:0/143643959 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f472005a650 con 0x7f472c0fe080 2024-05-06T12:23:44.904 INFO:teuthology.orchestra.run.smithi092.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-06T12:23:44.904 INFO:teuthology.orchestra.run.smithi092.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-06T12:23:44.904 INFO:teuthology.orchestra.run.smithi092.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-06T12:23:44.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.904+0000 7f473440f700 1 -- 172.21.15.92:0/143643959 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f471406e150 msgr2=0x7f4714070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:44.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.904+0000 7f473440f700 1 --2- 172.21.15.92:0/143643959 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f471406e150 0x7f4714070600 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4724000f80 tx=0x7f4724004d10 comp rx=0 tx=0).stop 2024-05-06T12:23:44.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.904+0000 7f473440f700 1 -- 172.21.15.92:0/143643959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f472c0fe080 msgr2=0x7f472c0702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:23:44.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.904+0000 7f473440f700 1 --2- 172.21.15.92:0/143643959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f472c0fe080 0x7f472c0702b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f472000c830 tx=0x7f4720004470 comp rx=0 tx=0).stop 2024-05-06T12:23:44.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.904+0000 7f473440f700 1 -- 172.21.15.92:0/143643959 shutdown_connections 2024-05-06T12:23:44.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.904+0000 7f473440f700 1 --2- 172.21.15.92:0/143643959 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f471406e150 0x7f4714070600 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:44.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.904+0000 7f473440f700 1 --2- 172.21.15.92:0/143643959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f472c0fce80 0x7f472c071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:44.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.904+0000 7f473440f700 1 --2- 172.21.15.92:0/143643959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f472c0fe080 0x7f472c0702b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:23:44.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.904+0000 7f473440f700 1 -- 172.21.15.92:0/143643959 >> 172.21.15.92:0/143643959 conn(0x7f472c0f8590 msgr2=0x7f472c0f9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:23:44.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.905+0000 7f473440f700 1 -- 172.21.15.92:0/143643959 shutdown_connections 2024-05-06T12:23:44.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:23:44.905+0000 7f473440f700 1 -- 172.21.15.92:0/143643959 wait complete. 2024-05-06T12:23:45.170 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:44 smithi092 ceph-mon[91933]: from='client.34542 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:23:45.171 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:44 smithi092 ceph-mon[91933]: Upgrade: Updating grafana.smithi092 2024-05-06T12:23:45.171 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:44 smithi092 ceph-mon[91933]: Deploying daemon grafana.smithi092 on smithi092 2024-05-06T12:23:45.171 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:44 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/435767573' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:23:45.171 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:44 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/143643959' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:23:45.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:44 smithi190 ceph-mon[79744]: from='client.34542 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:23:45.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:44 smithi190 ceph-mon[79744]: Upgrade: Updating grafana.smithi092 2024-05-06T12:23:45.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:44 smithi190 ceph-mon[79744]: Deploying daemon grafana.smithi092 on smithi092 2024-05-06T12:23:45.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:44 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/435767573' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:23:45.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:44 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/143643959' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:23:46.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:45 smithi092 ceph-mon[91933]: from='client.34554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:23:46.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:45 smithi092 ceph-mon[91933]: pgmap v252: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:23:46.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:45 smithi190 ceph-mon[79744]: from='client.34554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:23:46.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:45 smithi190 ceph-mon[79744]: pgmap v252: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:23:48.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:47 smithi092 ceph-mon[91933]: pgmap v253: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:23:48.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:47 smithi190 ceph-mon[79744]: pgmap v253: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:23:50.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:49 smithi092 ceph-mon[91933]: pgmap v254: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:23:50.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:49 smithi190 ceph-mon[79744]: pgmap v254: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:23:52.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:51 smithi092 ceph-mon[91933]: pgmap v255: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:23:52.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:51 smithi190 ceph-mon[79744]: pgmap v255: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:23:54.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:53 smithi092 ceph-mon[91933]: pgmap v256: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:23:54.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:53 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:23:54.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:53 smithi190 ceph-mon[79744]: pgmap v256: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:23:54.351 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:53 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:23:56.305 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:55 smithi092 ceph-mon[91933]: pgmap v257: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:23:56.305 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:55 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:56.305 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:55 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:56.306 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:55 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:23:56.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:55 smithi190 ceph-mon[79744]: pgmap v257: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:23:56.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:55 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:56.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:55 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:23:56.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:55 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:23:58.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:57 smithi092 ceph-mon[91933]: pgmap v258: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:23:58.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:57 smithi190 ceph-mon[79744]: pgmap v258: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:24:00.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:59 smithi190 ceph-mon[79744]: pgmap v259: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:24:00.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:59 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:24:00.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:23:59 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:24:00.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:59 smithi092 ceph-mon[91933]: pgmap v259: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:24:00.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:59 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:24:00.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:23:59 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:24:02.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:01 smithi092 ceph-mon[91933]: pgmap v260: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:24:02.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:01 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:24:02.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:01 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:24:02.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:01 smithi190 ceph-mon[79744]: pgmap v260: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:24:02.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:01 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:24:02.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:01 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:24:04.204 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:03 smithi092 ceph-mon[91933]: pgmap v261: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:24:04.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:03 smithi190 ceph-mon[79744]: pgmap v261: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:24:06.195 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:05 smithi092 ceph-mon[91933]: pgmap v262: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:24:06.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:05 smithi190 ceph-mon[79744]: pgmap v262: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:24:08.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:24:08.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:24:08.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:24:08.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: pgmap v263: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:24:08.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:24:08.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:24:08.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-06T12:24:08.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-06T12:24:08.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi092:3000"}]: dispatch 2024-05-06T12:24:08.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi092:3000"}]: dispatch 2024-05-06T12:24:08.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:24:08.223 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:24:08.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.224 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: Upgrade: Finalizing container_image settings 2024-05-06T12:24:08.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:24:08.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-05-06T12:24:08.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-05-06T12:24:08.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-06T12:24:08.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-05-06T12:24:08.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-05-06T12:24:08.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-05-06T12:24:08.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-05-06T12:24:08.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-05-06T12:24:08.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-05-06T12:24:08.225 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-05-06T12:24:08.226 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-05-06T12:24:08.226 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-05-06T12:24:08.226 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-05-06T12:24:08.226 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-05-06T12:24:08.226 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-06T12:24:08.226 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-05-06T12:24:08.226 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-05-06T12:24:08.226 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-05-06T12:24:08.226 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-05-06T12:24:08.226 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-05-06T12:24:08.226 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-05-06T12:24:08.226 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-05-06T12:24:08.226 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-05-06T12:24:08.226 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-06T12:24:08.227 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-06T12:24:08.227 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-06T12:24:08.227 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-06T12:24:08.227 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-06T12:24:08.227 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-06T12:24:08.227 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-05-06T12:24:08.227 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-05-06T12:24:08.227 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:24:08.227 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:24:08.227 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:24:08.227 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:24:08.227 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:24:08.227 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:24:08.227 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:24:08.227 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:07 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:24:08.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:24:08.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:24:08.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:24:08.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: pgmap v263: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:24:08.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:24:08.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:24:08.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-06T12:24:08.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-06T12:24:08.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi092:3000"}]: dispatch 2024-05-06T12:24:08.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi092:3000"}]: dispatch 2024-05-06T12:24:08.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:24:08.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:24:08.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:08.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: Upgrade: Finalizing container_image settings 2024-05-06T12:24:08.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:24:08.351 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-05-06T12:24:08.351 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-05-06T12:24:08.351 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-06T12:24:08.351 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-05-06T12:24:08.351 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-05-06T12:24:08.351 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-05-06T12:24:08.351 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-05-06T12:24:08.351 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-05-06T12:24:08.351 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-05-06T12:24:08.351 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-05-06T12:24:08.351 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-05-06T12:24:08.351 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-05-06T12:24:08.351 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-05-06T12:24:08.351 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-05-06T12:24:08.352 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-06T12:24:08.352 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-05-06T12:24:08.352 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-05-06T12:24:08.352 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-05-06T12:24:08.352 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-05-06T12:24:08.352 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-05-06T12:24:08.352 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-05-06T12:24:08.352 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-05-06T12:24:08.352 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-05-06T12:24:08.352 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-06T12:24:08.352 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-06T12:24:08.352 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-06T12:24:08.352 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-06T12:24:08.352 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-06T12:24:08.353 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-06T12:24:08.353 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-05-06T12:24:08.353 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-05-06T12:24:08.353 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:24:08.353 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:24:08.353 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:24:08.353 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:24:08.353 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:24:08.353 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:24:08.353 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:24:08.353 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:07 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:24:09.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:08 smithi092 ceph-mon[91933]: Upgrade: Complete! 2024-05-06T12:24:09.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:08 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:24:09.249 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:08 smithi190 ceph-mon[79744]: Upgrade: Complete! 2024-05-06T12:24:09.249 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:08 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:24:10.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:09 smithi092 ceph-mon[91933]: pgmap v264: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:24:10.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:24:10.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:09 smithi190 ceph-mon[79744]: pgmap v264: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:24:10.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:24:12.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:11 smithi092 ceph-mon[91933]: pgmap v265: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:24:12.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:11 smithi190 ceph-mon[79744]: pgmap v265: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:24:14.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:13 smithi092 ceph-mon[91933]: pgmap v266: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:24:14.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:13 smithi190 ceph-mon[79744]: pgmap v266: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:24:15.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.023+0000 7f30a1556700 1 -- 172.21.15.92:0/417976703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f309c0fce60 msgr2=0x7f309c0fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:24:15.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.023+0000 7f30a1556700 1 --2- 172.21.15.92:0/417976703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f309c0fce60 0x7f309c0fd270 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f308c004940 tx=0x7f308c009e30 comp rx=0 tx=0).stop 2024-05-06T12:24:15.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.023+0000 7f30a1556700 1 -- 172.21.15.92:0/417976703 shutdown_connections 2024-05-06T12:24:15.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.023+0000 7f30a1556700 1 --2- 172.21.15.92:0/417976703 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f309c0fdfd0 0x7f309c0fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:24:15.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.023+0000 7f30a1556700 1 --2- 172.21.15.92:0/417976703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f309c0fce60 0x7f309c0fd270 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:24:15.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.023+0000 7f30a1556700 1 -- 172.21.15.92:0/417976703 >> 172.21.15.92:0/417976703 conn(0x7f309c0f8540 msgr2=0x7f309c0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:24:15.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.023+0000 7f30a1556700 1 -- 172.21.15.92:0/417976703 shutdown_connections 2024-05-06T12:24:15.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.023+0000 7f30a1556700 1 -- 172.21.15.92:0/417976703 wait complete. 2024-05-06T12:24:15.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.023+0000 7f30a1556700 1 Processor -- start 2024-05-06T12:24:15.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.023+0000 7f30a1556700 1 -- start start 2024-05-06T12:24:15.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.024+0000 7f30a1556700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f309c0fce60 0x7f309c071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:24:15.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.024+0000 7f30a1556700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f309c0fdfd0 0x7f309c0702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:24:15.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.024+0000 7f30a1556700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c072200 con 0x7f309c0fce60 2024-05-06T12:24:15.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.024+0000 7f30a1556700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c0707f0 con 0x7f309c0fdfd0 2024-05-06T12:24:15.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.024+0000 7f309affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f309c0fce60 0x7f309c071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:24:15.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.024+0000 7f309affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f309c0fce60 0x7f309c071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58082/0 (socket says 172.21.15.92:58082) 2024-05-06T12:24:15.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.024+0000 7f309affd700 1 -- 172.21.15.92:0/1636078029 learned_addr learned my addr 172.21.15.92:0/1636078029 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:24:15.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.024+0000 7f309a7fc700 1 --2- 172.21.15.92:0/1636078029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f309c0fdfd0 0x7f309c0702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:24:15.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.024+0000 7f309affd700 1 -- 172.21.15.92:0/1636078029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f309c0fdfd0 msgr2=0x7f309c0702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:24:15.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.024+0000 7f309affd700 1 --2- 172.21.15.92:0/1636078029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f309c0fdfd0 0x7f309c0702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:24:15.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.024+0000 7f309affd700 1 -- 172.21.15.92:0/1636078029 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308c005040 con 0x7f309c0fce60 2024-05-06T12:24:15.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.024+0000 7f309affd700 1 --2- 172.21.15.92:0/1636078029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f309c0fce60 0x7f309c071c00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f308c009150 tx=0x7f308c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:24:15.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.024+0000 7f30837fe700 1 -- 172.21.15.92:0/1636078029 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f308c00e040 con 0x7f309c0fce60 2024-05-06T12:24:15.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.024+0000 7f30837fe700 1 -- 172.21.15.92:0/1636078029 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f308c004290 con 0x7f309c0fce60 2024-05-06T12:24:15.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.024+0000 7f30837fe700 1 -- 172.21.15.92:0/1636078029 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f308c01c630 con 0x7f309c0fce60 2024-05-06T12:24:15.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.024+0000 7f30a1556700 1 -- 172.21.15.92:0/1636078029 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f309c070a40 con 0x7f309c0fce60 2024-05-06T12:24:15.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.025+0000 7f30a1556700 1 -- 172.21.15.92:0/1636078029 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f309c071000 con 0x7f309c0fce60 2024-05-06T12:24:15.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.025+0000 7f30a1556700 1 -- 172.21.15.92:0/1636078029 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f309c04c710 con 0x7f309c0fce60 2024-05-06T12:24:15.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.026+0000 7f30837fe700 1 -- 172.21.15.92:0/1636078029 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f308c00bbf0 con 0x7f309c0fce60 2024-05-06T12:24:15.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.026+0000 7f30837fe700 1 --2- 172.21.15.92:0/1636078029 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f308406e140 0x7f30840705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:24:15.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.026+0000 7f30837fe700 1 -- 172.21.15.92:0/1636078029 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f308c08e750 con 0x7f309c0fce60 2024-05-06T12:24:15.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.027+0000 7f309a7fc700 1 --2- 172.21.15.92:0/1636078029 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f308406e140 0x7f30840705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:24:15.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.027+0000 7f309a7fc700 1 --2- 172.21.15.92:0/1636078029 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f308406e140 0x7f30840705f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3090009fd0 tx=0x7f309000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:24:15.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.031+0000 7f30837fe700 1 -- 172.21.15.92:0/1636078029 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f308c05b630 con 0x7f309c0fce60 2024-05-06T12:24:15.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.246+0000 7f30a1556700 1 -- 172.21.15.92:0/1636078029 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f309c1022a0 con 0x7f308406e140 2024-05-06T12:24:15.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.247+0000 7f30837fe700 1 -- 172.21.15.92:0/1636078029 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f309c1022a0 con 0x7f308406e140 2024-05-06T12:24:15.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.249+0000 7f30a1556700 1 -- 172.21.15.92:0/1636078029 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f308406e140 msgr2=0x7f30840705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:24:15.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.249+0000 7f30a1556700 1 --2- 172.21.15.92:0/1636078029 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f308406e140 0x7f30840705f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3090009fd0 tx=0x7f309000b040 comp rx=0 tx=0).stop 2024-05-06T12:24:15.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.249+0000 7f30a1556700 1 -- 172.21.15.92:0/1636078029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f309c0fce60 msgr2=0x7f309c071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:24:15.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.249+0000 7f30a1556700 1 --2- 172.21.15.92:0/1636078029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f309c0fce60 0x7f309c071c00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f308c009150 tx=0x7f308c009e30 comp rx=0 tx=0).stop 2024-05-06T12:24:15.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.249+0000 7f30a1556700 1 -- 172.21.15.92:0/1636078029 shutdown_connections 2024-05-06T12:24:15.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.249+0000 7f30a1556700 1 --2- 172.21.15.92:0/1636078029 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f308406e140 0x7f30840705f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:24:15.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.249+0000 7f30a1556700 1 --2- 172.21.15.92:0/1636078029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f309c0fce60 0x7f309c071c00 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:24:15.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.249+0000 7f30a1556700 1 --2- 172.21.15.92:0/1636078029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f309c0fdfd0 0x7f309c0702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:24:15.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.249+0000 7f30a1556700 1 -- 172.21.15.92:0/1636078029 >> 172.21.15.92:0/1636078029 conn(0x7f309c0f8540 msgr2=0x7f309c0f9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:24:15.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.250+0000 7f30a1556700 1 -- 172.21.15.92:0/1636078029 shutdown_connections 2024-05-06T12:24:15.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:15.250+0000 7f30a1556700 1 -- 172.21.15.92:0/1636078029 wait complete. 2024-05-06T12:24:15.969 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -e sha1=70ce9539b62a15b9de429c1f0e91a1e012b099aa -- bash -c 'ceph orch ps' 2024-05-06T12:24:16.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:15 smithi092 ceph-mon[91933]: pgmap v267: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:24:16.264 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:24:16.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:15 smithi190 ceph-mon[79744]: pgmap v267: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:24:17.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:16 smithi092 ceph-mon[91933]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:24:17.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:16 smithi190 ceph-mon[79744]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:24:17.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.893+0000 7f11e7130700 1 -- 172.21.15.92:0/4038866502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11e0102de0 msgr2=0x7f11e01031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:24:17.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.893+0000 7f11e7130700 1 --2- 172.21.15.92:0/4038866502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11e0102de0 0x7f11e01031b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f11d4004940 tx=0x7f11d4009e30 comp rx=0 tx=0).stop 2024-05-06T12:24:17.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.893+0000 7f11e7130700 1 -- 172.21.15.92:0/4038866502 shutdown_connections 2024-05-06T12:24:17.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.893+0000 7f11e7130700 1 --2- 172.21.15.92:0/4038866502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11e0103780 0x7f11e00fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:24:17.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.893+0000 7f11e7130700 1 --2- 172.21.15.92:0/4038866502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11e0102de0 0x7f11e01031b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:24:17.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.893+0000 7f11e7130700 1 -- 172.21.15.92:0/4038866502 >> 172.21.15.92:0/4038866502 conn(0x7f11e00f8990 msgr2=0x7f11e00fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:24:17.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.893+0000 7f11e7130700 1 -- 172.21.15.92:0/4038866502 shutdown_connections 2024-05-06T12:24:17.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.893+0000 7f11e7130700 1 -- 172.21.15.92:0/4038866502 wait complete. 2024-05-06T12:24:17.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.893+0000 7f11e7130700 1 Processor -- start 2024-05-06T12:24:17.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.893+0000 7f11e7130700 1 -- start start 2024-05-06T12:24:17.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.894+0000 7f11e7130700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11e0102de0 0x7f11e0070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:24:17.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.894+0000 7f11e7130700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11e0103780 0x7f11e00707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:24:17.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.894+0000 7f11e7130700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e0072160 con 0x7f11e0102de0 2024-05-06T12:24:17.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.894+0000 7f11e7130700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e0070ce0 con 0x7f11e0103780 2024-05-06T12:24:17.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.894+0000 7f11e4ecc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11e0102de0 0x7f11e0070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:24:17.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.894+0000 7f11e4ecc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11e0102de0 0x7f11e0070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58106/0 (socket says 172.21.15.92:58106) 2024-05-06T12:24:17.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.894+0000 7f11e4ecc700 1 -- 172.21.15.92:0/4216796034 learned_addr learned my addr 172.21.15.92:0/4216796034 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:24:17.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.894+0000 7f11dffff700 1 --2- 172.21.15.92:0/4216796034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11e0103780 0x7f11e00707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:24:17.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.894+0000 7f11e4ecc700 1 -- 172.21.15.92:0/4216796034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11e0103780 msgr2=0x7f11e00707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:24:17.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.894+0000 7f11e4ecc700 1 --2- 172.21.15.92:0/4216796034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11e0103780 0x7f11e00707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:24:17.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.894+0000 7f11e4ecc700 1 -- 172.21.15.92:0/4216796034 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d4005040 con 0x7f11e0102de0 2024-05-06T12:24:17.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.894+0000 7f11e4ecc700 1 --2- 172.21.15.92:0/4216796034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11e0102de0 0x7f11e0070260 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f11d400b3b0 tx=0x7f11d4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:24:17.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.895+0000 7f11dd7fa700 1 -- 172.21.15.92:0/4216796034 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11d400e040 con 0x7f11e0102de0 2024-05-06T12:24:17.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.895+0000 7f11e7130700 1 -- 172.21.15.92:0/4216796034 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e0070f60 con 0x7f11e0102de0 2024-05-06T12:24:17.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.895+0000 7f11dd7fa700 1 -- 172.21.15.92:0/4216796034 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f11d4013420 con 0x7f11e0102de0 2024-05-06T12:24:17.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.895+0000 7f11dd7fa700 1 -- 172.21.15.92:0/4216796034 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11d401c630 con 0x7f11e0102de0 2024-05-06T12:24:17.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.895+0000 7f11e7130700 1 -- 172.21.15.92:0/4216796034 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e011a110 con 0x7f11e0102de0 2024-05-06T12:24:17.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.896+0000 7f11e7130700 1 -- 172.21.15.92:0/4216796034 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11e01000f0 con 0x7f11e0102de0 2024-05-06T12:24:17.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.896+0000 7f11dd7fa700 1 -- 172.21.15.92:0/4216796034 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f11d4013590 con 0x7f11e0102de0 2024-05-06T12:24:17.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.897+0000 7f11dd7fa700 1 --2- 172.21.15.92:0/4216796034 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f11d006e140 0x7f11d00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:24:17.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.897+0000 7f11dffff700 1 --2- 172.21.15.92:0/4216796034 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f11d006e140 0x7f11d00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:24:17.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.897+0000 7f11dd7fa700 1 -- 172.21.15.92:0/4216796034 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f11d408ed40 con 0x7f11e0102de0 2024-05-06T12:24:17.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.897+0000 7f11dffff700 1 --2- 172.21.15.92:0/4216796034 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f11d006e140 0x7f11d00705f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f11e0071940 tx=0x7f11cc00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:24:17.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:17.901+0000 7f11dd7fa700 1 -- 172.21.15.92:0/4216796034 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f11d4092030 con 0x7f11e0102de0 2024-05-06T12:24:18.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:18.109+0000 7f11e7130700 1 -- 172.21.15.92:0/4216796034 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f11e00716f0 con 0x7f11d006e140 2024-05-06T12:24:18.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:18.119+0000 7f11dd7fa700 1 -- 172.21.15.92:0/4216796034 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f11e00716f0 con 0x7f11d006e140 2024-05-06T12:24:18.121 INFO:teuthology.orchestra.run.smithi092.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-06T12:24:18.121 INFO:teuthology.orchestra.run.smithi092.stdout:alertmanager.smithi092 smithi092 *:9093,9094 running (46s) 18s ago 17m 15.1M - 0.25.0 c8568f914cd2 c9a1046648dc 2024-05-06T12:24:18.121 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi092 smithi092 running (6m) 18s ago 18m 6660k - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 8da0494e1bdc 2024-05-06T12:24:18.122 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi190 smithi190 running (6m) 75s ago 17m 6665k - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 641a7cdaa56e 2024-05-06T12:24:18.122 INFO:teuthology.orchestra.run.smithi092.stdout:grafana.smithi092 smithi092 *:3000 running (23s) 18s ago 17m 52.6M - 9.4.12 2bacad6d85d8 55135681ef06 2024-05-06T12:24:18.122 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.ijsqgp smithi092 running (2m) 18s ago 12m 66.7M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 40b4cdd3be40 2024-05-06T12:24:18.122 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.rosbll smithi092 running (2m) 18s ago 12m 17.0M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 550f8238d95f 2024-05-06T12:24:18.122 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.byjvgf smithi190 running (113s) 75s ago 12m 63.4M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 0d26051c908e 2024-05-06T12:24:18.122 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.idzoff smithi190 running (97s) 75s ago 12m 12.8M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 7cfd0097389b 2024-05-06T12:24:18.122 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi092.gyzeqb smithi092 *:8443,9283,8765 running (8m) 18s ago 19m 519M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 de9d2198327a 2024-05-06T12:24:18.122 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi190.kczxkh smithi190 *:8443,9283,8765 running (8m) 75s ago 16m 452M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 e84ab3758ac7 2024-05-06T12:24:18.122 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi092 smithi092 running (7m) 18s ago 19m 56.2M 2048M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 ac07f7158087 2024-05-06T12:24:18.122 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi190 smithi190 running (7m) 75s ago 16m 39.7M 2048M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 7a3c08eb716f 2024-05-06T12:24:18.122 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi092 smithi092 *:9100 running (83s) 18s ago 17m 8292k - 1.5.0 0da6a335fe13 590fa41f0538 2024-05-06T12:24:18.122 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi190 smithi190 *:9100 running (79s) 75s ago 16m 3590k - 1.5.0 0da6a335fe13 a57191dac5b5 2024-05-06T12:24:18.122 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 smithi092 running (6m) 18s ago 16m 94.2M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 8c96c75b25f4 2024-05-06T12:24:18.123 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 smithi092 running (5m) 18s ago 15m 97.4M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 e8007a510f08 2024-05-06T12:24:18.123 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 smithi092 running (5m) 18s ago 15m 91.6M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 2beca7448789 2024-05-06T12:24:18.123 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 smithi190 running (4m) 75s ago 14m 101M 2581M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 914ada199f93 2024-05-06T12:24:18.123 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 smithi190 running (3m) 75s ago 14m 89.4M 2581M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 371c0be776c7 2024-05-06T12:24:18.123 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 smithi190 running (3m) 75s ago 13m 86.8M 2581M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 067bc34cdfb1 2024-05-06T12:24:18.123 INFO:teuthology.orchestra.run.smithi092.stdout:prometheus.smithi092 smithi092 *:9095 running (62s) 18s ago 17m 46.9M - 2.43.0 a07b618ecd1d eadf61684ae7 2024-05-06T12:24:18.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:18.121+0000 7f11e7130700 1 -- 172.21.15.92:0/4216796034 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f11d006e140 msgr2=0x7f11d00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:24:18.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:18.121+0000 7f11e7130700 1 --2- 172.21.15.92:0/4216796034 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f11d006e140 0x7f11d00705f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f11e0071940 tx=0x7f11cc00b040 comp rx=0 tx=0).stop 2024-05-06T12:24:18.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:18.121+0000 7f11e7130700 1 -- 172.21.15.92:0/4216796034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11e0102de0 msgr2=0x7f11e0070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:24:18.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:18.121+0000 7f11e7130700 1 --2- 172.21.15.92:0/4216796034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11e0102de0 0x7f11e0070260 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f11d400b3b0 tx=0x7f11d4009e30 comp rx=0 tx=0).stop 2024-05-06T12:24:18.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:18.121+0000 7f11e7130700 1 -- 172.21.15.92:0/4216796034 shutdown_connections 2024-05-06T12:24:18.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:18.121+0000 7f11e7130700 1 --2- 172.21.15.92:0/4216796034 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f11d006e140 0x7f11d00705f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:24:18.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:18.121+0000 7f11e7130700 1 --2- 172.21.15.92:0/4216796034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11e0102de0 0x7f11e0070260 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:24:18.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:18.121+0000 7f11e7130700 1 --2- 172.21.15.92:0/4216796034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11e0103780 0x7f11e00707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:24:18.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:18.121+0000 7f11e7130700 1 -- 172.21.15.92:0/4216796034 >> 172.21.15.92:0/4216796034 conn(0x7f11e00f8990 msgr2=0x7f11e0107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:24:18.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:18.121+0000 7f11e7130700 1 -- 172.21.15.92:0/4216796034 shutdown_connections 2024-05-06T12:24:18.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:18.121+0000 7f11e7130700 1 -- 172.21.15.92:0/4216796034 wait complete. 2024-05-06T12:24:18.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:17 smithi092 ceph-mon[91933]: pgmap v268: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:24:18.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:17 smithi190 ceph-mon[79744]: pgmap v268: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:24:18.758 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -e sha1=70ce9539b62a15b9de429c1f0e91a1e012b099aa -- bash -c 'ceph orch upgrade status' 2024-05-06T12:24:19.014 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:24:20.177 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:19 smithi092 ceph-mon[91933]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:24:20.177 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:19 smithi092 ceph-mon[91933]: pgmap v269: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:24:20.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:19 smithi190 ceph-mon[79744]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:24:20.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:19 smithi190 ceph-mon[79744]: pgmap v269: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:24:20.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.657+0000 7f4f40a3a700 1 -- 172.21.15.92:0/3351961288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f3c0fe810 msgr2=0x7f4f3c106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:24:20.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.657+0000 7f4f40a3a700 1 --2- 172.21.15.92:0/3351961288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f3c0fe810 0x7f4f3c106e50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4f30004940 tx=0x7f4f30009e30 comp rx=0 tx=0).stop 2024-05-06T12:24:20.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.657+0000 7f4f40a3a700 1 -- 172.21.15.92:0/3351961288 shutdown_connections 2024-05-06T12:24:20.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.657+0000 7f4f40a3a700 1 --2- 172.21.15.92:0/3351961288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f3c0fe810 0x7f4f3c106e50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:24:20.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.657+0000 7f4f40a3a700 1 --2- 172.21.15.92:0/3351961288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f3c0fde70 0x7f4f3c0fe240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:24:20.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.657+0000 7f4f40a3a700 1 -- 172.21.15.92:0/3351961288 >> 172.21.15.92:0/3351961288 conn(0x7f4f3c0f7860 msgr2=0x7f4f3c0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:24:20.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.657+0000 7f4f40a3a700 1 -- 172.21.15.92:0/3351961288 shutdown_connections 2024-05-06T12:24:20.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.657+0000 7f4f40a3a700 1 -- 172.21.15.92:0/3351961288 wait complete. 2024-05-06T12:24:20.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.657+0000 7f4f40a3a700 1 Processor -- start 2024-05-06T12:24:20.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.657+0000 7f4f40a3a700 1 -- start start 2024-05-06T12:24:20.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.658+0000 7f4f40a3a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f3c0fde70 0x7f4f3c113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:24:20.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.658+0000 7f4f40a3a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f3c0fe810 0x7f4f3c113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:24:20.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.658+0000 7f4f40a3a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c1144e0 con 0x7f4f3c0fe810 2024-05-06T12:24:20.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.658+0000 7f4f40a3a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c10dae0 con 0x7f4f3c0fde70 2024-05-06T12:24:20.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.658+0000 7f4f39d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f3c0fe810 0x7f4f3c113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:24:20.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.658+0000 7f4f39d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f3c0fe810 0x7f4f3c113e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58126/0 (socket says 172.21.15.92:58126) 2024-05-06T12:24:20.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.658+0000 7f4f39d9b700 1 -- 172.21.15.92:0/914922642 learned_addr learned my addr 172.21.15.92:0/914922642 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:24:20.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.658+0000 7f4f3a59c700 1 --2- 172.21.15.92:0/914922642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f3c0fde70 0x7f4f3c113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:24:20.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.658+0000 7f4f39d9b700 1 -- 172.21.15.92:0/914922642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f3c0fde70 msgr2=0x7f4f3c113950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:24:20.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.658+0000 7f4f39d9b700 1 --2- 172.21.15.92:0/914922642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f3c0fde70 0x7f4f3c113950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:24:20.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.658+0000 7f4f39d9b700 1 -- 172.21.15.92:0/914922642 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f30005040 con 0x7f4f3c0fe810 2024-05-06T12:24:20.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.658+0000 7f4f3a59c700 1 --2- 172.21.15.92:0/914922642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f3c0fde70 0x7f4f3c113950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-06T12:24:20.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.658+0000 7f4f39d9b700 1 --2- 172.21.15.92:0/914922642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f3c0fe810 0x7f4f3c113e90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009380 tx=0x7f4f30009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:24:20.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.658+0000 7f4f2affd700 1 -- 172.21.15.92:0/914922642 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f3000e040 con 0x7f4f3c0fe810 2024-05-06T12:24:20.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.659+0000 7f4f40a3a700 1 -- 172.21.15.92:0/914922642 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f3c10ddc0 con 0x7f4f3c0fe810 2024-05-06T12:24:20.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.659+0000 7f4f2affd700 1 -- 172.21.15.92:0/914922642 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f30009af0 con 0x7f4f3c0fe810 2024-05-06T12:24:20.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.659+0000 7f4f2affd700 1 -- 172.21.15.92:0/914922642 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f3001c630 con 0x7f4f3c0fe810 2024-05-06T12:24:20.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.659+0000 7f4f40a3a700 1 -- 172.21.15.92:0/914922642 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f3c10e300 con 0x7f4f3c0fe810 2024-05-06T12:24:20.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.660+0000 7f4f40a3a700 1 -- 172.21.15.92:0/914922642 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f1c0052b0 con 0x7f4f3c0fe810 2024-05-06T12:24:20.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.660+0000 7f4f2affd700 1 -- 172.21.15.92:0/914922642 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4f3000bbc0 con 0x7f4f3c0fe810 2024-05-06T12:24:20.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.660+0000 7f4f2affd700 1 --2- 172.21.15.92:0/914922642 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4f2406e140 0x7f4f240705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:24:20.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.661+0000 7f4f2affd700 1 -- 172.21.15.92:0/914922642 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4f3008ee50 con 0x7f4f3c0fe810 2024-05-06T12:24:20.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.661+0000 7f4f3a59c700 1 --2- 172.21.15.92:0/914922642 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4f2406e140 0x7f4f240705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:24:20.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.661+0000 7f4f3a59c700 1 --2- 172.21.15.92:0/914922642 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4f2406e140 0x7f4f240705f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c007fd0 tx=0x7f4f2c009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:24:20.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.665+0000 7f4f2affd700 1 -- 172.21.15.92:0/914922642 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4f3005bd30 con 0x7f4f3c0fe810 2024-05-06T12:24:20.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.885+0000 7f4f40a3a700 1 -- 172.21.15.92:0/914922642 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f1c000bc0 con 0x7f4f2406e140 2024-05-06T12:24:20.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.887+0000 7f4f2affd700 1 -- 172.21.15.92:0/914922642 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f4f1c000bc0 con 0x7f4f2406e140 2024-05-06T12:24:20.889 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:24:20.889 INFO:teuthology.orchestra.run.smithi092.stdout: "target_image": null, 2024-05-06T12:24:20.889 INFO:teuthology.orchestra.run.smithi092.stdout: "in_progress": false, 2024-05-06T12:24:20.889 INFO:teuthology.orchestra.run.smithi092.stdout: "which": "", 2024-05-06T12:24:20.889 INFO:teuthology.orchestra.run.smithi092.stdout: "services_complete": [], 2024-05-06T12:24:20.889 INFO:teuthology.orchestra.run.smithi092.stdout: "progress": null, 2024-05-06T12:24:20.889 INFO:teuthology.orchestra.run.smithi092.stdout: "message": "", 2024-05-06T12:24:20.890 INFO:teuthology.orchestra.run.smithi092.stdout: "is_paused": false 2024-05-06T12:24:20.890 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:24:20.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.889+0000 7f4f40a3a700 1 -- 172.21.15.92:0/914922642 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4f2406e140 msgr2=0x7f4f240705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:24:20.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.889+0000 7f4f40a3a700 1 --2- 172.21.15.92:0/914922642 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4f2406e140 0x7f4f240705f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c007fd0 tx=0x7f4f2c009040 comp rx=0 tx=0).stop 2024-05-06T12:24:20.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.889+0000 7f4f40a3a700 1 -- 172.21.15.92:0/914922642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f3c0fe810 msgr2=0x7f4f3c113e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:24:20.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.889+0000 7f4f40a3a700 1 --2- 172.21.15.92:0/914922642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f3c0fe810 0x7f4f3c113e90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009380 tx=0x7f4f30009e30 comp rx=0 tx=0).stop 2024-05-06T12:24:20.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.889+0000 7f4f40a3a700 1 -- 172.21.15.92:0/914922642 shutdown_connections 2024-05-06T12:24:20.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.889+0000 7f4f40a3a700 1 --2- 172.21.15.92:0/914922642 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4f2406e140 0x7f4f240705f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:24:20.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.889+0000 7f4f40a3a700 1 --2- 172.21.15.92:0/914922642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f3c0fde70 0x7f4f3c113950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:24:20.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.889+0000 7f4f40a3a700 1 --2- 172.21.15.92:0/914922642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f3c0fe810 0x7f4f3c113e90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:24:20.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.889+0000 7f4f40a3a700 1 -- 172.21.15.92:0/914922642 >> 172.21.15.92:0/914922642 conn(0x7f4f3c0f7860 msgr2=0x7f4f3c0f8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:24:20.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.889+0000 7f4f40a3a700 1 -- 172.21.15.92:0/914922642 shutdown_connections 2024-05-06T12:24:20.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:20.889+0000 7f4f40a3a700 1 -- 172.21.15.92:0/914922642 wait complete. 2024-05-06T12:24:21.590 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -e sha1=70ce9539b62a15b9de429c1f0e91a1e012b099aa -- bash -c 'ceph health detail' 2024-05-06T12:24:21.859 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:24:22.209 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:21 smithi092 ceph-mon[91933]: from='client.34570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:24:22.209 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:21 smithi092 ceph-mon[91933]: pgmap v270: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:24:22.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:21 smithi190 ceph-mon[79744]: from='client.34570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:24:22.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:21 smithi190 ceph-mon[79744]: pgmap v270: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:24:23.491 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.487+0000 7fbbd4ff3700 1 -- 172.21.15.92:0/1987895777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbd0101d90 msgr2=0x7fbbd0102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:24:23.492 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.487+0000 7fbbd4ff3700 1 --2- 172.21.15.92:0/1987895777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbd0101d90 0x7fbbd0102160 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8004940 tx=0x7fbbc8009e30 comp rx=0 tx=0).stop 2024-05-06T12:24:23.492 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.487+0000 7fbbd4ff3700 1 -- 172.21.15.92:0/1987895777 shutdown_connections 2024-05-06T12:24:23.492 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.487+0000 7fbbd4ff3700 1 --2- 172.21.15.92:0/1987895777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbbd00fbd40 0x7fbbd00fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:24:23.492 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.487+0000 7fbbd4ff3700 1 --2- 172.21.15.92:0/1987895777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbd0101d90 0x7fbbd0102160 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:24:23.492 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.487+0000 7fbbd4ff3700 1 -- 172.21.15.92:0/1987895777 >> 172.21.15.92:0/1987895777 conn(0x7fbbd0071d00 msgr2=0x7fbbd0072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:24:23.492 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.487+0000 7fbbd4ff3700 1 -- 172.21.15.92:0/1987895777 shutdown_connections 2024-05-06T12:24:23.492 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.487+0000 7fbbd4ff3700 1 -- 172.21.15.92:0/1987895777 wait complete. 2024-05-06T12:24:23.492 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.488+0000 7fbbd4ff3700 1 Processor -- start 2024-05-06T12:24:23.492 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.488+0000 7fbbd4ff3700 1 -- start start 2024-05-06T12:24:23.492 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.488+0000 7fbbd4ff3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbd00fbd40 0x7fbbd010eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:24:23.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.488+0000 7fbbd4ff3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbbd0101d90 0x7fbbd010f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:24:23.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.488+0000 7fbbd4ff3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd010f740 con 0x7fbbd00fbd40 2024-05-06T12:24:23.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.488+0000 7fbbd4ff3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd010f880 con 0x7fbbd0101d90 2024-05-06T12:24:23.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.488+0000 7fbbce59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbd00fbd40 0x7fbbd010eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:24:23.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.488+0000 7fbbce59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbd00fbd40 0x7fbbd010eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34782/0 (socket says 172.21.15.92:34782) 2024-05-06T12:24:23.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.488+0000 7fbbce59c700 1 -- 172.21.15.92:0/1056038864 learned_addr learned my addr 172.21.15.92:0/1056038864 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:24:23.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.488+0000 7fbbcdd9b700 1 --2- 172.21.15.92:0/1056038864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbbd0101d90 0x7fbbd010f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:24:23.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.488+0000 7fbbce59c700 1 -- 172.21.15.92:0/1056038864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbbd0101d90 msgr2=0x7fbbd010f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:24:23.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.488+0000 7fbbce59c700 1 --2- 172.21.15.92:0/1056038864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbbd0101d90 0x7fbbd010f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:24:23.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.488+0000 7fbbce59c700 1 -- 172.21.15.92:0/1056038864 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc8005040 con 0x7fbbd00fbd40 2024-05-06T12:24:23.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.488+0000 7fbbce59c700 1 --2- 172.21.15.92:0/1056038864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbd00fbd40 0x7fbbd010eb20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbbc800b3b0 tx=0x7fbbc8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:24:23.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.489+0000 7fbbbeffd700 1 -- 172.21.15.92:0/1056038864 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbc800e040 con 0x7fbbd00fbd40 2024-05-06T12:24:23.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.489+0000 7fbbbeffd700 1 -- 172.21.15.92:0/1056038864 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbbc800bbc0 con 0x7fbbd00fbd40 2024-05-06T12:24:23.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.489+0000 7fbbd4ff3700 1 -- 172.21.15.92:0/1056038864 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbd0117b90 con 0x7fbbd00fbd40 2024-05-06T12:24:23.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.489+0000 7fbbbeffd700 1 -- 172.21.15.92:0/1056038864 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbc801c3f0 con 0x7fbbd00fbd40 2024-05-06T12:24:23.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.489+0000 7fbbd4ff3700 1 -- 172.21.15.92:0/1056038864 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbd0118120 con 0x7fbbd00fbd40 2024-05-06T12:24:23.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.490+0000 7fbbd4ff3700 1 -- 172.21.15.92:0/1056038864 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbd00fd430 con 0x7fbbd00fbd40 2024-05-06T12:24:23.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.490+0000 7fbbbeffd700 1 -- 172.21.15.92:0/1056038864 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fbbc800bd30 con 0x7fbbd00fbd40 2024-05-06T12:24:23.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.490+0000 7fbbbeffd700 1 --2- 172.21.15.92:0/1056038864 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fbbb806e140 0x7fbbb80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:24:23.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.491+0000 7fbbbeffd700 1 -- 172.21.15.92:0/1056038864 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fbbc808ee60 con 0x7fbbd00fbd40 2024-05-06T12:24:23.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.491+0000 7fbbcdd9b700 1 --2- 172.21.15.92:0/1056038864 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fbbb806e140 0x7fbbb80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:24:23.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.491+0000 7fbbcdd9b700 1 --2- 172.21.15.92:0/1056038864 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fbbb806e140 0x7fbbb80705f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4006fd0 tx=0x7fbbc4008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:24:23.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.495+0000 7fbbbeffd700 1 -- 172.21.15.92:0/1056038864 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fbbc8058190 con 0x7fbbd00fbd40 2024-05-06T12:24:23.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.775+0000 7fbbd4ff3700 1 -- 172.21.15.92:0/1056038864 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fbbd004bf60 con 0x7fbbd00fbd40 2024-05-06T12:24:23.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.776+0000 7fbbbeffd700 1 -- 172.21.15.92:0/1056038864 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fbbc805b850 con 0x7fbbd00fbd40 2024-05-06T12:24:23.778 INFO:teuthology.orchestra.run.smithi092.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-06T12:24:23.779 INFO:teuthology.orchestra.run.smithi092.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-06T12:24:23.779 INFO:teuthology.orchestra.run.smithi092.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-06T12:24:23.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.778+0000 7fbbd4ff3700 1 -- 172.21.15.92:0/1056038864 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fbbb806e140 msgr2=0x7fbbb80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:24:23.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.778+0000 7fbbd4ff3700 1 --2- 172.21.15.92:0/1056038864 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fbbb806e140 0x7fbbb80705f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4006fd0 tx=0x7fbbc4008040 comp rx=0 tx=0).stop 2024-05-06T12:24:23.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.778+0000 7fbbd4ff3700 1 -- 172.21.15.92:0/1056038864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbd00fbd40 msgr2=0x7fbbd010eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:24:23.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.778+0000 7fbbd4ff3700 1 --2- 172.21.15.92:0/1056038864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbd00fbd40 0x7fbbd010eb20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbbc800b3b0 tx=0x7fbbc8009e30 comp rx=0 tx=0).stop 2024-05-06T12:24:23.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.778+0000 7fbbd4ff3700 1 -- 172.21.15.92:0/1056038864 shutdown_connections 2024-05-06T12:24:23.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.778+0000 7fbbd4ff3700 1 --2- 172.21.15.92:0/1056038864 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fbbb806e140 0x7fbbb80705f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:24:23.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.778+0000 7fbbd4ff3700 1 --2- 172.21.15.92:0/1056038864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbd00fbd40 0x7fbbd010eb20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:24:23.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.778+0000 7fbbd4ff3700 1 --2- 172.21.15.92:0/1056038864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbbd0101d90 0x7fbbd010f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:24:23.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.778+0000 7fbbd4ff3700 1 -- 172.21.15.92:0/1056038864 >> 172.21.15.92:0/1056038864 conn(0x7fbbd0071d00 msgr2=0x7fbbd00fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:24:23.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.778+0000 7fbbd4ff3700 1 -- 172.21.15.92:0/1056038864 shutdown_connections 2024-05-06T12:24:23.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:23.778+0000 7fbbd4ff3700 1 -- 172.21.15.92:0/1056038864 wait complete. 2024-05-06T12:24:24.136 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:23 smithi092 ceph-mon[91933]: pgmap v271: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:24:24.136 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:23 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/1056038864' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:24:24.136 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:23 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:24:24.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:23 smithi190 ceph-mon[79744]: pgmap v271: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:24:24.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:23 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/1056038864' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-06T12:24:24.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:23 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:24:24.464 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -e sha1=70ce9539b62a15b9de429c1f0e91a1e012b099aa -- bash -c 'ceph versions' 2024-05-06T12:24:24.722 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:24:26.128 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:25 smithi092 ceph-mon[91933]: pgmap v272: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:24:26.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:25 smithi190 ceph-mon[79744]: pgmap v272: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:24:26.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.443+0000 7f826e63e700 1 -- 172.21.15.92:0/3882416127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82681026a0 msgr2=0x7f82680fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:24:26.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.443+0000 7f826e63e700 1 --2- 172.21.15.92:0/3882416127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82681026a0 0x7f82680fbd10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8258004940 tx=0x7f8258009e30 comp rx=0 tx=0).stop 2024-05-06T12:24:26.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.443+0000 7f826e63e700 1 -- 172.21.15.92:0/3882416127 shutdown_connections 2024-05-06T12:24:26.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.443+0000 7f826e63e700 1 --2- 172.21.15.92:0/3882416127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82681026a0 0x7f82680fbd10 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:24:26.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.443+0000 7f826e63e700 1 --2- 172.21.15.92:0/3882416127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8268101d00 0x7f82681020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:24:26.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.443+0000 7f826e63e700 1 -- 172.21.15.92:0/3882416127 >> 172.21.15.92:0/3882416127 conn(0x7f82680f7870 msgr2=0x7f82680f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:24:26.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.443+0000 7f826e63e700 1 -- 172.21.15.92:0/3882416127 shutdown_connections 2024-05-06T12:24:26.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.443+0000 7f826e63e700 1 -- 172.21.15.92:0/3882416127 wait complete. 2024-05-06T12:24:26.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.444+0000 7f826e63e700 1 Processor -- start 2024-05-06T12:24:26.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.444+0000 7f826e63e700 1 -- start start 2024-05-06T12:24:26.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.445+0000 7f826e63e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8268101d00 0x7f8268115be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:24:26.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.445+0000 7f826e63e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82681026a0 0x7f8268116120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:24:26.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.445+0000 7f826e63e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8268116800 con 0x7f82681026a0 2024-05-06T12:24:26.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.445+0000 7f826e63e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826810fd50 con 0x7f8268101d00 2024-05-06T12:24:26.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.445+0000 7f82677fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82681026a0 0x7f8268116120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:24:26.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.445+0000 7f82677fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82681026a0 0x7f8268116120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34800/0 (socket says 172.21.15.92:34800) 2024-05-06T12:24:26.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.445+0000 7f82677fe700 1 -- 172.21.15.92:0/297579293 learned_addr learned my addr 172.21.15.92:0/297579293 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:24:26.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.445+0000 7f8267fff700 1 --2- 172.21.15.92:0/297579293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8268101d00 0x7f8268115be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:24:26.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.445+0000 7f82677fe700 1 -- 172.21.15.92:0/297579293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8268101d00 msgr2=0x7f8268115be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:24:26.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.445+0000 7f82677fe700 1 --2- 172.21.15.92:0/297579293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8268101d00 0x7f8268115be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:24:26.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.445+0000 7f82677fe700 1 -- 172.21.15.92:0/297579293 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8258005040 con 0x7f82681026a0 2024-05-06T12:24:26.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.445+0000 7f82677fe700 1 --2- 172.21.15.92:0/297579293 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82681026a0 0x7f8268116120 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8258009100 tx=0x7f8258009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:24:26.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.445+0000 7f8264ff9700 1 -- 172.21.15.92:0/297579293 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f825800e040 con 0x7f82681026a0 2024-05-06T12:24:26.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.446+0000 7f826e63e700 1 -- 172.21.15.92:0/297579293 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f826810ff70 con 0x7f82681026a0 2024-05-06T12:24:26.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.446+0000 7f8264ff9700 1 -- 172.21.15.92:0/297579293 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8258009af0 con 0x7f82681026a0 2024-05-06T12:24:26.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.446+0000 7f8264ff9700 1 -- 172.21.15.92:0/297579293 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f825801c3f0 con 0x7f82681026a0 2024-05-06T12:24:26.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.446+0000 7f826e63e700 1 -- 172.21.15.92:0/297579293 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8268110530 con 0x7f82681026a0 2024-05-06T12:24:26.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.447+0000 7f826e63e700 1 -- 172.21.15.92:0/297579293 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82681041f0 con 0x7f82681026a0 2024-05-06T12:24:26.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.447+0000 7f8264ff9700 1 -- 172.21.15.92:0/297579293 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f825801c6b0 con 0x7f82681026a0 2024-05-06T12:24:26.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.447+0000 7f8264ff9700 1 --2- 172.21.15.92:0/297579293 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f825006e140 0x7f82500705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:24:26.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.447+0000 7f8264ff9700 1 -- 172.21.15.92:0/297579293 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f825808ed70 con 0x7f82681026a0 2024-05-06T12:24:26.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.447+0000 7f8267fff700 1 --2- 172.21.15.92:0/297579293 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f825006e140 0x7f82500705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:24:26.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.448+0000 7f8267fff700 1 --2- 172.21.15.92:0/297579293 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f825006e140 0x7f82500705f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f825c009b60 tx=0x7f825c0091a0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:24:26.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.452+0000 7f8264ff9700 1 -- 172.21.15.92:0/297579293 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f825805bc50 con 0x7f82681026a0 2024-05-06T12:24:26.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.738+0000 7f826e63e700 1 -- 172.21.15.92:0/297579293 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f82680fd450 con 0x7f82681026a0 2024-05-06T12:24:26.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.739+0000 7f8264ff9700 1 -- 172.21.15.92:0/297579293 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f825805b7e0 con 0x7f82681026a0 2024-05-06T12:24:26.743 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:24:26.743 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": { 2024-05-06T12:24:26.743 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:24:26.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:24:26.743 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": { 2024-05-06T12:24:26.743 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:24:26.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:24:26.743 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": { 2024-05-06T12:24:26.743 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 6 2024-05-06T12:24:26.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:24:26.743 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-05-06T12:24:26.743 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 4 2024-05-06T12:24:26.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:24:26.744 INFO:teuthology.orchestra.run.smithi092.stdout: "overall": { 2024-05-06T12:24:26.744 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 14 2024-05-06T12:24:26.744 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-06T12:24:26.744 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:24:26.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.741+0000 7f826e63e700 1 -- 172.21.15.92:0/297579293 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f825006e140 msgr2=0x7f82500705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:24:26.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.741+0000 7f826e63e700 1 --2- 172.21.15.92:0/297579293 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f825006e140 0x7f82500705f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f825c009b60 tx=0x7f825c0091a0 comp rx=0 tx=0).stop 2024-05-06T12:24:26.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.741+0000 7f826e63e700 1 -- 172.21.15.92:0/297579293 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82681026a0 msgr2=0x7f8268116120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:24:26.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.741+0000 7f826e63e700 1 --2- 172.21.15.92:0/297579293 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82681026a0 0x7f8268116120 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8258009100 tx=0x7f8258009e30 comp rx=0 tx=0).stop 2024-05-06T12:24:26.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.741+0000 7f826e63e700 1 -- 172.21.15.92:0/297579293 shutdown_connections 2024-05-06T12:24:26.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.741+0000 7f826e63e700 1 --2- 172.21.15.92:0/297579293 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f825006e140 0x7f82500705f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:24:26.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.741+0000 7f826e63e700 1 --2- 172.21.15.92:0/297579293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8268101d00 0x7f8268115be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:24:26.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.741+0000 7f826e63e700 1 --2- 172.21.15.92:0/297579293 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82681026a0 0x7f8268116120 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:24:26.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.741+0000 7f826e63e700 1 -- 172.21.15.92:0/297579293 >> 172.21.15.92:0/297579293 conn(0x7f82680f7870 msgr2=0x7f82681010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:24:26.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.741+0000 7f826e63e700 1 -- 172.21.15.92:0/297579293 shutdown_connections 2024-05-06T12:24:26.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:24:26.741+0000 7f826e63e700 1 -- 172.21.15.92:0/297579293 wait complete. 2024-05-06T12:24:27.058 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:26 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/297579293' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:27.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:26 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/297579293' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:24:27.384 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -e sha1=70ce9539b62a15b9de429c1f0e91a1e012b099aa -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-05-06T12:24:27.638 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:24:28.029 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:27 smithi092 ceph-mon[91933]: pgmap v273: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:24:28.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:27 smithi190 ceph-mon[79744]: pgmap v273: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:24:29.252 INFO:teuthology.orchestra.run.smithi092.stdout:wait for servicemap items w/ changing names to refresh 2024-05-06T12:24:29.823 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -e sha1=70ce9539b62a15b9de429c1f0e91a1e012b099aa -- bash -c 'sleep 60' 2024-05-06T12:24:30.077 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:24:30.199 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:29 smithi092 ceph-mon[91933]: pgmap v274: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:24:30.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:29 smithi190 ceph-mon[79744]: pgmap v274: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:24:32.313 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:31 smithi092 ceph-mon[91933]: pgmap v275: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:24:32.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:31 smithi190 ceph-mon[79744]: pgmap v275: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:24:34.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:33 smithi190 ceph-mon[79744]: pgmap v276: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:24:34.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:33 smithi092 ceph-mon[91933]: pgmap v276: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:24:36.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:35 smithi190 ceph-mon[79744]: pgmap v277: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:24:36.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:35 smithi092 ceph-mon[91933]: pgmap v277: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:24:38.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:37 smithi190 ceph-mon[79744]: pgmap v278: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:24:38.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:37 smithi092 ceph-mon[91933]: pgmap v278: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:24:39.249 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:38 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:24:39.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:38 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:24:40.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:40 smithi190 ceph-mon[79744]: pgmap v279: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:24:40.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:40 smithi092 ceph-mon[91933]: pgmap v279: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:24:42.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:42 smithi190 ceph-mon[79744]: pgmap v280: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:24:42.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:42 smithi092 ceph-mon[91933]: pgmap v280: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:24:44.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:44 smithi190 ceph-mon[79744]: pgmap v281: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:24:44.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:44 smithi092 ceph-mon[91933]: pgmap v281: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:24:46.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:46 smithi190 ceph-mon[79744]: pgmap v282: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:24:46.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:46 smithi092 ceph-mon[91933]: pgmap v282: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:24:48.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:48 smithi190 ceph-mon[79744]: pgmap v283: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:24:48.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:48 smithi092 ceph-mon[91933]: pgmap v283: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:24:50.274 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:50 smithi092 ceph-mon[91933]: pgmap v284: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:24:50.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:50 smithi190 ceph-mon[79744]: pgmap v284: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:24:52.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:52 smithi190 ceph-mon[79744]: pgmap v285: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:24:52.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:52 smithi092 ceph-mon[91933]: pgmap v285: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:24:54.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:54 smithi190 ceph-mon[79744]: pgmap v286: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:24:54.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:54 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:24:54.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:54 smithi092 ceph-mon[91933]: pgmap v286: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:24:54.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:54 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:24:56.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:56 smithi190 ceph-mon[79744]: pgmap v287: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:24:56.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:56 smithi092 ceph-mon[91933]: pgmap v287: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:24:58.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:24:58 smithi190 ceph-mon[79744]: pgmap v288: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:24:58.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:24:58 smithi092 ceph-mon[91933]: pgmap v288: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:25:00.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:00 smithi190 ceph-mon[79744]: pgmap v289: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:25:00.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:00 smithi092 ceph-mon[91933]: pgmap v289: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:25:02.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:02 smithi190 ceph-mon[79744]: pgmap v290: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:25:02.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:02 smithi092 ceph-mon[91933]: pgmap v290: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:25:04.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:04 smithi190 ceph-mon[79744]: pgmap v291: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:25:04.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:04 smithi092 ceph-mon[91933]: pgmap v291: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:25:06.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:06 smithi190 ceph-mon[79744]: pgmap v292: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:25:06.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:06 smithi092 ceph-mon[91933]: pgmap v292: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:25:08.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:08 smithi190 ceph-mon[79744]: pgmap v293: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:25:08.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:08 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:25:08.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:08 smithi092 ceph-mon[91933]: pgmap v293: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:25:08.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:08 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:25:09.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:25:09.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:25:09.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:25:09.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:25:09.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:25:09.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:25:09.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:25:09.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:25:10.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:10 smithi190 ceph-mon[79744]: pgmap v294: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:25:10.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:10 smithi092 ceph-mon[91933]: pgmap v294: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:25:12.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:12 smithi190 ceph-mon[79744]: pgmap v295: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:25:12.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:12 smithi092 ceph-mon[91933]: pgmap v295: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:25:14.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:14 smithi190 ceph-mon[79744]: pgmap v296: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:25:14.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:14 smithi092 ceph-mon[91933]: pgmap v296: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:25:16.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:16 smithi190 ceph-mon[79744]: pgmap v297: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:25:16.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:16 smithi092 ceph-mon[91933]: pgmap v297: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:25:18.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:18 smithi190 ceph-mon[79744]: pgmap v298: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:25:18.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:18 smithi092 ceph-mon[91933]: pgmap v298: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:25:20.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:20 smithi190 ceph-mon[79744]: pgmap v299: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:25:20.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:20 smithi092 ceph-mon[91933]: pgmap v299: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:25:22.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:22 smithi190 ceph-mon[79744]: pgmap v300: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:25:22.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:22 smithi092 ceph-mon[91933]: pgmap v300: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:25:24.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:24 smithi190 ceph-mon[79744]: pgmap v301: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:25:24.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:24 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:25:24.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:24 smithi092 ceph-mon[91933]: pgmap v301: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:25:24.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:24 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:25:26.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:26 smithi190 ceph-mon[79744]: pgmap v302: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:25:26.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:26 smithi092 ceph-mon[91933]: pgmap v302: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:25:28.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:28 smithi190 ceph-mon[79744]: pgmap v303: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:25:28.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:28 smithi092 ceph-mon[91933]: pgmap v303: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:25:30.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:30 smithi190 ceph-mon[79744]: pgmap v304: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:25:30.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:30 smithi092 ceph-mon[91933]: pgmap v304: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:25:32.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:32 smithi190 ceph-mon[79744]: pgmap v305: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:25:32.396 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:32 smithi092 ceph-mon[91933]: pgmap v305: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:25:32.430 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -e sha1=70ce9539b62a15b9de429c1f0e91a1e012b099aa -- bash -c 'ceph orch ps' 2024-05-06T12:25:32.690 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:25:34.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:34 smithi190 ceph-mon[79744]: pgmap v306: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:25:34.356 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:34 smithi092 ceph-mon[91933]: pgmap v306: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:25:34.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.378+0000 7fd1f6310700 1 -- 172.21.15.92:0/162890817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1f0102de0 msgr2=0x7fd1f01031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:34.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.378+0000 7fd1f6310700 1 --2- 172.21.15.92:0/162890817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1f0102de0 0x7fd1f01031b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0004940 tx=0x7fd1e0009e30 comp rx=0 tx=0).stop 2024-05-06T12:25:34.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.378+0000 7fd1f6310700 1 -- 172.21.15.92:0/162890817 shutdown_connections 2024-05-06T12:25:34.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.378+0000 7fd1f6310700 1 --2- 172.21.15.92:0/162890817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1f0103780 0x7fd1f00fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:34.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.378+0000 7fd1f6310700 1 --2- 172.21.15.92:0/162890817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1f0102de0 0x7fd1f01031b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:34.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.378+0000 7fd1f6310700 1 -- 172.21.15.92:0/162890817 >> 172.21.15.92:0/162890817 conn(0x7fd1f00f8990 msgr2=0x7fd1f00fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:34.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.378+0000 7fd1f6310700 1 -- 172.21.15.92:0/162890817 shutdown_connections 2024-05-06T12:25:34.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.378+0000 7fd1f6310700 1 -- 172.21.15.92:0/162890817 wait complete. 2024-05-06T12:25:34.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.379+0000 7fd1f6310700 1 Processor -- start 2024-05-06T12:25:34.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.379+0000 7fd1f6310700 1 -- start start 2024-05-06T12:25:34.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.379+0000 7fd1f6310700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1f0102de0 0x7fd1f0113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:34.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.379+0000 7fd1f6310700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1f0103780 0x7fd1f0113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:34.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.379+0000 7fd1f6310700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f0114500 con 0x7fd1f0102de0 2024-05-06T12:25:34.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.379+0000 7fd1f6310700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f010db50 con 0x7fd1f0103780 2024-05-06T12:25:34.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.379+0000 7fd1effff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1f0102de0 0x7fd1f0113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:34.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.380+0000 7fd1effff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1f0102de0 0x7fd1f0113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55774/0 (socket says 172.21.15.92:55774) 2024-05-06T12:25:34.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.380+0000 7fd1effff700 1 -- 172.21.15.92:0/808093344 learned_addr learned my addr 172.21.15.92:0/808093344 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:34.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.380+0000 7fd1ef7fe700 1 --2- 172.21.15.92:0/808093344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1f0103780 0x7fd1f0113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:34.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.380+0000 7fd1effff700 1 -- 172.21.15.92:0/808093344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1f0103780 msgr2=0x7fd1f0113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:34.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.380+0000 7fd1effff700 1 --2- 172.21.15.92:0/808093344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1f0103780 0x7fd1f0113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:34.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.380+0000 7fd1effff700 1 -- 172.21.15.92:0/808093344 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e0005040 con 0x7fd1f0102de0 2024-05-06T12:25:34.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.380+0000 7fd1effff700 1 --2- 172.21.15.92:0/808093344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1f0102de0 0x7fd1f0113970 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000b3b0 tx=0x7fd1e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:34.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.380+0000 7fd1ecff9700 1 -- 172.21.15.92:0/808093344 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1e000e040 con 0x7fd1f0102de0 2024-05-06T12:25:34.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.380+0000 7fd1ecff9700 1 -- 172.21.15.92:0/808093344 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd1e0013420 con 0x7fd1f0102de0 2024-05-06T12:25:34.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.380+0000 7fd1ecff9700 1 -- 172.21.15.92:0/808093344 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1e001c630 con 0x7fd1f0102de0 2024-05-06T12:25:34.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.380+0000 7fd1f6310700 1 -- 172.21.15.92:0/808093344 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1f010ddd0 con 0x7fd1f0102de0 2024-05-06T12:25:34.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.380+0000 7fd1f6310700 1 -- 172.21.15.92:0/808093344 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1f010e390 con 0x7fd1f0102de0 2024-05-06T12:25:34.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.381+0000 7fd1f6310700 1 -- 172.21.15.92:0/808093344 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1f01000f0 con 0x7fd1f0102de0 2024-05-06T12:25:34.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.382+0000 7fd1ecff9700 1 -- 172.21.15.92:0/808093344 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd1e000baf0 con 0x7fd1f0102de0 2024-05-06T12:25:34.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.382+0000 7fd1ecff9700 1 --2- 172.21.15.92:0/808093344 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd1d806e140 0x7fd1d80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:34.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.382+0000 7fd1ecff9700 1 -- 172.21.15.92:0/808093344 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd1e008ed60 con 0x7fd1f0102de0 2024-05-06T12:25:34.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.382+0000 7fd1ef7fe700 1 --2- 172.21.15.92:0/808093344 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd1d806e140 0x7fd1d80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:34.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.383+0000 7fd1ef7fe700 1 --2- 172.21.15.92:0/808093344 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd1d806e140 0x7fd1d80705f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd1f010ecc0 tx=0x7fd1e400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:34.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.387+0000 7fd1ecff9700 1 -- 172.21.15.92:0/808093344 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd1e005bc40 con 0x7fd1f0102de0 2024-05-06T12:25:34.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.595+0000 7fd1f6310700 1 -- 172.21.15.92:0/808093344 --> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1f010efc0 con 0x7fd1d806e140 2024-05-06T12:25:34.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.604+0000 7fd1ecff9700 1 -- 172.21.15.92:0/808093344 <== mgr.34102 v2:172.21.15.92:6800/114121430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fd1f010efc0 con 0x7fd1d806e140 2024-05-06T12:25:34.607 INFO:teuthology.orchestra.run.smithi092.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-06T12:25:34.607 INFO:teuthology.orchestra.run.smithi092.stdout:alertmanager.smithi092 smithi092 *:9093,9094 running (2m) 94s ago 19m 15.1M - 0.25.0 c8568f914cd2 c9a1046648dc 2024-05-06T12:25:34.607 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi092 smithi092 running (8m) 94s ago 19m 6660k - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 8da0494e1bdc 2024-05-06T12:25:34.607 INFO:teuthology.orchestra.run.smithi092.stdout:crash.smithi190 smithi190 running (7m) 2m ago 18m 6665k - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 641a7cdaa56e 2024-05-06T12:25:34.607 INFO:teuthology.orchestra.run.smithi092.stdout:grafana.smithi092 smithi092 *:3000 running (99s) 94s ago 18m 52.6M - 9.4.12 2bacad6d85d8 55135681ef06 2024-05-06T12:25:34.607 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.ijsqgp smithi092 running (3m) 94s ago 13m 66.7M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 40b4cdd3be40 2024-05-06T12:25:34.607 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi092.rosbll smithi092 running (3m) 94s ago 13m 17.0M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 550f8238d95f 2024-05-06T12:25:34.607 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.byjvgf smithi190 running (3m) 2m ago 13m 63.4M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 0d26051c908e 2024-05-06T12:25:34.607 INFO:teuthology.orchestra.run.smithi092.stdout:mds.cephfs.smithi190.idzoff smithi190 running (2m) 2m ago 13m 12.8M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 7cfd0097389b 2024-05-06T12:25:34.607 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi092.gyzeqb smithi092 *:8443,9283,8765 running (10m) 94s ago 20m 519M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 de9d2198327a 2024-05-06T12:25:34.607 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.smithi190.kczxkh smithi190 *:8443,9283,8765 running (9m) 2m ago 18m 452M - 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 e84ab3758ac7 2024-05-06T12:25:34.608 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi092 smithi092 running (9m) 94s ago 20m 56.2M 2048M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 ac07f7158087 2024-05-06T12:25:34.608 INFO:teuthology.orchestra.run.smithi092.stdout:mon.smithi190 smithi190 running (8m) 2m ago 18m 39.7M 2048M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 7a3c08eb716f 2024-05-06T12:25:34.608 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi092 smithi092 *:9100 running (2m) 94s ago 19m 8292k - 1.5.0 0da6a335fe13 590fa41f0538 2024-05-06T12:25:34.608 INFO:teuthology.orchestra.run.smithi092.stdout:node-exporter.smithi190 smithi190 *:9100 running (2m) 2m ago 18m 3590k - 1.5.0 0da6a335fe13 a57191dac5b5 2024-05-06T12:25:34.608 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 smithi092 running (7m) 94s ago 17m 94.2M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 8c96c75b25f4 2024-05-06T12:25:34.608 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 smithi092 running (6m) 94s ago 16m 97.4M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 e8007a510f08 2024-05-06T12:25:34.608 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 smithi092 running (6m) 94s ago 16m 91.6M 1557M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 2beca7448789 2024-05-06T12:25:34.608 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 smithi190 running (5m) 2m ago 16m 101M 2581M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 914ada199f93 2024-05-06T12:25:34.608 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 smithi190 running (5m) 2m ago 15m 89.4M 2581M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 371c0be776c7 2024-05-06T12:25:34.608 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 smithi190 running (4m) 2m ago 15m 86.8M 2581M 19.0.0-2498-g70ce9539 e4ce8ad5d0c4 067bc34cdfb1 2024-05-06T12:25:34.608 INFO:teuthology.orchestra.run.smithi092.stdout:prometheus.smithi092 smithi092 *:9095 running (2m) 94s ago 18m 46.9M - 2.43.0 a07b618ecd1d eadf61684ae7 2024-05-06T12:25:34.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.607+0000 7fd1f6310700 1 -- 172.21.15.92:0/808093344 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd1d806e140 msgr2=0x7fd1d80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:34.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.607+0000 7fd1f6310700 1 --2- 172.21.15.92:0/808093344 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd1d806e140 0x7fd1d80705f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd1f010ecc0 tx=0x7fd1e400b040 comp rx=0 tx=0).stop 2024-05-06T12:25:34.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.607+0000 7fd1f6310700 1 -- 172.21.15.92:0/808093344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1f0102de0 msgr2=0x7fd1f0113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:34.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.607+0000 7fd1f6310700 1 --2- 172.21.15.92:0/808093344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1f0102de0 0x7fd1f0113970 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000b3b0 tx=0x7fd1e0009e30 comp rx=0 tx=0).stop 2024-05-06T12:25:34.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.607+0000 7fd1f6310700 1 -- 172.21.15.92:0/808093344 shutdown_connections 2024-05-06T12:25:34.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.607+0000 7fd1f6310700 1 --2- 172.21.15.92:0/808093344 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd1d806e140 0x7fd1d80705f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:34.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.607+0000 7fd1f6310700 1 --2- 172.21.15.92:0/808093344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1f0102de0 0x7fd1f0113970 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:34.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.607+0000 7fd1f6310700 1 --2- 172.21.15.92:0/808093344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1f0103780 0x7fd1f0113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:34.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.607+0000 7fd1f6310700 1 -- 172.21.15.92:0/808093344 >> 172.21.15.92:0/808093344 conn(0x7fd1f00f8990 msgr2=0x7fd1f0107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:34.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.607+0000 7fd1f6310700 1 -- 172.21.15.92:0/808093344 shutdown_connections 2024-05-06T12:25:34.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:34.607+0000 7fd1f6310700 1 -- 172.21.15.92:0/808093344 wait complete. 2024-05-06T12:25:35.204 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -e sha1=70ce9539b62a15b9de429c1f0e91a1e012b099aa -- bash -c 'ceph versions' 2024-05-06T12:25:35.472 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:25:36.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:36 smithi190 ceph-mon[79744]: from='client.34582 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:25:36.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:36 smithi190 ceph-mon[79744]: pgmap v307: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:25:36.451 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:36 smithi092 ceph-mon[91933]: from='client.34582 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-06T12:25:36.452 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:36 smithi092 ceph-mon[91933]: pgmap v307: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:25:37.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.008+0000 7fa058b90700 1 -- 172.21.15.92:0/1505357409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0541026b0 msgr2=0x7fa0540fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:37.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.008+0000 7fa058b90700 1 --2- 172.21.15.92:0/1505357409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0541026b0 0x7fa0540fbd20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa03c004940 tx=0x7fa03c009e30 comp rx=0 tx=0).stop 2024-05-06T12:25:37.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.008+0000 7fa058b90700 1 -- 172.21.15.92:0/1505357409 shutdown_connections 2024-05-06T12:25:37.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.008+0000 7fa058b90700 1 --2- 172.21.15.92:0/1505357409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0541026b0 0x7fa0540fbd20 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:37.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.008+0000 7fa058b90700 1 --2- 172.21.15.92:0/1505357409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa054101d10 0x7fa0541020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:37.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.008+0000 7fa058b90700 1 -- 172.21.15.92:0/1505357409 >> 172.21.15.92:0/1505357409 conn(0x7fa0540f78a0 msgr2=0x7fa0540f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:37.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.008+0000 7fa058b90700 1 -- 172.21.15.92:0/1505357409 shutdown_connections 2024-05-06T12:25:37.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.008+0000 7fa058b90700 1 -- 172.21.15.92:0/1505357409 wait complete. 2024-05-06T12:25:37.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.008+0000 7fa058b90700 1 Processor -- start 2024-05-06T12:25:37.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.008+0000 7fa058b90700 1 -- start start 2024-05-06T12:25:37.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.009+0000 7fa058b90700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa054101d10 0x7fa05410eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:37.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.009+0000 7fa058b90700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0541026b0 0x7fa05410f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:37.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.009+0000 7fa058b90700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05410f590 con 0x7fa0541026b0 2024-05-06T12:25:37.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.009+0000 7fa058b90700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05410f6d0 con 0x7fa054101d10 2024-05-06T12:25:37.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.009+0000 7fa049d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0541026b0 0x7fa05410f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:37.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.009+0000 7fa049d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0541026b0 0x7fa05410f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55788/0 (socket says 172.21.15.92:55788) 2024-05-06T12:25:37.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.009+0000 7fa049d9b700 1 -- 172.21.15.92:0/3199905676 learned_addr learned my addr 172.21.15.92:0/3199905676 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:37.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.009+0000 7fa05259c700 1 --2- 172.21.15.92:0/3199905676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa054101d10 0x7fa05410eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:37.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.009+0000 7fa049d9b700 1 -- 172.21.15.92:0/3199905676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa054101d10 msgr2=0x7fa05410eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:37.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.009+0000 7fa049d9b700 1 --2- 172.21.15.92:0/3199905676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa054101d10 0x7fa05410eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:37.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.009+0000 7fa049d9b700 1 -- 172.21.15.92:0/3199905676 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa03c005040 con 0x7fa0541026b0 2024-05-06T12:25:37.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.009+0000 7fa049d9b700 1 --2- 172.21.15.92:0/3199905676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0541026b0 0x7fa05410f050 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa03c009180 tx=0x7fa03c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:37.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.009+0000 7fa04b7fe700 1 -- 172.21.15.92:0/3199905676 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa03c00e040 con 0x7fa0541026b0 2024-05-06T12:25:37.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.009+0000 7fa058b90700 1 -- 172.21.15.92:0/3199905676 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa05410f900 con 0x7fa0541026b0 2024-05-06T12:25:37.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.009+0000 7fa04b7fe700 1 -- 172.21.15.92:0/3199905676 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa03c013420 con 0x7fa0541026b0 2024-05-06T12:25:37.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.009+0000 7fa04b7fe700 1 -- 172.21.15.92:0/3199905676 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa03c01c610 con 0x7fa0541026b0 2024-05-06T12:25:37.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.009+0000 7fa058b90700 1 -- 172.21.15.92:0/3199905676 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa054117f20 con 0x7fa0541026b0 2024-05-06T12:25:37.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.010+0000 7fa058b90700 1 -- 172.21.15.92:0/3199905676 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0540fd460 con 0x7fa0541026b0 2024-05-06T12:25:37.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.011+0000 7fa04b7fe700 1 -- 172.21.15.92:0/3199905676 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa03c00baf0 con 0x7fa0541026b0 2024-05-06T12:25:37.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.011+0000 7fa04b7fe700 1 --2- 172.21.15.92:0/3199905676 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fa04006e140 0x7fa0400705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:37.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.012+0000 7fa04b7fe700 1 -- 172.21.15.92:0/3199905676 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa03c08ecd0 con 0x7fa0541026b0 2024-05-06T12:25:37.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.012+0000 7fa05259c700 1 --2- 172.21.15.92:0/3199905676 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fa04006e140 0x7fa0400705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:37.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.012+0000 7fa05259c700 1 --2- 172.21.15.92:0/3199905676 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fa04006e140 0x7fa0400705f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa044009bf0 tx=0x7fa0440091a0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:37.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.017+0000 7fa04b7fe700 1 -- 172.21.15.92:0/3199905676 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa03c058080 con 0x7fa0541026b0 2024-05-06T12:25:37.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.305+0000 7fa058b90700 1 -- 172.21.15.92:0/3199905676 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa0540fd460 con 0x7fa0541026b0 2024-05-06T12:25:37.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.305+0000 7fa04b7fe700 1 -- 172.21.15.92:0/3199905676 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fa03c05b740 con 0x7fa0541026b0 2024-05-06T12:25:37.308 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-06T12:25:37.308 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": { 2024-05-06T12:25:37.308 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:25:37.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:25:37.308 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": { 2024-05-06T12:25:37.308 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 2 2024-05-06T12:25:37.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:25:37.308 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": { 2024-05-06T12:25:37.308 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 6 2024-05-06T12:25:37.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:25:37.308 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-05-06T12:25:37.308 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 4 2024-05-06T12:25:37.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-06T12:25:37.308 INFO:teuthology.orchestra.run.smithi092.stdout: "overall": { 2024-05-06T12:25:37.308 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)": 14 2024-05-06T12:25:37.309 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-06T12:25:37.309 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-06T12:25:37.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.307+0000 7fa058b90700 1 -- 172.21.15.92:0/3199905676 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fa04006e140 msgr2=0x7fa0400705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:37.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.307+0000 7fa058b90700 1 --2- 172.21.15.92:0/3199905676 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fa04006e140 0x7fa0400705f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa044009bf0 tx=0x7fa0440091a0 comp rx=0 tx=0).stop 2024-05-06T12:25:37.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.307+0000 7fa058b90700 1 -- 172.21.15.92:0/3199905676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0541026b0 msgr2=0x7fa05410f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:37.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.307+0000 7fa058b90700 1 --2- 172.21.15.92:0/3199905676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0541026b0 0x7fa05410f050 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa03c009180 tx=0x7fa03c009e30 comp rx=0 tx=0).stop 2024-05-06T12:25:37.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.307+0000 7fa058b90700 1 -- 172.21.15.92:0/3199905676 shutdown_connections 2024-05-06T12:25:37.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.307+0000 7fa058b90700 1 --2- 172.21.15.92:0/3199905676 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fa04006e140 0x7fa0400705f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:37.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.307+0000 7fa058b90700 1 --2- 172.21.15.92:0/3199905676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa054101d10 0x7fa05410eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:37.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.307+0000 7fa058b90700 1 --2- 172.21.15.92:0/3199905676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0541026b0 0x7fa05410f050 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:37.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.307+0000 7fa058b90700 1 -- 172.21.15.92:0/3199905676 >> 172.21.15.92:0/3199905676 conn(0x7fa0540f78a0 msgr2=0x7fa0540f8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:37.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.307+0000 7fa058b90700 1 -- 172.21.15.92:0/3199905676 shutdown_connections 2024-05-06T12:25:37.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:37.307+0000 7fa058b90700 1 -- 172.21.15.92:0/3199905676 wait complete. 2024-05-06T12:25:37.965 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -e sha1=70ce9539b62a15b9de429c1f0e91a1e012b099aa -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-05-06T12:25:38.221 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:25:38.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:38 smithi092 ceph-mon[91933]: pgmap v308: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:25:38.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:38 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/3199905676' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:25:38.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:38 smithi190 ceph-mon[79744]: pgmap v308: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:25:38.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:38 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/3199905676' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:25:39.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:39 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:25:39.414 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:39 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:25:39.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:39.980+0000 7ff4f93b4700 1 -- 172.21.15.92:0/2060959185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4f4101ce0 msgr2=0x7ff4f41020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:39.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:39.980+0000 7ff4f93b4700 1 --2- 172.21.15.92:0/2060959185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4f4101ce0 0x7ff4f41020b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec009ce0 tx=0x7ff4ec004d10 comp rx=0 tx=0).stop 2024-05-06T12:25:39.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:39.980+0000 7ff4f93b4700 1 -- 172.21.15.92:0/2060959185 shutdown_connections 2024-05-06T12:25:39.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:39.980+0000 7ff4f93b4700 1 --2- 172.21.15.92:0/2060959185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4f4102680 0x7ff4f40fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:39.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:39.980+0000 7ff4f93b4700 1 --2- 172.21.15.92:0/2060959185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4f4101ce0 0x7ff4f41020b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:39.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:39.980+0000 7ff4f93b4700 1 -- 172.21.15.92:0/2060959185 >> 172.21.15.92:0/2060959185 conn(0x7ff4f40f7870 msgr2=0x7ff4f40f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:39.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:39.980+0000 7ff4f93b4700 1 -- 172.21.15.92:0/2060959185 shutdown_connections 2024-05-06T12:25:39.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:39.980+0000 7ff4f93b4700 1 -- 172.21.15.92:0/2060959185 wait complete. 2024-05-06T12:25:39.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:39.981+0000 7ff4f93b4700 1 Processor -- start 2024-05-06T12:25:39.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:39.981+0000 7ff4f93b4700 1 -- start start 2024-05-06T12:25:39.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:39.981+0000 7ff4f93b4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4f4101ce0 0x7ff4f410ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:39.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:39.981+0000 7ff4f93b4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4f4102680 0x7ff4f410f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:39.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:39.981+0000 7ff4f93b4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f410f6b0 con 0x7ff4f4101ce0 2024-05-06T12:25:39.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:39.981+0000 7ff4f93b4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f410f7f0 con 0x7ff4f4102680 2024-05-06T12:25:39.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:39.981+0000 7ff4f2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4f4101ce0 0x7ff4f410ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:39.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:39.981+0000 7ff4f2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4f4101ce0 0x7ff4f410ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55802/0 (socket says 172.21.15.92:55802) 2024-05-06T12:25:39.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:39.981+0000 7ff4f2ffd700 1 -- 172.21.15.92:0/1505556869 learned_addr learned my addr 172.21.15.92:0/1505556869 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:39.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:39.981+0000 7ff4f2ffd700 1 -- 172.21.15.92:0/1505556869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4f4102680 msgr2=0x7ff4f410f170 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-06T12:25:39.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:39.981+0000 7ff4f2ffd700 1 --2- 172.21.15.92:0/1505556869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4f4102680 0x7ff4f410f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:39.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:39.981+0000 7ff4f2ffd700 1 -- 172.21.15.92:0/1505556869 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ec009940 con 0x7ff4f4101ce0 2024-05-06T12:25:39.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:39.981+0000 7ff4f2ffd700 1 --2- 172.21.15.92:0/1505556869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4f4101ce0 0x7ff4f410ec30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00be90 tx=0x7ff4ec004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:39.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:39.982+0000 7ff4db7fe700 1 -- 172.21.15.92:0/1505556869 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4ec00e040 con 0x7ff4f4101ce0 2024-05-06T12:25:39.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:39.982+0000 7ff4db7fe700 1 -- 172.21.15.92:0/1505556869 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff4ec008ea0 con 0x7ff4f4101ce0 2024-05-06T12:25:39.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:39.982+0000 7ff4db7fe700 1 -- 172.21.15.92:0/1505556869 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4ec01c940 con 0x7ff4f4101ce0 2024-05-06T12:25:39.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:39.982+0000 7ff4f93b4700 1 -- 172.21.15.92:0/1505556869 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4f410fa70 con 0x7ff4f4101ce0 2024-05-06T12:25:39.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:39.982+0000 7ff4f93b4700 1 -- 172.21.15.92:0/1505556869 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f41180a0 con 0x7ff4f4101ce0 2024-05-06T12:25:39.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:39.983+0000 7ff4f93b4700 1 -- 172.21.15.92:0/1505556869 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4f40fd430 con 0x7ff4f4101ce0 2024-05-06T12:25:39.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:39.983+0000 7ff4db7fe700 1 -- 172.21.15.92:0/1505556869 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff4ec012d00 con 0x7ff4f4101ce0 2024-05-06T12:25:39.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:39.983+0000 7ff4db7fe700 1 --2- 172.21.15.92:0/1505556869 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff4dc06e140 0x7ff4dc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:39.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:39.984+0000 7ff4db7fe700 1 -- 172.21.15.92:0/1505556869 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff4ec08f770 con 0x7ff4f4101ce0 2024-05-06T12:25:39.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:39.984+0000 7ff4f27fc700 1 --2- 172.21.15.92:0/1505556869 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff4dc06e140 0x7ff4dc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:39.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:39.984+0000 7ff4f27fc700 1 --2- 172.21.15.92:0/1505556869 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff4dc06e140 0x7ff4dc0705f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8006fd0 tx=0x7ff4e8006d30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:39.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:39.988+0000 7ff4db7fe700 1 -- 172.21.15.92:0/1505556869 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff4ec058aa0 con 0x7ff4f4101ce0 2024-05-06T12:25:40.273 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:40 smithi092 ceph-mon[91933]: pgmap v309: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:25:40.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:40.271+0000 7ff4f93b4700 1 -- 172.21.15.92:0/1505556869 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff4f4118290 con 0x7ff4f4101ce0 2024-05-06T12:25:40.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:40.271+0000 7ff4db7fe700 1 -- 172.21.15.92:0/1505556869 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7ff4ec004e10 con 0x7ff4f4101ce0 2024-05-06T12:25:40.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:40.274+0000 7ff4f93b4700 1 -- 172.21.15.92:0/1505556869 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff4dc06e140 msgr2=0x7ff4dc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:40.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:40.274+0000 7ff4f93b4700 1 --2- 172.21.15.92:0/1505556869 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff4dc06e140 0x7ff4dc0705f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8006fd0 tx=0x7ff4e8006d30 comp rx=0 tx=0).stop 2024-05-06T12:25:40.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:40.274+0000 7ff4f93b4700 1 -- 172.21.15.92:0/1505556869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4f4101ce0 msgr2=0x7ff4f410ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:40.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:40.274+0000 7ff4f93b4700 1 --2- 172.21.15.92:0/1505556869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4f4101ce0 0x7ff4f410ec30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00be90 tx=0x7ff4ec004d10 comp rx=0 tx=0).stop 2024-05-06T12:25:40.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:40.274+0000 7ff4f93b4700 1 -- 172.21.15.92:0/1505556869 shutdown_connections 2024-05-06T12:25:40.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:40.274+0000 7ff4f93b4700 1 --2- 172.21.15.92:0/1505556869 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff4dc06e140 0x7ff4dc0705f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:40.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:40.274+0000 7ff4f93b4700 1 --2- 172.21.15.92:0/1505556869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4f4101ce0 0x7ff4f410ec30 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:40.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:40.274+0000 7ff4f93b4700 1 --2- 172.21.15.92:0/1505556869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4f4102680 0x7ff4f410f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:40.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:40.274+0000 7ff4f93b4700 1 -- 172.21.15.92:0/1505556869 >> 172.21.15.92:0/1505556869 conn(0x7ff4f40f7870 msgr2=0x7ff4f40f87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:40.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:40.274+0000 7ff4f93b4700 1 -- 172.21.15.92:0/1505556869 shutdown_connections 2024-05-06T12:25:40.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:40.274+0000 7ff4f93b4700 1 -- 172.21.15.92:0/1505556869 wait complete. 2024-05-06T12:25:40.286 INFO:teuthology.orchestra.run.smithi092.stdout:true 2024-05-06T12:25:40.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:40 smithi190 ceph-mon[79744]: pgmap v309: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:25:40.840 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -e sha1=70ce9539b62a15b9de429c1f0e91a1e012b099aa -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-05-06T12:25:41.093 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:25:41.450 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:41 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/1505556869' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:25:41.450 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:41 smithi092 ceph-mon[91933]: pgmap v310: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:25:41.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:41 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/1505556869' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:25:41.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:41 smithi190 ceph-mon[79744]: pgmap v310: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:25:42.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.669+0000 7f7ffc180700 1 -- 172.21.15.92:0/1520933777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ff4103780 msgr2=0x7f7ff40fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:42.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.669+0000 7f7ffc180700 1 --2- 172.21.15.92:0/1520933777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ff4103780 0x7f7ff40fcdf0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8004940 tx=0x7f7fe8009e30 comp rx=0 tx=0).stop 2024-05-06T12:25:42.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.669+0000 7f7ffc180700 1 -- 172.21.15.92:0/1520933777 shutdown_connections 2024-05-06T12:25:42.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.669+0000 7f7ffc180700 1 --2- 172.21.15.92:0/1520933777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ff4103780 0x7f7ff40fcdf0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:42.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.669+0000 7f7ffc180700 1 --2- 172.21.15.92:0/1520933777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ff4102de0 0x7f7ff41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:42.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.669+0000 7f7ffc180700 1 -- 172.21.15.92:0/1520933777 >> 172.21.15.92:0/1520933777 conn(0x7f7ff40f8990 msgr2=0x7f7ff40fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:42.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.669+0000 7f7ffc180700 1 -- 172.21.15.92:0/1520933777 shutdown_connections 2024-05-06T12:25:42.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.669+0000 7f7ffc180700 1 -- 172.21.15.92:0/1520933777 wait complete. 2024-05-06T12:25:42.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.669+0000 7f7ffc180700 1 Processor -- start 2024-05-06T12:25:42.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.670+0000 7f7ffc180700 1 -- start start 2024-05-06T12:25:42.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.670+0000 7f7ffc180700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ff4102de0 0x7f7ff4070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:42.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.670+0000 7f7ffc180700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ff4103780 0x7f7ff40707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:42.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.670+0000 7f7ffc180700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff40721b0 con 0x7f7ff4102de0 2024-05-06T12:25:42.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.670+0000 7f7ffc180700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff4070ce0 con 0x7f7ff4103780 2024-05-06T12:25:42.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.670+0000 7f7ff9f1c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ff4102de0 0x7f7ff4070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:42.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.670+0000 7f7ff9f1c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ff4102de0 0x7f7ff4070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35742/0 (socket says 172.21.15.92:35742) 2024-05-06T12:25:42.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.670+0000 7f7ff9f1c700 1 -- 172.21.15.92:0/130662631 learned_addr learned my addr 172.21.15.92:0/130662631 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:42.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.670+0000 7f7ff971b700 1 --2- 172.21.15.92:0/130662631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ff4103780 0x7f7ff40707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:42.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.670+0000 7f7ff9f1c700 1 -- 172.21.15.92:0/130662631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ff4103780 msgr2=0x7f7ff40707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:42.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.670+0000 7f7ff9f1c700 1 --2- 172.21.15.92:0/130662631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ff4103780 0x7f7ff40707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:42.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.670+0000 7f7ff9f1c700 1 -- 172.21.15.92:0/130662631 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe8005040 con 0x7f7ff4102de0 2024-05-06T12:25:42.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.670+0000 7f7ff9f1c700 1 --2- 172.21.15.92:0/130662631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ff4102de0 0x7f7ff4070260 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7fec001d80 tx=0x7f7fec009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:42.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.670+0000 7f7ff27fc700 1 -- 172.21.15.92:0/130662631 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fec007d10 con 0x7f7ff4102de0 2024-05-06T12:25:42.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.670+0000 7f7ff27fc700 1 -- 172.21.15.92:0/130662631 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7fec007e70 con 0x7f7ff4102de0 2024-05-06T12:25:42.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.670+0000 7f7ffc180700 1 -- 172.21.15.92:0/130662631 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ff4070fc0 con 0x7f7ff4102de0 2024-05-06T12:25:42.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.670+0000 7f7ff27fc700 1 -- 172.21.15.92:0/130662631 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fec014640 con 0x7f7ff4102de0 2024-05-06T12:25:42.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.671+0000 7f7ffc180700 1 -- 172.21.15.92:0/130662631 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff411a140 con 0x7f7ff4102de0 2024-05-06T12:25:42.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.671+0000 7f7ffc180700 1 -- 172.21.15.92:0/130662631 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ff40fe530 con 0x7f7ff4102de0 2024-05-06T12:25:42.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.672+0000 7f7ff27fc700 1 -- 172.21.15.92:0/130662631 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7fec00e960 con 0x7f7ff4102de0 2024-05-06T12:25:42.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.672+0000 7f7ff27fc700 1 --2- 172.21.15.92:0/130662631 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7fdc06e140 0x7f7fdc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:42.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.672+0000 7f7ff27fc700 1 -- 172.21.15.92:0/130662631 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7fec08d960 con 0x7f7ff4102de0 2024-05-06T12:25:42.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.672+0000 7f7ff971b700 1 --2- 172.21.15.92:0/130662631 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7fdc06e140 0x7f7fdc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:42.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.673+0000 7f7ff971b700 1 --2- 172.21.15.92:0/130662631 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7fdc06e140 0x7f7fdc0705f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7fe800b3b0 tx=0x7f7fe8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:42.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.677+0000 7f7ff27fc700 1 -- 172.21.15.92:0/130662631 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7fec056d40 con 0x7f7ff4102de0 2024-05-06T12:25:42.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.959+0000 7f7ffc180700 1 -- 172.21.15.92:0/130662631 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7ff411a390 con 0x7f7ff4102de0 2024-05-06T12:25:42.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.959+0000 7f7ff27fc700 1 -- 172.21.15.92:0/130662631 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f7fec05a400 con 0x7f7ff4102de0 2024-05-06T12:25:42.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.962+0000 7f7ffc180700 1 -- 172.21.15.92:0/130662631 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7fdc06e140 msgr2=0x7f7fdc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:42.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.962+0000 7f7ffc180700 1 --2- 172.21.15.92:0/130662631 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7fdc06e140 0x7f7fdc0705f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7fe800b3b0 tx=0x7f7fe8009e30 comp rx=0 tx=0).stop 2024-05-06T12:25:42.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.962+0000 7f7ffc180700 1 -- 172.21.15.92:0/130662631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ff4102de0 msgr2=0x7f7ff4070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:42.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.962+0000 7f7ffc180700 1 --2- 172.21.15.92:0/130662631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ff4102de0 0x7f7ff4070260 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7fec001d80 tx=0x7f7fec009ec0 comp rx=0 tx=0).stop 2024-05-06T12:25:42.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.962+0000 7f7ffc180700 1 -- 172.21.15.92:0/130662631 shutdown_connections 2024-05-06T12:25:42.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.962+0000 7f7ffc180700 1 --2- 172.21.15.92:0/130662631 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7fdc06e140 0x7f7fdc0705f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:42.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.962+0000 7f7ffc180700 1 --2- 172.21.15.92:0/130662631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ff4102de0 0x7f7ff4070260 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:42.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.962+0000 7f7ffc180700 1 --2- 172.21.15.92:0/130662631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ff4103780 0x7f7ff40707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:42.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.962+0000 7f7ffc180700 1 -- 172.21.15.92:0/130662631 >> 172.21.15.92:0/130662631 conn(0x7f7ff40f8990 msgr2=0x7f7ff40f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:42.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.962+0000 7f7ffc180700 1 -- 172.21.15.92:0/130662631 shutdown_connections 2024-05-06T12:25:42.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:42.962+0000 7f7ffc180700 1 -- 172.21.15.92:0/130662631 wait complete. 2024-05-06T12:25:42.974 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph version 19.0.0-2498-g70ce9539 (70ce9539b62a15b9de429c1f0e91a1e012b099aa) squid (dev)" 2024-05-06T12:25:43.221 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:43 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/130662631' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:25:43.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:43 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/130662631' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-06T12:25:43.631 DEBUG:teuthology.parallel:result is None 2024-05-06T12:25:43.632 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-06T12:25:43.644 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi092.front.sepia.ceph.com 2024-05-06T12:25:43.644 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-06T12:25:43.903 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/mon.smithi092/config 2024-05-06T12:25:44.344 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:44 smithi092 ceph-mon[91933]: pgmap v311: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:25:44.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:44 smithi190 ceph-mon[79744]: pgmap v311: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:25:45.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.725+0000 7f4b6b089700 1 -- 172.21.15.92:0/3763160898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b64102de0 msgr2=0x7f4b641031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.725+0000 7f4b6b089700 1 --2- 172.21.15.92:0/3763160898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b64102de0 0x7f4b641031b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c009ce0 tx=0x7f4b5c004d10 comp rx=0 tx=0).stop 2024-05-06T12:25:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.725+0000 7f4b6b089700 1 -- 172.21.15.92:0/3763160898 shutdown_connections 2024-05-06T12:25:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.725+0000 7f4b6b089700 1 --2- 172.21.15.92:0/3763160898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b64103780 0x7f4b640fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.725+0000 7f4b6b089700 1 --2- 172.21.15.92:0/3763160898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b64102de0 0x7f4b641031b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.725+0000 7f4b6b089700 1 -- 172.21.15.92:0/3763160898 >> 172.21.15.92:0/3763160898 conn(0x7f4b640f8990 msgr2=0x7f4b640fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.725+0000 7f4b6b089700 1 -- 172.21.15.92:0/3763160898 shutdown_connections 2024-05-06T12:25:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.725+0000 7f4b6b089700 1 -- 172.21.15.92:0/3763160898 wait complete. 2024-05-06T12:25:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.726+0000 7f4b6b089700 1 Processor -- start 2024-05-06T12:25:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.726+0000 7f4b6b089700 1 -- start start 2024-05-06T12:25:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.726+0000 7f4b6b089700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b64102de0 0x7f4b64113b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.726+0000 7f4b6b089700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b64103780 0x7f4b64114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:45.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.726+0000 7f4b6b089700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b641146b0 con 0x7f4b64102de0 2024-05-06T12:25:45.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.726+0000 7f4b6b089700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6410dc90 con 0x7f4b64103780 2024-05-06T12:25:45.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.726+0000 7f4b68e25700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b64102de0 0x7f4b64113b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:45.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.727+0000 7f4b68e25700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b64102de0 0x7f4b64113b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35762/0 (socket says 172.21.15.92:35762) 2024-05-06T12:25:45.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.727+0000 7f4b68e25700 1 -- 172.21.15.92:0/1059529848 learned_addr learned my addr 172.21.15.92:0/1059529848 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:45.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.727+0000 7f4b63fff700 1 --2- 172.21.15.92:0/1059529848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b64103780 0x7f4b64114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:45.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.727+0000 7f4b68e25700 1 -- 172.21.15.92:0/1059529848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b64103780 msgr2=0x7f4b64114060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:45.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.727+0000 7f4b68e25700 1 --2- 172.21.15.92:0/1059529848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b64103780 0x7f4b64114060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:45.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.727+0000 7f4b68e25700 1 -- 172.21.15.92:0/1059529848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b5c009940 con 0x7f4b64102de0 2024-05-06T12:25:45.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.727+0000 7f4b68e25700 1 --2- 172.21.15.92:0/1059529848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b64102de0 0x7f4b64113b20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c005010 tx=0x7f4b5c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:45.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.727+0000 7f4b617fa700 1 -- 172.21.15.92:0/1059529848 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b5c00e040 con 0x7f4b64102de0 2024-05-06T12:25:45.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.727+0000 7f4b6b089700 1 -- 172.21.15.92:0/1059529848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b6410df10 con 0x7f4b64102de0 2024-05-06T12:25:45.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.727+0000 7f4b617fa700 1 -- 172.21.15.92:0/1059529848 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b5c008e20 con 0x7f4b64102de0 2024-05-06T12:25:45.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.727+0000 7f4b617fa700 1 -- 172.21.15.92:0/1059529848 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b5c01c660 con 0x7f4b64102de0 2024-05-06T12:25:45.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.727+0000 7f4b6b089700 1 -- 172.21.15.92:0/1059529848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b6410e4d0 con 0x7f4b64102de0 2024-05-06T12:25:45.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.728+0000 7f4b6b089700 1 -- 172.21.15.92:0/1059529848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b641000f0 con 0x7f4b64102de0 2024-05-06T12:25:45.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.729+0000 7f4b617fa700 1 -- 172.21.15.92:0/1059529848 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4b5c012a70 con 0x7f4b64102de0 2024-05-06T12:25:45.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.729+0000 7f4b617fa700 1 --2- 172.21.15.92:0/1059529848 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4b4c06e140 0x7f4b4c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:45.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.729+0000 7f4b617fa700 1 -- 172.21.15.92:0/1059529848 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4b5c08f760 con 0x7f4b64102de0 2024-05-06T12:25:45.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.729+0000 7f4b63fff700 1 --2- 172.21.15.92:0/1059529848 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4b4c06e140 0x7f4b4c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:45.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.730+0000 7f4b63fff700 1 --2- 172.21.15.92:0/1059529848 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4b4c06e140 0x7f4b4c0705f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4b58007fd0 tx=0x7f4b58009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:45.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.734+0000 7f4b617fa700 1 -- 172.21.15.92:0/1059529848 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4b5c05c5c0 con 0x7f4b64102de0 2024-05-06T12:25:45.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.977+0000 7f4b6b089700 1 -- 172.21.15.92:0/1059529848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4b6410e0a0 con 0x7f4b64102de0 2024-05-06T12:25:45.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.977+0000 7f4b617fa700 1 -- 172.21.15.92:0/1059529848 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 42 v42) v1 ==== 76+0+1967 (secure 0 0 0) 0x7f4b6410e0a0 con 0x7f4b64102de0 2024-05-06T12:25:45.980 INFO:teuthology.orchestra.run.smithi092.stdout:e42 2024-05-06T12:25:45.980 INFO:teuthology.orchestra.run.smithi092.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-06T12:25:45.980 INFO:teuthology.orchestra.run.smithi092.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-06T12:25:45.980 INFO:teuthology.orchestra.run.smithi092.stdout:legacy client fscid: 1 2024-05-06T12:25:45.980 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:25:45.980 INFO:teuthology.orchestra.run.smithi092.stdout:Filesystem 'cephfs' (1) 2024-05-06T12:25:45.980 INFO:teuthology.orchestra.run.smithi092.stdout:fs_name cephfs 2024-05-06T12:25:45.980 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 42 2024-05-06T12:25:45.980 INFO:teuthology.orchestra.run.smithi092.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-06T12:25:45.980 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-05-06T12:11:44.952671+0000 2024-05-06T12:25:45.980 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2024-05-06T12:22:50.526300+0000 2024-05-06T12:25:45.980 INFO:teuthology.orchestra.run.smithi092.stdout:tableserver 0 2024-05-06T12:25:45.981 INFO:teuthology.orchestra.run.smithi092.stdout:root 0 2024-05-06T12:25:45.981 INFO:teuthology.orchestra.run.smithi092.stdout:session_timeout 60 2024-05-06T12:25:45.981 INFO:teuthology.orchestra.run.smithi092.stdout:session_autoclose 300 2024-05-06T12:25:45.981 INFO:teuthology.orchestra.run.smithi092.stdout:max_file_size 1099511627776 2024-05-06T12:25:45.981 INFO:teuthology.orchestra.run.smithi092.stdout:max_xattr_size 65536 2024-05-06T12:25:45.981 INFO:teuthology.orchestra.run.smithi092.stdout:required_client_features {} 2024-05-06T12:25:45.981 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure 0 2024-05-06T12:25:45.981 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure_osd_epoch 84 2024-05-06T12:25:45.981 INFO:teuthology.orchestra.run.smithi092.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-06T12:25:45.981 INFO:teuthology.orchestra.run.smithi092.stdout:max_mds 2 2024-05-06T12:25:45.981 INFO:teuthology.orchestra.run.smithi092.stdout:in 0,1 2024-05-06T12:25:45.981 INFO:teuthology.orchestra.run.smithi092.stdout:up {0=34438,1=34466} 2024-05-06T12:25:45.981 INFO:teuthology.orchestra.run.smithi092.stdout:failed 2024-05-06T12:25:45.981 INFO:teuthology.orchestra.run.smithi092.stdout:damaged 2024-05-06T12:25:45.981 INFO:teuthology.orchestra.run.smithi092.stdout:stopped 2024-05-06T12:25:45.982 INFO:teuthology.orchestra.run.smithi092.stdout:data_pools [3] 2024-05-06T12:25:45.982 INFO:teuthology.orchestra.run.smithi092.stdout:metadata_pool 2 2024-05-06T12:25:45.982 INFO:teuthology.orchestra.run.smithi092.stdout:inline_data enabled 2024-05-06T12:25:45.982 INFO:teuthology.orchestra.run.smithi092.stdout:balancer 2024-05-06T12:25:45.982 INFO:teuthology.orchestra.run.smithi092.stdout:bal_rank_mask -1 2024-05-06T12:25:45.982 INFO:teuthology.orchestra.run.smithi092.stdout:standby_count_wanted 1 2024-05-06T12:25:45.982 INFO:teuthology.orchestra.run.smithi092.stdout:qdb_cluster leader: 34438 members: 34466,34438 2024-05-06T12:25:45.982 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.ijsqgp{0:34438} state up:active seq 8 join_fscid=1 addr [v2:172.21.15.92:6826/4190573007,v1:172.21.15.92:6827/4190573007] compat {c=[1],r=[1],i=[1fff]}] 2024-05-06T12:25:45.982 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.byjvgf{0:44309} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.190:6824/1219274291,v1:172.21.15.190:6825/1219274291] compat {c=[1],r=[1],i=[1fff]}] 2024-05-06T12:25:45.982 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi092.rosbll{1:34466} state up:active seq 12 join_fscid=1 addr [v2:172.21.15.92:6828/2398082173,v1:172.21.15.92:6829/2398082173] compat {c=[1],r=[1],i=[1fff]}] 2024-05-06T12:25:45.982 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.cephfs.smithi190.idzoff{1:44329} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.190:6826/2486629325,v1:172.21.15.190:6827/2486629325] compat {c=[1],r=[1],i=[1fff]}] 2024-05-06T12:25:45.982 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:25:45.982 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:25:45.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.980+0000 7f4b6b089700 1 -- 172.21.15.92:0/1059529848 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4b4c06e140 msgr2=0x7f4b4c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:45.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.980+0000 7f4b6b089700 1 --2- 172.21.15.92:0/1059529848 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4b4c06e140 0x7f4b4c0705f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4b58007fd0 tx=0x7f4b58009040 comp rx=0 tx=0).stop 2024-05-06T12:25:45.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.980+0000 7f4b6b089700 1 -- 172.21.15.92:0/1059529848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b64102de0 msgr2=0x7f4b64113b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:45.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.980+0000 7f4b6b089700 1 --2- 172.21.15.92:0/1059529848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b64102de0 0x7f4b64113b20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c005010 tx=0x7f4b5c004d10 comp rx=0 tx=0).stop 2024-05-06T12:25:45.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.980+0000 7f4b6b089700 1 -- 172.21.15.92:0/1059529848 shutdown_connections 2024-05-06T12:25:45.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.980+0000 7f4b6b089700 1 --2- 172.21.15.92:0/1059529848 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4b4c06e140 0x7f4b4c0705f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:45.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.980+0000 7f4b6b089700 1 --2- 172.21.15.92:0/1059529848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b64102de0 0x7f4b64113b20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:45.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.980+0000 7f4b6b089700 1 --2- 172.21.15.92:0/1059529848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b64103780 0x7f4b64114060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:45.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.980+0000 7f4b6b089700 1 -- 172.21.15.92:0/1059529848 >> 172.21.15.92:0/1059529848 conn(0x7f4b640f8990 msgr2=0x7f4b64107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:45.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.980+0000 7f4b6b089700 1 -- 172.21.15.92:0/1059529848 shutdown_connections 2024-05-06T12:25:45.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:45.980+0000 7f4b6b089700 1 -- 172.21.15.92:0/1059529848 wait complete. 2024-05-06T12:25:45.984 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 42 2024-05-06T12:25:46.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:46 smithi092 ceph-mon[91933]: pgmap v312: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:25:46.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:46 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/1059529848' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:25:46.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:46 smithi190 ceph-mon[79744]: pgmap v312: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:25:46.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:46 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/1059529848' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-06T12:25:46.632 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-05-06T12:25:46.661 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-06T12:25:46.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.753+0000 7f7a6a6ae640 1 -- 172.21.15.92:0/2555302367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a64101a30 msgr2=0x7f7a640fb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:46.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.753+0000 7f7a6a6ae640 1 --2- 172.21.15.92:0/2555302367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a64101a30 0x7f7a640fb0c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7a540048d0 tx=0x7f7a54009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:46.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.753+0000 7f7a6a6ae640 1 -- 172.21.15.92:0/2555302367 shutdown_connections 2024-05-06T12:25:46.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.753+0000 7f7a6a6ae640 1 --2- 172.21.15.92:0/2555302367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a64101a30 0x7f7a640fb0c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:46.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.753+0000 7f7a6a6ae640 1 --2- 172.21.15.92:0/2555302367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a64101090 0x7f7a64101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:46.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.753+0000 7f7a6a6ae640 1 -- 172.21.15.92:0/2555302367 >> 172.21.15.92:0/2555302367 conn(0x7f7a640f6c20 msgr2=0x7f7a640f9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:46.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.753+0000 7f7a6a6ae640 1 -- 172.21.15.92:0/2555302367 shutdown_connections 2024-05-06T12:25:46.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.753+0000 7f7a6a6ae640 1 -- 172.21.15.92:0/2555302367 wait complete. 2024-05-06T12:25:46.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.754+0000 7f7a6a6ae640 1 Processor -- start 2024-05-06T12:25:46.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.754+0000 7f7a6a6ae640 1 -- start start 2024-05-06T12:25:46.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.754+0000 7f7a6a6ae640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a64101090 0x7f7a6410ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:46.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.754+0000 7f7a6a6ae640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a64101a30 0x7f7a6410e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:46.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.754+0000 7f7a6a6ae640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a64066490 con 0x7f7a64101a30 2024-05-06T12:25:46.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.754+0000 7f7a6a6ae640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6410e950 con 0x7f7a64101090 2024-05-06T12:25:46.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.754+0000 7f7a637fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a64101a30 0x7f7a6410e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:46.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.754+0000 7f7a637fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a64101a30 0x7f7a6410e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35782/0 (socket says 172.21.15.92:35782) 2024-05-06T12:25:46.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.754+0000 7f7a637fe640 1 -- 172.21.15.92:0/1598353901 learned_addr learned my addr 172.21.15.92:0/1598353901 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:46.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.755+0000 7f7a63fff640 1 --2- 172.21.15.92:0/1598353901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a64101090 0x7f7a6410ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:46.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.755+0000 7f7a637fe640 1 -- 172.21.15.92:0/1598353901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a64101090 msgr2=0x7f7a6410ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:46.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.755+0000 7f7a637fe640 1 --2- 172.21.15.92:0/1598353901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a64101090 0x7f7a6410ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:46.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.755+0000 7f7a637fe640 1 -- 172.21.15.92:0/1598353901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a54005040 con 0x7f7a64101a30 2024-05-06T12:25:46.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.755+0000 7f7a637fe640 1 --2- 172.21.15.92:0/1598353901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a64101a30 0x7f7a6410e410 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7a54004930 tx=0x7f7a54009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:46.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.755+0000 7f7a60ff9640 1 -- 172.21.15.92:0/1598353901 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a540094c0 con 0x7f7a64101a30 2024-05-06T12:25:46.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.755+0000 7f7a60ff9640 1 -- 172.21.15.92:0/1598353901 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a54032e50 con 0x7f7a64101a30 2024-05-06T12:25:46.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.755+0000 7f7a6a6ae640 1 -- 172.21.15.92:0/1598353901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a6410ebd0 con 0x7f7a64101a30 2024-05-06T12:25:46.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.755+0000 7f7a60ff9640 1 -- 172.21.15.92:0/1598353901 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a540433b0 con 0x7f7a64101a30 2024-05-06T12:25:46.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.755+0000 7f7a6a6ae640 1 -- 172.21.15.92:0/1598353901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a64117130 con 0x7f7a64101a30 2024-05-06T12:25:46.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.756+0000 7f7a6a6ae640 1 -- 172.21.15.92:0/1598353901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a640fc800 con 0x7f7a64101a30 2024-05-06T12:25:46.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.757+0000 7f7a60ff9640 1 -- 172.21.15.92:0/1598353901 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7a5402f070 con 0x7f7a64101a30 2024-05-06T12:25:46.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.757+0000 7f7a60ff9640 1 --2- 172.21.15.92:0/1598353901 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7a4c06e0b0 0x7f7a4c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:46.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.757+0000 7f7a63fff640 1 --2- 172.21.15.92:0/1598353901 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7a4c06e0b0 0x7f7a4c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:46.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.757+0000 7f7a60ff9640 1 -- 172.21.15.92:0/1598353901 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7a540af9f0 con 0x7f7a64101a30 2024-05-06T12:25:46.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.757+0000 7f7a63fff640 1 --2- 172.21.15.92:0/1598353901 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7a4c06e0b0 0x7f7a4c070560 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7a58004ed0 tx=0x7f7a5800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:46.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.762+0000 7f7a60ff9640 1 -- 172.21.15.92:0/1598353901 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7a54078d50 con 0x7f7a64101a30 2024-05-06T12:25:46.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.940+0000 7f7a6a6ae640 1 -- 172.21.15.92:0/1598353901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a6404bcc0 con 0x7f7a64101a30 2024-05-06T12:25:46.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.941+0000 7f7a60ff9640 1 -- 172.21.15.92:0/1598353901 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16615 (secure 0 0 0) 0x7f7a5407c410 con 0x7f7a64101a30 2024-05-06T12:25:46.944 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:25:46.944 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":86,"fsid":"aee1d912-0ba0-11ef-bc97-c7b262605968","created":"2024-05-06T12:04:51.220909+0000","modified":"2024-05-06T12:22:36.092329+0000","last_up_change":"2024-05-06T12:20:59.798275+0000","last_in_change":"2024-05-06T12:10:07.815689+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-06T12:09:12.129668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-05-06T12:11:43.666342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-05-06T12:11:44.183559+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b276435-4698-4017-9535-4be41a4e1379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3125171617},{"type":"v1","addr":"172.21.15.92:6803","nonce":3125171617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3125171617},{"type":"v1","addr":"172.21.15.92:6805","nonce":3125171617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3125171617},{"type":"v1","addr":"172.21.15.92:6809","nonce":3125171617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3125171617},{"type":"v1","addr":"172.21.15.92:6807","nonce":3125171617}]},"public_addr":"172.21.15.92:6803/3125171617","cluster_addr":"172.21.15.92:6805/3125171617","heartbeat_back_addr":"172.21.15.92:6809/3125171617","heartbeat_front_addr":"172.21.15.92:6807/3125171617","state":["exists","up"]},{"osd":1,"uuid":"21688119-d15f-4b31-8842-090cc539c76d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1875265240},{"type":"v1","addr":"172.21.15.92:6811","nonce":1875265240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1875265240},{"type":"v1","addr":"172.21.15.92:6813","nonce":1875265240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1875265240},{"type":"v1","addr":"172.21.15.92:6817","nonce":1875265240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1875265240},{"type":"v1","addr":"172.21.15.92:6815","nonce":1875265240}]},"public_addr":"172.21.15.92:6811/1875265240","cluster_addr":"172.21.15.92:6813/1875265240","heartbeat_back_addr":"172.21.15.92:6817/1875265240","heartbeat_front_addr":"172.21.15.92:6815/1875265240","state":["exists","up"]},{"osd":2,"uuid":"74d16dfb-b118-432d-827a-59706d69c393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":629593219},{"type":"v1","addr":"172.21.15.92:6819","nonce":629593219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":629593219},{"type":"v1","addr":"172.21.15.92:6821","nonce":629593219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":629593219},{"type":"v1","addr":"172.21.15.92:6825","nonce":629593219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":629593219},{"type":"v1","addr":"172.21.15.92:6823","nonce":629593219}]},"public_addr":"172.21.15.92:6819/629593219","cluster_addr":"172.21.15.92:6821/629593219","heartbeat_back_addr":"172.21.15.92:6825/629593219","heartbeat_front_addr":"172.21.15.92:6823/629593219","state":["exists","up"]},{"osd":3,"uuid":"5a9aa5ed-2502-4c73-a8ac-cb7d0259500b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":3441479977},{"type":"v1","addr":"172.21.15.190:6801","nonce":3441479977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":3441479977},{"type":"v1","addr":"172.21.15.190:6803","nonce":3441479977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":3441479977},{"type":"v1","addr":"172.21.15.190:6807","nonce":3441479977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":3441479977},{"type":"v1","addr":"172.21.15.190:6805","nonce":3441479977}]},"public_addr":"172.21.15.190:6801/3441479977","cluster_addr":"172.21.15.190:6803/3441479977","heartbeat_back_addr":"172.21.15.190:6807/3441479977","heartbeat_front_addr":"172.21.15.190:6805/3441479977","state":["exists","up"]},{"osd":4,"uuid":"d4039e4a-c758-4632-85d6-1b7e725c0293","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":839214066},{"type":"v1","addr":"172.21.15.190:6809","nonce":839214066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":839214066},{"type":"v1","addr":"172.21.15.190:6811","nonce":839214066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":839214066},{"type":"v1","addr":"172.21.15.190:6815","nonce":839214066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":839214066},{"type":"v1","addr":"172.21.15.190:6813","nonce":839214066}]},"public_addr":"172.21.15.190:6809/839214066","cluster_addr":"172.21.15.190:6811/839214066","heartbeat_back_addr":"172.21.15.190:6815/839214066","heartbeat_front_addr":"172.21.15.190:6813/839214066","state":["exists","up"]},{"osd":5,"uuid":"ec707eee-5f24-48bb-818a-547f804fc41b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":10700099},{"type":"v1","addr":"172.21.15.190:6817","nonce":10700099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":10700099},{"type":"v1","addr":"172.21.15.190:6819","nonce":10700099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":10700099},{"type":"v1","addr":"172.21.15.190:6823","nonce":10700099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":10700099},{"type":"v1","addr":"172.21.15.190:6821","nonce":10700099}]},"public_addr":"172.21.15.190:6817/10700099","cluster_addr":"172.21.15.190:6819/10700099","heartbeat_back_addr":"172.21.15.190:6823/10700099","heartbeat_front_addr":"172.21.15.190:6821/10700099","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:08:15.348582+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:08:42.571577+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:09:10.093700+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:09:34.257829+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:09:59.795522+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:10:25.131646+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:6828/1663253598":"2024-05-07T12:21:58.517697+0000","172.21.15.92:6801/648200129":"2024-05-07T12:16:38.777479+0000","172.21.15.92:6801/543287293":"2024-05-07T12:07:03.802107+0000","172.21.15.92:0/3693315250":"2024-05-07T12:05:56.546947+0000","172.21.15.92:0/3493498366":"2024-05-07T12:14:01.169339+0000","172.21.15.190:0/3060689070":"2024-05-07T12:15:47.314494+0000","172.21.15.92:0/2210153116":"2024-05-07T12:05:56.546947+0000","172.21.15.92:0/3417170577":"2024-05-07T12:14:01.169339+0000","172.21.15.92:0/1054764607":"2024-05-07T12:16:38.777479+0000","172.21.15.92:0/856987777":"2024-05-07T12:14:01.169339+0000","172.21.15.92:0/895013058":"2024-05-07T12:05:25.954232+0000","172.21.15.92:0/2440233745":"2024-05-07T12:07:03.802107+0000","172.21.15.92:6800/543287293":"2024-05-07T12:07:03.802107+0000","172.21.15.190:6829/3484477933":"2024-05-07T12:15:47.314494+0000","172.21.15.92:0/758782094":"2024-05-07T12:05:25.954232+0000","172.21.15.92:0/1552275072":"2024-05-07T12:07:03.802107+0000","172.21.15.92:6800/3731057893":"2024-05-07T12:05:56.546947+0000","172.21.15.92:0/1407601857":"2024-05-07T12:05:56.546947+0000","172.21.15.92:0/1345929606":"2024-05-07T12:16:38.777479+0000","172.21.15.92:6801/3045905943":"2024-05-07T12:14:01.169339+0000","172.21.15.92:6829/1663253598":"2024-05-07T12:21:58.517697+0000","172.21.15.190:0/2858276177":"2024-05-07T12:15:47.314494+0000","172.21.15.92:6827/4255634245":"2024-05-07T12:12:00.896508+0000","172.21.15.92:0/4121052357":"2024-05-07T12:14:01.169339+0000","172.21.15.92:0/997820112":"2024-05-07T12:05:25.954232+0000","172.21.15.92:6800/3656822751":"2024-05-07T12:05:25.954232+0000","172.21.15.92:0/1532977673":"2024-05-07T12:07:03.802107+0000","172.21.15.92:6801/3656822751":"2024-05-07T12:05:25.954232+0000","172.21.15.92:6801/3731057893":"2024-05-07T12:05:56.546947+0000","172.21.15.92:6826/4255634245":"2024-05-07T12:12:00.896508+0000","172.21.15.190:6828/3484477933":"2024-05-07T12:15:47.314494+0000","172.21.15.190:0/2419054582":"2024-05-07T12:15:47.314494+0000","172.21.15.92:0/418878214":"2024-05-07T12:16:38.777479+0000","172.21.15.92:0/2098378816":"2024-05-07T12:16:38.777479+0000","172.21.15.92:6800/3045905943":"2024-05-07T12:14:01.169339+0000","172.21.15.92:0/2150816131":"2024-05-07T12:16:38.777479+0000","172.21.15.190:0/2974973416":"2024-05-07T12:15:47.314494+0000","172.21.15.92:0/1862221479":"2024-05-07T12:16:38.777479+0000","172.21.15.92:6800/648200129":"2024-05-07T12:16:38.777479+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-06T12:25:46.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.943+0000 7f7a6a6ae640 1 -- 172.21.15.92:0/1598353901 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7a4c06e0b0 msgr2=0x7f7a4c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:46.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.944+0000 7f7a6a6ae640 1 --2- 172.21.15.92:0/1598353901 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7a4c06e0b0 0x7f7a4c070560 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7a58004ed0 tx=0x7f7a5800b040 comp rx=0 tx=0).stop 2024-05-06T12:25:46.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.944+0000 7f7a6a6ae640 1 -- 172.21.15.92:0/1598353901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a64101a30 msgr2=0x7f7a6410e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:46.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.944+0000 7f7a6a6ae640 1 --2- 172.21.15.92:0/1598353901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a64101a30 0x7f7a6410e410 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7a54004930 tx=0x7f7a54009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:46.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.944+0000 7f7a6a6ae640 1 -- 172.21.15.92:0/1598353901 shutdown_connections 2024-05-06T12:25:46.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.944+0000 7f7a6a6ae640 1 --2- 172.21.15.92:0/1598353901 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7a4c06e0b0 0x7f7a4c070560 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:46.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.944+0000 7f7a6a6ae640 1 --2- 172.21.15.92:0/1598353901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a64101a30 0x7f7a6410e410 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:46.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.944+0000 7f7a6a6ae640 1 --2- 172.21.15.92:0/1598353901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a64101090 0x7f7a6410ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:46.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.944+0000 7f7a6a6ae640 1 -- 172.21.15.92:0/1598353901 >> 172.21.15.92:0/1598353901 conn(0x7f7a640f6c20 msgr2=0x7f7a640f7bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:46.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.944+0000 7f7a6a6ae640 1 -- 172.21.15.92:0/1598353901 shutdown_connections 2024-05-06T12:25:46.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:46.944+0000 7f7a6a6ae640 1 -- 172.21.15.92:0/1598353901 wait complete. 2024-05-06T12:25:46.957 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-06T12:09:12.129668+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-05-06T12:11:43.666342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-05-06T12:11:44.183559+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-06T12:25:46.957 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-06T12:25:47.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.053+0000 7fc769c18640 1 -- 172.21.15.92:0/1762966337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc764102b40 msgr2=0x7fc7640fa860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:47.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.053+0000 7fc769c18640 1 --2- 172.21.15.92:0/1762966337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc764102b40 0x7fc7640fa860 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc7580048d0 tx=0x7fc758009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:47.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.053+0000 7fc769c18640 1 -- 172.21.15.92:0/1762966337 shutdown_connections 2024-05-06T12:25:47.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.053+0000 7fc769c18640 1 --2- 172.21.15.92:0/1762966337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc764102b40 0x7fc7640fa860 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:47.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.053+0000 7fc769c18640 1 --2- 172.21.15.92:0/1762966337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7641021a0 0x7fc764102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:47.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.053+0000 7fc769c18640 1 -- 172.21.15.92:0/1762966337 >> 172.21.15.92:0/1762966337 conn(0x7fc764071700 msgr2=0x7fc764071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:47.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.053+0000 7fc769c18640 1 -- 172.21.15.92:0/1762966337 shutdown_connections 2024-05-06T12:25:47.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.053+0000 7fc769c18640 1 -- 172.21.15.92:0/1762966337 wait complete. 2024-05-06T12:25:47.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.054+0000 7fc769c18640 1 Processor -- start 2024-05-06T12:25:47.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.054+0000 7fc769c18640 1 -- start start 2024-05-06T12:25:47.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.054+0000 7fc769c18640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7641021a0 0x7fc764112ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:47.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.054+0000 7fc769c18640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc764102b40 0x7fc764113220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:47.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.054+0000 7fc769c18640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc764113820 con 0x7fc764102b40 2024-05-06T12:25:47.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.054+0000 7fc769c18640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76410ce70 con 0x7fc7641021a0 2024-05-06T12:25:47.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.054+0000 7fc762ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc764102b40 0x7fc764113220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:47.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.054+0000 7fc762ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc764102b40 0x7fc764113220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35800/0 (socket says 172.21.15.92:35800) 2024-05-06T12:25:47.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.054+0000 7fc762ffd640 1 -- 172.21.15.92:0/3289578396 learned_addr learned my addr 172.21.15.92:0/3289578396 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:47.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.054+0000 7fc7637fe640 1 --2- 172.21.15.92:0/3289578396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7641021a0 0x7fc764112ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:47.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.054+0000 7fc762ffd640 1 -- 172.21.15.92:0/3289578396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7641021a0 msgr2=0x7fc764112ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:47.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.054+0000 7fc762ffd640 1 --2- 172.21.15.92:0/3289578396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7641021a0 0x7fc764112ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:47.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.054+0000 7fc762ffd640 1 -- 172.21.15.92:0/3289578396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc758005040 con 0x7fc764102b40 2024-05-06T12:25:47.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.054+0000 7fc7637fe640 1 --2- 172.21.15.92:0/3289578396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7641021a0 0x7fc764112ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-06T12:25:47.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.055+0000 7fc762ffd640 1 --2- 172.21.15.92:0/3289578396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc764102b40 0x7fc764113220 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc758031720 tx=0x7fc758009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:47.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.055+0000 7fc74bfff640 1 -- 172.21.15.92:0/3289578396 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7580094c0 con 0x7fc764102b40 2024-05-06T12:25:47.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.055+0000 7fc74bfff640 1 -- 172.21.15.92:0/3289578396 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc758032e50 con 0x7fc764102b40 2024-05-06T12:25:47.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.055+0000 7fc769c18640 1 -- 172.21.15.92:0/3289578396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc76410d0f0 con 0x7fc764102b40 2024-05-06T12:25:47.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.055+0000 7fc74bfff640 1 -- 172.21.15.92:0/3289578396 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc75803b400 con 0x7fc764102b40 2024-05-06T12:25:47.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.055+0000 7fc769c18640 1 -- 172.21.15.92:0/3289578396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc76410d610 con 0x7fc764102b40 2024-05-06T12:25:47.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.056+0000 7fc769c18640 1 -- 172.21.15.92:0/3289578396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7640652b0 con 0x7fc764102b40 2024-05-06T12:25:47.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.056+0000 7fc74bfff640 1 -- 172.21.15.92:0/3289578396 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc75802f070 con 0x7fc764102b40 2024-05-06T12:25:47.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.057+0000 7fc74bfff640 1 --2- 172.21.15.92:0/3289578396 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fc74c06e0b0 0x7fc74c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:47.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.057+0000 7fc74bfff640 1 -- 172.21.15.92:0/3289578396 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc7580b0130 con 0x7fc764102b40 2024-05-06T12:25:47.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.057+0000 7fc7637fe640 1 --2- 172.21.15.92:0/3289578396 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fc74c06e0b0 0x7fc74c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:47.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.057+0000 7fc7637fe640 1 --2- 172.21.15.92:0/3289578396 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fc74c06e0b0 0x7fc74c070560 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc754007fc0 tx=0x7fc754004d00 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:47.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.061+0000 7fc74bfff640 1 -- 172.21.15.92:0/3289578396 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc758079420 con 0x7fc764102b40 2024-05-06T12:25:47.221 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:47 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/1598353901' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-06T12:25:47.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.239+0000 7fc769c18640 1 -- 172.21.15.92:0/3289578396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc7641092e0 con 0x7fc764102b40 2024-05-06T12:25:47.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.239+0000 7fc74bfff640 1 -- 172.21.15.92:0/3289578396 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc75802d040 con 0x7fc764102b40 2024-05-06T12:25:47.241 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2024-05-06T12:25:47.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.241+0000 7fc769c18640 1 -- 172.21.15.92:0/3289578396 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fc74c06e0b0 msgr2=0x7fc74c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:47.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.241+0000 7fc769c18640 1 --2- 172.21.15.92:0/3289578396 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fc74c06e0b0 0x7fc74c070560 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc754007fc0 tx=0x7fc754004d00 comp rx=0 tx=0).stop 2024-05-06T12:25:47.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.241+0000 7fc769c18640 1 -- 172.21.15.92:0/3289578396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc764102b40 msgr2=0x7fc764113220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:47.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.241+0000 7fc769c18640 1 --2- 172.21.15.92:0/3289578396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc764102b40 0x7fc764113220 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc758031720 tx=0x7fc758009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:47.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.241+0000 7fc769c18640 1 -- 172.21.15.92:0/3289578396 shutdown_connections 2024-05-06T12:25:47.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.241+0000 7fc769c18640 1 --2- 172.21.15.92:0/3289578396 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fc74c06e0b0 0x7fc74c070560 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:47.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.241+0000 7fc769c18640 1 --2- 172.21.15.92:0/3289578396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc764102b40 0x7fc764113220 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:47.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.241+0000 7fc769c18640 1 --2- 172.21.15.92:0/3289578396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7641021a0 0x7fc764112ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:47.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.241+0000 7fc769c18640 1 -- 172.21.15.92:0/3289578396 >> 172.21.15.92:0/3289578396 conn(0x7fc764071700 msgr2=0x7fc764104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:47.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.242+0000 7fc769c18640 1 -- 172.21.15.92:0/3289578396 shutdown_connections 2024-05-06T12:25:47.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.242+0000 7fc769c18640 1 -- 172.21.15.92:0/3289578396 wait complete. 2024-05-06T12:25:47.255 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-05-06T12:25:47.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:47 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/1598353901' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-06T12:25:47.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.349+0000 7f60f6c6e640 1 -- 172.21.15.92:0/3699396235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60f0101a30 msgr2=0x7f60f00fb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:47.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.349+0000 7f60f6c6e640 1 --2- 172.21.15.92:0/3699396235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60f0101a30 0x7f60f00fb0c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f60dc0048d0 tx=0x7f60dc009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:47.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.349+0000 7f60f6c6e640 1 -- 172.21.15.92:0/3699396235 shutdown_connections 2024-05-06T12:25:47.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.349+0000 7f60f6c6e640 1 --2- 172.21.15.92:0/3699396235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60f0101a30 0x7f60f00fb0c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:47.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.349+0000 7f60f6c6e640 1 --2- 172.21.15.92:0/3699396235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60f0101090 0x7f60f0101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:47.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.349+0000 7f60f6c6e640 1 -- 172.21.15.92:0/3699396235 >> 172.21.15.92:0/3699396235 conn(0x7f60f00f6c20 msgr2=0x7f60f00f9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:47.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.349+0000 7f60f6c6e640 1 -- 172.21.15.92:0/3699396235 shutdown_connections 2024-05-06T12:25:47.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.349+0000 7f60f6c6e640 1 -- 172.21.15.92:0/3699396235 wait complete. 2024-05-06T12:25:47.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.349+0000 7f60f6c6e640 1 Processor -- start 2024-05-06T12:25:47.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.349+0000 7f60f6c6e640 1 -- start start 2024-05-06T12:25:47.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.350+0000 7f60f6c6e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60f0101090 0x7f60f010ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:47.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.350+0000 7f60f6c6e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60f0101a30 0x7f60f010e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:47.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.350+0000 7f60f6c6e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f0066490 con 0x7f60f0101a30 2024-05-06T12:25:47.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.350+0000 7f60f6c6e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f010e950 con 0x7f60f0101090 2024-05-06T12:25:47.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.350+0000 7f60effff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60f0101a30 0x7f60f010e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:47.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.350+0000 7f60effff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60f0101a30 0x7f60f010e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35820/0 (socket says 172.21.15.92:35820) 2024-05-06T12:25:47.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.350+0000 7f60effff640 1 -- 172.21.15.92:0/2210673954 learned_addr learned my addr 172.21.15.92:0/2210673954 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:47.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.350+0000 7f60f4a0a640 1 --2- 172.21.15.92:0/2210673954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60f0101090 0x7f60f010ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:47.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.350+0000 7f60effff640 1 -- 172.21.15.92:0/2210673954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60f0101090 msgr2=0x7f60f010ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:47.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.350+0000 7f60effff640 1 --2- 172.21.15.92:0/2210673954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60f0101090 0x7f60f010ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:47.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.350+0000 7f60effff640 1 -- 172.21.15.92:0/2210673954 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60dc005040 con 0x7f60f0101a30 2024-05-06T12:25:47.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.350+0000 7f60effff640 1 --2- 172.21.15.92:0/2210673954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60f0101a30 0x7f60f010e410 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f60dc034040 tx=0x7f60dc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:47.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.350+0000 7f60ed7fa640 1 -- 172.21.15.92:0/2210673954 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60dc0092b0 con 0x7f60f0101a30 2024-05-06T12:25:47.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.350+0000 7f60f6c6e640 1 -- 172.21.15.92:0/2210673954 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f010ebd0 con 0x7f60f0101a30 2024-05-06T12:25:47.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.350+0000 7f60ed7fa640 1 -- 172.21.15.92:0/2210673954 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f60dc03b400 con 0x7f60f0101a30 2024-05-06T12:25:47.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.351+0000 7f60ed7fa640 1 -- 172.21.15.92:0/2210673954 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60dc0435d0 con 0x7f60f0101a30 2024-05-06T12:25:47.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.351+0000 7f60f6c6e640 1 -- 172.21.15.92:0/2210673954 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f0117130 con 0x7f60f0101a30 2024-05-06T12:25:47.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.351+0000 7f60f6c6e640 1 -- 172.21.15.92:0/2210673954 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f00fc800 con 0x7f60f0101a30 2024-05-06T12:25:47.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.352+0000 7f60ed7fa640 1 -- 172.21.15.92:0/2210673954 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f60dc02f070 con 0x7f60f0101a30 2024-05-06T12:25:47.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.352+0000 7f60ed7fa640 1 --2- 172.21.15.92:0/2210673954 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f60e006e0b0 0x7f60e0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:47.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.352+0000 7f60ed7fa640 1 -- 172.21.15.92:0/2210673954 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f60dc0af910 con 0x7f60f0101a30 2024-05-06T12:25:47.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.352+0000 7f60f4a0a640 1 --2- 172.21.15.92:0/2210673954 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f60e006e0b0 0x7f60e0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:47.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.353+0000 7f60f4a0a640 1 --2- 172.21.15.92:0/2210673954 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f60e006e0b0 0x7f60e0070560 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f60e40049b0 tx=0x7f60e400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:47.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.357+0000 7f60ed7fa640 1 -- 172.21.15.92:0/2210673954 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f60dc078c40 con 0x7f60f0101a30 2024-05-06T12:25:47.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.530+0000 7f60f6c6e640 1 -- 172.21.15.92:0/2210673954 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f60f0108250 con 0x7f60f0101a30 2024-05-06T12:25:47.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.531+0000 7f60ed7fa640 1 -- 172.21.15.92:0/2210673954 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7f60dc07c300 con 0x7f60f0101a30 2024-05-06T12:25:47.533 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2024-05-06T12:25:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.533+0000 7f60f6c6e640 1 -- 172.21.15.92:0/2210673954 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f60e006e0b0 msgr2=0x7f60e0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.533+0000 7f60f6c6e640 1 --2- 172.21.15.92:0/2210673954 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f60e006e0b0 0x7f60e0070560 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f60e40049b0 tx=0x7f60e400b040 comp rx=0 tx=0).stop 2024-05-06T12:25:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.533+0000 7f60f6c6e640 1 -- 172.21.15.92:0/2210673954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60f0101a30 msgr2=0x7f60f010e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.533+0000 7f60f6c6e640 1 --2- 172.21.15.92:0/2210673954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60f0101a30 0x7f60f010e410 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f60dc034040 tx=0x7f60dc009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:47.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.533+0000 7f60f6c6e640 1 -- 172.21.15.92:0/2210673954 shutdown_connections 2024-05-06T12:25:47.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.533+0000 7f60f6c6e640 1 --2- 172.21.15.92:0/2210673954 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f60e006e0b0 0x7f60e0070560 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:47.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.533+0000 7f60f6c6e640 1 --2- 172.21.15.92:0/2210673954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60f0101a30 0x7f60f010e410 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:47.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.533+0000 7f60f6c6e640 1 --2- 172.21.15.92:0/2210673954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60f0101090 0x7f60f010ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:47.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.533+0000 7f60f6c6e640 1 -- 172.21.15.92:0/2210673954 >> 172.21.15.92:0/2210673954 conn(0x7f60f00f6c20 msgr2=0x7f60f00f7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:47.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.533+0000 7f60f6c6e640 1 -- 172.21.15.92:0/2210673954 shutdown_connections 2024-05-06T12:25:47.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.533+0000 7f60f6c6e640 1 -- 172.21.15.92:0/2210673954 wait complete. 2024-05-06T12:25:47.546 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-05-06T12:25:47.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.640+0000 7fedae06e640 1 -- 172.21.15.92:0/3215283956 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feda8102190 msgr2=0x7feda8102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:47.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.640+0000 7fedae06e640 1 --2- 172.21.15.92:0/3215283956 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feda8102190 0x7feda8102560 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fed980048d0 tx=0x7fed98009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:47.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.641+0000 7fedae06e640 1 -- 172.21.15.92:0/3215283956 shutdown_connections 2024-05-06T12:25:47.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.641+0000 7fedae06e640 1 --2- 172.21.15.92:0/3215283956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feda8102b30 0x7feda80fc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:47.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.641+0000 7fedae06e640 1 --2- 172.21.15.92:0/3215283956 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feda8102190 0x7feda8102560 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:47.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.641+0000 7fedae06e640 1 -- 172.21.15.92:0/3215283956 >> 172.21.15.92:0/3215283956 conn(0x7feda80f7de0 msgr2=0x7feda80fa1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:47.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.641+0000 7fedae06e640 1 -- 172.21.15.92:0/3215283956 shutdown_connections 2024-05-06T12:25:47.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.641+0000 7fedae06e640 1 -- 172.21.15.92:0/3215283956 wait complete. 2024-05-06T12:25:47.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.641+0000 7fedae06e640 1 Processor -- start 2024-05-06T12:25:47.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.641+0000 7fedae06e640 1 -- start start 2024-05-06T12:25:47.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.641+0000 7fedae06e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feda8102190 0x7feda8112cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:47.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.641+0000 7fedae06e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feda8102b30 0x7feda8113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:47.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.641+0000 7fedae06e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda8113810 con 0x7feda8102b30 2024-05-06T12:25:47.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.641+0000 7fedae06e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda810ce60 con 0x7feda8102190 2024-05-06T12:25:47.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.641+0000 7fed9ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feda8102b30 0x7feda8113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:47.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.641+0000 7fed9ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feda8102b30 0x7feda8113210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35848/0 (socket says 172.21.15.92:35848) 2024-05-06T12:25:47.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.641+0000 7fed9ffff640 1 -- 172.21.15.92:0/547791025 learned_addr learned my addr 172.21.15.92:0/547791025 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:47.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.641+0000 7feda77fe640 1 --2- 172.21.15.92:0/547791025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feda8102190 0x7feda8112cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:47.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.642+0000 7fed9ffff640 1 -- 172.21.15.92:0/547791025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feda8102190 msgr2=0x7feda8112cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:47.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.642+0000 7fed9ffff640 1 --2- 172.21.15.92:0/547791025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feda8102190 0x7feda8112cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:47.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.642+0000 7fed9ffff640 1 -- 172.21.15.92:0/547791025 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed98005040 con 0x7feda8102b30 2024-05-06T12:25:47.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.642+0000 7fed9ffff640 1 --2- 172.21.15.92:0/547791025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feda8102b30 0x7feda8113210 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fed90002a50 tx=0x7fed90004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:47.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.642+0000 7feda4ff9640 1 -- 172.21.15.92:0/547791025 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed90007c30 con 0x7feda8102b30 2024-05-06T12:25:47.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.642+0000 7feda4ff9640 1 -- 172.21.15.92:0/547791025 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed9000f040 con 0x7feda8102b30 2024-05-06T12:25:47.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.642+0000 7fedae06e640 1 -- 172.21.15.92:0/547791025 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feda810d140 con 0x7feda8102b30 2024-05-06T12:25:47.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.642+0000 7feda4ff9640 1 -- 172.21.15.92:0/547791025 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed9000db50 con 0x7feda8102b30 2024-05-06T12:25:47.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.642+0000 7fedae06e640 1 -- 172.21.15.92:0/547791025 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda810d660 con 0x7feda8102b30 2024-05-06T12:25:47.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.643+0000 7fedae06e640 1 -- 172.21.15.92:0/547791025 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feda80fd900 con 0x7feda8102b30 2024-05-06T12:25:47.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.644+0000 7feda4ff9640 1 -- 172.21.15.92:0/547791025 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fed9001c400 con 0x7feda8102b30 2024-05-06T12:25:47.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.644+0000 7feda4ff9640 1 --2- 172.21.15.92:0/547791025 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fed9406e0b0 0x7fed94070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:47.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.644+0000 7feda4ff9640 1 -- 172.21.15.92:0/547791025 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fed9008d3c0 con 0x7feda8102b30 2024-05-06T12:25:47.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.644+0000 7feda77fe640 1 --2- 172.21.15.92:0/547791025 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fed9406e0b0 0x7fed94070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:47.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.645+0000 7feda77fe640 1 --2- 172.21.15.92:0/547791025 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fed9406e0b0 0x7fed94070560 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fed98003080 tx=0x7fed98009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:47.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.649+0000 7feda4ff9640 1 -- 172.21.15.92:0/547791025 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fed900567a0 con 0x7feda8102b30 2024-05-06T12:25:47.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.821+0000 7fedae06e640 1 -- 172.21.15.92:0/547791025 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7feda81092f0 con 0x7feda8102b30 2024-05-06T12:25:47.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.822+0000 7feda4ff9640 1 -- 172.21.15.92:0/547791025 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7fed90059e60 con 0x7feda8102b30 2024-05-06T12:25:47.824 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2024-05-06T12:25:47.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.824+0000 7fedae06e640 1 -- 172.21.15.92:0/547791025 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fed9406e0b0 msgr2=0x7fed94070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:47.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.824+0000 7fedae06e640 1 --2- 172.21.15.92:0/547791025 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fed9406e0b0 0x7fed94070560 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fed98003080 tx=0x7fed98009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:47.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.824+0000 7fedae06e640 1 -- 172.21.15.92:0/547791025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feda8102b30 msgr2=0x7feda8113210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:47.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.824+0000 7fedae06e640 1 --2- 172.21.15.92:0/547791025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feda8102b30 0x7feda8113210 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fed90002a50 tx=0x7fed90004600 comp rx=0 tx=0).stop 2024-05-06T12:25:47.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.824+0000 7fedae06e640 1 -- 172.21.15.92:0/547791025 shutdown_connections 2024-05-06T12:25:47.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.824+0000 7fedae06e640 1 --2- 172.21.15.92:0/547791025 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fed9406e0b0 0x7fed94070560 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:47.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.824+0000 7fedae06e640 1 --2- 172.21.15.92:0/547791025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feda8102b30 0x7feda8113210 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:47.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.824+0000 7fedae06e640 1 --2- 172.21.15.92:0/547791025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feda8102190 0x7feda8112cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:47.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.824+0000 7fedae06e640 1 -- 172.21.15.92:0/547791025 >> 172.21.15.92:0/547791025 conn(0x7feda80f7de0 msgr2=0x7feda80f8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:47.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.824+0000 7fedae06e640 1 -- 172.21.15.92:0/547791025 shutdown_connections 2024-05-06T12:25:47.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.824+0000 7fedae06e640 1 -- 172.21.15.92:0/547791025 wait complete. 2024-05-06T12:25:47.837 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-06T12:25:47.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.928+0000 7f3e6246e640 1 -- 172.21.15.92:0/448334632 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e5c102b40 msgr2=0x7f3e5c0fc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:47.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.928+0000 7f3e6246e640 1 --2- 172.21.15.92:0/448334632 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e5c102b40 0x7f3e5c0fc1d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3e500048d0 tx=0x7f3e50009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:47.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.928+0000 7f3e6246e640 1 -- 172.21.15.92:0/448334632 shutdown_connections 2024-05-06T12:25:47.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.928+0000 7f3e6246e640 1 --2- 172.21.15.92:0/448334632 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e5c102b40 0x7f3e5c0fc1d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:47.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.928+0000 7f3e6246e640 1 --2- 172.21.15.92:0/448334632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e5c1021a0 0x7f3e5c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:47.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.928+0000 7f3e6246e640 1 -- 172.21.15.92:0/448334632 >> 172.21.15.92:0/448334632 conn(0x7f3e5c0f7d30 msgr2=0x7f3e5c0fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:47.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.928+0000 7f3e6246e640 1 -- 172.21.15.92:0/448334632 shutdown_connections 2024-05-06T12:25:47.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.928+0000 7f3e6246e640 1 -- 172.21.15.92:0/448334632 wait complete. 2024-05-06T12:25:47.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.929+0000 7f3e6246e640 1 Processor -- start 2024-05-06T12:25:47.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.929+0000 7f3e6246e640 1 -- start start 2024-05-06T12:25:47.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.929+0000 7f3e6246e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e5c1021a0 0x7f3e5c112ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:47.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.929+0000 7f3e6246e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e5c102b40 0x7f3e5c1131e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:47.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.929+0000 7f3e6246e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c1137e0 con 0x7f3e5c1021a0 2024-05-06T12:25:47.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.929+0000 7f3e6246e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c10ce30 con 0x7f3e5c102b40 2024-05-06T12:25:47.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.929+0000 7f3e5bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e5c1021a0 0x7f3e5c112ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:47.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.929+0000 7f3e5bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e5c1021a0 0x7f3e5c112ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35852/0 (socket says 172.21.15.92:35852) 2024-05-06T12:25:47.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.929+0000 7f3e5bfff640 1 -- 172.21.15.92:0/1137993949 learned_addr learned my addr 172.21.15.92:0/1137993949 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:47.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.930+0000 7f3e5b7fe640 1 --2- 172.21.15.92:0/1137993949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e5c102b40 0x7f3e5c1131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:47.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.930+0000 7f3e5bfff640 1 -- 172.21.15.92:0/1137993949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e5c102b40 msgr2=0x7f3e5c1131e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:47.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.930+0000 7f3e5bfff640 1 --2- 172.21.15.92:0/1137993949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e5c102b40 0x7f3e5c1131e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:47.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.930+0000 7f3e5bfff640 1 -- 172.21.15.92:0/1137993949 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e50005040 con 0x7f3e5c1021a0 2024-05-06T12:25:47.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.930+0000 7f3e5b7fe640 1 --2- 172.21.15.92:0/1137993949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e5c102b40 0x7f3e5c1131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-06T12:25:47.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.930+0000 7f3e5bfff640 1 --2- 172.21.15.92:0/1137993949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e5c1021a0 0x7f3e5c112ca0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c002a50 tx=0x7f3e4c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:47.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.930+0000 7f3e58ff9640 1 -- 172.21.15.92:0/1137993949 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e4c007c30 con 0x7f3e5c1021a0 2024-05-06T12:25:47.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.930+0000 7f3e58ff9640 1 -- 172.21.15.92:0/1137993949 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e4c00f040 con 0x7f3e5c1021a0 2024-05-06T12:25:47.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.930+0000 7f3e6246e640 1 -- 172.21.15.92:0/1137993949 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e5c10d110 con 0x7f3e5c1021a0 2024-05-06T12:25:47.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.930+0000 7f3e58ff9640 1 -- 172.21.15.92:0/1137993949 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e4c00db50 con 0x7f3e5c1021a0 2024-05-06T12:25:47.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.930+0000 7f3e6246e640 1 -- 172.21.15.92:0/1137993949 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e5c10d630 con 0x7f3e5c1021a0 2024-05-06T12:25:47.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.931+0000 7f3e6246e640 1 -- 172.21.15.92:0/1137993949 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e5c0fd910 con 0x7f3e5c1021a0 2024-05-06T12:25:47.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.931+0000 7f3e58ff9640 1 -- 172.21.15.92:0/1137993949 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3e4c01c400 con 0x7f3e5c1021a0 2024-05-06T12:25:47.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.932+0000 7f3e58ff9640 1 --2- 172.21.15.92:0/1137993949 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f3e4406e0b0 0x7f3e44070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:47.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.932+0000 7f3e58ff9640 1 -- 172.21.15.92:0/1137993949 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f3e4c08d470 con 0x7f3e5c1021a0 2024-05-06T12:25:47.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.932+0000 7f3e5b7fe640 1 --2- 172.21.15.92:0/1137993949 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f3e4406e0b0 0x7f3e44070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:47.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.932+0000 7f3e5b7fe640 1 --2- 172.21.15.92:0/1137993949 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f3e4406e0b0 0x7f3e44070560 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3e50000c00 tx=0x7f3e50009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:47.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:47.936+0000 7f3e58ff9640 1 -- 172.21.15.92:0/1137993949 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3e4c0567a0 con 0x7f3e5c1021a0 2024-05-06T12:25:48.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.150+0000 7f3e6246e640 1 -- 172.21.15.92:0/1137993949 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e5c109300 con 0x7f3e5c1021a0 2024-05-06T12:25:48.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.150+0000 7f3e58ff9640 1 -- 172.21.15.92:0/1137993949 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+5251 (secure 0 0 0) 0x7f3e4c059e60 con 0x7f3e5c1021a0 2024-05-06T12:25:48.152 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:25:48.152 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-06T12:11:44.952671+0000","modified":"2024-05-06T12:22:50.526300+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34438,"mds_1":34466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34438":{"gid":34438,"name":"cephfs.smithi092.ijsqgp","rank":0,"incarnation":31,"state":"up:active","state_seq":8,"addr":"172.21.15.92:6827/4190573007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4190573007},{"type":"v1","addr":"172.21.15.92:6827","nonce":4190573007}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"cephfs.smithi092.rosbll","rank":1,"incarnation":40,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6829/2398082173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":2398082173},{"type":"v1","addr":"172.21.15.92:6829","nonce":2398082173}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44309":{"gid":44309,"name":"cephfs.smithi190.byjvgf","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6825/1219274291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1219274291},{"type":"v1","addr":"172.21.15.190:6825","nonce":1219274291}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44329":{"gid":44329,"name":"cephfs.smithi190.idzoff","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6827/2486629325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2486629325},{"type":"v1","addr":"172.21.15.190:6827","nonce":2486629325}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34438,"qdb_cluster":[34466,34438]},"id":1}]} 2024-05-06T12:25:48.153 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 42 2024-05-06T12:25:48.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.152+0000 7f3e6246e640 1 -- 172.21.15.92:0/1137993949 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f3e4406e0b0 msgr2=0x7f3e44070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:48.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.152+0000 7f3e6246e640 1 --2- 172.21.15.92:0/1137993949 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f3e4406e0b0 0x7f3e44070560 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3e50000c00 tx=0x7f3e50009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:48.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.152+0000 7f3e6246e640 1 -- 172.21.15.92:0/1137993949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e5c1021a0 msgr2=0x7f3e5c112ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:48.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.152+0000 7f3e6246e640 1 --2- 172.21.15.92:0/1137993949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e5c1021a0 0x7f3e5c112ca0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c002a50 tx=0x7f3e4c004600 comp rx=0 tx=0).stop 2024-05-06T12:25:48.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.152+0000 7f3e6246e640 1 -- 172.21.15.92:0/1137993949 shutdown_connections 2024-05-06T12:25:48.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.152+0000 7f3e6246e640 1 --2- 172.21.15.92:0/1137993949 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f3e4406e0b0 0x7f3e44070560 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:48.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.152+0000 7f3e6246e640 1 --2- 172.21.15.92:0/1137993949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e5c102b40 0x7f3e5c1131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:48.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.152+0000 7f3e6246e640 1 --2- 172.21.15.92:0/1137993949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e5c1021a0 0x7f3e5c112ca0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:48.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.152+0000 7f3e6246e640 1 -- 172.21.15.92:0/1137993949 >> 172.21.15.92:0/1137993949 conn(0x7f3e5c0f7d30 msgr2=0x7f3e5c0f8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:48.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.152+0000 7f3e6246e640 1 -- 172.21.15.92:0/1137993949 shutdown_connections 2024-05-06T12:25:48.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.152+0000 7f3e6246e640 1 -- 172.21.15.92:0/1137993949 wait complete. 2024-05-06T12:25:48.164 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:48 smithi092 ceph-mon[91933]: pgmap v313: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:25:48.164 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:48 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/3289578396' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-06T12:25:48.164 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:48 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/2210673954' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-06T12:25:48.165 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:48 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/547791025' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-06T12:25:48.165 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 18, 'max_mds': 2, 'flags': 50} 2024-05-06T12:25:48.166 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-05-06T12:25:48.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.258+0000 7f3865c65640 1 -- 172.21.15.92:0/826817016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38600fdb00 msgr2=0x7f38601061d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:48.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.258+0000 7f3865c65640 1 --2- 172.21.15.92:0/826817016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38600fdb00 0x7f38601061d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3850004900 tx=0x7f3850009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:48.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.258+0000 7f3865c65640 1 -- 172.21.15.92:0/826817016 shutdown_connections 2024-05-06T12:25:48.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.258+0000 7f3865c65640 1 --2- 172.21.15.92:0/826817016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38600fdb00 0x7f38601061d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:48.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.258+0000 7f3865c65640 1 --2- 172.21.15.92:0/826817016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38600fd160 0x7f38600fd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:48.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.258+0000 7f3865c65640 1 -- 172.21.15.92:0/826817016 >> 172.21.15.92:0/826817016 conn(0x7f38600f6be0 msgr2=0x7f38600f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:48.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.258+0000 7f3865c65640 1 -- 172.21.15.92:0/826817016 shutdown_connections 2024-05-06T12:25:48.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.258+0000 7f3865c65640 1 -- 172.21.15.92:0/826817016 wait complete. 2024-05-06T12:25:48.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.259+0000 7f3865c65640 1 Processor -- start 2024-05-06T12:25:48.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.259+0000 7f3865c65640 1 -- start start 2024-05-06T12:25:48.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.259+0000 7f3865c65640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38600fd160 0x7f3860112d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:48.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.259+0000 7f3865c65640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38600fdb00 0x7f38601132a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:48.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.259+0000 7f3865c65640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38601138a0 con 0x7f38600fdb00 2024-05-06T12:25:48.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.259+0000 7f3865c65640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386010cef0 con 0x7f38600fd160 2024-05-06T12:25:48.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.259+0000 7f385effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38600fdb00 0x7f38601132a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:48.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.259+0000 7f385effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38600fdb00 0x7f38601132a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35868/0 (socket says 172.21.15.92:35868) 2024-05-06T12:25:48.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.259+0000 7f385effd640 1 -- 172.21.15.92:0/3230244859 learned_addr learned my addr 172.21.15.92:0/3230244859 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:48.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.259+0000 7f385f7fe640 1 --2- 172.21.15.92:0/3230244859 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38600fd160 0x7f3860112d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:48.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.259+0000 7f385effd640 1 -- 172.21.15.92:0/3230244859 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38600fd160 msgr2=0x7f3860112d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:48.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.259+0000 7f385effd640 1 --2- 172.21.15.92:0/3230244859 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38600fd160 0x7f3860112d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:48.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.259+0000 7f385effd640 1 -- 172.21.15.92:0/3230244859 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3850005040 con 0x7f38600fdb00 2024-05-06T12:25:48.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.259+0000 7f385f7fe640 1 --2- 172.21.15.92:0/3230244859 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38600fd160 0x7f3860112d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-06T12:25:48.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.260+0000 7f385effd640 1 --2- 172.21.15.92:0/3230244859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38600fdb00 0x7f38601132a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3850000c00 tx=0x7f3850009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:48.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.260+0000 7f3847fff640 1 -- 172.21.15.92:0/3230244859 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38500094c0 con 0x7f38600fdb00 2024-05-06T12:25:48.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.260+0000 7f3847fff640 1 -- 172.21.15.92:0/3230244859 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f385003d920 con 0x7f38600fdb00 2024-05-06T12:25:48.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.260+0000 7f3847fff640 1 -- 172.21.15.92:0/3230244859 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f385003b930 con 0x7f38600fdb00 2024-05-06T12:25:48.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.260+0000 7f3865c65640 1 -- 172.21.15.92:0/3230244859 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f386010d170 con 0x7f38600fdb00 2024-05-06T12:25:48.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.260+0000 7f3865c65640 1 -- 172.21.15.92:0/3230244859 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f386010d690 con 0x7f38600fdb00 2024-05-06T12:25:48.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.261+0000 7f3865c65640 1 -- 172.21.15.92:0/3230244859 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3860066000 con 0x7f38600fdb00 2024-05-06T12:25:48.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.261+0000 7f3847fff640 1 -- 172.21.15.92:0/3230244859 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f385003da90 con 0x7f38600fdb00 2024-05-06T12:25:48.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.262+0000 7f3847fff640 1 --2- 172.21.15.92:0/3230244859 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f384806e0b0 0x7f3848070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:48.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.262+0000 7f3847fff640 1 -- 172.21.15.92:0/3230244859 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f38500b05a0 con 0x7f38600fdb00 2024-05-06T12:25:48.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.262+0000 7f385f7fe640 1 --2- 172.21.15.92:0/3230244859 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f384806e0b0 0x7f3848070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:48.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.262+0000 7f385f7fe640 1 --2- 172.21.15.92:0/3230244859 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f384806e0b0 0x7f3848070560 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f38540041d0 tx=0x7f3854009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:48.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.266+0000 7f3847fff640 1 -- 172.21.15.92:0/3230244859 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3850079990 con 0x7f38600fdb00 2024-05-06T12:25:48.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:48 smithi190 ceph-mon[79744]: pgmap v313: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:25:48.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:48 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/3289578396' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-06T12:25:48.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:48 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/2210673954' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-06T12:25:48.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:48 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/547791025' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-06T12:25:48.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.479+0000 7f3865c65640 1 -- 172.21.15.92:0/3230244859 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7f3860064350 con 0x7f38600fdb00 2024-05-06T12:25:48.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.480+0000 7f3847fff640 1 -- 172.21.15.92:0/3230244859 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v42) v1 ==== 107+0+4895 (secure 0 0 0) 0x7f385007d050 con 0x7f38600fdb00 2024-05-06T12:25:48.482 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:25:48.483 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-06T12:11:44.952671+0000","modified":"2024-05-06T12:13:52.244842+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14472,"mds_1":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14472":{"gid":14472,"name":"cephfs.smithi092.rosbll","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6829/1663253598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1663253598},{"type":"v1","addr":"172.21.15.92:6829","nonce":1663253598}]},"join_fscid":1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14484":{"gid":14484,"name":"cephfs.smithi092.ijsqgp","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6827/3613441054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3613441054},{"type":"v1","addr":"172.21.15.92:6827","nonce":3613441054}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24267":{"gid":24267,"name":"cephfs.smithi190.byjvgf","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.190:6825/1565151699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1565151699},{"type":"v1","addr":"172.21.15.190:6825","nonce":1565151699}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24277":{"gid":24277,"name":"cephfs.smithi190.idzoff","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.190:6827/2453728380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2453728380},{"type":"v1","addr":"172.21.15.190:6827","nonce":2453728380}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-06T12:25:48.483 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 19 2024-05-06T12:25:48.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.482+0000 7f3865c65640 1 -- 172.21.15.92:0/3230244859 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f384806e0b0 msgr2=0x7f3848070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:48.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.482+0000 7f3865c65640 1 --2- 172.21.15.92:0/3230244859 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f384806e0b0 0x7f3848070560 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f38540041d0 tx=0x7f3854009040 comp rx=0 tx=0).stop 2024-05-06T12:25:48.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.482+0000 7f3865c65640 1 -- 172.21.15.92:0/3230244859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38600fdb00 msgr2=0x7f38601132a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:48.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.482+0000 7f3865c65640 1 --2- 172.21.15.92:0/3230244859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38600fdb00 0x7f38601132a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3850000c00 tx=0x7f3850009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:48.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.482+0000 7f3865c65640 1 -- 172.21.15.92:0/3230244859 shutdown_connections 2024-05-06T12:25:48.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.482+0000 7f3865c65640 1 --2- 172.21.15.92:0/3230244859 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f384806e0b0 0x7f3848070560 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:48.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.482+0000 7f3865c65640 1 --2- 172.21.15.92:0/3230244859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38600fdb00 0x7f38601132a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:48.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.482+0000 7f3865c65640 1 --2- 172.21.15.92:0/3230244859 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38600fd160 0x7f3860112d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:48.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.482+0000 7f3865c65640 1 -- 172.21.15.92:0/3230244859 >> 172.21.15.92:0/3230244859 conn(0x7f38600f6be0 msgr2=0x7f38600f8050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:48.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.482+0000 7f3865c65640 1 -- 172.21.15.92:0/3230244859 shutdown_connections 2024-05-06T12:25:48.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.482+0000 7f3865c65640 1 -- 172.21.15.92:0/3230244859 wait complete. 2024-05-06T12:25:48.495 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-05-06T12:25:48.591 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.587+0000 7f52480ae640 1 -- 172.21.15.92:0/1947429838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52401021e0 msgr2=0x7f52401025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:48.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.587+0000 7f52480ae640 1 --2- 172.21.15.92:0/1947429838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52401021e0 0x7f52401025b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f523c0048d0 tx=0x7f523c009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:48.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.587+0000 7f52480ae640 1 -- 172.21.15.92:0/1947429838 shutdown_connections 2024-05-06T12:25:48.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.587+0000 7f52480ae640 1 --2- 172.21.15.92:0/1947429838 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5240102b80 0x7f52400fc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:48.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.587+0000 7f52480ae640 1 --2- 172.21.15.92:0/1947429838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52401021e0 0x7f52401025b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:48.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.587+0000 7f52480ae640 1 -- 172.21.15.92:0/1947429838 >> 172.21.15.92:0/1947429838 conn(0x7f52400f7db0 msgr2=0x7f52400fa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:48.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.587+0000 7f52480ae640 1 -- 172.21.15.92:0/1947429838 shutdown_connections 2024-05-06T12:25:48.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.587+0000 7f52480ae640 1 -- 172.21.15.92:0/1947429838 wait complete. 2024-05-06T12:25:48.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.588+0000 7f52480ae640 1 Processor -- start 2024-05-06T12:25:48.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.588+0000 7f52480ae640 1 -- start start 2024-05-06T12:25:48.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.588+0000 7f52480ae640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52401021e0 0x7f5240112ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:48.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.588+0000 7f52480ae640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5240102b80 0x7f52401131e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:48.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.588+0000 7f52480ae640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52401137e0 con 0x7f52401021e0 2024-05-06T12:25:48.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.588+0000 7f52480ae640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524010ce30 con 0x7f5240102b80 2024-05-06T12:25:48.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.588+0000 7f5245e4a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52401021e0 0x7f5240112ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:48.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.588+0000 7f5245e4a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52401021e0 0x7f5240112ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35892/0 (socket says 172.21.15.92:35892) 2024-05-06T12:25:48.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.588+0000 7f5245e4a640 1 -- 172.21.15.92:0/4139050108 learned_addr learned my addr 172.21.15.92:0/4139050108 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:48.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.588+0000 7f5245649640 1 --2- 172.21.15.92:0/4139050108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5240102b80 0x7f52401131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:48.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.588+0000 7f5245e4a640 1 -- 172.21.15.92:0/4139050108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5240102b80 msgr2=0x7f52401131e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:48.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.588+0000 7f5245e4a640 1 --2- 172.21.15.92:0/4139050108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5240102b80 0x7f52401131e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:48.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.588+0000 7f5245e4a640 1 -- 172.21.15.92:0/4139050108 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f523c005040 con 0x7f52401021e0 2024-05-06T12:25:48.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.588+0000 7f5245649640 1 --2- 172.21.15.92:0/4139050108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5240102b80 0x7f52401131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-06T12:25:48.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.589+0000 7f5245e4a640 1 --2- 172.21.15.92:0/4139050108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52401021e0 0x7f5240112ca0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f523c004930 tx=0x7f523c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:48.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.589+0000 7f52367fc640 1 -- 172.21.15.92:0/4139050108 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f523c0094c0 con 0x7f52401021e0 2024-05-06T12:25:48.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.589+0000 7f52367fc640 1 -- 172.21.15.92:0/4139050108 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f523c032e50 con 0x7f52401021e0 2024-05-06T12:25:48.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.589+0000 7f52480ae640 1 -- 172.21.15.92:0/4139050108 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f524010d0b0 con 0x7f52401021e0 2024-05-06T12:25:48.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.589+0000 7f52367fc640 1 -- 172.21.15.92:0/4139050108 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f523c0433b0 con 0x7f52401021e0 2024-05-06T12:25:48.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.589+0000 7f52480ae640 1 -- 172.21.15.92:0/4139050108 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f524010d570 con 0x7f52401021e0 2024-05-06T12:25:48.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.590+0000 7f52480ae640 1 -- 172.21.15.92:0/4139050108 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52400fd950 con 0x7f52401021e0 2024-05-06T12:25:48.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.590+0000 7f52367fc640 1 -- 172.21.15.92:0/4139050108 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f523c02f070 con 0x7f52401021e0 2024-05-06T12:25:48.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.590+0000 7f52367fc640 1 --2- 172.21.15.92:0/4139050108 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f522c06e0b0 0x7f522c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:48.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.591+0000 7f52367fc640 1 -- 172.21.15.92:0/4139050108 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f523c0afa40 con 0x7f52401021e0 2024-05-06T12:25:48.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.591+0000 7f5245649640 1 --2- 172.21.15.92:0/4139050108 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f522c06e0b0 0x7f522c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:48.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.591+0000 7f5245649640 1 --2- 172.21.15.92:0/4139050108 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f522c06e0b0 0x7f522c070560 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f524010e560 tx=0x7f523000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:48.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.595+0000 7f52367fc640 1 -- 172.21.15.92:0/4139050108 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f523c078d70 con 0x7f52401021e0 2024-05-06T12:25:48.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.807+0000 7f52480ae640 1 -- 172.21.15.92:0/4139050108 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f524010e310 con 0x7f52401021e0 2024-05-06T12:25:48.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.808+0000 7f52367fc640 1 -- 172.21.15.92:0/4139050108 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v42) v1 ==== 107+0+4896 (secure 0 0 0) 0x7f523c07c430 con 0x7f52401021e0 2024-05-06T12:25:48.810 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:25:48.811 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-06T12:11:44.952671+0000","modified":"2024-05-06T12:15:07.568167+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14472,"mds_1":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14472":{"gid":14472,"name":"cephfs.smithi092.rosbll","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6829/1663253598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1663253598},{"type":"v1","addr":"172.21.15.92:6829","nonce":1663253598}]},"join_fscid":1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14484":{"gid":14484,"name":"cephfs.smithi092.ijsqgp","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6827/3613441054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3613441054},{"type":"v1","addr":"172.21.15.92:6827","nonce":3613441054}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24267":{"gid":24267,"name":"cephfs.smithi190.byjvgf","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.190:6825/1565151699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1565151699},{"type":"v1","addr":"172.21.15.190:6825","nonce":1565151699}]},"join_fscid":1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24277":{"gid":24277,"name":"cephfs.smithi190.idzoff","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.190:6827/2453728380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2453728380},{"type":"v1","addr":"172.21.15.190:6827","nonce":2453728380}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-06T12:25:48.811 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 20 2024-05-06T12:25:48.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.810+0000 7f52480ae640 1 -- 172.21.15.92:0/4139050108 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f522c06e0b0 msgr2=0x7f522c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:48.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.810+0000 7f52480ae640 1 --2- 172.21.15.92:0/4139050108 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f522c06e0b0 0x7f522c070560 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f524010e560 tx=0x7f523000b040 comp rx=0 tx=0).stop 2024-05-06T12:25:48.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.810+0000 7f52480ae640 1 -- 172.21.15.92:0/4139050108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52401021e0 msgr2=0x7f5240112ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:48.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.810+0000 7f52480ae640 1 --2- 172.21.15.92:0/4139050108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52401021e0 0x7f5240112ca0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f523c004930 tx=0x7f523c009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:48.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.810+0000 7f52480ae640 1 -- 172.21.15.92:0/4139050108 shutdown_connections 2024-05-06T12:25:48.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.810+0000 7f52480ae640 1 --2- 172.21.15.92:0/4139050108 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f522c06e0b0 0x7f522c070560 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:48.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.810+0000 7f52480ae640 1 --2- 172.21.15.92:0/4139050108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5240102b80 0x7f52401131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:48.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.810+0000 7f52480ae640 1 --2- 172.21.15.92:0/4139050108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52401021e0 0x7f5240112ca0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:48.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.810+0000 7f52480ae640 1 -- 172.21.15.92:0/4139050108 >> 172.21.15.92:0/4139050108 conn(0x7f52400f7db0 msgr2=0x7f52400f9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:48.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.810+0000 7f52480ae640 1 -- 172.21.15.92:0/4139050108 shutdown_connections 2024-05-06T12:25:48.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.810+0000 7f52480ae640 1 -- 172.21.15.92:0/4139050108 wait complete. 2024-05-06T12:25:48.823 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-05-06T12:25:48.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.916+0000 7f98d859d640 1 -- 172.21.15.92:0/3189125708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98d0102b50 msgr2=0x7f98d00fc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:48.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.916+0000 7f98d859d640 1 --2- 172.21.15.92:0/3189125708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98d0102b50 0x7f98d00fc1e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f98c00048d0 tx=0x7f98c0009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:48.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.916+0000 7f98d859d640 1 -- 172.21.15.92:0/3189125708 shutdown_connections 2024-05-06T12:25:48.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.916+0000 7f98d859d640 1 --2- 172.21.15.92:0/3189125708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98d0102b50 0x7f98d00fc1e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:48.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.916+0000 7f98d859d640 1 --2- 172.21.15.92:0/3189125708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98d01021b0 0x7f98d0102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:48.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.916+0000 7f98d859d640 1 -- 172.21.15.92:0/3189125708 >> 172.21.15.92:0/3189125708 conn(0x7f98d00f7da0 msgr2=0x7f98d00fa1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:48.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.916+0000 7f98d859d640 1 -- 172.21.15.92:0/3189125708 shutdown_connections 2024-05-06T12:25:48.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.916+0000 7f98d859d640 1 -- 172.21.15.92:0/3189125708 wait complete. 2024-05-06T12:25:48.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.917+0000 7f98d859d640 1 Processor -- start 2024-05-06T12:25:48.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.917+0000 7f98d859d640 1 -- start start 2024-05-06T12:25:48.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.917+0000 7f98d859d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98d01021b0 0x7f98d0112c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:48.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.917+0000 7f98d859d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98d0102b50 0x7f98d01131c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:48.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.917+0000 7f98d859d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d01137c0 con 0x7f98d01021b0 2024-05-06T12:25:48.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.917+0000 7f98d859d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d010ce10 con 0x7f98d0102b50 2024-05-06T12:25:48.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.917+0000 7f98d6339640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98d01021b0 0x7f98d0112c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:48.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.917+0000 7f98d6339640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98d01021b0 0x7f98d0112c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35910/0 (socket says 172.21.15.92:35910) 2024-05-06T12:25:48.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.917+0000 7f98d6339640 1 -- 172.21.15.92:0/3870076728 learned_addr learned my addr 172.21.15.92:0/3870076728 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:48.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.917+0000 7f98d5b38640 1 --2- 172.21.15.92:0/3870076728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98d0102b50 0x7f98d01131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:48.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.917+0000 7f98d6339640 1 -- 172.21.15.92:0/3870076728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98d0102b50 msgr2=0x7f98d01131c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:48.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.917+0000 7f98d6339640 1 --2- 172.21.15.92:0/3870076728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98d0102b50 0x7f98d01131c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:48.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.917+0000 7f98d6339640 1 -- 172.21.15.92:0/3870076728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c0005040 con 0x7f98d01021b0 2024-05-06T12:25:48.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.917+0000 7f98d5b38640 1 --2- 172.21.15.92:0/3870076728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98d0102b50 0x7f98d01131c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-06T12:25:48.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.917+0000 7f98d6339640 1 --2- 172.21.15.92:0/3870076728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98d01021b0 0x7f98d0112c80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f98cc00c830 tx=0x7f98cc004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:48.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.917+0000 7f98c6ffd640 1 -- 172.21.15.92:0/3870076728 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98cc00a850 con 0x7f98d01021b0 2024-05-06T12:25:48.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.917+0000 7f98c6ffd640 1 -- 172.21.15.92:0/3870076728 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f98cc00ae80 con 0x7f98d01021b0 2024-05-06T12:25:48.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.917+0000 7f98d859d640 1 -- 172.21.15.92:0/3870076728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98d010d0f0 con 0x7f98d01021b0 2024-05-06T12:25:48.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.917+0000 7f98c6ffd640 1 -- 172.21.15.92:0/3870076728 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98cc012ea0 con 0x7f98d01021b0 2024-05-06T12:25:48.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.917+0000 7f98d859d640 1 -- 172.21.15.92:0/3870076728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98d010d610 con 0x7f98d01021b0 2024-05-06T12:25:48.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.918+0000 7f98d859d640 1 -- 172.21.15.92:0/3870076728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98d00fd920 con 0x7f98d01021b0 2024-05-06T12:25:48.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.919+0000 7f98c6ffd640 1 -- 172.21.15.92:0/3870076728 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f98cc012400 con 0x7f98d01021b0 2024-05-06T12:25:48.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.919+0000 7f98c6ffd640 1 --2- 172.21.15.92:0/3870076728 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f98bc06e0b0 0x7f98bc070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:48.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.919+0000 7f98c6ffd640 1 -- 172.21.15.92:0/3870076728 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f98cc08dde0 con 0x7f98d01021b0 2024-05-06T12:25:48.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.919+0000 7f98d5b38640 1 --2- 172.21.15.92:0/3870076728 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f98bc06e0b0 0x7f98bc070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:48.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.920+0000 7f98d5b38640 1 --2- 172.21.15.92:0/3870076728 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f98bc06e0b0 0x7f98bc070560 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f98c0000c00 tx=0x7f98c0009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:48.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:48.924+0000 7f98c6ffd640 1 -- 172.21.15.92:0/3870076728 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f98cc057190 con 0x7f98d01021b0 2024-05-06T12:25:49.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.137+0000 7f98d859d640 1 -- 172.21.15.92:0/3870076728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f98d0063f00 con 0x7f98d01021b0 2024-05-06T12:25:49.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.138+0000 7f98c6ffd640 1 -- 172.21.15.92:0/3870076728 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v42) v1 ==== 107+0+4910 (secure 0 0 0) 0x7f98cc0126b0 con 0x7f98d01021b0 2024-05-06T12:25:49.140 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:25:49.140 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-06T12:11:44.952671+0000","modified":"2024-05-06T12:18:36.975472+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14472,"mds_1":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14472":{"gid":14472,"name":"cephfs.smithi092.rosbll","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6829/1663253598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1663253598},{"type":"v1","addr":"172.21.15.92:6829","nonce":1663253598}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14484":{"gid":14484,"name":"cephfs.smithi092.ijsqgp","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6827/3613441054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3613441054},{"type":"v1","addr":"172.21.15.92:6827","nonce":3613441054}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24267":{"gid":24267,"name":"cephfs.smithi190.byjvgf","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.190:6825/1565151699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1565151699},{"type":"v1","addr":"172.21.15.190:6825","nonce":1565151699}]},"join_fscid":1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24277":{"gid":24277,"name":"cephfs.smithi190.idzoff","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.190:6827/2453728380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2453728380},{"type":"v1","addr":"172.21.15.190:6827","nonce":2453728380}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14472,"qdb_cluster":[14472,24267]},"id":1}]} 2024-05-06T12:25:49.140 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 21 2024-05-06T12:25:49.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.139+0000 7f98d859d640 1 -- 172.21.15.92:0/3870076728 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f98bc06e0b0 msgr2=0x7f98bc070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:49.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.139+0000 7f98d859d640 1 --2- 172.21.15.92:0/3870076728 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f98bc06e0b0 0x7f98bc070560 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f98c0000c00 tx=0x7f98c0009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:49.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.139+0000 7f98d859d640 1 -- 172.21.15.92:0/3870076728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98d01021b0 msgr2=0x7f98d0112c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:49.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.139+0000 7f98d859d640 1 --2- 172.21.15.92:0/3870076728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98d01021b0 0x7f98d0112c80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f98cc00c830 tx=0x7f98cc004430 comp rx=0 tx=0).stop 2024-05-06T12:25:49.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.139+0000 7f98d859d640 1 -- 172.21.15.92:0/3870076728 shutdown_connections 2024-05-06T12:25:49.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.139+0000 7f98d859d640 1 --2- 172.21.15.92:0/3870076728 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f98bc06e0b0 0x7f98bc070560 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:49.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.139+0000 7f98d859d640 1 --2- 172.21.15.92:0/3870076728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98d0102b50 0x7f98d01131c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:49.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.140+0000 7f98d859d640 1 --2- 172.21.15.92:0/3870076728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98d01021b0 0x7f98d0112c80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:49.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.140+0000 7f98d859d640 1 -- 172.21.15.92:0/3870076728 >> 172.21.15.92:0/3870076728 conn(0x7f98d00f7da0 msgr2=0x7f98d00f8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:49.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.140+0000 7f98d859d640 1 -- 172.21.15.92:0/3870076728 shutdown_connections 2024-05-06T12:25:49.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.140+0000 7f98d859d640 1 -- 172.21.15.92:0/3870076728 wait complete. 2024-05-06T12:25:49.152 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:49 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/1137993949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-06T12:25:49.152 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:49 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/3230244859' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-05-06T12:25:49.152 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:49 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/4139050108' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-05-06T12:25:49.153 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-05-06T12:25:49.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.245+0000 7fca11f9d640 1 -- 172.21.15.92:0/2356245693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca0c0faa50 msgr2=0x7fca0c0faea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:49.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.245+0000 7fca11f9d640 1 --2- 172.21.15.92:0/2356245693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca0c0faa50 0x7fca0c0faea0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fca000048d0 tx=0x7fca00009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:49.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.245+0000 7fca11f9d640 1 -- 172.21.15.92:0/2356245693 shutdown_connections 2024-05-06T12:25:49.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.245+0000 7fca11f9d640 1 --2- 172.21.15.92:0/2356245693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca0c0faa50 0x7fca0c0faea0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:49.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.245+0000 7fca11f9d640 1 --2- 172.21.15.92:0/2356245693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca0c102420 0x7fca0c1027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:49.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.245+0000 7fca11f9d640 1 -- 172.21.15.92:0/2356245693 >> 172.21.15.92:0/2356245693 conn(0x7fca0c071990 msgr2=0x7fca0c071d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:49.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.245+0000 7fca11f9d640 1 -- 172.21.15.92:0/2356245693 shutdown_connections 2024-05-06T12:25:49.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.245+0000 7fca11f9d640 1 -- 172.21.15.92:0/2356245693 wait complete. 2024-05-06T12:25:49.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.246+0000 7fca11f9d640 1 Processor -- start 2024-05-06T12:25:49.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.246+0000 7fca11f9d640 1 -- start start 2024-05-06T12:25:49.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.246+0000 7fca11f9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca0c0faa50 0x7fca0c112ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:49.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.246+0000 7fca11f9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca0c102420 0x7fca0c113430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:49.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.246+0000 7fca11f9d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c113a30 con 0x7fca0c102420 2024-05-06T12:25:49.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.246+0000 7fca11f9d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c10d080 con 0x7fca0c0faa50 2024-05-06T12:25:49.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.246+0000 7fca0affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca0c102420 0x7fca0c113430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:49.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.246+0000 7fca0affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca0c102420 0x7fca0c113430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35936/0 (socket says 172.21.15.92:35936) 2024-05-06T12:25:49.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.246+0000 7fca0affd640 1 -- 172.21.15.92:0/716081465 learned_addr learned my addr 172.21.15.92:0/716081465 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:49.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.246+0000 7fca0b7fe640 1 --2- 172.21.15.92:0/716081465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca0c0faa50 0x7fca0c112ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:49.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.247+0000 7fca0affd640 1 -- 172.21.15.92:0/716081465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca0c0faa50 msgr2=0x7fca0c112ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:49.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.247+0000 7fca0affd640 1 --2- 172.21.15.92:0/716081465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca0c0faa50 0x7fca0c112ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:49.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.247+0000 7fca0affd640 1 -- 172.21.15.92:0/716081465 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca00005040 con 0x7fca0c102420 2024-05-06T12:25:49.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.247+0000 7fca0affd640 1 --2- 172.21.15.92:0/716081465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca0c102420 0x7fca0c113430 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fca00034040 tx=0x7fca00009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:49.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.247+0000 7fc9f3fff640 1 -- 172.21.15.92:0/716081465 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca000092b0 con 0x7fca0c102420 2024-05-06T12:25:49.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.247+0000 7fca11f9d640 1 -- 172.21.15.92:0/716081465 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca0c10d300 con 0x7fca0c102420 2024-05-06T12:25:49.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.247+0000 7fc9f3fff640 1 -- 172.21.15.92:0/716081465 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca0003b400 con 0x7fca0c102420 2024-05-06T12:25:49.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.247+0000 7fc9f3fff640 1 -- 172.21.15.92:0/716081465 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca000435d0 con 0x7fca0c102420 2024-05-06T12:25:49.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.247+0000 7fca11f9d640 1 -- 172.21.15.92:0/716081465 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca0c10d7c0 con 0x7fca0c102420 2024-05-06T12:25:49.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.248+0000 7fca11f9d640 1 -- 172.21.15.92:0/716081465 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca0c064350 con 0x7fca0c102420 2024-05-06T12:25:49.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.248+0000 7fc9f3fff640 1 -- 172.21.15.92:0/716081465 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fca0002f070 con 0x7fca0c102420 2024-05-06T12:25:49.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.249+0000 7fc9f3fff640 1 --2- 172.21.15.92:0/716081465 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fc9f406e0b0 0x7fc9f4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:49.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.249+0000 7fc9f3fff640 1 -- 172.21.15.92:0/716081465 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fca000af7c0 con 0x7fca0c102420 2024-05-06T12:25:49.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.249+0000 7fca0b7fe640 1 --2- 172.21.15.92:0/716081465 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fc9f406e0b0 0x7fc9f4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:49.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.249+0000 7fca0b7fe640 1 --2- 172.21.15.92:0/716081465 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fc9f406e0b0 0x7fc9f4070560 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc009af0 tx=0x7fc9fc00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:49.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.253+0000 7fc9f3fff640 1 -- 172.21.15.92:0/716081465 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fca00078ba0 con 0x7fca0c102420 2024-05-06T12:25:49.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:49 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/1137993949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-06T12:25:49.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:49 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/3230244859' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-05-06T12:25:49.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:49 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/4139050108' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-05-06T12:25:49.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.464+0000 7fca11f9d640 1 -- 172.21.15.92:0/716081465 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7fca0c0638a0 con 0x7fca0c102420 2024-05-06T12:25:49.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.465+0000 7fc9f3fff640 1 -- 172.21.15.92:0/716081465 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v42) v1 ==== 107+0+4909 (secure 0 0 0) 0x7fca0007c260 con 0x7fca0c102420 2024-05-06T12:25:49.467 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:25:49.467 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-06T12:11:44.952671+0000","modified":"2024-05-06T12:19:06.643003+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14472,"mds_1":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14472":{"gid":14472,"name":"cephfs.smithi092.rosbll","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6829/1663253598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1663253598},{"type":"v1","addr":"172.21.15.92:6829","nonce":1663253598}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14484":{"gid":14484,"name":"cephfs.smithi092.ijsqgp","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6827/3613441054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3613441054},{"type":"v1","addr":"172.21.15.92:6827","nonce":3613441054}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24267":{"gid":24267,"name":"cephfs.smithi190.byjvgf","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.190:6825/1565151699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1565151699},{"type":"v1","addr":"172.21.15.190:6825","nonce":1565151699}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24277":{"gid":24277,"name":"cephfs.smithi190.idzoff","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.190:6827/2453728380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2453728380},{"type":"v1","addr":"172.21.15.190:6827","nonce":2453728380}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14472,"qdb_cluster":[24267,14472]},"id":1}]} 2024-05-06T12:25:49.467 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 22 2024-05-06T12:25:49.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.467+0000 7fca11f9d640 1 -- 172.21.15.92:0/716081465 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fc9f406e0b0 msgr2=0x7fc9f4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:49.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.467+0000 7fca11f9d640 1 --2- 172.21.15.92:0/716081465 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fc9f406e0b0 0x7fc9f4070560 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc009af0 tx=0x7fc9fc00b040 comp rx=0 tx=0).stop 2024-05-06T12:25:49.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.467+0000 7fca11f9d640 1 -- 172.21.15.92:0/716081465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca0c102420 msgr2=0x7fca0c113430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:49.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.467+0000 7fca11f9d640 1 --2- 172.21.15.92:0/716081465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca0c102420 0x7fca0c113430 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fca00034040 tx=0x7fca00009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:49.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.467+0000 7fca11f9d640 1 -- 172.21.15.92:0/716081465 shutdown_connections 2024-05-06T12:25:49.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.467+0000 7fca11f9d640 1 --2- 172.21.15.92:0/716081465 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fc9f406e0b0 0x7fc9f4070560 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:49.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.467+0000 7fca11f9d640 1 --2- 172.21.15.92:0/716081465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca0c102420 0x7fca0c113430 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:49.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.467+0000 7fca11f9d640 1 --2- 172.21.15.92:0/716081465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca0c0faa50 0x7fca0c112ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:49.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.467+0000 7fca11f9d640 1 -- 172.21.15.92:0/716081465 >> 172.21.15.92:0/716081465 conn(0x7fca0c071990 msgr2=0x7fca0c0f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:49.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.467+0000 7fca11f9d640 1 -- 172.21.15.92:0/716081465 shutdown_connections 2024-05-06T12:25:49.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.467+0000 7fca11f9d640 1 -- 172.21.15.92:0/716081465 wait complete. 2024-05-06T12:25:49.480 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-05-06T12:25:49.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.572+0000 7f67d019d640 1 -- 172.21.15.92:0/1962908675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67c80fd160 msgr2=0x7f67c80fd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:49.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.572+0000 7f67d019d640 1 --2- 172.21.15.92:0/1962908675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67c80fd160 0x7f67c80fd530 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f67c4004900 tx=0x7f67c4009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:49.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.572+0000 7f67d019d640 1 -- 172.21.15.92:0/1962908675 shutdown_connections 2024-05-06T12:25:49.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.572+0000 7f67d019d640 1 --2- 172.21.15.92:0/1962908675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67c80fdb00 0x7f67c81061d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:49.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.572+0000 7f67d019d640 1 --2- 172.21.15.92:0/1962908675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67c80fd160 0x7f67c80fd530 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:49.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.572+0000 7f67d019d640 1 -- 172.21.15.92:0/1962908675 >> 172.21.15.92:0/1962908675 conn(0x7f67c80f6be0 msgr2=0x7f67c80f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:49.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.572+0000 7f67d019d640 1 -- 172.21.15.92:0/1962908675 shutdown_connections 2024-05-06T12:25:49.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.572+0000 7f67d019d640 1 -- 172.21.15.92:0/1962908675 wait complete. 2024-05-06T12:25:49.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.573+0000 7f67d019d640 1 Processor -- start 2024-05-06T12:25:49.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.573+0000 7f67d019d640 1 -- start start 2024-05-06T12:25:49.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.573+0000 7f67d019d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67c80fd160 0x7f67c8112d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:49.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.573+0000 7f67d019d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67c80fdb00 0x7f67c8113270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:49.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.573+0000 7f67d019d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c8113870 con 0x7f67c80fd160 2024-05-06T12:25:49.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.573+0000 7f67d019d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c810cec0 con 0x7f67c80fdb00 2024-05-06T12:25:49.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.573+0000 7f67cdf39640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67c80fd160 0x7f67c8112d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:49.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.573+0000 7f67cdf39640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67c80fd160 0x7f67c8112d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35958/0 (socket says 172.21.15.92:35958) 2024-05-06T12:25:49.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.573+0000 7f67cdf39640 1 -- 172.21.15.92:0/647659855 learned_addr learned my addr 172.21.15.92:0/647659855 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:49.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.573+0000 7f67cd738640 1 --2- 172.21.15.92:0/647659855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67c80fdb00 0x7f67c8113270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:49.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.573+0000 7f67cdf39640 1 -- 172.21.15.92:0/647659855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67c80fdb00 msgr2=0x7f67c8113270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:49.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.573+0000 7f67cdf39640 1 --2- 172.21.15.92:0/647659855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67c80fdb00 0x7f67c8113270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:49.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.573+0000 7f67cdf39640 1 -- 172.21.15.92:0/647659855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c4005040 con 0x7f67c80fd160 2024-05-06T12:25:49.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.574+0000 7f67cdf39640 1 --2- 172.21.15.92:0/647659855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67c80fd160 0x7f67c8112d30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f67c4000c00 tx=0x7f67c4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:49.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.574+0000 7f67be7fc640 1 -- 172.21.15.92:0/647659855 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67c40094c0 con 0x7f67c80fd160 2024-05-06T12:25:49.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.574+0000 7f67be7fc640 1 -- 172.21.15.92:0/647659855 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f67c403d920 con 0x7f67c80fd160 2024-05-06T12:25:49.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.574+0000 7f67d019d640 1 -- 172.21.15.92:0/647659855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c810d140 con 0x7f67c80fd160 2024-05-06T12:25:49.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.574+0000 7f67be7fc640 1 -- 172.21.15.92:0/647659855 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67c403b8c0 con 0x7f67c80fd160 2024-05-06T12:25:49.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.574+0000 7f67d019d640 1 -- 172.21.15.92:0/647659855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c810d660 con 0x7f67c80fd160 2024-05-06T12:25:49.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.575+0000 7f67d019d640 1 -- 172.21.15.92:0/647659855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67c8066000 con 0x7f67c80fd160 2024-05-06T12:25:49.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.575+0000 7f67be7fc640 1 -- 172.21.15.92:0/647659855 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f67c403da90 con 0x7f67c80fd160 2024-05-06T12:25:49.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.576+0000 7f67be7fc640 1 --2- 172.21.15.92:0/647659855 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f67b406e0b0 0x7f67b4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:49.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.576+0000 7f67cd738640 1 --2- 172.21.15.92:0/647659855 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f67b406e0b0 0x7f67b4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:49.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.576+0000 7f67be7fc640 1 -- 172.21.15.92:0/647659855 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f67c40b0340 con 0x7f67c80fd160 2024-05-06T12:25:49.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.576+0000 7f67cd738640 1 --2- 172.21.15.92:0/647659855 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f67b406e0b0 0x7f67b4070560 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f67c810e5f0 tx=0x7f67b800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:49.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.581+0000 7f67be7fc640 1 -- 172.21.15.92:0/647659855 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f67c4079730 con 0x7f67c80fd160 2024-05-06T12:25:49.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.791+0000 7f67d019d640 1 -- 172.21.15.92:0/647659855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f67c8064350 con 0x7f67c80fd160 2024-05-06T12:25:49.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.792+0000 7f67be7fc640 1 -- 172.21.15.92:0/647659855 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v42) v1 ==== 107+0+3334 (secure 0 0 0) 0x7f67c4036d20 con 0x7f67c80fd160 2024-05-06T12:25:49.794 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:25:49.794 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-06T12:11:44.952671+0000","modified":"2024-05-06T12:21:02.481001+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14472,"mds_1":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14472":{"gid":14472,"name":"cephfs.smithi092.rosbll","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6829/1663253598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1663253598},{"type":"v1","addr":"172.21.15.92:6829","nonce":1663253598}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24267":{"gid":24267,"name":"cephfs.smithi190.byjvgf","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.190:6825/1565151699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1565151699},{"type":"v1","addr":"172.21.15.190:6825","nonce":1565151699}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14472,"qdb_cluster":[14472,24267]},"id":1}]} 2024-05-06T12:25:49.794 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 23 2024-05-06T12:25:49.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.794+0000 7f67d019d640 1 -- 172.21.15.92:0/647659855 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f67b406e0b0 msgr2=0x7f67b4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:49.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.794+0000 7f67d019d640 1 --2- 172.21.15.92:0/647659855 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f67b406e0b0 0x7f67b4070560 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f67c810e5f0 tx=0x7f67b800b040 comp rx=0 tx=0).stop 2024-05-06T12:25:49.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.794+0000 7f67d019d640 1 -- 172.21.15.92:0/647659855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67c80fd160 msgr2=0x7f67c8112d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:49.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.794+0000 7f67d019d640 1 --2- 172.21.15.92:0/647659855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67c80fd160 0x7f67c8112d30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f67c4000c00 tx=0x7f67c4009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:49.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.794+0000 7f67d019d640 1 -- 172.21.15.92:0/647659855 shutdown_connections 2024-05-06T12:25:49.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.794+0000 7f67d019d640 1 --2- 172.21.15.92:0/647659855 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f67b406e0b0 0x7f67b4070560 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:49.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.794+0000 7f67d019d640 1 --2- 172.21.15.92:0/647659855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67c80fdb00 0x7f67c8113270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:49.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.794+0000 7f67d019d640 1 --2- 172.21.15.92:0/647659855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67c80fd160 0x7f67c8112d30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:49.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.794+0000 7f67d019d640 1 -- 172.21.15.92:0/647659855 >> 172.21.15.92:0/647659855 conn(0x7f67c80f6be0 msgr2=0x7f67c80f80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:49.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.794+0000 7f67d019d640 1 -- 172.21.15.92:0/647659855 shutdown_connections 2024-05-06T12:25:49.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.794+0000 7f67d019d640 1 -- 172.21.15.92:0/647659855 wait complete. 2024-05-06T12:25:49.807 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 23 2024-05-06T12:25:49.807 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-05-06T12:25:49.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.900+0000 7f25b1007640 1 -- 172.21.15.92:0/70816274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25ac102b60 msgr2=0x7f25ac0fa8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:49.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.900+0000 7f25b1007640 1 --2- 172.21.15.92:0/70816274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25ac102b60 0x7f25ac0fa8a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f25a00048d0 tx=0x7f25a0009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:49.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.900+0000 7f25b1007640 1 -- 172.21.15.92:0/70816274 shutdown_connections 2024-05-06T12:25:49.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.900+0000 7f25b1007640 1 --2- 172.21.15.92:0/70816274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25ac102b60 0x7f25ac0fa8a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:49.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.900+0000 7f25b1007640 1 --2- 172.21.15.92:0/70816274 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25ac102250 0x7f25ac102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:49.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.900+0000 7f25b1007640 1 -- 172.21.15.92:0/70816274 >> 172.21.15.92:0/70816274 conn(0x7f25ac070900 msgr2=0x7f25ac070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:49.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.900+0000 7f25b1007640 1 -- 172.21.15.92:0/70816274 shutdown_connections 2024-05-06T12:25:49.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.900+0000 7f25b1007640 1 -- 172.21.15.92:0/70816274 wait complete. 2024-05-06T12:25:49.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.901+0000 7f25b1007640 1 Processor -- start 2024-05-06T12:25:49.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.901+0000 7f25b1007640 1 -- start start 2024-05-06T12:25:49.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.901+0000 7f25b1007640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25ac102250 0x7f25ac112d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:49.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.901+0000 7f25b1007640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25ac102b60 0x7f25ac113290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:49.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.901+0000 7f25b1007640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac113920 con 0x7f25ac102b60 2024-05-06T12:25:49.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.901+0000 7f25b1007640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac10cee0 con 0x7f25ac102250 2024-05-06T12:25:49.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.901+0000 7f25aa59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25ac102b60 0x7f25ac113290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:49.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.901+0000 7f25aa59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25ac102b60 0x7f25ac113290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35974/0 (socket says 172.21.15.92:35974) 2024-05-06T12:25:49.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.901+0000 7f25aa59c640 1 -- 172.21.15.92:0/1620051236 learned_addr learned my addr 172.21.15.92:0/1620051236 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:49.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.901+0000 7f25aad9d640 1 --2- 172.21.15.92:0/1620051236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25ac102250 0x7f25ac112d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:49.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.901+0000 7f25aa59c640 1 -- 172.21.15.92:0/1620051236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25ac102250 msgr2=0x7f25ac112d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:49.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.901+0000 7f25aa59c640 1 --2- 172.21.15.92:0/1620051236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25ac102250 0x7f25ac112d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:49.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.901+0000 7f25aa59c640 1 -- 172.21.15.92:0/1620051236 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a0005040 con 0x7f25ac102b60 2024-05-06T12:25:49.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.902+0000 7f25aa59c640 1 --2- 172.21.15.92:0/1620051236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25ac102b60 0x7f25ac113290 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f25a0009fa0 tx=0x7f25a0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:49.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.902+0000 7f25937fe640 1 -- 172.21.15.92:0/1620051236 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25a0030870 con 0x7f25ac102b60 2024-05-06T12:25:49.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.902+0000 7f25b1007640 1 -- 172.21.15.92:0/1620051236 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25ac10d160 con 0x7f25ac102b60 2024-05-06T12:25:49.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.902+0000 7f25937fe640 1 -- 172.21.15.92:0/1620051236 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f25a00092e0 con 0x7f25ac102b60 2024-05-06T12:25:49.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.902+0000 7f25937fe640 1 -- 172.21.15.92:0/1620051236 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25a0043640 con 0x7f25ac102b60 2024-05-06T12:25:49.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.902+0000 7f25b1007640 1 -- 172.21.15.92:0/1620051236 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25ac10d680 con 0x7f25ac102b60 2024-05-06T12:25:49.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.903+0000 7f25b1007640 1 -- 172.21.15.92:0/1620051236 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25ac0654f0 con 0x7f25ac102b60 2024-05-06T12:25:49.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.903+0000 7f25937fe640 1 -- 172.21.15.92:0/1620051236 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f25a00438b0 con 0x7f25ac102b60 2024-05-06T12:25:49.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.904+0000 7f25937fe640 1 --2- 172.21.15.92:0/1620051236 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f259406e0b0 0x7f2594070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:49.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.904+0000 7f25937fe640 1 -- 172.21.15.92:0/1620051236 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f25a00b0920 con 0x7f25ac102b60 2024-05-06T12:25:49.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.904+0000 7f25aad9d640 1 --2- 172.21.15.92:0/1620051236 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f259406e0b0 0x7f2594070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:49.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.904+0000 7f25aad9d640 1 --2- 172.21.15.92:0/1620051236 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f259406e0b0 0x7f2594070560 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f259c00a900 tx=0x7f259c008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:49.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:49.908+0000 7f25937fe640 1 -- 172.21.15.92:0/1620051236 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f25a0079cd0 con 0x7f25ac102b60 2024-05-06T12:25:50.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.123+0000 7f25b1007640 1 -- 172.21.15.92:0/1620051236 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f25ac109400 con 0x7f25ac102b60 2024-05-06T12:25:50.126 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.124+0000 7f25937fe640 1 -- 172.21.15.92:0/1620051236 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v42) v1 ==== 107+0+4894 (secure 0 0 0) 0x7f25a007d390 con 0x7f25ac102b60 2024-05-06T12:25:50.126 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:25:50.126 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34402,"name":"cephfs.smithi092.ijsqgp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6827/3131141280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3131141280},{"type":"v1","addr":"172.21.15.92:6827","nonce":3131141280}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44261,"name":"cephfs.smithi190.idzoff","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6827/1642657019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1642657019},{"type":"v1","addr":"172.21.15.190:6827","nonce":1642657019}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-06T12:11:44.952671+0000","modified":"2024-05-06T12:21:03.644853+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0,1],"up":{"mds_0":14472,"mds_1":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14472":{"gid":14472,"name":"cephfs.smithi092.rosbll","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6829/1663253598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1663253598},{"type":"v1","addr":"172.21.15.92:6829","nonce":1663253598}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24267":{"gid":24267,"name":"cephfs.smithi190.byjvgf","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.190:6825/1565151699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1565151699},{"type":"v1","addr":"172.21.15.190:6825","nonce":1565151699}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14472,"qdb_cluster":[24267,14472]},"id":1}]} 2024-05-06T12:25:50.126 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 24 2024-05-06T12:25:50.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.126+0000 7f25b1007640 1 -- 172.21.15.92:0/1620051236 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f259406e0b0 msgr2=0x7f2594070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:50.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.126+0000 7f25b1007640 1 --2- 172.21.15.92:0/1620051236 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f259406e0b0 0x7f2594070560 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f259c00a900 tx=0x7f259c008040 comp rx=0 tx=0).stop 2024-05-06T12:25:50.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.126+0000 7f25b1007640 1 -- 172.21.15.92:0/1620051236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25ac102b60 msgr2=0x7f25ac113290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:50.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.126+0000 7f25b1007640 1 --2- 172.21.15.92:0/1620051236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25ac102b60 0x7f25ac113290 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f25a0009fa0 tx=0x7f25a0009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:50.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.126+0000 7f25b1007640 1 -- 172.21.15.92:0/1620051236 shutdown_connections 2024-05-06T12:25:50.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.126+0000 7f25b1007640 1 --2- 172.21.15.92:0/1620051236 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f259406e0b0 0x7f2594070560 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:50.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.126+0000 7f25b1007640 1 --2- 172.21.15.92:0/1620051236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25ac102b60 0x7f25ac113290 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:50.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.126+0000 7f25b1007640 1 --2- 172.21.15.92:0/1620051236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25ac102250 0x7f25ac112d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:50.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.126+0000 7f25b1007640 1 -- 172.21.15.92:0/1620051236 >> 172.21.15.92:0/1620051236 conn(0x7f25ac070900 msgr2=0x7f25ac104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:50.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.126+0000 7f25b1007640 1 -- 172.21.15.92:0/1620051236 shutdown_connections 2024-05-06T12:25:50.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.126+0000 7f25b1007640 1 -- 172.21.15.92:0/1620051236 wait complete. 2024-05-06T12:25:50.138 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:50 smithi092 ceph-mon[91933]: pgmap v314: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:25:50.138 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:50 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/3870076728' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-05-06T12:25:50.138 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:50 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/716081465' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-05-06T12:25:50.138 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:50 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/647659855' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-05-06T12:25:50.139 DEBUG:tasks.fs:max_mds reduced in epoch 24 2024-05-06T12:25:50.139 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 24 2024-05-06T12:25:50.139 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-05-06T12:25:50.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.234+0000 7fedc8b9d640 1 -- 172.21.15.92:0/753773731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedc40fb150 msgr2=0x7fedc40fb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:50.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.234+0000 7fedc8b9d640 1 --2- 172.21.15.92:0/753773731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedc40fb150 0x7fedc40fb5c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fedb80048d0 tx=0x7fedb8009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:50.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.234+0000 7fedc8b9d640 1 -- 172.21.15.92:0/753773731 shutdown_connections 2024-05-06T12:25:50.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.234+0000 7fedc8b9d640 1 --2- 172.21.15.92:0/753773731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedc40fb150 0x7fedc40fb5c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:50.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.234+0000 7fedc8b9d640 1 --2- 172.21.15.92:0/753773731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedc4101170 0x7fedc4101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:50.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.234+0000 7fedc8b9d640 1 -- 172.21.15.92:0/753773731 >> 172.21.15.92:0/753773731 conn(0x7fedc4071fd0 msgr2=0x7fedc40723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:50.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.234+0000 7fedc8b9d640 1 -- 172.21.15.92:0/753773731 shutdown_connections 2024-05-06T12:25:50.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.234+0000 7fedc8b9d640 1 -- 172.21.15.92:0/753773731 wait complete. 2024-05-06T12:25:50.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.234+0000 7fedc8b9d640 1 Processor -- start 2024-05-06T12:25:50.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.235+0000 7fedc8b9d640 1 -- start start 2024-05-06T12:25:50.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.235+0000 7fedc8b9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedc40fb150 0x7fedc4112c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:50.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.235+0000 7fedc8b9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedc4101170 0x7fedc4113150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:50.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.235+0000 7fedc8b9d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc4113750 con 0x7fedc4101170 2024-05-06T12:25:50.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.235+0000 7fedc8b9d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc41138c0 con 0x7fedc40fb150 2024-05-06T12:25:50.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.235+0000 7fedc1d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedc4101170 0x7fedc4113150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:50.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.235+0000 7fedc1d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedc4101170 0x7fedc4113150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35982/0 (socket says 172.21.15.92:35982) 2024-05-06T12:25:50.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.235+0000 7fedc1d9b640 1 -- 172.21.15.92:0/3307828074 learned_addr learned my addr 172.21.15.92:0/3307828074 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:50.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.235+0000 7fedc259c640 1 --2- 172.21.15.92:0/3307828074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedc40fb150 0x7fedc4112c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:50.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.235+0000 7fedc1d9b640 1 -- 172.21.15.92:0/3307828074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedc40fb150 msgr2=0x7fedc4112c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:50.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.235+0000 7fedc1d9b640 1 --2- 172.21.15.92:0/3307828074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedc40fb150 0x7fedc4112c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:50.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.235+0000 7fedc1d9b640 1 -- 172.21.15.92:0/3307828074 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb8005040 con 0x7fedc4101170 2024-05-06T12:25:50.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.235+0000 7fedc1d9b640 1 --2- 172.21.15.92:0/3307828074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedc4101170 0x7fedc4113150 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fedb8004930 tx=0x7fedb8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:50.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.235+0000 7fedb2ffd640 1 -- 172.21.15.92:0/3307828074 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedb80094c0 con 0x7fedc4101170 2024-05-06T12:25:50.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.235+0000 7fedb2ffd640 1 -- 172.21.15.92:0/3307828074 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fedb8032e50 con 0x7fedc4101170 2024-05-06T12:25:50.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.235+0000 7fedc8b9d640 1 -- 172.21.15.92:0/3307828074 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc410cef0 con 0x7fedc4101170 2024-05-06T12:25:50.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.236+0000 7fedb2ffd640 1 -- 172.21.15.92:0/3307828074 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedb80433b0 con 0x7fedc4101170 2024-05-06T12:25:50.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.236+0000 7fedc8b9d640 1 -- 172.21.15.92:0/3307828074 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc410d3b0 con 0x7fedc4101170 2024-05-06T12:25:50.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.236+0000 7fedc8b9d640 1 -- 172.21.15.92:0/3307828074 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc40fc870 con 0x7fedc4101170 2024-05-06T12:25:50.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.237+0000 7fedb2ffd640 1 -- 172.21.15.92:0/3307828074 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fedb802f070 con 0x7fedc4101170 2024-05-06T12:25:50.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.237+0000 7fedb2ffd640 1 --2- 172.21.15.92:0/3307828074 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fedac06e0b0 0x7fedac070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:50.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.238+0000 7fedb2ffd640 1 -- 172.21.15.92:0/3307828074 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fedb80af990 con 0x7fedc4101170 2024-05-06T12:25:50.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.238+0000 7fedc259c640 1 --2- 172.21.15.92:0/3307828074 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fedac06e0b0 0x7fedac070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:50.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.238+0000 7fedc259c640 1 --2- 172.21.15.92:0/3307828074 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fedac06e0b0 0x7fedac070560 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fedb4004eb0 tx=0x7fedb400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:50.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.242+0000 7fedb2ffd640 1 -- 172.21.15.92:0/3307828074 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fedb8078d70 con 0x7fedc4101170 2024-05-06T12:25:50.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:50 smithi190 ceph-mon[79744]: pgmap v314: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:25:50.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:50 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/3870076728' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-05-06T12:25:50.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:50 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/716081465' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-05-06T12:25:50.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:50 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/647659855' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-05-06T12:25:50.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.462+0000 7fedc8b9d640 1 -- 172.21.15.92:0/3307828074 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7fedc410d5d0 con 0x7fedc4101170 2024-05-06T12:25:50.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.463+0000 7fedb2ffd640 1 -- 172.21.15.92:0/3307828074 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v42) v1 ==== 107+0+4890 (secure 0 0 0) 0x7fedb807c430 con 0x7fedc4101170 2024-05-06T12:25:50.465 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:25:50.466 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34402,"name":"cephfs.smithi092.ijsqgp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6827/3131141280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3131141280},{"type":"v1","addr":"172.21.15.92:6827","nonce":3131141280}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44261,"name":"cephfs.smithi190.idzoff","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6827/1642657019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1642657019},{"type":"v1","addr":"172.21.15.190:6827","nonce":1642657019}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-06T12:11:44.952671+0000","modified":"2024-05-06T12:21:04.496119+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0,1],"up":{"mds_0":14472,"mds_1":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14472":{"gid":14472,"name":"cephfs.smithi092.rosbll","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6829/1663253598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1663253598},{"type":"v1","addr":"172.21.15.92:6829","nonce":1663253598}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24267":{"gid":24267,"name":"cephfs.smithi190.byjvgf","rank":1,"incarnation":7,"state":"up:stopping","state_seq":4,"addr":"172.21.15.190:6825/1565151699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1565151699},{"type":"v1","addr":"172.21.15.190:6825","nonce":1565151699}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14472,"qdb_cluster":[14472]},"id":1}]} 2024-05-06T12:25:50.466 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 25 2024-05-06T12:25:50.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.465+0000 7fedc8b9d640 1 -- 172.21.15.92:0/3307828074 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fedac06e0b0 msgr2=0x7fedac070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:50.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.465+0000 7fedc8b9d640 1 --2- 172.21.15.92:0/3307828074 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fedac06e0b0 0x7fedac070560 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fedb4004eb0 tx=0x7fedb400b040 comp rx=0 tx=0).stop 2024-05-06T12:25:50.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.465+0000 7fedc8b9d640 1 -- 172.21.15.92:0/3307828074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedc4101170 msgr2=0x7fedc4113150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:50.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.465+0000 7fedc8b9d640 1 --2- 172.21.15.92:0/3307828074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedc4101170 0x7fedc4113150 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fedb8004930 tx=0x7fedb8009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:50.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.465+0000 7fedc8b9d640 1 -- 172.21.15.92:0/3307828074 shutdown_connections 2024-05-06T12:25:50.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.465+0000 7fedc8b9d640 1 --2- 172.21.15.92:0/3307828074 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fedac06e0b0 0x7fedac070560 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:50.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.465+0000 7fedc8b9d640 1 --2- 172.21.15.92:0/3307828074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedc4101170 0x7fedc4113150 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:50.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.465+0000 7fedc8b9d640 1 --2- 172.21.15.92:0/3307828074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedc40fb150 0x7fedc4112c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:50.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.465+0000 7fedc8b9d640 1 -- 172.21.15.92:0/3307828074 >> 172.21.15.92:0/3307828074 conn(0x7fedc4071fd0 msgr2=0x7fedc4064630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:50.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.465+0000 7fedc8b9d640 1 -- 172.21.15.92:0/3307828074 shutdown_connections 2024-05-06T12:25:50.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.465+0000 7fedc8b9d640 1 -- 172.21.15.92:0/3307828074 wait complete. 2024-05-06T12:25:50.478 DEBUG:tasks.fs:max_mds reduced in epoch 25 2024-05-06T12:25:50.479 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 25 2024-05-06T12:25:50.479 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-05-06T12:25:50.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.575+0000 7f33ef15d640 1 -- 172.21.15.92:0/3362846756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33e8102270 msgr2=0x7f33e8102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:50.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.575+0000 7f33ef15d640 1 --2- 172.21.15.92:0/3362846756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33e8102270 0x7f33e8102640 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f33e0009c50 tx=0x7f33e0004d10 comp rx=0 tx=0).stop 2024-05-06T12:25:50.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.575+0000 7f33ef15d640 1 -- 172.21.15.92:0/3362846756 shutdown_connections 2024-05-06T12:25:50.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.575+0000 7f33ef15d640 1 --2- 172.21.15.92:0/3362846756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33e8064390 0x7f33e8064800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:50.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.575+0000 7f33ef15d640 1 --2- 172.21.15.92:0/3362846756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33e8102270 0x7f33e8102640 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:50.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.575+0000 7f33ef15d640 1 -- 172.21.15.92:0/3362846756 >> 172.21.15.92:0/3362846756 conn(0x7f33e8071fd0 msgr2=0x7f33e80723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:50.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.575+0000 7f33ef15d640 1 -- 172.21.15.92:0/3362846756 shutdown_connections 2024-05-06T12:25:50.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.575+0000 7f33ef15d640 1 -- 172.21.15.92:0/3362846756 wait complete. 2024-05-06T12:25:50.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.575+0000 7f33ef15d640 1 Processor -- start 2024-05-06T12:25:50.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.575+0000 7f33ef15d640 1 -- start start 2024-05-06T12:25:50.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.576+0000 7f33ef15d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33e8064390 0x7f33e8112c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:50.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.576+0000 7f33ef15d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33e8102270 0x7f33e81131a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:50.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.576+0000 7f33ef15d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e81137a0 con 0x7f33e8064390 2024-05-06T12:25:50.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.576+0000 7f33ef15d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e810cdf0 con 0x7f33e8102270 2024-05-06T12:25:50.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.576+0000 7f33ecef9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33e8064390 0x7f33e8112c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:50.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.576+0000 7f33ecef9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33e8064390 0x7f33e8112c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36002/0 (socket says 172.21.15.92:36002) 2024-05-06T12:25:50.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.576+0000 7f33ecef9640 1 -- 172.21.15.92:0/373996061 learned_addr learned my addr 172.21.15.92:0/373996061 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:50.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.576+0000 7f33ecef9640 1 -- 172.21.15.92:0/373996061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33e8102270 msgr2=0x7f33e81131a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:50.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.576+0000 7f33ecef9640 1 --2- 172.21.15.92:0/373996061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33e8102270 0x7f33e81131a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:50.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.576+0000 7f33ecef9640 1 -- 172.21.15.92:0/373996061 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e00098b0 con 0x7f33e8064390 2024-05-06T12:25:50.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.576+0000 7f33ecef9640 1 --2- 172.21.15.92:0/373996061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33e8064390 0x7f33e8112c60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f33e0004a30 tx=0x7f33e0004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:50.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.576+0000 7f33e57fa640 1 -- 172.21.15.92:0/373996061 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33e003a070 con 0x7f33e8064390 2024-05-06T12:25:50.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.576+0000 7f33e57fa640 1 -- 172.21.15.92:0/373996061 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f33e0035b20 con 0x7f33e8064390 2024-05-06T12:25:50.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.576+0000 7f33ef15d640 1 -- 172.21.15.92:0/373996061 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33e810d070 con 0x7f33e8064390 2024-05-06T12:25:50.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.576+0000 7f33e57fa640 1 -- 172.21.15.92:0/373996061 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33e003ea70 con 0x7f33e8064390 2024-05-06T12:25:50.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.577+0000 7f33ef15d640 1 -- 172.21.15.92:0/373996061 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e810d610 con 0x7f33e8064390 2024-05-06T12:25:50.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.578+0000 7f33ef15d640 1 -- 172.21.15.92:0/373996061 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33e80fd970 con 0x7f33e8064390 2024-05-06T12:25:50.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.578+0000 7f33e57fa640 1 -- 172.21.15.92:0/373996061 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f33e0032070 con 0x7f33e8064390 2024-05-06T12:25:50.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.579+0000 7f33e57fa640 1 --2- 172.21.15.92:0/373996061 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f33d006e0b0 0x7f33d0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:50.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.579+0000 7f33e57fa640 1 -- 172.21.15.92:0/373996061 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f33e00afc30 con 0x7f33e8064390 2024-05-06T12:25:50.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.579+0000 7f33e7fff640 1 --2- 172.21.15.92:0/373996061 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f33d006e0b0 0x7f33d0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:50.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.579+0000 7f33e7fff640 1 --2- 172.21.15.92:0/373996061 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f33d006e0b0 0x7f33d0070560 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f33e810e520 tx=0x7f33dc009c50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:50.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.584+0000 7f33e57fa640 1 -- 172.21.15.92:0/373996061 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f33e0079090 con 0x7f33e8064390 2024-05-06T12:25:50.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.800+0000 7f33ef15d640 1 -- 172.21.15.92:0/373996061 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f33e8109380 con 0x7f33e8064390 2024-05-06T12:25:50.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.801+0000 7f33e57fa640 1 -- 172.21.15.92:0/373996061 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v42) v1 ==== 107+0+4092 (secure 0 0 0) 0x7f33e007c750 con 0x7f33e8064390 2024-05-06T12:25:50.803 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:25:50.803 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34402,"name":"cephfs.smithi092.ijsqgp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6827/3131141280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3131141280},{"type":"v1","addr":"172.21.15.92:6827","nonce":3131141280}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44261,"name":"cephfs.smithi190.idzoff","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6827/1642657019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1642657019},{"type":"v1","addr":"172.21.15.190:6827","nonce":1642657019}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-06T12:11:44.952671+0000","modified":"2024-05-06T12:21:26.647044+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14472},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_14472":{"gid":14472,"name":"cephfs.smithi092.rosbll","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6829/1663253598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1663253598},{"type":"v1","addr":"172.21.15.92:6829","nonce":1663253598}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14472,"qdb_cluster":[14472]},"id":1}]} 2024-05-06T12:25:50.803 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 26 2024-05-06T12:25:50.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.803+0000 7f33ef15d640 1 -- 172.21.15.92:0/373996061 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f33d006e0b0 msgr2=0x7f33d0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:50.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.803+0000 7f33ef15d640 1 --2- 172.21.15.92:0/373996061 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f33d006e0b0 0x7f33d0070560 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f33e810e520 tx=0x7f33dc009c50 comp rx=0 tx=0).stop 2024-05-06T12:25:50.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.803+0000 7f33ef15d640 1 -- 172.21.15.92:0/373996061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33e8064390 msgr2=0x7f33e8112c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:50.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.803+0000 7f33ef15d640 1 --2- 172.21.15.92:0/373996061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33e8064390 0x7f33e8112c60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f33e0004a30 tx=0x7f33e0004d10 comp rx=0 tx=0).stop 2024-05-06T12:25:50.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.803+0000 7f33ef15d640 1 -- 172.21.15.92:0/373996061 shutdown_connections 2024-05-06T12:25:50.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.803+0000 7f33ef15d640 1 --2- 172.21.15.92:0/373996061 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f33d006e0b0 0x7f33d0070560 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:50.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.803+0000 7f33ef15d640 1 --2- 172.21.15.92:0/373996061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33e8102270 0x7f33e81131a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:50.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.803+0000 7f33ef15d640 1 --2- 172.21.15.92:0/373996061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33e8064390 0x7f33e8112c60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:50.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.803+0000 7f33ef15d640 1 -- 172.21.15.92:0/373996061 >> 172.21.15.92:0/373996061 conn(0x7f33e8071fd0 msgr2=0x7f33e80fa2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:50.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.803+0000 7f33ef15d640 1 -- 172.21.15.92:0/373996061 shutdown_connections 2024-05-06T12:25:50.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.803+0000 7f33ef15d640 1 -- 172.21.15.92:0/373996061 wait complete. 2024-05-06T12:25:50.817 DEBUG:tasks.fs:max_mds reduced in epoch 26 2024-05-06T12:25:50.817 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 26 2024-05-06T12:25:50.817 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-05-06T12:25:50.915 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.911+0000 7fbec4c6e640 1 -- 172.21.15.92:0/3146153820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbec0064390 msgr2=0x7fbec0064800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:50.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.911+0000 7fbec4c6e640 1 --2- 172.21.15.92:0/3146153820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbec0064390 0x7fbec0064800 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbeb00048d0 tx=0x7fbeb0009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:50.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.912+0000 7fbec4c6e640 1 -- 172.21.15.92:0/3146153820 shutdown_connections 2024-05-06T12:25:50.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.912+0000 7fbec4c6e640 1 --2- 172.21.15.92:0/3146153820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbec0064390 0x7fbec0064800 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:50.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.912+0000 7fbec4c6e640 1 --2- 172.21.15.92:0/3146153820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbec0102270 0x7fbec0102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:50.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.912+0000 7fbec4c6e640 1 -- 172.21.15.92:0/3146153820 >> 172.21.15.92:0/3146153820 conn(0x7fbec0071fd0 msgr2=0x7fbec00723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:50.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.912+0000 7fbec4c6e640 1 -- 172.21.15.92:0/3146153820 shutdown_connections 2024-05-06T12:25:50.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.912+0000 7fbec4c6e640 1 -- 172.21.15.92:0/3146153820 wait complete. 2024-05-06T12:25:50.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.912+0000 7fbec4c6e640 1 Processor -- start 2024-05-06T12:25:50.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.912+0000 7fbec4c6e640 1 -- start start 2024-05-06T12:25:50.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.912+0000 7fbec4c6e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbec0064390 0x7fbec0112c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:50.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.912+0000 7fbec4c6e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbec0102270 0x7fbec01131a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:50.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.912+0000 7fbec4c6e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec01137a0 con 0x7fbec0102270 2024-05-06T12:25:50.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.912+0000 7fbec4c6e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec010cdf0 con 0x7fbec0064390 2024-05-06T12:25:50.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.912+0000 7fbebdd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbec0102270 0x7fbec01131a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:50.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.912+0000 7fbebdd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbec0102270 0x7fbec01131a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36018/0 (socket says 172.21.15.92:36018) 2024-05-06T12:25:50.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.912+0000 7fbebdd9b640 1 -- 172.21.15.92:0/1126518445 learned_addr learned my addr 172.21.15.92:0/1126518445 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:50.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.912+0000 7fbebe59c640 1 --2- 172.21.15.92:0/1126518445 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbec0064390 0x7fbec0112c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:50.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.913+0000 7fbebdd9b640 1 -- 172.21.15.92:0/1126518445 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbec0064390 msgr2=0x7fbec0112c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:50.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.913+0000 7fbebdd9b640 1 --2- 172.21.15.92:0/1126518445 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbec0064390 0x7fbec0112c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:50.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.913+0000 7fbebdd9b640 1 -- 172.21.15.92:0/1126518445 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb0005040 con 0x7fbec0102270 2024-05-06T12:25:50.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.913+0000 7fbebdd9b640 1 --2- 172.21.15.92:0/1126518445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbec0102270 0x7fbec01131a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0004930 tx=0x7fbeb0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:50.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.913+0000 7fbeaeffd640 1 -- 172.21.15.92:0/1126518445 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbeb00094c0 con 0x7fbec0102270 2024-05-06T12:25:50.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.913+0000 7fbec4c6e640 1 -- 172.21.15.92:0/1126518445 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec010d070 con 0x7fbec0102270 2024-05-06T12:25:50.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.913+0000 7fbeaeffd640 1 -- 172.21.15.92:0/1126518445 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbeb0032e50 con 0x7fbec0102270 2024-05-06T12:25:50.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.913+0000 7fbeaeffd640 1 -- 172.21.15.92:0/1126518445 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbeb00433b0 con 0x7fbec0102270 2024-05-06T12:25:50.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.913+0000 7fbec4c6e640 1 -- 172.21.15.92:0/1126518445 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec010d530 con 0x7fbec0102270 2024-05-06T12:25:50.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.914+0000 7fbec4c6e640 1 -- 172.21.15.92:0/1126518445 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbec00fd970 con 0x7fbec0102270 2024-05-06T12:25:50.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.915+0000 7fbeaeffd640 1 -- 172.21.15.92:0/1126518445 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fbeb002f070 con 0x7fbec0102270 2024-05-06T12:25:50.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.915+0000 7fbeaeffd640 1 --2- 172.21.15.92:0/1126518445 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fbea806e0b0 0x7fbea8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:50.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.915+0000 7fbeaeffd640 1 -- 172.21.15.92:0/1126518445 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fbeb00afa40 con 0x7fbec0102270 2024-05-06T12:25:50.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.915+0000 7fbebe59c640 1 --2- 172.21.15.92:0/1126518445 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fbea806e0b0 0x7fbea8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:50.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.916+0000 7fbebe59c640 1 --2- 172.21.15.92:0/1126518445 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fbea806e0b0 0x7fbea8070560 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4009fd0 tx=0x7fbeb4009e50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:50.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:50.920+0000 7fbeaeffd640 1 -- 172.21.15.92:0/1126518445 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fbeb0078d70 con 0x7fbec0102270 2024-05-06T12:25:51.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.137+0000 7fbec4c6e640 1 -- 172.21.15.92:0/1126518445 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7fbec0109380 con 0x7fbec0102270 2024-05-06T12:25:51.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.138+0000 7fbeaeffd640 1 -- 172.21.15.92:0/1126518445 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v42) v1 ==== 107+0+4874 (secure 0 0 0) 0x7fbeb007c430 con 0x7fbec0102270 2024-05-06T12:25:51.140 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:25:51.140 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34402,"name":"cephfs.smithi092.ijsqgp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6827/3131141280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3131141280},{"type":"v1","addr":"172.21.15.92:6827","nonce":3131141280}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44261,"name":"cephfs.smithi190.idzoff","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6827/1642657019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1642657019},{"type":"v1","addr":"172.21.15.190:6827","nonce":1642657019}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44267,"name":"cephfs.smithi190.byjvgf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6825/3031977277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3031977277},{"type":"v1","addr":"172.21.15.190:6825","nonce":3031977277}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-06T12:11:44.952671+0000","modified":"2024-05-06T12:21:26.647044+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14472},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_14472":{"gid":14472,"name":"cephfs.smithi092.rosbll","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6829/1663253598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1663253598},{"type":"v1","addr":"172.21.15.92:6829","nonce":1663253598}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14472,"qdb_cluster":[14472]},"id":1}]} 2024-05-06T12:25:51.140 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 27 2024-05-06T12:25:51.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.140+0000 7fbec4c6e640 1 -- 172.21.15.92:0/1126518445 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fbea806e0b0 msgr2=0x7fbea8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:51.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.140+0000 7fbec4c6e640 1 --2- 172.21.15.92:0/1126518445 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fbea806e0b0 0x7fbea8070560 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4009fd0 tx=0x7fbeb4009e50 comp rx=0 tx=0).stop 2024-05-06T12:25:51.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.140+0000 7fbec4c6e640 1 -- 172.21.15.92:0/1126518445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbec0102270 msgr2=0x7fbec01131a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:51.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.140+0000 7fbec4c6e640 1 --2- 172.21.15.92:0/1126518445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbec0102270 0x7fbec01131a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0004930 tx=0x7fbeb0009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:51.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.140+0000 7fbec4c6e640 1 -- 172.21.15.92:0/1126518445 shutdown_connections 2024-05-06T12:25:51.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.140+0000 7fbec4c6e640 1 --2- 172.21.15.92:0/1126518445 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fbea806e0b0 0x7fbea8070560 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:51.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.140+0000 7fbec4c6e640 1 --2- 172.21.15.92:0/1126518445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbec0102270 0x7fbec01131a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:51.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.140+0000 7fbec4c6e640 1 --2- 172.21.15.92:0/1126518445 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbec0064390 0x7fbec0112c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:51.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.140+0000 7fbec4c6e640 1 -- 172.21.15.92:0/1126518445 >> 172.21.15.92:0/1126518445 conn(0x7fbec0071fd0 msgr2=0x7fbec00fa320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:51.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.140+0000 7fbec4c6e640 1 -- 172.21.15.92:0/1126518445 shutdown_connections 2024-05-06T12:25:51.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.140+0000 7fbec4c6e640 1 -- 172.21.15.92:0/1126518445 wait complete. 2024-05-06T12:25:51.153 DEBUG:tasks.fs:max_mds reduced in epoch 27 2024-05-06T12:25:51.153 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 27 2024-05-06T12:25:51.153 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-05-06T12:25:51.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.248+0000 7fc092d9d640 1 -- 172.21.15.92:0/4149932803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc08c102270 msgr2=0x7fc08c102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:51.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.248+0000 7fc092d9d640 1 --2- 172.21.15.92:0/4149932803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc08c102270 0x7fc08c102640 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc0800048d0 tx=0x7fc080009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:51.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.248+0000 7fc092d9d640 1 -- 172.21.15.92:0/4149932803 shutdown_connections 2024-05-06T12:25:51.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.248+0000 7fc092d9d640 1 --2- 172.21.15.92:0/4149932803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc08c064390 0x7fc08c064800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:51.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.248+0000 7fc092d9d640 1 --2- 172.21.15.92:0/4149932803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc08c102270 0x7fc08c102640 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:51.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.248+0000 7fc092d9d640 1 -- 172.21.15.92:0/4149932803 >> 172.21.15.92:0/4149932803 conn(0x7fc08c071fd0 msgr2=0x7fc08c0723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:51.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.248+0000 7fc092d9d640 1 -- 172.21.15.92:0/4149932803 shutdown_connections 2024-05-06T12:25:51.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.248+0000 7fc092d9d640 1 -- 172.21.15.92:0/4149932803 wait complete. 2024-05-06T12:25:51.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.248+0000 7fc092d9d640 1 Processor -- start 2024-05-06T12:25:51.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.248+0000 7fc092d9d640 1 -- start start 2024-05-06T12:25:51.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.249+0000 7fc092d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc08c064390 0x7fc08c112c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:51.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.249+0000 7fc092d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc08c102270 0x7fc08c1131a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:51.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.249+0000 7fc092d9d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c1137a0 con 0x7fc08c064390 2024-05-06T12:25:51.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.249+0000 7fc092d9d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c10cdf0 con 0x7fc08c102270 2024-05-06T12:25:51.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.249+0000 7fc090b39640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc08c064390 0x7fc08c112c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:51.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.249+0000 7fc090b39640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc08c064390 0x7fc08c112c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36042/0 (socket says 172.21.15.92:36042) 2024-05-06T12:25:51.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.249+0000 7fc090b39640 1 -- 172.21.15.92:0/3455016012 learned_addr learned my addr 172.21.15.92:0/3455016012 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:51.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.249+0000 7fc08bfff640 1 --2- 172.21.15.92:0/3455016012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc08c102270 0x7fc08c1131a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:51.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.249+0000 7fc090b39640 1 -- 172.21.15.92:0/3455016012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc08c102270 msgr2=0x7fc08c1131a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:51.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.249+0000 7fc090b39640 1 --2- 172.21.15.92:0/3455016012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc08c102270 0x7fc08c1131a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:51.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.249+0000 7fc090b39640 1 -- 172.21.15.92:0/3455016012 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc080005040 con 0x7fc08c064390 2024-05-06T12:25:51.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.249+0000 7fc090b39640 1 --2- 172.21.15.92:0/3455016012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc08c064390 0x7fc08c112c60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc0800047f0 tx=0x7fc080009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:51.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.249+0000 7fc0897fa640 1 -- 172.21.15.92:0/3455016012 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0800094c0 con 0x7fc08c064390 2024-05-06T12:25:51.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.249+0000 7fc0897fa640 1 -- 172.21.15.92:0/3455016012 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc080032e50 con 0x7fc08c064390 2024-05-06T12:25:51.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.249+0000 7fc092d9d640 1 -- 172.21.15.92:0/3455016012 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc08c10d070 con 0x7fc08c064390 2024-05-06T12:25:51.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.249+0000 7fc0897fa640 1 -- 172.21.15.92:0/3455016012 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0800433b0 con 0x7fc08c064390 2024-05-06T12:25:51.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.249+0000 7fc092d9d640 1 -- 172.21.15.92:0/3455016012 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08c10d530 con 0x7fc08c064390 2024-05-06T12:25:51.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.250+0000 7fc092d9d640 1 -- 172.21.15.92:0/3455016012 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc08c0fd970 con 0x7fc08c064390 2024-05-06T12:25:51.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.251+0000 7fc0897fa640 1 -- 172.21.15.92:0/3455016012 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc08002f070 con 0x7fc08c064390 2024-05-06T12:25:51.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.251+0000 7fc0897fa640 1 --2- 172.21.15.92:0/3455016012 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fc07c06e0b0 0x7fc07c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:51.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.252+0000 7fc0897fa640 1 -- 172.21.15.92:0/3455016012 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc0800af940 con 0x7fc08c064390 2024-05-06T12:25:51.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.252+0000 7fc08bfff640 1 --2- 172.21.15.92:0/3455016012 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fc07c06e0b0 0x7fc07c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:51.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.252+0000 7fc08bfff640 1 --2- 172.21.15.92:0/3455016012 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fc07c06e0b0 0x7fc07c070560 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc08c10e520 tx=0x7fc07800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:51.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.256+0000 7fc0897fa640 1 -- 172.21.15.92:0/3455016012 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc080078d20 con 0x7fc08c064390 2024-05-06T12:25:51.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:51 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/1620051236' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-05-06T12:25:51.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:51 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/3307828074' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-05-06T12:25:51.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:51 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/373996061' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-05-06T12:25:51.366 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:51 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/1620051236' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-05-06T12:25:51.366 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:51 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/3307828074' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-05-06T12:25:51.366 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:51 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/373996061' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-05-06T12:25:51.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.472+0000 7fc092d9d640 1 -- 172.21.15.92:0/3455016012 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7fc08c10e360 con 0x7fc08c064390 2024-05-06T12:25:51.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.472+0000 7fc0897fa640 1 -- 172.21.15.92:0/3455016012 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v42) v1 ==== 107+0+4095 (secure 0 0 0) 0x7fc08007c3e0 con 0x7fc08c064390 2024-05-06T12:25:51.475 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:25:51.475 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44261,"name":"cephfs.smithi190.idzoff","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6827/1642657019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1642657019},{"type":"v1","addr":"172.21.15.190:6827","nonce":1642657019}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44267,"name":"cephfs.smithi190.byjvgf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6825/3031977277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3031977277},{"type":"v1","addr":"172.21.15.190:6825","nonce":3031977277}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-06T12:11:44.952671+0000","modified":"2024-05-06T12:21:26.647044+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14472},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_14472":{"gid":14472,"name":"cephfs.smithi092.rosbll","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6829/1663253598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1663253598},{"type":"v1","addr":"172.21.15.92:6829","nonce":1663253598}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14472,"qdb_cluster":[14472]},"id":1}]} 2024-05-06T12:25:51.475 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 28 2024-05-06T12:25:51.477 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.474+0000 7fc092d9d640 1 -- 172.21.15.92:0/3455016012 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fc07c06e0b0 msgr2=0x7fc07c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:51.477 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.474+0000 7fc092d9d640 1 --2- 172.21.15.92:0/3455016012 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fc07c06e0b0 0x7fc07c070560 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc08c10e520 tx=0x7fc07800b040 comp rx=0 tx=0).stop 2024-05-06T12:25:51.477 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.474+0000 7fc092d9d640 1 -- 172.21.15.92:0/3455016012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc08c064390 msgr2=0x7fc08c112c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:51.477 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.474+0000 7fc092d9d640 1 --2- 172.21.15.92:0/3455016012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc08c064390 0x7fc08c112c60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc0800047f0 tx=0x7fc080009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:51.477 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.474+0000 7fc092d9d640 1 -- 172.21.15.92:0/3455016012 shutdown_connections 2024-05-06T12:25:51.477 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.474+0000 7fc092d9d640 1 --2- 172.21.15.92:0/3455016012 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fc07c06e0b0 0x7fc07c070560 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:51.477 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.474+0000 7fc092d9d640 1 --2- 172.21.15.92:0/3455016012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc08c102270 0x7fc08c1131a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:51.477 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.474+0000 7fc092d9d640 1 --2- 172.21.15.92:0/3455016012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc08c064390 0x7fc08c112c60 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:51.477 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.474+0000 7fc092d9d640 1 -- 172.21.15.92:0/3455016012 >> 172.21.15.92:0/3455016012 conn(0x7fc08c071fd0 msgr2=0x7fc08c0fa2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:51.477 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.474+0000 7fc092d9d640 1 -- 172.21.15.92:0/3455016012 shutdown_connections 2024-05-06T12:25:51.477 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.474+0000 7fc092d9d640 1 -- 172.21.15.92:0/3455016012 wait complete. 2024-05-06T12:25:51.487 DEBUG:tasks.fs:max_mds reduced in epoch 28 2024-05-06T12:25:51.487 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 28 2024-05-06T12:25:51.488 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-05-06T12:25:51.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.580+0000 7f7be0f9d640 1 -- 172.21.15.92:0/3083916204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bdc0fb150 msgr2=0x7f7bdc0fb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:51.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.580+0000 7f7be0f9d640 1 --2- 172.21.15.92:0/3083916204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bdc0fb150 0x7f7bdc0fb5c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7bd00048d0 tx=0x7f7bd0009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:51.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.580+0000 7f7be0f9d640 1 -- 172.21.15.92:0/3083916204 shutdown_connections 2024-05-06T12:25:51.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.580+0000 7f7be0f9d640 1 --2- 172.21.15.92:0/3083916204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bdc0fb150 0x7f7bdc0fb5c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:51.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.580+0000 7f7be0f9d640 1 --2- 172.21.15.92:0/3083916204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bdc101170 0x7f7bdc101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:51.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.580+0000 7f7be0f9d640 1 -- 172.21.15.92:0/3083916204 >> 172.21.15.92:0/3083916204 conn(0x7f7bdc071fd0 msgr2=0x7f7bdc0723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:51.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.580+0000 7f7be0f9d640 1 -- 172.21.15.92:0/3083916204 shutdown_connections 2024-05-06T12:25:51.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.580+0000 7f7be0f9d640 1 -- 172.21.15.92:0/3083916204 wait complete. 2024-05-06T12:25:51.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.580+0000 7f7be0f9d640 1 Processor -- start 2024-05-06T12:25:51.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.581+0000 7f7be0f9d640 1 -- start start 2024-05-06T12:25:51.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.581+0000 7f7be0f9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bdc0fb150 0x7f7bdc10de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:51.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.581+0000 7f7be0f9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bdc101170 0x7f7bdc10e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:51.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.581+0000 7f7be0f9d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc0652f0 con 0x7f7bdc0fb150 2024-05-06T12:25:51.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.581+0000 7f7be0f9d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc10e8d0 con 0x7f7bdc101170 2024-05-06T12:25:51.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.581+0000 7f7bda59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bdc0fb150 0x7f7bdc10de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:51.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.581+0000 7f7bda59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bdc0fb150 0x7f7bdc10de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36062/0 (socket says 172.21.15.92:36062) 2024-05-06T12:25:51.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.581+0000 7f7bda59c640 1 -- 172.21.15.92:0/837197790 learned_addr learned my addr 172.21.15.92:0/837197790 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:51.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.581+0000 7f7bd9d9b640 1 --2- 172.21.15.92:0/837197790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bdc101170 0x7f7bdc10e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:51.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.581+0000 7f7bda59c640 1 -- 172.21.15.92:0/837197790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bdc101170 msgr2=0x7f7bdc10e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:51.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.581+0000 7f7bda59c640 1 --2- 172.21.15.92:0/837197790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bdc101170 0x7f7bdc10e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:51.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.581+0000 7f7bda59c640 1 -- 172.21.15.92:0/837197790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd0005040 con 0x7f7bdc0fb150 2024-05-06T12:25:51.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.581+0000 7f7bda59c640 1 --2- 172.21.15.92:0/837197790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bdc0fb150 0x7f7bdc10de50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc00eb70 tx=0x7f7bcc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:51.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.581+0000 7f7bcaffd640 1 -- 172.21.15.92:0/837197790 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bcc007c30 con 0x7f7bdc0fb150 2024-05-06T12:25:51.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.582+0000 7f7bcaffd640 1 -- 172.21.15.92:0/837197790 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7bcc00f040 con 0x7f7bdc0fb150 2024-05-06T12:25:51.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.582+0000 7f7be0f9d640 1 -- 172.21.15.92:0/837197790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bdc10eb50 con 0x7f7bdc0fb150 2024-05-06T12:25:51.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.582+0000 7f7bcaffd640 1 -- 172.21.15.92:0/837197790 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bcc0095b0 con 0x7f7bdc0fb150 2024-05-06T12:25:51.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.582+0000 7f7be0f9d640 1 -- 172.21.15.92:0/837197790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bdc117110 con 0x7f7bdc0fb150 2024-05-06T12:25:51.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.583+0000 7f7be0f9d640 1 -- 172.21.15.92:0/837197790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bdc0fc870 con 0x7f7bdc0fb150 2024-05-06T12:25:51.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.583+0000 7f7bcaffd640 1 -- 172.21.15.92:0/837197790 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7bcc009730 con 0x7f7bdc0fb150 2024-05-06T12:25:51.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.583+0000 7f7bcaffd640 1 --2- 172.21.15.92:0/837197790 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7bc406e0b0 0x7f7bc4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:51.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.583+0000 7f7bcaffd640 1 -- 172.21.15.92:0/837197790 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7bcc08d840 con 0x7f7bdc0fb150 2024-05-06T12:25:51.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.583+0000 7f7bd9d9b640 1 --2- 172.21.15.92:0/837197790 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7bc406e0b0 0x7f7bc4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:51.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.584+0000 7f7bd9d9b640 1 --2- 172.21.15.92:0/837197790 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7bc406e0b0 0x7f7bc4070560 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc064b40 tx=0x7f7bd0009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:51.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.588+0000 7f7bcaffd640 1 -- 172.21.15.92:0/837197790 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7bcc056c20 con 0x7f7bdc0fb150 2024-05-06T12:25:51.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.799+0000 7f7be0f9d640 1 -- 172.21.15.92:0/837197790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f7bdc0fe3a0 con 0x7f7bdc0fb150 2024-05-06T12:25:51.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.799+0000 7f7bcaffd640 1 -- 172.21.15.92:0/837197790 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v42) v1 ==== 107+0+4942 (secure 0 0 0) 0x7f7bcc05a2e0 con 0x7f7bdc0fb150 2024-05-06T12:25:51.802 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:25:51.802 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34438,"name":"cephfs.smithi092.ijsqgp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6827/4190573007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4190573007},{"type":"v1","addr":"172.21.15.92:6827","nonce":4190573007}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":44261,"name":"cephfs.smithi190.idzoff","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6827/1642657019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1642657019},{"type":"v1","addr":"172.21.15.190:6827","nonce":1642657019}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44267,"name":"cephfs.smithi190.byjvgf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6825/3031977277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3031977277},{"type":"v1","addr":"172.21.15.190:6825","nonce":3031977277}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-06T12:11:44.952671+0000","modified":"2024-05-06T12:21:26.647044+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14472},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_14472":{"gid":14472,"name":"cephfs.smithi092.rosbll","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6829/1663253598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1663253598},{"type":"v1","addr":"172.21.15.92:6829","nonce":1663253598}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14472,"qdb_cluster":[14472]},"id":1}]} 2024-05-06T12:25:51.802 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 29 2024-05-06T12:25:51.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.802+0000 7f7be0f9d640 1 -- 172.21.15.92:0/837197790 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7bc406e0b0 msgr2=0x7f7bc4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:51.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.802+0000 7f7be0f9d640 1 --2- 172.21.15.92:0/837197790 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7bc406e0b0 0x7f7bc4070560 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc064b40 tx=0x7f7bd0009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:51.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.802+0000 7f7be0f9d640 1 -- 172.21.15.92:0/837197790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bdc0fb150 msgr2=0x7f7bdc10de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:51.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.802+0000 7f7be0f9d640 1 --2- 172.21.15.92:0/837197790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bdc0fb150 0x7f7bdc10de50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc00eb70 tx=0x7f7bcc00b040 comp rx=0 tx=0).stop 2024-05-06T12:25:51.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.802+0000 7f7be0f9d640 1 -- 172.21.15.92:0/837197790 shutdown_connections 2024-05-06T12:25:51.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.802+0000 7f7be0f9d640 1 --2- 172.21.15.92:0/837197790 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7bc406e0b0 0x7f7bc4070560 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:51.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.802+0000 7f7be0f9d640 1 --2- 172.21.15.92:0/837197790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bdc101170 0x7f7bdc10e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:51.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.802+0000 7f7be0f9d640 1 --2- 172.21.15.92:0/837197790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bdc0fb150 0x7f7bdc10de50 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:51.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.802+0000 7f7be0f9d640 1 -- 172.21.15.92:0/837197790 >> 172.21.15.92:0/837197790 conn(0x7f7bdc071fd0 msgr2=0x7f7bdc0fa300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:51.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.802+0000 7f7be0f9d640 1 -- 172.21.15.92:0/837197790 shutdown_connections 2024-05-06T12:25:51.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.802+0000 7f7be0f9d640 1 -- 172.21.15.92:0/837197790 wait complete. 2024-05-06T12:25:51.815 DEBUG:tasks.fs:max_mds reduced in epoch 29 2024-05-06T12:25:51.815 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 29 2024-05-06T12:25:51.815 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-05-06T12:25:51.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.908+0000 7f5d7075d640 1 -- 172.21.15.92:0/3047301108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d68102270 msgr2=0x7f5d68102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:51.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.908+0000 7f5d7075d640 1 --2- 172.21.15.92:0/3047301108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d68102270 0x7f5d68102640 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5d580048d0 tx=0x7f5d58009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:51.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.908+0000 7f5d7075d640 1 -- 172.21.15.92:0/3047301108 shutdown_connections 2024-05-06T12:25:51.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.908+0000 7f5d7075d640 1 --2- 172.21.15.92:0/3047301108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d68064390 0x7f5d68064800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:51.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.908+0000 7f5d7075d640 1 --2- 172.21.15.92:0/3047301108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d68102270 0x7f5d68102640 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:51.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.908+0000 7f5d7075d640 1 -- 172.21.15.92:0/3047301108 >> 172.21.15.92:0/3047301108 conn(0x7f5d68071fd0 msgr2=0x7f5d680723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:51.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.908+0000 7f5d7075d640 1 -- 172.21.15.92:0/3047301108 shutdown_connections 2024-05-06T12:25:51.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.908+0000 7f5d7075d640 1 -- 172.21.15.92:0/3047301108 wait complete. 2024-05-06T12:25:51.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.908+0000 7f5d7075d640 1 Processor -- start 2024-05-06T12:25:51.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.908+0000 7f5d7075d640 1 -- start start 2024-05-06T12:25:51.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.909+0000 7f5d7075d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d68064390 0x7f5d68112c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:51.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.909+0000 7f5d7075d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d68102270 0x7f5d68113170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:51.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.909+0000 7f5d7075d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d68113770 con 0x7f5d68064390 2024-05-06T12:25:51.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.909+0000 7f5d7075d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6810cdc0 con 0x7f5d68102270 2024-05-06T12:25:51.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.909+0000 7f5d6e4f9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d68064390 0x7f5d68112c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:51.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.909+0000 7f5d6e4f9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d68064390 0x7f5d68112c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36084/0 (socket says 172.21.15.92:36084) 2024-05-06T12:25:51.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.909+0000 7f5d6e4f9640 1 -- 172.21.15.92:0/2735680722 learned_addr learned my addr 172.21.15.92:0/2735680722 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:51.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.909+0000 7f5d6e4f9640 1 -- 172.21.15.92:0/2735680722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d68102270 msgr2=0x7f5d68113170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:51.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.909+0000 7f5d6e4f9640 1 --2- 172.21.15.92:0/2735680722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d68102270 0x7f5d68113170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:51.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.909+0000 7f5d6e4f9640 1 -- 172.21.15.92:0/2735680722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d58005040 con 0x7f5d68064390 2024-05-06T12:25:51.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.909+0000 7f5d6e4f9640 1 --2- 172.21.15.92:0/2735680722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d68064390 0x7f5d68112c30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5d58004930 tx=0x7f5d58009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:51.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.909+0000 7f5d5effd640 1 -- 172.21.15.92:0/2735680722 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d580094c0 con 0x7f5d68064390 2024-05-06T12:25:51.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.909+0000 7f5d5effd640 1 -- 172.21.15.92:0/2735680722 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d58032e50 con 0x7f5d68064390 2024-05-06T12:25:51.914 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.909+0000 7f5d7075d640 1 -- 172.21.15.92:0/2735680722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d6810d040 con 0x7f5d68064390 2024-05-06T12:25:51.914 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.909+0000 7f5d5effd640 1 -- 172.21.15.92:0/2735680722 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d580433b0 con 0x7f5d68064390 2024-05-06T12:25:51.914 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.909+0000 7f5d7075d640 1 -- 172.21.15.92:0/2735680722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d6810d500 con 0x7f5d68064390 2024-05-06T12:25:51.915 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.910+0000 7f5d7075d640 1 -- 172.21.15.92:0/2735680722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d680fd970 con 0x7f5d68064390 2024-05-06T12:25:51.915 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.911+0000 7f5d5effd640 1 -- 172.21.15.92:0/2735680722 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5d5802f070 con 0x7f5d68064390 2024-05-06T12:25:51.915 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.911+0000 7f5d5effd640 1 --2- 172.21.15.92:0/2735680722 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f5d5406e0b0 0x7f5d54070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:51.915 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.911+0000 7f5d5effd640 1 -- 172.21.15.92:0/2735680722 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f5d580af960 con 0x7f5d68064390 2024-05-06T12:25:51.915 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.911+0000 7f5d6dcf8640 1 --2- 172.21.15.92:0/2735680722 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f5d5406e0b0 0x7f5d54070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:51.915 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.912+0000 7f5d6dcf8640 1 --2- 172.21.15.92:0/2735680722 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f5d5406e0b0 0x7f5d54070560 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5d6810e4f0 tx=0x7f5d64009c50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:51.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:51.916+0000 7f5d5effd640 1 -- 172.21.15.92:0/2735680722 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5d58078c90 con 0x7f5d68064390 2024-05-06T12:25:52.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.134+0000 7f5d7075d640 1 -- 172.21.15.92:0/2735680722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7f5d68109370 con 0x7f5d68064390 2024-05-06T12:25:52.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.134+0000 7f5d5effd640 1 -- 172.21.15.92:0/2735680722 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v42) v1 ==== 107+0+4144 (secure 0 0 0) 0x7f5d5807c350 con 0x7f5d68064390 2024-05-06T12:25:52.137 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:25:52.137 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34438,"name":"cephfs.smithi092.ijsqgp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6827/4190573007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4190573007},{"type":"v1","addr":"172.21.15.92:6827","nonce":4190573007}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":44261,"name":"cephfs.smithi190.idzoff","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6827/1642657019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1642657019},{"type":"v1","addr":"172.21.15.190:6827","nonce":1642657019}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44267,"name":"cephfs.smithi190.byjvgf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6825/3031977277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3031977277},{"type":"v1","addr":"172.21.15.190:6825","nonce":3031977277}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-06T12:11:44.952671+0000","modified":"2024-05-06T12:21:58.517920+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[1],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-06T12:25:52.137 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 30 2024-05-06T12:25:52.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.137+0000 7f5d7075d640 1 -- 172.21.15.92:0/2735680722 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f5d5406e0b0 msgr2=0x7f5d54070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:52.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.137+0000 7f5d7075d640 1 --2- 172.21.15.92:0/2735680722 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f5d5406e0b0 0x7f5d54070560 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5d6810e4f0 tx=0x7f5d64009c50 comp rx=0 tx=0).stop 2024-05-06T12:25:52.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.137+0000 7f5d7075d640 1 -- 172.21.15.92:0/2735680722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d68064390 msgr2=0x7f5d68112c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:52.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.137+0000 7f5d7075d640 1 --2- 172.21.15.92:0/2735680722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d68064390 0x7f5d68112c30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5d58004930 tx=0x7f5d58009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:52.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.137+0000 7f5d7075d640 1 -- 172.21.15.92:0/2735680722 shutdown_connections 2024-05-06T12:25:52.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.137+0000 7f5d7075d640 1 --2- 172.21.15.92:0/2735680722 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f5d5406e0b0 0x7f5d54070560 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:52.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.137+0000 7f5d7075d640 1 --2- 172.21.15.92:0/2735680722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d68102270 0x7f5d68113170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:52.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.137+0000 7f5d7075d640 1 --2- 172.21.15.92:0/2735680722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d68064390 0x7f5d68112c30 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:52.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.137+0000 7f5d7075d640 1 -- 172.21.15.92:0/2735680722 >> 172.21.15.92:0/2735680722 conn(0x7f5d68071fd0 msgr2=0x7f5d680fa260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:52.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.137+0000 7f5d7075d640 1 -- 172.21.15.92:0/2735680722 shutdown_connections 2024-05-06T12:25:52.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.137+0000 7f5d7075d640 1 -- 172.21.15.92:0/2735680722 wait complete. 2024-05-06T12:25:52.148 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:52 smithi092 ceph-mon[91933]: pgmap v315: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:25:52.149 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:52 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/1126518445' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-05-06T12:25:52.149 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:52 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/3455016012' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-05-06T12:25:52.149 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:52 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/837197790' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-05-06T12:25:52.150 DEBUG:tasks.fs:max_mds reduced in epoch 30 2024-05-06T12:25:52.150 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 30 2024-05-06T12:25:52.150 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-05-06T12:25:52.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.242+0000 7f206ab9d640 1 -- 172.21.15.92:0/715312657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2064101070 msgr2=0x7f2064101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:52.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.242+0000 7f206ab9d640 1 --2- 172.21.15.92:0/715312657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2064101070 0x7f2064101440 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f205c009c50 tx=0x7f205c004d10 comp rx=0 tx=0).stop 2024-05-06T12:25:52.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.242+0000 7f206ab9d640 1 -- 172.21.15.92:0/715312657 shutdown_connections 2024-05-06T12:25:52.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.242+0000 7f206ab9d640 1 --2- 172.21.15.92:0/715312657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2064101a10 0x7f20640fb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:52.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.242+0000 7f206ab9d640 1 --2- 172.21.15.92:0/715312657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2064101070 0x7f2064101440 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:52.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.242+0000 7f206ab9d640 1 -- 172.21.15.92:0/715312657 >> 172.21.15.92:0/715312657 conn(0x7f20640f6be0 msgr2=0x7f20640f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:52.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.242+0000 7f206ab9d640 1 -- 172.21.15.92:0/715312657 shutdown_connections 2024-05-06T12:25:52.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.242+0000 7f206ab9d640 1 -- 172.21.15.92:0/715312657 wait complete. 2024-05-06T12:25:52.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.242+0000 7f206ab9d640 1 Processor -- start 2024-05-06T12:25:52.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.242+0000 7f206ab9d640 1 -- start start 2024-05-06T12:25:52.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.243+0000 7f206ab9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2064101070 0x7f206410de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:52.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.243+0000 7f206ab9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2064101a10 0x7f206410e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:52.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.243+0000 7f206ab9d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2064066740 con 0x7f2064101a10 2024-05-06T12:25:52.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.243+0000 7f206ab9d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206410e900 con 0x7f2064101070 2024-05-06T12:25:52.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.243+0000 7f2063fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2064101a10 0x7f206410e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:52.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.243+0000 7f2063fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2064101a10 0x7f206410e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36104/0 (socket says 172.21.15.92:36104) 2024-05-06T12:25:52.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.243+0000 7f2063fff640 1 -- 172.21.15.92:0/1803459146 learned_addr learned my addr 172.21.15.92:0/1803459146 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:52.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.243+0000 7f2068939640 1 --2- 172.21.15.92:0/1803459146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2064101070 0x7f206410de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:52.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.243+0000 7f2063fff640 1 -- 172.21.15.92:0/1803459146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2064101070 msgr2=0x7f206410de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:52.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.243+0000 7f2063fff640 1 --2- 172.21.15.92:0/1803459146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2064101070 0x7f206410de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:52.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.243+0000 7f2063fff640 1 -- 172.21.15.92:0/1803459146 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f205c0098b0 con 0x7f2064101a10 2024-05-06T12:25:52.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.243+0000 7f2068939640 1 --2- 172.21.15.92:0/1803459146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2064101070 0x7f206410de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-06T12:25:52.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.243+0000 7f2063fff640 1 --2- 172.21.15.92:0/1803459146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2064101a10 0x7f206410e390 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2054002a50 tx=0x7f2054004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:52.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.243+0000 7f20617fa640 1 -- 172.21.15.92:0/1803459146 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2054007c30 con 0x7f2064101a10 2024-05-06T12:25:52.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.243+0000 7f206ab9d640 1 -- 172.21.15.92:0/1803459146 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f206410ebe0 con 0x7f2064101a10 2024-05-06T12:25:52.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.243+0000 7f20617fa640 1 -- 172.21.15.92:0/1803459146 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f205400f040 con 0x7f2064101a10 2024-05-06T12:25:52.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.243+0000 7f20617fa640 1 -- 172.21.15.92:0/1803459146 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f205400db50 con 0x7f2064101a10 2024-05-06T12:25:52.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.243+0000 7f206ab9d640 1 -- 172.21.15.92:0/1803459146 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20641171f0 con 0x7f2064101a10 2024-05-06T12:25:52.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.245+0000 7f206ab9d640 1 -- 172.21.15.92:0/1803459146 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20640fc7e0 con 0x7f2064101a10 2024-05-06T12:25:52.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.245+0000 7f20617fa640 1 -- 172.21.15.92:0/1803459146 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f205401c400 con 0x7f2064101a10 2024-05-06T12:25:52.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.245+0000 7f20617fa640 1 --2- 172.21.15.92:0/1803459146 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f205006e0b0 0x7f2050070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:52.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.246+0000 7f2068939640 1 --2- 172.21.15.92:0/1803459146 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f205006e0b0 0x7f2050070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:52.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.246+0000 7f20617fa640 1 -- 172.21.15.92:0/1803459146 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f205408d460 con 0x7f2064101a10 2024-05-06T12:25:52.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.246+0000 7f2068939640 1 --2- 172.21.15.92:0/1803459146 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f205006e0b0 0x7f2050070560 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f205c000f80 tx=0x7f205c02e040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:52.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.250+0000 7f20617fa640 1 -- 172.21.15.92:0/1803459146 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2054056790 con 0x7f2064101a10 2024-05-06T12:25:52.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:52 smithi190 ceph-mon[79744]: pgmap v315: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:25:52.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:52 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/1126518445' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-05-06T12:25:52.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:52 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/3455016012' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-05-06T12:25:52.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:52 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/837197790' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-05-06T12:25:52.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.462+0000 7f206ab9d640 1 -- 172.21.15.92:0/1803459146 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f2064065d40 con 0x7f2064101a10 2024-05-06T12:25:52.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.463+0000 7f20617fa640 1 -- 172.21.15.92:0/1803459146 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v42) v1 ==== 107+0+4223 (secure 0 0 0) 0x7f2054059e50 con 0x7f2064101a10 2024-05-06T12:25:52.465 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:25:52.465 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44261,"name":"cephfs.smithi190.idzoff","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6827/1642657019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1642657019},{"type":"v1","addr":"172.21.15.190:6827","nonce":1642657019}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44267,"name":"cephfs.smithi190.byjvgf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6825/3031977277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3031977277},{"type":"v1","addr":"172.21.15.190:6825","nonce":3031977277}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-06T12:11:44.952671+0000","modified":"2024-05-06T12:21:58.522512+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34438},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34438":{"gid":34438,"name":"cephfs.smithi092.ijsqgp","rank":0,"incarnation":31,"state":"up:replay","state_seq":1,"addr":"172.21.15.92:6827/4190573007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4190573007},{"type":"v1","addr":"172.21.15.92:6827","nonce":4190573007}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-06T12:25:52.466 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 31 2024-05-06T12:25:52.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.465+0000 7f206ab9d640 1 -- 172.21.15.92:0/1803459146 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f205006e0b0 msgr2=0x7f2050070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:52.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.465+0000 7f206ab9d640 1 --2- 172.21.15.92:0/1803459146 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f205006e0b0 0x7f2050070560 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f205c000f80 tx=0x7f205c02e040 comp rx=0 tx=0).stop 2024-05-06T12:25:52.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.465+0000 7f206ab9d640 1 -- 172.21.15.92:0/1803459146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2064101a10 msgr2=0x7f206410e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:52.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.465+0000 7f206ab9d640 1 --2- 172.21.15.92:0/1803459146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2064101a10 0x7f206410e390 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2054002a50 tx=0x7f2054004600 comp rx=0 tx=0).stop 2024-05-06T12:25:52.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.465+0000 7f206ab9d640 1 -- 172.21.15.92:0/1803459146 shutdown_connections 2024-05-06T12:25:52.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.465+0000 7f206ab9d640 1 --2- 172.21.15.92:0/1803459146 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f205006e0b0 0x7f2050070560 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:52.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.465+0000 7f206ab9d640 1 --2- 172.21.15.92:0/1803459146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2064101a10 0x7f206410e390 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:52.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.465+0000 7f206ab9d640 1 --2- 172.21.15.92:0/1803459146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2064101070 0x7f206410de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:52.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.465+0000 7f206ab9d640 1 -- 172.21.15.92:0/1803459146 >> 172.21.15.92:0/1803459146 conn(0x7f20640f6be0 msgr2=0x7f20640f7aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:52.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.465+0000 7f206ab9d640 1 -- 172.21.15.92:0/1803459146 shutdown_connections 2024-05-06T12:25:52.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.465+0000 7f206ab9d640 1 -- 172.21.15.92:0/1803459146 wait complete. 2024-05-06T12:25:52.478 DEBUG:tasks.fs:max_mds reduced in epoch 31 2024-05-06T12:25:52.478 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 31 2024-05-06T12:25:52.478 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-05-06T12:25:52.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.570+0000 7fce2d06e640 1 -- 172.21.15.92:0/1763020601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce28102b80 msgr2=0x7fce280fc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:52.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.570+0000 7fce2d06e640 1 --2- 172.21.15.92:0/1763020601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce28102b80 0x7fce280fc210 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fce1c009c50 tx=0x7fce1c004d10 comp rx=0 tx=0).stop 2024-05-06T12:25:52.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.571+0000 7fce2d06e640 1 -- 172.21.15.92:0/1763020601 shutdown_connections 2024-05-06T12:25:52.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.571+0000 7fce2d06e640 1 --2- 172.21.15.92:0/1763020601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce28102b80 0x7fce280fc210 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:52.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.571+0000 7fce2d06e640 1 --2- 172.21.15.92:0/1763020601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce281021e0 0x7fce281025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:52.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.571+0000 7fce2d06e640 1 -- 172.21.15.92:0/1763020601 >> 172.21.15.92:0/1763020601 conn(0x7fce280f7db0 msgr2=0x7fce280fa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:52.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.571+0000 7fce2d06e640 1 -- 172.21.15.92:0/1763020601 shutdown_connections 2024-05-06T12:25:52.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.571+0000 7fce2d06e640 1 -- 172.21.15.92:0/1763020601 wait complete. 2024-05-06T12:25:52.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.571+0000 7fce2d06e640 1 Processor -- start 2024-05-06T12:25:52.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.571+0000 7fce2d06e640 1 -- start start 2024-05-06T12:25:52.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.571+0000 7fce2d06e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce281021e0 0x7fce28112c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:52.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.571+0000 7fce2d06e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce28102b80 0x7fce281131a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:52.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.571+0000 7fce2d06e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce281137a0 con 0x7fce28102b80 2024-05-06T12:25:52.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.571+0000 7fce2d06e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2810cdf0 con 0x7fce281021e0 2024-05-06T12:25:52.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.571+0000 7fce2659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce28102b80 0x7fce281131a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:52.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.571+0000 7fce2659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce28102b80 0x7fce281131a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51880/0 (socket says 172.21.15.92:51880) 2024-05-06T12:25:52.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.571+0000 7fce2659c640 1 -- 172.21.15.92:0/2020132853 learned_addr learned my addr 172.21.15.92:0/2020132853 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:52.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.572+0000 7fce26d9d640 1 --2- 172.21.15.92:0/2020132853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce281021e0 0x7fce28112c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:52.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.572+0000 7fce2659c640 1 -- 172.21.15.92:0/2020132853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce281021e0 msgr2=0x7fce28112c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:52.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.572+0000 7fce2659c640 1 --2- 172.21.15.92:0/2020132853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce281021e0 0x7fce28112c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:52.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.572+0000 7fce2659c640 1 -- 172.21.15.92:0/2020132853 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce1c0098b0 con 0x7fce28102b80 2024-05-06T12:25:52.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.572+0000 7fce2659c640 1 --2- 172.21.15.92:0/2020132853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce28102b80 0x7fce281131a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fce1c004a30 tx=0x7fce1c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:52.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.572+0000 7fce0f7fe640 1 -- 172.21.15.92:0/2020132853 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce1c03a070 con 0x7fce28102b80 2024-05-06T12:25:52.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.572+0000 7fce2d06e640 1 -- 172.21.15.92:0/2020132853 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce2810d070 con 0x7fce28102b80 2024-05-06T12:25:52.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.572+0000 7fce0f7fe640 1 -- 172.21.15.92:0/2020132853 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce1c035b20 con 0x7fce28102b80 2024-05-06T12:25:52.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.572+0000 7fce0f7fe640 1 -- 172.21.15.92:0/2020132853 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce1c03eaa0 con 0x7fce28102b80 2024-05-06T12:25:52.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.572+0000 7fce2d06e640 1 -- 172.21.15.92:0/2020132853 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce2810d610 con 0x7fce28102b80 2024-05-06T12:25:52.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.573+0000 7fce2d06e640 1 -- 172.21.15.92:0/2020132853 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce280fd950 con 0x7fce28102b80 2024-05-06T12:25:52.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.573+0000 7fce0f7fe640 1 -- 172.21.15.92:0/2020132853 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fce1c032070 con 0x7fce28102b80 2024-05-06T12:25:52.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.574+0000 7fce0f7fe640 1 --2- 172.21.15.92:0/2020132853 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fce1006e0b0 0x7fce10070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:52.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.574+0000 7fce26d9d640 1 --2- 172.21.15.92:0/2020132853 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fce1006e0b0 0x7fce10070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:52.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.574+0000 7fce0f7fe640 1 -- 172.21.15.92:0/2020132853 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fce1c0b03d0 con 0x7fce28102b80 2024-05-06T12:25:52.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.574+0000 7fce26d9d640 1 --2- 172.21.15.92:0/2020132853 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fce1006e0b0 0x7fce10070560 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fce20004c40 tx=0x7fce20009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:52.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.579+0000 7fce0f7fe640 1 -- 172.21.15.92:0/2020132853 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fce1c079700 con 0x7fce28102b80 2024-05-06T12:25:52.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.790+0000 7fce2d06e640 1 -- 172.21.15.92:0/2020132853 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7fce28064300 con 0x7fce28102b80 2024-05-06T12:25:52.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.791+0000 7fce0f7fe640 1 -- 172.21.15.92:0/2020132853 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v42) v1 ==== 107+0+4226 (secure 0 0 0) 0x7fce1c07cdc0 con 0x7fce28102b80 2024-05-06T12:25:52.793 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:25:52.793 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44261,"name":"cephfs.smithi190.idzoff","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6827/1642657019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1642657019},{"type":"v1","addr":"172.21.15.190:6827","nonce":1642657019}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44267,"name":"cephfs.smithi190.byjvgf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6825/3031977277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3031977277},{"type":"v1","addr":"172.21.15.190:6825","nonce":3031977277}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-06T12:11:44.952671+0000","modified":"2024-05-06T12:22:04.366347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34438},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34438":{"gid":34438,"name":"cephfs.smithi092.ijsqgp","rank":0,"incarnation":31,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.92:6827/4190573007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4190573007},{"type":"v1","addr":"172.21.15.92:6827","nonce":4190573007}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-06T12:25:52.793 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 32 2024-05-06T12:25:52.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.793+0000 7fce2d06e640 1 -- 172.21.15.92:0/2020132853 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fce1006e0b0 msgr2=0x7fce10070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:52.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.793+0000 7fce2d06e640 1 --2- 172.21.15.92:0/2020132853 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fce1006e0b0 0x7fce10070560 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fce20004c40 tx=0x7fce20009040 comp rx=0 tx=0).stop 2024-05-06T12:25:52.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.793+0000 7fce2d06e640 1 -- 172.21.15.92:0/2020132853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce28102b80 msgr2=0x7fce281131a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:52.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.793+0000 7fce2d06e640 1 --2- 172.21.15.92:0/2020132853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce28102b80 0x7fce281131a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fce1c004a30 tx=0x7fce1c004d10 comp rx=0 tx=0).stop 2024-05-06T12:25:52.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.793+0000 7fce2d06e640 1 -- 172.21.15.92:0/2020132853 shutdown_connections 2024-05-06T12:25:52.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.793+0000 7fce2d06e640 1 --2- 172.21.15.92:0/2020132853 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fce1006e0b0 0x7fce10070560 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:52.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.793+0000 7fce2d06e640 1 --2- 172.21.15.92:0/2020132853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce28102b80 0x7fce281131a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:52.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.793+0000 7fce2d06e640 1 --2- 172.21.15.92:0/2020132853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce281021e0 0x7fce28112c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:52.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.793+0000 7fce2d06e640 1 -- 172.21.15.92:0/2020132853 >> 172.21.15.92:0/2020132853 conn(0x7fce280f7db0 msgr2=0x7fce280f9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:52.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.793+0000 7fce2d06e640 1 -- 172.21.15.92:0/2020132853 shutdown_connections 2024-05-06T12:25:52.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.793+0000 7fce2d06e640 1 -- 172.21.15.92:0/2020132853 wait complete. 2024-05-06T12:25:52.806 DEBUG:tasks.fs:max_mds reduced in epoch 32 2024-05-06T12:25:52.806 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 32 2024-05-06T12:25:52.806 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-05-06T12:25:52.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.898+0000 7fd458d5d640 1 -- 172.21.15.92:0/412421653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4540fdb60 msgr2=0x7fd4541061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:52.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.898+0000 7fd458d5d640 1 --2- 172.21.15.92:0/412421653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4540fdb60 0x7fd4541061a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0048d0 tx=0x7fd43c009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:52.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.898+0000 7fd458d5d640 1 -- 172.21.15.92:0/412421653 shutdown_connections 2024-05-06T12:25:52.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.898+0000 7fd458d5d640 1 --2- 172.21.15.92:0/412421653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4540fdb60 0x7fd4541061a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:52.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.898+0000 7fd458d5d640 1 --2- 172.21.15.92:0/412421653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4540fd1c0 0x7fd4540fd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:52.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.898+0000 7fd458d5d640 1 -- 172.21.15.92:0/412421653 >> 172.21.15.92:0/412421653 conn(0x7fd4540f6c10 msgr2=0x7fd4540f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:52.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.898+0000 7fd458d5d640 1 -- 172.21.15.92:0/412421653 shutdown_connections 2024-05-06T12:25:52.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.898+0000 7fd458d5d640 1 -- 172.21.15.92:0/412421653 wait complete. 2024-05-06T12:25:52.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.898+0000 7fd458d5d640 1 Processor -- start 2024-05-06T12:25:52.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.899+0000 7fd458d5d640 1 -- start start 2024-05-06T12:25:52.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.899+0000 7fd458d5d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4540fd1c0 0x7fd454112d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:52.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.899+0000 7fd458d5d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4540fdb60 0x7fd454113290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:52.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.899+0000 7fd458d5d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd454113890 con 0x7fd4540fd1c0 2024-05-06T12:25:52.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.899+0000 7fd458d5d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45410cee0 con 0x7fd4540fdb60 2024-05-06T12:25:52.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.899+0000 7fd45259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4540fd1c0 0x7fd454112d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:52.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.899+0000 7fd45259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4540fd1c0 0x7fd454112d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51898/0 (socket says 172.21.15.92:51898) 2024-05-06T12:25:52.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.899+0000 7fd45259c640 1 -- 172.21.15.92:0/3187000059 learned_addr learned my addr 172.21.15.92:0/3187000059 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:52.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.899+0000 7fd45259c640 1 -- 172.21.15.92:0/3187000059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4540fdb60 msgr2=0x7fd454113290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:52.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.899+0000 7fd45259c640 1 --2- 172.21.15.92:0/3187000059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4540fdb60 0x7fd454113290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:52.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.899+0000 7fd45259c640 1 -- 172.21.15.92:0/3187000059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd43c005040 con 0x7fd4540fd1c0 2024-05-06T12:25:52.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.899+0000 7fd45259c640 1 --2- 172.21.15.92:0/3187000059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4540fd1c0 0x7fd454112d50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd444002a50 tx=0x7fd444004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:52.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.899+0000 7fd44affd640 1 -- 172.21.15.92:0/3187000059 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd444007c30 con 0x7fd4540fd1c0 2024-05-06T12:25:52.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.899+0000 7fd44affd640 1 -- 172.21.15.92:0/3187000059 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd44400f040 con 0x7fd4540fd1c0 2024-05-06T12:25:52.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.899+0000 7fd458d5d640 1 -- 172.21.15.92:0/3187000059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd45410d1c0 con 0x7fd4540fd1c0 2024-05-06T12:25:52.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.899+0000 7fd44affd640 1 -- 172.21.15.92:0/3187000059 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd44400db50 con 0x7fd4540fd1c0 2024-05-06T12:25:52.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.899+0000 7fd458d5d640 1 -- 172.21.15.92:0/3187000059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd45410d6e0 con 0x7fd4540fd1c0 2024-05-06T12:25:52.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.900+0000 7fd458d5d640 1 -- 172.21.15.92:0/3187000059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd454066060 con 0x7fd4540fd1c0 2024-05-06T12:25:52.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.901+0000 7fd44affd640 1 -- 172.21.15.92:0/3187000059 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd44401c400 con 0x7fd4540fd1c0 2024-05-06T12:25:52.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.901+0000 7fd44affd640 1 --2- 172.21.15.92:0/3187000059 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd44006e0b0 0x7fd440070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:52.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.901+0000 7fd44bfff640 1 --2- 172.21.15.92:0/3187000059 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd44006e0b0 0x7fd440070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:52.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.901+0000 7fd44affd640 1 -- 172.21.15.92:0/3187000059 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd44408d470 con 0x7fd4540fd1c0 2024-05-06T12:25:52.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.902+0000 7fd44bfff640 1 --2- 172.21.15.92:0/3187000059 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd44006e0b0 0x7fd440070560 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0043d0 tx=0x7fd43c009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:52.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:52.906+0000 7fd44affd640 1 -- 172.21.15.92:0/3187000059 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd4440567a0 con 0x7fd4540fd1c0 2024-05-06T12:25:53.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.118+0000 7fd458d5d640 1 -- 172.21.15.92:0/3187000059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7fd454064350 con 0x7fd4540fd1c0 2024-05-06T12:25:53.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.119+0000 7fd44affd640 1 -- 172.21.15.92:0/3187000059 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v42) v1 ==== 107+0+4223 (secure 0 0 0) 0x7fd444059e60 con 0x7fd4540fd1c0 2024-05-06T12:25:53.121 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:25:53.121 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44261,"name":"cephfs.smithi190.idzoff","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6827/1642657019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1642657019},{"type":"v1","addr":"172.21.15.190:6827","nonce":1642657019}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44267,"name":"cephfs.smithi190.byjvgf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6825/3031977277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3031977277},{"type":"v1","addr":"172.21.15.190:6825","nonce":3031977277}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-06T12:11:44.952671+0000","modified":"2024-05-06T12:22:04.922678+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34438},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34438":{"gid":34438,"name":"cephfs.smithi092.ijsqgp","rank":0,"incarnation":31,"state":"up:rejoin","state_seq":7,"addr":"172.21.15.92:6827/4190573007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4190573007},{"type":"v1","addr":"172.21.15.92:6827","nonce":4190573007}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-06T12:25:53.121 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 33 2024-05-06T12:25:53.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.121+0000 7fd458d5d640 1 -- 172.21.15.92:0/3187000059 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd44006e0b0 msgr2=0x7fd440070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:53.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.121+0000 7fd458d5d640 1 --2- 172.21.15.92:0/3187000059 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd44006e0b0 0x7fd440070560 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0043d0 tx=0x7fd43c009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:53.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.121+0000 7fd458d5d640 1 -- 172.21.15.92:0/3187000059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4540fd1c0 msgr2=0x7fd454112d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:53.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.121+0000 7fd458d5d640 1 --2- 172.21.15.92:0/3187000059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4540fd1c0 0x7fd454112d50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd444002a50 tx=0x7fd444004600 comp rx=0 tx=0).stop 2024-05-06T12:25:53.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.121+0000 7fd458d5d640 1 -- 172.21.15.92:0/3187000059 shutdown_connections 2024-05-06T12:25:53.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.121+0000 7fd458d5d640 1 --2- 172.21.15.92:0/3187000059 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd44006e0b0 0x7fd440070560 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:53.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.121+0000 7fd458d5d640 1 --2- 172.21.15.92:0/3187000059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4540fdb60 0x7fd454113290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:53.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.121+0000 7fd458d5d640 1 --2- 172.21.15.92:0/3187000059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4540fd1c0 0x7fd454112d50 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:53.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.121+0000 7fd458d5d640 1 -- 172.21.15.92:0/3187000059 >> 172.21.15.92:0/3187000059 conn(0x7fd4540f6c10 msgr2=0x7fd4540f8050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:53.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.121+0000 7fd458d5d640 1 -- 172.21.15.92:0/3187000059 shutdown_connections 2024-05-06T12:25:53.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.121+0000 7fd458d5d640 1 -- 172.21.15.92:0/3187000059 wait complete. 2024-05-06T12:25:53.133 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:53 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/2735680722' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-05-06T12:25:53.133 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:53 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/1803459146' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-05-06T12:25:53.133 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:53 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/2020132853' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-05-06T12:25:53.134 DEBUG:tasks.fs:max_mds reduced in epoch 33 2024-05-06T12:25:53.134 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 33 2024-05-06T12:25:53.135 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-05-06T12:25:53.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.229+0000 7f178682e640 1 -- 172.21.15.92:0/3753459056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1780101070 msgr2=0x7f1780101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:53.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.229+0000 7f178682e640 1 --2- 172.21.15.92:0/3753459056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1780101070 0x7f1780101440 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f17700048d0 tx=0x7f1770009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:53.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.229+0000 7f178682e640 1 -- 172.21.15.92:0/3753459056 shutdown_connections 2024-05-06T12:25:53.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.229+0000 7f178682e640 1 --2- 172.21.15.92:0/3753459056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1780101a10 0x7f17800fb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:53.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.229+0000 7f178682e640 1 --2- 172.21.15.92:0/3753459056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1780101070 0x7f1780101440 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:53.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.229+0000 7f178682e640 1 -- 172.21.15.92:0/3753459056 >> 172.21.15.92:0/3753459056 conn(0x7f17800f6be0 msgr2=0x7f17800f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:53.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.229+0000 7f178682e640 1 -- 172.21.15.92:0/3753459056 shutdown_connections 2024-05-06T12:25:53.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.229+0000 7f178682e640 1 -- 172.21.15.92:0/3753459056 wait complete. 2024-05-06T12:25:53.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.229+0000 7f178682e640 1 Processor -- start 2024-05-06T12:25:53.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.229+0000 7f178682e640 1 -- start start 2024-05-06T12:25:53.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.230+0000 7f178682e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1780101070 0x7f178010dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:53.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.230+0000 7f178682e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1780101a10 0x7f178010e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:53.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.230+0000 7f178682e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178010e920 con 0x7f1780101070 2024-05-06T12:25:53.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.230+0000 7f178682e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178010ea90 con 0x7f1780101a10 2024-05-06T12:25:53.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.230+0000 7f177ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1780101070 0x7f178010dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:53.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.230+0000 7f177ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1780101070 0x7f178010dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51910/0 (socket says 172.21.15.92:51910) 2024-05-06T12:25:53.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.230+0000 7f177ffff640 1 -- 172.21.15.92:0/140677388 learned_addr learned my addr 172.21.15.92:0/140677388 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:53.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.230+0000 7f177ffff640 1 -- 172.21.15.92:0/140677388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1780101a10 msgr2=0x7f178010e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:53.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.230+0000 7f177f7fe640 1 --2- 172.21.15.92:0/140677388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1780101a10 0x7f178010e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:53.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.230+0000 7f177ffff640 1 --2- 172.21.15.92:0/140677388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1780101a10 0x7f178010e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:53.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.230+0000 7f177ffff640 1 -- 172.21.15.92:0/140677388 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1770005040 con 0x7f1780101070 2024-05-06T12:25:53.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.230+0000 7f177ffff640 1 --2- 172.21.15.92:0/140677388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1780101070 0x7f178010dea0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1770004930 tx=0x7f1770009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:53.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.230+0000 7f177cff9640 1 -- 172.21.15.92:0/140677388 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17700094c0 con 0x7f1780101070 2024-05-06T12:25:53.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.230+0000 7f177cff9640 1 -- 172.21.15.92:0/140677388 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1770032e50 con 0x7f1780101070 2024-05-06T12:25:53.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.230+0000 7f178682e640 1 -- 172.21.15.92:0/140677388 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f178010ed10 con 0x7f1780101070 2024-05-06T12:25:53.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.230+0000 7f177cff9640 1 -- 172.21.15.92:0/140677388 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17700433b0 con 0x7f1780101070 2024-05-06T12:25:53.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.230+0000 7f178682e640 1 -- 172.21.15.92:0/140677388 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17801171c0 con 0x7f1780101070 2024-05-06T12:25:53.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.231+0000 7f178682e640 1 -- 172.21.15.92:0/140677388 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17800fc7e0 con 0x7f1780101070 2024-05-06T12:25:53.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.232+0000 7f177cff9640 1 -- 172.21.15.92:0/140677388 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f177002f070 con 0x7f1780101070 2024-05-06T12:25:53.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.232+0000 7f177cff9640 1 --2- 172.21.15.92:0/140677388 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f176806e0b0 0x7f1768070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:53.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.232+0000 7f177cff9640 1 -- 172.21.15.92:0/140677388 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f17700af8a0 con 0x7f1780101070 2024-05-06T12:25:53.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.232+0000 7f177f7fe640 1 --2- 172.21.15.92:0/140677388 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f176806e0b0 0x7f1768070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:53.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.233+0000 7f177f7fe640 1 --2- 172.21.15.92:0/140677388 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f176806e0b0 0x7f1768070560 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1780065f70 tx=0x7f1774008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:53.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.237+0000 7f177cff9640 1 -- 172.21.15.92:0/140677388 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1770078bd0 con 0x7f1780101070 2024-05-06T12:25:53.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:53 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/2735680722' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-05-06T12:25:53.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:53 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/1803459146' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-05-06T12:25:53.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:53 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/2020132853' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-05-06T12:25:53.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.451+0000 7f178682e640 1 -- 172.21.15.92:0/140677388 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f1780064350 con 0x7f1780101070 2024-05-06T12:25:53.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.452+0000 7f177cff9640 1 -- 172.21.15.92:0/140677388 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v42) v1 ==== 107+0+5079 (secure 0 0 0) 0x7f177007c290 con 0x7f1780101070 2024-05-06T12:25:53.454 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:25:53.454 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34466,"name":"cephfs.smithi092.rosbll","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6829/2398082173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":2398082173},{"type":"v1","addr":"172.21.15.92:6829","nonce":2398082173}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":44261,"name":"cephfs.smithi190.idzoff","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6827/1642657019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1642657019},{"type":"v1","addr":"172.21.15.190:6827","nonce":1642657019}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44267,"name":"cephfs.smithi190.byjvgf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6825/3031977277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3031977277},{"type":"v1","addr":"172.21.15.190:6825","nonce":3031977277}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-06T12:11:44.952671+0000","modified":"2024-05-06T12:22:06.925528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34438},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34438":{"gid":34438,"name":"cephfs.smithi092.ijsqgp","rank":0,"incarnation":31,"state":"up:active","state_seq":8,"addr":"172.21.15.92:6827/4190573007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4190573007},{"type":"v1","addr":"172.21.15.92:6827","nonce":4190573007}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34438,"qdb_cluster":[34438]},"id":1}]} 2024-05-06T12:25:53.455 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 34 2024-05-06T12:25:53.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.454+0000 7f178682e640 1 -- 172.21.15.92:0/140677388 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f176806e0b0 msgr2=0x7f1768070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:53.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.454+0000 7f178682e640 1 --2- 172.21.15.92:0/140677388 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f176806e0b0 0x7f1768070560 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1780065f70 tx=0x7f1774008040 comp rx=0 tx=0).stop 2024-05-06T12:25:53.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.454+0000 7f178682e640 1 -- 172.21.15.92:0/140677388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1780101070 msgr2=0x7f178010dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:53.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.454+0000 7f178682e640 1 --2- 172.21.15.92:0/140677388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1780101070 0x7f178010dea0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1770004930 tx=0x7f1770009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:53.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.454+0000 7f178682e640 1 -- 172.21.15.92:0/140677388 shutdown_connections 2024-05-06T12:25:53.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.454+0000 7f178682e640 1 --2- 172.21.15.92:0/140677388 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f176806e0b0 0x7f1768070560 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:53.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.454+0000 7f178682e640 1 --2- 172.21.15.92:0/140677388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1780101a10 0x7f178010e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:53.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.454+0000 7f178682e640 1 --2- 172.21.15.92:0/140677388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1780101070 0x7f178010dea0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:53.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.454+0000 7f178682e640 1 -- 172.21.15.92:0/140677388 >> 172.21.15.92:0/140677388 conn(0x7f17800f6be0 msgr2=0x7f17800f7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:53.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.454+0000 7f178682e640 1 -- 172.21.15.92:0/140677388 shutdown_connections 2024-05-06T12:25:53.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.454+0000 7f178682e640 1 -- 172.21.15.92:0/140677388 wait complete. 2024-05-06T12:25:53.467 DEBUG:tasks.fs:max_mds reduced in epoch 34 2024-05-06T12:25:53.468 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 34 2024-05-06T12:25:53.468 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-05-06T12:25:53.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.560+0000 7ff09a06e640 1 -- 172.21.15.92:0/4050363354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff094102b80 msgr2=0x7ff0940fc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:53.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.560+0000 7ff09a06e640 1 --2- 172.21.15.92:0/4050363354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff094102b80 0x7ff0940fc210 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff0840048d0 tx=0x7ff084009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:53.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.561+0000 7ff09a06e640 1 -- 172.21.15.92:0/4050363354 shutdown_connections 2024-05-06T12:25:53.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.561+0000 7ff09a06e640 1 --2- 172.21.15.92:0/4050363354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff094102b80 0x7ff0940fc210 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:53.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.561+0000 7ff09a06e640 1 --2- 172.21.15.92:0/4050363354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0941021e0 0x7ff0941025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:53.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.561+0000 7ff09a06e640 1 -- 172.21.15.92:0/4050363354 >> 172.21.15.92:0/4050363354 conn(0x7ff0940f7db0 msgr2=0x7ff0940fa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:53.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.561+0000 7ff09a06e640 1 -- 172.21.15.92:0/4050363354 shutdown_connections 2024-05-06T12:25:53.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.561+0000 7ff09a06e640 1 -- 172.21.15.92:0/4050363354 wait complete. 2024-05-06T12:25:53.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.561+0000 7ff09a06e640 1 Processor -- start 2024-05-06T12:25:53.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.561+0000 7ff09a06e640 1 -- start start 2024-05-06T12:25:53.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.561+0000 7ff09a06e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0941021e0 0x7ff094112c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:53.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.561+0000 7ff09a06e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff094102b80 0x7ff0941131a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:53.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.561+0000 7ff09a06e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0941137a0 con 0x7ff0941021e0 2024-05-06T12:25:53.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.561+0000 7ff09a06e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09410cdf0 con 0x7ff094102b80 2024-05-06T12:25:53.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.561+0000 7ff0937fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0941021e0 0x7ff094112c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:53.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.561+0000 7ff0937fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0941021e0 0x7ff094112c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51924/0 (socket says 172.21.15.92:51924) 2024-05-06T12:25:53.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.561+0000 7ff0937fe640 1 -- 172.21.15.92:0/2230938214 learned_addr learned my addr 172.21.15.92:0/2230938214 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:53.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.562+0000 7ff092ffd640 1 --2- 172.21.15.92:0/2230938214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff094102b80 0x7ff0941131a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:53.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.562+0000 7ff0937fe640 1 -- 172.21.15.92:0/2230938214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff094102b80 msgr2=0x7ff0941131a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:53.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.562+0000 7ff0937fe640 1 --2- 172.21.15.92:0/2230938214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff094102b80 0x7ff0941131a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:53.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.562+0000 7ff0937fe640 1 -- 172.21.15.92:0/2230938214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff084005040 con 0x7ff0941021e0 2024-05-06T12:25:53.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.562+0000 7ff0937fe640 1 --2- 172.21.15.92:0/2230938214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0941021e0 0x7ff094112c60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff08800c830 tx=0x7ff088004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:53.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.562+0000 7ff07bfff640 1 -- 172.21.15.92:0/2230938214 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff08800a850 con 0x7ff0941021e0 2024-05-06T12:25:53.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.562+0000 7ff09a06e640 1 -- 172.21.15.92:0/2230938214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff09410d0d0 con 0x7ff0941021e0 2024-05-06T12:25:53.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.562+0000 7ff07bfff640 1 -- 172.21.15.92:0/2230938214 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff08800ae80 con 0x7ff0941021e0 2024-05-06T12:25:53.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.562+0000 7ff07bfff640 1 -- 172.21.15.92:0/2230938214 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff088012ea0 con 0x7ff0941021e0 2024-05-06T12:25:53.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.562+0000 7ff09a06e640 1 -- 172.21.15.92:0/2230938214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff09410d5c0 con 0x7ff0941021e0 2024-05-06T12:25:53.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.563+0000 7ff09a06e640 1 -- 172.21.15.92:0/2230938214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0940fd950 con 0x7ff0941021e0 2024-05-06T12:25:53.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.563+0000 7ff07bfff640 1 -- 172.21.15.92:0/2230938214 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff088012400 con 0x7ff0941021e0 2024-05-06T12:25:53.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.564+0000 7ff07bfff640 1 --2- 172.21.15.92:0/2230938214 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff07c06e0b0 0x7ff07c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:53.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.564+0000 7ff07bfff640 1 -- 172.21.15.92:0/2230938214 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff08808dde0 con 0x7ff0941021e0 2024-05-06T12:25:53.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.564+0000 7ff092ffd640 1 --2- 172.21.15.92:0/2230938214 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff07c06e0b0 0x7ff07c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:53.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.564+0000 7ff092ffd640 1 --2- 172.21.15.92:0/2230938214 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff07c06e0b0 0x7ff07c070560 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff0940f93f0 tx=0x7ff084009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:53.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.569+0000 7ff07bfff640 1 -- 172.21.15.92:0/2230938214 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff088057190 con 0x7ff0941021e0 2024-05-06T12:25:53.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.781+0000 7ff09a06e640 1 -- 172.21.15.92:0/2230938214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7ff094109320 con 0x7ff0941021e0 2024-05-06T12:25:53.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.781+0000 7ff07bfff640 1 -- 172.21.15.92:0/2230938214 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v42) v1 ==== 107+0+4297 (secure 0 0 0) 0x7ff0880126b0 con 0x7ff0941021e0 2024-05-06T12:25:53.784 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:25:53.784 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34466,"name":"cephfs.smithi092.rosbll","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6829/2398082173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":2398082173},{"type":"v1","addr":"172.21.15.92:6829","nonce":2398082173}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":44261,"name":"cephfs.smithi190.idzoff","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6827/1642657019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1642657019},{"type":"v1","addr":"172.21.15.190:6827","nonce":1642657019}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-06T12:11:44.952671+0000","modified":"2024-05-06T12:22:06.925528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34438},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34438":{"gid":34438,"name":"cephfs.smithi092.ijsqgp","rank":0,"incarnation":31,"state":"up:active","state_seq":8,"addr":"172.21.15.92:6827/4190573007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4190573007},{"type":"v1","addr":"172.21.15.92:6827","nonce":4190573007}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34438,"qdb_cluster":[34438]},"id":1}]} 2024-05-06T12:25:53.784 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 35 2024-05-06T12:25:53.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.783+0000 7ff09a06e640 1 -- 172.21.15.92:0/2230938214 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff07c06e0b0 msgr2=0x7ff07c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:53.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.783+0000 7ff09a06e640 1 --2- 172.21.15.92:0/2230938214 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff07c06e0b0 0x7ff07c070560 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff0940f93f0 tx=0x7ff084009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:53.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.784+0000 7ff09a06e640 1 -- 172.21.15.92:0/2230938214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0941021e0 msgr2=0x7ff094112c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:53.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.784+0000 7ff09a06e640 1 --2- 172.21.15.92:0/2230938214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0941021e0 0x7ff094112c60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff08800c830 tx=0x7ff088004430 comp rx=0 tx=0).stop 2024-05-06T12:25:53.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.784+0000 7ff09a06e640 1 -- 172.21.15.92:0/2230938214 shutdown_connections 2024-05-06T12:25:53.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.784+0000 7ff09a06e640 1 --2- 172.21.15.92:0/2230938214 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff07c06e0b0 0x7ff07c070560 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:53.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.784+0000 7ff09a06e640 1 --2- 172.21.15.92:0/2230938214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff094102b80 0x7ff0941131a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:53.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.784+0000 7ff09a06e640 1 --2- 172.21.15.92:0/2230938214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0941021e0 0x7ff094112c60 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:53.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.784+0000 7ff09a06e640 1 -- 172.21.15.92:0/2230938214 >> 172.21.15.92:0/2230938214 conn(0x7ff0940f7db0 msgr2=0x7ff0940f9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:53.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.784+0000 7ff09a06e640 1 -- 172.21.15.92:0/2230938214 shutdown_connections 2024-05-06T12:25:53.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.784+0000 7ff09a06e640 1 -- 172.21.15.92:0/2230938214 wait complete. 2024-05-06T12:25:53.797 DEBUG:tasks.fs:max_mds reduced in epoch 35 2024-05-06T12:25:53.797 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 35 2024-05-06T12:25:53.797 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 36 2024-05-06T12:25:53.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.891+0000 7f520c59d640 1 -- 172.21.15.92:0/3413661654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5204101a10 msgr2=0x7f52040fb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:53.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.891+0000 7f520c59d640 1 --2- 172.21.15.92:0/3413661654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5204101a10 0x7f52040fb0a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f51f80048d0 tx=0x7f51f8009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:53.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.891+0000 7f520c59d640 1 -- 172.21.15.92:0/3413661654 shutdown_connections 2024-05-06T12:25:53.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.891+0000 7f520c59d640 1 --2- 172.21.15.92:0/3413661654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5204101a10 0x7f52040fb0a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:53.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.891+0000 7f520c59d640 1 --2- 172.21.15.92:0/3413661654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5204101070 0x7f5204101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:53.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.891+0000 7f520c59d640 1 -- 172.21.15.92:0/3413661654 >> 172.21.15.92:0/3413661654 conn(0x7f52040f6be0 msgr2=0x7f52040f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:53.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.891+0000 7f520c59d640 1 -- 172.21.15.92:0/3413661654 shutdown_connections 2024-05-06T12:25:53.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.891+0000 7f520c59d640 1 -- 172.21.15.92:0/3413661654 wait complete. 2024-05-06T12:25:53.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.892+0000 7f520c59d640 1 Processor -- start 2024-05-06T12:25:53.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.892+0000 7f520c59d640 1 -- start start 2024-05-06T12:25:53.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.892+0000 7f520c59d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5204101070 0x7f5204112c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:53.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.892+0000 7f520c59d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5204101a10 0x7f52041131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:53.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.892+0000 7f520c59d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52041137d0 con 0x7f5204101070 2024-05-06T12:25:53.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.892+0000 7f520c59d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520410ce20 con 0x7f5204101a10 2024-05-06T12:25:53.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.892+0000 7f520a339640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5204101070 0x7f5204112c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:53.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.892+0000 7f520a339640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5204101070 0x7f5204112c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51940/0 (socket says 172.21.15.92:51940) 2024-05-06T12:25:53.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.892+0000 7f520a339640 1 -- 172.21.15.92:0/766706485 learned_addr learned my addr 172.21.15.92:0/766706485 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:53.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.892+0000 7f5209b38640 1 --2- 172.21.15.92:0/766706485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5204101a10 0x7f52041131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:53.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.892+0000 7f520a339640 1 -- 172.21.15.92:0/766706485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5204101a10 msgr2=0x7f52041131d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:53.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.892+0000 7f520a339640 1 --2- 172.21.15.92:0/766706485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5204101a10 0x7f52041131d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:53.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.892+0000 7f520a339640 1 -- 172.21.15.92:0/766706485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f8005040 con 0x7f5204101070 2024-05-06T12:25:53.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.892+0000 7f520a339640 1 --2- 172.21.15.92:0/766706485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5204101070 0x7f5204112c90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00cb10 tx=0x7f51fc007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:53.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.892+0000 7f5202ffd640 1 -- 172.21.15.92:0/766706485 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51fc004200 con 0x7f5204101070 2024-05-06T12:25:53.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.892+0000 7f5202ffd640 1 -- 172.21.15.92:0/766706485 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f51fc00ac70 con 0x7f5204101070 2024-05-06T12:25:53.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.892+0000 7f520c59d640 1 -- 172.21.15.92:0/766706485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f520410d100 con 0x7f5204101070 2024-05-06T12:25:53.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.892+0000 7f5202ffd640 1 -- 172.21.15.92:0/766706485 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51fc003db0 con 0x7f5204101070 2024-05-06T12:25:53.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.893+0000 7f520c59d640 1 -- 172.21.15.92:0/766706485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f520410d5f0 con 0x7f5204101070 2024-05-06T12:25:53.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.893+0000 7f520c59d640 1 -- 172.21.15.92:0/766706485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52040fc7e0 con 0x7f5204101070 2024-05-06T12:25:53.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.894+0000 7f5202ffd640 1 -- 172.21.15.92:0/766706485 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f51fc00a7a0 con 0x7f5204101070 2024-05-06T12:25:53.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.894+0000 7f5202ffd640 1 --2- 172.21.15.92:0/766706485 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f51ec06e0b0 0x7f51ec070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:53.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.894+0000 7f5202ffd640 1 -- 172.21.15.92:0/766706485 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f51fc08e1a0 con 0x7f5204101070 2024-05-06T12:25:53.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.894+0000 7f5209b38640 1 --2- 172.21.15.92:0/766706485 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f51ec06e0b0 0x7f51ec070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:53.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.895+0000 7f5209b38640 1 --2- 172.21.15.92:0/766706485 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f51ec06e0b0 0x7f51ec070560 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f51f80048a0 tx=0x7f51f8009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:53.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:53.899+0000 7f5202ffd640 1 -- 172.21.15.92:0/766706485 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f51fc0574d0 con 0x7f5204101070 2024-05-06T12:25:54.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.109+0000 7f520c59d640 1 -- 172.21.15.92:0/766706485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7f520410e330 con 0x7f5204101070 2024-05-06T12:25:54.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.110+0000 7f5202ffd640 1 -- 172.21.15.92:0/766706485 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v42) v1 ==== 107+0+5147 (secure 0 0 0) 0x7f51fc05ab90 con 0x7f5204101070 2024-05-06T12:25:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:25:54.112 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34466,"name":"cephfs.smithi092.rosbll","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6829/2398082173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":2398082173},{"type":"v1","addr":"172.21.15.92:6829","nonce":2398082173}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":44261,"name":"cephfs.smithi190.idzoff","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6827/1642657019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1642657019},{"type":"v1","addr":"172.21.15.190:6827","nonce":1642657019}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44309,"name":"cephfs.smithi190.byjvgf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6825/1219274291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1219274291},{"type":"v1","addr":"172.21.15.190:6825","nonce":1219274291}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-06T12:11:44.952671+0000","modified":"2024-05-06T12:22:06.925528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34438},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34438":{"gid":34438,"name":"cephfs.smithi092.ijsqgp","rank":0,"incarnation":31,"state":"up:active","state_seq":8,"addr":"172.21.15.92:6827/4190573007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4190573007},{"type":"v1","addr":"172.21.15.92:6827","nonce":4190573007}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34438,"qdb_cluster":[34438]},"id":1}]} 2024-05-06T12:25:54.113 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 36 2024-05-06T12:25:54.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.112+0000 7f520c59d640 1 -- 172.21.15.92:0/766706485 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f51ec06e0b0 msgr2=0x7f51ec070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:54.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.112+0000 7f520c59d640 1 --2- 172.21.15.92:0/766706485 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f51ec06e0b0 0x7f51ec070560 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f51f80048a0 tx=0x7f51f8009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:54.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.112+0000 7f520c59d640 1 -- 172.21.15.92:0/766706485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5204101070 msgr2=0x7f5204112c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:54.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.112+0000 7f520c59d640 1 --2- 172.21.15.92:0/766706485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5204101070 0x7f5204112c90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00cb10 tx=0x7f51fc007ee0 comp rx=0 tx=0).stop 2024-05-06T12:25:54.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.112+0000 7f520c59d640 1 -- 172.21.15.92:0/766706485 shutdown_connections 2024-05-06T12:25:54.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.112+0000 7f520c59d640 1 --2- 172.21.15.92:0/766706485 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f51ec06e0b0 0x7f51ec070560 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:54.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.112+0000 7f520c59d640 1 --2- 172.21.15.92:0/766706485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5204101a10 0x7f52041131d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:54.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.112+0000 7f520c59d640 1 --2- 172.21.15.92:0/766706485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5204101070 0x7f5204112c90 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:54.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.112+0000 7f520c59d640 1 -- 172.21.15.92:0/766706485 >> 172.21.15.92:0/766706485 conn(0x7f52040f6be0 msgr2=0x7f52040657b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:54.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.112+0000 7f520c59d640 1 -- 172.21.15.92:0/766706485 shutdown_connections 2024-05-06T12:25:54.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.112+0000 7f520c59d640 1 -- 172.21.15.92:0/766706485 wait complete. 2024-05-06T12:25:54.124 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:54 smithi092 ceph-mon[91933]: pgmap v316: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:25:54.124 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:54 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/3187000059' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-05-06T12:25:54.124 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:54 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/140677388' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-05-06T12:25:54.124 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:54 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/2230938214' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-05-06T12:25:54.124 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:54 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:25:54.125 DEBUG:tasks.fs:max_mds reduced in epoch 36 2024-05-06T12:25:54.125 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 36 2024-05-06T12:25:54.125 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 37 2024-05-06T12:25:54.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.217+0000 7f7f4c26e640 1 -- 172.21.15.92:0/3724737615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f441010a0 msgr2=0x7f7f44101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:54.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.217+0000 7f7f4c26e640 1 --2- 172.21.15.92:0/3724737615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f441010a0 0x7f7f44101470 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7f380048d0 tx=0x7f7f38009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:54.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.218+0000 7f7f4c26e640 1 -- 172.21.15.92:0/3724737615 shutdown_connections 2024-05-06T12:25:54.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.218+0000 7f7f4c26e640 1 --2- 172.21.15.92:0/3724737615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f44101a40 0x7f7f440fb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:54.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.218+0000 7f7f4c26e640 1 --2- 172.21.15.92:0/3724737615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f441010a0 0x7f7f44101470 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:54.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.218+0000 7f7f4c26e640 1 -- 172.21.15.92:0/3724737615 >> 172.21.15.92:0/3724737615 conn(0x7f7f440f6c50 msgr2=0x7f7f440f9060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:54.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.218+0000 7f7f4c26e640 1 -- 172.21.15.92:0/3724737615 shutdown_connections 2024-05-06T12:25:54.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.218+0000 7f7f4c26e640 1 -- 172.21.15.92:0/3724737615 wait complete. 2024-05-06T12:25:54.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.218+0000 7f7f4c26e640 1 Processor -- start 2024-05-06T12:25:54.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.218+0000 7f7f4c26e640 1 -- start start 2024-05-06T12:25:54.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.218+0000 7f7f4c26e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f441010a0 0x7f7f44101f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:54.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.218+0000 7f7f4c26e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f44101a40 0x7f7f44102460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:54.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.218+0000 7f7f4c26e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f44103e00 con 0x7f7f441010a0 2024-05-06T12:25:54.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.218+0000 7f7f4c26e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f441029a0 con 0x7f7f44101a40 2024-05-06T12:25:54.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.218+0000 7f7f4a00a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f441010a0 0x7f7f44101f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:54.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.218+0000 7f7f4a00a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f441010a0 0x7f7f44101f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51960/0 (socket says 172.21.15.92:51960) 2024-05-06T12:25:54.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.218+0000 7f7f4a00a640 1 -- 172.21.15.92:0/3546812651 learned_addr learned my addr 172.21.15.92:0/3546812651 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:54.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.218+0000 7f7f49809640 1 --2- 172.21.15.92:0/3546812651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f44101a40 0x7f7f44102460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:54.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.219+0000 7f7f4a00a640 1 -- 172.21.15.92:0/3546812651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f44101a40 msgr2=0x7f7f44102460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:54.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.219+0000 7f7f4a00a640 1 --2- 172.21.15.92:0/3546812651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f44101a40 0x7f7f44102460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:54.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.219+0000 7f7f4a00a640 1 -- 172.21.15.92:0/3546812651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f38005040 con 0x7f7f441010a0 2024-05-06T12:25:54.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.219+0000 7f7f49809640 1 --2- 172.21.15.92:0/3546812651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f44101a40 0x7f7f44102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-06T12:25:54.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.219+0000 7f7f4a00a640 1 --2- 172.21.15.92:0/3546812651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f441010a0 0x7f7f44101f20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7f38004930 tx=0x7f7f38009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:54.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.219+0000 7f7f36ffd640 1 -- 172.21.15.92:0/3546812651 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f380094c0 con 0x7f7f441010a0 2024-05-06T12:25:54.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.219+0000 7f7f36ffd640 1 -- 172.21.15.92:0/3546812651 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f38032e50 con 0x7f7f441010a0 2024-05-06T12:25:54.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.219+0000 7f7f4c26e640 1 -- 172.21.15.92:0/3546812651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f44102c20 con 0x7f7f441010a0 2024-05-06T12:25:54.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.219+0000 7f7f36ffd640 1 -- 172.21.15.92:0/3546812651 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f380433b0 con 0x7f7f441010a0 2024-05-06T12:25:54.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.219+0000 7f7f4c26e640 1 -- 172.21.15.92:0/3546812651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f44119390 con 0x7f7f441010a0 2024-05-06T12:25:54.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.220+0000 7f7f4c26e640 1 -- 172.21.15.92:0/3546812651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f440fc810 con 0x7f7f441010a0 2024-05-06T12:25:54.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.220+0000 7f7f36ffd640 1 -- 172.21.15.92:0/3546812651 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7f3802f070 con 0x7f7f441010a0 2024-05-06T12:25:54.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.221+0000 7f7f36ffd640 1 --2- 172.21.15.92:0/3546812651 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7f3006e0b0 0x7f7f30070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:54.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.221+0000 7f7f36ffd640 1 -- 172.21.15.92:0/3546812651 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7f380af960 con 0x7f7f441010a0 2024-05-06T12:25:54.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.221+0000 7f7f49809640 1 --2- 172.21.15.92:0/3546812651 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7f3006e0b0 0x7f7f30070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:54.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.221+0000 7f7f49809640 1 --2- 172.21.15.92:0/3546812651 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7f3006e0b0 0x7f7f30070560 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7f40007fd0 tx=0x7f7f40009e90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:54.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.225+0000 7f7f36ffd640 1 -- 172.21.15.92:0/3546812651 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7f38078c90 con 0x7f7f441010a0 2024-05-06T12:25:54.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:54 smithi190 ceph-mon[79744]: pgmap v316: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:25:54.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:54 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/3187000059' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-05-06T12:25:54.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:54 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/140677388' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-05-06T12:25:54.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:54 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/2230938214' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-05-06T12:25:54.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:54 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:25:54.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.436+0000 7f7f4c26e640 1 -- 172.21.15.92:0/3546812651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 37, "format": "json"} v 0) v1 -- 0x7f7f44064350 con 0x7f7f441010a0 2024-05-06T12:25:54.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.436+0000 7f7f36ffd640 1 -- 172.21.15.92:0/3546812651 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 37, "format": "json"}]=0 dumped fsmap epoch 37 v42) v1 ==== 107+0+4365 (secure 0 0 0) 0x7f7f3807c350 con 0x7f7f441010a0 2024-05-06T12:25:54.439 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:25:54.439 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34466,"name":"cephfs.smithi092.rosbll","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6829/2398082173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":2398082173},{"type":"v1","addr":"172.21.15.92:6829","nonce":2398082173}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":44309,"name":"cephfs.smithi190.byjvgf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6825/1219274291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1219274291},{"type":"v1","addr":"172.21.15.190:6825","nonce":1219274291}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-06T12:11:44.952671+0000","modified":"2024-05-06T12:22:06.925528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34438},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34438":{"gid":34438,"name":"cephfs.smithi092.ijsqgp","rank":0,"incarnation":31,"state":"up:active","state_seq":8,"addr":"172.21.15.92:6827/4190573007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4190573007},{"type":"v1","addr":"172.21.15.92:6827","nonce":4190573007}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34438,"qdb_cluster":[34438]},"id":1}]} 2024-05-06T12:25:54.439 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 37 2024-05-06T12:25:54.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.438+0000 7f7f4c26e640 1 -- 172.21.15.92:0/3546812651 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7f3006e0b0 msgr2=0x7f7f30070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:54.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.438+0000 7f7f4c26e640 1 --2- 172.21.15.92:0/3546812651 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7f3006e0b0 0x7f7f30070560 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7f40007fd0 tx=0x7f7f40009e90 comp rx=0 tx=0).stop 2024-05-06T12:25:54.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.438+0000 7f7f4c26e640 1 -- 172.21.15.92:0/3546812651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f441010a0 msgr2=0x7f7f44101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:54.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.438+0000 7f7f4c26e640 1 --2- 172.21.15.92:0/3546812651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f441010a0 0x7f7f44101f20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7f38004930 tx=0x7f7f38009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:54.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.438+0000 7f7f4c26e640 1 -- 172.21.15.92:0/3546812651 shutdown_connections 2024-05-06T12:25:54.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.438+0000 7f7f4c26e640 1 --2- 172.21.15.92:0/3546812651 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f7f3006e0b0 0x7f7f30070560 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:54.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.438+0000 7f7f4c26e640 1 --2- 172.21.15.92:0/3546812651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f44101a40 0x7f7f44102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:54.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.438+0000 7f7f4c26e640 1 --2- 172.21.15.92:0/3546812651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f441010a0 0x7f7f44101f20 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:54.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.438+0000 7f7f4c26e640 1 -- 172.21.15.92:0/3546812651 >> 172.21.15.92:0/3546812651 conn(0x7f7f440f6c50 msgr2=0x7f7f440f7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:54.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.439+0000 7f7f4c26e640 1 -- 172.21.15.92:0/3546812651 shutdown_connections 2024-05-06T12:25:54.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.439+0000 7f7f4c26e640 1 -- 172.21.15.92:0/3546812651 wait complete. 2024-05-06T12:25:54.452 DEBUG:tasks.fs:max_mds reduced in epoch 37 2024-05-06T12:25:54.452 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 37 2024-05-06T12:25:54.452 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 38 2024-05-06T12:25:54.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.545+0000 7fa5fc06e640 1 -- 172.21.15.92:0/304590388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5f4101070 msgr2=0x7fa5f4101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:54.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.545+0000 7fa5fc06e640 1 --2- 172.21.15.92:0/304590388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5f4101070 0x7fa5f4101440 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa5f00048d0 tx=0x7fa5f0009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:54.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.545+0000 7fa5fc06e640 1 -- 172.21.15.92:0/304590388 shutdown_connections 2024-05-06T12:25:54.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.545+0000 7fa5fc06e640 1 --2- 172.21.15.92:0/304590388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5f4101a10 0x7fa5f40fb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:54.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.545+0000 7fa5fc06e640 1 --2- 172.21.15.92:0/304590388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5f4101070 0x7fa5f4101440 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:54.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.545+0000 7fa5fc06e640 1 -- 172.21.15.92:0/304590388 >> 172.21.15.92:0/304590388 conn(0x7fa5f40f6be0 msgr2=0x7fa5f40f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:54.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.545+0000 7fa5fc06e640 1 -- 172.21.15.92:0/304590388 shutdown_connections 2024-05-06T12:25:54.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.545+0000 7fa5fc06e640 1 -- 172.21.15.92:0/304590388 wait complete. 2024-05-06T12:25:54.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.545+0000 7fa5fc06e640 1 Processor -- start 2024-05-06T12:25:54.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.545+0000 7fa5fc06e640 1 -- start start 2024-05-06T12:25:54.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.546+0000 7fa5fc06e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5f4101070 0x7fa5f4112ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:54.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.546+0000 7fa5fc06e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5f4101a10 0x7fa5f4113220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:54.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.546+0000 7fa5fc06e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f41138b0 con 0x7fa5f4101a10 2024-05-06T12:25:54.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.546+0000 7fa5fc06e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f410ce70 con 0x7fa5f4101070 2024-05-06T12:25:54.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.546+0000 7fa5f9609640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5f4101a10 0x7fa5f4113220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:54.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.546+0000 7fa5f9609640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5f4101a10 0x7fa5f4113220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51974/0 (socket says 172.21.15.92:51974) 2024-05-06T12:25:54.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.546+0000 7fa5f9609640 1 -- 172.21.15.92:0/3139196499 learned_addr learned my addr 172.21.15.92:0/3139196499 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:54.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.546+0000 7fa5f9e0a640 1 --2- 172.21.15.92:0/3139196499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5f4101070 0x7fa5f4112ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:54.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.546+0000 7fa5f9609640 1 -- 172.21.15.92:0/3139196499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5f4101070 msgr2=0x7fa5f4112ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:54.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.546+0000 7fa5f9609640 1 --2- 172.21.15.92:0/3139196499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5f4101070 0x7fa5f4112ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:54.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.546+0000 7fa5f9609640 1 -- 172.21.15.92:0/3139196499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f0005040 con 0x7fa5f4101a10 2024-05-06T12:25:54.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.546+0000 7fa5f9609640 1 --2- 172.21.15.92:0/3139196499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5f4101a10 0x7fa5f4113220 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400b790 tx=0x7fa5e40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:54.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.546+0000 7fa5ea7fc640 1 -- 172.21.15.92:0/3139196499 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5e400c700 con 0x7fa5f4101a10 2024-05-06T12:25:54.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.546+0000 7fa5ea7fc640 1 -- 172.21.15.92:0/3139196499 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa5e400cd30 con 0x7fa5f4101a10 2024-05-06T12:25:54.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.546+0000 7fa5fc06e640 1 -- 172.21.15.92:0/3139196499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5f410d150 con 0x7fa5f4101a10 2024-05-06T12:25:54.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.546+0000 7fa5ea7fc640 1 -- 172.21.15.92:0/3139196499 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5e4013370 con 0x7fa5f4101a10 2024-05-06T12:25:54.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.546+0000 7fa5fc06e640 1 -- 172.21.15.92:0/3139196499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5f410d670 con 0x7fa5f4101a10 2024-05-06T12:25:54.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.547+0000 7fa5fc06e640 1 -- 172.21.15.92:0/3139196499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5f40fc7e0 con 0x7fa5f4101a10 2024-05-06T12:25:54.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.548+0000 7fa5ea7fc640 1 -- 172.21.15.92:0/3139196499 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa5e4012070 con 0x7fa5f4101a10 2024-05-06T12:25:54.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.548+0000 7fa5ea7fc640 1 --2- 172.21.15.92:0/3139196499 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fa5e006e0b0 0x7fa5e0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:54.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.548+0000 7fa5ea7fc640 1 -- 172.21.15.92:0/3139196499 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa5e408d7c0 con 0x7fa5f4101a10 2024-05-06T12:25:54.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.548+0000 7fa5f9e0a640 1 --2- 172.21.15.92:0/3139196499 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fa5e006e0b0 0x7fa5e0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:54.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.549+0000 7fa5f9e0a640 1 --2- 172.21.15.92:0/3139196499 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fa5e006e0b0 0x7fa5e0070560 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0004540 tx=0x7fa5f0009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:54.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.553+0000 7fa5ea7fc640 1 -- 172.21.15.92:0/3139196499 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa5e4056af0 con 0x7fa5f4101a10 2024-05-06T12:25:54.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.764+0000 7fa5fc06e640 1 -- 172.21.15.92:0/3139196499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 38, "format": "json"} v 0) v1 -- 0x7fa5f4108260 con 0x7fa5f4101a10 2024-05-06T12:25:54.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.765+0000 7fa5ea7fc640 1 -- 172.21.15.92:0/3139196499 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 38, "format": "json"}]=0 dumped fsmap epoch 38 v42) v1 ==== 107+0+5215 (secure 0 0 0) 0x7fa5e405a1b0 con 0x7fa5f4101a10 2024-05-06T12:25:54.767 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:25:54.768 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34466,"name":"cephfs.smithi092.rosbll","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6829/2398082173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":2398082173},{"type":"v1","addr":"172.21.15.92:6829","nonce":2398082173}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":44309,"name":"cephfs.smithi190.byjvgf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6825/1219274291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1219274291},{"type":"v1","addr":"172.21.15.190:6825","nonce":1219274291}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":44329,"name":"cephfs.smithi190.idzoff","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6827/2486629325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2486629325},{"type":"v1","addr":"172.21.15.190:6827","nonce":2486629325}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-06T12:11:44.952671+0000","modified":"2024-05-06T12:22:06.925528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34438},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34438":{"gid":34438,"name":"cephfs.smithi092.ijsqgp","rank":0,"incarnation":31,"state":"up:active","state_seq":8,"addr":"172.21.15.92:6827/4190573007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4190573007},{"type":"v1","addr":"172.21.15.92:6827","nonce":4190573007}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34438,"qdb_cluster":[34438]},"id":1}]} 2024-05-06T12:25:54.768 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 38 2024-05-06T12:25:54.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.767+0000 7fa5fc06e640 1 -- 172.21.15.92:0/3139196499 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fa5e006e0b0 msgr2=0x7fa5e0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:54.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.767+0000 7fa5fc06e640 1 --2- 172.21.15.92:0/3139196499 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fa5e006e0b0 0x7fa5e0070560 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0004540 tx=0x7fa5f0009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:54.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.767+0000 7fa5fc06e640 1 -- 172.21.15.92:0/3139196499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5f4101a10 msgr2=0x7fa5f4113220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:54.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.767+0000 7fa5fc06e640 1 --2- 172.21.15.92:0/3139196499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5f4101a10 0x7fa5f4113220 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400b790 tx=0x7fa5e40091d0 comp rx=0 tx=0).stop 2024-05-06T12:25:54.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.767+0000 7fa5fc06e640 1 -- 172.21.15.92:0/3139196499 shutdown_connections 2024-05-06T12:25:54.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.767+0000 7fa5fc06e640 1 --2- 172.21.15.92:0/3139196499 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fa5e006e0b0 0x7fa5e0070560 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:54.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.767+0000 7fa5fc06e640 1 --2- 172.21.15.92:0/3139196499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5f4101a10 0x7fa5f4113220 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:54.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.767+0000 7fa5fc06e640 1 --2- 172.21.15.92:0/3139196499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5f4101070 0x7fa5f4112ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:54.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.767+0000 7fa5fc06e640 1 -- 172.21.15.92:0/3139196499 >> 172.21.15.92:0/3139196499 conn(0x7fa5f40f6be0 msgr2=0x7fa5f40f7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:54.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.767+0000 7fa5fc06e640 1 -- 172.21.15.92:0/3139196499 shutdown_connections 2024-05-06T12:25:54.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.767+0000 7fa5fc06e640 1 -- 172.21.15.92:0/3139196499 wait complete. 2024-05-06T12:25:54.780 DEBUG:tasks.fs:max_mds reduced in epoch 38 2024-05-06T12:25:54.780 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 38 2024-05-06T12:25:54.780 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 39 2024-05-06T12:25:54.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.873+0000 7f754e59d640 1 -- 172.21.15.92:0/2176699586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7548101170 msgr2=0x7f7548101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:54.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.873+0000 7f754e59d640 1 --2- 172.21.15.92:0/2176699586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7548101170 0x7f7548101540 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f75300048d0 tx=0x7f7530009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:54.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.873+0000 7f754e59d640 1 -- 172.21.15.92:0/2176699586 shutdown_connections 2024-05-06T12:25:54.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.873+0000 7f754e59d640 1 --2- 172.21.15.92:0/2176699586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75480fb150 0x7f75480fb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:54.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.873+0000 7f754e59d640 1 --2- 172.21.15.92:0/2176699586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7548101170 0x7f7548101540 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:54.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.873+0000 7f754e59d640 1 -- 172.21.15.92:0/2176699586 >> 172.21.15.92:0/2176699586 conn(0x7f7548071fd0 msgr2=0x7f75480723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:54.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.873+0000 7f754e59d640 1 -- 172.21.15.92:0/2176699586 shutdown_connections 2024-05-06T12:25:54.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.874+0000 7f754e59d640 1 -- 172.21.15.92:0/2176699586 wait complete. 2024-05-06T12:25:54.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.874+0000 7f754e59d640 1 Processor -- start 2024-05-06T12:25:54.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.874+0000 7f754e59d640 1 -- start start 2024-05-06T12:25:54.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.874+0000 7f754e59d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75480fb150 0x7f7548102600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:54.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.874+0000 7f754e59d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7548101170 0x7f7548102b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:54.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.874+0000 7f754e59d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7548104570 con 0x7f75480fb150 2024-05-06T12:25:54.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.874+0000 7f754e59d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7548103080 con 0x7f7548101170 2024-05-06T12:25:54.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.874+0000 7f7547fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75480fb150 0x7f7548102600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:54.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.874+0000 7f7547fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75480fb150 0x7f7548102600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51994/0 (socket says 172.21.15.92:51994) 2024-05-06T12:25:54.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.874+0000 7f7547fff640 1 -- 172.21.15.92:0/2400723284 learned_addr learned my addr 172.21.15.92:0/2400723284 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:54.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.875+0000 7f7547fff640 1 -- 172.21.15.92:0/2400723284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7548101170 msgr2=0x7f7548102b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:54.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.875+0000 7f7547fff640 1 --2- 172.21.15.92:0/2400723284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7548101170 0x7f7548102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:54.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.875+0000 7f7547fff640 1 -- 172.21.15.92:0/2400723284 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7530005040 con 0x7f75480fb150 2024-05-06T12:25:54.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.875+0000 7f7547fff640 1 --2- 172.21.15.92:0/2400723284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75480fb150 0x7f7548102600 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7530004930 tx=0x7f7530009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:54.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.875+0000 7f7544ff9640 1 -- 172.21.15.92:0/2400723284 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75300094c0 con 0x7f75480fb150 2024-05-06T12:25:54.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.875+0000 7f7544ff9640 1 -- 172.21.15.92:0/2400723284 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7530032e50 con 0x7f75480fb150 2024-05-06T12:25:54.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.875+0000 7f7544ff9640 1 -- 172.21.15.92:0/2400723284 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75300433b0 con 0x7f75480fb150 2024-05-06T12:25:54.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.875+0000 7f754e59d640 1 -- 172.21.15.92:0/2400723284 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75481032a0 con 0x7f75480fb150 2024-05-06T12:25:54.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.875+0000 7f754e59d640 1 -- 172.21.15.92:0/2400723284 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75481193d0 con 0x7f75480fb150 2024-05-06T12:25:54.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.876+0000 7f754e59d640 1 -- 172.21.15.92:0/2400723284 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75480fc870 con 0x7f75480fb150 2024-05-06T12:25:54.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.876+0000 7f7544ff9640 1 -- 172.21.15.92:0/2400723284 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f753002f070 con 0x7f75480fb150 2024-05-06T12:25:54.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.877+0000 7f7544ff9640 1 --2- 172.21.15.92:0/2400723284 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f753406e0b0 0x7f7534070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:54.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.877+0000 7f75477fe640 1 --2- 172.21.15.92:0/2400723284 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f753406e0b0 0x7f7534070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:54.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.877+0000 7f7544ff9640 1 -- 172.21.15.92:0/2400723284 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f75300afb50 con 0x7f75480fb150 2024-05-06T12:25:54.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.877+0000 7f75477fe640 1 --2- 172.21.15.92:0/2400723284 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f753406e0b0 0x7f7534070560 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7548103d30 tx=0x7f7538006d20 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:54.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:54.882+0000 7f7544ff9640 1 -- 172.21.15.92:0/2400723284 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7530078e80 con 0x7f75480fb150 2024-05-06T12:25:55.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.096+0000 7f754e59d640 1 -- 172.21.15.92:0/2400723284 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 39, "format": "json"} v 0) v1 -- 0x7f7548103ae0 con 0x7f75480fb150 2024-05-06T12:25:55.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.096+0000 7f7544ff9640 1 -- 172.21.15.92:0/2400723284 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 39, "format": "json"}]=0 dumped fsmap epoch 39 v42) v1 ==== 107+0+5215 (secure 0 0 0) 0x7f753007c540 con 0x7f75480fb150 2024-05-06T12:25:55.099 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:25:55.099 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34466,"name":"cephfs.smithi092.rosbll","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6829/2398082173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":2398082173},{"type":"v1","addr":"172.21.15.92:6829","nonce":2398082173}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":44309,"name":"cephfs.smithi190.byjvgf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6825/1219274291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1219274291},{"type":"v1","addr":"172.21.15.190:6825","nonce":1219274291}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":44329,"name":"cephfs.smithi190.idzoff","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6827/2486629325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2486629325},{"type":"v1","addr":"172.21.15.190:6827","nonce":2486629325}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-06T12:11:44.952671+0000","modified":"2024-05-06T12:22:48.502326+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0],"up":{"mds_0":34438},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34438":{"gid":34438,"name":"cephfs.smithi092.ijsqgp","rank":0,"incarnation":31,"state":"up:active","state_seq":8,"addr":"172.21.15.92:6827/4190573007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4190573007},{"type":"v1","addr":"172.21.15.92:6827","nonce":4190573007}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34438,"qdb_cluster":[34438]},"id":1}]} 2024-05-06T12:25:55.099 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 39 2024-05-06T12:25:55.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.098+0000 7f754e59d640 1 -- 172.21.15.92:0/2400723284 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f753406e0b0 msgr2=0x7f7534070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:55.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.098+0000 7f754e59d640 1 --2- 172.21.15.92:0/2400723284 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f753406e0b0 0x7f7534070560 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7548103d30 tx=0x7f7538006d20 comp rx=0 tx=0).stop 2024-05-06T12:25:55.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.098+0000 7f754e59d640 1 -- 172.21.15.92:0/2400723284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75480fb150 msgr2=0x7f7548102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:55.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.098+0000 7f754e59d640 1 --2- 172.21.15.92:0/2400723284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75480fb150 0x7f7548102600 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7530004930 tx=0x7f7530009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:55.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.098+0000 7f754e59d640 1 -- 172.21.15.92:0/2400723284 shutdown_connections 2024-05-06T12:25:55.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.098+0000 7f754e59d640 1 --2- 172.21.15.92:0/2400723284 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f753406e0b0 0x7f7534070560 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:55.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.098+0000 7f754e59d640 1 --2- 172.21.15.92:0/2400723284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7548101170 0x7f7548102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:55.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.098+0000 7f754e59d640 1 --2- 172.21.15.92:0/2400723284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75480fb150 0x7f7548102600 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:55.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.098+0000 7f754e59d640 1 -- 172.21.15.92:0/2400723284 >> 172.21.15.92:0/2400723284 conn(0x7f7548071fd0 msgr2=0x7f7548104a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:55.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.098+0000 7f754e59d640 1 -- 172.21.15.92:0/2400723284 shutdown_connections 2024-05-06T12:25:55.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.098+0000 7f754e59d640 1 -- 172.21.15.92:0/2400723284 wait complete. 2024-05-06T12:25:55.111 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 39 2024-05-06T12:25:55.112 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 40 2024-05-06T12:25:55.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.203+0000 7fe8efd9d640 1 -- 172.21.15.92:0/2362573204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8e80fb150 msgr2=0x7fe8e80fb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:55.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.203+0000 7fe8efd9d640 1 --2- 172.21.15.92:0/2362573204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8e80fb150 0x7fe8e80fb5c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009c50 tx=0x7fe8dc004d10 comp rx=0 tx=0).stop 2024-05-06T12:25:55.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.203+0000 7fe8efd9d640 1 -- 172.21.15.92:0/2362573204 shutdown_connections 2024-05-06T12:25:55.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.203+0000 7fe8efd9d640 1 --2- 172.21.15.92:0/2362573204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8e80fb150 0x7fe8e80fb5c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:55.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.203+0000 7fe8efd9d640 1 --2- 172.21.15.92:0/2362573204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8e8101170 0x7fe8e8101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:55.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.203+0000 7fe8efd9d640 1 -- 172.21.15.92:0/2362573204 >> 172.21.15.92:0/2362573204 conn(0x7fe8e8071fd0 msgr2=0x7fe8e80723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:55.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.204+0000 7fe8efd9d640 1 -- 172.21.15.92:0/2362573204 shutdown_connections 2024-05-06T12:25:55.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.204+0000 7fe8efd9d640 1 -- 172.21.15.92:0/2362573204 wait complete. 2024-05-06T12:25:55.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.204+0000 7fe8efd9d640 1 Processor -- start 2024-05-06T12:25:55.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.204+0000 7fe8efd9d640 1 -- start start 2024-05-06T12:25:55.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.204+0000 7fe8efd9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8e80fb150 0x7fe8e8112c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:55.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.204+0000 7fe8efd9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8e8101170 0x7fe8e81131c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:55.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.204+0000 7fe8efd9d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e81137c0 con 0x7fe8e8101170 2024-05-06T12:25:55.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.204+0000 7fe8efd9d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e810ce10 con 0x7fe8e80fb150 2024-05-06T12:25:55.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.204+0000 7fe8ed338640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8e8101170 0x7fe8e81131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:55.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.204+0000 7fe8ed338640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8e8101170 0x7fe8e81131c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52010/0 (socket says 172.21.15.92:52010) 2024-05-06T12:25:55.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.204+0000 7fe8ed338640 1 -- 172.21.15.92:0/3090867747 learned_addr learned my addr 172.21.15.92:0/3090867747 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:55.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.204+0000 7fe8edb39640 1 --2- 172.21.15.92:0/3090867747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8e80fb150 0x7fe8e8112c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:55.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.204+0000 7fe8ed338640 1 -- 172.21.15.92:0/3090867747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8e80fb150 msgr2=0x7fe8e8112c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:55.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.204+0000 7fe8ed338640 1 --2- 172.21.15.92:0/3090867747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8e80fb150 0x7fe8e8112c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:55.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.204+0000 7fe8ed338640 1 -- 172.21.15.92:0/3090867747 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8dc0098b0 con 0x7fe8e8101170 2024-05-06T12:25:55.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.205+0000 7fe8ed338640 1 --2- 172.21.15.92:0/3090867747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8e8101170 0x7fe8e81131c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc004a30 tx=0x7fe8dc004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:55.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.205+0000 7fe8e67fc640 1 -- 172.21.15.92:0/3090867747 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8dc03a070 con 0x7fe8e8101170 2024-05-06T12:25:55.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.205+0000 7fe8e67fc640 1 -- 172.21.15.92:0/3090867747 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe8dc035b20 con 0x7fe8e8101170 2024-05-06T12:25:55.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.205+0000 7fe8e67fc640 1 -- 172.21.15.92:0/3090867747 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8dc03eaa0 con 0x7fe8e8101170 2024-05-06T12:25:55.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.205+0000 7fe8efd9d640 1 -- 172.21.15.92:0/3090867747 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e810d090 con 0x7fe8e8101170 2024-05-06T12:25:55.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.205+0000 7fe8efd9d640 1 -- 172.21.15.92:0/3090867747 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e810d630 con 0x7fe8e8101170 2024-05-06T12:25:55.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.206+0000 7fe8efd9d640 1 -- 172.21.15.92:0/3090867747 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8e80fc870 con 0x7fe8e8101170 2024-05-06T12:25:55.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.206+0000 7fe8e67fc640 1 -- 172.21.15.92:0/3090867747 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe8dc032070 con 0x7fe8e8101170 2024-05-06T12:25:55.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.207+0000 7fe8e67fc640 1 --2- 172.21.15.92:0/3090867747 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fe8d006e0b0 0x7fe8d0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:55.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.207+0000 7fe8e67fc640 1 -- 172.21.15.92:0/3090867747 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe8dc0b0320 con 0x7fe8e8101170 2024-05-06T12:25:55.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.207+0000 7fe8edb39640 1 --2- 172.21.15.92:0/3090867747 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fe8d006e0b0 0x7fe8d0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:55.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.207+0000 7fe8edb39640 1 --2- 172.21.15.92:0/3090867747 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fe8d006e0b0 0x7fe8d0070560 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80041d0 tx=0x7fe8d8009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:55.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.211+0000 7fe8e67fc640 1 -- 172.21.15.92:0/3090867747 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe8dc0796d0 con 0x7fe8e8101170 2024-05-06T12:25:55.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:55 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/766706485' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-05-06T12:25:55.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:55 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/3546812651' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-05-06T12:25:55.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:55 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/3139196499' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-05-06T12:25:55.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.423+0000 7fe8efd9d640 1 -- 172.21.15.92:0/3090867747 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 40, "format": "json"} v 0) v1 -- 0x7fe8e81082a0 con 0x7fe8e8101170 2024-05-06T12:25:55.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.424+0000 7fe8e67fc640 1 -- 172.21.15.92:0/3090867747 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 40, "format": "json"}]=0 dumped fsmap epoch 40 v42) v1 ==== 107+0+5232 (secure 0 0 0) 0x7fe8dc07cd90 con 0x7fe8e8101170 2024-05-06T12:25:55.426 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:25:55.427 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44309,"name":"cephfs.smithi190.byjvgf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6825/1219274291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1219274291},{"type":"v1","addr":"172.21.15.190:6825","nonce":1219274291}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":44329,"name":"cephfs.smithi190.idzoff","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6827/2486629325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2486629325},{"type":"v1","addr":"172.21.15.190:6827","nonce":2486629325}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-06T12:11:44.952671+0000","modified":"2024-05-06T12:22:49.510559+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34438,"mds_1":34466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34438":{"gid":34438,"name":"cephfs.smithi092.ijsqgp","rank":0,"incarnation":31,"state":"up:active","state_seq":8,"addr":"172.21.15.92:6827/4190573007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4190573007},{"type":"v1","addr":"172.21.15.92:6827","nonce":4190573007}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"cephfs.smithi092.rosbll","rank":1,"incarnation":40,"state":"up:starting","state_seq":1,"addr":"172.21.15.92:6829/2398082173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":2398082173},{"type":"v1","addr":"172.21.15.92:6829","nonce":2398082173}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34438,"qdb_cluster":[34438]},"id":1}]} 2024-05-06T12:25:55.427 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 40 2024-05-06T12:25:55.428 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.426+0000 7fe8efd9d640 1 -- 172.21.15.92:0/3090867747 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fe8d006e0b0 msgr2=0x7fe8d0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:55.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.426+0000 7fe8efd9d640 1 --2- 172.21.15.92:0/3090867747 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fe8d006e0b0 0x7fe8d0070560 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80041d0 tx=0x7fe8d8009040 comp rx=0 tx=0).stop 2024-05-06T12:25:55.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.426+0000 7fe8efd9d640 1 -- 172.21.15.92:0/3090867747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8e8101170 msgr2=0x7fe8e81131c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:55.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.426+0000 7fe8efd9d640 1 --2- 172.21.15.92:0/3090867747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8e8101170 0x7fe8e81131c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc004a30 tx=0x7fe8dc004d10 comp rx=0 tx=0).stop 2024-05-06T12:25:55.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.426+0000 7fe8efd9d640 1 -- 172.21.15.92:0/3090867747 shutdown_connections 2024-05-06T12:25:55.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.426+0000 7fe8efd9d640 1 --2- 172.21.15.92:0/3090867747 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fe8d006e0b0 0x7fe8d0070560 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:55.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.426+0000 7fe8efd9d640 1 --2- 172.21.15.92:0/3090867747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8e8101170 0x7fe8e81131c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:55.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.426+0000 7fe8efd9d640 1 --2- 172.21.15.92:0/3090867747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8e80fb150 0x7fe8e8112c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:55.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.426+0000 7fe8efd9d640 1 -- 172.21.15.92:0/3090867747 >> 172.21.15.92:0/3090867747 conn(0x7fe8e8071fd0 msgr2=0x7fe8e8104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:55.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.426+0000 7fe8efd9d640 1 -- 172.21.15.92:0/3090867747 shutdown_connections 2024-05-06T12:25:55.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.426+0000 7fe8efd9d640 1 -- 172.21.15.92:0/3090867747 wait complete. 2024-05-06T12:25:55.438 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:55 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/766706485' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-05-06T12:25:55.438 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:55 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/3546812651' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-05-06T12:25:55.438 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:55 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/3139196499' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-05-06T12:25:55.439 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 40 2024-05-06T12:25:55.439 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 41 2024-05-06T12:25:55.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.531+0000 7f56b7c2e640 1 -- 172.21.15.92:0/3175465255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56b0101070 msgr2=0x7f56b0101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:55.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.531+0000 7f56b7c2e640 1 --2- 172.21.15.92:0/3175465255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56b0101070 0x7f56b0101440 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f56ac0048d0 tx=0x7f56ac009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:55.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.531+0000 7f56b7c2e640 1 -- 172.21.15.92:0/3175465255 shutdown_connections 2024-05-06T12:25:55.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.531+0000 7f56b7c2e640 1 --2- 172.21.15.92:0/3175465255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56b0101a10 0x7f56b00fb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:55.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.531+0000 7f56b7c2e640 1 --2- 172.21.15.92:0/3175465255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56b0101070 0x7f56b0101440 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:55.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.531+0000 7f56b7c2e640 1 -- 172.21.15.92:0/3175465255 >> 172.21.15.92:0/3175465255 conn(0x7f56b00f6be0 msgr2=0x7f56b00f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:55.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.531+0000 7f56b7c2e640 1 -- 172.21.15.92:0/3175465255 shutdown_connections 2024-05-06T12:25:55.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.531+0000 7f56b7c2e640 1 -- 172.21.15.92:0/3175465255 wait complete. 2024-05-06T12:25:55.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.532+0000 7f56b7c2e640 1 Processor -- start 2024-05-06T12:25:55.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.532+0000 7f56b7c2e640 1 -- start start 2024-05-06T12:25:55.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.532+0000 7f56b7c2e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56b0101070 0x7f56b0111ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:55.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.532+0000 7f56b7c2e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56b0101a10 0x7f56b01120e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:55.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.532+0000 7f56b7c2e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b01126e0 con 0x7f56b0101a10 2024-05-06T12:25:55.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.532+0000 7f56b7c2e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b010bd30 con 0x7f56b0101070 2024-05-06T12:25:55.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.532+0000 7f56b51c9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56b0101a10 0x7f56b01120e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:55.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.532+0000 7f56b51c9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56b0101a10 0x7f56b01120e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52030/0 (socket says 172.21.15.92:52030) 2024-05-06T12:25:55.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.532+0000 7f56b51c9640 1 -- 172.21.15.92:0/1791368791 learned_addr learned my addr 172.21.15.92:0/1791368791 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:55.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.533+0000 7f56b59ca640 1 --2- 172.21.15.92:0/1791368791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56b0101070 0x7f56b0111ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:55.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.533+0000 7f56b51c9640 1 -- 172.21.15.92:0/1791368791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56b0101070 msgr2=0x7f56b0111ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:55.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.533+0000 7f56b51c9640 1 --2- 172.21.15.92:0/1791368791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56b0101070 0x7f56b0111ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:55.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.533+0000 7f56b51c9640 1 -- 172.21.15.92:0/1791368791 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ac005040 con 0x7f56b0101a10 2024-05-06T12:25:55.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.533+0000 7f56b59ca640 1 --2- 172.21.15.92:0/1791368791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56b0101070 0x7f56b0111ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-06T12:25:55.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.533+0000 7f56b51c9640 1 --2- 172.21.15.92:0/1791368791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56b0101a10 0x7f56b01120e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f56a000c790 tx=0x7f56a00043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:55.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.533+0000 7f56a67fc640 1 -- 172.21.15.92:0/1791368791 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56a000a820 con 0x7f56b0101a10 2024-05-06T12:25:55.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.533+0000 7f56b7c2e640 1 -- 172.21.15.92:0/1791368791 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56b010c010 con 0x7f56b0101a10 2024-05-06T12:25:55.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.533+0000 7f56a67fc640 1 -- 172.21.15.92:0/1791368791 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f56a000ae50 con 0x7f56b0101a10 2024-05-06T12:25:55.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.533+0000 7f56a67fc640 1 -- 172.21.15.92:0/1791368791 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56a00115e0 con 0x7f56b0101a10 2024-05-06T12:25:55.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.533+0000 7f56b7c2e640 1 -- 172.21.15.92:0/1791368791 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b010c530 con 0x7f56b0101a10 2024-05-06T12:25:55.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.534+0000 7f56b7c2e640 1 -- 172.21.15.92:0/1791368791 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56b00fc7e0 con 0x7f56b0101a10 2024-05-06T12:25:55.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.534+0000 7f56a67fc640 1 -- 172.21.15.92:0/1791368791 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f56a0012400 con 0x7f56b0101a10 2024-05-06T12:25:55.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.535+0000 7f56a67fc640 1 --2- 172.21.15.92:0/1791368791 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f569c06e0b0 0x7f569c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:55.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.535+0000 7f56a67fc640 1 -- 172.21.15.92:0/1791368791 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f56a008dd30 con 0x7f56b0101a10 2024-05-06T12:25:55.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.535+0000 7f56b59ca640 1 --2- 172.21.15.92:0/1791368791 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f569c06e0b0 0x7f569c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:55.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.535+0000 7f56b59ca640 1 --2- 172.21.15.92:0/1791368791 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f569c06e0b0 0x7f569c070560 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f56ac000c00 tx=0x7f56ac009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:55.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.539+0000 7f56a67fc640 1 -- 172.21.15.92:0/1791368791 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f56a00570e0 con 0x7f56b0101a10 2024-05-06T12:25:55.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.752+0000 7f56b7c2e640 1 -- 172.21.15.92:0/1791368791 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 41, "format": "json"} v 0) v1 -- 0x7f56b0108200 con 0x7f56b0101a10 2024-05-06T12:25:55.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.753+0000 7f56a67fc640 1 -- 172.21.15.92:0/1791368791 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 41, "format": "json"}]=0 dumped fsmap epoch 41 v42) v1 ==== 107+0+5236 (secure 0 0 0) 0x7f56a005a7a0 con 0x7f56b0101a10 2024-05-06T12:25:55.756 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:25:55.756 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44309,"name":"cephfs.smithi190.byjvgf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6825/1219274291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1219274291},{"type":"v1","addr":"172.21.15.190:6825","nonce":1219274291}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":44329,"name":"cephfs.smithi190.idzoff","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6827/2486629325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2486629325},{"type":"v1","addr":"172.21.15.190:6827","nonce":2486629325}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-06T12:11:44.952671+0000","modified":"2024-05-06T12:22:50.519706+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34438,"mds_1":34466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34438":{"gid":34438,"name":"cephfs.smithi092.ijsqgp","rank":0,"incarnation":31,"state":"up:active","state_seq":8,"addr":"172.21.15.92:6827/4190573007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4190573007},{"type":"v1","addr":"172.21.15.92:6827","nonce":4190573007}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34466":{"gid":34466,"name":"cephfs.smithi092.rosbll","rank":1,"incarnation":40,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6829/2398082173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":2398082173},{"type":"v1","addr":"172.21.15.92:6829","nonce":2398082173}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34438,"qdb_cluster":[34438,34466]},"id":1}]} 2024-05-06T12:25:55.756 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 41 2024-05-06T12:25:55.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.755+0000 7f56b7c2e640 1 -- 172.21.15.92:0/1791368791 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f569c06e0b0 msgr2=0x7f569c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:55.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.755+0000 7f56b7c2e640 1 --2- 172.21.15.92:0/1791368791 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f569c06e0b0 0x7f569c070560 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f56ac000c00 tx=0x7f56ac009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:55.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.755+0000 7f56b7c2e640 1 -- 172.21.15.92:0/1791368791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56b0101a10 msgr2=0x7f56b01120e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:55.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.755+0000 7f56b7c2e640 1 --2- 172.21.15.92:0/1791368791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56b0101a10 0x7f56b01120e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f56a000c790 tx=0x7f56a00043c0 comp rx=0 tx=0).stop 2024-05-06T12:25:55.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.755+0000 7f56b7c2e640 1 -- 172.21.15.92:0/1791368791 shutdown_connections 2024-05-06T12:25:55.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.755+0000 7f56b7c2e640 1 --2- 172.21.15.92:0/1791368791 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f569c06e0b0 0x7f569c070560 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:55.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.755+0000 7f56b7c2e640 1 --2- 172.21.15.92:0/1791368791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56b0101a10 0x7f56b01120e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:55.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.755+0000 7f56b7c2e640 1 --2- 172.21.15.92:0/1791368791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56b0101070 0x7f56b0111ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:55.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.755+0000 7f56b7c2e640 1 -- 172.21.15.92:0/1791368791 >> 172.21.15.92:0/1791368791 conn(0x7f56b00f6be0 msgr2=0x7f56b00f7aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:55.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.755+0000 7f56b7c2e640 1 -- 172.21.15.92:0/1791368791 shutdown_connections 2024-05-06T12:25:55.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.755+0000 7f56b7c2e640 1 -- 172.21.15.92:0/1791368791 wait complete. 2024-05-06T12:25:55.768 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-06T12:25:55.779 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-06T12:25:55.780 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-06T12:25:55.780 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-06T12:25:55.796 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-06T12:25:55.796 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-06T12:25:55.852 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-06T12:25:55.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.983+0000 7fd90519d640 1 -- 172.21.15.92:0/873152108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd900101090 msgr2=0x7fd900101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:55.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.983+0000 7fd90519d640 1 --2- 172.21.15.92:0/873152108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd900101090 0x7fd900101460 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd8f00048d0 tx=0x7fd8f0009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:55.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.983+0000 7fd90519d640 1 -- 172.21.15.92:0/873152108 shutdown_connections 2024-05-06T12:25:55.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.983+0000 7fd90519d640 1 --2- 172.21.15.92:0/873152108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd900101a30 0x7fd9000fb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:55.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.983+0000 7fd90519d640 1 --2- 172.21.15.92:0/873152108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd900101090 0x7fd900101460 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:55.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.983+0000 7fd90519d640 1 -- 172.21.15.92:0/873152108 >> 172.21.15.92:0/873152108 conn(0x7fd9000f6c20 msgr2=0x7fd9000f9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:55.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.983+0000 7fd90519d640 1 -- 172.21.15.92:0/873152108 shutdown_connections 2024-05-06T12:25:55.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.983+0000 7fd90519d640 1 -- 172.21.15.92:0/873152108 wait complete. 2024-05-06T12:25:55.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.984+0000 7fd90519d640 1 Processor -- start 2024-05-06T12:25:55.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.984+0000 7fd90519d640 1 -- start start 2024-05-06T12:25:55.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.984+0000 7fd90519d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd900101090 0x7fd90010de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:55.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.984+0000 7fd90519d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd900101a30 0x7fd90010e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:55.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.984+0000 7fd90519d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90010e900 con 0x7fd900101090 2024-05-06T12:25:55.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.984+0000 7fd90519d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90010ea70 con 0x7fd900101a30 2024-05-06T12:25:55.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.984+0000 7fd8fed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd900101090 0x7fd90010de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:55.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.984+0000 7fd8fed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd900101090 0x7fd90010de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52056/0 (socket says 172.21.15.92:52056) 2024-05-06T12:25:55.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.984+0000 7fd8fed9d640 1 -- 172.21.15.92:0/766609827 learned_addr learned my addr 172.21.15.92:0/766609827 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:55.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.984+0000 7fd8fe59c640 1 --2- 172.21.15.92:0/766609827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd900101a30 0x7fd90010e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:55.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.984+0000 7fd8fed9d640 1 -- 172.21.15.92:0/766609827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd900101a30 msgr2=0x7fd90010e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:55.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.984+0000 7fd8fed9d640 1 --2- 172.21.15.92:0/766609827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd900101a30 0x7fd90010e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:55.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.984+0000 7fd8fed9d640 1 -- 172.21.15.92:0/766609827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f0005040 con 0x7fd900101090 2024-05-06T12:25:55.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.984+0000 7fd8fe59c640 1 --2- 172.21.15.92:0/766609827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd900101a30 0x7fd90010e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-06T12:25:55.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.985+0000 7fd8fed9d640 1 --2- 172.21.15.92:0/766609827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd900101090 0x7fd90010de80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0004930 tx=0x7fd8f0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:55.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.985+0000 7fd8e77fe640 1 -- 172.21.15.92:0/766609827 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8f00094c0 con 0x7fd900101090 2024-05-06T12:25:55.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.985+0000 7fd90519d640 1 -- 172.21.15.92:0/766609827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd90010ecf0 con 0x7fd900101090 2024-05-06T12:25:55.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.985+0000 7fd8e77fe640 1 -- 172.21.15.92:0/766609827 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd8f0032e50 con 0x7fd900101090 2024-05-06T12:25:55.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.985+0000 7fd8e77fe640 1 -- 172.21.15.92:0/766609827 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8f00433b0 con 0x7fd900101090 2024-05-06T12:25:55.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.985+0000 7fd90519d640 1 -- 172.21.15.92:0/766609827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd900117140 con 0x7fd900101090 2024-05-06T12:25:55.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.986+0000 7fd90519d640 1 -- 172.21.15.92:0/766609827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9000fc800 con 0x7fd900101090 2024-05-06T12:25:55.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.986+0000 7fd8e77fe640 1 -- 172.21.15.92:0/766609827 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd8f002f070 con 0x7fd900101090 2024-05-06T12:25:55.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.986+0000 7fd8e77fe640 1 --2- 172.21.15.92:0/766609827 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd8e806e0b0 0x7fd8e8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:55.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.987+0000 7fd8e77fe640 1 -- 172.21.15.92:0/766609827 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd8f00af7f0 con 0x7fd900101090 2024-05-06T12:25:55.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.987+0000 7fd8fe59c640 1 --2- 172.21.15.92:0/766609827 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd8e806e0b0 0x7fd8e8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:55.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.987+0000 7fd8fe59c640 1 --2- 172.21.15.92:0/766609827 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd8e806e0b0 0x7fd8e8070560 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd900065ce0 tx=0x7fd8f400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:55.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:55.991+0000 7fd8e77fe640 1 -- 172.21.15.92:0/766609827 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd8f0078b20 con 0x7fd900101090 2024-05-06T12:25:56.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.168+0000 7fd90519d640 1 -- 172.21.15.92:0/766609827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd90004bcc0 con 0x7fd900101090 2024-05-06T12:25:56.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.170+0000 7fd8e77fe640 1 -- 172.21.15.92:0/766609827 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16615 (secure 0 0 0) 0x7fd8f007c1e0 con 0x7fd900101090 2024-05-06T12:25:56.173 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:25:56.173 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":86,"fsid":"aee1d912-0ba0-11ef-bc97-c7b262605968","created":"2024-05-06T12:04:51.220909+0000","modified":"2024-05-06T12:22:36.092329+0000","last_up_change":"2024-05-06T12:20:59.798275+0000","last_in_change":"2024-05-06T12:10:07.815689+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-06T12:09:12.129668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-05-06T12:11:43.666342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-05-06T12:11:44.183559+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b276435-4698-4017-9535-4be41a4e1379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3125171617},{"type":"v1","addr":"172.21.15.92:6803","nonce":3125171617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3125171617},{"type":"v1","addr":"172.21.15.92:6805","nonce":3125171617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3125171617},{"type":"v1","addr":"172.21.15.92:6809","nonce":3125171617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3125171617},{"type":"v1","addr":"172.21.15.92:6807","nonce":3125171617}]},"public_addr":"172.21.15.92:6803/3125171617","cluster_addr":"172.21.15.92:6805/3125171617","heartbeat_back_addr":"172.21.15.92:6809/3125171617","heartbeat_front_addr":"172.21.15.92:6807/3125171617","state":["exists","up"]},{"osd":1,"uuid":"21688119-d15f-4b31-8842-090cc539c76d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1875265240},{"type":"v1","addr":"172.21.15.92:6811","nonce":1875265240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1875265240},{"type":"v1","addr":"172.21.15.92:6813","nonce":1875265240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1875265240},{"type":"v1","addr":"172.21.15.92:6817","nonce":1875265240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1875265240},{"type":"v1","addr":"172.21.15.92:6815","nonce":1875265240}]},"public_addr":"172.21.15.92:6811/1875265240","cluster_addr":"172.21.15.92:6813/1875265240","heartbeat_back_addr":"172.21.15.92:6817/1875265240","heartbeat_front_addr":"172.21.15.92:6815/1875265240","state":["exists","up"]},{"osd":2,"uuid":"74d16dfb-b118-432d-827a-59706d69c393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":629593219},{"type":"v1","addr":"172.21.15.92:6819","nonce":629593219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":629593219},{"type":"v1","addr":"172.21.15.92:6821","nonce":629593219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":629593219},{"type":"v1","addr":"172.21.15.92:6825","nonce":629593219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":629593219},{"type":"v1","addr":"172.21.15.92:6823","nonce":629593219}]},"public_addr":"172.21.15.92:6819/629593219","cluster_addr":"172.21.15.92:6821/629593219","heartbeat_back_addr":"172.21.15.92:6825/629593219","heartbeat_front_addr":"172.21.15.92:6823/629593219","state":["exists","up"]},{"osd":3,"uuid":"5a9aa5ed-2502-4c73-a8ac-cb7d0259500b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":3441479977},{"type":"v1","addr":"172.21.15.190:6801","nonce":3441479977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":3441479977},{"type":"v1","addr":"172.21.15.190:6803","nonce":3441479977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":3441479977},{"type":"v1","addr":"172.21.15.190:6807","nonce":3441479977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":3441479977},{"type":"v1","addr":"172.21.15.190:6805","nonce":3441479977}]},"public_addr":"172.21.15.190:6801/3441479977","cluster_addr":"172.21.15.190:6803/3441479977","heartbeat_back_addr":"172.21.15.190:6807/3441479977","heartbeat_front_addr":"172.21.15.190:6805/3441479977","state":["exists","up"]},{"osd":4,"uuid":"d4039e4a-c758-4632-85d6-1b7e725c0293","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":839214066},{"type":"v1","addr":"172.21.15.190:6809","nonce":839214066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":839214066},{"type":"v1","addr":"172.21.15.190:6811","nonce":839214066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":839214066},{"type":"v1","addr":"172.21.15.190:6815","nonce":839214066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":839214066},{"type":"v1","addr":"172.21.15.190:6813","nonce":839214066}]},"public_addr":"172.21.15.190:6809/839214066","cluster_addr":"172.21.15.190:6811/839214066","heartbeat_back_addr":"172.21.15.190:6815/839214066","heartbeat_front_addr":"172.21.15.190:6813/839214066","state":["exists","up"]},{"osd":5,"uuid":"ec707eee-5f24-48bb-818a-547f804fc41b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":10700099},{"type":"v1","addr":"172.21.15.190:6817","nonce":10700099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":10700099},{"type":"v1","addr":"172.21.15.190:6819","nonce":10700099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":10700099},{"type":"v1","addr":"172.21.15.190:6823","nonce":10700099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":10700099},{"type":"v1","addr":"172.21.15.190:6821","nonce":10700099}]},"public_addr":"172.21.15.190:6817/10700099","cluster_addr":"172.21.15.190:6819/10700099","heartbeat_back_addr":"172.21.15.190:6823/10700099","heartbeat_front_addr":"172.21.15.190:6821/10700099","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:08:15.348582+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:08:42.571577+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:09:10.093700+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:09:34.257829+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:09:59.795522+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:10:25.131646+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:6828/1663253598":"2024-05-07T12:21:58.517697+0000","172.21.15.92:6801/648200129":"2024-05-07T12:16:38.777479+0000","172.21.15.92:6801/543287293":"2024-05-07T12:07:03.802107+0000","172.21.15.92:0/3693315250":"2024-05-07T12:05:56.546947+0000","172.21.15.92:0/3493498366":"2024-05-07T12:14:01.169339+0000","172.21.15.190:0/3060689070":"2024-05-07T12:15:47.314494+0000","172.21.15.92:0/2210153116":"2024-05-07T12:05:56.546947+0000","172.21.15.92:0/3417170577":"2024-05-07T12:14:01.169339+0000","172.21.15.92:0/1054764607":"2024-05-07T12:16:38.777479+0000","172.21.15.92:0/856987777":"2024-05-07T12:14:01.169339+0000","172.21.15.92:0/895013058":"2024-05-07T12:05:25.954232+0000","172.21.15.92:0/2440233745":"2024-05-07T12:07:03.802107+0000","172.21.15.92:6800/543287293":"2024-05-07T12:07:03.802107+0000","172.21.15.190:6829/3484477933":"2024-05-07T12:15:47.314494+0000","172.21.15.92:0/758782094":"2024-05-07T12:05:25.954232+0000","172.21.15.92:0/1552275072":"2024-05-07T12:07:03.802107+0000","172.21.15.92:6800/3731057893":"2024-05-07T12:05:56.546947+0000","172.21.15.92:0/1407601857":"2024-05-07T12:05:56.546947+0000","172.21.15.92:0/1345929606":"2024-05-07T12:16:38.777479+0000","172.21.15.92:6801/3045905943":"2024-05-07T12:14:01.169339+0000","172.21.15.92:6829/1663253598":"2024-05-07T12:21:58.517697+0000","172.21.15.190:0/2858276177":"2024-05-07T12:15:47.314494+0000","172.21.15.92:6827/4255634245":"2024-05-07T12:12:00.896508+0000","172.21.15.92:0/4121052357":"2024-05-07T12:14:01.169339+0000","172.21.15.92:0/997820112":"2024-05-07T12:05:25.954232+0000","172.21.15.92:6800/3656822751":"2024-05-07T12:05:25.954232+0000","172.21.15.92:0/1532977673":"2024-05-07T12:07:03.802107+0000","172.21.15.92:6801/3656822751":"2024-05-07T12:05:25.954232+0000","172.21.15.92:6801/3731057893":"2024-05-07T12:05:56.546947+0000","172.21.15.92:6826/4255634245":"2024-05-07T12:12:00.896508+0000","172.21.15.190:6828/3484477933":"2024-05-07T12:15:47.314494+0000","172.21.15.190:0/2419054582":"2024-05-07T12:15:47.314494+0000","172.21.15.92:0/418878214":"2024-05-07T12:16:38.777479+0000","172.21.15.92:0/2098378816":"2024-05-07T12:16:38.777479+0000","172.21.15.92:6800/3045905943":"2024-05-07T12:14:01.169339+0000","172.21.15.92:0/2150816131":"2024-05-07T12:16:38.777479+0000","172.21.15.190:0/2974973416":"2024-05-07T12:15:47.314494+0000","172.21.15.92:0/1862221479":"2024-05-07T12:16:38.777479+0000","172.21.15.92:6800/648200129":"2024-05-07T12:16:38.777479+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-06T12:25:56.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.173+0000 7fd90519d640 1 -- 172.21.15.92:0/766609827 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd8e806e0b0 msgr2=0x7fd8e8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:56.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.173+0000 7fd90519d640 1 --2- 172.21.15.92:0/766609827 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd8e806e0b0 0x7fd8e8070560 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd900065ce0 tx=0x7fd8f400b040 comp rx=0 tx=0).stop 2024-05-06T12:25:56.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.173+0000 7fd90519d640 1 -- 172.21.15.92:0/766609827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd900101090 msgr2=0x7fd90010de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:56.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.173+0000 7fd90519d640 1 --2- 172.21.15.92:0/766609827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd900101090 0x7fd90010de80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0004930 tx=0x7fd8f0009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:56.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.173+0000 7fd90519d640 1 -- 172.21.15.92:0/766609827 shutdown_connections 2024-05-06T12:25:56.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.173+0000 7fd90519d640 1 --2- 172.21.15.92:0/766609827 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd8e806e0b0 0x7fd8e8070560 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:56.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.173+0000 7fd90519d640 1 --2- 172.21.15.92:0/766609827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd900101a30 0x7fd90010e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:56.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.173+0000 7fd90519d640 1 --2- 172.21.15.92:0/766609827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd900101090 0x7fd90010de80 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:56.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.173+0000 7fd90519d640 1 -- 172.21.15.92:0/766609827 >> 172.21.15.92:0/766609827 conn(0x7fd9000f6c20 msgr2=0x7fd9000f7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:56.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.173+0000 7fd90519d640 1 -- 172.21.15.92:0/766609827 shutdown_connections 2024-05-06T12:25:56.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.173+0000 7fd90519d640 1 -- 172.21.15.92:0/766609827 wait complete. 2024-05-06T12:25:56.186 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-06T12:09:12.129668+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-05-06T12:11:43.666342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-05-06T12:11:44.183559+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-06T12:25:56.186 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-06T12:25:56.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.280+0000 7f330d75d640 1 -- 172.21.15.92:0/1524800214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3308101a40 msgr2=0x7f33080fb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:56.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.280+0000 7f330d75d640 1 --2- 172.21.15.92:0/1524800214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3308101a40 0x7f33080fb0d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f32fc0048d0 tx=0x7f32fc009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:56.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.280+0000 7f330d75d640 1 -- 172.21.15.92:0/1524800214 shutdown_connections 2024-05-06T12:25:56.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.280+0000 7f330d75d640 1 --2- 172.21.15.92:0/1524800214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3308101a40 0x7f33080fb0d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:56.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.281+0000 7f330d75d640 1 --2- 172.21.15.92:0/1524800214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33081010a0 0x7f3308101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:56.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.281+0000 7f330d75d640 1 -- 172.21.15.92:0/1524800214 >> 172.21.15.92:0/1524800214 conn(0x7f33080f6c90 msgr2=0x7f33080f90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:56.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.281+0000 7f330d75d640 1 -- 172.21.15.92:0/1524800214 shutdown_connections 2024-05-06T12:25:56.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.281+0000 7f330d75d640 1 -- 172.21.15.92:0/1524800214 wait complete. 2024-05-06T12:25:56.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.281+0000 7f330d75d640 1 Processor -- start 2024-05-06T12:25:56.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.281+0000 7f330d75d640 1 -- start start 2024-05-06T12:25:56.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.281+0000 7f330d75d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33081010a0 0x7f3308114f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:56.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.281+0000 7f330d75d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3308101a40 0x7f33081154a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:56.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.281+0000 7f330d75d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3308115aa0 con 0x7f33081010a0 2024-05-06T12:25:56.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.281+0000 7f330d75d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330810f0f0 con 0x7f3308101a40 2024-05-06T12:25:56.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.281+0000 7f3306ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33081010a0 0x7f3308114f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:56.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.281+0000 7f3306ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33081010a0 0x7f3308114f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52062/0 (socket says 172.21.15.92:52062) 2024-05-06T12:25:56.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.282+0000 7f3306ffd640 1 -- 172.21.15.92:0/1012141198 learned_addr learned my addr 172.21.15.92:0/1012141198 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:56.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.282+0000 7f33067fc640 1 --2- 172.21.15.92:0/1012141198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3308101a40 0x7f33081154a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:56.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.282+0000 7f3306ffd640 1 -- 172.21.15.92:0/1012141198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3308101a40 msgr2=0x7f33081154a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:56.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.282+0000 7f3306ffd640 1 --2- 172.21.15.92:0/1012141198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3308101a40 0x7f33081154a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:56.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.282+0000 7f3306ffd640 1 -- 172.21.15.92:0/1012141198 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32fc005040 con 0x7f33081010a0 2024-05-06T12:25:56.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.282+0000 7f3306ffd640 1 --2- 172.21.15.92:0/1012141198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33081010a0 0x7f3308114f60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f32f800cb10 tx=0x7f32f8007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:56.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.282+0000 7f32ef7fe640 1 -- 172.21.15.92:0/1012141198 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32f8004200 con 0x7f33081010a0 2024-05-06T12:25:56.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.282+0000 7f330d75d640 1 -- 172.21.15.92:0/1012141198 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f330810f3d0 con 0x7f33081010a0 2024-05-06T12:25:56.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.282+0000 7f32ef7fe640 1 -- 172.21.15.92:0/1012141198 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f32f800acf0 con 0x7f33081010a0 2024-05-06T12:25:56.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.282+0000 7f32ef7fe640 1 -- 172.21.15.92:0/1012141198 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32f8003e40 con 0x7f33081010a0 2024-05-06T12:25:56.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.282+0000 7f330d75d640 1 -- 172.21.15.92:0/1012141198 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f330810f8c0 con 0x7f33081010a0 2024-05-06T12:25:56.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.283+0000 7f330d75d640 1 -- 172.21.15.92:0/1012141198 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33080fc810 con 0x7f33081010a0 2024-05-06T12:25:56.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.283+0000 7f32ef7fe640 1 -- 172.21.15.92:0/1012141198 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f32f800a820 con 0x7f33081010a0 2024-05-06T12:25:56.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.284+0000 7f32ef7fe640 1 --2- 172.21.15.92:0/1012141198 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f32f006e050 0x7f32f0070500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:56.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.284+0000 7f32ef7fe640 1 -- 172.21.15.92:0/1012141198 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f32f808e2c0 con 0x7f33081010a0 2024-05-06T12:25:56.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.284+0000 7f33067fc640 1 --2- 172.21.15.92:0/1012141198 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f32f006e050 0x7f32f0070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:56.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.284+0000 7f33067fc640 1 --2- 172.21.15.92:0/1012141198 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f32f006e050 0x7f32f0070500 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f33080f96e0 tx=0x7f32fc009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:56.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.289+0000 7f32ef7fe640 1 -- 172.21.15.92:0/1012141198 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f32f80575f0 con 0x7f33081010a0 2024-05-06T12:25:56.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:56 smithi190 ceph-mon[79744]: pgmap v317: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:25:56.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:56 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/2400723284' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-05-06T12:25:56.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:56 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/3090867747' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 40, "format": "json"}]: dispatch 2024-05-06T12:25:56.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:56 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/1791368791' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 41, "format": "json"}]: dispatch 2024-05-06T12:25:56.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.467+0000 7f330d75d640 1 -- 172.21.15.92:0/1012141198 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f33081105d0 con 0x7f33081010a0 2024-05-06T12:25:56.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.468+0000 7f32ef7fe640 1 -- 172.21.15.92:0/1012141198 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f32f805acb0 con 0x7f33081010a0 2024-05-06T12:25:56.470 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2024-05-06T12:25:56.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:56 smithi092 ceph-mon[91933]: pgmap v317: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:25:56.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:56 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/2400723284' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-05-06T12:25:56.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:56 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/3090867747' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 40, "format": "json"}]: dispatch 2024-05-06T12:25:56.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:56 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/1791368791' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 41, "format": "json"}]: dispatch 2024-05-06T12:25:56.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.469+0000 7f330d75d640 1 -- 172.21.15.92:0/1012141198 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f32f006e050 msgr2=0x7f32f0070500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:56.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.469+0000 7f330d75d640 1 --2- 172.21.15.92:0/1012141198 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f32f006e050 0x7f32f0070500 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f33080f96e0 tx=0x7f32fc009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:56.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.470+0000 7f330d75d640 1 -- 172.21.15.92:0/1012141198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33081010a0 msgr2=0x7f3308114f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:56.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.470+0000 7f330d75d640 1 --2- 172.21.15.92:0/1012141198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33081010a0 0x7f3308114f60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f32f800cb10 tx=0x7f32f8007ee0 comp rx=0 tx=0).stop 2024-05-06T12:25:56.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.470+0000 7f330d75d640 1 -- 172.21.15.92:0/1012141198 shutdown_connections 2024-05-06T12:25:56.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.470+0000 7f330d75d640 1 --2- 172.21.15.92:0/1012141198 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f32f006e050 0x7f32f0070500 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:56.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.470+0000 7f330d75d640 1 --2- 172.21.15.92:0/1012141198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3308101a40 0x7f33081154a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:56.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.470+0000 7f330d75d640 1 --2- 172.21.15.92:0/1012141198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33081010a0 0x7f3308114f60 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:56.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.470+0000 7f330d75d640 1 -- 172.21.15.92:0/1012141198 >> 172.21.15.92:0/1012141198 conn(0x7f33080f6c90 msgr2=0x7f3308065540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:56.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.470+0000 7f330d75d640 1 -- 172.21.15.92:0/1012141198 shutdown_connections 2024-05-06T12:25:56.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.470+0000 7f330d75d640 1 -- 172.21.15.92:0/1012141198 wait complete. 2024-05-06T12:25:56.483 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-05-06T12:25:56.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.577+0000 7ff94535d640 1 -- 172.21.15.92:0/2261053085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff940101090 msgr2=0x7ff940101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:56.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.577+0000 7ff94535d640 1 --2- 172.21.15.92:0/2261053085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff940101090 0x7ff940101460 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff9300048d0 tx=0x7ff930009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:56.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.577+0000 7ff94535d640 1 -- 172.21.15.92:0/2261053085 shutdown_connections 2024-05-06T12:25:56.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.577+0000 7ff94535d640 1 --2- 172.21.15.92:0/2261053085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff940101a30 0x7ff9400fb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:56.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.577+0000 7ff94535d640 1 --2- 172.21.15.92:0/2261053085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff940101090 0x7ff940101460 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:56.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.577+0000 7ff94535d640 1 -- 172.21.15.92:0/2261053085 >> 172.21.15.92:0/2261053085 conn(0x7ff9400f6c20 msgr2=0x7ff9400f9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:56.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.577+0000 7ff94535d640 1 -- 172.21.15.92:0/2261053085 shutdown_connections 2024-05-06T12:25:56.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.577+0000 7ff94535d640 1 -- 172.21.15.92:0/2261053085 wait complete. 2024-05-06T12:25:56.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.578+0000 7ff94535d640 1 Processor -- start 2024-05-06T12:25:56.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.578+0000 7ff94535d640 1 -- start start 2024-05-06T12:25:56.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.578+0000 7ff94535d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff940101090 0x7ff94010de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:56.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.578+0000 7ff94535d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff940101a30 0x7ff94010e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:56.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.578+0000 7ff94535d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9400664b0 con 0x7ff940101a30 2024-05-06T12:25:56.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.578+0000 7ff94535d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94010e940 con 0x7ff940101090 2024-05-06T12:25:56.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.578+0000 7ff93e7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff940101a30 0x7ff94010e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:56.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.578+0000 7ff93e7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff940101a30 0x7ff94010e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52072/0 (socket says 172.21.15.92:52072) 2024-05-06T12:25:56.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.578+0000 7ff93e7fc640 1 -- 172.21.15.92:0/3177932868 learned_addr learned my addr 172.21.15.92:0/3177932868 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:56.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.578+0000 7ff93effd640 1 --2- 172.21.15.92:0/3177932868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff940101090 0x7ff94010de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:56.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.578+0000 7ff93e7fc640 1 -- 172.21.15.92:0/3177932868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff940101090 msgr2=0x7ff94010de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:56.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.578+0000 7ff93e7fc640 1 --2- 172.21.15.92:0/3177932868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff940101090 0x7ff94010de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:56.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.579+0000 7ff93e7fc640 1 -- 172.21.15.92:0/3177932868 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff930005040 con 0x7ff940101a30 2024-05-06T12:25:56.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.579+0000 7ff93e7fc640 1 --2- 172.21.15.92:0/3177932868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff940101a30 0x7ff94010e3d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff93400c830 tx=0x7ff934004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:56.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.579+0000 7ff9277fe640 1 -- 172.21.15.92:0/3177932868 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff93400a850 con 0x7ff940101a30 2024-05-06T12:25:56.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.579+0000 7ff9277fe640 1 -- 172.21.15.92:0/3177932868 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff93400ae80 con 0x7ff940101a30 2024-05-06T12:25:56.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.579+0000 7ff94535d640 1 -- 172.21.15.92:0/3177932868 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff94010ec20 con 0x7ff940101a30 2024-05-06T12:25:56.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.579+0000 7ff9277fe640 1 -- 172.21.15.92:0/3177932868 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff934012ea0 con 0x7ff940101a30 2024-05-06T12:25:56.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.579+0000 7ff94535d640 1 -- 172.21.15.92:0/3177932868 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9401171b0 con 0x7ff940101a30 2024-05-06T12:25:56.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.580+0000 7ff94535d640 1 -- 172.21.15.92:0/3177932868 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9400fc800 con 0x7ff940101a30 2024-05-06T12:25:56.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.580+0000 7ff9277fe640 1 -- 172.21.15.92:0/3177932868 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff934012400 con 0x7ff940101a30 2024-05-06T12:25:56.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.581+0000 7ff9277fe640 1 --2- 172.21.15.92:0/3177932868 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff92806e0b0 0x7ff928070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:56.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.581+0000 7ff9277fe640 1 -- 172.21.15.92:0/3177932868 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff93408ddb0 con 0x7ff940101a30 2024-05-06T12:25:56.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.581+0000 7ff93effd640 1 --2- 172.21.15.92:0/3177932868 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff92806e0b0 0x7ff928070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:56.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.581+0000 7ff93effd640 1 --2- 172.21.15.92:0/3177932868 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff92806e0b0 0x7ff928070560 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff930000c00 tx=0x7ff930009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:56.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.586+0000 7ff9277fe640 1 -- 172.21.15.92:0/3177932868 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff934057160 con 0x7ff940101a30 2024-05-06T12:25:56.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.761+0000 7ff94535d640 1 -- 172.21.15.92:0/3177932868 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7ff94004bcc0 con 0x7ff940101a30 2024-05-06T12:25:56.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.761+0000 7ff9277fe640 1 -- 172.21.15.92:0/3177932868 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7ff9340126b0 con 0x7ff940101a30 2024-05-06T12:25:56.764 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2024-05-06T12:25:56.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.763+0000 7ff94535d640 1 -- 172.21.15.92:0/3177932868 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff92806e0b0 msgr2=0x7ff928070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:56.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.763+0000 7ff94535d640 1 --2- 172.21.15.92:0/3177932868 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff92806e0b0 0x7ff928070560 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff930000c00 tx=0x7ff930009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:56.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.763+0000 7ff94535d640 1 -- 172.21.15.92:0/3177932868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff940101a30 msgr2=0x7ff94010e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:56.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.763+0000 7ff94535d640 1 --2- 172.21.15.92:0/3177932868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff940101a30 0x7ff94010e3d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff93400c830 tx=0x7ff934004430 comp rx=0 tx=0).stop 2024-05-06T12:25:56.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.763+0000 7ff94535d640 1 -- 172.21.15.92:0/3177932868 shutdown_connections 2024-05-06T12:25:56.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.763+0000 7ff94535d640 1 --2- 172.21.15.92:0/3177932868 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7ff92806e0b0 0x7ff928070560 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:56.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.763+0000 7ff94535d640 1 --2- 172.21.15.92:0/3177932868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff940101a30 0x7ff94010e3d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:56.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.763+0000 7ff94535d640 1 --2- 172.21.15.92:0/3177932868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff940101090 0x7ff94010de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:56.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.763+0000 7ff94535d640 1 -- 172.21.15.92:0/3177932868 >> 172.21.15.92:0/3177932868 conn(0x7ff9400f6c20 msgr2=0x7ff9400f7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:56.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.764+0000 7ff94535d640 1 -- 172.21.15.92:0/3177932868 shutdown_connections 2024-05-06T12:25:56.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.764+0000 7ff94535d640 1 -- 172.21.15.92:0/3177932868 wait complete. 2024-05-06T12:25:56.777 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-05-06T12:25:56.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.873+0000 7fd2e4954640 1 -- 172.21.15.92:0/2608358873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2e0101090 msgr2=0x7fd2e0101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:56.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.873+0000 7fd2e4954640 1 --2- 172.21.15.92:0/2608358873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2e0101090 0x7fd2e0101460 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd2d00048d0 tx=0x7fd2d0009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:56.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.873+0000 7fd2e4954640 1 -- 172.21.15.92:0/2608358873 shutdown_connections 2024-05-06T12:25:56.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.873+0000 7fd2e4954640 1 --2- 172.21.15.92:0/2608358873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2e0101a30 0x7fd2e00fb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:56.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.873+0000 7fd2e4954640 1 --2- 172.21.15.92:0/2608358873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2e0101090 0x7fd2e0101460 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:56.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.873+0000 7fd2e4954640 1 -- 172.21.15.92:0/2608358873 >> 172.21.15.92:0/2608358873 conn(0x7fd2e00f6c20 msgr2=0x7fd2e00f9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:56.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.873+0000 7fd2e4954640 1 -- 172.21.15.92:0/2608358873 shutdown_connections 2024-05-06T12:25:56.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.873+0000 7fd2e4954640 1 -- 172.21.15.92:0/2608358873 wait complete. 2024-05-06T12:25:56.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.873+0000 7fd2e4954640 1 Processor -- start 2024-05-06T12:25:56.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.874+0000 7fd2e4954640 1 -- start start 2024-05-06T12:25:56.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.874+0000 7fd2e4954640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2e0101090 0x7fd2e0112ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:56.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.874+0000 7fd2e4954640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2e0101a30 0x7fd2e0113220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:56.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.874+0000 7fd2e4954640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e0113820 con 0x7fd2e0101a30 2024-05-06T12:25:56.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.874+0000 7fd2e4954640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e010ce70 con 0x7fd2e0101090 2024-05-06T12:25:56.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.874+0000 7fd2ddd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2e0101a30 0x7fd2e0113220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:56.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.874+0000 7fd2ddd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2e0101a30 0x7fd2e0113220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52090/0 (socket says 172.21.15.92:52090) 2024-05-06T12:25:56.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.874+0000 7fd2ddd9b640 1 -- 172.21.15.92:0/1215851323 learned_addr learned my addr 172.21.15.92:0/1215851323 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:56.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.874+0000 7fd2ddd9b640 1 -- 172.21.15.92:0/1215851323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2e0101090 msgr2=0x7fd2e0112ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:56.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.874+0000 7fd2ddd9b640 1 --2- 172.21.15.92:0/1215851323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2e0101090 0x7fd2e0112ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:56.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.874+0000 7fd2ddd9b640 1 -- 172.21.15.92:0/1215851323 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d0005040 con 0x7fd2e0101a30 2024-05-06T12:25:56.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.874+0000 7fd2ddd9b640 1 --2- 172.21.15.92:0/1215851323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2e0101a30 0x7fd2e0113220 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd2d400eb70 tx=0x7fd2d400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:56.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.874+0000 7fd2ceffd640 1 -- 172.21.15.92:0/1215851323 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2d4007c30 con 0x7fd2e0101a30 2024-05-06T12:25:56.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.874+0000 7fd2ceffd640 1 -- 172.21.15.92:0/1215851323 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2d400f040 con 0x7fd2e0101a30 2024-05-06T12:25:56.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.874+0000 7fd2e4954640 1 -- 172.21.15.92:0/1215851323 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2e010d0f0 con 0x7fd2e0101a30 2024-05-06T12:25:56.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.874+0000 7fd2ceffd640 1 -- 172.21.15.92:0/1215851323 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2d4009680 con 0x7fd2e0101a30 2024-05-06T12:25:56.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.874+0000 7fd2e4954640 1 -- 172.21.15.92:0/1215851323 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e010d610 con 0x7fd2e0101a30 2024-05-06T12:25:56.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.875+0000 7fd2e4954640 1 -- 172.21.15.92:0/1215851323 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2e00fc800 con 0x7fd2e0101a30 2024-05-06T12:25:56.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.876+0000 7fd2ceffd640 1 -- 172.21.15.92:0/1215851323 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd2d400cd40 con 0x7fd2e0101a30 2024-05-06T12:25:56.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.876+0000 7fd2ceffd640 1 --2- 172.21.15.92:0/1215851323 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd2c806e0b0 0x7fd2c8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:56.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.876+0000 7fd2ceffd640 1 -- 172.21.15.92:0/1215851323 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd2d408d800 con 0x7fd2e0101a30 2024-05-06T12:25:56.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.876+0000 7fd2de59c640 1 --2- 172.21.15.92:0/1215851323 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd2c806e0b0 0x7fd2c8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:56.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.877+0000 7fd2de59c640 1 --2- 172.21.15.92:0/1215851323 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd2c806e0b0 0x7fd2c8070560 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd2d00043d0 tx=0x7fd2d0009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:56.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:56.882+0000 7fd2ceffd640 1 -- 172.21.15.92:0/1215851323 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd2d4056be0 con 0x7fd2e0101a30 2024-05-06T12:25:57.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.060+0000 7fd2e4954640 1 -- 172.21.15.92:0/1215851323 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fd2e004bcc0 con 0x7fd2e0101a30 2024-05-06T12:25:57.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.060+0000 7fd2ceffd640 1 -- 172.21.15.92:0/1215851323 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7fd2d405a2a0 con 0x7fd2e0101a30 2024-05-06T12:25:57.062 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2024-05-06T12:25:57.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.062+0000 7fd2e4954640 1 -- 172.21.15.92:0/1215851323 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd2c806e0b0 msgr2=0x7fd2c8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:57.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.062+0000 7fd2e4954640 1 --2- 172.21.15.92:0/1215851323 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd2c806e0b0 0x7fd2c8070560 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd2d00043d0 tx=0x7fd2d0009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:57.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.062+0000 7fd2e4954640 1 -- 172.21.15.92:0/1215851323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2e0101a30 msgr2=0x7fd2e0113220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:57.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.062+0000 7fd2e4954640 1 --2- 172.21.15.92:0/1215851323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2e0101a30 0x7fd2e0113220 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd2d400eb70 tx=0x7fd2d400b040 comp rx=0 tx=0).stop 2024-05-06T12:25:57.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.062+0000 7fd2e4954640 1 -- 172.21.15.92:0/1215851323 shutdown_connections 2024-05-06T12:25:57.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.062+0000 7fd2e4954640 1 --2- 172.21.15.92:0/1215851323 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7fd2c806e0b0 0x7fd2c8070560 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:57.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.062+0000 7fd2e4954640 1 --2- 172.21.15.92:0/1215851323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2e0101a30 0x7fd2e0113220 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:57.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.062+0000 7fd2e4954640 1 --2- 172.21.15.92:0/1215851323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2e0101090 0x7fd2e0112ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:57.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.062+0000 7fd2e4954640 1 -- 172.21.15.92:0/1215851323 >> 172.21.15.92:0/1215851323 conn(0x7fd2e00f6c20 msgr2=0x7fd2e00f8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:57.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.062+0000 7fd2e4954640 1 -- 172.21.15.92:0/1215851323 shutdown_connections 2024-05-06T12:25:57.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.062+0000 7fd2e4954640 1 -- 172.21.15.92:0/1215851323 wait complete. 2024-05-06T12:25:57.074 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-06T12:25:57.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.168+0000 7efe98c6e640 1 -- 172.21.15.92:0/1404545078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe94101a80 msgr2=0x7efe940fb110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:57.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.168+0000 7efe98c6e640 1 --2- 172.21.15.92:0/1404545078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe94101a80 0x7efe940fb110 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7efe88009c50 tx=0x7efe88004d10 comp rx=0 tx=0).stop 2024-05-06T12:25:57.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.168+0000 7efe98c6e640 1 -- 172.21.15.92:0/1404545078 shutdown_connections 2024-05-06T12:25:57.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.168+0000 7efe98c6e640 1 --2- 172.21.15.92:0/1404545078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe94101a80 0x7efe940fb110 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:57.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.168+0000 7efe98c6e640 1 --2- 172.21.15.92:0/1404545078 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe941010e0 0x7efe941014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:57.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.168+0000 7efe98c6e640 1 -- 172.21.15.92:0/1404545078 >> 172.21.15.92:0/1404545078 conn(0x7efe940f6cb0 msgr2=0x7efe940f90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:57.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.168+0000 7efe98c6e640 1 -- 172.21.15.92:0/1404545078 shutdown_connections 2024-05-06T12:25:57.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.168+0000 7efe98c6e640 1 -- 172.21.15.92:0/1404545078 wait complete. 2024-05-06T12:25:57.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.168+0000 7efe98c6e640 1 Processor -- start 2024-05-06T12:25:57.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.168+0000 7efe98c6e640 1 -- start start 2024-05-06T12:25:57.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.168+0000 7efe98c6e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe941010e0 0x7efe940fcd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:57.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.168+0000 7efe98c6e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe94101a80 0x7efe940fd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:57.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.168+0000 7efe98c6e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe940fec60 con 0x7efe94101a80 2024-05-06T12:25:57.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.168+0000 7efe98c6e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe940fd800 con 0x7efe941010e0 2024-05-06T12:25:57.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.169+0000 7efe91d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe94101a80 0x7efe940fd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:57.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.169+0000 7efe91d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe94101a80 0x7efe940fd2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52110/0 (socket says 172.21.15.92:52110) 2024-05-06T12:25:57.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.169+0000 7efe91d9b640 1 -- 172.21.15.92:0/920450210 learned_addr learned my addr 172.21.15.92:0/920450210 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:57.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.169+0000 7efe9259c640 1 --2- 172.21.15.92:0/920450210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe941010e0 0x7efe940fcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:57.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.169+0000 7efe91d9b640 1 -- 172.21.15.92:0/920450210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe941010e0 msgr2=0x7efe940fcd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:57.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.169+0000 7efe91d9b640 1 --2- 172.21.15.92:0/920450210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe941010e0 0x7efe940fcd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:57.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.169+0000 7efe91d9b640 1 -- 172.21.15.92:0/920450210 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe880098b0 con 0x7efe94101a80 2024-05-06T12:25:57.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.169+0000 7efe91d9b640 1 --2- 172.21.15.92:0/920450210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe94101a80 0x7efe940fd2c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7efe88004a30 tx=0x7efe88004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:57.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.169+0000 7efe82ffd640 1 -- 172.21.15.92:0/920450210 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe8803a070 con 0x7efe94101a80 2024-05-06T12:25:57.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.169+0000 7efe82ffd640 1 -- 172.21.15.92:0/920450210 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe88035b20 con 0x7efe94101a80 2024-05-06T12:25:57.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.169+0000 7efe98c6e640 1 -- 172.21.15.92:0/920450210 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe940fda80 con 0x7efe94101a80 2024-05-06T12:25:57.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.169+0000 7efe82ffd640 1 -- 172.21.15.92:0/920450210 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe8803ea70 con 0x7efe94101a80 2024-05-06T12:25:57.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.169+0000 7efe98c6e640 1 -- 172.21.15.92:0/920450210 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe94119780 con 0x7efe94101a80 2024-05-06T12:25:57.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.170+0000 7efe98c6e640 1 -- 172.21.15.92:0/920450210 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe940fc850 con 0x7efe94101a80 2024-05-06T12:25:57.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.170+0000 7efe82ffd640 1 -- 172.21.15.92:0/920450210 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7efe88032070 con 0x7efe94101a80 2024-05-06T12:25:57.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.171+0000 7efe82ffd640 1 --2- 172.21.15.92:0/920450210 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7efe7c06e0b0 0x7efe7c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:57.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.171+0000 7efe82ffd640 1 -- 172.21.15.92:0/920450210 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7efe880afce0 con 0x7efe94101a80 2024-05-06T12:25:57.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.171+0000 7efe9259c640 1 --2- 172.21.15.92:0/920450210 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7efe7c06e0b0 0x7efe7c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:57.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.171+0000 7efe9259c640 1 --2- 172.21.15.92:0/920450210 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7efe7c06e0b0 0x7efe7c070560 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efe8c007fd0 tx=0x7efe8c009e90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:57.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.176+0000 7efe82ffd640 1 -- 172.21.15.92:0/920450210 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7efe880b4070 con 0x7efe94101a80 2024-05-06T12:25:57.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:57 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/766609827' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-06T12:25:57.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:57 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/1012141198' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-06T12:25:57.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:57 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/3177932868' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-06T12:25:57.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.362+0000 7efe98c6e640 1 -- 172.21.15.92:0/920450210 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7efe94108270 con 0x7efe94101a80 2024-05-06T12:25:57.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.363+0000 7efe82ffd640 1 -- 172.21.15.92:0/920450210 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 39 entries v86) v1 ==== 81+0+2307 (secure 0 0 0) 0x7efe88079180 con 0x7efe94101a80 2024-05-06T12:25:57.364 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6828/1663253598 2024-05-07T12:21:58.517697+0000 2024-05-06T12:25:57.364 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6801/648200129 2024-05-07T12:16:38.777479+0000 2024-05-06T12:25:57.364 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6801/543287293 2024-05-07T12:07:03.802107+0000 2024-05-06T12:25:57.364 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/3693315250 2024-05-07T12:05:56.546947+0000 2024-05-06T12:25:57.364 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/3493498366 2024-05-07T12:14:01.169339+0000 2024-05-06T12:25:57.365 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.190:0/3060689070 2024-05-07T12:15:47.314494+0000 2024-05-06T12:25:57.365 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/2210153116 2024-05-07T12:05:56.546947+0000 2024-05-06T12:25:57.365 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/3417170577 2024-05-07T12:14:01.169339+0000 2024-05-06T12:25:57.365 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/1054764607 2024-05-07T12:16:38.777479+0000 2024-05-06T12:25:57.365 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/856987777 2024-05-07T12:14:01.169339+0000 2024-05-06T12:25:57.365 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/895013058 2024-05-07T12:05:25.954232+0000 2024-05-06T12:25:57.365 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/2440233745 2024-05-07T12:07:03.802107+0000 2024-05-06T12:25:57.365 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6800/543287293 2024-05-07T12:07:03.802107+0000 2024-05-06T12:25:57.365 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.190:6829/3484477933 2024-05-07T12:15:47.314494+0000 2024-05-06T12:25:57.365 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/758782094 2024-05-07T12:05:25.954232+0000 2024-05-06T12:25:57.365 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/1552275072 2024-05-07T12:07:03.802107+0000 2024-05-06T12:25:57.365 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6800/3731057893 2024-05-07T12:05:56.546947+0000 2024-05-06T12:25:57.365 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/1407601857 2024-05-07T12:05:56.546947+0000 2024-05-06T12:25:57.365 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/1345929606 2024-05-07T12:16:38.777479+0000 2024-05-06T12:25:57.365 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6801/3045905943 2024-05-07T12:14:01.169339+0000 2024-05-06T12:25:57.365 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6829/1663253598 2024-05-07T12:21:58.517697+0000 2024-05-06T12:25:57.366 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.190:0/2858276177 2024-05-07T12:15:47.314494+0000 2024-05-06T12:25:57.366 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6827/4255634245 2024-05-07T12:12:00.896508+0000 2024-05-06T12:25:57.366 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/4121052357 2024-05-07T12:14:01.169339+0000 2024-05-06T12:25:57.366 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/997820112 2024-05-07T12:05:25.954232+0000 2024-05-06T12:25:57.366 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6800/3656822751 2024-05-07T12:05:25.954232+0000 2024-05-06T12:25:57.366 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/1532977673 2024-05-07T12:07:03.802107+0000 2024-05-06T12:25:57.366 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6801/3656822751 2024-05-07T12:05:25.954232+0000 2024-05-06T12:25:57.366 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6801/3731057893 2024-05-07T12:05:56.546947+0000 2024-05-06T12:25:57.366 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6826/4255634245 2024-05-07T12:12:00.896508+0000 2024-05-06T12:25:57.366 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.190:6828/3484477933 2024-05-07T12:15:47.314494+0000 2024-05-06T12:25:57.366 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.190:0/2419054582 2024-05-07T12:15:47.314494+0000 2024-05-06T12:25:57.366 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/418878214 2024-05-07T12:16:38.777479+0000 2024-05-06T12:25:57.366 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/2098378816 2024-05-07T12:16:38.777479+0000 2024-05-06T12:25:57.366 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6800/3045905943 2024-05-07T12:14:01.169339+0000 2024-05-06T12:25:57.367 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/2150816131 2024-05-07T12:16:38.777479+0000 2024-05-06T12:25:57.367 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.190:0/2974973416 2024-05-07T12:15:47.314494+0000 2024-05-06T12:25:57.367 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/1862221479 2024-05-07T12:16:38.777479+0000 2024-05-06T12:25:57.367 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6800/648200129 2024-05-07T12:16:38.777479+0000 2024-05-06T12:25:57.367 INFO:teuthology.orchestra.run.smithi092.stderr:listed 39 entries 2024-05-06T12:25:57.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.365+0000 7efe98c6e640 1 -- 172.21.15.92:0/920450210 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7efe7c06e0b0 msgr2=0x7efe7c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:57.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.365+0000 7efe98c6e640 1 --2- 172.21.15.92:0/920450210 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7efe7c06e0b0 0x7efe7c070560 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efe8c007fd0 tx=0x7efe8c009e90 comp rx=0 tx=0).stop 2024-05-06T12:25:57.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.365+0000 7efe98c6e640 1 -- 172.21.15.92:0/920450210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe94101a80 msgr2=0x7efe940fd2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:57.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.365+0000 7efe98c6e640 1 --2- 172.21.15.92:0/920450210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe94101a80 0x7efe940fd2c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7efe88004a30 tx=0x7efe88004d10 comp rx=0 tx=0).stop 2024-05-06T12:25:57.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.365+0000 7efe98c6e640 1 -- 172.21.15.92:0/920450210 shutdown_connections 2024-05-06T12:25:57.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.365+0000 7efe98c6e640 1 --2- 172.21.15.92:0/920450210 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7efe7c06e0b0 0x7efe7c070560 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:57.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.365+0000 7efe98c6e640 1 --2- 172.21.15.92:0/920450210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe94101a80 0x7efe940fd2c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:57.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.365+0000 7efe98c6e640 1 --2- 172.21.15.92:0/920450210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe941010e0 0x7efe940fcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:57.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.365+0000 7efe98c6e640 1 -- 172.21.15.92:0/920450210 >> 172.21.15.92:0/920450210 conn(0x7efe940f6cb0 msgr2=0x7efe940f7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:57.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.365+0000 7efe98c6e640 1 -- 172.21.15.92:0/920450210 shutdown_connections 2024-05-06T12:25:57.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.365+0000 7efe98c6e640 1 -- 172.21.15.92:0/920450210 wait complete. 2024-05-06T12:25:57.378 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:57 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/766609827' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-06T12:25:57.378 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:57 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/1012141198' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-06T12:25:57.378 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:57 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/3177932868' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-06T12:25:57.379 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-06T12:25:57.379 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-06T12:25:57.394 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-06T12:25:57.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.527+0000 7f68d075d640 1 -- 172.21.15.92:0/3874065823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68c8102210 msgr2=0x7f68c81025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:57.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.527+0000 7f68d075d640 1 --2- 172.21.15.92:0/3874065823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68c8102210 0x7f68c81025e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f68c40048d0 tx=0x7f68c4009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:57.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.527+0000 7f68d075d640 1 -- 172.21.15.92:0/3874065823 shutdown_connections 2024-05-06T12:25:57.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.527+0000 7f68d075d640 1 --2- 172.21.15.92:0/3874065823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68c8102b20 0x7f68c80fa7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:57.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.527+0000 7f68d075d640 1 --2- 172.21.15.92:0/3874065823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68c8102210 0x7f68c81025e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:57.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.527+0000 7f68d075d640 1 -- 172.21.15.92:0/3874065823 >> 172.21.15.92:0/3874065823 conn(0x7f68c8071770 msgr2=0x7f68c8071b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:57.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.527+0000 7f68d075d640 1 -- 172.21.15.92:0/3874065823 shutdown_connections 2024-05-06T12:25:57.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.528+0000 7f68d075d640 1 -- 172.21.15.92:0/3874065823 wait complete. 2024-05-06T12:25:57.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.528+0000 7f68d075d640 1 Processor -- start 2024-05-06T12:25:57.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.528+0000 7f68d075d640 1 -- start start 2024-05-06T12:25:57.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.528+0000 7f68d075d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68c8102210 0x7f68c8112d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:57.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.528+0000 7f68d075d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68c8102b20 0x7f68c81132a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:57.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.528+0000 7f68d075d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c81138a0 con 0x7f68c8102b20 2024-05-06T12:25:57.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.528+0000 7f68d075d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c810cef0 con 0x7f68c8102210 2024-05-06T12:25:57.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.528+0000 7f68cdcf8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68c8102b20 0x7f68c81132a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:57.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.528+0000 7f68cdcf8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68c8102b20 0x7f68c81132a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52120/0 (socket says 172.21.15.92:52120) 2024-05-06T12:25:57.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.528+0000 7f68cdcf8640 1 -- 172.21.15.92:0/2846312846 learned_addr learned my addr 172.21.15.92:0/2846312846 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:57.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.528+0000 7f68ce4f9640 1 --2- 172.21.15.92:0/2846312846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68c8102210 0x7f68c8112d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:57.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.529+0000 7f68cdcf8640 1 -- 172.21.15.92:0/2846312846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68c8102210 msgr2=0x7f68c8112d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:57.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.529+0000 7f68cdcf8640 1 --2- 172.21.15.92:0/2846312846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68c8102210 0x7f68c8112d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:57.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.529+0000 7f68cdcf8640 1 -- 172.21.15.92:0/2846312846 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c4005040 con 0x7f68c8102b20 2024-05-06T12:25:57.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.529+0000 7f68ce4f9640 1 --2- 172.21.15.92:0/2846312846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68c8102210 0x7f68c8112d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-06T12:25:57.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.529+0000 7f68cdcf8640 1 --2- 172.21.15.92:0/2846312846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68c8102b20 0x7f68c81132a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f68b800c7e0 tx=0x7f68b80043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:57.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.529+0000 7f68beffd640 1 -- 172.21.15.92:0/2846312846 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68b800a820 con 0x7f68c8102b20 2024-05-06T12:25:57.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.529+0000 7f68beffd640 1 -- 172.21.15.92:0/2846312846 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f68b800ae50 con 0x7f68c8102b20 2024-05-06T12:25:57.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.529+0000 7f68beffd640 1 -- 172.21.15.92:0/2846312846 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68b80113a0 con 0x7f68c8102b20 2024-05-06T12:25:57.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.529+0000 7f68d075d640 1 -- 172.21.15.92:0/2846312846 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c810d1d0 con 0x7f68c8102b20 2024-05-06T12:25:57.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.529+0000 7f68d075d640 1 -- 172.21.15.92:0/2846312846 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c810d6a0 con 0x7f68c8102b20 2024-05-06T12:25:57.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.530+0000 7f68d075d640 1 -- 172.21.15.92:0/2846312846 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c80fdb10 con 0x7f68c8102b20 2024-05-06T12:25:57.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.530+0000 7f68beffd640 1 -- 172.21.15.92:0/2846312846 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f68b800a980 con 0x7f68c8102b20 2024-05-06T12:25:57.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.531+0000 7f68beffd640 1 --2- 172.21.15.92:0/2846312846 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f68b406e0b0 0x7f68b4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:57.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.531+0000 7f68beffd640 1 -- 172.21.15.92:0/2846312846 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f68b808dd00 con 0x7f68c8102b20 2024-05-06T12:25:57.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.531+0000 7f68ce4f9640 1 --2- 172.21.15.92:0/2846312846 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f68b406e0b0 0x7f68b4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:57.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.531+0000 7f68ce4f9640 1 --2- 172.21.15.92:0/2846312846 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f68b406e0b0 0x7f68b4070560 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f68c4003080 tx=0x7f68c4009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:57.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.535+0000 7f68beffd640 1 -- 172.21.15.92:0/2846312846 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f68b80570b0 con 0x7f68c8102b20 2024-05-06T12:25:57.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.711+0000 7f68d075d640 1 -- 172.21.15.92:0/2846312846 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68c8109370 con 0x7f68c8102b20 2024-05-06T12:25:57.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.712+0000 7f68beffd640 1 -- 172.21.15.92:0/2846312846 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16615 (secure 0 0 0) 0x7f68b805a770 con 0x7f68c8102b20 2024-05-06T12:25:57.714 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-06T12:25:57.714 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":86,"fsid":"aee1d912-0ba0-11ef-bc97-c7b262605968","created":"2024-05-06T12:04:51.220909+0000","modified":"2024-05-06T12:22:36.092329+0000","last_up_change":"2024-05-06T12:20:59.798275+0000","last_in_change":"2024-05-06T12:10:07.815689+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-06T12:09:12.129668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-05-06T12:11:43.666342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-05-06T12:11:44.183559+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b276435-4698-4017-9535-4be41a4e1379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3125171617},{"type":"v1","addr":"172.21.15.92:6803","nonce":3125171617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3125171617},{"type":"v1","addr":"172.21.15.92:6805","nonce":3125171617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3125171617},{"type":"v1","addr":"172.21.15.92:6809","nonce":3125171617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3125171617},{"type":"v1","addr":"172.21.15.92:6807","nonce":3125171617}]},"public_addr":"172.21.15.92:6803/3125171617","cluster_addr":"172.21.15.92:6805/3125171617","heartbeat_back_addr":"172.21.15.92:6809/3125171617","heartbeat_front_addr":"172.21.15.92:6807/3125171617","state":["exists","up"]},{"osd":1,"uuid":"21688119-d15f-4b31-8842-090cc539c76d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1875265240},{"type":"v1","addr":"172.21.15.92:6811","nonce":1875265240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1875265240},{"type":"v1","addr":"172.21.15.92:6813","nonce":1875265240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1875265240},{"type":"v1","addr":"172.21.15.92:6817","nonce":1875265240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1875265240},{"type":"v1","addr":"172.21.15.92:6815","nonce":1875265240}]},"public_addr":"172.21.15.92:6811/1875265240","cluster_addr":"172.21.15.92:6813/1875265240","heartbeat_back_addr":"172.21.15.92:6817/1875265240","heartbeat_front_addr":"172.21.15.92:6815/1875265240","state":["exists","up"]},{"osd":2,"uuid":"74d16dfb-b118-432d-827a-59706d69c393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":629593219},{"type":"v1","addr":"172.21.15.92:6819","nonce":629593219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":629593219},{"type":"v1","addr":"172.21.15.92:6821","nonce":629593219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":629593219},{"type":"v1","addr":"172.21.15.92:6825","nonce":629593219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":629593219},{"type":"v1","addr":"172.21.15.92:6823","nonce":629593219}]},"public_addr":"172.21.15.92:6819/629593219","cluster_addr":"172.21.15.92:6821/629593219","heartbeat_back_addr":"172.21.15.92:6825/629593219","heartbeat_front_addr":"172.21.15.92:6823/629593219","state":["exists","up"]},{"osd":3,"uuid":"5a9aa5ed-2502-4c73-a8ac-cb7d0259500b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":3441479977},{"type":"v1","addr":"172.21.15.190:6801","nonce":3441479977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":3441479977},{"type":"v1","addr":"172.21.15.190:6803","nonce":3441479977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":3441479977},{"type":"v1","addr":"172.21.15.190:6807","nonce":3441479977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":3441479977},{"type":"v1","addr":"172.21.15.190:6805","nonce":3441479977}]},"public_addr":"172.21.15.190:6801/3441479977","cluster_addr":"172.21.15.190:6803/3441479977","heartbeat_back_addr":"172.21.15.190:6807/3441479977","heartbeat_front_addr":"172.21.15.190:6805/3441479977","state":["exists","up"]},{"osd":4,"uuid":"d4039e4a-c758-4632-85d6-1b7e725c0293","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":839214066},{"type":"v1","addr":"172.21.15.190:6809","nonce":839214066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":839214066},{"type":"v1","addr":"172.21.15.190:6811","nonce":839214066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":839214066},{"type":"v1","addr":"172.21.15.190:6815","nonce":839214066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":839214066},{"type":"v1","addr":"172.21.15.190:6813","nonce":839214066}]},"public_addr":"172.21.15.190:6809/839214066","cluster_addr":"172.21.15.190:6811/839214066","heartbeat_back_addr":"172.21.15.190:6815/839214066","heartbeat_front_addr":"172.21.15.190:6813/839214066","state":["exists","up"]},{"osd":5,"uuid":"ec707eee-5f24-48bb-818a-547f804fc41b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":10700099},{"type":"v1","addr":"172.21.15.190:6817","nonce":10700099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":10700099},{"type":"v1","addr":"172.21.15.190:6819","nonce":10700099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":10700099},{"type":"v1","addr":"172.21.15.190:6823","nonce":10700099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":10700099},{"type":"v1","addr":"172.21.15.190:6821","nonce":10700099}]},"public_addr":"172.21.15.190:6817/10700099","cluster_addr":"172.21.15.190:6819/10700099","heartbeat_back_addr":"172.21.15.190:6823/10700099","heartbeat_front_addr":"172.21.15.190:6821/10700099","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:08:15.348582+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:08:42.571577+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:09:10.093700+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:09:34.257829+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:09:59.795522+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-06T12:10:25.131646+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:6828/1663253598":"2024-05-07T12:21:58.517697+0000","172.21.15.92:6801/648200129":"2024-05-07T12:16:38.777479+0000","172.21.15.92:6801/543287293":"2024-05-07T12:07:03.802107+0000","172.21.15.92:0/3693315250":"2024-05-07T12:05:56.546947+0000","172.21.15.92:0/3493498366":"2024-05-07T12:14:01.169339+0000","172.21.15.190:0/3060689070":"2024-05-07T12:15:47.314494+0000","172.21.15.92:0/2210153116":"2024-05-07T12:05:56.546947+0000","172.21.15.92:0/3417170577":"2024-05-07T12:14:01.169339+0000","172.21.15.92:0/1054764607":"2024-05-07T12:16:38.777479+0000","172.21.15.92:0/856987777":"2024-05-07T12:14:01.169339+0000","172.21.15.92:0/895013058":"2024-05-07T12:05:25.954232+0000","172.21.15.92:0/2440233745":"2024-05-07T12:07:03.802107+0000","172.21.15.92:6800/543287293":"2024-05-07T12:07:03.802107+0000","172.21.15.190:6829/3484477933":"2024-05-07T12:15:47.314494+0000","172.21.15.92:0/758782094":"2024-05-07T12:05:25.954232+0000","172.21.15.92:0/1552275072":"2024-05-07T12:07:03.802107+0000","172.21.15.92:6800/3731057893":"2024-05-07T12:05:56.546947+0000","172.21.15.92:0/1407601857":"2024-05-07T12:05:56.546947+0000","172.21.15.92:0/1345929606":"2024-05-07T12:16:38.777479+0000","172.21.15.92:6801/3045905943":"2024-05-07T12:14:01.169339+0000","172.21.15.92:6829/1663253598":"2024-05-07T12:21:58.517697+0000","172.21.15.190:0/2858276177":"2024-05-07T12:15:47.314494+0000","172.21.15.92:6827/4255634245":"2024-05-07T12:12:00.896508+0000","172.21.15.92:0/4121052357":"2024-05-07T12:14:01.169339+0000","172.21.15.92:0/997820112":"2024-05-07T12:05:25.954232+0000","172.21.15.92:6800/3656822751":"2024-05-07T12:05:25.954232+0000","172.21.15.92:0/1532977673":"2024-05-07T12:07:03.802107+0000","172.21.15.92:6801/3656822751":"2024-05-07T12:05:25.954232+0000","172.21.15.92:6801/3731057893":"2024-05-07T12:05:56.546947+0000","172.21.15.92:6826/4255634245":"2024-05-07T12:12:00.896508+0000","172.21.15.190:6828/3484477933":"2024-05-07T12:15:47.314494+0000","172.21.15.190:0/2419054582":"2024-05-07T12:15:47.314494+0000","172.21.15.92:0/418878214":"2024-05-07T12:16:38.777479+0000","172.21.15.92:0/2098378816":"2024-05-07T12:16:38.777479+0000","172.21.15.92:6800/3045905943":"2024-05-07T12:14:01.169339+0000","172.21.15.92:0/2150816131":"2024-05-07T12:16:38.777479+0000","172.21.15.190:0/2974973416":"2024-05-07T12:15:47.314494+0000","172.21.15.92:0/1862221479":"2024-05-07T12:16:38.777479+0000","172.21.15.92:6800/648200129":"2024-05-07T12:16:38.777479+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-06T12:25:57.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.714+0000 7f68d075d640 1 -- 172.21.15.92:0/2846312846 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f68b406e0b0 msgr2=0x7f68b4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:57.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.714+0000 7f68d075d640 1 --2- 172.21.15.92:0/2846312846 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f68b406e0b0 0x7f68b4070560 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f68c4003080 tx=0x7f68c4009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:57.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.714+0000 7f68d075d640 1 -- 172.21.15.92:0/2846312846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68c8102b20 msgr2=0x7f68c81132a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:57.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.714+0000 7f68d075d640 1 --2- 172.21.15.92:0/2846312846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68c8102b20 0x7f68c81132a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f68b800c7e0 tx=0x7f68b80043c0 comp rx=0 tx=0).stop 2024-05-06T12:25:57.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.714+0000 7f68d075d640 1 -- 172.21.15.92:0/2846312846 shutdown_connections 2024-05-06T12:25:57.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.714+0000 7f68d075d640 1 --2- 172.21.15.92:0/2846312846 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f68b406e0b0 0x7f68b4070560 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:57.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.714+0000 7f68d075d640 1 --2- 172.21.15.92:0/2846312846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68c8102b20 0x7f68c81132a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:57.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.714+0000 7f68d075d640 1 --2- 172.21.15.92:0/2846312846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68c8102210 0x7f68c8112d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:57.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.714+0000 7f68d075d640 1 -- 172.21.15.92:0/2846312846 >> 172.21.15.92:0/2846312846 conn(0x7f68c8071770 msgr2=0x7f68c80f92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:57.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.715+0000 7f68d075d640 1 -- 172.21.15.92:0/2846312846 shutdown_connections 2024-05-06T12:25:57.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.715+0000 7f68d075d640 1 -- 172.21.15.92:0/2846312846 wait complete. 2024-05-06T12:25:57.727 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-06T12:09:12.129668+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-05-06T12:11:43.666342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-05-06T12:11:44.183559+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-06T12:25:57.727 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-06T12:25:57.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.822+0000 7f5c3826e640 1 -- 172.21.15.92:0/3127250706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c30102b30 msgr2=0x7f5c300fc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:57.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.822+0000 7f5c3826e640 1 --2- 172.21.15.92:0/3127250706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c30102b30 0x7f5c300fc1c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5c200048d0 tx=0x7f5c20009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:57.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.822+0000 7f5c3826e640 1 -- 172.21.15.92:0/3127250706 shutdown_connections 2024-05-06T12:25:57.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.822+0000 7f5c3826e640 1 --2- 172.21.15.92:0/3127250706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c30102b30 0x7f5c300fc1c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:57.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.822+0000 7f5c3826e640 1 --2- 172.21.15.92:0/3127250706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c30102190 0x7f5c30102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:57.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.822+0000 7f5c3826e640 1 -- 172.21.15.92:0/3127250706 >> 172.21.15.92:0/3127250706 conn(0x7f5c300f7da0 msgr2=0x7f5c300fa1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:57.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.822+0000 7f5c3826e640 1 -- 172.21.15.92:0/3127250706 shutdown_connections 2024-05-06T12:25:57.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.822+0000 7f5c3826e640 1 -- 172.21.15.92:0/3127250706 wait complete. 2024-05-06T12:25:57.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.823+0000 7f5c3826e640 1 Processor -- start 2024-05-06T12:25:57.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.823+0000 7f5c3826e640 1 -- start start 2024-05-06T12:25:57.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.823+0000 7f5c3826e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c30102190 0x7f5c3006f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:57.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.823+0000 7f5c3826e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c30102b30 0x7f5c3006fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:57.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.823+0000 7f5c3826e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c30071650 con 0x7f5c30102b30 2024-05-06T12:25:57.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.823+0000 7f5c3826e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c300701f0 con 0x7f5c30102190 2024-05-06T12:25:57.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.823+0000 7f5c35809640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c30102b30 0x7f5c3006fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:57.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.823+0000 7f5c35809640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c30102b30 0x7f5c3006fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52132/0 (socket says 172.21.15.92:52132) 2024-05-06T12:25:57.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.823+0000 7f5c35809640 1 -- 172.21.15.92:0/2201083051 learned_addr learned my addr 172.21.15.92:0/2201083051 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:57.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.823+0000 7f5c3600a640 1 --2- 172.21.15.92:0/2201083051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c30102190 0x7f5c3006f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:57.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.823+0000 7f5c35809640 1 -- 172.21.15.92:0/2201083051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c30102190 msgr2=0x7f5c3006f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:57.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.823+0000 7f5c35809640 1 --2- 172.21.15.92:0/2201083051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c30102190 0x7f5c3006f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:57.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.823+0000 7f5c35809640 1 -- 172.21.15.92:0/2201083051 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c20005040 con 0x7f5c30102b30 2024-05-06T12:25:57.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.823+0000 7f5c3600a640 1 --2- 172.21.15.92:0/2201083051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c30102190 0x7f5c3006f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-06T12:25:57.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.823+0000 7f5c35809640 1 --2- 172.21.15.92:0/2201083051 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c30102b30 0x7f5c3006fcb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5c20004930 tx=0x7f5c20009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:57.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.824+0000 7f5c26ffd640 1 -- 172.21.15.92:0/2201083051 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c200094c0 con 0x7f5c30102b30 2024-05-06T12:25:57.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.824+0000 7f5c3826e640 1 -- 172.21.15.92:0/2201083051 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c30070470 con 0x7f5c30102b30 2024-05-06T12:25:57.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.824+0000 7f5c26ffd640 1 -- 172.21.15.92:0/2201083051 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c20032e50 con 0x7f5c30102b30 2024-05-06T12:25:57.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.824+0000 7f5c26ffd640 1 -- 172.21.15.92:0/2201083051 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c200433b0 con 0x7f5c30102b30 2024-05-06T12:25:57.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.824+0000 7f5c3826e640 1 -- 172.21.15.92:0/2201083051 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c301193d0 con 0x7f5c30102b30 2024-05-06T12:25:57.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.825+0000 7f5c3826e640 1 -- 172.21.15.92:0/2201083051 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c300fd900 con 0x7f5c30102b30 2024-05-06T12:25:57.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.825+0000 7f5c26ffd640 1 -- 172.21.15.92:0/2201083051 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5c2002f070 con 0x7f5c30102b30 2024-05-06T12:25:57.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.826+0000 7f5c26ffd640 1 --2- 172.21.15.92:0/2201083051 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f5c1c06e0b0 0x7f5c1c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:57.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.826+0000 7f5c26ffd640 1 -- 172.21.15.92:0/2201083051 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f5c200afa20 con 0x7f5c30102b30 2024-05-06T12:25:57.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.826+0000 7f5c3600a640 1 --2- 172.21.15.92:0/2201083051 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f5c1c06e0b0 0x7f5c1c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:57.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.826+0000 7f5c3600a640 1 --2- 172.21.15.92:0/2201083051 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f5c1c06e0b0 0x7f5c1c070560 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c007fd0 tx=0x7f5c2c009e90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:57.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:57.830+0000 7f5c26ffd640 1 -- 172.21.15.92:0/2201083051 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5c20078d50 con 0x7f5c30102b30 2024-05-06T12:25:58.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.006+0000 7f5c3826e640 1 -- 172.21.15.92:0/2201083051 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5c301092f0 con 0x7f5c30102b30 2024-05-06T12:25:58.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.007+0000 7f5c26ffd640 1 -- 172.21.15.92:0/2201083051 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5c2007c410 con 0x7f5c30102b30 2024-05-06T12:25:58.008 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2024-05-06T12:25:58.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.008+0000 7f5c3826e640 1 -- 172.21.15.92:0/2201083051 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f5c1c06e0b0 msgr2=0x7f5c1c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:58.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.008+0000 7f5c3826e640 1 --2- 172.21.15.92:0/2201083051 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f5c1c06e0b0 0x7f5c1c070560 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c007fd0 tx=0x7f5c2c009e90 comp rx=0 tx=0).stop 2024-05-06T12:25:58.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.008+0000 7f5c3826e640 1 -- 172.21.15.92:0/2201083051 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c30102b30 msgr2=0x7f5c3006fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:58.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.008+0000 7f5c3826e640 1 --2- 172.21.15.92:0/2201083051 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c30102b30 0x7f5c3006fcb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5c20004930 tx=0x7f5c20009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:58.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.009+0000 7f5c3826e640 1 -- 172.21.15.92:0/2201083051 shutdown_connections 2024-05-06T12:25:58.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.009+0000 7f5c3826e640 1 --2- 172.21.15.92:0/2201083051 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f5c1c06e0b0 0x7f5c1c070560 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:58.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.009+0000 7f5c3826e640 1 --2- 172.21.15.92:0/2201083051 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c30102b30 0x7f5c3006fcb0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:58.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.009+0000 7f5c3826e640 1 --2- 172.21.15.92:0/2201083051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c30102190 0x7f5c3006f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:58.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.009+0000 7f5c3826e640 1 -- 172.21.15.92:0/2201083051 >> 172.21.15.92:0/2201083051 conn(0x7f5c300f7da0 msgr2=0x7f5c300f8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:58.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.009+0000 7f5c3826e640 1 -- 172.21.15.92:0/2201083051 shutdown_connections 2024-05-06T12:25:58.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.009+0000 7f5c3826e640 1 -- 172.21.15.92:0/2201083051 wait complete. 2024-05-06T12:25:58.020 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-05-06T12:25:58.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.115+0000 7f4a62aae640 1 -- 172.21.15.92:0/912805769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a5c101110 msgr2=0x7f4a5c1014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:58.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.115+0000 7f4a62aae640 1 --2- 172.21.15.92:0/912805769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a5c101110 0x7f4a5c1014e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c0048d0 tx=0x7f4a4c009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:58.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.115+0000 7f4a62aae640 1 -- 172.21.15.92:0/912805769 shutdown_connections 2024-05-06T12:25:58.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.115+0000 7f4a62aae640 1 --2- 172.21.15.92:0/912805769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a5c0fb0e0 0x7f4a5c0fb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:58.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.115+0000 7f4a62aae640 1 --2- 172.21.15.92:0/912805769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a5c101110 0x7f4a5c1014e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:58.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.116+0000 7f4a62aae640 1 -- 172.21.15.92:0/912805769 >> 172.21.15.92:0/912805769 conn(0x7f4a5c0710c0 msgr2=0x7f4a5c0714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:58.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.116+0000 7f4a62aae640 1 -- 172.21.15.92:0/912805769 shutdown_connections 2024-05-06T12:25:58.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.116+0000 7f4a62aae640 1 -- 172.21.15.92:0/912805769 wait complete. 2024-05-06T12:25:58.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.116+0000 7f4a62aae640 1 Processor -- start 2024-05-06T12:25:58.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.116+0000 7f4a62aae640 1 -- start start 2024-05-06T12:25:58.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.116+0000 7f4a62aae640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a5c0fb0e0 0x7f4a5c111b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:58.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.116+0000 7f4a62aae640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a5c101110 0x7f4a5c1120a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:58.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.116+0000 7f4a62aae640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c1126a0 con 0x7f4a5c0fb0e0 2024-05-06T12:25:58.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.116+0000 7f4a62aae640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c10bcf0 con 0x7f4a5c101110 2024-05-06T12:25:58.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.116+0000 7f4a6084a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a5c0fb0e0 0x7f4a5c111b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:58.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.116+0000 7f4a6084a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a5c0fb0e0 0x7f4a5c111b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52152/0 (socket says 172.21.15.92:52152) 2024-05-06T12:25:58.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.116+0000 7f4a6084a640 1 -- 172.21.15.92:0/3978536106 learned_addr learned my addr 172.21.15.92:0/3978536106 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:58.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.117+0000 7f4a5bfff640 1 --2- 172.21.15.92:0/3978536106 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a5c101110 0x7f4a5c1120a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:58.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.117+0000 7f4a6084a640 1 -- 172.21.15.92:0/3978536106 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a5c101110 msgr2=0x7f4a5c1120a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:58.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.117+0000 7f4a6084a640 1 --2- 172.21.15.92:0/3978536106 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a5c101110 0x7f4a5c1120a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:58.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.117+0000 7f4a6084a640 1 -- 172.21.15.92:0/3978536106 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4c005040 con 0x7f4a5c0fb0e0 2024-05-06T12:25:58.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.117+0000 7f4a6084a640 1 --2- 172.21.15.92:0/3978536106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a5c0fb0e0 0x7f4a5c111b60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c0041d0 tx=0x7f4a4c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:58.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.117+0000 7f4a597fa640 1 -- 172.21.15.92:0/3978536106 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a4c03e070 con 0x7f4a5c0fb0e0 2024-05-06T12:25:58.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.117+0000 7f4a597fa640 1 -- 172.21.15.92:0/3978536106 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a4c0092e0 con 0x7f4a5c0fb0e0 2024-05-06T12:25:58.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.117+0000 7f4a62aae640 1 -- 172.21.15.92:0/3978536106 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a5c10bf70 con 0x7f4a5c0fb0e0 2024-05-06T12:25:58.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.117+0000 7f4a597fa640 1 -- 172.21.15.92:0/3978536106 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a4c02f5f0 con 0x7f4a5c0fb0e0 2024-05-06T12:25:58.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.117+0000 7f4a62aae640 1 -- 172.21.15.92:0/3978536106 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a5c10c430 con 0x7f4a5c0fb0e0 2024-05-06T12:25:58.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.118+0000 7f4a62aae640 1 -- 172.21.15.92:0/3978536106 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a5c0fc7d0 con 0x7f4a5c0fb0e0 2024-05-06T12:25:58.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.119+0000 7f4a597fa640 1 -- 172.21.15.92:0/3978536106 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4a4c0384c0 con 0x7f4a5c0fb0e0 2024-05-06T12:25:58.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.119+0000 7f4a597fa640 1 --2- 172.21.15.92:0/3978536106 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4a4406e0b0 0x7f4a44070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:58.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.119+0000 7f4a597fa640 1 -- 172.21.15.92:0/3978536106 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4a4c0b1a30 con 0x7f4a5c0fb0e0 2024-05-06T12:25:58.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.119+0000 7f4a5bfff640 1 --2- 172.21.15.92:0/3978536106 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4a4406e0b0 0x7f4a44070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:58.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.119+0000 7f4a5bfff640 1 --2- 172.21.15.92:0/3978536106 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4a4406e0b0 0x7f4a44070560 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c10d420 tx=0x7f4a50009ed0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:58.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.124+0000 7f4a597fa640 1 -- 172.21.15.92:0/3978536106 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4a4c0b3e60 con 0x7f4a5c0fb0e0 2024-05-06T12:25:58.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.298+0000 7f4a62aae640 1 -- 172.21.15.92:0/3978536106 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f4a5c1081d0 con 0x7f4a5c0fb0e0 2024-05-06T12:25:58.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.298+0000 7f4a597fa640 1 -- 172.21.15.92:0/3978536106 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7f4a4c07aea0 con 0x7f4a5c0fb0e0 2024-05-06T12:25:58.300 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2024-05-06T12:25:58.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.300+0000 7f4a62aae640 1 -- 172.21.15.92:0/3978536106 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4a4406e0b0 msgr2=0x7f4a44070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:58.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.300+0000 7f4a62aae640 1 --2- 172.21.15.92:0/3978536106 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4a4406e0b0 0x7f4a44070560 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c10d420 tx=0x7f4a50009ed0 comp rx=0 tx=0).stop 2024-05-06T12:25:58.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.300+0000 7f4a62aae640 1 -- 172.21.15.92:0/3978536106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a5c0fb0e0 msgr2=0x7f4a5c111b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:58.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.300+0000 7f4a62aae640 1 --2- 172.21.15.92:0/3978536106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a5c0fb0e0 0x7f4a5c111b60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c0041d0 tx=0x7f4a4c009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:58.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.300+0000 7f4a62aae640 1 -- 172.21.15.92:0/3978536106 shutdown_connections 2024-05-06T12:25:58.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.300+0000 7f4a62aae640 1 --2- 172.21.15.92:0/3978536106 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f4a4406e0b0 0x7f4a44070560 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:58.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.300+0000 7f4a62aae640 1 --2- 172.21.15.92:0/3978536106 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a5c101110 0x7f4a5c1120a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:58.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.300+0000 7f4a62aae640 1 --2- 172.21.15.92:0/3978536106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a5c0fb0e0 0x7f4a5c111b60 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:58.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.300+0000 7f4a62aae640 1 -- 172.21.15.92:0/3978536106 >> 172.21.15.92:0/3978536106 conn(0x7f4a5c0710c0 msgr2=0x7f4a5c0f9510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:58.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.300+0000 7f4a62aae640 1 -- 172.21.15.92:0/3978536106 shutdown_connections 2024-05-06T12:25:58.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.300+0000 7f4a62aae640 1 -- 172.21.15.92:0/3978536106 wait complete. 2024-05-06T12:25:58.311 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:58 smithi092 ceph-mon[91933]: pgmap v318: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:25:58.311 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:58 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/1215851323' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-06T12:25:58.311 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:58 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/920450210' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-06T12:25:58.311 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:58 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/2846312846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-06T12:25:58.311 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:58 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/2201083051' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-06T12:25:58.312 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-05-06T12:25:58.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:58 smithi190 ceph-mon[79744]: pgmap v318: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:25:58.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:58 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/1215851323' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-06T12:25:58.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:58 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/920450210' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-06T12:25:58.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:58 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/2846312846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-06T12:25:58.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:58 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/2201083051' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-06T12:25:58.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.407+0000 7f62c9c14640 1 -- 172.21.15.92:0/2918021048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62c40fa840 msgr2=0x7f62c40fac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:58.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.407+0000 7f62c9c14640 1 --2- 172.21.15.92:0/2918021048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62c40fa840 0x7f62c40fac90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f62b80048d0 tx=0x7f62b8009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:58.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.407+0000 7f62c9c14640 1 -- 172.21.15.92:0/2918021048 shutdown_connections 2024-05-06T12:25:58.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.407+0000 7f62c9c14640 1 --2- 172.21.15.92:0/2918021048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62c40fa840 0x7f62c40fac90 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:58.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.408+0000 7f62c9c14640 1 --2- 172.21.15.92:0/2918021048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62c4102200 0x7f62c41025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:58.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.408+0000 7f62c9c14640 1 -- 172.21.15.92:0/2918021048 >> 172.21.15.92:0/2918021048 conn(0x7f62c4071700 msgr2=0x7f62c4071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:58.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.408+0000 7f62c9c14640 1 -- 172.21.15.92:0/2918021048 shutdown_connections 2024-05-06T12:25:58.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.408+0000 7f62c9c14640 1 -- 172.21.15.92:0/2918021048 wait complete. 2024-05-06T12:25:58.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.408+0000 7f62c9c14640 1 Processor -- start 2024-05-06T12:25:58.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.408+0000 7f62c9c14640 1 -- start start 2024-05-06T12:25:58.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.408+0000 7f62c9c14640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62c40fa840 0x7f62c4112db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:58.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.408+0000 7f62c9c14640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62c4102200 0x7f62c41132f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:58.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.408+0000 7f62c9c14640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c4113980 con 0x7f62c4102200 2024-05-06T12:25:58.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.408+0000 7f62c9c14640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c410cf40 con 0x7f62c40fa840 2024-05-06T12:25:58.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.408+0000 7f62c2ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62c4102200 0x7f62c41132f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:58.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.409+0000 7f62c2ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62c4102200 0x7f62c41132f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52162/0 (socket says 172.21.15.92:52162) 2024-05-06T12:25:58.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.409+0000 7f62c2ffd640 1 -- 172.21.15.92:0/3939064781 learned_addr learned my addr 172.21.15.92:0/3939064781 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:58.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.409+0000 7f62c37fe640 1 --2- 172.21.15.92:0/3939064781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62c40fa840 0x7f62c4112db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:58.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.409+0000 7f62c2ffd640 1 -- 172.21.15.92:0/3939064781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62c40fa840 msgr2=0x7f62c4112db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:58.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.409+0000 7f62c2ffd640 1 --2- 172.21.15.92:0/3939064781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62c40fa840 0x7f62c4112db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:58.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.409+0000 7f62c2ffd640 1 -- 172.21.15.92:0/3939064781 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b8005040 con 0x7f62c4102200 2024-05-06T12:25:58.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.409+0000 7f62c37fe640 1 --2- 172.21.15.92:0/3939064781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62c40fa840 0x7f62c4112db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-06T12:25:58.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.409+0000 7f62c2ffd640 1 --2- 172.21.15.92:0/3939064781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62c4102200 0x7f62c41132f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f62b8004930 tx=0x7f62b8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:58.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.409+0000 7f62abfff640 1 -- 172.21.15.92:0/3939064781 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62b80094c0 con 0x7f62c4102200 2024-05-06T12:25:58.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.409+0000 7f62c9c14640 1 -- 172.21.15.92:0/3939064781 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62c410d1c0 con 0x7f62c4102200 2024-05-06T12:25:58.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.409+0000 7f62abfff640 1 -- 172.21.15.92:0/3939064781 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62b8032e50 con 0x7f62c4102200 2024-05-06T12:25:58.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.409+0000 7f62abfff640 1 -- 172.21.15.92:0/3939064781 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62b80433b0 con 0x7f62c4102200 2024-05-06T12:25:58.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.409+0000 7f62c9c14640 1 -- 172.21.15.92:0/3939064781 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62c410d680 con 0x7f62c4102200 2024-05-06T12:25:58.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.410+0000 7f62c9c14640 1 -- 172.21.15.92:0/3939064781 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62c40652a0 con 0x7f62c4102200 2024-05-06T12:25:58.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.411+0000 7f62abfff640 1 -- 172.21.15.92:0/3939064781 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f62b802f070 con 0x7f62c4102200 2024-05-06T12:25:58.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.411+0000 7f62abfff640 1 --2- 172.21.15.92:0/3939064781 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f62ac06e0b0 0x7f62ac070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:58.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.411+0000 7f62abfff640 1 -- 172.21.15.92:0/3939064781 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f62b80afad0 con 0x7f62c4102200 2024-05-06T12:25:58.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.411+0000 7f62c37fe640 1 --2- 172.21.15.92:0/3939064781 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f62ac06e0b0 0x7f62ac070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:58.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.412+0000 7f62c37fe640 1 --2- 172.21.15.92:0/3939064781 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f62ac06e0b0 0x7f62ac070560 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f62b4006fd0 tx=0x7f62b4008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:58.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.416+0000 7f62abfff640 1 -- 172.21.15.92:0/3939064781 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f62b8078e00 con 0x7f62c4102200 2024-05-06T12:25:58.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.589+0000 7f62c9c14640 1 -- 172.21.15.92:0/3939064781 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f62c404bcc0 con 0x7f62c4102200 2024-05-06T12:25:58.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.589+0000 7f62abfff640 1 -- 172.21.15.92:0/3939064781 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7f62b807c4c0 con 0x7f62c4102200 2024-05-06T12:25:58.591 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2024-05-06T12:25:58.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.591+0000 7f62c9c14640 1 -- 172.21.15.92:0/3939064781 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f62ac06e0b0 msgr2=0x7f62ac070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:58.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.591+0000 7f62c9c14640 1 --2- 172.21.15.92:0/3939064781 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f62ac06e0b0 0x7f62ac070560 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f62b4006fd0 tx=0x7f62b4008040 comp rx=0 tx=0).stop 2024-05-06T12:25:58.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.591+0000 7f62c9c14640 1 -- 172.21.15.92:0/3939064781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62c4102200 msgr2=0x7f62c41132f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:58.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.591+0000 7f62c9c14640 1 --2- 172.21.15.92:0/3939064781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62c4102200 0x7f62c41132f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f62b8004930 tx=0x7f62b8009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:58.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.591+0000 7f62c9c14640 1 -- 172.21.15.92:0/3939064781 shutdown_connections 2024-05-06T12:25:58.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.591+0000 7f62c9c14640 1 --2- 172.21.15.92:0/3939064781 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f62ac06e0b0 0x7f62ac070560 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:58.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.591+0000 7f62c9c14640 1 --2- 172.21.15.92:0/3939064781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62c4102200 0x7f62c41132f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:58.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.591+0000 7f62c9c14640 1 --2- 172.21.15.92:0/3939064781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62c40fa840 0x7f62c4112db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:58.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.591+0000 7f62c9c14640 1 -- 172.21.15.92:0/3939064781 >> 172.21.15.92:0/3939064781 conn(0x7f62c4071700 msgr2=0x7f62c4104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:58.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.591+0000 7f62c9c14640 1 -- 172.21.15.92:0/3939064781 shutdown_connections 2024-05-06T12:25:58.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.591+0000 7f62c9c14640 1 -- 172.21.15.92:0/3939064781 wait complete. 2024-05-06T12:25:58.603 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-06T12:25:58.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.697+0000 7f122c15d640 1 -- 172.21.15.92:0/201676247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1224102d60 msgr2=0x7f12240fc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:58.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.697+0000 7f122c15d640 1 --2- 172.21.15.92:0/201676247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1224102d60 0x7f12240fc3f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f12180048d0 tx=0x7f1218009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:58.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.697+0000 7f122c15d640 1 -- 172.21.15.92:0/201676247 shutdown_connections 2024-05-06T12:25:58.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.697+0000 7f122c15d640 1 --2- 172.21.15.92:0/201676247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1224102d60 0x7f12240fc3f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:58.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.697+0000 7f122c15d640 1 --2- 172.21.15.92:0/201676247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12241023c0 0x7f1224102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:58.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.697+0000 7f122c15d640 1 -- 172.21.15.92:0/201676247 >> 172.21.15.92:0/201676247 conn(0x7f12240f7f90 msgr2=0x7f12240fa3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:58.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.697+0000 7f122c15d640 1 -- 172.21.15.92:0/201676247 shutdown_connections 2024-05-06T12:25:58.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.697+0000 7f122c15d640 1 -- 172.21.15.92:0/201676247 wait complete. 2024-05-06T12:25:58.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.697+0000 7f122c15d640 1 Processor -- start 2024-05-06T12:25:58.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.697+0000 7f122c15d640 1 -- start start 2024-05-06T12:25:58.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.698+0000 7f122c15d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12241023c0 0x7f1224112f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:58.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.698+0000 7f122c15d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1224102d60 0x7f1224113480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:58.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.698+0000 7f122c15d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1224113b10 con 0x7f1224102d60 2024-05-06T12:25:58.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.698+0000 7f122c15d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122410d0d0 con 0x7f12241023c0 2024-05-06T12:25:58.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.698+0000 7f12296f8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1224102d60 0x7f1224113480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:58.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.698+0000 7f12296f8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1224102d60 0x7f1224113480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52188/0 (socket says 172.21.15.92:52188) 2024-05-06T12:25:58.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.698+0000 7f12296f8640 1 -- 172.21.15.92:0/3898742593 learned_addr learned my addr 172.21.15.92:0/3898742593 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-06T12:25:58.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.698+0000 7f1229ef9640 1 --2- 172.21.15.92:0/3898742593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12241023c0 0x7f1224112f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:58.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.698+0000 7f12296f8640 1 -- 172.21.15.92:0/3898742593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12241023c0 msgr2=0x7f1224112f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:58.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.698+0000 7f12296f8640 1 --2- 172.21.15.92:0/3898742593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12241023c0 0x7f1224112f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:58.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.698+0000 7f12296f8640 1 -- 172.21.15.92:0/3898742593 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1218005040 con 0x7f1224102d60 2024-05-06T12:25:58.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.698+0000 7f1229ef9640 1 --2- 172.21.15.92:0/3898742593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12241023c0 0x7f1224112f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-06T12:25:58.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.698+0000 7f12296f8640 1 --2- 172.21.15.92:0/3898742593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1224102d60 0x7f1224113480 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1218004930 tx=0x7f1218009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:58.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.698+0000 7f12167fc640 1 -- 172.21.15.92:0/3898742593 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12180094c0 con 0x7f1224102d60 2024-05-06T12:25:58.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.698+0000 7f12167fc640 1 -- 172.21.15.92:0/3898742593 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1218032e50 con 0x7f1224102d60 2024-05-06T12:25:58.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.698+0000 7f122c15d640 1 -- 172.21.15.92:0/3898742593 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f122410d350 con 0x7f1224102d60 2024-05-06T12:25:58.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.698+0000 7f12167fc640 1 -- 172.21.15.92:0/3898742593 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12180433b0 con 0x7f1224102d60 2024-05-06T12:25:58.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.698+0000 7f122c15d640 1 -- 172.21.15.92:0/3898742593 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f122410d810 con 0x7f1224102d60 2024-05-06T12:25:58.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.699+0000 7f122c15d640 1 -- 172.21.15.92:0/3898742593 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12240fdb30 con 0x7f1224102d60 2024-05-06T12:25:58.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.700+0000 7f12167fc640 1 -- 172.21.15.92:0/3898742593 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f121802f070 con 0x7f1224102d60 2024-05-06T12:25:58.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.700+0000 7f12167fc640 1 --2- 172.21.15.92:0/3898742593 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f121006e0b0 0x7f1210070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-06T12:25:58.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.700+0000 7f12167fc640 1 -- 172.21.15.92:0/3898742593 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f12180afb20 con 0x7f1224102d60 2024-05-06T12:25:58.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.700+0000 7f1229ef9640 1 --2- 172.21.15.92:0/3898742593 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f121006e0b0 0x7f1210070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-06T12:25:58.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.700+0000 7f1229ef9640 1 --2- 172.21.15.92:0/3898742593 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f121006e0b0 0x7f1210070560 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1220006fd0 tx=0x7f1220008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-06T12:25:58.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.705+0000 7f12167fc640 1 -- 172.21.15.92:0/3898742593 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1218078e50 con 0x7f1224102d60 2024-05-06T12:25:58.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.893+0000 7f122c15d640 1 -- 172.21.15.92:0/3898742593 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f12241095e0 con 0x7f1224102d60 2024-05-06T12:25:58.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.893+0000 7f12167fc640 1 -- 172.21.15.92:0/3898742593 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 39 entries v86) v1 ==== 81+0+2307 (secure 0 0 0) 0x7f121807c510 con 0x7f1224102d60 2024-05-06T12:25:58.895 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6828/1663253598 2024-05-07T12:21:58.517697+0000 2024-05-06T12:25:58.895 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6801/648200129 2024-05-07T12:16:38.777479+0000 2024-05-06T12:25:58.895 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6801/543287293 2024-05-07T12:07:03.802107+0000 2024-05-06T12:25:58.895 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/3693315250 2024-05-07T12:05:56.546947+0000 2024-05-06T12:25:58.895 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/3493498366 2024-05-07T12:14:01.169339+0000 2024-05-06T12:25:58.895 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.190:0/3060689070 2024-05-07T12:15:47.314494+0000 2024-05-06T12:25:58.895 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/2210153116 2024-05-07T12:05:56.546947+0000 2024-05-06T12:25:58.896 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/3417170577 2024-05-07T12:14:01.169339+0000 2024-05-06T12:25:58.896 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/1054764607 2024-05-07T12:16:38.777479+0000 2024-05-06T12:25:58.896 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/856987777 2024-05-07T12:14:01.169339+0000 2024-05-06T12:25:58.896 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/895013058 2024-05-07T12:05:25.954232+0000 2024-05-06T12:25:58.896 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/2440233745 2024-05-07T12:07:03.802107+0000 2024-05-06T12:25:58.896 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6800/543287293 2024-05-07T12:07:03.802107+0000 2024-05-06T12:25:58.896 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.190:6829/3484477933 2024-05-07T12:15:47.314494+0000 2024-05-06T12:25:58.896 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/758782094 2024-05-07T12:05:25.954232+0000 2024-05-06T12:25:58.896 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/1552275072 2024-05-07T12:07:03.802107+0000 2024-05-06T12:25:58.896 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6800/3731057893 2024-05-07T12:05:56.546947+0000 2024-05-06T12:25:58.896 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/1407601857 2024-05-07T12:05:56.546947+0000 2024-05-06T12:25:58.896 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/1345929606 2024-05-07T12:16:38.777479+0000 2024-05-06T12:25:58.896 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6801/3045905943 2024-05-07T12:14:01.169339+0000 2024-05-06T12:25:58.896 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6829/1663253598 2024-05-07T12:21:58.517697+0000 2024-05-06T12:25:58.897 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.190:0/2858276177 2024-05-07T12:15:47.314494+0000 2024-05-06T12:25:58.897 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6827/4255634245 2024-05-07T12:12:00.896508+0000 2024-05-06T12:25:58.897 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/4121052357 2024-05-07T12:14:01.169339+0000 2024-05-06T12:25:58.897 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/997820112 2024-05-07T12:05:25.954232+0000 2024-05-06T12:25:58.897 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6800/3656822751 2024-05-07T12:05:25.954232+0000 2024-05-06T12:25:58.897 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/1532977673 2024-05-07T12:07:03.802107+0000 2024-05-06T12:25:58.897 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6801/3656822751 2024-05-07T12:05:25.954232+0000 2024-05-06T12:25:58.897 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6801/3731057893 2024-05-07T12:05:56.546947+0000 2024-05-06T12:25:58.897 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6826/4255634245 2024-05-07T12:12:00.896508+0000 2024-05-06T12:25:58.897 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.190:6828/3484477933 2024-05-07T12:15:47.314494+0000 2024-05-06T12:25:58.897 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.190:0/2419054582 2024-05-07T12:15:47.314494+0000 2024-05-06T12:25:58.897 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/418878214 2024-05-07T12:16:38.777479+0000 2024-05-06T12:25:58.897 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/2098378816 2024-05-07T12:16:38.777479+0000 2024-05-06T12:25:58.897 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6800/3045905943 2024-05-07T12:14:01.169339+0000 2024-05-06T12:25:58.897 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/2150816131 2024-05-07T12:16:38.777479+0000 2024-05-06T12:25:58.898 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.190:0/2974973416 2024-05-07T12:15:47.314494+0000 2024-05-06T12:25:58.898 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/1862221479 2024-05-07T12:16:38.777479+0000 2024-05-06T12:25:58.898 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6800/648200129 2024-05-07T12:16:38.777479+0000 2024-05-06T12:25:58.899 INFO:teuthology.orchestra.run.smithi092.stderr:listed 39 entries 2024-05-06T12:25:58.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.896+0000 7f122c15d640 1 -- 172.21.15.92:0/3898742593 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f121006e0b0 msgr2=0x7f1210070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:58.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.896+0000 7f122c15d640 1 --2- 172.21.15.92:0/3898742593 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f121006e0b0 0x7f1210070560 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1220006fd0 tx=0x7f1220008040 comp rx=0 tx=0).stop 2024-05-06T12:25:58.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.896+0000 7f122c15d640 1 -- 172.21.15.92:0/3898742593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1224102d60 msgr2=0x7f1224113480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-06T12:25:58.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.896+0000 7f122c15d640 1 --2- 172.21.15.92:0/3898742593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1224102d60 0x7f1224113480 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1218004930 tx=0x7f1218009da0 comp rx=0 tx=0).stop 2024-05-06T12:25:58.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.896+0000 7f122c15d640 1 -- 172.21.15.92:0/3898742593 shutdown_connections 2024-05-06T12:25:58.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.896+0000 7f122c15d640 1 --2- 172.21.15.92:0/3898742593 >> [v2:172.21.15.92:6800/114121430,v1:172.21.15.92:6801/114121430] conn(0x7f121006e0b0 0x7f1210070560 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:58.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.896+0000 7f122c15d640 1 --2- 172.21.15.92:0/3898742593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1224102d60 0x7f1224113480 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:58.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.896+0000 7f122c15d640 1 --2- 172.21.15.92:0/3898742593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12241023c0 0x7f1224112f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-06T12:25:58.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.896+0000 7f122c15d640 1 -- 172.21.15.92:0/3898742593 >> 172.21.15.92:0/3898742593 conn(0x7f12240f7f90 msgr2=0x7f12240f8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-06T12:25:58.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.896+0000 7f122c15d640 1 -- 172.21.15.92:0/3898742593 shutdown_connections 2024-05-06T12:25:58.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-06T12:25:58.896+0000 7f122c15d640 1 -- 172.21.15.92:0/3898742593 wait complete. 2024-05-06T12:25:58.907 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi092.front.sepia.ceph.com... 2024-05-06T12:25:58.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-06T12:25:58.908 DEBUG:teuthology.orchestra.run.smithi092:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-06T12:25:58.931 INFO:teuthology.orchestra.run:waiting for 300 2024-05-06T12:25:59.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:59 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/3978536106' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-06T12:25:59.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:59 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/3939064781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-06T12:25:59.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:25:59 smithi092 ceph-mon[91933]: from='client.? 172.21.15.92:0/3898742593' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-06T12:25:59.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:59 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/3978536106' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-06T12:25:59.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:59 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/3939064781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-06T12:25:59.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:25:59 smithi190 ceph-mon[79744]: from='client.? 172.21.15.92:0/3898742593' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-06T12:26:00.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:00 smithi190 ceph-mon[79744]: pgmap v319: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:26:00.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:00 smithi092 ceph-mon[91933]: pgmap v319: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:26:02.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:02 smithi190 ceph-mon[79744]: pgmap v320: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:26:02.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:02 smithi092 ceph-mon[91933]: pgmap v320: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:26:04.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:04 smithi190 ceph-mon[79744]: pgmap v321: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:26:04.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:04 smithi092 ceph-mon[91933]: pgmap v321: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:26:06.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:06 smithi190 ceph-mon[79744]: pgmap v322: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:26:06.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:06 smithi092 ceph-mon[91933]: pgmap v322: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:26:08.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:08 smithi190 ceph-mon[79744]: pgmap v323: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:26:08.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:08 smithi092 ceph-mon[91933]: pgmap v323: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:26:09.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:26:09.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:26:09.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:26:09.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:26:10.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:10 smithi190 ceph-mon[79744]: pgmap v324: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:26:10.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:10 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:26:10.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:10 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:26:10.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:10 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:26:10.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:10 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:26:10.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:10 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:26:10.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:10 smithi092 ceph-mon[91933]: pgmap v324: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:26:10.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:10 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:26:10.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:10 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:26:10.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:10 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:26:10.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:10 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:26:10.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:10 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:26:12.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:12 smithi190 ceph-mon[79744]: pgmap v325: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:26:12.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:12 smithi092 ceph-mon[91933]: pgmap v325: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:26:14.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:14 smithi190 ceph-mon[79744]: pgmap v326: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:26:14.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:14 smithi092 ceph-mon[91933]: pgmap v326: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:26:16.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:16 smithi190 ceph-mon[79744]: pgmap v327: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:26:16.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:16 smithi092 ceph-mon[91933]: pgmap v327: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-06T12:26:18.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:18 smithi190 ceph-mon[79744]: pgmap v328: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:26:18.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:18 smithi092 ceph-mon[91933]: pgmap v328: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:26:20.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:20 smithi190 ceph-mon[79744]: pgmap v329: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:26:20.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:20 smithi092 ceph-mon[91933]: pgmap v329: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:26:22.378 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:22 smithi190 ceph-mon[79744]: pgmap v330: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:26:22.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:22 smithi092 ceph-mon[91933]: pgmap v330: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:26:24.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:24 smithi092 ceph-mon[91933]: pgmap v331: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:26:24.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:24 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:26:24.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:24 smithi190 ceph-mon[79744]: pgmap v331: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:26:24.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:24 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:26:25.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:25 smithi092 ceph-mon[91933]: pgmap v332: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-06T12:26:25.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:25 smithi190 ceph-mon[79744]: pgmap v332: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-06T12:26:28.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:27 smithi190 ceph-mon[79744]: pgmap v333: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-05-06T12:26:28.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:27 smithi092 ceph-mon[91933]: pgmap v333: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-05-06T12:26:30.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:29 smithi190 ceph-mon[79744]: pgmap v334: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-06T12:26:30.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:29 smithi092 ceph-mon[91933]: pgmap v334: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-06T12:26:32.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:31 smithi190 ceph-mon[79744]: pgmap v335: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-05-06T12:26:32.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:31 smithi092 ceph-mon[91933]: pgmap v335: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-05-06T12:26:34.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:33 smithi190 ceph-mon[79744]: pgmap v336: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-05-06T12:26:34.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:33 smithi092 ceph-mon[91933]: pgmap v336: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-05-06T12:26:36.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:35 smithi190 ceph-mon[79744]: pgmap v337: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-06T12:26:36.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:35 smithi092 ceph-mon[91933]: pgmap v337: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-06T12:26:38.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:38 smithi190 ceph-mon[79744]: pgmap v338: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:26:38.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:38 smithi092 ceph-mon[91933]: pgmap v338: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:26:39.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:39 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:26:39.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:39 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:26:40.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:40 smithi190 ceph-mon[79744]: pgmap v339: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:26:40.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:40 smithi092 ceph-mon[91933]: pgmap v339: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:26:42.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:42 smithi190 ceph-mon[79744]: pgmap v340: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:26:42.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:42 smithi092 ceph-mon[91933]: pgmap v340: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:26:44.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:44 smithi190 ceph-mon[79744]: pgmap v341: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:26:44.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:44 smithi092 ceph-mon[91933]: pgmap v341: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:26:46.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:46 smithi190 ceph-mon[79744]: pgmap v342: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:26:46.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:46 smithi092 ceph-mon[91933]: pgmap v342: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:26:48.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:48 smithi190 ceph-mon[79744]: pgmap v343: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:26:48.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:48 smithi092 ceph-mon[91933]: pgmap v343: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:26:50.274 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:50 smithi092 ceph-mon[91933]: pgmap v344: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:26:50.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:50 smithi190 ceph-mon[79744]: pgmap v344: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:26:52.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:52 smithi190 ceph-mon[79744]: pgmap v345: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:26:52.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:52 smithi092 ceph-mon[91933]: pgmap v345: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:26:54.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:54 smithi190 ceph-mon[79744]: pgmap v346: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:26:54.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:54 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:26:54.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:54 smithi092 ceph-mon[91933]: pgmap v346: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:26:54.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:54 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:26:56.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:56 smithi190 ceph-mon[79744]: pgmap v347: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:26:56.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:56 smithi092 ceph-mon[91933]: pgmap v347: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:26:58.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:26:58 smithi190 ceph-mon[79744]: pgmap v348: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:26:58.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:26:58 smithi092 ceph-mon[91933]: pgmap v348: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:27:00.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:00 smithi190 ceph-mon[79744]: pgmap v349: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:27:00.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:00 smithi092 ceph-mon[91933]: pgmap v349: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:27:02.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:02 smithi190 ceph-mon[79744]: pgmap v350: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:27:02.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:02 smithi092 ceph-mon[91933]: pgmap v350: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:27:04.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:04 smithi190 ceph-mon[79744]: pgmap v351: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:27:04.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:04 smithi092 ceph-mon[91933]: pgmap v351: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:27:06.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:06 smithi190 ceph-mon[79744]: pgmap v352: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:27:06.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:06 smithi092 ceph-mon[91933]: pgmap v352: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:27:08.349 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:08 smithi190 ceph-mon[79744]: pgmap v353: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:27:08.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:08 smithi092 ceph-mon[91933]: pgmap v353: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:27:09.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:27:09.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:27:10.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:10 smithi190 ceph-mon[79744]: pgmap v354: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:27:10.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:10 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:27:10.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:10 smithi092 ceph-mon[91933]: pgmap v354: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:27:10.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:10 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:27:11.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:11 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:27:11.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:11 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:27:11.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:11 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:27:11.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:11 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:27:11.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:11 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:27:11.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:11 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:27:12.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:12 smithi190 ceph-mon[79744]: pgmap v355: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:27:12.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:12 smithi092 ceph-mon[91933]: pgmap v355: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:27:14.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:14 smithi190 ceph-mon[79744]: pgmap v356: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:27:14.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:14 smithi092 ceph-mon[91933]: pgmap v356: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:27:16.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:16 smithi190 ceph-mon[79744]: pgmap v357: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:27:16.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:16 smithi092 ceph-mon[91933]: pgmap v357: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:27:18.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:18 smithi190 ceph-mon[79744]: pgmap v358: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:27:18.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:18 smithi092 ceph-mon[91933]: pgmap v358: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:27:20.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:20 smithi190 ceph-mon[79744]: pgmap v359: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:27:20.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:20 smithi092 ceph-mon[91933]: pgmap v359: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:27:22.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:22 smithi190 ceph-mon[79744]: pgmap v360: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:27:22.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:22 smithi092 ceph-mon[91933]: pgmap v360: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:27:24.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:24 smithi190 ceph-mon[79744]: pgmap v361: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:27:24.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:24 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:27:24.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:24 smithi092 ceph-mon[91933]: pgmap v361: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:27:24.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:24 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:27:26.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:26 smithi190 ceph-mon[79744]: pgmap v362: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:27:26.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:26 smithi092 ceph-mon[91933]: pgmap v362: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:27:28.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:28 smithi190 ceph-mon[79744]: pgmap v363: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:27:28.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:28 smithi092 ceph-mon[91933]: pgmap v363: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:27:30.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:30 smithi190 ceph-mon[79744]: pgmap v364: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:27:30.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:30 smithi092 ceph-mon[91933]: pgmap v364: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:27:32.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:32 smithi190 ceph-mon[79744]: pgmap v365: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:27:32.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:32 smithi092 ceph-mon[91933]: pgmap v365: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:27:34.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:34 smithi092 ceph-mon[91933]: pgmap v366: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:27:34.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:34 smithi190 ceph-mon[79744]: pgmap v366: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:27:36.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:36 smithi092 ceph-mon[91933]: pgmap v367: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:27:36.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:36 smithi190 ceph-mon[79744]: pgmap v367: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:27:38.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:38 smithi092 ceph-mon[91933]: pgmap v368: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:27:38.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:38 smithi190 ceph-mon[79744]: pgmap v368: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:27:39.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:39 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:27:39.498 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:39 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:27:40.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:40 smithi092 ceph-mon[91933]: pgmap v369: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:27:40.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:40 smithi190 ceph-mon[79744]: pgmap v369: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:27:41.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:41 smithi092 ceph-mon[91933]: pgmap v370: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:27:41.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:41 smithi190 ceph-mon[79744]: pgmap v370: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:27:44.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:43 smithi190 ceph-mon[79744]: pgmap v371: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:27:44.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:43 smithi092 ceph-mon[91933]: pgmap v371: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:27:46.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:46 smithi190 ceph-mon[79744]: pgmap v372: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:27:46.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:46 smithi092 ceph-mon[91933]: pgmap v372: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:27:48.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:48 smithi190 ceph-mon[79744]: pgmap v373: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:27:48.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:48 smithi092 ceph-mon[91933]: pgmap v373: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:27:50.273 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:50 smithi092 ceph-mon[91933]: pgmap v374: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:27:50.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:50 smithi190 ceph-mon[79744]: pgmap v374: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:27:52.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:52 smithi190 ceph-mon[79744]: pgmap v375: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:27:52.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:52 smithi092 ceph-mon[91933]: pgmap v375: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:27:54.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:54 smithi190 ceph-mon[79744]: pgmap v376: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-05-06T12:27:54.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:54 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:27:54.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:54 smithi092 ceph-mon[91933]: pgmap v376: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-05-06T12:27:54.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:54 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:27:56.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:56 smithi190 ceph-mon[79744]: pgmap v377: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-05-06T12:27:56.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:56 smithi092 ceph-mon[91933]: pgmap v377: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-05-06T12:27:58.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:27:58 smithi190 ceph-mon[79744]: pgmap v378: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-05-06T12:27:58.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:27:58 smithi092 ceph-mon[91933]: pgmap v378: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-05-06T12:28:00.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:00 smithi190 ceph-mon[79744]: pgmap v379: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-06T12:28:00.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:00 smithi092 ceph-mon[91933]: pgmap v379: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-06T12:28:02.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:02 smithi190 ceph-mon[79744]: pgmap v380: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-05-06T12:28:02.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:02 smithi092 ceph-mon[91933]: pgmap v380: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-05-06T12:28:04.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:04 smithi190 ceph-mon[79744]: pgmap v381: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-05-06T12:28:04.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:04 smithi092 ceph-mon[91933]: pgmap v381: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-05-06T12:28:06.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:06 smithi190 ceph-mon[79744]: pgmap v382: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:28:06.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:06 smithi092 ceph-mon[91933]: pgmap v382: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:28:08.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:08 smithi190 ceph-mon[79744]: pgmap v383: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:28:08.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:08 smithi092 ceph-mon[91933]: pgmap v383: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:28:09.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:28:09.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:28:10.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:10 smithi190 ceph-mon[79744]: pgmap v384: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:28:10.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:10 smithi092 ceph-mon[91933]: pgmap v384: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:28:11.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:11 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:28:11.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:11 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:28:12.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:12 smithi190 ceph-mon[79744]: pgmap v385: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:28:12.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:12 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:28:12.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:12 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:28:12.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:12 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:28:12.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:12 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:28:12.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:12 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:28:12.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:12 smithi092 ceph-mon[91933]: pgmap v385: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:28:12.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:12 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:28:12.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:12 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:28:12.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:12 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:28:12.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:12 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:28:12.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:12 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:28:14.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:14 smithi190 ceph-mon[79744]: pgmap v386: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:28:14.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:14 smithi092 ceph-mon[91933]: pgmap v386: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:28:16.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:16 smithi190 ceph-mon[79744]: pgmap v387: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:28:16.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:16 smithi092 ceph-mon[91933]: pgmap v387: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:28:18.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:18 smithi190 ceph-mon[79744]: pgmap v388: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:28:18.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:18 smithi092 ceph-mon[91933]: pgmap v388: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:28:20.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:20 smithi190 ceph-mon[79744]: pgmap v389: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:28:20.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:20 smithi092 ceph-mon[91933]: pgmap v389: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:28:22.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:22 smithi190 ceph-mon[79744]: pgmap v390: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:28:22.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:22 smithi092 ceph-mon[91933]: pgmap v390: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:28:24.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:24 smithi190 ceph-mon[79744]: pgmap v391: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:28:24.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:24 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:28:24.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:24 smithi092 ceph-mon[91933]: pgmap v391: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:28:24.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:24 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:28:26.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:26 smithi190 ceph-mon[79744]: pgmap v392: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:28:26.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:26 smithi092 ceph-mon[91933]: pgmap v392: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:28:28.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:28 smithi190 ceph-mon[79744]: pgmap v393: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:28:28.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:28 smithi092 ceph-mon[91933]: pgmap v393: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:28:30.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:30 smithi190 ceph-mon[79744]: pgmap v394: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:28:30.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:30 smithi092 ceph-mon[91933]: pgmap v394: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:28:32.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:32 smithi190 ceph-mon[79744]: pgmap v395: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:28:32.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:32 smithi092 ceph-mon[91933]: pgmap v395: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:28:34.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:34 smithi190 ceph-mon[79744]: pgmap v396: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:28:34.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:34 smithi092 ceph-mon[91933]: pgmap v396: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:28:36.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:36 smithi190 ceph-mon[79744]: pgmap v397: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:28:36.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:36 smithi092 ceph-mon[91933]: pgmap v397: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:28:38.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:38 smithi190 ceph-mon[79744]: pgmap v398: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:28:38.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:38 smithi092 ceph-mon[91933]: pgmap v398: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:28:39.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:39 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:28:39.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:39 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:28:40.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:40 smithi092 ceph-mon[91933]: pgmap v399: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:28:40.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:40 smithi190 ceph-mon[79744]: pgmap v399: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:28:42.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:42 smithi092 ceph-mon[91933]: pgmap v400: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:28:42.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:42 smithi190 ceph-mon[79744]: pgmap v400: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:28:44.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:44 smithi092 ceph-mon[91933]: pgmap v401: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:28:44.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:44 smithi190 ceph-mon[79744]: pgmap v401: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:28:46.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:46 smithi092 ceph-mon[91933]: pgmap v402: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:28:46.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:46 smithi190 ceph-mon[79744]: pgmap v402: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:28:48.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:48 smithi092 ceph-mon[91933]: pgmap v403: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:28:48.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:48 smithi190 ceph-mon[79744]: pgmap v403: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:28:50.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:50 smithi092 ceph-mon[91933]: pgmap v404: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:28:50.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:50 smithi190 ceph-mon[79744]: pgmap v404: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:28:52.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:52 smithi092 ceph-mon[91933]: pgmap v405: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:28:52.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:52 smithi190 ceph-mon[79744]: pgmap v405: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:28:54.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:54 smithi092 ceph-mon[91933]: pgmap v406: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:28:54.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:54 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:28:54.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:54 smithi190 ceph-mon[79744]: pgmap v406: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:28:54.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:54 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:28:56.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:56 smithi092 ceph-mon[91933]: pgmap v407: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:28:56.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:56 smithi190 ceph-mon[79744]: pgmap v407: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:28:57.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:28:57 smithi092 ceph-mon[91933]: pgmap v408: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:28:57.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:28:57 smithi190 ceph-mon[79744]: pgmap v408: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:29:00.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:00 smithi190 ceph-mon[79744]: pgmap v409: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:29:00.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:00 smithi092 ceph-mon[91933]: pgmap v409: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:29:02.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:02 smithi190 ceph-mon[79744]: pgmap v410: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:29:02.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:02 smithi092 ceph-mon[91933]: pgmap v410: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:29:04.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:04 smithi190 ceph-mon[79744]: pgmap v411: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:29:04.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:04 smithi092 ceph-mon[91933]: pgmap v411: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:29:06.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:06 smithi190 ceph-mon[79744]: pgmap v412: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:29:06.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:06 smithi092 ceph-mon[91933]: pgmap v412: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:29:08.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:08 smithi190 ceph-mon[79744]: pgmap v413: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:29:08.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:08 smithi092 ceph-mon[91933]: pgmap v413: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:29:09.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:29:09.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:29:10.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:10 smithi190 ceph-mon[79744]: pgmap v414: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:29:10.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:10 smithi092 ceph-mon[91933]: pgmap v414: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:29:12.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:12 smithi190 ceph-mon[79744]: pgmap v415: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:29:12.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:12 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:29:12.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:12 smithi092 ceph-mon[91933]: pgmap v415: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:29:12.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:12 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:29:13.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:13 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:29:13.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:13 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:29:13.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:13 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:29:13.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:13 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:29:13.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:13 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:29:13.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:13 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:29:14.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:14 smithi190 ceph-mon[79744]: pgmap v416: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:29:14.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:14 smithi092 ceph-mon[91933]: pgmap v416: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:29:16.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:16 smithi190 ceph-mon[79744]: pgmap v417: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:29:16.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:16 smithi092 ceph-mon[91933]: pgmap v417: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:29:18.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:18 smithi190 ceph-mon[79744]: pgmap v418: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:29:18.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:18 smithi092 ceph-mon[91933]: pgmap v418: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:29:20.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:20 smithi190 ceph-mon[79744]: pgmap v419: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:29:20.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:20 smithi092 ceph-mon[91933]: pgmap v419: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:29:22.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:22 smithi190 ceph-mon[79744]: pgmap v420: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:29:22.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:22 smithi092 ceph-mon[91933]: pgmap v420: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:29:24.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:24 smithi190 ceph-mon[79744]: pgmap v421: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:29:24.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:24 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:29:24.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:24 smithi092 ceph-mon[91933]: pgmap v421: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:29:24.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:24 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:29:26.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:26 smithi190 ceph-mon[79744]: pgmap v422: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:29:26.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:26 smithi092 ceph-mon[91933]: pgmap v422: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:29:28.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:28 smithi190 ceph-mon[79744]: pgmap v423: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:29:28.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:28 smithi092 ceph-mon[91933]: pgmap v423: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:29:30.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:30 smithi190 ceph-mon[79744]: pgmap v424: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:29:30.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:30 smithi092 ceph-mon[91933]: pgmap v424: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:29:32.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:32 smithi190 ceph-mon[79744]: pgmap v425: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:29:32.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:32 smithi092 ceph-mon[91933]: pgmap v425: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:29:34.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:34 smithi190 ceph-mon[79744]: pgmap v426: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:29:34.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:34 smithi092 ceph-mon[91933]: pgmap v426: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:29:36.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:36 smithi190 ceph-mon[79744]: pgmap v427: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:29:36.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:36 smithi092 ceph-mon[91933]: pgmap v427: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:29:38.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:38 smithi190 ceph-mon[79744]: pgmap v428: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:29:38.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:38 smithi092 ceph-mon[91933]: pgmap v428: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:29:39.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:39 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:29:39.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:39 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:29:40.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:40 smithi190 ceph-mon[79744]: pgmap v429: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:29:40.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:40 smithi092 ceph-mon[91933]: pgmap v429: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:29:42.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:42 smithi190 ceph-mon[79744]: pgmap v430: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:29:42.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:42 smithi092 ceph-mon[91933]: pgmap v430: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:29:44.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:44 smithi190 ceph-mon[79744]: pgmap v431: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:29:44.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:44 smithi092 ceph-mon[91933]: pgmap v431: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:29:46.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:46 smithi092 ceph-mon[91933]: pgmap v432: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:29:46.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:46 smithi190 ceph-mon[79744]: pgmap v432: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:29:48.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:48 smithi092 ceph-mon[91933]: pgmap v433: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:29:48.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:48 smithi190 ceph-mon[79744]: pgmap v433: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:29:50.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:50 smithi092 ceph-mon[91933]: pgmap v434: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:29:50.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:50 smithi190 ceph-mon[79744]: pgmap v434: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:29:52.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:52 smithi092 ceph-mon[91933]: pgmap v435: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:29:52.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:52 smithi190 ceph-mon[79744]: pgmap v435: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:29:54.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:54 smithi092 ceph-mon[91933]: pgmap v436: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:29:54.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:54 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:29:54.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:54 smithi190 ceph-mon[79744]: pgmap v436: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:29:54.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:54 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:29:56.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:56 smithi092 ceph-mon[91933]: pgmap v437: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:29:56.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:56 smithi190 ceph-mon[79744]: pgmap v437: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:29:58.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:29:58 smithi092 ceph-mon[91933]: pgmap v438: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:29:58.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:29:58 smithi190 ceph-mon[79744]: pgmap v438: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:30:00.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:00 smithi092 ceph-mon[91933]: pgmap v439: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:30:00.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:00 smithi092 ceph-mon[91933]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-06T12:30:00.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:00 smithi190 ceph-mon[79744]: pgmap v439: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:30:00.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:00 smithi190 ceph-mon[79744]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-06T12:30:02.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:02 smithi092 ceph-mon[91933]: pgmap v440: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:30:02.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:02 smithi190 ceph-mon[79744]: pgmap v440: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:30:04.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:04 smithi092 ceph-mon[91933]: pgmap v441: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:30:04.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:04 smithi190 ceph-mon[79744]: pgmap v441: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:30:06.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:06 smithi092 ceph-mon[91933]: pgmap v442: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:30:06.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:06 smithi190 ceph-mon[79744]: pgmap v442: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:30:08.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:08 smithi092 ceph-mon[91933]: pgmap v443: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:30:08.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:08 smithi190 ceph-mon[79744]: pgmap v443: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:30:09.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:09 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:30:09.498 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:09 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:30:10.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:10 smithi092 ceph-mon[91933]: pgmap v444: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:30:10.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:10 smithi190 ceph-mon[79744]: pgmap v444: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:30:11.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:11 smithi092 ceph-mon[91933]: pgmap v445: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:30:11.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:11 smithi190 ceph-mon[79744]: pgmap v445: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:30:12.971 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:12 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:30:13.097 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:12 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-06T12:30:14.097 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:13 smithi190 ceph-mon[79744]: pgmap v446: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:30:14.097 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:13 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:30:14.097 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:13 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:30:14.097 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:13 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:30:14.221 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:13 smithi092 ceph-mon[91933]: pgmap v446: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:30:14.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:13 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-06T12:30:14.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:13 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-06T12:30:14.222 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:13 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' 2024-05-06T12:30:16.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:16 smithi190 ceph-mon[79744]: pgmap v447: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:30:16.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:16 smithi092 ceph-mon[91933]: pgmap v447: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:30:18.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:18 smithi190 ceph-mon[79744]: pgmap v448: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:30:18.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:18 smithi092 ceph-mon[91933]: pgmap v448: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:30:20.348 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:20 smithi190 ceph-mon[79744]: pgmap v449: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:30:20.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:20 smithi092 ceph-mon[91933]: pgmap v449: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:30:22.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:22 smithi190 ceph-mon[79744]: pgmap v450: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:30:22.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:22 smithi092 ceph-mon[91933]: pgmap v450: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:30:24.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:24 smithi190 ceph-mon[79744]: pgmap v451: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:30:24.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:24 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:30:24.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:24 smithi092 ceph-mon[91933]: pgmap v451: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:30:24.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:24 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:30:26.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:26 smithi190 ceph-mon[79744]: pgmap v452: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:30:26.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:26 smithi092 ceph-mon[91933]: pgmap v452: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:30:28.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:28 smithi190 ceph-mon[79744]: pgmap v453: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:30:28.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:28 smithi092 ceph-mon[91933]: pgmap v453: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:30:30.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:30 smithi190 ceph-mon[79744]: pgmap v454: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:30:30.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:30 smithi092 ceph-mon[91933]: pgmap v454: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:30:32.359 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:32 smithi190 ceph-mon[79744]: pgmap v455: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:30:32.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:32 smithi092 ceph-mon[91933]: pgmap v455: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:30:34.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:34 smithi190 ceph-mon[79744]: pgmap v456: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:30:34.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:34 smithi092 ceph-mon[91933]: pgmap v456: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:30:36.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:36 smithi190 ceph-mon[79744]: pgmap v457: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:30:36.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:36 smithi092 ceph-mon[91933]: pgmap v457: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:30:38.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:38 smithi190 ceph-mon[79744]: pgmap v458: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:30:38.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:38 smithi092 ceph-mon[91933]: pgmap v458: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:30:39.350 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:39 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:30:39.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:39 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:30:40.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:40 smithi190 ceph-mon[79744]: pgmap v459: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:30:40.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:40 smithi092 ceph-mon[91933]: pgmap v459: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:30:42.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:42 smithi190 ceph-mon[79744]: pgmap v460: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:30:42.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:42 smithi092 ceph-mon[91933]: pgmap v460: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:30:44.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:44 smithi190 ceph-mon[79744]: pgmap v461: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:30:44.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:44 smithi092 ceph-mon[91933]: pgmap v461: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:30:46.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:46 smithi190 ceph-mon[79744]: pgmap v462: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:30:46.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:46 smithi092 ceph-mon[91933]: pgmap v462: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:30:48.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:48 smithi190 ceph-mon[79744]: pgmap v463: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:30:48.474 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:48 smithi092 ceph-mon[91933]: pgmap v463: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:30:50.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:50 smithi190 ceph-mon[79744]: pgmap v464: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:30:50.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:50 smithi092 ceph-mon[91933]: pgmap v464: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-06T12:30:52.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:52 smithi092 ceph-mon[91933]: pgmap v465: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:30:52.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:52 smithi190 ceph-mon[79744]: pgmap v465: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:30:54.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:54 smithi092 ceph-mon[91933]: pgmap v466: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:30:54.472 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:54 smithi092 ceph-mon[91933]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:30:54.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:54 smithi190 ceph-mon[79744]: pgmap v466: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:30:54.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:54 smithi190 ceph-mon[79744]: from='mgr.34102 172.21.15.92:0/472379392' entity='mgr.smithi092.gyzeqb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-06T12:30:56.471 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:56 smithi092 ceph-mon[91933]: pgmap v467: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:30:56.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:56 smithi190 ceph-mon[79744]: pgmap v467: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-06T12:30:58.002 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-05-06T12:30:58.002 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_70ce9539b62a15b9de429c1f0e91a1e012b099aa/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_70ce9539b62a15b9de429c1f0e91a1e012b099aa/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-05-06T12:30:58.004 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-06T12:30:58.016 INFO:tasks.cephadm:Teardown begin 2024-05-06T12:30:58.016 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph_70ce9539b62a15b9de429c1f0e91a1e012b099aa/qa/tasks/cephadm.py", line 2240, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_70ce9539b62a15b9de429c1f0e91a1e012b099aa/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_70ce9539b62a15b9de429c1f0e91a1e012b099aa/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-05-06T12:30:58.017 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-06T12:30:58.050 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-06T12:30:58.086 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-06T12:30:58.086 DEBUG:teuthology.orchestra.run.smithi092:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-06T12:30:58.108 DEBUG:teuthology.orchestra.run.smithi190:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-06T12:30:58.140 INFO:tasks.cephadm:Stopping all daemons... 2024-05-06T12:30:58.140 INFO:tasks.cephadm.mon.smithi092:Stopping mon.smithi092... 2024-05-06T12:30:58.141 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@mon.smithi092 2024-05-06T12:30:58.347 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:58 smithi190 ceph-mon[79744]: pgmap v468: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:30:58.442 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:58 smithi092 ceph-mon[91933]: pgmap v468: 65 pgs: 65 active+clean; 175 MiB data, 773 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-06T12:30:58.443 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:58 smithi092 systemd[1]: Stopping Ceph mon.smithi092 for aee1d912-0ba0-11ef-bc97-c7b262605968... 2024-05-06T12:30:58.721 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:58 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi092[91929]: 2024-05-06T12:30:58.440+0000 7f73db817700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi092 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-06T12:30:58.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:58 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi092[91929]: 2024-05-06T12:30:58.440+0000 7f73db817700 -1 mon.smithi092@0(leader) e3 *** Got Signal Terminated *** 2024-05-06T12:30:58.722 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:58 smithi092 podman[148157]: 2024-05-06 12:30:58.594935632 +0000 UTC m=+0.251486442 container died ac07f7158087a05c9a6a634101382e16d869dfd83d110786151d67cee252b2f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi092, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-70ce953, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True) 2024-05-06T12:30:59.221 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:59 smithi092 podman[148157]: 2024-05-06 12:30:59.085527234 +0000 UTC m=+0.742078090 container cleanup ac07f7158087a05c9a6a634101382e16d869dfd83d110786151d67cee252b2f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi092, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-70ce953, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5) 2024-05-06T12:30:59.221 INFO:journalctl@ceph.mon.smithi092.smithi092.stdout:May 06 12:30:59 smithi092 bash[148157]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi092 2024-05-06T12:30:59.508 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@mon.smithi092.service' 2024-05-06T12:30:59.576 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-06T12:30:59.577 INFO:tasks.cephadm.mon.smithi092:Stopped mon.smithi092 2024-05-06T12:30:59.577 INFO:tasks.cephadm.mon.smithi190:Stopping mon.smithi190... 2024-05-06T12:30:59.577 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@mon.smithi190 2024-05-06T12:30:59.879 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:59 smithi190 systemd[1]: Stopping Ceph mon.smithi190 for aee1d912-0ba0-11ef-bc97-c7b262605968... 2024-05-06T12:31:00.214 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:59 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi190[79740]: 2024-05-06T12:30:59.878+0000 7f2503ca0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi190 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-06T12:31:00.214 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:59 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi190[79740]: 2024-05-06T12:30:59.878+0000 7f2503ca0700 -1 mon.smithi190@1(peon) e3 *** Got Signal Terminated *** 2024-05-06T12:31:00.214 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:30:59 smithi190 podman[117874]: 2024-05-06 12:30:59.919647522 +0000 UTC m=+0.144915490 container died 7a3c08eb716fcaa9b3412d4694100f785857fdb65f0f0c02037f77fa2ad1a43a (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi190, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-70ce953, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5) 2024-05-06T12:31:00.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:31:00 smithi190 podman[117874]: 2024-05-06 12:31:00.437896415 +0000 UTC m=+0.663164375 container cleanup 7a3c08eb716fcaa9b3412d4694100f785857fdb65f0f0c02037f77fa2ad1a43a (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi190, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-70ce953, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-06T12:31:00.597 INFO:journalctl@ceph.mon.smithi190.smithi190.stdout:May 06 12:31:00 smithi190 bash[117874]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-mon-smithi190 2024-05-06T12:31:00.781 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@mon.smithi190.service' 2024-05-06T12:31:00.826 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-06T12:31:00.826 INFO:tasks.cephadm.mon.smithi190:Stopped mon.smithi190 2024-05-06T12:31:00.826 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-06T12:31:00.827 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@osd.0 2024-05-06T12:31:01.166 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:31:00 smithi092 systemd[1]: Stopping Ceph osd.0 for aee1d912-0ba0-11ef-bc97-c7b262605968... 2024-05-06T12:31:01.471 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:31:01 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0[100391]: 2024-05-06T12:31:01.163+0000 7f562c5ce700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-06T12:31:01.471 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:31:01 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0[100391]: 2024-05-06T12:31:01.163+0000 7f562c5ce700 -1 osd.0 86 *** Got signal Terminated *** 2024-05-06T12:31:01.472 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:31:01 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0[100391]: 2024-05-06T12:31:01.163+0000 7f562c5ce700 -1 osd.0 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-06T12:31:06.809 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:31:06 smithi092 podman[148290]: 2024-05-06 12:31:06.505845215 +0000 UTC m=+5.486693354 container died 8c96c75b25f48f162b86e1fee897eff5f25b6f588fef97aeb12e05a1084df8aa (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-70ce953, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-06T12:31:07.150 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:31:06 smithi092 podman[148290]: 2024-05-06 12:31:06.90652065 +0000 UTC m=+5.887368824 container cleanup 8c96c75b25f48f162b86e1fee897eff5f25b6f588fef97aeb12e05a1084df8aa (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-70ce953, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-06T12:31:07.151 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:31:06 smithi092 bash[148290]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0 2024-05-06T12:31:07.471 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:31:07 smithi092 podman[148307]: 2024-05-06 12:31:07.149935375 +0000 UTC m=+0.640243386 container remove 8c96c75b25f48f162b86e1fee897eff5f25b6f588fef97aeb12e05a1084df8aa (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0, org.label-schema.build-date=20240430, RELEASE=squid-70ce953, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-06T12:31:07.754 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:31:07 smithi092 podman[148402]: 2024-05-06 12:31:07.38231312 +0000 UTC m=+0.018400873 image pull e4ce8ad5d0c4b9ea60bc5f6b8b0fdb5efbe59bf7771fb0920fc2200e6ae546b0 quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb 2024-05-06T12:31:07.754 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:31:07 smithi092 podman[148402]: 2024-05-06 12:31:07.517583721 +0000 UTC m=+0.153671456 container create 58d2837cbd220668d68a957c0da2fc731352684366d9f9bee6edb90c9d8b140a (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-70ce953, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2) 2024-05-06T12:31:08.050 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:31:07 smithi092 podman[148402]: 2024-05-06 12:31:07.753526744 +0000 UTC m=+0.389614481 container init 58d2837cbd220668d68a957c0da2fc731352684366d9f9bee6edb90c9d8b140a (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-70ce953, org.label-schema.license=GPLv2) 2024-05-06T12:31:08.050 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:31:07 smithi092 podman[148402]: 2024-05-06 12:31:07.757024696 +0000 UTC m=+0.393112438 container start 58d2837cbd220668d68a957c0da2fc731352684366d9f9bee6edb90c9d8b140a (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-70ce953, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-06T12:31:08.050 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:31:07 smithi092 podman[148402]: 2024-05-06 12:31:07.845096363 +0000 UTC m=+0.481184111 container attach 58d2837cbd220668d68a957c0da2fc731352684366d9f9bee6edb90c9d8b140a (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0-deactivate, RELEASE=squid-70ce953, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5) 2024-05-06T12:31:08.318 INFO:journalctl@ceph.osd.0.smithi092.stdout:May 06 12:31:08 smithi092 podman[148402]: 2024-05-06 12:31:08.049996603 +0000 UTC m=+0.686084338 container died 58d2837cbd220668d68a957c0da2fc731352684366d9f9bee6edb90c9d8b140a (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-70ce953, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-06T12:31:08.694 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@osd.0.service' 2024-05-06T12:31:08.733 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-06T12:31:08.733 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-06T12:31:08.733 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-06T12:31:08.733 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@osd.1 2024-05-06T12:31:08.972 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:31:08 smithi092 systemd[1]: Stopping Ceph osd.1 for aee1d912-0ba0-11ef-bc97-c7b262605968... 2024-05-06T12:31:09.471 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:31:09 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1[106225]: 2024-05-06T12:31:09.102+0000 7f8707e8a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-06T12:31:09.472 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:31:09 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1[106225]: 2024-05-06T12:31:09.102+0000 7f8707e8a700 -1 osd.1 86 *** Got signal Terminated *** 2024-05-06T12:31:09.472 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:31:09 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1[106225]: 2024-05-06T12:31:09.102+0000 7f8707e8a700 -1 osd.1 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-06T12:31:14.888 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:31:14 smithi092 podman[148517]: 2024-05-06 12:31:14.56990679 +0000 UTC m=+5.593773009 container died e8007a510f08eae84f224f0551305393278f755fa1814b5c88fd14c114f32df0 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-70ce953, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-06T12:31:15.221 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:31:14 smithi092 podman[148517]: 2024-05-06 12:31:14.968614416 +0000 UTC m=+5.992480646 container cleanup e8007a510f08eae84f224f0551305393278f755fa1814b5c88fd14c114f32df0 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-70ce953, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-06T12:31:15.221 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:31:14 smithi092 bash[148517]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1 2024-05-06T12:31:15.588 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:31:15 smithi092 podman[148537]: 2024-05-06 12:31:15.228485083 +0000 UTC m=+0.660938124 container remove e8007a510f08eae84f224f0551305393278f755fa1814b5c88fd14c114f32df0 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1, RELEASE=squid-70ce953, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240430, io.buildah.version=1.33.5) 2024-05-06T12:31:15.858 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:31:15 smithi092 podman[148632]: 2024-05-06 12:31:15.489836883 +0000 UTC m=+0.018442337 image pull e4ce8ad5d0c4b9ea60bc5f6b8b0fdb5efbe59bf7771fb0920fc2200e6ae546b0 quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb 2024-05-06T12:31:15.858 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:31:15 smithi092 podman[148632]: 2024-05-06 12:31:15.604302786 +0000 UTC m=+0.132908240 container create fa10bd8c5ca54066f02df21dbd8f5c55ae69ffea93335cfea55998c95f08395d (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-70ce953, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-06T12:31:15.858 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:31:15 smithi092 podman[148632]: 2024-05-06 12:31:15.857317988 +0000 UTC m=+0.385923441 container init fa10bd8c5ca54066f02df21dbd8f5c55ae69ffea93335cfea55998c95f08395d (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-70ce953, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5) 2024-05-06T12:31:16.147 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:31:15 smithi092 podman[148632]: 2024-05-06 12:31:15.860719161 +0000 UTC m=+0.389324615 container start fa10bd8c5ca54066f02df21dbd8f5c55ae69ffea93335cfea55998c95f08395d (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, RELEASE=squid-70ce953, org.label-schema.schema-version=1.0, ceph=True) 2024-05-06T12:31:16.147 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:31:15 smithi092 podman[148632]: 2024-05-06 12:31:15.976534461 +0000 UTC m=+0.505139929 container attach fa10bd8c5ca54066f02df21dbd8f5c55ae69ffea93335cfea55998c95f08395d (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-70ce953, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-06T12:31:16.436 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 06 12:31:16 smithi092 podman[148632]: 2024-05-06 12:31:16.146584294 +0000 UTC m=+0.675189752 container died fa10bd8c5ca54066f02df21dbd8f5c55ae69ffea93335cfea55998c95f08395d (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-70ce953, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240430) 2024-05-06T12:31:16.751 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@osd.1.service' 2024-05-06T12:31:16.789 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-06T12:31:16.789 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-06T12:31:16.789 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-06T12:31:16.789 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@osd.2 2024-05-06T12:31:17.131 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:31:16 smithi092 systemd[1]: Stopping Ceph osd.2 for aee1d912-0ba0-11ef-bc97-c7b262605968... 2024-05-06T12:31:17.472 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:31:17 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2[112023]: 2024-05-06T12:31:17.129+0000 7f674d83b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-06T12:31:17.472 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:31:17 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2[112023]: 2024-05-06T12:31:17.129+0000 7f674d83b700 -1 osd.2 86 *** Got signal Terminated *** 2024-05-06T12:31:17.472 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:31:17 smithi092 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2[112023]: 2024-05-06T12:31:17.129+0000 7f674d83b700 -1 osd.2 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-06T12:31:22.721 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:31:22 smithi092 podman[148738]: 2024-05-06 12:31:22.538690339 +0000 UTC m=+5.527823742 container died 2beca7448789c87dcd787d8ab5b224f68af7a62a5d44e9462ba2defce1ebeefa (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-70ce953, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-06T12:31:23.182 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:31:22 smithi092 podman[148738]: 2024-05-06 12:31:22.921546797 +0000 UTC m=+5.910680247 container cleanup 2beca7448789c87dcd787d8ab5b224f68af7a62a5d44e9462ba2defce1ebeefa (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240430, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-70ce953, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-06T12:31:23.182 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:31:22 smithi092 bash[148738]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2 2024-05-06T12:31:23.471 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:31:23 smithi092 podman[148768]: 2024-05-06 12:31:23.181470396 +0000 UTC m=+0.640765201 container remove 2beca7448789c87dcd787d8ab5b224f68af7a62a5d44e9462ba2defce1ebeefa (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-70ce953, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-06T12:31:23.802 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:31:23 smithi092 podman[148861]: 2024-05-06 12:31:23.437057129 +0000 UTC m=+0.018397268 image pull e4ce8ad5d0c4b9ea60bc5f6b8b0fdb5efbe59bf7771fb0920fc2200e6ae546b0 quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb 2024-05-06T12:31:23.802 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:31:23 smithi092 podman[148861]: 2024-05-06 12:31:23.557228464 +0000 UTC m=+0.138568592 container create 0b12dd4c932206dc0013dbf4a022cbe159e243d2bf30a5f9a4c3e4fd5be5c0e1 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-70ce953, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-06T12:31:23.802 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:31:23 smithi092 podman[148861]: 2024-05-06 12:31:23.80133631 +0000 UTC m=+0.382676439 container init 0b12dd4c932206dc0013dbf4a022cbe159e243d2bf30a5f9a4c3e4fd5be5c0e1 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-70ce953, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-06T12:31:24.101 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:31:23 smithi092 podman[148861]: 2024-05-06 12:31:23.805692626 +0000 UTC m=+0.387032758 container start 0b12dd4c932206dc0013dbf4a022cbe159e243d2bf30a5f9a4c3e4fd5be5c0e1 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-70ce953, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-06T12:31:24.101 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:31:23 smithi092 podman[148861]: 2024-05-06 12:31:23.909702874 +0000 UTC m=+0.491043016 container attach 0b12dd4c932206dc0013dbf4a022cbe159e243d2bf30a5f9a4c3e4fd5be5c0e1 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-70ce953, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-06T12:31:24.471 INFO:journalctl@ceph.osd.2.smithi092.stdout:May 06 12:31:24 smithi092 podman[148861]: 2024-05-06 12:31:24.098275038 +0000 UTC m=+0.679615159 container died 0b12dd4c932206dc0013dbf4a022cbe159e243d2bf30a5f9a4c3e4fd5be5c0e1 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-70ce953) 2024-05-06T12:31:24.887 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@osd.2.service' 2024-05-06T12:31:24.925 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-06T12:31:24.925 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-06T12:31:24.926 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-06T12:31:24.926 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@osd.3 2024-05-06T12:31:25.217 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:31:24 smithi190 systemd[1]: Stopping Ceph osd.3 for aee1d912-0ba0-11ef-bc97-c7b262605968... 2024-05-06T12:31:25.473 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:31:25 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3[89271]: 2024-05-06T12:31:25.215+0000 7f874c50d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-06T12:31:25.474 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:31:25 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3[89271]: 2024-05-06T12:31:25.215+0000 7f874c50d700 -1 osd.3 86 *** Got signal Terminated *** 2024-05-06T12:31:25.474 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:31:25 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3[89271]: 2024-05-06T12:31:25.215+0000 7f874c50d700 -1 osd.3 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-06T12:31:27.347 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:31:27 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3[89271]: 2024-05-06T12:31:27.015+0000 7f8749302700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-05-06T12:31:02.307830+0000 front 2024-05-06T12:31:02.307876+0000 (oldest deadline 2024-05-06T12:31:27.007897+0000) 2024-05-06T12:31:28.097 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:31:27 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3[89271]: 2024-05-06T12:31:27.985+0000 7f8749302700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-05-06T12:31:02.307830+0000 front 2024-05-06T12:31:02.307876+0000 (oldest deadline 2024-05-06T12:31:27.007897+0000) 2024-05-06T12:31:28.097 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:27 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:31:27.820+0000 7f33ac227700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-05-06T12:31:02.879001+0000 front 2024-05-06T12:31:02.878925+0000 (oldest deadline 2024-05-06T12:31:26.879315+0000) 2024-05-06T12:31:29.097 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:31:28 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3[89271]: 2024-05-06T12:31:28.972+0000 7f8749302700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-05-06T12:31:02.307830+0000 front 2024-05-06T12:31:02.307876+0000 (oldest deadline 2024-05-06T12:31:27.007897+0000) 2024-05-06T12:31:29.097 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:28 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:31:28.844+0000 7f33ac227700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-05-06T12:31:02.879001+0000 front 2024-05-06T12:31:02.878925+0000 (oldest deadline 2024-05-06T12:31:26.879315+0000) 2024-05-06T12:31:30.347 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:31:29 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3[89271]: 2024-05-06T12:31:29.933+0000 7f8749302700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-05-06T12:31:02.307830+0000 front 2024-05-06T12:31:02.307876+0000 (oldest deadline 2024-05-06T12:31:27.007897+0000) 2024-05-06T12:31:30.347 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:29 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:31:29.889+0000 7f33ac227700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-05-06T12:31:02.879001+0000 front 2024-05-06T12:31:02.878925+0000 (oldest deadline 2024-05-06T12:31:26.879315+0000) 2024-05-06T12:31:31.001 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:31:30 smithi190 podman[117997]: 2024-05-06 12:31:30.729606033 +0000 UTC m=+5.609042811 container died 914ada199f9336cd9b65640fead36c31aaed835630ba514007feaa0bcffe83e3 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-70ce953, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-05-06T12:31:31.001 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:31:30 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4[94601]: 2024-05-06T12:31:30.977+0000 7fcbc52bf700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-05-06T12:31:05.186985+0000 front 2024-05-06T12:31:05.186835+0000 (oldest deadline 2024-05-06T12:31:30.486568+0000) 2024-05-06T12:31:31.002 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:30 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:31:30.909+0000 7f33ac227700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-05-06T12:31:02.879001+0000 front 2024-05-06T12:31:02.878925+0000 (oldest deadline 2024-05-06T12:31:26.879315+0000) 2024-05-06T12:31:31.347 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:31:31 smithi190 podman[117997]: 2024-05-06 12:31:31.098834806 +0000 UTC m=+5.978271542 container cleanup 914ada199f9336cd9b65640fead36c31aaed835630ba514007feaa0bcffe83e3 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-70ce953, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-06T12:31:31.347 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:31:31 smithi190 bash[117997]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3 2024-05-06T12:31:31.722 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:31:31 smithi190 podman[118026]: 2024-05-06 12:31:31.375351785 +0000 UTC m=+0.639220794 container remove 914ada199f9336cd9b65640fead36c31aaed835630ba514007feaa0bcffe83e3 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-70ce953, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430) 2024-05-06T12:31:32.013 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:31:31 smithi190 podman[118118]: 2024-05-06 12:31:31.625073332 +0000 UTC m=+0.017934833 image pull e4ce8ad5d0c4b9ea60bc5f6b8b0fdb5efbe59bf7771fb0920fc2200e6ae546b0 quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb 2024-05-06T12:31:32.013 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:31:31 smithi190 podman[118118]: 2024-05-06 12:31:31.759791069 +0000 UTC m=+0.152652596 container create 94a84deea7e3d7e14895c1d9601cdb98036aab0ba592d324224fc3e5668efec7 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240430, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-70ce953, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-06T12:31:32.014 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:31:32 smithi190 podman[118118]: 2024-05-06 12:31:32.012717987 +0000 UTC m=+0.405579477 container init 94a84deea7e3d7e14895c1d9601cdb98036aab0ba592d324224fc3e5668efec7 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-70ce953) 2024-05-06T12:31:32.014 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:31 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:31:31.896+0000 7f33ac227700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-05-06T12:31:02.879001+0000 front 2024-05-06T12:31:02.878925+0000 (oldest deadline 2024-05-06T12:31:26.879315+0000) 2024-05-06T12:31:32.309 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:31:32 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4[94601]: 2024-05-06T12:31:32.016+0000 7fcbc52bf700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-05-06T12:31:05.186985+0000 front 2024-05-06T12:31:05.186835+0000 (oldest deadline 2024-05-06T12:31:30.486568+0000) 2024-05-06T12:31:32.310 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:31:32 smithi190 podman[118118]: 2024-05-06 12:31:32.016701455 +0000 UTC m=+0.409562940 container start 94a84deea7e3d7e14895c1d9601cdb98036aab0ba592d324224fc3e5668efec7 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3-deactivate, RELEASE=squid-70ce953, ceph=True, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-06T12:31:32.310 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:31:32 smithi190 podman[118118]: 2024-05-06 12:31:32.120610302 +0000 UTC m=+0.513471806 container attach 94a84deea7e3d7e14895c1d9601cdb98036aab0ba592d324224fc3e5668efec7 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-70ce953, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-05-06T12:31:32.597 INFO:journalctl@ceph.osd.3.smithi190.stdout:May 06 12:31:32 smithi190 podman[118118]: 2024-05-06 12:31:32.309091304 +0000 UTC m=+0.701952787 container died 94a84deea7e3d7e14895c1d9601cdb98036aab0ba592d324224fc3e5668efec7 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-3-deactivate, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, RELEASE=squid-70ce953, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-06T12:31:32.966 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:32 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:31:32.902+0000 7f33ac227700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-05-06T12:31:02.879001+0000 front 2024-05-06T12:31:02.878925+0000 (oldest deadline 2024-05-06T12:31:26.879315+0000) 2024-05-06T12:31:32.988 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@osd.3.service' 2024-05-06T12:31:33.026 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-06T12:31:33.026 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-06T12:31:33.026 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-06T12:31:33.026 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@osd.4 2024-05-06T12:31:33.346 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:31:33 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4[94601]: 2024-05-06T12:31:33.017+0000 7fcbc52bf700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-05-06T12:31:05.186985+0000 front 2024-05-06T12:31:05.186835+0000 (oldest deadline 2024-05-06T12:31:30.486568+0000) 2024-05-06T12:31:33.346 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:31:33 smithi190 systemd[1]: Stopping Ceph osd.4 for aee1d912-0ba0-11ef-bc97-c7b262605968... 2024-05-06T12:31:33.346 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:31:33 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4[94601]: 2024-05-06T12:31:33.344+0000 7fcbc84ca700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-06T12:31:33.346 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:31:33 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4[94601]: 2024-05-06T12:31:33.344+0000 7fcbc84ca700 -1 osd.4 86 *** Got signal Terminated *** 2024-05-06T12:31:33.346 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:31:33 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4[94601]: 2024-05-06T12:31:33.344+0000 7fcbc84ca700 -1 osd.4 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-06T12:31:34.347 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:31:33 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4[94601]: 2024-05-06T12:31:33.983+0000 7fcbc52bf700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-05-06T12:31:05.186985+0000 front 2024-05-06T12:31:05.186835+0000 (oldest deadline 2024-05-06T12:31:30.486568+0000) 2024-05-06T12:31:34.347 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:33 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:31:33.862+0000 7f33ac227700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-05-06T12:31:02.879001+0000 front 2024-05-06T12:31:02.878925+0000 (oldest deadline 2024-05-06T12:31:26.879315+0000) 2024-05-06T12:31:35.097 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:31:34 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4[94601]: 2024-05-06T12:31:34.970+0000 7fcbc52bf700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-05-06T12:31:05.186985+0000 front 2024-05-06T12:31:05.186835+0000 (oldest deadline 2024-05-06T12:31:30.486568+0000) 2024-05-06T12:31:35.097 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:34 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:31:34.832+0000 7f33ac227700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-05-06T12:31:02.879001+0000 front 2024-05-06T12:31:02.878925+0000 (oldest deadline 2024-05-06T12:31:26.879315+0000) 2024-05-06T12:31:36.347 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:31:35 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4[94601]: 2024-05-06T12:31:35.984+0000 7fcbc52bf700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-05-06T12:31:05.186985+0000 front 2024-05-06T12:31:05.186835+0000 (oldest deadline 2024-05-06T12:31:30.486568+0000) 2024-05-06T12:31:36.347 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:35 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:31:35.857+0000 7f33ac227700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-05-06T12:31:02.879001+0000 front 2024-05-06T12:31:02.878925+0000 (oldest deadline 2024-05-06T12:31:26.879315+0000) 2024-05-06T12:31:37.347 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:31:37 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4[94601]: 2024-05-06T12:31:37.004+0000 7fcbc52bf700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-05-06T12:31:05.186985+0000 front 2024-05-06T12:31:05.186835+0000 (oldest deadline 2024-05-06T12:31:30.486568+0000) 2024-05-06T12:31:37.347 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:36 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:31:36.848+0000 7f33ac227700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-05-06T12:31:02.879001+0000 front 2024-05-06T12:31:02.878925+0000 (oldest deadline 2024-05-06T12:31:26.879315+0000) 2024-05-06T12:31:38.097 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:31:38 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4[94601]: 2024-05-06T12:31:38.024+0000 7fcbc52bf700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-05-06T12:31:05.186985+0000 front 2024-05-06T12:31:05.186835+0000 (oldest deadline 2024-05-06T12:31:30.486568+0000) 2024-05-06T12:31:38.097 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:37 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:31:37.836+0000 7f33ac227700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-05-06T12:31:02.879001+0000 front 2024-05-06T12:31:02.878925+0000 (oldest deadline 2024-05-06T12:31:26.879315+0000) 2024-05-06T12:31:39.097 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:31:38 smithi190 podman[118225]: 2024-05-06 12:31:38.851548025 +0000 UTC m=+5.610031447 container died 371c0be776c7d2e21c34bc6bb7b3703d88bbecca2bae00c5b3b8a0c5faa8d5d1 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4, RELEASE=squid-70ce953, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5) 2024-05-06T12:31:39.097 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:38 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:31:38.790+0000 7f33ac227700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-05-06T12:31:02.879001+0000 front 2024-05-06T12:31:02.878925+0000 (oldest deadline 2024-05-06T12:31:26.879315+0000) 2024-05-06T12:31:39.471 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:31:39 smithi190 podman[118225]: 2024-05-06 12:31:39.210997129 +0000 UTC m=+5.969480556 container cleanup 371c0be776c7d2e21c34bc6bb7b3703d88bbecca2bae00c5b3b8a0c5faa8d5d1 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-70ce953, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-06T12:31:39.471 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:31:39 smithi190 bash[118225]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4 2024-05-06T12:31:39.771 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:31:39 smithi190 podman[118244]: 2024-05-06 12:31:39.470356941 +0000 UTC m=+0.615846841 container remove 371c0be776c7d2e21c34bc6bb7b3703d88bbecca2bae00c5b3b8a0c5faa8d5d1 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4, RELEASE=squid-70ce953, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-06T12:31:40.082 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:31:39 smithi190 podman[118337]: 2024-05-06 12:31:39.770964233 +0000 UTC m=+0.114331273 container create 4d34f3deee3d55d9819ec4b616bdf6cb2acdabe4372955affafe164c3b6be9ab (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4-deactivate, RELEASE=squid-70ce953, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-06T12:31:40.083 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:31:39 smithi190 podman[118337]: 2024-05-06 12:31:39.674423913 +0000 UTC m=+0.017790960 image pull e4ce8ad5d0c4b9ea60bc5f6b8b0fdb5efbe59bf7771fb0920fc2200e6ae546b0 quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb 2024-05-06T12:31:40.083 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:31:39 smithi190 podman[118337]: 2024-05-06 12:31:39.998570927 +0000 UTC m=+0.341937966 container init 4d34f3deee3d55d9819ec4b616bdf6cb2acdabe4372955affafe164c3b6be9ab (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4-deactivate, RELEASE=squid-70ce953, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-06T12:31:40.083 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:31:40 smithi190 podman[118337]: 2024-05-06 12:31:40.001803651 +0000 UTC m=+0.345170693 container start 4d34f3deee3d55d9819ec4b616bdf6cb2acdabe4372955affafe164c3b6be9ab (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, RELEASE=squid-70ce953, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD) 2024-05-06T12:31:40.083 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:31:40 smithi190 podman[118337]: 2024-05-06 12:31:40.081802533 +0000 UTC m=+0.425169593 container attach 4d34f3deee3d55d9819ec4b616bdf6cb2acdabe4372955affafe164c3b6be9ab (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-70ce953) 2024-05-06T12:31:40.083 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:39 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:31:39.797+0000 7f33ac227700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-05-06T12:31:02.879001+0000 front 2024-05-06T12:31:02.878925+0000 (oldest deadline 2024-05-06T12:31:26.879315+0000) 2024-05-06T12:31:40.083 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:39 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:31:39.797+0000 7f33ac227700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-05-06T12:31:13.781427+0000 front 2024-05-06T12:31:13.781483+0000 (oldest deadline 2024-05-06T12:31:39.681448+0000) 2024-05-06T12:31:40.347 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:31:40 smithi190 podman[118337]: 2024-05-06 12:31:40.289489318 +0000 UTC m=+0.632856360 container died 4d34f3deee3d55d9819ec4b616bdf6cb2acdabe4372955affafe164c3b6be9ab (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-70ce953, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-06T12:31:40.901 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@osd.4.service' 2024-05-06T12:31:40.924 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:40 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:31:40.756+0000 7f33ac227700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-05-06T12:31:02.879001+0000 front 2024-05-06T12:31:02.878925+0000 (oldest deadline 2024-05-06T12:31:26.879315+0000) 2024-05-06T12:31:40.925 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:40 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:31:40.756+0000 7f33ac227700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-05-06T12:31:13.781427+0000 front 2024-05-06T12:31:13.781483+0000 (oldest deadline 2024-05-06T12:31:39.681448+0000) 2024-05-06T12:31:40.925 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:31:40 smithi190 podman[118362]: 2024-05-06 12:31:40.868932207 +0000 UTC m=+0.574066428 container remove 4d34f3deee3d55d9819ec4b616bdf6cb2acdabe4372955affafe164c3b6be9ab (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-70ce953, org.label-schema.license=GPLv2) 2024-05-06T12:31:40.925 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:31:40 smithi190 systemd[1]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@osd.4.service: Deactivated successfully. 2024-05-06T12:31:40.925 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:31:40 smithi190 systemd[1]: Stopped Ceph osd.4 for aee1d912-0ba0-11ef-bc97-c7b262605968. 2024-05-06T12:31:40.925 INFO:journalctl@ceph.osd.4.smithi190.stdout:May 06 12:31:40 smithi190 systemd[1]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@osd.4.service: Consumed 7.078s CPU time. 2024-05-06T12:31:40.940 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-06T12:31:40.940 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-06T12:31:40.940 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-06T12:31:40.940 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@osd.5 2024-05-06T12:31:41.264 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:41 smithi190 systemd[1]: Stopping Ceph osd.5 for aee1d912-0ba0-11ef-bc97-c7b262605968... 2024-05-06T12:31:41.597 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:41 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:31:41.262+0000 7f33af432700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-06T12:31:41.597 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:41 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:31:41.262+0000 7f33af432700 -1 osd.5 86 *** Got signal Terminated *** 2024-05-06T12:31:41.597 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:41 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:31:41.262+0000 7f33af432700 -1 osd.5 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-06T12:31:42.061 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:41 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:31:41.747+0000 7f33ac227700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-05-06T12:31:02.879001+0000 front 2024-05-06T12:31:02.878925+0000 (oldest deadline 2024-05-06T12:31:26.879315+0000) 2024-05-06T12:31:42.061 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:41 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:31:41.747+0000 7f33ac227700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-05-06T12:31:13.781427+0000 front 2024-05-06T12:31:13.781483+0000 (oldest deadline 2024-05-06T12:31:39.681448+0000) 2024-05-06T12:31:43.097 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:42 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:31:42.791+0000 7f33ac227700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-05-06T12:31:02.879001+0000 front 2024-05-06T12:31:02.878925+0000 (oldest deadline 2024-05-06T12:31:26.879315+0000) 2024-05-06T12:31:43.097 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:42 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:31:42.791+0000 7f33ac227700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-05-06T12:31:13.781427+0000 front 2024-05-06T12:31:13.781483+0000 (oldest deadline 2024-05-06T12:31:39.681448+0000) 2024-05-06T12:31:44.097 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:43 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:31:43.821+0000 7f33ac227700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-05-06T12:31:02.879001+0000 front 2024-05-06T12:31:02.878925+0000 (oldest deadline 2024-05-06T12:31:26.879315+0000) 2024-05-06T12:31:44.097 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:43 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:31:43.821+0000 7f33ac227700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-05-06T12:31:13.781427+0000 front 2024-05-06T12:31:13.781483+0000 (oldest deadline 2024-05-06T12:31:39.681448+0000) 2024-05-06T12:31:44.097 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:43 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:31:43.821+0000 7f33ac227700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-05-06T12:31:21.982251+0000 front 2024-05-06T12:31:21.982285+0000 (oldest deadline 2024-05-06T12:31:43.682271+0000) 2024-05-06T12:31:45.097 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:44 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:31:44.807+0000 7f33ac227700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-05-06T12:31:02.879001+0000 front 2024-05-06T12:31:02.878925+0000 (oldest deadline 2024-05-06T12:31:26.879315+0000) 2024-05-06T12:31:45.097 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:44 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:31:44.807+0000 7f33ac227700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-05-06T12:31:13.781427+0000 front 2024-05-06T12:31:13.781483+0000 (oldest deadline 2024-05-06T12:31:39.681448+0000) 2024-05-06T12:31:45.097 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:44 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:31:44.807+0000 7f33ac227700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-05-06T12:31:21.982251+0000 front 2024-05-06T12:31:21.982285+0000 (oldest deadline 2024-05-06T12:31:43.682271+0000) 2024-05-06T12:31:46.097 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:45 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:31:45.784+0000 7f33ac227700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-05-06T12:31:02.879001+0000 front 2024-05-06T12:31:02.878925+0000 (oldest deadline 2024-05-06T12:31:26.879315+0000) 2024-05-06T12:31:46.097 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:45 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:31:45.784+0000 7f33ac227700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-05-06T12:31:13.781427+0000 front 2024-05-06T12:31:13.781483+0000 (oldest deadline 2024-05-06T12:31:39.681448+0000) 2024-05-06T12:31:46.097 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:45 smithi190 ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5[99923]: 2024-05-06T12:31:45.784+0000 7f33ac227700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-05-06T12:31:21.982251+0000 front 2024-05-06T12:31:21.982285+0000 (oldest deadline 2024-05-06T12:31:43.682271+0000) 2024-05-06T12:31:46.906 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:46 smithi190 podman[118441]: 2024-05-06 12:31:46.653406342 +0000 UTC m=+5.496739969 container died 067bc34cdfb18f8ad7b176d1c3cf140c1b22c4f15aff4f6de93ac05bf9fa621b (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-70ce953, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-06T12:31:47.281 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:47 smithi190 podman[118441]: 2024-05-06 12:31:47.004606075 +0000 UTC m=+5.847939711 container cleanup 067bc34cdfb18f8ad7b176d1c3cf140c1b22c4f15aff4f6de93ac05bf9fa621b (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-70ce953, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-05-06T12:31:47.281 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:47 smithi190 bash[118441]: ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5 2024-05-06T12:31:47.597 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:47 smithi190 podman[118458]: 2024-05-06 12:31:47.280894256 +0000 UTC m=+0.626274203 container remove 067bc34cdfb18f8ad7b176d1c3cf140c1b22c4f15aff4f6de93ac05bf9fa621b (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5, ceph=True, RELEASE=squid-70ce953, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-06T12:31:47.909 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:47 smithi190 podman[118548]: 2024-05-06 12:31:47.530534137 +0000 UTC m=+0.017788760 image pull e4ce8ad5d0c4b9ea60bc5f6b8b0fdb5efbe59bf7771fb0920fc2200e6ae546b0 quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb 2024-05-06T12:31:47.909 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:47 smithi190 podman[118548]: 2024-05-06 12:31:47.631774645 +0000 UTC m=+0.119029250 container create 6c877b6a805cea4b351529252053da1bf04151873110d95d93aeb900531892a8 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-70ce953, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-06T12:31:47.910 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:47 smithi190 podman[118548]: 2024-05-06 12:31:47.843161846 +0000 UTC m=+0.330416461 container init 6c877b6a805cea4b351529252053da1bf04151873110d95d93aeb900531892a8 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-70ce953) 2024-05-06T12:31:47.910 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:47 smithi190 podman[118548]: 2024-05-06 12:31:47.846556908 +0000 UTC m=+0.333811520 container start 6c877b6a805cea4b351529252053da1bf04151873110d95d93aeb900531892a8 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5-deactivate, RELEASE=squid-70ce953, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-06T12:31:47.910 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:47 smithi190 podman[118548]: 2024-05-06 12:31:47.909085562 +0000 UTC m=+0.396340179 container attach 6c877b6a805cea4b351529252053da1bf04151873110d95d93aeb900531892a8 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-70ce953, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-06T12:31:48.347 INFO:journalctl@ceph.osd.5.smithi190.stdout:May 06 12:31:48 smithi190 podman[118548]: 2024-05-06 12:31:48.144123471 +0000 UTC m=+0.631378068 container died 6c877b6a805cea4b351529252053da1bf04151873110d95d93aeb900531892a8 (image=quay.ceph.io/ceph-ci/ceph@sha256:0150ca77ab7330820ade0517ccbade81a6c60dbd326dfc0e8f481c3af7950ccb, name=ceph-aee1d912-0ba0-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-70ce953) 2024-05-06T12:31:48.773 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-aee1d912-0ba0-11ef-bc97-c7b262605968@osd.5.service' 2024-05-06T12:31:48.813 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-06T12:31:48.813 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-06T12:31:48.813 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-06T12:31:51.991 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi092.stderr:ceph-fuse[74605]: fuse finished with error 0 and tester_r 0 2024-05-06T12:32:12.043 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-06T12:32:26.816 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-06T12:32:26.848 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-06T12:32:26.875 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-06T12:32:26.877 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-05-03_21:08:03-orch-squid-distro-default-smithi/7688103/remote/smithi092/crash 2024-05-06T12:32:26.878 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/crash -- . 2024-05-06T12:32:26.916 INFO:teuthology.orchestra.run.smithi092.stderr:tar: /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-06T12:32:26.916 INFO:teuthology.orchestra.run.smithi092.stderr:tar: Error is not recoverable: exiting now 2024-05-06T12:32:26.918 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-05-03_21:08:03-orch-squid-distro-default-smithi/7688103/remote/smithi190/crash 2024-05-06T12:32:26.919 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/crash -- . 2024-05-06T12:32:26.942 INFO:teuthology.orchestra.run.smithi190.stderr:tar: /var/lib/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-06T12:32:26.942 INFO:teuthology.orchestra.run.smithi190.stderr:tar: Error is not recoverable: exiting now 2024-05-06T12:32:26.943 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-06T12:32:26.943 DEBUG:teuthology.orchestra.run.smithi092:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-05-06T12:32:26.994 INFO:tasks.cephadm:Compressing logs... 2024-05-06T12:32:26.994 DEBUG:teuthology.orchestra.run.smithi092:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-06T12:32:27.037 DEBUG:teuthology.orchestra.run.smithi190:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-06T12:32:27.061 INFO:teuthology.orchestra.run.smithi190.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log 2024-05-06T12:32:27.061 INFO:teuthology.orchestra.run.smithi190.stderr:: No such file or directory 2024-05-06T12:32:27.061 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-mds.cephfs.smithi190.idzoff.log 2024-05-06T12:32:27.061 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph.log 2024-05-06T12:32:27.061 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.1.log: /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-mds.cephfs.smithi190.idzoff.log: gzip -5 --verbose -- /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-mds.cephfs.smithi190.byjvgf.log 2024-05-06T12:32:27.061 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-06T12:32:27.061 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-mds.cephfs.smithi190.byjvgf.log: gzip -5 --verbose -- /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-mon.smithi190.log 2024-05-06T12:32:27.070 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-osd.5.log 2024-05-06T12:32:27.071 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-mon.smithi190.log: gzip -5 --verbose -- /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-osd.3.log 2024-05-06T12:32:27.071 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-osd.4.log 2024-05-06T12:32:27.071 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-osd.3.log: 88.0% -- replaced with /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-06T12:32:27.071 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-06T12:32:27.071 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-osd.4.log: 91.3% -- replaced with /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-06T12:32:27.071 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-mgr.smithi190.kczxkh.log 2024-05-06T12:32:27.071 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph.cephadm.log: 85.1% -- replaced with /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-06T12:32:27.071 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-06T12:32:27.072 INFO:teuthology.orchestra.run.smithi092.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-05-06T12:32:27.072 INFO:teuthology.orchestra.run.smithi092.stderr:: No such file or directory 2024-05-06T12:32:27.072 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-mgr.smithi092.gyzeqb.log 2024-05-06T12:32:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph.log 2024-05-06T12:32:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-osd.2.log 2024-05-06T12:32:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-mgr.smithi092.gyzeqb.log: /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-06T12:32:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-mon.smithi092.log 2024-05-06T12:32:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-osd.2.log: /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-osd.1.log 2024-05-06T12:32:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-mon.smithi092.log: gzip -5 --verbose -- /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-osd.0.log 2024-05-06T12:32:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-mds.cephfs.smithi092.rosbll.log 2024-05-06T12:32:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-osd.0.log: 87.9% -- replaced with /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-06T12:32:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-06T12:32:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-mds.cephfs.smithi092.rosbll.log: 91.1% -- replaced with /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-06T12:32:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-mds.cephfs.smithi092.ijsqgp.log 2024-05-06T12:32:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph.cephadm.log: 85.1% -- replaced with /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-06T12:32:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-06T12:32:27.084 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-mds.cephfs.smithi092.ijsqgp.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-06T12:32:27.088 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-mgr.smithi190.kczxkh.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-06T12:32:27.112 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-volume.log: /var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-06T12:32:27.123 INFO:teuthology.orchestra.run.smithi190.stderr: 91.1% -- replaced with /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-06T12:32:27.134 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-volume.log: /var/log/ceph/cephadm.log: 91.1% -- replaced with /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-06T12:32:27.139 INFO:teuthology.orchestra.run.smithi190.stderr: 89.9% -- replaced with /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-mgr.smithi190.kczxkh.log.gz 2024-05-06T12:32:27.163 INFO:teuthology.orchestra.run.smithi092.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-06T12:32:27.463 INFO:teuthology.orchestra.run.smithi092.stderr: 88.7% -- replaced with /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-mgr.smithi092.gyzeqb.log.gz 2024-05-06T12:32:27.504 INFO:teuthology.orchestra.run.smithi190.stderr: 92.5% -- replaced with /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-mon.smithi190.log.gz 2024-05-06T12:32:28.276 INFO:teuthology.orchestra.run.smithi092.stderr: 90.5% -- replaced with /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-mon.smithi092.log.gz 2024-05-06T12:32:30.501 INFO:teuthology.orchestra.run.smithi190.stderr: 95.1% -- replaced with /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-mds.cephfs.smithi190.idzoff.log.gz 2024-05-06T12:32:31.460 INFO:teuthology.orchestra.run.smithi092.stderr: 93.5% -- replaced with /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-osd.1.log.gz 2024-05-06T12:32:31.658 INFO:teuthology.orchestra.run.smithi190.stderr: 93.5% -- replaced with /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-osd.4.log.gz 2024-05-06T12:32:31.804 INFO:teuthology.orchestra.run.smithi092.stderr: 93.5% -- replaced with /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-osd.2.log.gz 2024-05-06T12:32:32.704 INFO:teuthology.orchestra.run.smithi092.stderr: 93.5% -- replaced with /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-osd.0.log.gz 2024-05-06T12:32:33.019 INFO:teuthology.orchestra.run.smithi190.stderr: 93.6% -- replaced with /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-osd.5.log.gz 2024-05-06T12:32:33.196 INFO:teuthology.orchestra.run.smithi190.stderr: 93.6% -- replaced with /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-osd.3.log.gz 2024-05-06T12:32:35.276 INFO:teuthology.orchestra.run.smithi092.stderr: 95.1% -- replaced with /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-mds.cephfs.smithi092.ijsqgp.log.gz 2024-05-06T12:32:41.427 INFO:teuthology.orchestra.run.smithi190.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-05-06T12:32:41.428 INFO:teuthology.orchestra.run.smithi190.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-05-06T12:32:42.899 INFO:teuthology.orchestra.run.smithi092.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-05-06T12:33:13.636 INFO:teuthology.orchestra.run.smithi190.stderr: 93.4% -- replaced with /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-mds.cephfs.smithi190.byjvgf.log.gz 2024-05-06T12:33:13.637 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-05-06T12:33:13.637 INFO:teuthology.orchestra.run.smithi190.stderr:real 0m46.586s 2024-05-06T12:33:13.637 INFO:teuthology.orchestra.run.smithi190.stderr:user 1m19.032s 2024-05-06T12:33:13.637 INFO:teuthology.orchestra.run.smithi190.stderr:sys 0m2.422s 2024-05-06T12:33:14.957 INFO:teuthology.orchestra.run.smithi092.stderr: 93.3% -- replaced with /var/log/ceph/aee1d912-0ba0-11ef-bc97-c7b262605968/ceph-mds.cephfs.smithi092.rosbll.log.gz 2024-05-06T12:33:14.959 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-05-06T12:33:14.959 INFO:teuthology.orchestra.run.smithi092.stderr:real 0m47.908s 2024-05-06T12:33:14.959 INFO:teuthology.orchestra.run.smithi092.stderr:user 1m25.409s 2024-05-06T12:33:14.959 INFO:teuthology.orchestra.run.smithi092.stderr:sys 0m2.779s 2024-05-06T12:33:14.960 INFO:tasks.cephadm:Archiving logs... 2024-05-06T12:33:14.960 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-03_21:08:03-orch-squid-distro-default-smithi/7688103/remote/smithi092/log 2024-05-06T12:33:14.961 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-06T12:33:27.386 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-03_21:08:03-orch-squid-distro-default-smithi/7688103/remote/smithi190/log 2024-05-06T12:33:27.387 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-06T12:33:38.937 INFO:tasks.cephadm:Removing cluster... 2024-05-06T12:33:38.938 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 --force 2024-05-06T12:33:39.482 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid aee1d912-0ba0-11ef-bc97-c7b262605968 --force 2024-05-06T12:33:40.029 INFO:tasks.cephadm:Removing cephadm ... 2024-05-06T12:33:40.030 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-06T12:33:40.046 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-06T12:33:40.061 INFO:tasks.cephadm:Teardown complete 2024-05-06T12:33:40.061 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-06T12:33:40.079 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_70ce9539b62a15b9de429c1f0e91a1e012b099aa/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_70ce9539b62a15b9de429c1f0e91a1e012b099aa/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-05-06T12:33:40.080 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-06T12:33:40.080 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-06T12:33:40.089 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-06T12:33:40.174 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-06T12:33:40.186 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-06T12:33:40.187 DEBUG:teuthology.orchestra.run.smithi092:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-06T12:33:40.189 DEBUG:teuthology.orchestra.run.smithi190:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-06T12:33:40.201 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-05-06T12:33:40.203 INFO:teuthology.orchestra.run.smithi190.stderr:bash: line 1: ntpq: command not found 2024-05-06T12:33:40.206 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-06T12:33:40.206 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-05-06T12:33:40.206 INFO:teuthology.orchestra.run.smithi092.stdout:^* hv01.front.sepia.ceph.com 3 8 377 44 +1926us[+1924us] +/- 69ms 2024-05-06T12:33:40.206 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv02.front.sepia.ceph.com 3 8 377 52 -1719us[-1721us] +/- 77ms 2024-05-06T12:33:40.206 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 50 -980us[ -983us] +/- 82ms 2024-05-06T12:33:40.206 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-06T12:33:40.207 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-06T12:33:40.207 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2024-05-06T12:33:40.207 INFO:teuthology.orchestra.run.smithi190.stdout:^* hv01.front.sepia.ceph.com 3 6 377 47 +2074us[+2073us] +/- 69ms 2024-05-06T12:33:40.207 INFO:teuthology.orchestra.run.smithi190.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 47 -1575us[-1575us] +/- 77ms 2024-05-06T12:33:40.207 INFO:teuthology.orchestra.run.smithi190.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 50 -783us[ -783us] +/- 82ms 2024-05-06T12:33:40.207 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-06T12:33:40.208 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-06T12:33:40.225 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-06T12:33:40.226 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-06T12:33:40.271 DEBUG:teuthology.orchestra.run.smithi092:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-06T12:33:40.351 DEBUG:teuthology.orchestra.run.smithi190:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-06T12:33:40.422 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-06T12:33:40.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-06T12:33:40.450 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-06T12:33:40.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-06T12:33:40.477 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-06T12:33:40.488 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-06T12:33:40.507 INFO:teuthology.task.internal:Duration was 2163.116895 seconds 2024-05-06T12:33:40.507 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-06T12:33:40.524 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-06T12:33:40.524 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-06T12:33:40.527 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-06T12:33:40.561 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-06T12:33:40.563 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-06T12:33:40.888 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-06T12:33:40.888 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi092.front.sepia.ceph.com 2024-05-06T12:33:40.888 DEBUG:teuthology.orchestra.run.smithi092:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-06T12:33:40.908 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi190.front.sepia.ceph.com 2024-05-06T12:33:40.908 DEBUG:teuthology.orchestra.run.smithi190:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-06T12:33:40.952 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-06T12:33:40.952 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-06T12:33:40.954 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-06T12:33:41.017 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-06T12:33:41.017 DEBUG:teuthology.orchestra.run.smithi092:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-06T12:33:41.020 DEBUG:teuthology.orchestra.run.smithi190:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-06T12:33:41.633 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-06T12:33:41.645 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-06T12:33:41.645 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-06T12:33:41.670 DEBUG:teuthology.orchestra.run.smithi190:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-06T12:33:41.696 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-06T12:33:41.708 DEBUG:teuthology.orchestra.run.smithi092:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-06T12:33:41.713 DEBUG:teuthology.orchestra.run.smithi190:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-06T12:33:41.731 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = core 2024-05-06T12:33:41.758 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = core 2024-05-06T12:33:41.790 DEBUG:teuthology.orchestra.run.smithi092:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-06T12:33:41.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-06T12:33:41.809 DEBUG:teuthology.orchestra.run.smithi190:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-06T12:33:41.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-06T12:33:41.844 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-06T12:33:41.874 INFO:teuthology.task.internal:Transferring archived files... 2024-05-06T12:33:41.876 DEBUG:teuthology.misc:Transferring archived files from smithi092:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-03_21:08:03-orch-squid-distro-default-smithi/7688103/remote/smithi092 2024-05-06T12:33:41.876 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-06T12:33:41.925 DEBUG:teuthology.misc:Transferring archived files from smithi190:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-03_21:08:03-orch-squid-distro-default-smithi/7688103/remote/smithi190 2024-05-06T12:33:41.926 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-06T12:33:41.966 INFO:teuthology.task.internal:Removing archive directory... 2024-05-06T12:33:41.966 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-06T12:33:41.968 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-06T12:33:42.015 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-06T12:33:42.029 INFO:teuthology.task.internal:Not uploading archives. 2024-05-06T12:33:42.029 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-06T12:33:42.045 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-06T12:33:42.045 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-06T12:33:42.048 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-06T12:33:42.060 INFO:teuthology.orchestra.run.smithi092.stdout: 262213 4 drwxr-xr-x 3 ubuntu ubuntu 4096 May 6 12:33 /home/ubuntu/cephtest 2024-05-06T12:33:42.060 INFO:teuthology.orchestra.run.smithi092.stdout: 655612 4 d--------- 2 ubuntu ubuntu 4096 May 6 12:12 /home/ubuntu/cephtest/mnt.0 2024-05-06T12:33:42.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-06T12:33:42.061 INFO:teuthology.orchestra.run.smithi092.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-05-06T12:33:42.061 INFO:teuthology.orchestra.run.smithi092.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-05-06T12:33:42.061 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_70ce9539b62a15b9de429c1f0e91a1e012b099aa/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_70ce9539b62a15b9de429c1f0e91a1e012b099aa/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi092 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-05-06T12:33:42.062 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-06T12:33:42.077 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-06T12:33:42.092 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-05-06T12:33:42.093 INFO:teuthology.nuke:Checking targets against current locks 2024-05-06T12:33:42.120 DEBUG:teuthology.nuke:shortname: smithi092 2024-05-06T12:33:42.120 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-06T12:33:42.146 DEBUG:teuthology.nuke:shortname: smithi190 2024-05-06T12:33:42.146 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-06T12:33:42.149 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-03_21:08:03-orch-squid-distro-default-smithi/7688103', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-06 11:51:58.915452', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKqUT7+t55ukAQG9vthWS3R1xTo9XqzCpOxghMF3+w79mpeABKvmHmuRCD2yNsyxPS74zILZH4gZzvX9HXxMCMM='} 2024-05-06T12:33:42.180 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-03_21:08:03-orch-squid-distro-default-smithi/7688103', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-06 11:51:58.917042', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCQOkrFXfkgx4LrkVXglbo8zoGLSCcrkRtvo9k3P/rHzsGmOkOoyg15LqCqsTkDLFNwKNPNRNxNIC3KdErT1gpA='} 2024-05-06T12:33:42.213 INFO:teuthology.orchestra.console.smithi092:Power off 2024-05-06T12:33:42.213 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-06T12:33:42.239 DEBUG:teuthology.orchestra.console.smithi092:power off output: Chassis Power Control: Down/Off 2024-05-06T12:33:42.240 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-06T12:33:42.255 INFO:teuthology.orchestra.console.smithi190:Power off 2024-05-06T12:33:42.256 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-06T12:33:42.266 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2024-05-06T12:33:42.281 DEBUG:teuthology.orchestra.console.smithi190:power off output: Chassis Power Control: Down/Off 2024-05-06T12:33:42.281 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-06T12:33:42.304 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on 2024-05-06T12:33:46.267 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-06T12:33:46.306 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-06T12:33:46.380 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2024-05-06T12:33:46.416 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on 2024-05-06T12:33:50.382 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-06T12:33:50.417 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-06T12:33:50.494 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is off 2024-05-06T12:33:50.528 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on 2024-05-06T12:33:50.596 INFO:teuthology.orchestra.console.smithi092:Power off completed 2024-05-06T12:33:54.531 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-06T12:33:54.641 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is off 2024-05-06T12:33:54.742 INFO:teuthology.orchestra.console.smithi190:Power off completed 2024-05-06T12:33:54.846 INFO:teuthology.run:Summary data: description: orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 2163.116894721985 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default owner: scheduled_teuthology@teuthology status: fail success: false 2024-05-06T12:33:54.847 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-06T12:33:54.939 INFO:teuthology.run:FAIL